Replication of Experiments Evaluating Impact of Psychological Distance on Moral Judgment

Most of the time people act and feel toward the construal of an object, rather than object itself. Construal Level Theory (CLT; Trope & Liberman, 2010) addresses universally relevant issues of mental representations and thus runs across the subdisciplines of psychology. CLT has been widely tested in different domains, and seems to have received relatively robust empirical confirmation. It is established as an influential theory of social cognition: A chapter is regularly devoted to it in contemporary handbooks (Shapira, Liberman, Trope, & Rim, 2012; Trope & Liberman, 2011), and the seminal article on Temporal construal (Trope & Liberman, 2003) has been highly cited (656 citations in Scopus, 689 in APA Psych Net, and 1,324 times in Google scholar¹).

CLT’s major premise is that mental representations of objects and situations vary depending on their distance from the perceiver: Distant objects tend to be represented by few essential characteristics (high-level construal), whereas close objects tend to be represented by detailed and contextual information (low-level construal). High-level construals are important for regulation toward distal objects, when it is important to focus on their invariant, central features, while low-level construals are more important for regulation toward close objects, as we need to act toward them immediately. Given that psychological distance is egocentric, with its reference point being here and now, it can be social (self vs. other), temporal (present self vs. future or past self), spatial (here vs. some other place), or hypothetical (highly likely vs. unlikely event).

CLT proposes that moral values have stronger impact on judgment from a greater psychological distance, while contextual information is more relevant when at a close distance. In other words, the same act will be evaluated differently depending on the perceivers’ distance from the presented event. In a series of experiments, Eyal, Liberman, and Trope (EL&T, 2008) tested how temporal and social distance impact the evaluation of moral acts (both transgressions and virtuous acts), followed by contextual information that intended to attenuate their severity. Temporal distance was manipulated by the instruction to imagine the situation was taking place now or later in time. Social distance was manipulated by the instruction to imagine the situation from one’s own or from a third person’s perspective. Four experiments in their article confirmed CLT propositions: Both moral transgressions and virtuous behaviors were evaluated differently depending on the psychological distance. When the distance was higher, values (as high-level construal) had a greater impact on evaluation. When the distance was smaller, contextual information (as low-level construal) was of greater concern. This study has been cited (33 times in Scopus, 76 in Google scholar), as supportive of CLT predictions within the moral evaluation domain. In a similar vein, Agerström and Bjorklund (2009) examined the impact of temporal distance on moral concerns (selfish vs. altruistic considerations) and reported that moral concerns were higher for temporarily distant situations. In a series of experiments, Lammers’ results (2012) mirrored those of EL&T: Subjects reacted more negatively to others’ morally questionable behaviors when adopting an abstract rather than concrete perspective.

A following project by Gong and Medin (G&M, 2012), however, yielded some directly conflicting results. The authors used the same stimulus material as EL&T (vignettes about moral transgressions and virtuous acts), but instead of manipulating distance, they directly manipulated construal level by priming tasks designed to stimulate either the abstract or concrete mindset. Contrary to expectations, when participants were primed with a concrete mindset, they showed more extreme moral judgments of both moral transgressions and virtuous behaviors. In order to test the possible impact of a different procedure (manipulation of distance vs. manipulation of construal level), G&M directly replicated one of four experiments from EL&T’s study. Again, G&M observed the opposite results. When EL&T tried to replicate G&M’s experiments (using the priming tasks), their results were not significant. Baring in mind these contradictory findings and the importance of the topic (impact of construal level on moral judgment), we proposed a two-stage replication project:

1. 1.
   Direct replication of EL&T’s Studies 2, 3, and 4 and G&M’s Study 1 in the Social Cognition Laboratory at Belgrade University, Serbia. The main authors of both studies agreed to provide us with the materials (vignettes, instruments, and instructions) used in their experiments.
2. 2.
   Aggregation of databases from three laboratories: Ben-Gurion University (Israel), Northwestern University (Ilinois, USA), and Belgrade University (Serbia). This provided us with the opportunity to directly compare: (a) The effects obtained by three laboratories (Experiment 2 from EL&T’s study, and Experiment 1 from G&M’s study); (b) the effects obtained by two laboratories (Israeli and Serbian), from Experiments 3 and 4 of EL&T’s.

Materials, data, and the preregistered proposal are available on the project page on the Open Science Framework ( https://osf.io/Z5TE6/).

We report all data exclusions, manipulations, and measures, and how we determined our sample sizes. The planned sample size² was based on the effect size from each of the original studies, so as to achieve .95 power (see Table 1 for details).

Table 1. Effect sizes from original studies and planned sample size

Original study	Original sample size	Cohen-s’ d from original study	Planned sample size
Exp 2 (EL&T)	58	.68	114 (57 per group)
Exp 3 (EL&T)	40	.72	102 (51 per group)
Exp 4 (EL&T)	47	.81	80 (40 per group)
Exp 1 (G&M)	34	1.07	48 (24 per group)

Table 1. Effect sizes from original studies and planned sample size

View as image HTML

Whenever we had an opportunity, we tested up to 5% respondents over the planned sample size, in case we had to omit some from further analysis. Participants were randomly assigned to experimental groups. They were recruited from a pool of psychology students from Belgrade University, in exchange for course credits. The same recruiting method was applied in EL&T’s original study, while in G&M’s study participation was on a voluntary basis. Gender information was registered, although it was not found to have a significant effect in either study.

We used the vignettes from EL&T’s studies, with the authors’ permission. The vignettes were translated into Serbian by two independent bilingual translators, and then translated back into English in order to provide maximum correspondence (as suggested in Brislin, 1970, 1976). As was the procedure in the original experiments, the respondents filled in booklets in paper format.

The experimenters were PhD students from the Social Cognition Laboratory of the Department of Psychology, Belgrade University, blind to the study hypothesis. Participants were tested in groups no larger than ten. At the beginning of each session, the experimenter presented themselves to the participants and explained the research purpose: “This study is about a judgment of different people’s actions. After reading each story, please provide your opinion below it. There are no right or wrong answers. You are expected to evaluate just as you think. Examination is anonymous, so you do not need to provide any personal information.” The experimenter was present until the end of the experiment, but not allowed to give any additional instructions, except to encourage participants to give their own opinions if they had any questions. After hearing the instructions, participants read the vignettes and evaluated the wrongness (in the first two studies), virtuousness (in the third study), or moral acceptability (in the fourth study) of the actions presented. We used the same scales as in corresponding studies of EL&T and G&M.

Our analysis was planned in a confirmatory fashion.³ We performed a series of standard t-tests, as well as default Bayesian t-tests, as proposed by Rouder, Speckman, Sun, Morey, and Iverson (2009), in order to test the differences:

1. 1.
   In perceived wrongness of the actions between high and low temporal distance primed groups (Study 1);
2. 2.
   In perceived wrongness of the actions between high and low social distance primed groups (Study 2);
3. 3.
   In perceived virtuousness of the actions between high and low temporal distance primed groups (Study 3);
4. 4.
   In perceived moral acceptability of the transgressions between high and low construal level primed groups (Study 4).

In a next step we performed an analysis on the aggregated database (with provided data from two or three laboratories), and at the end we introduced “laboratory” as a factor.

The main known difference from the original study was the cultural and linguistic backgrounds of the samples (Serbian vs. Israeli/Hebrew vs. American/English). One minor difference was that our sample consisted exclusively of students, while in one of EL&T’s Study 3 participants were workers from security service organizations. Apart from that, a full methodological and procedural equivalence was set up.

Study 1

The aim of the Study 1 was to replicate the findings of EL&T (Study 2, 2008) indicating that people would judge immoral acts more harshly if presented to them as temporally distant rather than presented as temporally close. Participants judged the wrongness of moral transgressions as expected to occur either the next day (near future condition) or next year (distant future condition).

Results for Aggregated Data From Three Laboratories

A mixed ANOVA with temporal distance (near vs. distant future) as between-subject factor and story (1–3) as within-subject factor again yielded a strong main effect of story, F(2, 209) = 186.61, p < .001, η² = .47 (“incest”: M = −4.06, SD = 0.14; “dog”: M = −3.93, SD = 0.13; “flag”: M = −0.50, SD = 0.24), meaning that wrongness of different transgressions was judged differently. As for the central research hypothesis, we ended up with conflicting findings: The original study found the expected impact of temporal distance to the wrongness assessment; the direct replication with the American sample yielded the opposite pattern of results, whereas direct replication with the Serbian sample yielded no significant effect. Analysis on the integrated sample revealed no effect of temporal distance F(1, 209) = 0.25, p = .618 η² = .001, g = −0.069, [CI = −0.323, 0.186] (distant future transgressions: M = −2.76; SD = 2.03; near future transgressions: M = −2.89; SD = 1.72).

Further analysis revealed the significant effect of “laboratory” as a factor, F(2, 209) = 11.80, p < .001, η² = .10 and a significant interaction between story and laboratory, F(4, 209) = 10.76, p < .001, η² = .09, indicating cultural differences in assessment of wrongness of different immoral acts.⁶ More importantly, there was also an interaction between temporal distance and “laboratory”, F(2, 209) = 4.08, p = .018, η² = .04. A post hoc Tukey test showed that the Israeli laboratory differed significantly from both Serbian one (at p < .001) and American one (at p = .022), while Serbian and American laboratories did not differ from one another (see Figure 1 ).

Study 2

Study 2 was designed to build upon the results of Study 1 by manipulation of social distance (self vs. other) rather than temporal, and to extend the number of acts to be evaluated (six instead of three). It presents a direct replication of EL&T’s study 3.

Results for Serbian Replication

In order to unify data from replication and original databases, we reversed the participants’ ratings so that higher ratings indicated higher wrongness scores. A mixed ANOVA with social distance (self vs. other) as between-subject factor and story (1–6) as within-subject factor yielded a main effect of story, F(5, 104) = 52.01, p < .001, η² = .34, indicating that the wrongness of events were judged differently. More importantly, there was also a main effect of social distance, F(1, 104) = 9.90, p = .002, η² = .09, g = 0.615, [CI = 0.479, 0.751]. All actions were judged as more wrong from a third person perspective (M = 3.91, SD = 0.61) then from a first person perspective (M = 3.47, SD = 0.80), which was in accordance with CLT predictions and the results of EL&T’s original Study 3. Across six scenarios, the effect was significant on two (“dog”, F(1, 104) = 5.17, p = .025, η² = .05; “broken promise”, F(1, 104) = 4.73, p = .032, η² = .04), and marginally significant on three (“swing”, F(1, 104) = 3.39, p = .068, η² = .03; “flag”, F(1, 104) = 3.34, p = .070, η² = .03; “kiss”, F(1, 104) = 2.89, p = .092, η² = .03). The pattern of means of six stories across social distance is presented in Figure 2 .

Scaled JZS Bayes Factor which was lower than one (0.07) strongly supported the alternative hypothesis, indicating that two social distance groups significantly differed from each other.

Results for Aggregated Data From Two Laboratories

As expected, mixed ANOVA with social distance (self vs. other) as between-subject factor and story (1–6) as within-subject factor again yielded a main effect of story, F(5, 144) = 68.83, p < .001, η² = .33. The main effect of social distance was once again significant, F(1, 144) = 13.77, p < .001, η² = .09, g = 0.624 [CI = 0.513, 0.736] (means are detailed in Figure 3 ).

Further analysis revealed the main effect of laboratory as a factor, F(1, 144) = 9.36, p = .003, η² = .06. There was no interaction between laboratory and social distance factors, and as in previous study, there was a significant interaction between story and laboratory that emerged, F(5, 144) = 9.05, p < .001, η² = .06.

Study 3

The main objective of Study 3 was to explore the effects of temporal distance on moral judgment of virtuous acts performed under attenuating circumstances (e.g., a fashion company donates to the poor and it positively affects its sales rate). In EL&T’s original Study 4, higher distance lead to more positive virtuousness ratings, that is ascribing less weight to attenuating contextual information.

Results for Aggregated Data From Two Laboratories

A mixed ANOVA with temporal distance (near vs. distant future) as between-subject factor, and story (1–3) as within-subject factor again yielded a main effect of story, F(2, 130) = 14.97, p < .001, η² = .10. There was no main effect of temporal distance, F(1, 130) = 0.11, p = .737, η² = .001, g = 0.061, [CI = −0.136, 0.257] meaning that the virtuousness of distant future acts (M = 4.61; SD = 1.13) was judged the same as the virtuousness of near future acts (M = 4.54; SD = 1.17).

Further analysis revealed the marginally significant effect of laboratory as a factor, F(1, 130) = 3.31, p = .071, η² = .025 (see Figure 4 ). There was also an interaction between temporal distance and laboratory, F(1, 130) = 6.79, p = .01, η² = .05, and as in previous studies, an interaction between story and laboratory, F(2, 130) = 4.80, p = .009, η² = .04.

Study 4

In this study we introduced a direct manipulation of construal level as employed in G&M’s (2012) first study. We primed our participants with a series of “how-and-why” questions that were expected to directly activate either a low or high construal mindset. By asking subjects to generate subordinate goals, we led them to adopt an instrumental, lower-level perspective. In contrast, generating superordinate goals led them to adopt a higher-level perspective. We aimed to explore if priming a high-or-low construal level would have an impact on the severity of our participants’ moral judgment. In G&M’s study, this manipulation yielded results contradictory to those of EL&T: Low-level construals led to harsher condemnation of moral transgressions in comparison to high-level construals.

Results for Aggregated Data From Three Laboratories

A mixed ANOVA with construal level (low vs. high) as between-subject factor and story (1–4) as within-subject factor again yielded a main effect of story, F(3, 162) = 54.56, p < .001, η² = .25. There was also a main effect of the construal level, F(1, 162) = 8.81, p = .003, η² = .05, g = −0.338, [CI = −0.564, −0.113] in contrast to CLT prediction: Low level priming led to harsher wrongness assessment (M = −3.03, SD = 1.41) than high level priming (M = −2.35, SD = 1.53).

Further analysis revealed the significant effect of the laboratory, F(2, 162) = 51.78, p < .001, η² = .40. A post hoc Tukey test demonstrated that the Serbian laboratory differed significantly from both Israeli and American, at p < .001), whereas Israeli and American laboratories did not differ from one another (as can be seen in Figure 5 ). Once more we discovered an interaction between story and laboratory, F(6, 162) = 26.98, p < .001, η² = .26. The difference was mainly due to the fact that Serbian participants judged two transgressions less harshly than the Israeli and American: “flag” and “cheat” (both at p < .001). We have already addressed the former; the latter might be because the concept of academic honesty is more vague in Serbian than in Israely/American university setting, with no explicit (written) ethical guidelines.

Following the recommendations from Valentine et al. (2011) and Lakens (2013), we summarized the results of four experiments in Table 2 .

Table 2. Results summary of multiple studies⁷

Study	Eyal, Liberman, and Trope, Hedge’s g ± CI	Zezelj and Jokic, Hedge’s g ± CI	Gong and Medin, Hedge’s g ± CI	Do the studies agree about the direction of the effect?	What is the pattern of statistical significance?	Is the effect size from original study within the CI of the Zezelj and Jokic study?
Notes. As in some of the original studies samples were relatively small (n < 20), and Cohen’s d gives a biased estimate of the population effect size especially for small samples, we opted for corrected effect size, Hedges’s g (recommended in Cumming, 2013; Lakens, 2013).
1 (EL&T Study 2, 2008)	0.66 ± 0.27	−0.06 ± 0.37	−0.78 ± 0.59	No	Zezelj and Jokic not, other two significant	No
2 (EL&T Study 3, 2008)	0.71 ± 0.17	0.61 ± 0.14		Yes	Both significant	Yes
3 (EL&T Study 4, 2008)	0.80 ± 0.26	−0.26 ± 0.26		No	Eyal et al. significant, Zezelj and Jokic not	No
4 (G&M Study 1, 2012)	−0.34 ± 0.24	−0.68 ± 0.36	−1.06 ± 0.35	Yes	Eyal et al. not, other two significant	No

Table 2. Results summary of multiple studies⁷

View as image HTML

Discussion

To set the right tone for this discussion, we must first acknowledge that there is no such thing as an exact replication – there are always known and unknown factors that possibly contribute to a certain outcome that might not be identical between two studies (from characteristics of participants to physical conditions and time of day). We share the view of other social scientists (e.g., Asendorpf et al., 2013; Cacioppo & Cacioppo, 2013; Spellman, 2013) that this fact, however, should not discourage researchers from performing replications, as it is a necessary step for further generalization and/or establishing the limits of a certain phenomenon. In our replication study, we put every effort to have identical stimuli, procedure and participants as in original studies. We were fortunate enough to have the full cooperation from the other two laboratories – they were fast in sharing materials, instruments, and databases upon our request. The only known difference between the studies was the fact that samples were drawn from different cultures. Strictly speaking, this should not be decisive to the main manipulation effect: Cognitive construals should be sensitive to psychological distance, and this mechanism should serve self-regulatory needs, that are culturally invariant. However, EL&T application of CLT was in the domain of moral reasoning, which has proven to be culturally sensitive (e.g., Boyes & Walker, 1988; Snarey, 1985; Tsui & Windsor, 2001). Moreover, there were attempts to attribute current inconclusive results to cultural differences (G&M, 2012), which is why a direct replication of the original experiments in different cultural setting seemed to be an appropriate starting point.

The original study’s hypotheses were derived from the premises of Construal Level Theory (Trope & Liberman, 2010). CLT proposes that distant events are represented more abstractly, globally than events that are psychologically closer. Therefore, a distant event should be evaluated in terms of more primary, high-level features. Applied in the domain of moral reasoning, this translates to expectation that one’s reliance on universal moral principles and neglect to the detail should be enhanced in evaluating psychologically distant events.

We begun our investigation following two conflicting sets of findings: One supporting the conclusion that high-level construal leads to less sensitivity to context – therefore to harsher judgment of moral transgressions and more appreciation to virtuous behavior (EL&T, 2008), and the other supporting the conclusion that low construal leads to more sensitivity to context and therefore to an opposite moral evaluation of acts (G&M, 2012). Given that the cooperative efforts of the two research groups did not lead to a resolution, it was necessary to further disentangle this puzzle.

Our four attempts to replicate experiments investigating the impact of psychological distance on moral judgment yielded three different outcomes: No systematic effect of temporal distance (regardless of the nature of the act: Transgression or virtuous), the effect of social distance compatible with CLT predictions, and the reversed effect of direct construal level manipulation.

There is accumulated empirical evidence demonstrating that both temporal and social distance indeed affects moral judgment. For example, research done in Sweden (Agerström & Björklund, 2009; Agerström, Björklund, & Carlsson, 2013) concluded that people make more extreme moral judgments of behavior from a distant than from a near time (or visual) perspective, and that this effect was driven by the level of construal. Lammers (2012) demonstrated in four studies that subjects reacted more negatively to others’ morally questionable behaviors when they took an abstract (high-level) perspective rather than a concrete (low-level) perspective. However, they were inclined to react less negatively to their own moral transgressions.

CLT does not assume different impact of temporal and social distance – in fact, it was initially a theory of temporal construal, exploring the effects of time perspective on mental representation (Liberman, Sagristano, & Trope, 2002; Liberman & Trope, 1998; Trope & Liberman, 2003). Only later it was generalized to other forms of psychological perspectives, namely social, spatial, and hypothetical (Trope & Liberman, 2010). It is therefore difficult to speculate the reasons as to why the temporal distance manipulation showed no consistent impact on morality judgments.

The effects of direct priming of construal level within the “how-and-why” task were in line with the results obtained by G&M. This task was designed to activate a low construal mindset (through a series of “how questions” emphasizing the means by which activities are carried out) or a high construal mindset (through a series of “why questions” emphasizing the end state activities lead to). The priming procedure was developed by Freitas, Gollwitzer, and Trope (2004), and successfully implemented for this purpose in other experiments within CLT framework (e.g., Wakslak & Trope, 2009). In our research it was expected to shift respondents’ focus in evaluation of immoral acts from moral universalities to contextual information that attenuated the severity of the act. However, it could be the case that if a person was prompted to think about the means of immoral acts, they tend to focus on the act itself and thus represent it very concretely and vividly. This representation could invoke strong emotional responses. Alternatively, it could be argued that the details provided with intention to mitigate the severity of the act could have been perceived as implausible excuses and therefore dismissed. Both could paradoxically lead to harsher instead of leaner judgments.

However, the obtained pattern of results is not easily attributable to cultural differences or to unexpected consequences of provided contextual information. Had we discovered, for example, that a low construal mindset consistently leads to a harsher judgment (as G&M had), we could have speculated that attenuating information would not be plausible to our respondents so that focusing on them did not have the expected effect. Yet we had the strongest effect of social distance manipulation, in which the judgment was harsher for a high construal mindset (i.e., judging moral behavior from a third person perspective).What our replication venture seems to show is that there is a complex interplay between (A) domain of judgment (moral judgment seems to be very specific), (B) procedures employed to invoke a specific mindset (direct priming, temporal distance, and social distance manipulation yield different results in morality assessment, maybe because they do not all necessarily lead to focus on moral universalities or context, as expected), and (C) the ethnicity/culture/society that respondents are recruited from.

Future research could benefit from developing clear manipulation checks aiming to assess if different priming techniques really lead to different levels of mental construal. It could also seek to directly compare the effects of different procedures (social, temporal distance, and direct priming) on moral judgments. This in turn could help in establishing the limits of CLT’s generalizability as one of the most promising and influential theories in the field.

Note From the Editors

Commentaries and a rejoinder on this paper are available (Eyal, Liberman, & Trope, 2014; Gong & Medin, 2014; Žeželj & Jokić, 2014; doi: 10.1027/1864-9335/a000206).