Proximity of Items Does Not Necessarily Affect Their Correlation
Two Failed Replications of Item Proximity Effect
Abstract
Abstract. In this paper, we attempted to detect the effect of provided answers on those that succeed them, the “item proximity effect,” within a questionnaire with randomized item order for each respondent. In two studies, we administered personality inventories to two samples of mostly undergraduate students, consisting of 742 and 205 respondents, respectively. We expected to see a strong item proximity effect that quickly diminishes with higher item distances, but we found no evidence of such effect in the data. Thus, we failed to conceptually replicate the findings of previous studies. After discussing the probable causes for these discrepancies, we argue that a general item proximity effect does not need to be considered a major factor when employing and evaluating common psychological inventories with randomized item order for research purposes.
References
(2010). Development of the Zuckerman-Kuhlman-Aluja Personality Questionnaire (ZKA-PQ): A factor/facet version of the Zuckerman-Kuhlman Personality Questionnaire (ZKPQ). Journal of Personality Assessment, 92(5), 416–431. https://doi.org/10.1080/00223891.2010.497406
(2019). Is reliability compromised towards the end of long personality inventories? European Journal of Psychological Assessment, 35(1), 14–21. https://doi.org/10.1027/1015-5759/a000363
(2012). Using the theory of satisficing to evaluate the quality of survey data. Research in Higher Education, 53(2), 182–200. https://doi.org/10.1007/s11162-011-9251-2
(2012). Anchoring and adjusting in questionnaire responses. Basic and Applied Social Psychology, 34(5), 417–433. https://doi.org/10.1080/01973533.2012.711691
(2006). Planned missing data designs in psychological research. Psychological Methods, 11(4), 323–343. https://doi.org/10.1037/1082-989X.11.4.323
(2021). Proximity of items does not necessarily affect their correlation: Two failed replications of item proximity effect. Supplementary material. https://osf.io/s7wn2/
(1993). Cognitive processes in self-report responses: Tests of item context effects in work attitude measures. Journal of Applied Psychology, 78(1), 129–140. https://doi.org/10.1037/0021-9010.78.1.129
(1985). The instability of response sets. Public Opinion Quarterly, 49(2), 253–260. https://doi.org/10.1086/268918
(1988). Item context effects on personality scales: Measuring changes the measure. Journal of Personality and Social Psychology, 55(2), 312–320.
(2005).
The measurement of attitudes . In D. AlbarracinB. T. JohnsonM. P. ZannaEds., Handbook of attitudes and attitude change (pp. 21–76). Erlbaum.(1996). Satisficing in surveys: Initial evidence. New Directions for Evaluation, 1996(70), 29–44. https://doi.org/10.1002/ev.1033
(1985). Implications for altering the context in which test items appear: A historical perspective on an immediate concern. Review of Educational Research, 55(3), 387–413.
(2019). Měření náchylnosti k nudě skrze ZKA-PQ: Adaptace inventáře a ověření konstruktové validity
[The measurement of boredom susceptibility with ZKA-PQ: Inventory adaptation and construct validity verification] (Bachelor thesis) . Masaryk University, Faculty of Social Studies. https://is.muni.cz/th/o8b8s/(2017). How does response bias emerge in lengthy sequential preference judgments? Behaviormetrika, 44(2), 575–591. https://doi.org/10.1007/s41237-017-0036-6
(1993). Personal Need for structure: individual differences in the desire for simple structure. Journal of Personality and Social Psychology, 65(1), 113–131. https://doi.org/10.1037/0022-3514.65.1.113
(2011). Too reliable to be true? Response bias as a potential source of inflation in Paper-And-Pencil Questionnaire reliability. Practical Assessment, Research & Evaluation, 16(9), 1–8. https://doi.org/10.7275/e482-n724
. (2019). foreign: Read Data Stored by “Minitab”, “S”, “SAS”, “SPSS”, “Stata”, “Systat”, “Weka”, “dBase”, …. https://CRAN.R-project.org/package=foreign
. (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
(2019). psych: Procedures for psychological, psychometric, and personality research. Northwestern University. https://CRAN.R-project.org/package=psych
(1981). Questions and answers: Experiments on question form, wording, and context in attitude surveys. Academic Press.
Schwarz, N.Sudman, S. (Eds.). (1992). Context effects in social and psychological research. Springer. https://doi.org/10.1007/978-1-4612-2848-6
(2020). DescTools: Tools for descriptive statistics. https://cran.r-project.org/package=DescTools
(1992).
“Order effects” in survey research: Activation and information functions of preceding questions . In N. SchwarzS. SudmanEds., Context effects in social and psychological research (pp. 23–34). Springer.(1996). Thinking about answers: The application of cognitive processes to survey methodology. Jossey-Bass.
(1988). Cognitive processes underlying context effects in attitude measurement. Psychological Bulletin, 103(3), 299–314.
(2000). The psychology of survey response. Cambridge University Press.
(2017). R package “corrplot”: Visualization of a correlation matrix. https://github.com/taiyun/corrplot
(2014). Discriminant validity where there should be none: Positioning same-scale items in separated blocks of a questionnaire. Applied Psychological Measurement, 38(6), 450–463. https://doi.org/10.1177/0146621614531850
(2009). The proximity effect: The role of inter-item distance on reverse-item bias. International Journal of Research in Marketing, 26(1), 2–12. https://doi.org/10.1016/j.ijresmar.2008.09.003
(2016). ggplot2: Elegant graphics for data analysis. Springer. https://ggplot2.tidyverse.org
(2020). dplyr: A grammar of data manipulation. https://CRAN.R-project.org/package=dplyr
(2020). tidyr: Tidy Messy data. https://CRAN.R-project.org/package=tidyr
