Abstract
Abstract. We constructed an online cheating paradigm that could be used to validate the Crosswise Model (Yu, Tian, & Tang, 2008), a promising indirect questioning technique designed to control for socially desirable responding on sensitive questions. Participants qualified for a reward only if they could identify the target words from three anagrams, one of which was virtually unsolvable as shown on a pretest. Of the 664 participants, 15.5% overreported their performance and were categorized as cheaters. When participants were asked to report whether they had cheated, a conventional direct question resulted in a substantial underestimate (5.1%) of the known prevalence of cheaters. Using a CWM question resulted in a more accurate estimate (13.0%). This result shows that the CWM can be used to control for socially desirable responding and provides estimates that are much closer to the known prevalence of a sensitive personal attribute than those obtained using a direct question.
References
(1998). Multinomial processing tree models and psychological assessment. Psychological Assessment, 10, 331–344. doi: 10.1037/1040-3590.10.4.331
(1966). Anagram solving as a function of letter moves. Journal of Verbal Learning and Verbal Behavior, 5, 107–111. doi: 10.1016/S0022-5371(66)80002-6
(1967). Anagram solving as a function of bigram rank and word frequency. Journal of Experimental Psychology, 75, 299–306. doi: 10.1037/H0025060
(1978). Effects of solution word attributes on anagram difficulty – a regression analysis. The Quarterly Journal of Experimental Psychology, 30, 57–70. doi: 10.1080/14640747808400654
(2014). Time, money, and morality. Psychological Science, 25, 414–421. doi: 10.1177/0956797613506438
(2013). Being honest about dishonesty: Correlating self-reports and actual lying. Human Communication Research, 40, 54–72.
(2013). What lies beneath: How the distance between truth and lie drives dishonesty. Journal of Experimental Social Psychology, 49, 263–266. doi: 10.1016/j.jesp.2012.11.010
(2014). Do RRT and the Crosswise Model produce valid measurements of sensitive items? An online validation study. Paper presented at the XVIII ISA World Congress of Sociology, July 13–19, 2014, Yokohama, Japan
(2015). Assessing the validity of two indirect questioning techniques: A Stochastic Lie Detector versus the Crosswise Model. Behavior Research Methods. Advance online publication. doi: 10.3758/s13428-015-0628-6
(2015). On the comprehensibility and perceived privacy protection of indirect questioning techniques.Behavior Research Methods. Advance online publication. doi: 10.3758/s13428-015-0628-6
(1994). The statistical-analysis of general processing tree models with the Em algorithm. Psychometrika, 59, 21–47. doi: 10.1007/Bf02294263
(2012). Asking sensitive questions using the Crosswise Model. Public Opinion Quarterly, 76, 32–49. doi: 10.1093/Poq/Nfr036
(1966). Solution of anagrams. Psychological Bulletin, 66, 371–384. doi: 10.1037/H0023886
(2010). The dirty dozen: A concise measure of the dark triad. Psychological Assessment, 22, 420–432. doi: 10.1037/A0019265
(2014). Measuring and explaining tax evasion: Improving self-reports using the Crosswise Model. Journal of Economic Psychology, 45, 18–32. doi: 10.1016/j.joep.2014.08.001
(2013). Determinants of social desirability bias in sensitive surveys: A literature review. Quality & Quantity, 47, 2025–2047. doi: 10.1007/s11135-011-9640-9
(2014). Das Dreckige Dutzend und die Niederträchtigen Neun – Kurzskalen zur Erfassung von Narzissmus, Machiavellismus und Psychopathie
[The Dirty Dozen and the Naughty Nine: Short scales for the assessment of narcissism, Machiavellianism, and psychopathy] . Diagnostica, 61, 1–16. doi: 10.1026/0012-1924/a000124(2013). Re-assessing the merits of measuring tax evasions through surveys: Evidence from Serbian firms. ZEW Discussion Papers, No. 13-047. Retrieved Dec 12th, 2013, from http://hdl.handle.net/10419/78625
(1972). Anagram solutions as a function of task variables and solution word models. Journal of Experimental Psychology, 92, 65–68. doi: 10.1037/H0032164
(2005). Meta-analysis of randomized response research: Thirty-five years of validation. Sociological Methods & Research, 33, 319–348. doi: 10.1177/0049124104268664
(1958). Anagram solution times: A function of letter order and word-frequency. Journal of Experimental Psychology, 56, 376–379. doi: 10.1037/H0041542
(1959). Anagram solution times: A function of transition-probabilities. Journal of Psychology, 47, 117–125.
(1976). An hypothesis approach to the solution of anagrams. Memory & Cognition, 4, 637–642. doi: 10.3758/Bf03213228
(1974). The solution of anagrams – a reexamination of the effects of transition letter probabilities, letter moves, and word frequency on anagram difficulty. Memory & Cognition, 2, 566–574. doi: 10.3758/Bf03196922
(2010). multiTree: A computer program for the analysis of multinomial processing tree models. Behavior Research Methods, 42, 42–54.
(2014). An experimental validation method for questioning techniques that assess sensitive issues. Experimental Psychology, 61, 48–54. doi: 10.1027/1618-3169/a000226
(2012). A stochastic lie detector. Behavior Research Methods, 44, 222–231. doi: 10.3758/s13428-011-0144-2 21858604
(2001).
Improving survey research on the world-wide web using the randomized response technique . In U. D. ReipsM. BosnjakEds., Dimensions of Internet science (pp. 179–192). Lengerich, Germany: Pabst.(2013). Prevalence of use of anabolic steroids by bodybuilders using three methods in a city of Iran. Addict Health, 5, 1–6.
(1985). Methods of coping with social desirability bias – a review. European Journal of Social Psychology, 15, 263–280. doi: 10.1002/ejsp.2420150303
(2009). Assessing sensitive attributes using the randomized response technique: Evidence for the importance of response symmetry. Journal of Educational and Behavioral Statistics, 34, 267–287. doi: 10.3102/1076998609332747
(2011). Estimating the prevalence of negative attitudes towards people with disability: A comparison of direct questioning, projective questioning and randomised response. Disability and Rehabilitation, 33, 1–13. doi: 10.3109/09638288.2010.492067
(1991).
Measurement and control of response bias . In J. P. RobinsonP. R. ShaverL. S. WrightsmanEds., Measures of personality and social psychological attitudes (Vol. 1, pp. 17–59). San Diego, CA: Academic Press.(1972). Some effects of social desirability in survey studies. American Journal of Sociology, 77, 921–940. doi: 10.1086/225231
(2006). Corpus portal for search in monolingual corpora. Paper presented at the fifth international conference on Language Resources and Evaluation, LREC, Genoa
(2014). Unipark EFS Survey 10.2. Retrieved from http://www.unipark.de
(1988). Goodness-of-fit statistics for discrete multivariate data. New York, NY: Springer.
(2013). The cheater’s high: The unexpected affective benefits of unethical behavior. Journal of Personality and Social Psychology, 105, 531–548. doi: 10.1037/A0034231
(2011). Justified ethicality: Observing desired counterfactuals modifies ethical perceptions and behavior. Organizational Behavior and Human Decision Processes, 115, 181–190. doi: 10.1016/j.obhdp.2011.02.001
(1959). Effects of letter-orders and material words on the anagram solution. Japanese Journal of Psychology, 30, 253–263.
(2014). Incomplete categorical data design: Non-randomized response techniques for sensitive questions in surveys. Boca Raton, FL: CRC Press, Taylor & Francis Group.
(2007). Sensitive questions in surveys. Psychological Bulletin, 133, 859–883. doi: 10.1037/0033-2909.133.5.859 17723033
(2012). Asking sensitive questions: A statistical power analysis of randomized response models. Psychological Methods, 17, 623–641. doi: 10.1037/A0029314
(1991). A critical evaluation of the randomized-response method – applications, validation, and research agenda. Sociological Methods & Research, 20, 104–138.
(2000). A comparison of randomized response, computer-assisted self-interview, and face-to-face direct questioning – eliciting sensitive information in the context of welfare and unemployment benefit. Sociological Methods & Research, 28, 505–537.
(1965). Randomized-response – a survey technique for eliminating evasive answer bias. Journal of the American Statistical Association, 60, 63–69.
(2011). Cheating more when the spoils are split. Organizational Behavior and Human Decision Processes, 115, 157–168. doi: 10.1016/j.obhdp.2010.10.001
(2008). Two new models for survey sampling with sensitive characteristic: Design and analysis. Metrika, 67, 251–263. doi: 10.1007/s00184-007-0131-x
(1935). The psycho-biology of language. Oxford, UK: Houghton-Mifflin.
