Experimental Vignette Studies in Survey Research
Abstract
Vignette studies use short descriptions of situations or persons (vignettes) that are usually shown to respondents within surveys in order to elicit their judgments about these scenarios. By systematically varying the levels of theoretically important vignette characteristics a large population of different vignettes is typically available – too large to be presented to each respondent. Therefore, each respondent gets only a subset of vignettes. These subsets may either be randomly selected in following the tradition of the factorial survey or systematically selected according to an experimental design. We show that these strategies in selecting vignette sets have strong implications for the analysis and interpretation of vignette data. Random selection strategies result in a random confounding of effects and heavily rely on the assumption of no interaction effects. In contrast, experimental strategies systematically confound interaction effects with main or set effects, thereby preserving a meaningful interpretation of main and important interaction effects. Using a pilot study on attitudes toward immigrants we demonstrate the implementation and analysis of a confounded factorial design.
References
(1978). The use of vignettes in survey research. Public Opinion Quarterly, 42, 93–104.
(1978). Who should get what? Fairness judgments of the distribution of earnings. American Journal of Sociology, 84, 541–564.
(2006). Die Vignettenanalyse als empirisches Messmodell: Methodische und methodologische Neuerungen gezeigt am Beispiel der Untersuchung normativer (Vor-)Urteile unter Jugendlichen zum Thema Integration von MigrantInnen [
The vignette analysis as empirical model: Methodological improvements shown as an example for a normative prejudice/judgement survey of adolecents with the topic “integration of immigrants” ]. University of Vienna: Vienna Unpublished doctorial dissertation.(2009). Komplexität von Vignetten, Lerneffekte und Plausibilität im Faktoriellen Survey [
Complexity, learning effects and plausibility of vignettes in the factorial survey design ]. Methoden – Daten – Analysen, 1, 59–96.(2002). Imitation and learning under uncertainty: A vignette experiment. ICS/Department of Sociology, Utrecht University.
(2001). Der faktorielle Survey und die Messung von Normen [
The factorial survey and the measurement of norms ]. Kölner Zeitschrift für Soziologie und Sozialpsychologie, 53, 283–306.(1999). How to show that 9 > 221: Collect judgments in a between-subjects design. Psychological Methods, 4, 243–249.
(1950). Experimental designs (2 nd ed.). New York: Wiley.
(1977). Differences in public support for seven social welfare groups: Description and explanation University of Chicago Doctoral dissertation.
(1979). Who should be helped? Public support for social services. London: Sage.
(2001). Bildung und der Einfluss von Argumenten auf das moralische Urteil. Eine empirische Analyse zur moralischen Entwicklungstheorie Kohlbergs [
Education and the influence of arguments for moral judgement: An empirical analysis of Kohlberg’s moral development theory ]. Kölner Zeitschrift für Soziologie und Sozialpsychologie, 53, 1–27.(2007). Experimental plans in factorial surveys. Random or quota design?. Sociological Methods & Research, 35, 382–409.
(1991). The analysis of factorial Surveys. Sociological Methods & Research, 19, 493–510.
(1991). Honoring accounts for sexual harassment: A factorial survey analysis. Sex Roles, 24, 725–752.
(2006). Factorial survey methods for studying beliefs and judgments. Sociological Methods & Research, 34, 334–423.
(1999). Assessing the gender gap in just earnings and its underlying mechanisms. Social Psychology Quarterly, 62, 367–380.
(1973). Attribution of fault to a rape victim as a function of respectability of the victim. Journal of Personality and Social Psychology, 26, 415–419.
(1995). Experimental design: Procedures for the behavioral sciences (3rd ed.). Pacific Grove, CA: Brooks/Cole.
(2005). Wann ist ein Steuersystem gerecht? Einstellungen zu allgemeinen Prinzipien der Besteuerung und zur Gerechtigkeit der eigenen Steuerlast [
When is a system of taxation fair? Attitudes towards general principles of taxation and fairness of the own tax burden ]. Duisburger Beiträge zur Soziologischen Forschung, 1.(2001). The factorial survey: An experimental method to replicate real world problems. Nursing Research, 50, 129–133.
(2005). Wann ist man Deutsch? Empirische Ergebnisse eines faktoriellen Surveys [
When are you German? Empirical results of a factorial survey ]. Kölner Zeitschrift für Soziologie und Sozialpsychologie, 57, 112–134.(2007). nlme: Linear and nonlinear mixed effects models. R package version, 3, 1–83.
(2003). Causal deviance and the attribution for moral responsibility. Journal of experimental social psychology, 39, 653–660.
. (2007). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved, April 15, 2007, from www.R-project.org.
(2002). Hierarchical linear models. Applications and data analysis methods (2nd ed.). London: Sage.
(1979). Vignette analysis: Uncovering the normative structure of complex judgments. In , Qualitative and quantitative social research. Papers in honor of Paul F. Larzarsfeld (pp. 176–186). New York: Free Press.
(1982). The factorial survey approach: An introduction. In , Measuring social judgments: The factorial survey approach (pp. 15–67). Beverly Hills, CA: Sage.
(1982). Measuring social judgments: The factorial survey approach. Beverly Hills, CA: Sage.
(1974). Measuring household social standing. Social Science Research, 3, 169–190.
(1985). Beyond crime seriousness: Fitting the punishment to the crime. Journal of Quantitative Criminology, 1, 59–90.
(1974). The seriousness of crimes: Normative structures and individual differences. American Sociological Review, 39, 224–237.
(1987). Linear models for unbalanced data. New York: Wiley.
(2002). Multicultural vignettes for teacher preparation. Multicultural Perspectives, 4, 10–16.
(2004). How parents evaluate child care: A factorial survey analysis of perceptions of child care quality, fair market price and willingness to pay by low-income, African American mothers. Philadelphia: Temple University.
(1996). Innovations in experimental design in attitude surveys. Annual Review of Sociology, 22, 377–399.
(2006). Experimentelle Vignettendesigns in faktoriellen Surveys [
Experimental vignette designs in factorial surveys ]. Kölner Zeitschrift für Soziologie und Sozialpsychologie, 58, 117–146.(1988). Sorting out the cuckoo’s nest: A factorial survey approach to the study of popular conceptions of mental illness. Sociological Quarterly, 29, 565–588.
(2002). Content specific vignettes as tools for research and teaching. Electronic Journal of Science Education, 6.
(1966). Assignment of responsibility for an accident. Journal of Personality and Social Psychology, 3, 73–79.
(2002). Designing vignette studies in marketing. Australasian Marketing Journal, 10, 41–58.
