Psychometric Comparison of Automated Versus Rational Methods of Scale Abbreviation
An Illustration Using a Brief Measure of Values
Abstract
A family of automated analyses known as Genetic Algorithms (GAs) have recently been applied to abbreviating scales in the psychological literature. This method allows long scales to be abbreviated in a matter of minutes, representing a huge potential savings of time over standard rational methods, which are based on psychometric analyses. To compare the results of the automated GA approach with those of the traditional approach, we used both methods to abbreviate a 40-item measure of psychological values (Portrait Values Questionnaire). The two methods yielded 10- and 20-item measures of values, which proved to be successful at recapturing the properties of the original measure. The abbreviated scales created from two approaches displayed strikingly similar psychometric qualities. The advantages and disadvantages of the different approaches are discussed.
References
2006). The NPI-16 as a short measure of narcissism. Journal of Research in Personality, 40, 440–450.
(2003). Values and behavior: strength and structure of relations. Personality & Social Psychology Bulletin, 29, 1207–1220.
(1997). Methodological approaches to Shortening Composite Measurement Scales. Journal of Clinical Epidemiology, 50, 247–252.
(2012). An evaluation of the consequences of using short measures of the Big Five personality traits. Journal of Personality and Social Psychology, 102, 874–888.
(1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302.
(2003). A very brief measure of the Big-Five personality domains. Journal of Research in Personality, 37, 504–528.
(2004). Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires. American Psychologist, 59, 93–104.
(1977). Empirical item keying versus a rational approach to analyzing a psychological climate questionnaire. Applied Psychological Measurement, 1, 489–500.
(1991). The Big Five Inventory – Versions 4a and 54. Berkeley, CA: University of California, Berkeley, Institute of Personality and Social Research.
(2007). The importance of being valid. In , Handbook of research methods in personality psychology (pp. 461–494). New York, NY: Guilford Press.
(2010). The dirty dozen: A concise measure of the dark triad. Psychological Assessment, 22, 420–432.
(1992). Comparing correlated correlation coefficients. Psychological Bulletin, 111, 172–175.
(2005). The application of short forms of the Wechsler Intelligence Scales in adults and children with high functioning autism. Journal of Autism and Developmental Disorders, 35, 45–52.
(1992). The effect of acquaintanceship on the validity of personality impressions: A longitudinal study. Journal of Personality and Social Psychology, 63, 816–824.
(2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna, Austria: R Core Team. Retrieved from www.R-project.org/
. (2001). Measuring global self-esteem: Construct validation of a single-item measure and the Rosenberg Self-Esteem Scale. Personality and Social Psychology Bulletin, 27, 151–161.
(2004). Values, intelligence and client behavior in career counseling: A field study. European Journal of Psychology of Education, 19, 237–255.
(2014). The development and validation of brief and ultra-brief measures of values. Manuscript in preparation.
(1992). Universals in the content and structure of values: Theoretical advantages and empirical tests in 20 countries. In , Advances in experimental and social psychology, Vol. 25, (pp. 1–65). New York, NY: Academic Press.
(2001). Value hierarchies across cultures: Taking a similarities perspective. Journal of Cross-Cultural Psychology, 32, 268–290.
(2001). Extending the cross-cultural validity of the theory of basic human values with a different method of measurement. Journal of Cross-Cultural Psychology, 32, 519–542.
(2011). Advice on total-score reliability issues in psychosomatic measurement. Journal of Psychosomatic Research, 70, 565–572.
(2000). On the sins of short-form development. Psychological Assessment, 12, 102–111.
(2007). The experiences in Close Relationship Scale (ECR)-short form: reliability, validity, and factor structure. Journal of Personality Assessment, 88, 187–204.
(2010a). The abbreviation of personality, or how to measure 200 Personality Scales with 200 items. Journal of Research in Personality, 44, 180–198.
(2010b, March 31). Abbreviating personality measures in R: A tutorial. Retrieved from www.talyarkoni.org/blog/2010/03/31/abbreviating-personality-measures-in-r-a-tutorial/
(