Abstract
Summary: The problem of measurement invariance in organizational surveys is discussed, and it is shown how mixture distribution models can be used to detect response styles in organizational surveys. The results of an analysis of a leadership performance scale with the polytomous mixed Rasch model is reported (N = 4578). The results revealed that two latent classes differing in response styles could be detected: One class (size: 71%) using the whole response scale without a strong preference for specific categories and one class (size: 29%) preferring the extreme response categories and avoiding the middle ones. Furthermore, it was shown that the two latent classes differ in demographic and other organizational variables. Finally, the implications of this study for comparing individuals across divisions and organizations as well as for future research on organizational assessment methods are discussed.
References
References
Berg, I.A. Collier, J.S. (1953). Personality and group differences in extreme response sets. Educational and Psychological Measurement, 13, 164– 169Berk, R.A. (1982). Handbook of methods for detecting test bias . Baltimore: The Johns Hopkins University PressBozdogan, H. (1987). Model selection for Akaike's information criterion (AIC). Psychometrika, 53, 345– 370Camilli, G. Shepard, L.A. (1994). Methods for identifying biased test items . Thousand Oaks, CA: SageChurch, A.H. Margiloff, A. Coruzzi, C. (1995). Using surveys for change: an applied example in a pharmaceutical organization. Leadership and Organization Development Journal, 16, 3– 11Drasgow, F. Kanfer, R. (1985). Equivalence of psychological measurement in heterogeneous populations. Journal of Applied Psychology, 70, 662– 680Dunham, R.B. Smith, F.J. (1979). Organizational surveys. An internal assessment of organizational health . Glenview, IL: Scott, Foresman and CompanyDutka, S. Frankel, L.R. (1993). Measurement errors in organizational surveys. American Behavioral Scientist, 36, 472– 484Eid, M. Emmermann, A. Schwenkmezger, P. (1996). Sind Ärgererleben und Ärgerausdruck universelle Konstrukte? Ergebnisse einer Analyse des State-Trait-Ärger-Inventars mit logistischen Mischverteilungsmodellen [Are the experience and expression of anger universal constructs? Results of an analysis of the State-trait Anger Expression Inventory with logistic mixture distribution models]. Diagnostica, 42, 3– 26Higgs, A.C. Ashworth, S.D. (1996). Organizational surveys. Tools for assessment and research . In I.A. Kraut (Ed.), Organizational surveys: Tools for assessment and research (pp. 19-40). San Francisco, CA: Jossey-BassHolland, P.W. Wainer, H. (1993). Differential item functioning . Hillsdale, NJ: ErlbaumKraut, I.A. (1996). An overview of organizational surveys. In I.A. Kraut (Ed.), Organizational surveys: Tools for assessment and research (pp.1-14). San Francisco, CA: Jossey-BassLangeheine, R. Pannekoek, J. van de Pol, F. (1996). Bootstrapping goodness-of-fit measures in categorical data analysis. Sociological Methods and Research, 24, 492– 516Leary, M.L. (1996). Self-presentation. Impression management and interpersonal behavior . Boulder, CO: WestviewLewis, N.A. Taylor, J.A. (1955). Anxiety and extreme response preferences. Educational and Psychological Measurement, 15, 111– 116Lim, R.G. Drasgow, F. (1990). Evaluation of two methods for estimating item response theory parameters when assessing differential item functioning. Journal of Applied Psychology, 75, 164– 174Masters, G.N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149– 174Millsap, R.E. Everson, H.T. (1993). Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement, 17, 297– 334Mislevy, R.J. Verhelst, N. (1990). Modeling item responses when different subjects employ different solution strategies. Psychometrika, 55, 195– 215Mislevy, R.J. Wingersky, M.S. Irvine, S.H. Dann, P.L. (1991). Resolving mixtures of strategies in spatial visualization tasks. British Journal of Mathematical and Statistical Psychology, 44, 265– 288Morris, G.W. LoVerde, M.A. (1993). Consortium surveys. American Behavioral Scientist, 36, 531– 550Nunally, J.C. (1967). Psychometric theory . New York: McGraw-HillOsterlind, S.J. (1985). Test item bias . Beverly Hills, CA: SageRead, T.R.C. Cressie, N.A.C. (1988). Goodness-of-fit statistics for discrete multivariate data . New York: Springer-VerlagReise, S.P. Gomel, J.N. (1995). Modeling qualitative variation within latent trait dimensions: Application of mixed measurement to personality assessment. Multivariate Behavioral Research, 30, 341– 358Reise, S.P. Widaman, K.F. Pugh, R.H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114, 552– 566Rost, J. (1990). Rasch models in latent classes: An integration of two approaches to item analysis. Applied Psychological Measurement, 14, 271– 282Rost, J. (1991). A logistic mixture distribution model for polychotomous item response. The British Journal of Mathematical and Statistical Psychology, 44, 75– 92Rost, J. (1996). Logistic mixture models. In W. van der Linden & R. Hambleton (Eds.), Handbook of modern item response theory (pp. 449-463). Berlin: Springer-VerlagRost, J. Carstensen, C. von Davier, M. (1997). Applying the mixed Rasch model to personality questionnaires. In J. Rost & R. Langeheine (Eds.), Applications of latent trait and latent class models in the social sciences (pp. 324-332). Münster: WaxmannRost, J. von Davier, M. (1995). Mixture distribution Rasch models. In G. Fischer & I. Molenaar (Eds.), Rasch models: Foundations, recent developments and applications (pp. 255-266). New York: Springer-VerlagVon Davier, M. (1997a). Bootstrapping goodness-of-fit statistics for sparse categorical data - results of a Monte Carlo study. Methods of Psychological Research Online, 2, 29– 48Von Davier, M. (1997b). WINMIRA - program description and recent enhancements. Methods of Psychological Research Online, 2, 25– 28Von Davier, M. Rost, J. (1995). Polytomous mixed Rasch models. In G.H. Fischer & I.W. Molenaar (Eds.), Rasch models. Foundations, recent developments, and applications (pp. 371 - 379). New York: Springer-VerlagYung, Y.-F. (1997). Finite mixtures in confirmatory factor-analysis models. Psychometrika, 62, 297– 330Zickar, M.J. Drasgow, F. (1996). Detecting faking on a personality instrument using appropriateness measurement. Applied Psychological Measurement, 20, 71– 87
