The Psychometric Evaluation of a Multidimensional Measurement Instrument of Professional Expertise1
Abstract
Summary:The present study of small- and medium-sized enterprises aimed to determine the reliability and validity of a measurement instrument for professional expertise. The instrument consists of five different scales (with contents ranging from 12 to 19 items each), each reflecting one of five dimensions: knowledge, metacognitive knowledge, skills, social recognition, growth, and flexibility. Participants were two different groups of raters: individual employees (N = 233) and their immediate supervisors (N = 217). The item-sets for both groups are nominally identical, except that wording was adapted to suit one group or the other. The reliabilities of the developed scales are high with Cronbach's Alpha ranging from .83 to .92, for both the self-ratings and the supervisor ratings. All five scales are homogeneous and distinctive as corroborated by the multitrait-multimethod analysis.
References
References
Alexander, P.A. Schallert, D.L. Hare, V.C. (1991). Coming to terms: How researchers in learning and literacy talk about knowledge.. Review of Educational Research, 61, 315– 344Arnold, J. Mackenzie Daveys, K. (1992). Self-ratings and supervisor ratings of graduate employees' competences during early career.. Journal of Occupational and Organizational Psychology, 65, 235– 250Bechger, T. (1997). Browne's composite direct product model for MTMM correlation matrices.. Structural Equation Modeling, 5(2), 191– 204Boerlijst, J.G. (1994). The neglect of growth and development of employees over 40 in organizations: A managerial and training problem.. In J. Snel & R. Cremer (Eds.), Work and aging (pp. 251-271). London: Taylor & Francis.Boerlijst, J.G. Van der Heijden, B.I.J.M. (1996). Human resources management in distress: Pitfalls for gifted workers in organizations.. In U. Munandar & C. Semiawan (Eds.), Optimizing excellence in human resource development (Proceedings of the Fourth Asia Pacific Conference on Giftedness, Jakarta, 4-8 August 1996, pp. 473-480). Jakarta: University of Indonesia Press.Boerlijst, J.G. Van der Heijden, B.I.J.M. Van Assen, A. (1993). Veertig-plussers in de onderneming ‘Over-forties in the organization’.. Assen: Van Gorcum/Stichting Management Studies.Campbell, D.T. Fiske, D.W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix.. Psychological Bulletin, 56, 81– 105Campbell, D.J. Lee, C. (1988). Self-appraisal in performance evaluation: Development versus evaluation.. Academy of Management Review, 13, 302– 314Chanaron, J. (1998). Managing innovation in European small and medium-sized enterprises. Nijmegen lectures on Innovation Management.. Nijmegen Business School, Catholic University Nijmegen. Antwerpen: Maklu.Cooper, W. (1981). Ubiquitous halo.. Psychological Bulletin, 90, 218– 244Cronbach, L.J. (1990). Essentials of psychological testing. . New York: Harper Collins Publishers.Cronbach, L.J. Gleser, G.C. Rajaratnam, N. Nanda, H. (1972). The dependability of behavioral measurements. . New York: Wiley.Deshpande, S.P. Golhar, D.Y. (1994). HRM practices in large and small manufacturing firms: A comparative study.. Journal of Small Business Management, 32(2), 49– 56Duberly, J.P. Walley, P. (1995). Assessing the adoption of HRM by small- and medium-sized manufacturing organizations.. The International Journal of Human Resource Management, 6, 891– 909Ericsson, K.A., Smith, J. (1991). Toward a general theory of expertise. Prospects and limits. . Cambridge: Cambridge University Press.Gulliksen, H. (1950). Theory of mental tests. . New York: Wiley.Harris, M.M. Schaubroeck, J. (1988). A metaanalysis of self-supervisor, self-peer, and peer-supervisor ratings.. Personnel Psychology, 41, 43– 62Hellriegel, D. Slocum, J.W. Jr. Woodman, R.W. (1995). Organizational behavior (7th edition).. St. Paul, MN: West Publishing.Hoffman, C.C. Nathan, B.R. Holden, L.M. (1991). A comparison of validation criteria: Objective versus subjective performance measures and self- versus supervisor ratings.. Personnel Psychology, 44, 601– 619Holzbach, R.L. (1978). Rater bias in performance ratings: Superior, self-, and peer ratings.. Journal of Applied Psychology, 63(5), 579– 588Hornsby, J.S. Kuratko, D.F. (1990). Human Resource Management in small business: Critical issues for the 1990's.. Journal of Small Business Management, 28(3), 9– 18Hunt, J. Collins, R.R. (1983). Managers in mid career crisis. . Sydney: Wellington Lane Press.Janssen, P. (1992). Relatieve deprivatie in de middenloopbaanfase bij hoger opgeleide mannen. Een vergelijking tussen drie leeftijdsgroepen ‘Relative deprivation in the mid-career phase in more highly educated males. A comparison of three age groups’.. Dissertation, Catholic University Nijmegen. Maastricht: Datawyse.Jöreskog, K.G. Sörbom, D. (1979). Advances in factor analysis and structural equation models. . Cambridge, MA: Abt Books.Jöreskog, K.G. Sörbom, D. (1993). LISREL VIII: User's reference guide. . Mooresville: Scientific Software.Koch, C. De Kok, J. (1999). A human-resource-based theory of the small firm. . Zoetermeer: EIM.Legge, K. (1995). Human Resource management: Rhetorics and realities. . London: Macmillan Business.Lord, F.M. Novick, M.R. (1968). Statistical theories of mental test scores. . Reading, MA: Addison-Wesley.Mabe, P. West, S. (1982). Validity of self-evaluation of ability: A review and metaanalysis.. Journal of Applied Psychology, 67, 280– 296Munnichs, J.M.A. (1989). Loopbaan, levensloop en de midleven-ervaring ‘Career, life course and the mid-life experience’.. In J. Munnichs & G. Uildriks (Eds.), Psychogerontologie. Een inleidend leerboek over ouder worden, persoonlijkheid, zingeving, levensloop en tijd, sociale context, gezondheid en interventie ‘Psychogerontology: An introductory textbook on aging, personality, meaning of life, course of life and time, social context, health, and intervention’ (pp. 223-228). Deventer: Van Loghum Slaterus.(1997). Globalisation and small and medium sized enterprises (SME's). Vol. 1: Synthesis report. . Paris: OECD.
Roe, R.A. Dani;ls, M.J.M. (1990). Personeelsbeoordeling: Achtergrond en toepassing ‘Personnel assessment: Background and application’.. Assen/Maastricht: Van Gorcum.(2000). Internet-site: www.mkb.nl/mkbnederland
Schoemaker, M.J.R. Geerdink, Th.D. (1991). Human talent management: Een visie of besturen, faciliteren en ontwikkelen van personeel ‘Human talent management: A perception of managing, facilitating and development of personnel’.. Deventer: Kluwer Bedrijfswetenschappen/NVP.Stoker, J.I. Van der Heijden, B.I.J.M. (2001). Self-perceptions of employees and supervisors and their consequences for competence development.. Journal of Career Development,Storey, D.J. (1992). Developments in the management of human resources. . Princeton: Princeton University Press.Storey, D.J. (1994). Understanding the small business sector. . London: Routledge.Sullivan, J.L. Feldman, S. (1979). Multiple indicators. An introduction (Series: Quantitative applications in the social sciences).. Beverly Hills: Sage.Thornton, G. (1980). Psychometric properties of self-appraisal of job performance.. Personnel Psychology, 33, 262– 271Van der Heijden, B.I.J.M. (1996). Life-long expertise development: Goal of the nineties. Proceedings of the Fifth Conference on International Human Resource Management . San Diego, CA, 24-28 June, 1996.Van der Heijden, B.I.J.M. (1998). The measurement and development of professional expertise throughout the career. A retrospective study among higher level Dutch professionals. . Dissertation, University of Twente, Enschede, The Netherlands. Enschede: PrintPartners Ipskamp.Van der Heijden, B.I.J.M. (2000). The development and psychometric evaluation of a multidimensional measurement instrument of professional expertise.. High Ability Studies. The Journal of the European Council for High Ability, 11(1), 9– 39Van Sluijs, E. Kluytmans, F. (1996). Management van competenties ‘Management of competences’.. M & O, 3, 200– 220Verhelst, N. Vander Steene, G. (1972). A Gulliksen item analysis program.. Behavioral Science, 17, 491– 493
