Current Issues in Competence Modeling and Assessment
Abstract
The goals of education and qualification in modern industrial societies can no longer be described by a fixed set of specialized skills that are transferable from one generation to the next. Nowadays, knowledge must be applicable to different, new, and complex situations and contexts. It is against this background that the concept of competence has attracted increased research attention. Competencies are conceptualized as complex ability constructs that are context-specific, trainable, and closely related to real life. The theoretical modeling of competencies, their assessment, and the usage of assessment results in practice present new challenges for psychological and educational research. This article reviews current issues in competence modeling, outlining research questions and the current state of research, and identifying the need for more interdisciplinary research. Finally, a research program recently initiated by the German Research Foundation (DFG) to address these questions and demands is presented.
References
2005). PISA 2003 technical report. Paris: OECD.
(1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1–23.
(1997). Multilevel item response modeling: An approach to errors in variables regression. Journal of Educational and Behavioral Statistics, 22, 47–76.
(2002). PISA 2000 technical report. Paris: OECD.
(2005). Diagnosing foreign language proficiency: The interface between learning and assessment. London: Continuum.
(2005). The Dutch CEF grid reading/listening (revised internet version available for test development and analysis). Retrieved from http://www.lancs.ac.uk/fss/projects/grid/
(2005). Entwicklung und Implementierung eines kombinierten Beratungs- und Auswahlverfahrens für die wichtigsten Studiengänge an der Universität Heidelberg [
(Development and implementation of a combined instrument for counseling and selection for the most important courses at the University of Heidelberg ]. Psychologische Rundschau, 56, 135–137.2001). Predictors of reading literacy. European Journal of Psychology of Education, 16, 363–383.
(2001). PISA 2000: Untersuchungsgegenstand, theoretische Grundlagen und Durchführung der Studie [
(Subject, theoretical background, and implementation of the study ]. In , PISA 2000. Basiskompetenzen von Schülerinnen und Schülern im internationalen Vergleich [PISA 2000. Students basic competencies in international comparison ] (pp. 15–68). Opladen: Leske & Budrich.2007). Sprachliche Kompetenzen – Konzepte und Messung [
(Language competencies – Concepts and measurement ]. Weinheim: Beltz.2004). Mathematische Kompetenz [
(Mathematical literacy ]. In , ISA 2003. Der Bildungsstand der Jugendlichen in Deutschland – Ergebnisse des zweiten internationalen Vergleichs [PISA 2003: Educational outcomes of German students – Results of the second international study ] (pp. 47–92). Münster: Waxmann.1997). Toward an understanding of scientific literacy. In , Scientific literacy, an international Symposium (pp. 37–68). Kiel: IPN.
(2004). On text structure, language proficiency, and reading comprehension test format interactions: A reply to Kobayashi, 2002. Language Testing, 21, 228–234.
(2004). Washback in language testing: Research contexts and methods. Mahwah, NJ: Erlbaum.
(in press ). Computer-based assessments to support distance learning. In , Assessment of competencies in educational contexts: State of the art and future prospects. Göttingen: Hogrefe & Huber.2001). More unintended consequences of high-stakes testing. Educational Measurement, Issues, and Practice, 20(4), 19–28.
(2004). A NCME instructional module on setting performance standards: Contemporary methods. Educational Measurement, Issues and Practice, 23, 31–50.
(2003). On abilities and domains. In , The psychology of abilities, competencies, and expertise (pp. 126–155). Cambridge, MA: Cambridge University Press.
(2004). Knowledge and competencies. In , The integrated person. How curriculum development relates to new competencies (pp. 35–49). Enschede: CIDREE/ SLO.
(2007). Guest editors introduction and overview: IRT-based cognitive diagnostic models and related methods. Journal of Educational Measurement, 44, 285–291.
(2006). A history of conceptual change research. In , The Cambridge handbook of the learning sciences (pp. 265–281). Cambridge, MA: Cambridge University Press.
(2004). Problem solving for tomorrows world. First measures of cross-curricular competencies from PISA 2003. Paris: OECD.
(2002). The work ahead: A psychometric infrastructure for computerized adaptive testing. In , Computer-based testing. Building the foundation for future assessments (pp. 1–35). Mahwah, NJ: Erlbaum.
(1983). Construct validity: construct representation vs. nomothetic span. Psychological Bulletin, 93, 179–197.
(2006). The continued search for nonarbitrary metrics in psychology. American Psychologist, 61, 50–55.
(2005). TOEFL iBT at a glance. Retrieved September 25, 2005, from http://www.ets.org/Media/Test/TOEFL/pdf/TOEFL_at_ a_Glance.pdf
(1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37, 359–374.
(2003). The route from implicit learning to awareness of what has been learned. In , Attention and implicit learning (pp. 335–366). New York: John Benjamins.
(2007). Usability and internal validity of a modification of the computer game Quake III Arena® for the use in psychological experiments. Computers in Human Behavior, 23, 2026–2039.
(2004). Redesigning accountability systems for education. New York: Teachers College Press.
(2007). Using the attribute hierarchy method to make inferences about examinees cognitive skills. In , Cognitive diagnostic assessment for education (pp. 242–274). Cambridge, MA: Cambridge University Press.
(2002). Linguistic and cultural diversity in Europe: A challenge for educational research and practice. European Educational Research Journal, 1, 123–138.
(2005). Prognose der Studierfähigkeit. Ergebnisse aus Längsschnittanalysen [
(Prediction of college graduation. Results from longitudinal studies ]. Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie, 37, 214–222.2004). Familial influences on sustained attention and inhibition in preschoolers. Journal of Child Psychology and Psychiatry and Allied Disciplines, 45, 306–314.
(2004). Validating standards-based test score interpretations. Measurement: Interdisciplinary Research and Perspectives, 2, 61–103.
(1996). The role of information reduction in skill acquisition. Cognitive Psychology, 30, 304–337.
(1997). Lernmechanismen des kognitiven Fertigkeitserwerbs [
(Learning mechanisms in cognitive skill acquisition ]. Zeitschrift für Experimentelle Psychologie, 44, 521–560.2002). Why individual learning does not follow the power law of practice but aggregated learning does: Comment on Rickard (1997, 1999), Delaney et al. (1998), and Palmeri (1999). Journal of Experimental Psychology: Learning, Memory, and Cognition, 28, 392–406.
(2005, July). Application of different explanatory item response models for model based proficiency scaling. Paper presented at the 70th Annual Meeting of the Psychometric Society in Tilburg.
(2008). Representation of competencies in multidimensional IRT models with within- and between-item multidimensionality. Zeitschrift für Psychologie / Journal of Psychology, 216, 88–100.
(2006). Kompetenz und Kompetenzdiagnostik [
(Competence and competence diagnosis ]. In , Leistung und Leistungsdiagnostik [Performance and assessment of performance] (pp. 127–143). Berlin: Springer-Verlag.2007, August). From theoretical notions of competence to adequate psychometric models. Paper presented at the 12th Biennial EARLI Conference, Budapest, Hungary.
(2007). Anforderungen an Computer- und Netzwerkbasiertes Assessment [
(Requirements for computer- and network-based assessments ]. In , Möglichkeiten und Voraussetzungen technologiebasierter Kompetenzdiagnostik [Possibilities and preconditions for technology-based assessment of competencies ] (pp. 57–67). Berlin: Federal Ministry of Education and Research (available at http://www.bmbf.de/pub/band_zwanzig_bildungsforschung.pdf).2003). The phonological similarity effect on memory span in children: Does it depend on age, speech rate, and articulatory suppression? International Journal of Behavioral Development, 27, 145–152.
(2007). The demand for cognitive diagnostic assessment. In , Cognitive diagnostic assessment for education (pp. 19–61). Cambridge, MA: Cambridge University Press.
(2004). Models with item and item group predictors. In , Explanatory item response models: A generalized linear and nonlinear approach (pp. 189–212). New York: Springer-Verlag.
(2000). A hierarchical IRT model for criterion-referenced measurement. Journal of Educational and Behavioral Statistics, 25, 285–306.
(2007). Neue Chancen bei der technologiebasierten Erfassung von Kompetenzen [
(New opportunities of technology based assessment of competencies ]. In , Möglichkeiten und Voraussetzungen technologiebasierter Kompetenzdiagnostik [Possibilities and preconditions for technology-based assessment of competencies ] (pp. 81–91). Berlin: Federal Ministry of Education and Research (available at www.bmbf.de/pub/band_zwanzig_bildungsforschung.pdf).2005). The affective virtual patient: An e-learning tool for social interaction training within medical field. In Proceedings TESI 2005 – Training Education & Education International Conference. Kent, UK: Nexus Media (available at http://isnm.de/aahad/Downloads/AVP_TESI. pdf).
(1978). Perspektiven pädagogischer Diagnostik [
(Perspectives of educational assessment at the individual level ]. In , Handbuch der Pädagogischen Diagnostik [Handbook of educational assessment ] (pp. 3–4). Düsseldorf: Schwann.1987). Kriteriumsorientierte Tests [
(Criterion-referenced tests ]. Göttingen: Hogrefe.2007). Lehren und Lernen. Einführung in die Instruktionspsychologie [
(Teaching and learning. Introduction to instructional psychology ]. Weinheim: Beltz-PVU.2003). The development of national educational standards. An expertise. Berlin: Federal Ministry of Education and Research (available at www.bmbf.de/pub/the_development_of_national_educational_standards.pdf).
(2008). Unterricht und Kompetenzerwerb in Deutsch und Englisch. Ergebnisse der DESI-Studie [
(Instruction and competence development in German and English. Results of the DESI study ]. Weinheim: Beltz.2001). Problemlösen als fächerübergreifende Kompetenz? Konzeption und erste Resultate aus einer Schulleistungsstudie [
(Problem solving as cross-curricular competence? Concepts and first results from an educational assessment ]. Zeitschrift für Pädagogik, 47, 179–200.2006). Kompetenzmodelle zur Erfassung individueller Lernergebnisse und zur Bilanzierung von Bildungsprozessen. Beschreibung eines neu eingerichteten Schwerpunktprogramms bei der DFG [
(Competence models for assessing individual learning outcomes and evaluating educational processes. Description of a new priority program of the German Research Foundation, DFG ]. Zeitschrift für Pädagogik, 52, 876–903.2007). Kompetenzbegriff und Bedeutung von Kompetenzen im Bildungswesen [
(The concept and relevance of competencies in education ]. In , Möglichkeiten und Voraussetzungen technologiebasierter Kompetenzdiagnostik [Possibilities and preconditions for technology-based assessment of competencies ] (pp. 5–16). Berlin: Federal Ministry of Education and Research (available at www.bmbf.de/pub/band_zwanzig_bildungsforschung.pdf).2001). Mathematische Grundbildung: Testkonzeption und Ergebnisse [
(Mathematical literacy: Assessment framework and results ]. In , PISA 2000. Basiskompetenzen von Schülerinnen und Schülern im internationalen Vergleich [PISA 2000. Students basic competencies in international comparison ] (pp. 139–190). Opladen: Leske & Budrich.2004). Konsequenzen von Leistungsgruppierungen [
(Consequences of homogeneous groups with regard to school performance ]. Münster: Waxmann.2002). Method effects on reading comprehension test performance: Text organization and response format. Language Testing, 19, 193–220.
(2005). Intelligence assessment with computer simulations. Intelligence, 33, 347–368.
(2008). Strategisches Experimentieren im naturwissenschaftlichen Unterricht [
(Strategic experimentation in science lessons ]. Psychologie in Erziehung und Unterricht, 55, 1–15.2002). The fuzzy relationship of intelligence and problem solving in computer simulations. Computers in Human Behavior, 18, 685–697.
(1998). Measuring learning styles with questionnaires versus direct observation of preferential choice behavior: Development of the Visualizer/Verbalizer Behavior Observation Scale (VV-BOS). Computers in Human Behavior, 14, 543–557.
(2007). Landesweite Lernstandserhebung zwischen Bildungsmonitoring und Individualdiagnostik [
(State-wide standardized assessments of learning between educational monitoring and individual diagnostics ]. Zeitschrift für Erziehungswissenschaft, Sonderheft 8, 149–167.1999). Estimating multiple classification latent class models. Psychometrika, 64, 187–212.
(1973). Testing for competence rather than for intelligence. American Psychologist, 28, 1–14.
(2000). A basis for multidimensional item response theory. Applied Psychological Measurement, 24, 99–114.
(2006). High-stakes testing and student achievement: Does accountability pressure increase student learning? Education Policy Analysis Archives, 14 (available at http://epaa.asu.edu/epaa/v14n1/).
(2003). DESI – A language assessment project in Germany and the pros and cons of large-scale testing. Empirische Pädagogik, 17, 368–379.
(2007a). Hörverstehen [
(Listening comprehension ]. In , Sprachliche Kompetenzen. Konzepte und Messung [Language competencies – Concepts and measurement ] (pp. 178–196). Weinheim: Beltz.2007b). Leseverstehen [
(Reading comprehension ]. In , Sprachliche Kompetenzen. Konzepte und Messung [Language competencies – Concepts and measurement ] (pp. 197–211). Weinheim: Beltz.2005). Working memory and intelligence – Their correlation and their relation: A comment on Ackerman, Beier, and Boyle (2005). Psychological Bulletin, 131, 61–65.
(2007a). PISA – Program for International Student Assessment (available at www.oecd.org/dataoecd/51/27/ 37474503.pdf).
(2007b). PISA 2006 Science competencies for tomorrows world (Volume 1: Analysis). Paris: OECD.
(2004). SET-10 test description and validation summary. Menlo Park, CA: Ordinate.
(2001). Knowing what students know. The science and design of educational assessment. Washington, DC: National Academic Press.
(1998). Supporting visualizer and verbalizer learning preferences in a second-language multimedia learning environment. Journal of Educational Psychology, 90, 25–36.
(2006). Untersuchungen zur Bildungsqualität von Schule. Abschlussbericht des DFG-Schwerpunktprogramms [
(Research on educational quality of schools. Final report of the DFG priority program ]. Münster: Waxmann.2007). PISA 2006. Die Ergebnisse der dritten internationalen Vergleichsstudie [
(PISA 2006. Results of the third international study ]. Münster: Waxmann.2004). PISA 2003: Der Bildungsstand der Jugendlichen in Deutschland – Ergebnisse des zweiten internationalen Vergleichs [
. (PISA 2003: Educational outcomes of German students – Results of the second international study ]. Münster: Waxmann.2005). PISA 2003: Der zweite Vergleich der Länder in Deutschland – Was wissen und können Jugendliche? [
. (The second comparison of the German states – What do students know? ]. Münster: Waxmann.2001). Naturwissenschaftliche Grundbildung: Testkonzeption und Ergebnisse [
(Scientific literacy: Assessment framework and results ]. In , PISA 2000. Basiskompetenzen von Schülerinnen und Schülern im internationalen Vergleich [PISA 2000. Students basic competencies in international comparison ] (pp. 192–248). Opladen: Leske & Budrich.2007). Naturwissenschaftliche Kompetenzen im internationalen Vergleich [
(Science competencies in international comparison ]. In , PISA 2006. Die Ergebnisse der dritten internationalen Vergleichsstudie [PISA 2006. Results of the third international comparison study ] (pp. 63–105). Münster: Waxmann.1997). A linear logistic multidimensional model for dichotomous item response data. In , Handbook of modern item response theory (pp. 271–286). New York: Springer-Verlag.
(2007). Technische Lösungen für ein computer- und internetbasiertes Assessment-System [
(Technical solutions for computer- and internet-based assessment systems ]. In , Möglichkeiten und Voraussetzungen technologiebasierter Kompetenzdiagnostik [Possibilities and preconditions for technology-based assessment of competencies ] (pp. 81–91). Berlin: Federal Ministry of Education and Research (available at www.bmbf.de/pub/band_zwanzig_bildungsforschung.pdf).2001). Defining and selecting key competencies. Seattle: Hogrefe & Huber.
(2003). Key competencies for a successful life and a well-functioning society. Washington: Hogrefe & Huber.
(2005). Interrelationships among theory of mind, executive control, language development, and working memory in young children: A longitudinal analysis. In , Young childrens cognitive development: Interrelationships among executive functioning, working memory, verbal ability, and theory of mind (pp. 259–284). Mahwah, NJ: Erlbaum.
(2004). Deconstructing instructional design models toward an integrative conceptual framework for instructional design research. In , Instructional design and multimedia learning (pp. 71–89). Münster: Waxmann.
(1999). New perspectives on conceptual change. Oxford: Elsevier.
(2003). Optimizing new modes of assessment: In search of quality and standards. Dordrecht: Kluwer.
(2003). Expertise, competence, and creative ability: The perplexing complexities. In , The psychology of abilities, competencies, and expertise (pp. 213–239). Cambridge, MA: Cambridge University Press.
(2002). Evidence-based education policies: Transforming educational practice and research. Educational Researcher, 31, 15–21.
(in press ). A model based test of competence profile and competence level in deductive reasoning. In , Assessment of competencies in educational contexts: State of the art and future prospects. Göttingen: Hogrefe.2003). The psychology of abilities, competencies, and expertise. New York: Cambridge University Press.
(2007). Skills diagnosis using IRT-based continuous latent trait models. Journal of Educational Measurement, 44, 313–324.
(2004). Person regression models. In , Explanatory item response models: A generalized linear and nonlinear approach (pp. 167–187). New York: Springer-Verlag.
(2005). A comparison of item-selection methods for adaptive tests with content constraints. Journal of Educational Measurement, 42, 283–302.
(1996). Item response theory: Brief history, common models, and extensions. In , Handbook of modern item-response theory (pp. 1–28). Berlin: Springer-Verlag.
(2005). A general diagnostic model applied to language testing data. ETS Research Report 0x-2005. RR-05-16.
(2001). Designing learning environments to promote conceptual change in science. Learning and Instruction, 15, 317–419.
(2003). Comparing multidimensional and unidimensional proficiency classifications: Multidimensional IRT as a diagnostic aid. Journal of Educational Measurement, 40, 255–275.
(2005). The Rasch testlet model. Applied Psychological Measurement, 29, 126–149.
(1999). Konzepte der Kompetenz [
(Concepts of competence ]. Paris: OECD.2001). Concept of competence: A conceptual clarification. In , Defining and selecting key competencies (pp. 45–65). Seattle: Hogrefe & Huber.
(1992). Research on the model teacher and the teaching model: Theoretical contradiction or conglutination? In , Effective and responsible teaching: The new synthesis (pp. 249–260). San Francisco: Jossey-Bass.
(1995). Memory performance and competencies: Issues in growth and development. Hillsdale, NJ: Erlbaum.
(1981). Measuring aptitude processes with multicomponent latent trait models. Journal of Educational Measurement, 18, 67–84.
(2005). Understanding and measuring intelligence. London: Sage.
(Cognitive diagnosis using item response models. Zeitschrift für Psychologie / Journal of Psychology, 216, 73–87.
2004). Some links between large-scale and classroom assessments: The case of the BEAR Assessment System. In , Toward coherence between classroom assessment and accountability (103rd Yearbook of the National Society for the Study of Education, Part II, pp. 132–154). Chicago: University of Chicago Press.
(2000). From principles to practice: An embedded assessment system. Applied Measurement in Education, 13, 181–208.
(2004). Descriptive and explanatory item response models. In , Explanatory item response models. A generalized linear and nonlinear approach (pp. 43–74). New York: Springer-Verlag.
(in press ). Explanatory item response models: A brief introduction. In , Assessment of competencies in educational contexts: State of the art and future prospects. Göttingen: Hogrefe & Huber.2004). Selbstregulation von Lernprozessen [
(Self-regulation in learning processes ]. Münster: Waxmann.2003). Computer-based assessment of problem-solving competence. Assessment in Education: Principles, Policy, and Practice, 10, 329–345.
(2008). Self-regulated learning as a competence. Implications of theoretical models for assessment methods. Zeitschrift für Psychologie / Journal of Psychology, 216, 101–109.
(