Theoretical and Methodological Problems of a 10-Year Follow-Up Program Evaluation Study
Abstract
Summary In this paper, conceptual and methodological problems of school program evaluation are discussed. The data were collected in conjunction with a 10 year cross-sectional/longitudinal investigation with partial inclusion of control groups. The experiences and conclusions resulting from this long-term study are revealing not only from the vantage point of the scientific evaluation of new scholastic models, but are also valuable for program evaluation studies in general, particularly in the field of gifted education.
References
References
Baltes, P.B. Nesselroade, J.R. (1979). History and rationale of longitudinal research.. In J.R. Nesselroade & P.B. Baltes (Eds.), Longitudinal research in the study of behavior and development (pp. 1-39). New York: Academic Press.Bergman, L.R. Magnusson, D. (1990). General issues about data quality in longitudinal research.. In D. Magnusson & L.R. Bergman (Eds.), Data quality in longitudinal research (pp. 1-31). New York, NY: Cambridge University Press.Callahan, C.M. (1983). Issues in evaluating programs for the gifted.. Gifted Child Quarterly, 27, 3– 7Callahan, C.M. (1986). Asking the right questions: The central issue in evaluating programs for the gifted.. Gifted Child Quarterly, 30, 38– 42Callahan, C.M. (1993). Evaluation programs and procedures for gifted education: International problems and solutions.. In K.A. Heller, F.J. Mönks, & A.H. Passow (Eds.), International handbook of research and development of giftedness and talent (pp. 605-618). Oxford: Pergamon Press.Callahan, C.M. (2000). Evaluation as a critical component of program development and implementation.. In K.A. Heller, F.J. Mönks, R.J. Sternberg, & R.F. Subotnik (Eds.), International handbook of giftedness and talent (2nd ed., pp. 537-547). Amsterdam: Elsevier.Capaldi, D. Patterson, G.R. (1987). An approach to the problem of recruitment and retention rates for longitudinal research.. Behavioral Assessment, 9, 169– 177Colangelo, N. Davis, G.A. Eds. (1997). Handbook of gifted education (2nd ed.).. Boston, MA: Allyn & Bacon.Cooley, W.W. Lohnes, P.R. (1976). Evaluation research in education. . New York: Irvington.Davis, G.A. Rimm, S.B. (1985). Education of the gifted and talented. . Englewood Cliffs, NJ: Prentice-Hall.Esteve, R. Godoy, A. Rodriguez-Naranjo, C. Fierro, A. (1993). Program evaluation use: An empirical study.. European Journal of Psychological Assessment, 9, 77– 93Farrington, D.P. Gallagher, B. Morley, L. St. Ledger, R.J. West, D.J. (1990). Minimizing attrition in longitudinal research: Methods of tracing and securing cooperation in a 24-year follow-up-study.. In D. Magnusson & L.R. Bergman (Eds.), Data quality in longitudinal research (pp. 122-147). New York: Cambridge University Press.Flynn, J.R. (1987). Massive IQ gains in 14 nations: What IQ tests really measure.. Psychological Bulletin, 101, 171– 191Hager, W. Hasselhorn, M. (1997). Wirkungen der Testwiederholung und entwicklungsbedingte Leistungssteigerung bei der Durchführung des CFT 1 mit Erstklässlern ‘Retest effects and effects of developmental changes on the CFT 1 score in testing first-grade students’.. Zeitschrift für Psychologie, 205, 205– 229Hansen, W.B. Collins, L.M. Malotte, C.K. Johnson, C.A. Fielding, J.E. (1985). Attrition in prevention research.. Journal of Behavioral Medicine, 8, 261– 275Heller, K.A. (1997). Individuelle Bedingungsfaktoren der Schulleistung: Literaturüberblick ‘Individual determinants of academic performance: Literature review’.. In F.E. Weinert & A. Helmke (Eds.), Entwicklung im Grundschulalter ‘Development in elementary school age’ (pp. 183-201). Weinheim: PVU.Heller, K.A. Ed. (2000). Begabungsdiagnostik in der Schul- und Erziehungsberatung ‘Diagnosis of giftedness in school counseling and child guidance’ (2nd ed.).. Bern: Huber.Heller, K.A. Ed. (2001). Hochbegabung im Kindes- und Jugendalter ‘High ability in childhood and adolescence’ (2nd ed.).. Göttingen: Hogrefe.Heller, K.A. Ed. (2002). Begabtenförderung im Gymnasium. Ergebnisse einer zehnjährigen Längsschnittstudie ‘Gifted education in the German gymnasium. Findings of a 10-year longitudinal study’.. Opladen: Leske + Budrich.Heller, K.A. Hany, E.A. (1986). Identification, development, and achievement analysis of talented and gifted children in West Germany.. In K.A. Heller & J.F. Feldhusen (Eds.), Identifying and nurturing the gifted (pp. 67-82). Toronto: Huber.Heller, K.A. Hany, E.A. (2001). Standardisierte Schulleistungsmessungen ‘Standardized measurement of academic achievement’.. In F.E. Weinert (Ed.), Leistungsmessungen in Schulen ‘Academic achievement measurement in schools’ (pp. 87-101). Weinheim: Beltz.Heller, K.A. Rindermann, H. (1999). Hochbegabung, Motivation und Leistungsexzellenz: Aktuelle Forschungsbefunde zum achtjährigen Gymnasium in Baden-Württemberg ‘High ability, motivation, and achievement excellence: Findings of the 8-year German gymnasium in Baden-Württemberg’.. In Th. Fitzner, W. Stark, H.P. Kagelmacher, & Th. Müller (Eds.), Erkennen, Anerkennen und Fördern von Hochbegabten ‘Identifying and nurturing the gifted’ (pp. 81-107). Stuttgart: Klett.Heller, K.A. Gaedike, A.-K. Weinläder, H. (1985). Kognitiver Fähigkeits-Test (KFT) ‘Cognitive Abilities Test (CAT)’ (2nd ed.).. Weinheim: Beltz.Heller, K.A. Mönks, F.J. Passow, A.H. Eds. (1993). International handbook of research and development of giftedness and talent. . Oxford: Pergamon Press.Heller, K.A. Osterrieder, K. Wystrychowski, W. (1995). A longitudinal follow-up evaluation study of a statewide acceleration program for highly gifted students at the German gymnasium.. In M. Katzko & F.J. Mönks (Eds.), Nurturing talent: Individual needs and social ability (pp. 269-274). Assen/Maastricht: Van Gorcum.Heller, K.A. Reimann, R. Rindermann, H. (2000). Evaluationsbefunde zum Gymnasium mit achtjährigem Bildungsgang in Baden-Württemberg ‘Findings of the evaluation study on the G8 program in Baden-Württemberg’.. Pädagogisches Handeln, 3, 9-15– 33-36.Heller, K.A. Rosemann, B. Steffens, K.-H. (1978). Prognose des Schulerfolgs - Eine Längsschnittstudie zur Schulaufbahnberatung ‘Prediction of school success - A longitudinal study’.. Weinheim: Beltz.(1994). The program evaluation standards: How to assess evaluations of educational programs. . Thousand Oaks: Sage.
Klann-Delius, G. (1996). Einige praktische Probleme von Langzeituntersuchungen und ihre methodologischen Implikationen ‘Practical problems of longitudinal studies and their methodological implications’.. In K. Ehlich (Eds.), Kindliche Sprachentwicklung: Konzepte und Empirie ‘Development of childrens' language: Conceptions and empirical findings’ (pp. 17- 29). Opladen: Westdt. VerlagKulik, J.A. Kulik, C.-L. Bangert, R.L. (1984). Effects of practice on aptitude and achievement test scores.. American Educational Research Journal, 21, 435– 447Lehwald, G. (1981). Verfahren zur Untersuchung des Erkenntnisstrebens ‘Measurement of strive for knowledge’.. In J. Guthke & G. Witzlack (Eds.), Zur Psychodiagnostik von Persönlichkeitsqualitäten bei Schülern ‘Diagnosis of students personality’ (pp. 345-406). Berlin: Volk und Wissen.Lubinski, D. Webb, R.M. Morelock, M.J. Benbow, C.P. (2001). Top 1 in 10.00: A 10-year follow-up of the profoundly gifted.. Journal of Applied Psychology, 86, 718– 729Madaus, G.F. Scriven, M. Stufflebeam, D. Eds. (1983). Evaluation models: Viewpoints on educational and human services education. . Boston, MA: Kluwer.Murphy, M. (1990). Minimizing attrition in longitudinal studies: Means or end?. In D. Magnusson & L.R. Bergman (Eds.), Data quality in longitudinal research (pp. 148-156). New York, NY: Cambridge University Press.Perleth, Ch. Heller, K.A. (1994). The Munich longitudinal study of giftedness.. In R.F. Subotnik & K.D. Arnold (Eds.), Beyond terman: Contemporary longitudinal studies of giftedness and talent (pp. 77-114). Norwood, NJ: Ablex.Renzulli, J.S. Smith, L.H. (1979). Issues and procedures in evaluating programs.. In A.H. Passow (Ed.), The gifted and the talented: Their education and development (pp. 289-307). Chicago: University of Chicago Press.Saunders, D.R. (1956). Moderator variables in prediction.. Educational and Psychological Measurement, 16, 209– 222Schneider, W. (1989). Problems of longitudinal studies with children: Practical, conceptual, and methodological issues.. In M. Brambring, F. Lösel, & H. Skowronek (Eds.), Children at risk: Assessment, longitudinal research, and intervention (pp. 313- 335). Berlin: de Gruyter.Schneider, W. (1991). Methodische Probleme und Möglichkeiten schulbezogener Längsschnittforschung ‘Methodological problems and possibilities of school-related longitudinal research’.. In R. Pekrun & H. Fend (Eds.), Schule und Persönlichkeitsentwicklung. Ein Resümee der Längsschnittforschung ‘School and personality development. A summary of longitudinal research’ (pp. 57-80). Stuttgart: Enke.Shea, D.L. Lubinski, D. Benbow, C.P. (2001). Importance of assessing spatial ability in intellectually talented young adolescents: A 20-year longitudinal study.. Journal of Educational Psychology, 93, 604– 614Sternberg, R.J. Ferrari, M. Clinkenbeard, P. Grigorenko, E.L. (1996). Identification, instruction, and assessment of gifted children: A construct validation of a triarchic model.. Gifted Child Quarterly, 40, 129– 137Stufflebeam, D. (1983). The CIPP model for program evaluation.. In G. F. Madaus, M. Scriven, & D. Stufflebeam (Eds.), Evaluation models: Viewpoints on educational and human services education (pp. 117-142). Boston, MA: Kluwer.Süß, H.-M. Schmiedek, F. (2000). Ermüdungs- und Übungseffekte bei mehrstündiger kognitiver Beanspruchung ‘Fatigue and practice effects during cognitive tasks lasting several hours’.. Zeitschrift für Experimentelle Psychologie, 47, 162– 179Tanaka, J.S. (1987). “How big is big enough?”: Sample size and goodness of fit in structural equation models with latent variables.. Child Development, 58, 134– 146Tucker, M.S. Codding, J.B. (1998). Standards for our schools. How to set them, measure them, and reach them. . San Francisco: Jossey-Bass.Wieczerkowski, W. Nickel, H. Janowski, A. Fittkau, B. Rauer, W. (1979). A-F-S. Angstfragebogen für Schüler ‘Anxiety Questionnaire for Students’ (4th ed.).. Braunschweig: Westermann.Winefield, A.H. Tiggemann, M. Winefield, H.R. (1991). Attrition bias and internal validity in a longitudinal study of youth unemployment.. Australian Journal of Psychology, 43, 69– 73Ziegler, A. Heller, K.A. (2000). Conceptions of giftedness from a meta-theoretical perspective.. In K.A. Heller, F.J. Mönks, R.J. Sternberg, & R.F. Subotnik (Eds.), International handbook of giftedness and talent (2nd ed., pp. 3-21). Amsterdam: Elsevier.
