Adaptive Measurement of Individual Change
Abstract
Adaptive measurement of change (AMC) was investigated by examining the recovery of true change. Monte Carlo simulation was used to compare three conventional testing (CT) methods with AMC. The CTs estimated individual change moderately well when the test was highly discriminating and when the θ level matched the test difficulty. However, AMC measured individual change equally well across the entire range of θ. AMC with more discriminating items produced the most precise estimates of individual change. AMC was shown to be superior to CTs under all conditions examined. In addition, AMC is efficient – it can dramatically reduce the number of items necessary to measure individual change. The results indicate that AMC is a viable and effective method for measuring individual change.
References
(1979). Introduction to measurement theory. Monterey, CA: Brooks/Cole.
(1985). Estimating latent correlations between repeated testings. Psychometrika, 50, 3–16.
. (1997). User“s manual for the PARDSIM parameter and response data simulation program. St. Paul, MN: Author.
. (1998). User“s manual for the conventional test scoring program. St. Paul, MN: Author.
(1992). Item response theory: Parameter estimation techniques. New York: Marcel Dekker.
(1963). Some persisting dilemmas in the measurement of change. In , Problems in measuring change (pp. 3–20). Madison, WI: University of Wisconsin Press.
(1976). Basic issues in the measurement of change. In , Advances in psychological and educational measurement (pp. 75–96). New York: Wiley.
(1990). Change measurement. In , Statistical methods in longitudinal research (Vol. 1, pp. 3–34). Boston, MA: Academic Press.
(1970). How we should measure “change” – or should we? Psychological Bulletin, 74, 68–80.
(1991a). A multidimensional latent trait model for measuring learning and change. Psychometrika, 56, 495–515.
(1991b). Implications of a multidimensional latent trait model for measuring change. In , Best methods for the analysis of change (pp. 184–197). Washington, DC: American Psychological Association.
(1995). A measurement model for linking individual learning to processes and knowledge: Application to mathematical reasoning. Journal of Educational Measurement, 32, 277–294.
(1976). Some probabilistic models for measuring change. In , Advances in psychological and educational measurement (pp. 97–110). New York: Wiley.
(1985). Item response theory: Principles and applications. Boston, MA: Kluwer-Nijhoff.
(1988). Statistics (4th ed.). New York: Holt, Rinehart, Winston.
(1992). Statistical methods for psychology (3rd ed.). Boston, MA: PWS-KENT.
(1975). Practical guidelines in applying current theories to the measurement of change. I. Problems in measuring change and recommended procedures. JSAS Catalog of Selected Documents in Psychology, 5, 226 (Ms. No. 916).
(1983). A comparison of IRT-based adaptive mastery testing and a sequential mastery testing procedure. In , New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 257–283). New York: Academic Press.
(1963). Elementary models for measuring change. In , Problems in measuring change (pp. 21–38). Madison, WI: The University of Wisconsin Press.
(1968). Statistical theories of mental test scores. Menlo Park, CA: Addison-Wesley.
(1962). Correlation methods in research on human learning. Perceptual and Motor Skills, 15, 287–321.
(1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum.
(1971). Planning the objective test. In , Educational measurement (2nd ed., pp. 46–80). Washington, DC: American Council on Education.
(1967). A note on the reliability of residual change scores. Journal of Educational Measurement, 4, 253–256.
(1966). A base-free measure of change. Psychometrika, 31, 457–473.
(1977). Tailored testing: A successful application of latent trait theory. Journal of Educational Measurement, 14, 181–196.
(1982). Improving measurement quality and efficiency with adaptive testing. Applied Psychological Measurement, 6, 473–492.
(1983). Introduction. In , New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 1–8). New York: Academic Press.
(1985). Adaptive testing by computer. Journal of Consulting and Clinical Psychology, 53, 774–789.
(1995). Improving individual differences measurement with item response theory and computerized adaptive testing. In , Assessing individual differences in human behavior: New concepts, methods, and findings (pp. 49–79). Palo Alto, CA: Davies-Black.
(2004). Computerized adaptive testing for effective and efficient measurement in counseling and education. Evaluation in Counseling and Development, 37, 70–84.
(2005). Manual for POSTSIM: Posthoc simulation of computerized adaptive testing. Version 2.0. St. Paul, MN: Assessment Systems Corporation.
(1984). Application of computerized adaptive testing to educational problems. Journal of Educational Measurement, 21, 361–375.
(1994). Measurement of change. In , The international encyclopedia of education (2nd ed., pp. 671–678). Oxford, UK: Pergamon.
(1997). Measuring change: What individual growth modeling buys you. In , Change and development: Issues of theory, method, and application (pp. 213–243). Mahwah, NJ: Erlbaum.
(1986). The choice of scale for educational measurement: An IRT perspective. Journal of Educational Measurement, 23, 299–325.
(1993). A comparison of the effectiveness of item parameter estimation techniques used with the 3-parameter logistic item response theory model. Unpublished doctoral dissertation, University of Minnesota, Minneapolis.
(1994). A simulation study of methods for assessing differential item functioning in computerized adaptive testing. Applied Psychological Measurement, 18, 121–140.
