Item Parameter Drift of the Self-Description Questionnaire I
Implications for Assessing Young Children’s Mathematics Self-Concept
Abstract
Abstract. The purpose of this study was to evaluate the psychometric properties of the Self-Description Questionnaire I (SDQI) with regard to children’s mathematics self-concept based on the item response theory (IRT) framework. The researchers adopted the IRT-based techniques for uncovering item-level characteristics including discrimination (a parameter) and thresholds (b parameters). The data for this study were drawn from the Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K) with a sample of 14,631 children. The IRT-based evaluations suggest that the SDQI items have sufficient discrimination, but insufficient item thresholds in assessing third- and fifth-graders’ mathematics self-concept. Thus, it is recommended that the current SDQI be reconsidered for its age appropriateness. In addition, affective items, which involve enjoyment, liking, and positive emotions, are found to be better indicators of self-concept than cognitive items, which involve perceived competence, ability beliefs, and self-efficacy. Results also indicate the presence of item parameter drift (IPD) in all SDQI items, suggesting beta change of the scale over time – children become more sensitive to the response anchors as they grow older. Findings of this study provide insights into the conceptualization and operationalization of academic self-concept, as well as practical implications for future use and development of self-concept measures.
References
(2000). The use of the Tobit model for analyzing measures of health status. Quality of Life Research, 9, 901–910.
(1985). The basics of item response theory. Portsmouth, NH: Heinemann.
(2001). The basics of item response theory (2nd ed.). College Park, MD: ERIC Clearinghouse on Assessment and Evaluation, University of Maryland.
(1999). Comparison between self-concept and self-efficacy in academic motivation research. Educational Psychologist, 34, 139–154.
(2003). Academic self-concept and self-efficacy: How different are they really. Educational Psychology Review, 15, 1–40. https://doi.org/I040-7Z6X/03/0300-0001/0
(2006). Confirmatory factor analysis for applied research. New York, NY: Guilford Press.
(2012). flexMIRT: Flexible multilevel item factor analysis and test scoring [Computer software]. Seattle, WA: Vector Psychometric Group, LLC.
(2011). IRTPRO: Flexible, multidimensional, multiple categorical IRT modeling [Computer software]. Lincolnwood, IL: Scientific Software International
(1998). An intrusive impact of anchors in children’s faces pain scales. Pain, 78, 27–37.
(2002). Developmental differences in children’s use of rating scales. Journal of Pediatric Psychology, 27, 27–36. https://doi.org/10.1093/jpepsy/27.1.27
(2013). The relationship between academic self-concept and achievement: A multicohort-multioccasion study. Learning and Individual Differences, 23, 172–178. https://doi.org/10.1016/j.lindif.2012.07.021
(2011). Math-gender stereotypes in elementary school children. Child Development, 82, 766–779. https://doi.org/10.1111/j.1467-8624.2010.01529.x
(2017). A continuous improvement approach to social and emotional competency measurement. Journal of Applied Developmental Psychology. Advance online publication. https://doi.org/10.1016/j.appdev.2017.03.002
(1983).
Achievement motivation . In P. H. MussenE. M. HeatheringtonEds., Handbook of child psychology: Vol 4. Socialization, personality, and social development (pp. 643–691). New York, NY: Wiley.(1983).
Expectations, values, and academic behaviors . In J. T. SpenceEd., Achievement and achievement motivation (pp. 75–146). San Francisco, CA: W. H. Freeman.(1995). In the mind of the actor: The structure of adolescents’ achievement task values and expectancy-related beliefs. Personality and Social Psychology Bulletin, 21, 215–225. https://doi.org/10.1177/0146167295213003
(1989). Self-concepts, domain values, and self-esteem: Relations and changes at early adolescence. Journal of Personality, 57, 283–310. https://doi.org/10.1111/j.1467-6494.1989.tb00484.x
(1998).
Motivation to succeed . In N. EisenbergEd., Social, emotional, and personality development in handbook of child psychology (Vol. III, pp. 1017–1096). New York, NY: Wiley.(2013). Discovering statistics using IBM SPSS (4th ed.). Thousand Oaks, CA: Sage.
(1995). An item response theory analysis of the general and academic scales of the Self-Description Questionnaire II. Journal of Research in Personality, 29, 168–188. https://doi.org/10.1006/jrpe.1995.1010
(2004). An examination of the psychometric properties of the physical self-description questionnaire using a polytomous. Psychology of Sport and Exercise, 5, 423–446. https://doi.org/10.1016/S1469-0292(03)00036-0
(2013). Using rating scales for the assessment of physical self-concept: Why the number of response categories matters. Measurement in Physical Education and Exercise Science, 17, 240–263. https://doi.org/10.1080/1091367X.2013.807265
(2010). Academic self-concept and emotion relations: Domain specificity and age effects. Contemporary Educational Psychology, 35, 44–58. https://doi.org/10.1016/j.cedpsych.2009.10.001
(1983). Measuring changes in educational attainment over time: Problems and possibilities. Journal of Educational Measurement, 33, 315–332. https://doi.org/10.1111/j.1745-3984.1983.tb00214.x
(1976). Measuring change and persistence in human affairs: Types of change generated by OD designs. Journal of Applied Behavioral Science, 12, 133–157. https://doi.org/10.1177/002188637601200201
(1997). An investigation of the placebo effect and age-related factors in the report of needle pain from venipuncture in children. Pain, 72, 383–391.
(2006). Understanding correlation: Factors that affect the size of r. The Journal of Experimental Education, 74, 251–266.
(1997). Exploratory factor analysis: Its role in item analysis. Journal of Personality Assessment, 68, 532–560. https://doi.org/10.1207/s15327752jpa6803_5
(1992).
Situational interest and its impact on reading and expository writing . In K. A. RenningerS. HidiA. KrappEds., The role of interest in learning and development (pp. 215–238). Hillsdale, NJ: Erlbaum.(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. https://doi.org/10.1080/10705519909540118
(1967). The behavior of maximum likelihood estimates under nonstandard conditions. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, 1, 221–233.
(2002). Changes in children’s self-competence and values: Gender and domain differences across grades one through twelve. Child Development, 73, 509–527. https://doi.org/10.1111/1467-8624.00421
(2013). Effect of multiple testing adjustment in differential item functioning detection. Educational and Psychological Measurement, 73, 458–470. https://doi.org/10.1177/0013164412467033
(1992).
Interest, learning, and development . In K. A. RenningerS. HidiA. KrappEds., The role of interest in learning and development (pp. 3–25). Hillsdale, NJ: Erlbaum.(2012). Measuring science interest: Rasch validation of the science interest survey. International Journal of Science and Mathematics Education, 10, 643–668. https://doi.org/10.1007/s10763-011-9314-z
(2008). A reexamination of Lord’s Wald test for differential item functioning using item response theory and modern error estimation (Doctoral dissertation). Retrieved from https://cdr.lib.unc.edu/
(2016). Measurement invariance of the Self-Description Questionnaire II in a Chinese sample. European Journal of Psychological Assessment, 32, 128–139. https://doi.org/10.1027/1015-5759/a000242
(2013). Gender differences in children’s math self-concept in the first years. Journal of Education and Learning, 2, 1–8. https://doi.org/10.5539/jel.v2n3p1
(2009).
Generalized estimating equations for longitudinal data analysis . In G. FitzmauriceM. DavidianG. VerbekeG. MolenberghsEds., Longitudinal data analysis (pp. 43–78). Boca Raton, FL: Chapman and Hall/CRC.(1992a). Self Description Questionnaire (SDQ) I: A theoretical and empirical basis for the measurement of multiple dimensions of preadolescent self-concept. An interim test manual and research monograph. Macarthur, New South Wales, Australia: University of Western Sydney, Faculty of Education.
(1992b). Self Description Questionnaire (SDQ) II: A theoretical and empirical basis for the measurement of multiple dimensions of adolescent self-concept. A test manual and research monograph. Macarthur, New South Wales, Australia: University of Western Sydney, Faculty of Education.
(1992c). Self Description Questionnaire (SDQ) III: A theoretical and empirical basis for the measurement of multiple dimensions of late adolescent self-concept. An interim test manual and research monograph. Macarthur, New South Wales, Australia: University of Western Sydney, Faculty of Education.
(1993). The multidimensional structure of academic self-concept: Invariance over gender and age. American Educational Research Journal, 30, 841–860.
(2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11, 320–341. https://doi.org/10.1207/s15328007sem1103_2
(1983). Self-concept: The construct validity of interpretations based upon the SDQ. Journal of Personality and Social Psychology, 45, 173–187. https://doi.org/10.1037/0022-3514.45.1.173
(1983). Multitrait-multimethod analysis of interpretations based upon the SDQ: Student-teacher agreement on multidimensional ratings of student self-concept. American Educational Research Journal, 20, 333–357. https://doi.org/10.3102/00028312020003333
(2010). Modeling outcomes with floor or ceiling effects: An introduction to the Tobit model. Gifted Child Quarterly, 54, 314–320. https://doi.org/10.1177/0016986210379095
(2005). Establishing measurement equivalence and invariance in longitudinal data with item response theory. International Journal of Testing, 5, 279–300. https://doi.org/10.1207/s15327574ijt0503_6
(2016). Peer reputations and gender differences in academic self-concept. Psychological Studies, 61, 21–31. https://doi.org/10.1007/s12646-015-0345-3
(2013). Predicting achievement: Confidence vs self-efficacy, anxiety, and self-concept in Confucian and European countries. International Journal of Educational Research, 58, 79–96. https://doi.org/10.1016/j.ijer.2012.11.002
(2010). Mplus user’s guide (6th ed.). Los Angeles, CA: Muthén & Muthén.
. (2011). Successful K-12 STEM education: Identifying effective approaches in science, technology, engineering, and mathematics. Washington, DC: The National Academies Press.
(2011). Who wants to have a career in science or math? Exploring adolescents’ future aspirations by gender and race/ethnicity. Science Education, 95, 458–476. https://doi.org/10.1002/sce.20431
(2008). Gender differences in gifted adolescents’ math/verbal self concepts and math/verbal achievement: Implications for the STEM fields. Journal for the Education of the Gifted, 32, 34–53. https://doi.org/10.4219/jeg-2008-818
(2010). Academic self-perceptions of ability and course planning among academically advanced students. Journal of Advanced Academics, 21, 300–329. https://doi.org/10.1177/1932202X1002100206
(1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, 19, 86–100. https://doi.org/10.1002/j.2333-8504.1968.tb00153.x
(1974). Normal ogive model on the continuous response level in the multidimensional latent space. Psychometrika, 39, 111–121. https://doi.org/10.1007/BF02291580
(1979). Increasing educational achievement via self concept change. Review of Educational Research, 49, 131–150. https://doi.org/10.2307/1169929
(1981). Self-concept: The interplay of theory and methods. Journal of Educational Psychology, 74, 3–17. https://doi.org/10.1037/0022-0663.74.1.3
(1976). Self-concept: Validation and construct interpretations. Review of Educational Research, 46, 407–441. https://doi.org/10.3102/00346543046003407
(1997).
Issues in research on self-concept . In M. MaehrP. R. PintrichEds., Advances in motivation and achievement (Vol. 10, pp. 51–97). New York, NY: JAI Press.(1996). Studies of academic self-concept using a Norwegian modification of the SDQ. Paper Presented at the XXVI International Congress of Psychology, Montreal, Canada.
(1989). Model modification. Psychomertika, 54, 371–384. https://doi.org/10.1007/BF02294623
(1996). Interest and competence as components of academic self-concepts. Paper Presented at the XXVI International Congress of Psychology, Montreal, Canada.
(2014). Testing measurement invariance across Spanish and English versions of the physical Self-Description Questionnaire: An application of exploratory structural equation modeling. Journal of Sport & Exercise Psychology, 36, 179–188. https://doi.org/10.1123/jsep.2013-0070
(2011). Self-concepts in Japanese and American elementary school children: Psychometric analysis of the Self-Description Questionnaire I (SDQ-I). Japanese Journal of Personality, 19, 191–205. https://doi.org/10.2132/personality.19.191
(2002). Development of interests and competency beliefs: A 1-year longitudinal study of fifth- to eighth-grade students using the ICA-R and structural equation modeling. Journal of Counseling Psychology, 49, 148–163. https://doi.org/10.1037/0022-0167.49.2.148
(1998). The structure of children’s interests and competence perceptions. Journal of Counseling Psychology, 45, 290–303. https://doi.org/10.1037/0022-0167.45.3.290
. (2010). Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K) kindergarten through fifth grade Approaches to Learning and Self-Description Questionnaire (SDQ) items and public-use data files (NCES 2010–070). Washington, DC: Author.
(1997). Underprediction of pain in children undergoing ear piercing. Behavioural Research and Therapy, 35, 399–404. https://doi.org/10.1016/S0005-7967(96)00127-1
(1995). The Self-Description Questionnaire-1: A Lithuanian application. European Journal of Psychological Assessment, 11, 41–51. https://doi.org/10.1027/1015-5759.11.1.41
(2004). Development of adolescents’ self-perceptions, values, and task perceptions according to gender and domain in 7th- through 11th-grade Australian students. Child Development, 75, 1556–1574. https://doi.org/10.1111/j.1467-8624.2004.00757.x
(1994). Expectancy-value theory of achievement motivation: A developmental perspective. Educational Psychology Review, 6, 49–78. https://doi.org/10.1007/BF02209024
(1992). The development of achievement task values: A theoretical analysis. Developmental Review, 12, 265–310. https://doi.org/10.1016/0273-2297(92)90011-P
(2009). Empirical selection of anchors for tests of differential item functioning. Applied Psychological Measurement, 33, 42–57. https://doi.org/10.1177/0146621607314044
