Construct Your Own Response
The Cube Construction Task as a Novel Format for the Assessment of Spatial Ability
Abstract
Abstract. The cube construction task represents a novel format in the assessment of spatial ability through mental cube rotation tasks. Instead of selecting the correct answer from several response options, testees construct their own response in a computerized test environment. The format has several advantages: It is no longer possible to guess the correct response or to compare the reference cube to the response options, resulting in a higher demand for spatial ability. Moreover, it is possible to create items with a particularly high difficulty which are needed for the assessment of intellectual giftedness. In the present study, we developed 28 items and presented them to a sample of 130 university students. Test results showed that the items possess a very high statistical difficulty. Furthermore, the item set yielded a very high internal consistency. The results of an exploratory factor analysis as well as of a multidimensional IRT analysis indicated that a two-factor solution (“spatial relations” vs. “spatial visualization”) is plausible. Response time had a negligible influence on accuracy. Perspectives on further research concerning the cube construction task and possibilities for practical applications are being discussed.
References
2001). Intelligenz-Struktur-Test 2000 R (I-S-T 2000 R)
([Intelligence-Structure-Test 2000 R] . Göttingen, Germany: Hogrefe.2013). Reducing response elimination strategies enhances the construct validity of figural matrices. Intelligence, 41, 234–243.
(2015). lme4: Linear mixed-effects models using Eigen and S4. R package version 1.1-8. Retrieved from http://CRAN.R-project.org/package=lme4
(2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in confirmatory factor analysis. Structural Equation Modeling, 13, 186–203.
(2015). Die Matrizenkonstruktionsaufgabe: Validierung eines distraktorfreien Aufgabenformats zur Vorgabe figuraler Matrizenaufgaben
([The Construction Task: Validation of a distractor-free item format for the presentation of figural matrices] . Diagnostica, 61, 22–33.1984). Adaptive reasoning: Componential and eye movement analysis of geometric analogy performance. Intelligence, 8, 205–238.
(1990). What one intelligence test measures: A theoretical account of the processing in the Raven Progressive Matrices Test. Psychological Review, 97, 404–431.
(1993). Human cognitive abilities: A survey of factor-analytic studies. New York, NY: Cambridge University Press.
(1996).
(A three-stratum theory of intelligence: Spearman’s contribution . In I. DennisP. TapsfieldEds., Human abilities: Their nature and measurement (pp. 1–17). Mahwah, NJ: Erlbaum.2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48, 1–29.
(2003). The assessment of spatial ability with a single computerized test. European Journal of Psychological Assessment, 19, 92–100.
(1983). An international directory of spatial tests. Windsor, Berkshire: NFER-Nelson.
(1990). Dreidimensionaler Würfeltest (3DW)
([Three dimensional-Cube-Test] . Weinheim, Germany: Beltz.2003). Spatial strategy selection: Interesting incremental information. International Journal of Testing, 3, 293–308.
(2015). More is not always better: The relation between item response and item response time in Raven’s Matrices. Journal of Intelligence, 3, 21–40.
(2014). The time on task effect in reading and problem solving is moderated by task difficulty and skill: Insights from a computer-based large-scale assessment. Journal of Educational Psychology, 106, 608–626.
(2014). A Kenward-Roger approximation and parametric bootstrap methods for tests in linear mixed models – the R package pbkrtest. Journal of Statistical Software, 59, 1–30.
(2005).
(Sex differences in visuospatial abilities: More than meets the eye . In P. ShahA. MiakeEds., The Cambridge Handbook of Visuospatial Thinking (pp. 170–212). New York, NY: Cambridge University Press.2011). A review of spatial ability literature, its connection to chemistry, and implications for instruction. Journal of Chemical Education, 88, 351–360.
(2008). Sex differences in mental rotation with polygons of different complexity: Do men utilize holistic processes whereas women prefer piecemeal ones? The Quarterly Journal of Experimental Psychology, 61, 683–689.
(2005). Sex differences in means and variability on the Progressive Matrices in university students: A meta-analysis. British Journal of Psychology, 96, 505–524.
(2012). Why does working memory capacity predict RAPM performance? A possible role of distraction. Intelligence, 40, 427–438.
(1997). Berliner Intelligenzstruktur-Test (BIS)
([Berlin-Intelligence-Structure-Test] . Göttingen, Germany: Hogrefe.1998). The g factor: The science of mental ability. Westport, CT: Praeger.
(1985). Emergence and characterization of sex differences in spatial ability: A meta-analysis. Child Development, 56, 1479–1498.
(1979). Spatial ability: A review and re-analysis of the correlational literature (Technical Report No. 8). Stanford, CA: Aptitudes Research Project, School of Education, Stanford University.
(1988).
(Spatial abilities as traits, processes, and knowledge . In R. J. SternbergEd., Advances in the psychology of human intelligence (Vol. 4, pp. 181–248). Hillsdale, NJ: Erlbaum.1996).
(Spatial ability and g . In I. DennisP. TapsfieldEds., Human abilities: Their nature and measurement (pp. 97–116). Mahwah, NJ: Erlbaum.1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
(2012). Modeling multiple strategies for solving geometric analogy problems. Proceedings of the 34th Annual Conference of the Cognitive Science Society. Sapporo, Japan.
(2009). CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research. Intelligence, 37, 1–10.
(1990). Modeling item responses when different subjects employ different solution strategies. Psychometrika, 55, 195–215.
(2008). Die verflixten Distraktoren: Über den Nutzen einer theoretischen Distraktorenanalyse bei Matrizentests (für besser Begabte und Hochbegabte)
([The nasty distracters. On the utility of a notional distracter analysis of items of matrices-test for the highly gifted] . Diagnostica, 54, 193–201.2007). MPlus Version 6. Los Angeles, CA: Muthén & Muthén.
(2000). Likelihood-based item fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24, 50–64.
(1981). Problemlöseprozesse und Intelligenzleistung
([Problem-solving processes and intelligence performance] . Bern, Switzerland: Huber.2015). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.
. (1987).
(The contribution of intelligence tests to the understanding of special children . In J. D. DayJ. B. BorkowskiEds., Intelligence and exceptionality: New directions for theory, assessment, and instructional practices (pp. 21–56). Norwood, NJ: Ablex.2007). Leseverständnis ohne Lesen? Zur Konstruktvalidität von multiple-choice-Leseverständnistestaufgaben
([Reading comprehension without reading? On the construct validity of multiple-choice Reading Comprehension Test items] . Zeitschrift für Pädagogische Psychologie, 21, 305–314.1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274.
(1980).
(Aptitude processes . In R. E. SnowP. A. FedericoW. E. MontagueEds., Aptitude, learning and instruction (Vol. 1, pp. 27–63). Hillsdale, NJ: Erlbaum.2006). Eye-movement analysis demonstrates strategic influences on intelligence. Intelligence, 34, 261–272.
(1995). Magnitude of sex differences in spatial abilities: A meta-analysis and consideration of critical variables. Psychological Bulletin, 117, 250–270.
(2009). Spatial ability for STEM domains: Aligning over 50 years of cumulative psychological knowledge solidifies its importance. Journal of Educational Psychology, 101, 817–835.
(2007). Spatial ability: A neglected dimension in talent searches for intellectually precocious youth. Journal of Educational Psychology, 99, 397–420.
(