Assessing Children in Developmental Research

Differences in Assessment Methods

The first challenge we identify is differences in assessment methods across age groups, since using the same assessment tool or even method is often impossible in developmental testing. This poses a problem especially in longitudinal studies that span multiple years – even though we note that those studies are the gold standard for understanding developmental trajectories. One example of how different assessment methods can lead to counter-intuitive results is studies on infants and young children suggesting that infants understand principles that preschool children apparently do not. This is the case, for instance, in the field of physical reasoning. Research on infants suggests that they reason intuitively about balance and support (Baillargeon, 1995), as measured by the violation of expectation paradigm. Since infants cannot speak about their reasoning on physical principles, the violation of expectation paradigm uses the method of gaze duration. The underlying assumption is that infants will gaze longer at a phenomenon that surprises them than at a phenomenon that they expect (Baillargeon, 1995). At the same time, research on somewhat older children, for instance, preschool children, usually applies a different set of methods, such as explicitly asking children to explain their reasoning (Bonawitz et al., 2012; Pine et al., 2004). Studies found that children struggle with physical reasoning during the preschool years, a task which infants seem to perform better at, which is a rather surprising finding. However, Krist and colleagues (2018) make the case that the methods applied in preschool are much more cognitively challenging than the violation of expectation paradigm and differences between infants and preschoolers might stem from this and not from actual differences in physical reasoning. In their study, Krist and his colleagues applied an eye-tracking paradigm with 2- to 6-year-old children and found that children became increasingly sensitive to physical principles such as the amount of support needed for objects to remain stable on a supporting surface. Their findings imply, contrary to the findings mentioned above that might have been confounded by different ways of measuring, development in physical reasoning.

This example shows that different ways of measuring the same construct can lead to different results. Therefore, the need to have comparable ways of measuring a construct, for example, with similar levels of cognitive demands, is one of the core challenges in establishing comparability through equivalence across age groups in developmental psychology (see Greiff & Iliescu, 2017).

Narrow Domains

The second challenge concerns very concrete and highly specific applications in real-world contexts and the measurement of constructs in a relatively narrow domain that assessments with young children often require. If broader domains are measured and the measure becomes too abstract, children might be unable to relate to the task, that is, to understand the questions or test stimuli (Arens et al., 2016), which leads to unreliable test results. For example, when measuring constructs related to the self, such as self-concept or self-efficacy, it is necessary to ask specific questions. In consequence, Arens and colleagues (2016) used highly specific questions to measure preschoolers’ mathematics self-concept, for example, “Do you know lots of numbers?”. Similarly, Oppermann and colleagues (2017) measured children’s life science self-concept in a highly specific way as well, “Can you tell me how much you already know about plants?”. The question arises whether results from these studies can be compared to broader and more abstract tests and questionnaires commonly used for adolescents or adults (Marsh et al., 2019), for example, “I have always believed that mathematics is one of my best subjects” for adolescents 15 years and older. It is likely that not only the structure but also the nature of the concepts changes as children and adolescents develop, which can also provide interesting insights into developmental trajectories. Therefore, in many cases, it remains unclear whether the reason for differences in mean, variance, or any other indicator of interest between age groups is an actual change in the underlying construct, or a change in the way the construct is measured, for example, from narrow and specific to broad and general. Differentiating between changes in the underlying construct and changes in measurement remains a challenge in developmental research.

Sample Sizes

The third challenge regards the relatively small sample sizes developmental researchers often work with, due to the nature of their experiments. Therefore, sample sizes larger than N = 100 children are scarce, limiting the possibilities to statistically check for equivalence across age groups, even if the assessment method is the same in these groups. For measures that can be implemented into larger surveys, for example, questionnaires or certain test instruments, (inter-)national surveys can enable researchers to investigate measurement invariance across age groups (Jones et al., 2016). Moreover, with samples of N = 150, non-complex CFAs can be conducted to some extent and the CFI can be used as an index for goodness of fit (Chen, 2007).

In any case, it is clear that small sample sizes hamper statistical approaches to check for comparability between age groups and equivalence of measures in many studies; therefore, different approaches to establishing comparability need to be applied for developmental research.

Lack of Standardized Test Instruments

The fourth challenge we identify is a lack of standardized test instruments. The number of standardized test instruments for children of different ages is growing and many established batteries are now available to researchers (Dunn et al., 2015; Wechsler, 2012). However, often, and particularly for young children, standardized test instruments are still missing. Instead, researchers use measurement paradigms in which they develop their own assessment tasks, for example, violation of expectation, or the blicket detector (Baillargeon, 1995; Gopnik & Sobel, 2000; Sobel & Kirkham, 2006). Investigating these paradigms for comparability across age groups, for example with standardized measures that exist for older children and adults, would allow more detailed insights into construct structure and stability. For this, assessment methods for younger children could be adapted to older children, for example by using eye-tracking procedures (Krist et al., 2018), and could be compared to the results obtained in these older children with standardized tests, employing multi-modal testing. This approach could test whether the different procedures measure the same construct.

The challenge for developmental research coming along with this limitation is twofold: (1) standardized test instruments need to be developed with a focus on younger children, ensuring direct comparability with older age groups, and (2) research procedures for younger children and infants, such as the violation of expectation paradigm, need to be adapted for older children. This way, cognitive demands across age groups would be similar, and differences between age groups would be less likely to be caused by differences in cognitive demand.

Differences in Knowledge and Differences in Construct Structure

The fifth challenge concerns the need to distinguish between differences between age groups that stem from construct structure, prior knowledge, or the way the construct is measured in different age groups. Research suggests that the mechanisms underlying constructs, for example, reasoning, maybe the same for children and adolescents/adults and that the differences we find between age groups stem at least partly from higher knowledge that is present in adolescents and adults than in smaller children (Goswami, 2014). For example, a number of studies show that children tend to rely on perceptual similarities for reasoning, whereas adults rely on causalities. However, if children have knowledge about underlying causal relations or can infer them, they prefer causal reasoning over perceptual similarities as well, just as adolescents and adults do (Goddu et al., 2020; Griffiths et al., 2011). These differences in knowledge can potentially affect comparability between age groups and it is desirable to have similar prior knowledge conditions under which children and adolescents/adults are tested to ensure that differences in cognition are not merely differences in knowledge (that might then be mistaken for differences in the underlying processes).

Cognition and behavior can also serve different purposes and might have different meanings in different age groups (Putnick & Bornstein, 2016). For example, spatial skills are known to develop early in life and are even discussed to be innate by some researchers (Newcombe et al., 2013). However, certain subcomponents of spatial skills might present differently in different age groups and may undergo considerable development. Thus, the construct structure might differ and even change over relatively short time intervals. For example, navigation is one component of spatial skills that is not well developed in very young children (Immel et al., 2022; Newcombe et al., 2013). Therefore, if spatial skills are measured with a high reliance on navigation skills, younger children will inevitably do worse than older children, because navigation skills have not yet developed and therefore cannot represent spatial skills in young children. As a viable alternative, spatial skills could be measured with object representation, a core principle of spatial development (Baillargeon et al., 1985; Newcombe et al., 2013). But assuming the same underlying construct structure for spatial skills even while measuring them by relying on navigation will inevitably lead to mean differences because spatial skills present differently in younger children than in adolescents or adults. Consequently, this complicates the interpretation of differences between age groups. Mean differences could stem from either differences in construct structure from the way the construct is measured or from both. It is, with standard designs, almost impossible to disentangle the two. This shows that construct structure can play a role in the results obtained and has the potential to skew their interpretation. Therefore, the investigation of construct structure across age groups is of fundamental importance.

Finding (statistical) approaches to disentangle differences stemming from knowledge instead of change in the construct as well as measuring constructs in a way that the underlying structure is the same across age groups remains another challenge for developmental research.

Conclusions

We encourage developmental researchers to consider measurement aspects related to the challenges outlined in this editorial, for instance when comparing age groups in longitudinal as well as cross-sectional developmental studies. Missing the subtle nuances of comparability between measures can jeopardize the inferences drawn from studies and in the worst case have negative implications for children’s lives. Challenges to comparability between age groups in developmental research can take the form of differences in assessment methods, specific issues in research with children, such as measuring constructs narrowly, or differences in knowledge between children and adults, as well as general threats to power, such as small sample sizes. We urge researchers to consider these challenges in their own work, particularly when submitting their work to EJPA.

In fact, we do notice that few if any of these challenges have been so far approached in papers published in EJPA and the assessment literature in general. Yet, they are fundamental assessment issues that provide researchers with important opportunities for creativity: not only these are problems that need to be solved, and any solution will have a rapid and noticeable impact, but solutions to these problems require creative approaches, and likely new methodological and statistical innovations. This is a seminal territory into which to delve to advance the field of psychological assessment. We encourage authors to submit papers dealing with comparability across age groups in general, and more specifically with the more unusual instances of (non-)comparability discussed in this Editorial.