The Role of IQ and Social Skills in Coping With Uncertainty in 7- to 11-Year-Old Children
Abstract
Abstract. Most feedback we receive or give is correct (deterministic feedback), though a small fraction can be wrong for various reasons. Children need to cope with receiving some portion of wrong feedback (stochastic feedback). It is still unknown if better social functioning and communication skills or outstanding intelligence (IQ) or chronological age support children in the coping process. We tested a sample of 7-, 9-, and 11-year-old children (N = 60) who deduced a sequence of four left and right button presses from a red and green stochastic feedback signal that was wrong in 15 % of the trials. Children performed worse with stochastic than with deterministic feedback but improved in the repeated trials, especially after receiving positive feedback about whether true or false. Controlling for IQ improved and confirmed these effects, while social and communicative competence explained little or no variance.
Zusammenfassung. Ein Großteil der Rückmeldungen, die wir bekommen oder geben ist richtig (deterministisch), aber ein kleiner Anteil der Rückmeldungen kann aus unterschiedlichen Gründen falsch sein. Kinder müssen lernen, mit manchmal falschen Rückmeldungen umzugehen (stochastisch). Bisher ist unklar, ob ihr Alter, gute soziale Fähigkeiten oder eine hohe Intelligenz (IQ) Kinder bei der Verarbeitung von falscher Rückmeldung unterstützen. Wir haben 7-, 9- und 11-jährige Kinder (N = 60) mit einer Sequenzlernaufgabe getestet, in der sie die richtige Sequenz von vier linken und rechten Knopfdrücken mit Hilfe eines roten und grünen Feedbacksignals herausbekommen sollten, das zu 15 % falsch war. Bei stochastischer Rückmeldung zeigten die Kinder einen stärkeren Leistungseinbruch als mit einem deterministischen Feedbacksignal, aber sie verbesserten sich durch Übung und nach positivem Feedback unabhängig von dessen Wahrheitsgehalt. Die Kontrolle von Intelligenzunterschieden verbesserte und bestätigte diese Effekte, während soziale und kommunikative Kompetenz wenig oder gar keine Varianz erklärte.
Most feedback we receive and hand out is correct though a small fraction thereof can be false – because the person is distracted, careless, or uncertain about the answer. Children can receive false feedback in daily situations and need to learn to cope without getting upset, disappointed, or frustrated. For example, when a child has written a text in the past and the teacher marked it as correct in the past, whereas on another occasion, the teacher marked a similar passage as wrong. The child knows that teachers too can make mistakes, but the child still tries to make the situation “consonant” by either changing the wording of the description or questioning the teacher’s marking. Thus, the feedback shows a discrepancy. This process of solving such contradictory feedback can be conceptualized by the theory of cognitive dissonance (Aronson, 1969; Festinger, 1957). Such dissonance may feel uncomfortable to children, and they would try to mediate the two cognitions. For this, intelligence quotient (IQ) and social skills may be involved in the process. An integrative framework, the sociocognitive integration of abilities model (SOCIAL), defines social skills within a developmental framework (Beauchamp & Anderson, 2010). One part of social skills is social interaction, which is influenced by factors within the individual, namely, communications skills, temperament, and personality. For instance, doubting oneself or the other person may result in uncertainty. Thus, internal factors are presumed to decisively influence the process of how a person mediates cognitive dissonance. To the best of our knowledge, no study has investigated the underlying social factors in children that mediate stochastic feedback uptake and induce uncertainty by providing a small proportion of partly wrong feedback.
The current study investigates how children learn a correct sequence of four button presses based on stochastic, that is sometimes randomly false feedback in repeated trials. We control not only for social skills, but also for the influence of IQ because previous research showed that nonverbal intelligence as per the Raven Progressive Matrices Test (Raven, Raven, & Court, 1998) was of major importance for this task in adolescence (Lange-Küttner et al., 2021). However, for this German sample, we used a German culture-fair IQ test. Because the task of this study is completely visual and requires some logical thinking, we did not control for general or verbal IQ. It should be noted that IQ can also refer to abilities in the visual arts and music (Gardner, 1999).
Previous research compared learning from randomly false, stochastic feedback with always correct deterministic feedback (Lange-Küttner et al., 2012). In children aged 8 to 11 years, an instant and pronounced performance decrease of more than 40 % occurred. Interestingly, after this meltdown, children improved their performance during repeated trials. Recovery occurred when the feedback signaled that they were right even though they were wrong (self-reflection), but they did not improve when the feedback falsely indicated that they were wrong when in fact they had been correct (self-assertion) (Lange-Küttner et al., 2012). Thus, it could be assumed that social skills such as communication in social interactions and an absence of restricted, repetitive patterns of behavior, interests, and activities might help to cope especially with false negative feedback. Negative feedback processing is particularly difficult for children at this age (Crone et al., 2006; Eppinger et al., 2009). However, in this previous research, social skills were not tested. Boys in the autistic spectrum who have difficulties with social skills performed on this task with the completely correct, deterministic feedback as if it was stochastic (Hentschel et al., 2016). On the other hand, they may have evaluated the deterministic feedback as if it were uncertain. It seemed as if they did not trust correct feedback. Thus, these results suggested that social skills may be involved in feedback processing. On the other hand, adolescents failed to show an effect of personality variables as working memory capacity explained decision-making and risk-taking in adolescence, but not self-control and impulsivity (Kray et al., 2021). Also, Lange-Küttner et al. (2021) found that nonverbal IQ was more relevant than social skills for reasoning under uncertain feedback. Social skills were only relevant for less than a handful of higher-than-average range intelligent boys who showed deficits in positive coping. Therefore, we have used this more controlled design with children from ages 7 to 11.
Sequence learning can be distinguished into learning the associations between subsequent stimuli and position-based sequence learning (Zhao et al., 2018, 2020). In the current study, we used the numbering of the positions in a sequence, although both types of sequence learning allow participants to develop knowledge of a sequence that can subsequently help them change from stimulus-based to memory-based responding (Gaschler et al., 2014).
Sequence learning can further be distinguished into implicit and explicit sequence learning. Explicit sequence learning implies that participants are explicitly told that the task is about detecting a sequence and is associated with IQ (Gebauer & Mackintosh, 2007; Reber et al., 1991) and age (Pufall & Furth, 1966; Thomas & Nelson, 2001). In contrast, implicit sequence learning implies learning a sequence in the absence of conscious awareness about the ongoing learning process itself or about the outcome of what was to be learned (Haider et al., 2014). Implicit sequence learning in children and adults showed no age differences and occurred independently of IQ (Gebauer & Mackintosh, 2007; Reber et al., 1991; Meulemans & Van der Linden, 1998; Weiermann & Meier, 2012). Nevertheless, even in an implicit sequence learning task, participants may develop explicit knowledge of the sequence.
Many studies investigating implicit sequence learning used serial reaction time (SRT) tasks. In such tasks, participants are asked to respond quickly and accurately to a black dot appearing in one of four locations on a screen by pressing four corresponding buttons. The locations of the dots are repeated in a particular sequence (Karatekin et al., 2007; Thomas & Nelson, 2001). Karatekin et al. showed that sequence learning in 12-year-old children is the same as in adults, which agrees with the results from other studies on conceptual development (Gzesh & Surber, 1985). Likewise, Janacsek et al. (2012) found in a large lifespan sample from 4 to 85 years that the largest amount of learning occurred in childhood between 4 and 12 years of age (when considering reaction time) (see also Thomas & Nelson).
Lange-Küttner et al. (2012) used the present four-button press sequence task with 8- to 11-year-old children who were explicitly told that they have a sequence to learn in a four-button-press task, and that sometimes the feedback would be wrong. Results showed a significant main effect of age: There was a significant increase in the accuracy of the deduction of the sequence between 8 and 10 years, but not thereafter. However, even though the level of success of the deduction of the sequence increased with age, recovery from stochastic feedback and improvement of learning was the same in all age groups.
Also, the type of feedback can influence the performance level (Crone et al., 2006; Van Duijvenvoorde et al., 2013). Feedback can be positive or negative. Several studies showed that positive feedback increases performance in children (Chiviacowsky et al., 2008; Puddefoot et al., 1997). However, van den Bos et al., (2009) suggested that children’s focus on positive feedback should shift toward negative feedback as they become more mature. Negative feedback is an important and useful source of error correction (see also Bolia et al., 1999; Eppinger et al., 2009; Lange-Küttner et al., 2012; Schmittmann et al., 2012; Van Duijvenvoorde et al., 2013; van den Bos et al., 2009). In the Lange-Küttner et al. (2012) sequence learning task with 100 % correct feedback, negative performance feedback informed learning only from age 9 years onward, but when feedback was uncertain, learning deteriorated after negative feedback in all age groups from 8 to 11 years. This negative influence of uncertain stochastic feedback only changed in adolescence from age 14 (Lange-Küttner et al., 2021). Crone et al. (2006) conducted a psychophysiological study on the processing of positive and negative feedback in 8- to 18-year-old children and adolescents using the Wisconsin Card Sorting Task. They could show that error monitoring increases with age: When young adults learned via feedback, they showed a stronger reaction after negative feedback than 8- to 9-year-old children and 11- to 13-year-old adolescents (Van Duijvenvoorde et al., 2013). This suggests that negative feedback processing and error monitoring develop later as well as for a much longer time – into adulthood – than, for instance, concept learning. This late onset of learning from negative feedback occurs despite a trend from external to internal monitoring of performance feedback (Groen et al., 2007; Chiviacowsky et al., 2008). Thus, we tested whether children’s social skills may play a role here, using the Social Communication Questionnaire (SCQ) of Rutter et al. (2001).
Below we focus on feedback processing and the relationship to IQ. Previous findings indicate an association between feedback processing and IQ (Brand et al., 2009; Luwel et al., 2011; van den Bos et al., 2012). Brand et al. (2009) showed that feedback processing was moderated by nonverbal IQ in a game of dice. Participants with high nonverbal intellectual abilities improved their performance with, but also without feedback, while participants with low nonverbal intellectual abilities relied more on feedback to improve their performance. Likewise, Luwel et al. (2011) showed that in a feedback intervention study, differences of strategic competence in low and high IQ participants disappeared because of the strong improvement of the participants with low IQ. Their study used a full-scale verbal and nonverbal IQ. Another study by van den Bos et al. (2012) with participants from 13 to 16 years of age using a probabilistic learning task with two stimulus pairs and a red/green feedback signal showed that IQ correlated positively with accuracy since, again, children with a higher IQ used positive feedback optimally to improve their performance. In their study, verbal and nonverbal IQ was averaged since there was a high correlation between them. These studies show that IQ positively influences feedback processing. In the current study, we investigate whether this also holds true for stochastic feedback.
To summarize, the current study aims to find out whether age, nonverbal IQ, or social skills is the most important factor explaining the development of coping with partially false, stochastic feedback in a sequence learning task. We tested 7-, 9-, and 11-year-old children with a stochastic feedback sequence learning task, and, in addition, we measured IQ with a standardized nonverbal IQ test and the social skills with a social communication questionnaire. Based on previous research, we hypothesized (1) that nonverbal IQ and age should be jointly predictive of children’s performance in the four-button-press task with stochastic feedback, while social skills would have a minor effect; (2) that children would learn more from positive than negative feedback. In short, the current study tests whether age differences, IQ, or social skills explain (a) the deterioration, (b) the recovery, and (c) responses to positive vs. negative feedback when learning to sequence four button presses with stochastic feedback.
Method
Participants
A posthoc power calculation using the program MorePower6.0.4 for an ANOVA with the two repeated factors (number of levels in brackets), repetition of response sets (6) and feedback type (2), and the between-subjects factor age groups (3), an alpha level of .05, an eta-squared effect size of .18, and a power level of .95 showed that a sample of N = 24 is required (Campbell & Thompson, 2012). For the calculation of the sample size, we used the reported effect size from the study of Lange-Küttner et al. (2012), which used the same task with similar age groups. The small sample suggestion can be explained by the extended within-subject design, which produces reliable data. In our sample size consideration, before collecting the data, we followed Bergmann et al. (2018), who suggest that at least 15 to 20 participants per age group produce reliable data.
The sample was tested in a small town in Northern Germany. We received parental consent for 68 children to participate in the study; 8 children were not tested because they did not meet the age requirements, resulting in a total of 60 participants. Table 1 shows the mean age and age range for the 7-year-olds, 9-year-olds, and 11-year-olds in two task order groups.
The total sample consisted of 29 boys and 31 girls. In each age group, there were 10 girls and 10 boys, except in the group of 11-year-olds, which consisted of 9 boys and 11 girls. All participants performed the sequence learning task twice, once with 85 % correct, stochastic feedback and once with 100 % correct, deterministic feedback, with the task sequence counterbalanced. Results of t-tests for independent samples (two-tailed) for the Social Communication Questionnaire (SCQ) (Rutter et al., 2001) and the IQ test, namely, the CFT-1 (Weiß & Osterland, 1997) and CFT 20-R (Weiß, 2006), showed no significant differences because of the counterbalanced test sequence samples in any of the tests, ps > .250. Thus, the samples were merged for further analyses. Also, the correlations between the SCQ and the IQ test in the three different age groups were not significant ps > .098.
Apparatus and Materials
Social Communication Questionnaire (SCQ)
Before testing the children with the sequence learning task, parents filled in the German version of the SCQ (Rutter et al., 2001), the Fragebogen zur sozialen Kommunikation (FSK) (Bölte & Poutska, 2006). The SCQ/FSK is a paper-form questionnaire consisting of 40 yes or no questions that evaluate communication skills, social interaction, and restricted, repetitive patterns of behavior, interests, and activities. The analyses are based on raw scores as no age norms are available. The internal consistency (Cronbach’s alpha) of the total scale is α = .83, the retest reliability of the questionnaire is rtt = .76 (Bölte & Poutska, 2006).
CFT 1 and CFT 20-R (Culture Fair Intelligence Test)
Before the sequence learning task, the 7- and 9-year-olds completed the CFT 1 (Weiß & Osterland, 1997) and the 11-year-olds the CFT 20-R (Weiß, 2006). Both tests are designed for German children and aim to test the general mental capacity of children while minimizing the influence of social background and learning experience, especially in the verbal area. The CFT 1 was designed for children aged 4 to 9, while the CFT 20-R tests the IQ of 9- to 19-year-old children and adolescents. The CFT 1 contains five subtests (Substitutions, Labyrinth, Classification, Similarities, and Matrices). The coefficient for the test repetition after 4 weeks (total: .95) as well as the reliability coefficient (total: .97) ensure reliable results for individual performance, but also for group comparisons (Weiß & Osterland, 1997). The CFT20-R has four subtests (Continue an Order, Classification, Matrices, and Topology). The coefficient for the test repetition after 3 months (.80 to .82) and the reliability coefficient (Cronbach’s alpha: .95) allow reliable results for individuals (Weiß, 2006).
The CFT1 and the CFT20-R both measure IQ but contain different items as they are used with different age groups, and they were normed about 10 years apart. Thus, cohort effects may play a role. Hence, for the statistical analyses, we z-transformed the scores to make the norms of the two psychometric IQ tests comparable.
Sequence Learning Task
The four-button sequence learning task is a computer task developed with Matlab (MATHWORKS). The child learns a sequence of four left (L) and right (R) button presses based on the visual red/green feedback signal on the screen (see Figure 1). Only the sequence of four button presses was required as response in the absence of any presented visual or auditory stimulus. The visual feedback appeared on the screen after each button press allowing deduction of the correct sequence of the four left and right button presses. The visual feedback after each button press was a circle in red (wrong) or green (right). Before each button press, a trial number of 1, 2, 3, or 4 inside a circle appeared which indicated the position of the subsequent button press in the four-button press sequence.
Deterministic Feedback Type
In this feedback type, children always received correct feedback (deterministic feedback, 100 % correct). If the child first pressed the left button and a green circle appeared, the child knew that the first button press in this sequence had to be the left one. However, if the child instead had pressed the right button and the red circle appeared, the child could deduce from the feedback that the first button press of the sequence was not the one on the right side and consequently would have to press the left button on the next four-button press set.
Stochastic Feedback Type
In this feedback type, children received only 85 % correct feedback, with 15 % randomly interspersed incorrect feedback. This means that even if the child pressed the correct button, the feedback could have indicated that this was the wrong choice. Likewise, if the child pressed the wrong button, the stochastic feedback could have indicated that the response was correct.
After children identified the correct sequence, they were asked to repeat this set of button presses six times before moving on to the next sequence, to show that the sequence had been properly learned, and that the first correct sequence was not a chance event. If the child failed to detect the correct sequence within 20 sets of button presses, the program automatically moved on to the next sequence. Before starting a new sequence, the instruction “You have a new sequence to learn” appeared on the screen to inform the child. There was no time constraint, the sequence learning task was self-paced, and only accuracy was measured.
The task consisted of the six sequences LLRR, RRLL, LRLR, RLRL, LRRL, and RLLR, that is, each sequence consisted of a response set of four button presses. Please note that no presentation was involved as with the Wisconsin Card Sorting Test (Crone et al., 2006). So, the children were required to find the correct sequence by feedback alone. The sequences were always tested in the same order in both feedback types. Each sequence was tested until it was correct, and the correct sequence was repeated six times or up to a maximum of 20 repetitions. Thus, there was not a fixed number of repetitions in this task. However, we only analyzed the first six sets of button presses per sequence, as, theoretically, children could complete one set within six repetitions. Hence, the raw data set consisted of 6 x 4 =24 button presses x 6 repetitions = 144 button press trials x 2 feedback type = 288 button press trials. The data were averaged across the six sequence sets to be used for statistical analyses.
Learning Parameter/Feedback Valence
In addition to accuracy, we measured whether learning progress depended on positive or negative feedback. Therefore, we used the formal learning model developed by Averbeck et al. (2011) with two learning parameters, a positive and a negative one. The positive learning parameter measures how likely the child was to press the correct button in the next response set after receiving positive feedback; the negative learning parameter measures whether negative feedback decreased the likelihood of learning to press the correct button in the next response set.
Procedure
Approval was obtained from the London Metropolitan University Ethical Board as well as from the Ministry of Education, Science, and Culture in Northern Germany, where the data were collected. After obtaining approval from the local educational officials overseeing the schools, we directly contacted the schools with an information letter. Once all people involved in the data collection (teachers, parents, and pupils) in three schools had agreed to the study, the information letter and consent form were sent to the parents. Once consent had been obtained, we sent the Social Communication Questionnaire (SCQ) home to the parents.
There were two sessions, each lasting about 30 – 45 minutes. For the first session, children were divided into small groups of 4 – 6 pupils to complete the psychometric assessment. The second session with the sequence learning task was carried out individually with each child. A child would either read the instructions from the screen, or the researcher would read them out aloud, depending on the reading ability and preference of the child (see Appendix A). After reading the initial instructions, the researcher showed the child the two buttons. The left index finger was put on the “z” button and the right index finger on the “-“ button on a UK/US keyboard layout (QUERTY). The keys were marked with stickers covering the letters. The child was asked to keep the fingers on the keyboard during the entire task.
Data Generation and Analyses
For the measurement of accuracy, data were averaged for each participant across the sequences LLRR, RRLL, LRLR, RLRL, LRRL, and RLLR for the first six repetitions of each of these sets of button presses. This resulted in six repeated dependent variables for the factor “repetitions,” per feedback type. We computed accuracy in % (score*100/6) for each of the six scores showing the total amount of correctly sequenced left and right button presses per set. Additionally, we computed a positive and a negative learning parameter using a formal learning model of Averbeck et al. (2011). The positive learning parameter measures how likely a child was to press the correct button in the next response set after receiving positive feedback; the negative learning parameter measures whether negative feedback decreased the likelihood of learning to press the correct button in the next response set.
The data were imported into IBM SPSS version 25 for statistical analyses. To make the IQ scores of the two different IQ tests equivalent, we z-transformed the scores of the CFT1 and the CFT20 using IBM SPSS version 25. IQ scores are therefore labeled IQz.
The MATLAB code for the sequence learning task itself – for averaging the first six repetitions, the code for the computation of the learning parameter, and the SPSS spreadsheet – are available via the link to the depository of the Open Science Foundation (OSF).
Results
We first analyzed sequence learning using the averaged response sets for six repetitions for both feedback types, and then compared the three different age groups in a MANOVA with repeated measures. This was followed up first with MANCOVAs with the IQz scores as a covariate, followed by adding the SCQ score as covariate thereafter, and vice versa. Thereafter, we report the analysis of the two learning parameters (positive or negative feedback valence) and both feedback types as a within-subjects factor and three age groups as a between-subjects factor in a MANOVA with repeated measures, followed by MANCOVAs in the same way. If the Mauchley’s test of sphericity was significant, we adjusted the degrees of freedom using Greenhouse-Geisser. Posthoc tests were deemed significant when the p-value was smaller than .05 and were Bonferroni adjusted for multiple testing.
Sequence Learning
Sex differences were not significant, ps > .174, and thus sex is not included as a between-subject factor. A 6 (Repetition) by 2 (Feedback type) by 3 (Age groups) analysis of variance was run with repeated measures on the first two factors and age groups as between-subject factors. Group means are listed in Table 2. We added both IQ and social skills as individual covariates in the two models. We report the results of the statistical analyses using the psychometric IQ values in Appendix B, and report here the results of the analyses using the z-transformed IQ values (IQz).
There was no significant difference between the three age groups F(2, 60) = 0.32, p = .726. As expected, a main effect of feedback type with a medium effect size was found, F(1, 60) = 71.91, p < .001, ŋp2 = .56, as children in the deterministic feedback type (M = 74.5 %) learned better than in the stochastic feedback type (M = 61.5 %). There was also the expected main effect of repetitions (rep) with a high effect size, F(3.91, 60) = 649.47, p < .001, ŋp2 = .92, (Rep 1: M = 0.5 %, Rep 2: M = 69.7 %, Rep 3: M = 79.3 %, Rep 4: M = 84.4 %, Rep 5: M = 85.4 % and Rep 6: M = 88.8 %). More importantly, the two-way interaction of feedback type and repetition was significant, F(1, 60) = 15.59, p < .001, ŋp2 = .22, showing performance improvement and recovery in the stochastic feedback type; see Figure 2. Posthoc pairwise t-tests (Bonferroni-corrected significance level p = .05/6 = .008) per feedback type showed that, after the first repetition until the fifth repetition, children performed better in the deterministic than in the stochastic feedback type, all ps < .001. However, the gap was nearly closed and no longer significant in the sixth repetition.
Contrasts (difference) showed that a significant learning gain occurred with each repetition, ps < .001, with large effect sizes of ŋp2s ˃ .87. These significant learning gains could be observed in the deterministic, ps < .001, ŋp2s ˃ .75, as well as in the stochastic feedback type, ps < .001, ŋp2s ˃ .72; see Figure 2.
After adding IQz as a covariate in the model (MANCOVA), all previous reported significant results remained significant; see Table 3. Additionally, a significant interaction of repetition and IQz appeared F(280, 60) = 1.59, p = .04, ŋp2 = .41, meaning children’s learning during the experiment was related to IQz in both feedback types. After we additionally added the overall score of the Social Communication Questionnaire (SCQ) as a second covariate to the model, the interaction of repetition and IQz was no longer significant F(275, 60) = 0.32, p = .163, showing that both social skills and IQz impacted on learning with repetition. However, all previously reported significant results remained significant.
In a second analysis, we added the SCQ first and the IQz second; see Table 4. While adding the overall SCQ score as a covariate in the model (MANCOVA) did not affect the significance of any of the effects, a new significant two-way interaction of repetition and SCQ, F(1, 60) = 2.43, p = .036, ŋp2 = .04, emerged.
This interaction was further explored by using the subscales instead of the total SCQ score, namely, communication skills, social interaction, and restricted, repetitive patterns of behavior, interests, and activities. Only restricted, repetitive patterns of behavior, interests, and activities showed a significant interaction with repetition, F(1, 60) = 3.30, p = .007, ŋp2 = .06. We then used the median of the variable restricted, repetitive patterns of behavior, interests, and activities to split the sample, with 56.7 % showing no restricted, repetitive patterns of behavior, interests, and activities (0), but 43.3 % showing restricted, repetitive patterns of behavior, interests, and activities on a scale of 1 – 7 (16.7 % scored 1, 13.3 % scored 2, and 13.3 % scored 3 and higher). Hence, a 6 (Repetition) by 2 (Feedback type) by 2 (Restricted, repetitive patterns) analysis of variance was run with repeated measures on the first two factors and the two stereotypy groups as between-subject factors. The only new effect was that restricted, repetitive patterns of behavior, interests, and activities was now a significant between-subject effect, F(1, 60) = 4.25, p = .044, ŋp2= .07, with children showing restricted and repetitive patterns performing less well (M = 66.0 %) than those without (M = 69.5 %). When adding the IQz score as the second covariate to the model, this interaction of repetition and SCQ score was no longer significant F(275, 60) = 0.32, p = .163, showing that IQz impacted on the effect of the SCQ. However, all other previously reported significant results remained significant. Thus, we find that an absence of perseverations and a higher IQ reveal shared variance when it comes to a general improvement via repetition.
To gain more information about learning during the repetitions, we calculated the correlations between the repetitions for each feedback type, using Pearson correlations (two-tailed) (see Table 5). A significant correlation implies that children were seeing the previous response set and the current repeated response set as related tasks, whereas no correlation implies that children treated each repetition as a new task. There were no significant correlations between the repetitions with deterministic feedback, ps > .148. However, with stochastic feedback, there was a significant correlation between repetitions 1 and 2 as well as repetitions 3 and 4 and 4 and 5. This shows that, with deterministic feedback, children did not relate one repetition with another, while with stochastic feedback, children made connections between repetitions.
Feedback Valence Processing
Here, we report the effect of positive vs. negative feedback on sequence learning accuracy. Note that this is not an analysis of performance level, but an analysis of the probability of a correct response following positive or negative feedback.
A 2 (Feedback valence) by 2 (Feedback type) by 3 (Age groups) analysis of variance was run with repeated measures on the first two factors and age groups as between-subject factors. There were no significant differences between age groups, F(2, 60) = 1.27, p = .289. We found a significant difference between the deterministic and stochastic feedback type, F(1,60)= 47.55, p < .001, ŋp2 = .46, showing that children learned more after deterministic (M = 8.0 %) than stochastic feedback (M = –2.1 %), with a difference of 10.1 %. A more than double as large effect size for feedback valence, F(1,60) = 676.29, p < .001, ŋp2 = .92, showed that feedback valence was more important than the veracity of the feedback: Children learned more from positive (M = 27.7 %) than from negative feedback (M = –21.7 %) which actually caused deterioration, with a difference of nearly 50 % in learning following positive vs. negative feedback. Moreover, the two factors feedback valence and feedback type interacted, F(1,60)= 19.11, p < .001, ŋp2 = .25; see Figure 3. Two posthoc pairwise t-tests (two-tailed) confirmed that positive feedback had more impact on learning when feedback was always correct (M = 36.2 %), compared to when it was only likely to be correct (M = 19.2 %), t(60)= 12.58, p < .001, while there was no significant difference for learning from negative feedback between the deterministic and stochastic feedback type, t(60)= 1.16, p = .252.
When the model was controlled for IQz scores as covariate (MANCOVA), the main effect of feedback type and feedback valence as well as the interaction of feedback valence and feedback type remained significant ps < .001; see Table 6. Additionally, the two-way interaction between IQz and feedback valence reached significance, F(56,60)= 4.57, p = .037, ŋp2 = .08.
The correlation between IQz and negative feedback processing, r = –.05, p = .674, was not significant. But the correlation between IQz and positive feedback processing did reach significance, r = .28, p = .030. Scatterplots showed that children with a higher IQz learned more from positive feedback than children with a lower IQz. Additionally, the two-way interaction of IQz and feedback type reached significance, F(56,60)= 9.16, p = .004, ŋp2 = .14. The correlations between IQz and feedback processing in the deterministic type, r = .07, p = .606, and in the stochastic type, r = –.12, p = .368, were not significant, but it was clear that only in the stochastic feedback type, IQz and learning were somewhat related. Another new significant interaction of age with feedback type emerged, F(56,60)= 579.41, p = .009, ŋp2 = .16. This was interesting because, in the previous analyses, no age effects were observed. Figure 4 shows that both 7- and 11-year-old children learned with deterministic feedback and deteriorated with stochastic feedback, but in the 9-year-olds deterioration, this did not occur.
Follow-up analyses with age-specific models showed that a significant main effect of feedback type was found in the 7-year-old children, F(1,20) = 29.01, p < .001, ŋp2 = .62, whereas in the 9-year-old children, the statistical effects of feedback type did not reach significance, ps > .056. Thus, when no deterioration because of stochastic feedback occurred (see Figure 4), IQ was not relevant either. Only in the 11-year-old children were both the main effect of feedback type, F(1,20)= 33.86, p < .001, ŋp2 = .65, and the interaction of feedback type with IQz, F(1,20) = 11.60, p = .003, ŋp2 = .39, significant, which shows that IQ became more important with age. However, after adding the SCQ score as a second covariate to the model, neither the interaction of feedback type and feedback valence, F(55, 60) = 2.70, p = .106, nor the interaction of feedback valence and IQz F(55, 60) = 3.70, p = .060, were still significant. All other previously reported significant results with IQz remained significant.
When, in the second analysis, the SCQ was added first and the IQz second as covariates, the effect of feedback type remained significant F(1,60) = 18.58, p < .001, ŋp2 = .25, with reduced effect size, indicating some influence of the SCQ on the processing of stochastic and deterministic feedback, see Table 7. Feedback valence also remained significant F(1,60) = 154.48, p < .001, ŋp2 = .74, showing that the superior effect of positive over negative feedback occurs independently of the level of communication skills, social interaction, and restricted, repetitive patterns of behavior, interests, and activities of children. However, the interaction of feedback valence and feedback type was no longer significant, F(1,60) = 29.01, p = .122. This effect demonstrated that positive feedback had more impact on learning when feedback was always correct than when it was only likely to be correct, while there was no significant difference for learning from negative feedback in either the deterministic or stochastic feedback type. Thus, the SCQ scores were negligible for the processing of positive and negative feedback in the deterministic and stochastic feedback types, although individual differences in the responses of the SCQ did explain the higher impact effect of positive deterministic feedback.
After adding the IQz scores as a second covariate to the model, feedback type and feedback valence remained significant, ps < .001. Additionally, as in the previous reverse entering of covariates, again the interaction of feedback type and age, F(55,60) = 4.43, p = .017, ŋp2 = .14, as well as the interaction of feedback type and IQz became significant, F(55,60) = 7.62, p = .008, ŋp2 = .12, confirming the above results.
Discussion
Young children may underestimate uncertainty because they are overly confident (Beck et al., 2011). Stochastic feedback with only a small percentage of false feedback leads to an instant and pronounced performance decrease in typically developing children, though they do recover in repetitions of the task (Lange-Küttner et al., 2012). Also, the current study showed that children in a sequence learning task recover rapidly from uncertain feedback. As a new result, we showed that uncertainty makes children look back and relate more to their own responses between each of the task repetitions. Moreover, we investigated whether differences in age, nonverbal IQ, or social skills can explain the pronounced performance deterioration and recovery in children when receiving stochastic feedback. In short, the results showed that nonverbal IQ is a better predictor of processing stochastic feedback than social interaction and communication skills as well as age. This result supports the study with adolescents by Lange-Küttner et al. (2021), who also found nonverbal IQ to be a better predictor than social skills. However, we also found a small effect that children’s recovery from stochastic feedback during repetitions was more limited when they tended to show perseverative responses.
Sequence Learning
As expected, all children performed better with deterministic than stochastic feedback, which is in line with the study by Lange-Küttner et al. (2012). Furthermore, all children significantly improved in the repeated trials, with a very large effect size speaking to the replicability of this effect also shown in previous research (Lange-Küttner et al. 2012; Hentschel et al. 2016).
As hypothesized, nonverbal IQ was linked to sequence learning, in line with previous research that also found that explicit but not implicit sequence learning was linked to IQ (Gebauer & Mackintosh, 2007; Reber et al., 1991) – and we had also explicitly told children about the stochastic feedback before the task. However, we also found that children’s stereotypical behavior was linked to sequence learning as children with stereotypies performed less well than those without. In the study by Hentschel et al. (2016), children with ASD performed equally well in the deterministic and stochastic feedback types. Since one defining symptom of ASD is stereotypical and perseverative behavior, the current study provides a possible explanation for the apparent lack of trust found in that study even when feedback was 100 % correct. The similar scores in both deterministic and stochastic feedback types are likely to have been caused by stereotypical behavior that would involve pressing the same buttons again and again (possibly indicating some exaggerated self-determination). The shared variance between IQ and the stereotypy subscale of the SCQ, when used as covariates controlling the recovery from stochastic feedback, shows an interesting contingency that would benefit from further research.
Feedback Valence Processing
In accordance with previous research, children learned more from positive than negative feedback (Chiviacowsky et al., 2008; Eppinger et al., 2009; Lange-Küttner et al., 2012; Puddefoot et al., 1997; Schmittmann et al., 2012; van den Bos et al., 2009; Van Duijvenvoorde et al., 2013). In addition, we showed that, as per effect sizes, feedback valence (positive/negative) is more important for children this age than the veracity of the feedback (deterministic/stochastic). The learning difference between deterministic and stochastic feedback in repeated trials was about 10 %, while the difference between positive and negative feedback was nearly 50 %. So, for children, positive feedback – whether true or not – has the biggest impact on learning. The developmental shift of focus from positive to negative feedback for error correction (Crone et al., 2006; Groen et al., 2007; Van Duijvenvoorde et al., 2013) from 12 years onwards was not found here because the children in the oldest age group in this study were only 11 years old. However, a follow-up study using this task indicates that positive feedback stays important for teenagers (Lange-Küttner, et al., 2021).
As predicted, results confirmed previous research that feedback processing is associated with IQ (Brand et al., 2009; Luwel et al., 2011; van den Bos et al., 2012) as children with a higher IQ learned more from feedback. The addition of the nonverbal IQ improved the model as it explained individual differences, especially in the stochastic feedback type, and less so also individual differences in the take-up of positive feedback. Different from recovery from stochastic feedback, the effect of IQ was not canceled out by social skills when predicting the correct next button press.
The results for the group of 9-year-olds sheds even more light on the association between feedback valence and nonverbal IQ. Only this age group showed no deterioration in the face of negative feedback and nonverbal IQ was not relevant either. This suggests that nonverbal IQ may have been deployed in the face of adversity of stochastic feedback rather than having been inherently required in feedback processing in the sequencing task.
We could add to the studies of Brand et al., (2009), who used nonverbal IQ, and Luwel et al. (2011), who used verbal and nonverbal IQ. Both found that children with low IQ benefited more from feedback and relied more on feedback than children with high IQ. In our study, children with high IQ improved more during the recovery from stochastic feedback. Moreover, IQ correlated with positive but not with negative feedback, which is in line with previous findings of van den Bos et al. (2012), even though they used verbal and nonverbal IQ.
One limitation of the study is that we did not measure reaction times. Latencies could have given more information if the observed differences resulted from thinking longer about responses because of an in/congruency effect (Zhao et al., 2020). Another limitation is that the study could have benefited from a bigger sample size to add more statistical power, though we did use an extensive within-subject measurement. Yet another limitation is that we had to z-transform the IQ scores of the established psychometric tests, because they were normed at different times with different cohorts in the last decades. It would have been more desirable if the IQ test had covered the entire age period, for instance, like the Raven Coloured Progressive Matrices test does (Raven et al., 1998), but the Raven does not offer IQ norms. Additionally, another limitation is that we only used a nonverbal IQ measurement and not an additional verbal IQ, or complete IQ score, which could have ruled out alternative explanations for recovery during repetition and the dependence on positive feedback.
In conclusion, this study investigated the underlying skills associated with the processing of stochastic feedback in a sequence learning task in children aged 7 to 11 years. We showed that IQ is an excellent predictor of children’s coping with stochastic feedback. In contrast, social interaction and communication skills, as well as age seem to play a minor role. Thus, we could add new evidence to previous findings on the importance of IQ in sequence learning. IQ was relevant not only for the level of sequence learning under stochastic feedback but also for the recovery after deterioration from such feedback. Moreover, IQ and positive reinforcement via positive feedback shared a significant amount of variance. We thus conclude that our study bridges two camps: behaviorism, which does not draw boundaries between humans and animals and aims to measure reflex-like responses (Skinner, 1966); and psychometric intelligence, which aims to measure higher cognitive processes (Binet & Simon, 1961). Moreover, past research on uncertainty focused especially on the social process of how children resolve uncertainty by taking into account information from other people and making majority-based decisions (e. g., Morgan et al., 2015). In contrast, the current study refers to the internal loops of thinking by analyzing how children recover during repetitions after their performance had suffered. As such, this research is particularly helpful when investigating how degraded internal representations can be repaired in an internal intraindividual process.
We would like to thank all children, parents, and teachers of the schools in Dithmarschen who helped us to collect the data.
Literatur
(1969). The theory of cognitive dissonance: A current perspective. Advances in Experimental Social Psychology, 4, 1 – 34. https://doi.org/10.1016/S0065-2601(08)60075-1
(2011). Probabilistic learning and inference in schizophrenia. Schizophrenia Research, 127(1 – 3), 115 – 122. https://doi.org/10.1016/j.schres.2010.08.009
(2011). Imagining what might be: Why children underestimate uncertainty. Journal of Experimental Child Psychology, 110 (4), 603 – 610. https://doi.org/10.1016/j.jecp.2011.06.010
(2010). SOCIAL?: An integrative framework for the development of social skills. Psychological Bulletin, 136 (1), 39 – 64. https://doi.org/10.1037/a0017768
(2018). Promoting replicability in developmental research through meta-analyses, insights from language acquisition research. Child Development, 89 (6), 1996 – 2009. https://doi.org/10.1111/cdev.13079
(1961).
The development of intelligence in children . In J. J. JenkinsD. G. Paterson (Eds.), Studies in individual differences: The search for intelligence (pp. 81 – 111). Appleton-Century-Crofts. https://doi.org/10.1037/11491-008(1999). Aurally aided visual search in three-dimensional space. Human Factors, 41 (4), 664 – 669. https://doi.org/10.1518/001872099779656789
(2006). Fragebogen zur Sozialen Kommunikation – Autismus Screening (FSK). Huber.
(2009). The decision is made with and without feedback: The role of intelligence, strategies, executive functions, and cognitive styles. Journal of Clinical and Experimental Neuropsychology, 31 (8), 984 – 998. https://doi.org/10.1080/13803390902776860
(2012). MorePower 6.0 for ANOVA with relational confidence intervals and Bayesian analysis. Behavior Research Methods, 44, 1255 – 1265. https://doi.org/10.3758/s13428-012-0186-0
(2008). Self-controlled feedback in 10-year-old children. Research Quarterly for Exercise and Sport, 79 (1), 122 – 127. https://doi.org/10.1080/02701367.2008.10599467
(2006). A heart rate analysis of developmental change in feedback processing and rule shifting from childhood to early adulthood. Journal of Experimental Child Psychology, 95 (2), 99 – 116. https://doi.org/10.1016/j.jecp.2006.03.007
(2009). Developmental differences in learning and error processing: Evidence from ERPs. Psychophysiology, 46 (5), 1043 – 1053. https://doi.org/10.1111/j.1469-8986.2009.00838.x
(1957). A theory of cognitive dissonance. Stanford University Press.
(1999). Intelligence reframed: Multiple intelligences. Basic Books.
(2014). Transferring control demands across incidental learning tasks: Stronger sequence usage in serial reaction task after shortcut option in letter string checking. Frontiers in Psychology, 5, 1 – 11. https://doi.org/10.3389/fpsyg.2014.01388
(2007). Psychometric intelligence dissociates implicit and explicit learning. Journal of Experimental Psychology, 33 (1), 34 – 54. https://doi.org/10.1037/0278-7393.33.1.34
(2007). Physiological correlates of learning from performance feedback in children: A study of EEG event-related potentials and evoked heart rate. Biological Psychology, 76 (3), 174 – 187. https://doi.org/10.1016/j.biopsycho.2007.07.006
(1985). Visual perspective-taking skills in children. Child Development, 56 (5), 1204 – 1213. https://doi.org/10.2307/1130235
(2014). Implicit visual learning: How the task sets modulates learning by determining the stimulus-response binding. Consciousness and Cognition, 26, 145 – 161. https://doi.org/10.1016/j.concog.2014.03.005
(2016). Sequence learning with stochastic feedback in a cross-cultural sample of boys in the autistic spectrum. Education and Training in Autism and Developmental Disabilities, 51 (2), 179 – 194.
(2012). The best time to acquire new skills: Age-related differences in implicit sequence learning across the human lifespan. Developmental Science, 15 (4), 496 – 505. https://doi.org/10.1111/j.1467-7687.2012.01150.x
(2007). Oculomotor and manual indexes of incidental and intentional spatial sequence learning during middle childhood and adolescence. Journal of Experimental Child Psychology, 96 (2), 107 – 130. https://doi.org/10.1016/j.jecp.2006.05.005
(2021). Age differences in decision making under known risk: The role of working memory and impulsivity. Developmental Psychology, 57 (2), 241 – 252. https://doi.org/10.1037/dev0001132
(2012). Sequence learning under uncertainty in children: Self-reflection vs. self-assertion. Frontiers in Psychology, 3, 2 – 11. https://doi.org/10.3389/fpsyg.2012.00127
(, B., Hentschel, M., & Baumbach, J. ). (2021). Intelligence matters for stochastic feedback processing during sequence learning in adolescents and young adults. Intelligence. https://doi.org/10.1016/j.intell.2021.101542
(2011). The role of intelligence and feedback in children’s strategy competence. Journal of Experimental Child Psychology, 108 (1), 61 – 67. https://doi.org/10.1016/j.jecp.2010.06.001
(1998). Implicit sequence learning in children. Journal of Experimental Child Psychology, 69 (3), 199 – 221. https://doi.org/10.1006/jecp.1998.2442
(2015). The development of adaptive conformity in young children: effects of uncertainty and consensus. Developmental Science, 18 (4), 511 – 524. https://doi.org/10.1111/desc.12231
(1997). Effect of verbal feedback on the physical performance of children. Physiotherapy, 83 (2), 76 – 81. https://doi.org/10.1016/S0031-9406(05)65581-2
(1966). Recognition and learning of visual sequences in young children. Child Development, 37 (4), 827 – 836. https://doi.org/10.1111/j.1467-8624.1966.tb05410.x
(1998). Raven manual: Section 2. Colored Progressive Matrices. Oxford Psychologists Press.
(1991). Implicit and explicit learning: Individual differences and IQ. Journal of Experimental Psychology, 17 (5), 888 – 896. https://doi.org/10.1037//0278-7393.17.5.888
(2001). Social communication questionnaire (SCQ). Western Psychological Services.
(2012). Distinct discrimination learning strategies and their relation with spatial memory and attentional control in 4- to 14-year-olds. Journal of Experimental Child Psychology, 111 (4), 644 – 662. https://doi.org/10.1016/j.jecp.2011.10.010
(2010). Effects of dopamine medication on sequence learning with stochastic feedback in Parkinson’s disease. Frontiers in Systems Neuroscience, 4, 1 – 9. Article 36. https://doi.org/10.3389/fnsys.2010.00036
(1966). The phylogeny and ontogeny of behavior. Science, 153 (3741), 1205 – 1213. https://doi.org/10.1126/science.153.3741.1205
(2001). Serial reaction time learning in preschool- and school-age children. Journal of Experimental Child Psychology, 79 (4), 364 – 387. https://doi.org/10.1006/jecp.2000.2613
(2012). Brain function during probabilistic learning in relation to IQ and level of education. Developmental Cognitive Neuroscience, 25, 578 – 589. https://doi.org/10.1016/j.Dcn.2011.09.007
(2009). Better than expected or as bad as you thought? The neurocognitive development of probabilistic feedback processing. Frontiers in Human Neuroscience, 3, 1 – 11. https://doi.org/10.3389/neuro.09.052.2009
(2013). Decomposing developmental differences in probabilistic feedback learning: A combined performance and heart-rate analysis. Biological Psychology, 93 (1), 175 – 183. https://doi.org/10.1016/j.biopsycho.2013.01.006
(2012). Incidental sequence learning across the lifespan. Cognition, 123 (3), 380 – 391. https://doi.org/10.1016/j.cognition.2012.02.010
(2006). CFT 20-R, Grundintelligenztest Skala 2. Hogrefe.
(1997). CFT 1, Grundintelligenztest Skala 1 (5th ed.). Hogrefe.
(2020). Sequential modulation of a cross-task congruency in the serial reaction time task. Acta Psychologica, 205, 1 – 10. https://doi.org/10.1016/j.actpsy.2020.103043
(2018). Effects of overlap between consecutive words on speeding typing inform about the representation of serial order within words. Advances in Cognitive Psychology, 14 (3), 126 – 138. https://doi.org/10.5709/acp-0244-6
Appendix A
Instructions of the Sequence Learning Task for Both Task Order Groups
Instructions for the deterministic condition first group were as follows:
Welcome to the “Learn the 4 Button Press”. You are given two buttons to press. You
need to learn to press these buttons in a particular sequence, making a total of 4 button
presses. For instance, Left, Right, Left, Right. Each time you press a button a GREEN
or a RED circle will appear. The circle is like a streetlight which shows whether you
pressed the right or wrong button. The computer always gives you the correct
feedback. When you press the CORRECT button a green circle will appear. When you
press the WRONG button a red circle will appear. After learning the correct 4 Button
Presses repeat the same sequence six times and you will proceed to learn another new
sequence.
Instructions for the stochastic feedback second were as follows:
Welcome to “Learn the 4 Button Presses task”. This task is the same task as before,
except this time the computer will also give you wrong feedback 15 percent of the time.
This means, even if you pressed the right button it may appear as false with a red
circle. Even if you pressed the wrong button it may appear as right with a green circle.
Therefore even if you think you know the sequence sometimes you may have to ignore
the feedback. Are you ready?
Instructions for the stochastic feedback first were as follows:
Welcome to the “Learn the 4 Button Press”. You are given two buttons to press. You
need to learn to press these buttons in a particular sequence, making a total of 4 button
presses. For instance, Left, Right, Left, Right. Each time you press a button a GREEN
or a RED circle will appear. The circle is like a streetlight which shows whether you
pressed the right or wrong button. The computer gives you also 15 percent wrong
feedback. This means, even if you pressed the right button it may appear as false with
a red circle. Even if you pressed the wrong button it may appear as right with a green
circle. Therefore even if you think you know the sequence sometimes you may have to
ignore the feedback. Are you ready?
Instructions for the deterministic feedback second were as follows:
Welcome to “Learn the 4 Button Presses task”. This task is the same task as before,
except this time the computer will only give you correct feedback. This means, when
you press the CORRECT button a green circle will appear. When you press the
WRONG button a red circle will appear. Are you ready?
Appendix B
Statistical Analyses Using Uncorrected IQ Norms
Sequence Learning
Adding the IQ score as covariate in the model showed that the performance level remained significant F(1,60)= 8.45, p < .001, ŋp2= .13 but with a reduced effect size, meaning children’s learning and improvement during the experiment was only partly dependent on their IQ. However, the main effect of feedback type and the two-way interaction of feedback type and performance were not significant anymore, ps > .807, suggesting that recovery from stochastic feedback is entirely determined by IQ.
Figure B1. Learning in the deterministic (black) and stochastic (grey) feedback type for the three different age groups.
Feedback Processing
When the model was controlled for IQ as covariate, the feedback valence and the interaction of feedback valence and feedback type were no longer significant ps > .319. This means IQ had replaced the effect of positive feedback. Only the main effect of feedback type (deterministic/stochastic) still reached significance F(1,60)= 4.25, p = .044, ŋp2= .07 with a much reduced effect.
Some further impact of IQ was highlighted by a new two-way interaction of IQ and feedback valence F(1,60)= 4.82, p = .032, ŋp2= .08. The correlations between IQ and negative feedback processing, r = -.16, p = .223, were not significant, indicating no shared variance between IQ and negative feedback processing. But the correlation between IQ and positive feedback processing reached significance, r = .35, p = .006. Scatterplots showed that children with a higher IQ learned more from positive feedback than children with a lower IQ.
Additionally the interaction of IQ and feedback type reached significance, F(1,60)= 8.45, p = .005, ŋp2= .13. The correlations between IQ and feedback processing in the deterministic condition, r = .23, p = .083, and feedback processing in the stochastic condition, r = -.19, p = .157, were not significant, but there was a clear difference in direction of the correlations. Children of high intelligence benefited from reliable feedback, while they were easily confused by stochastic feedback.
Another new significant interaction of age with feedback type emerged, F(1,60)= 4.40, p = .017, ŋp2= .14. This was interesting as in the previous analyses, no age effects could be observed. Figure shows that both 7- and 11-year old children learned with deterministic feedback and deteriorated with stochastic feedback, but in the 9-year-olds deterioration did not occur.
Follow-up analyses with age-specific models of analyses of variance with repeated measures were carried out with feedback type as dependent variable and IQ as covariate.
In the 7-year-old children, only a significant interaction of feedback type and IQ, F(1,20)=5.35, p = .033, ŋp2= .23 was found. No other effect reached significance, ps > .066. In the 9-year-old children there was no significant effect , ps > .333. However, in the 11-year-old children, both the main effect of feedback type, F(1,20)= 5.75, p = .027, ŋp2= .24 and the interaction of feedback type with IQ, F(1,20)= 9.72, p = .006, ŋp2= .35, were significant. Thus, when no deterioration due to stochastic feedback occurred (see Figure B1), intelligence was not relevant either.
