Task Switching Hurts Memory Encoding

With the beginning of the industrial world, it was a major issue to find the most efficient way to execute work procedure. According to Taylorism, the parsing of a procedure into small parts and the repetition of those small elements by eliminating all unnecessary movements was this “one best way” (Kanigel, 2005). However, in order to specify how goal-directed behavior is implemented, in the quest to understand cognitive processing, Miller, Galanter, and Pribam (1960) suggested a “test-operate-test-exit” (TOTE) unit, which, by definition, includes task switches as an optimal way to efficient performance. While successful performance necessarily requires flexibility, investigating the consequences of switching tasks on memory has just begun. In laboratory situations, such behavior is typically explored with the task-switching paradigm (e.g., Allport, Styles, & Hsieh, 1994; Jersild, 1927; Rogers & Monsell, 1995). The main goal of the present study was to investigate how task switching affects subsequent memory performance.

Cognitive control refers to the ability to form a plan, to maintain it in face of distraction, and to adjust behavior appropriately in case of cognitive conflict (Norman & Shallice, 1986; Posner & Snyder, 1975; Botvinick, Braver, Barch, Carter, & Cohen, 2001). Task switching is a typical example in which cognitive control is necessary. The increase in cognitive control associated with the requirement to switch between two tasks usually results in slower and less accurate performance compared to repeating the same task (e.g., Rogers & Monsell, 1995). The conflict produced by task switching is assumed to reflect the involvement of endogenous control processes that are needed to reconfigure the task set (Vandierendonck, Liefooghe, & Verbruggen, 2010; Rogers & Monsell, 1995). The requirement for cognitive control is further enhanced when the material involves bivalent stimuli, that is, stimuli that can be used to perform both tasks rather than univalent stimuli. For example, if one task requires participants to classify animals as birds or mammals and the other task requires participants to classify objects as musical instruments or kitchen utensils, a sparrow would be a univalent stimulus because it can only be used for the animal task but not for the object task. In contrast, if one task requires participants to classify a stimulus by size (e.g., as bigger or smaller than a soccer ball) and the other task requires participants to classify a stimulus by animacy (i.e., as living or non-living), a sparrow would be a bivalent stimulus because it can be used for both, the size and the animacy task. Bivalent stimuli create an additional conflict because they not only require to switch task, but also to select which task to perform (Allport et al., 1994; Woodward, Meier, Tipper, & Graf, 2003). Responding to bivalent stimuli causes slower reaction times compared to responding to univalent stimuli and even leads to long-lasting slowing on subsequent performance (i.e., the “bivalency effect,” Meier, Woodward, Rey-Mermet, & Graf, 2009; Woodward et al., 2003). Both types of conflicts – task switching and bivalency – contribute to “switch costs” as they both slow down reaction times and increase error rates (Jersild, 1927; Rogers & Monsell, 1995). However, as most task-switching experiments involve bivalent stimuli, the effects of switching and bivalency on switch costs are typically confounded. By using one experiment with univalent stimuli and one experiment with bivalent stimuli, we aimed to assess the separate impact of task switching and bivalency on subsequent memory performance in the present study.

So far, only a few studies have examined the effect of task switching on memory and all of them used bivalent stimuli. Reynolds, Donaldson, Wagner, and Braver (2004) investigated encoding processes during switching and repeating a task. In the study phase, participants performed two semantic classification tasks with single words. In two blocks, they performed one of the tasks alone (single-task condition), and in one block, they switched between the two tasks (task-switching condition). In a subsequent memory test, more words from the single-task compared to the task-switching condition were recognized correctly. Thus, memory performance was lower when control demands were higher. More interestingly for the purpose of the present study, within the task-switching blocks, memory performance for repeat stimuli was better than for switch stimuli, suggesting not only a block-specific but also a trial-specific effect. Together, the higher cognitive demands associated with task switching reduced memory performance.

Richter and Yeung (2012) also investigated the effect of task switching on memory. They used compound stimuli consisting of pictures and words and participants had to switch between classifying them. Thus, each trial consisted of task-relevant (target) and task-irrelevant (distractor) information. The results showed that task switching compared to task repetition impaired memory performance for targets, but improved memory performance for distractors. The authors explained the latter with interference from previously active task sets (i.e., task-set inertia; Allport et al., 1994). Due to residual attention to the competing, now-irrelevant task, encoding of the distractor would be facilitated in switch trials (Yeung, Nystrom, Aronson, & Cohen, 2006). In contrast, attention toward task-relevant information was unimpeded in repeat trials, resulting in better encoding for targets in repeat compared to switch trials. In a follow-up study, Richter and Yeung (2015) replicated these results.

Chiu and Egner (2016) focused on task-irrelevant stimulus features by investigating two distractor categories. In one group, participants switched between two classification tasks, the distractors were relevant in one task and irrelevant in the competing task. In the other group, the distractors (objects in the background) were never task relevant. The results showed better memory for distractors which were task relevant in one of the two tasks on switch compared to repeat trials, indicating that task-set inertia enhanced distractor encoding (Yeung et al., 2006). In the other condition with the truly irrelevant distractors, the results showed that memory for distractors was lower in switch than in repeat trials, indicating that the higher cognitive demands associated with task switching reduced encoding of completely irrelevant information (Jenkins, Lavie, & Driver, 2005).

Together, these findings suggest that task switching affects incidental memory performance. The interference associated with task switching results in less focused attention toward task-relevant information, leading to lower memory performance (Richter & Yeung, 2012, 2015). However, as all the previous studies have used bivalent stimuli, task switching and stimulus bivalency were confounded. In order to address the pure impact of task switching, we used univalent stimuli in Experiment 1 of the present study. Moreover, all the previous studies have used a task-cueing procedure in which a cue signals which task is to be performed such that switch and repeat trials appear in a random order (e.g., Shaffer, 1965). Task cueing requires the active maintenance of both task sets and may thus present additional attentional monitoring demands (Braver, Reynolds, & Donaldson, 2003). In contrast, in the present study, we used the alternating run paradigm in which switch and repeat trials appear in a predictable order (e.g., AABB) in order to reduce these demands (cf. Rogers & Monsell, 1995).

Experiment 1

The aim of Experiment 1 was to test whether the conflict triggered by task switching affects subsequent recognition memory performance. Participants performed two different tasks (animal and object classification) in a regular AABB-order. For half of the participants, the stimuli were presented as words, and for the other half, they were presented as pictures. Importantly, all the stimuli were univalent.

Method

Participants

The participants were 80 volunteers (43 male and 37 female) from the general population, recruited by word of mouth, and all of them were German speaking with an age from 18 to 35 years (M = 24.70, SD = 4.51). The study was approved by the local ethical committee of the University of Bern; all participants gave written consent.

Material

For the condition with pictures, the material consisted of 160 photographs of easy to name stimuli. The pictures were collected from a web search. Half were animals (mammals or birds), and the other half were objects (musical instruments or kitchen utensils). The size of the photographs was approximately 300 × 300 pixels. For the condition with words, 160 words were used. They were typical exemplars of the same four categories and consisted of 3–10 letters. The words were displayed in black letters against a white background in Courier New font.¹

The stimuli were divided into two lists of 80 pictures and words, respectively, and contained an equal number of stimuli of the four categories. One of the lists was used in the study phase, and both lists were presented in the test phase. The stimuli were counterbalanced across participants, so that each stimulus occurred equally often in the repeat and switch condition.

Procedure

One half of the participants were tested with words and the other half with pictures; they were randomly assigned to each condition and were tested individually in a computer laboratory. In the study phase, they were instructed to categorize the stimuli as quickly and correctly as possible. For animals, participants had to classify them as mammal or bird, and for objects, they had to classify them as musical instrument or kitchen utensil. The stimuli were presented randomized in the middle of the screen, each task twice in succession (see Figure 1). After a practice phase with 10 trials, participants performed the study phase with 80 trials. They responded on a standard computer keyboard using their index fingers. They had to press the a-key when the stimulus was either a mammal or a musical instrument and the l-key when the stimulus was either a bird or a kitchen utensil. The stimuli were presented until a response key was pressed, and then the next stimulus was presented after 200 ms of blank screen.

Following the study phase, participants had to complete a demanding reading span task (Daneman & Carpenter, 1980). The main purpose of this task was to create a filled retention interval between study and test phase. Participants had to read a series of two to six sentences. For each sentence, they had to indicate whether it was meaningful and they had to recall the last word of the sentence. Reading span was defined as the size of the largest set in which all words were correctly recalled in at least three of the five consecutive trials.

The third part of the experiment involved an incidental recognition memory test and an additional remember/know judgment (cf. Meier, Rey-Mermet, Rothen, & Graf, 2013). Participants had to indicate whether they had seen a stimulus already during the task-switching phase by pressing the j-key for “old” stimuli or not by pressing the n-key for “new” stimuli. In case of an “old”-response, they were required to give an additional remember/know judgment by pressing the 1-key for “remember” or the 2-key for “know” on the number pad. For each trial, the stimulus was presented in the middle of the screen until a response key was pressed. The stimuli appeared in randomized order with an interval of 200 ms. One half of the stimuli were old (presented in the study phase) and the other half new (unseen). The entire experiment lasted about 25 min. All raw data for Experiment 1 are listed in the Electronic Supplementary Material, ESM 1.

Analysis

For the study phase, mean reaction times and accuracy in the task-switching phase were analyzed separately using an analysis of variance (ANOVA) with the within-subject factor trial type (repeat vs. switch) and the between-subject factor material (words vs. pictures). For the test phase, the hit and the false alarms for each participant were computed. As it was not possible to assign the false alarm rates to the repeat or switch condition, we used hit rates only as recognition scores (cf. Ortiz-Tudela, Milliken, Botta, LaPointe, & Lupiañez, 2016). Memory performance was also analyzed with the within-subject factor trial type (repeat vs. switch) and the between-subject factor material (words vs. pictures). In addition, remember and know responses were analyzed separately. Reading span score was correlated with accuracy, reaction times, and the hit rate. We excluded one participant with an error rate > 30% in the study phase. An α level of .05 was used. Effect sizes are expressed as η_p² values.

Results

Study Phase

As expected, participants were faster to respond to repeat (M = 1,110 ms, SE = 29) than to switch trials (M = 1,234 ms, SE = 37), F(1, 77) = 41.5, p < .001, η_p² = .35. Overall, participants were faster to respond to pictures (M = 970 ms, SE = 46) than to words (M = 1,375 ms, SE = 45), F(1, 77) = 39.6, p < .001, η_p² = .34, but the interaction was not significant, F(1, 77) = 0.37, p = .543, η_p² = .05). The same ANOVA on the accuracy data revealed that performance was lower on switch (M = 0.93, SE = 0.01) than on repeat trials (M = 0.95, SE = 0.01), F(1, 77) = 10.1, p = .002, η_p² = .12. Accuracy was lower for words (M = 0.92, SE = 0.01) than for pictures (M = 0.96, SE = 0.01), F(1, 77) = 16.3, p < .001, η_p² = .18, but the interaction was not significant F(1, 77) = 3.02, p = .086, η_p² = .04, indicating that switch costs were not different for words and pictures. Together, our results showed typical switch costs.

Test Phase

Overall, the proportion of hits was M = 0.71, SE = 0.14, and the proportion of false alarms was M = 0.23, SE = 0.13. The ANOVA with the factors trial type and materials revealed that memory was significantly better for repeat (M = 0.72, SE = 0.13) than for switch trials (M = 0.70, SE = 0.17), F(1, 77) = 6.8, p = .011, η_p² = .08. Words and pictures did not differ, F(1, 77) = 1.74, p = .19, η_p² = .02, and the interaction was not significant, F(1, 77) = 1.37, p = .245, η_p² = .02. The critical analysis is depicted in Figure 2.

To assess the contribution of remember and know judgments on memory performance, additional ANOVAs with the same design were conducted. Significantly more remember responses were associated with repeat (M = 0.53, SE = 0.02) than with switch trials (M = 0.49, SE = 0.02), F(1, 77) = 12.75, p = .001, η_p² = .14; know responses did not vary with trial type, F(1, 77) = 1.50, p = .225, η_p² = .02. No other effect was significant, F < 2.06, p > .155. Thus, the difference in memory performance between switch and repeat trials was due to higher recollection than familiarity.

Follow-up Analysis

In order to explore the relationship between the task switching and memory results and working memory capacity, we analyzed the reading span task. The average reading span was 2.72 (SD = .95). This score was not significantly correlated to the scores of hits (r = −.04), accuracy (r = .11), or reaction times (r = −.16). Therefore, working memory capacity did not seem to be related to task or memory performance.

Experiment 2

In Experiment 2, we used pictures as stimuli and participants had do classify them as smaller or bigger than a soccer ball or as living or non-living. As all the stimuli could be used for both tasks, they were bivalent. Moreover, as we used the same set of response keys for both tasks, a third kind of conflict occurred on some trials, that is, response incompatibility. If a stimulus would require the same key for both tasks, for example, the a-key to classify a picture of an elephant as bigger than a soccer ball in the size task and as living in the animacy task, the response mapping was compatible. In contrast, when the stimulus required different response keys for each of the tasks, for example, the a-key to classify a house as bigger than a soccer ball and the l-key to classify it as non-living, the response mapping was incompatible. For incompatible response mappings, the inappropriate response has to be suppressed and this usually slows down performance (Gade & Koch, 2007; Kornblum, Hasbroucq, & Osman, 1990). We expected lower memory performance for incompatible and switch stimuli due to the presence of conflict. Moreover, we expected a stronger effect for bivalent compared to univalent materials because of the between-task conflict with bivalent materials (Allport et al., 1994; Meier et al., 2009).

Method

Participants and Design

The participants were 40 undergraduate students (4 male and 36 female) from the University of Bern, and all of them were German speaking. The age ranged from 19 to 33 years (M = 21.79, SD = 2.75), and they participated in the study for course credits. The study was approved by the local ethical committee of the University of Bern, and all participants gave written consent.

Material

A total of 128 colored photographs were used which were collected from a web search (see Footnote 1). They could be classified both as smaller or bigger than a soccer ball and as living or non-living. The stimuli were arranged in separate lists of 64 pictures, counterbalanced across category and trial type, such that each stimulus occurred equally often in the repeat and switch condition and in each task. One of the lists was used in the study phase, and both lists were presented in the test phase. Lists were counterbalanced across participants.

Procedure

The procedure was identical as in Experiment 1 with the following exceptions. Participants were instructed to perform the size task when the stimulus appeared in the upper part of the screen and to perform the animacy task when it appeared in the lower part. The stimuli were presented clockwise, beginning in the upper half on the left, which led to a predictable AABB sequence of the two tasks as depicted in Figure 3. Participants had to press the a-key when an object was bigger than a soccer ball or living and the l-key when the object was smaller than a soccer ball or non-living. After a brief practice phase with 8 trials, participants performed the study phase with 64 trials. After the reading span task which was identical to Experiment 1, the recognition memory test was administered with 128 stimuli, half of them old and the other half new. The entire experiment lasted about 25 min. All raw data for Experiment 2 are listed in ESM 2.

Analysis

For the study phase, task-switching performance was analyzed using a 2 (Trial Type: switch vs. repeat) × 2 (Response Type: compatible vs. incompatible) ANOVA for both reaction times and accuracy. For the test phase, the proportion of hits and the false alarms were analyzed. As it was not possible to assign the false alarm rates to repeat or switch trials, we used hit rates only as recognition scores (cf. Ortiz-Tudela et al., 2016). Memory performance and the remember/know judgments were analyzed using the same two factors trial type and response type. One participant was excluded because reaction time performance was more than 3 SD slower than all other participants. An α level of .05 was used. Effect sizes are expressed as η_p² values.

Results

Study Phase

Reaction time analysis revealed that the participants responded significantly faster on repeat (M = 1,098 ms, SE = 41) than on switch trials (M = 1,536 ms, SE = 64), F(1, 38) = 118.72, p < .001, η_p² = .76. Response type, F(1, 38) = .30, p = .59, η_p² = .01, and the interaction between trial type and response type were not significant, F(1, 38) < .01, p = .99, η_p² < .01.

Accuracy analysis revealed that participants were more accurate on repeat (M = 0.95, SE = 0.01) than on switch trials (M = 0.92, SE = 0.01), F(1, 38) = 10.15, p = .003, η_p² = .21. Response type, F(1, 38) = 1.96, p = .170, η_p² = .05, and the interaction between response type and trial type were not significant, F(1, 38) < 1, p = .922, η_p² < .01.

Test Phase

The proportion of hits was M = 0.76, SE = 0.16 and the proportion of false alarms was M = 0.08, SE = 0.07. Hit rates only for each conflict type were further analyzed, and the results are presented in Figure 4. The ANOVA revealed that repeat stimuli were better recognized (M = 0.80, SE = 0.02) than switch stimuli (M = 0.73, SE = 0.02) as indicated by a main effect of trial type, F(1, 38) = 18.23, p < .001, η_p² = .32. Neither the main effect of response type, F(1, 38) = .01, p = .92, η_p² < .01, nor the interaction between trial type and response type were significant, F(1, 38) = .56, p = .46, η_p² < .01.

To assess the contribution of recollection and familiarity on memory performance, additional ANOVAs with the same design were conducted. Significantly more remember responses were associated with repeat (M = 0.63, SE = 0.03) than with switch trials (M = 0.56, SE = 0.03), F(1, 38) = 11.7, p < .01, η_p² = .24. In contrast, know responses did not vary with trial type, F(1, 38) = 0.11, p = .744, η_p² < .01. No other effect was significant, F < 2.88, p > .098. Thus, as in Experiment 1, the difference between switch and repeat trials was due to higher recollection than familiarity.

Follow-Up Analysis

The average reading span was 2.59 (SD = 0.68). This score was not significantly correlated to the scores of hits (r = .02), accuracy (r = .17) or reaction times (r = −.12). Again, working memory capacity did not seem to be related to task or memory performance.

General Discussion

The aim of the study was to investigate the impact of task switching on subsequent memory performance. In two experiments, we combined a task-switching procedure with an incidental recognition memory test. The stimuli were either univalent (Experiment 1) or bivalent (Experiment 2); switch and bivalent stimuli were considered as conflict stimuli. Another conflict was induced by incompatible stimulus-response mappings.

The conflict produced by task switching impaired memory performance in both experiments, as memory was lower for switch than for repeat stimuli. As there is no between-task conflict with univalent materials (Mayr & Keele, 2010; Wylie & Allport, 2000), the requirement to reconfigure the task set in switch trials may have produced this effect in Experiment 1 (Rogers & Monsell, 1995). In Experiment 2, bivalency further impaired memory performance for switch trials, reflected in a larger switch effect than in Experiment 1. The results are in line with the studies by Reynolds et al. (2004) and Richter and Yeung (2012, 2015). They also found lower memory performance with task-relevant switch stimuli. As all the previous studies used bivalent stimuli, our study is the first that provides evidence that even univalent task switching hurts memory encoding for target events.

We suggest that task switching produced interference which resulted in less focused attention toward the target events (Lavie et al., 2004), rather than diminishing a general encoding capacity. In other words, the selectivity of memory encoding was reduced under high cognitive control demands (Richter & Yeung, 2012, 2015). The results of the remember/know procedure revealed that in both experiments, fewer “remember” responses were given for switch than for repeat trials. In contrast, “know” responses did not vary according to the encoding condition. Moreover, the effect of “remember” responses regarding the difference between switch and repeat trials was stronger in Experiment 2 than in Experiment 1. This corresponds with the idea that attention was more focused in repeat than in switch trials, rendering participants more certain about their decisions. Recollection is found to be sensitive to attention manipulations (Yonelinas, 2002). For example, in an experiment by Gardiner and Parkin (1990), participants learned word lists in a full and a divided attention condition. The following word recognition test showed that divided attention reduced the “remember” responses while the “know” responses did not differ. The same pattern was found in our results: Stimuli from repeat trials, in which attention was unimpeded, led to more remember responses than stimuli from switch trials, in which attention had to be shared between target processing and task switching. This effect was more pronounced with bivalent materials, as selecting the appropriate task required more attention due to overlapping stimulus features (Allport et al., 1994; Woodward et al., 2003).

In summary, both task switching and bivalency impair memory. Interestingly, this does not generalize to all kinds of conflict. Studies on the effects of Stroop conflict on subsequent memory performance found improved memory performance for Stroop compared to non-conflicting stimuli. For example, in a study by Krebs, Boehler, De Belder, and Egner (2015), faces were presented in a study phase either with congruent information (the word man over a male face) or with incongruent information (the word woman over a male face). The subsequent face recognition test showed that irrelevant incongruent information improved subsequent memory for faces, that is, a conflict-induced memory benefit. Similar results were reported by Rosner, D’Angelo, MacLellan, and Milliken (2015). Their participants had to read one word of a word pair. Half of the items were congruent (the words had the same identity), and the other half were incongruent (the words had different identities). The results of the subsequent recognition test showed better memory for incongruent than for congruent stimuli.

Crucially, in these studies the conflict arose from the co-activation of two incompatible responses (Egner, Delano, & Hirsch, 2007), for example, the picture of a woman with the superimposed word “man” (cf. Krebs et al., 2015). In Stroop conflict, the focus of attention is strategically directed at the target in order to avoid errors (Botvinick et al., 2001; Verguts & Notebaert, 2009). As a consequence, encoding mechanisms are up-regulated, leading to better memory performance for targets. In contrast, in the present study, the conflict arose from selecting the relevant task set in a task-switching environment. When participants have to switch tasks, the focus of attention toward the target is reduced because attention is required for selecting the appropriate task. Therefore, memory performance is reduced in switch trials. In the case of bivalent stimuli, even more attention is required for selecting the relevant task due to the overlapping stimulus features and thus memory performance is further affected.

Conclusion

Finding the most efficient way to execute work procedure is a major issue of mankind. To be efficient, most approaches – as, for example, the TOTE unit (Miller et al., 1960) – favor fast and flexible shifts. While goal-directed performance can be improved by switching tasks, our results suggest that this may be unprofitable for memory: The experiments presented here provide evidence that task switching impairs memory performance for task-relevant materials. Moreover, our study is the first that provides evidence that even task switching with univalent stimuli affects memory encoding.

Electronic Supplementary Materials