The Coupled Brains of Captivated Audiences

The Collective Nature of Continuous Audience Responses to Movies

Watching a movie creates a myriad of microlevel media effects (Weber et al., 2015). The eyes and ears of each audience member will continuously respond to the stream of images and sounds, which are converted into neural signals and propagated to the visual and auditory regions of each viewer’s brain. Studies on the mechanisms of vision and audition reveal the brain regions involved in elementary processes related to color vision, form, and object perception, as well as auditory processes such as pitch analysis and voice perception (Mesulam, 1998). Because the architecture of the visual and auditory systems is similar across all humans, watching the same movie will evoke similar functional responses across different viewers’ brains. The varying on-screen images will elicit similar brain activity time series in the visual cortices of all viewers, and listening to the soundtrack will elicit similar processing in auditory regions.

To measure these brain responses and compare their similarity between audience members, we can record their brain activity using functional magnetic resonance imaging (fMRI) while they watch movies and we can compute inter-subject correlations (ISC). In ISC analysis, one calculates cross-recipient correlations between the brain activity time series measured in corresponding regions of different viewers’ brains while they are exposed to the same message (Hasson et al., 2008; Hasson, Malach, & Heeger, 2010; Schmälzle & Grall, in press). As illustrated in Figure 1, fMRI time series from viewers’ visual cortex can be extracted and compared to identify the degree to which they concur. By repeating this analysis for all viewers and across all brain regions, one can assess where in the brain and how strongly a given movie aligns neural processing across audiences. In this sense, ISC analysis yields an objective measure to examine collectively shared audience responses.

Importantly, when an audience watches a movie, the brain regions that exhibit such collectively shared processes extend beyond sensory regions and include postperceptual regions associated with various cognitive and affective functions. Decades of media psychology research demonstrate that movies do much more than only stimulate vision and audition. Instead, cognitive and emotional audience responses are key to their popularity (Tamborini, Bowman, Eden, Grizzard, & Organ, 2010). For example, ISC in higher-order brain regions differs between listeners of the same story depending on whether they possess language knowledge and thus either can or cannot follow the content (Honey, Thompson, Lerner, & Hasson, 2012). Furthermore, when people are given different background knowledge that suggests different interpretations of the same story, people with similar interpretations show more similarity in their brain responses (Yeshurun et al., 2017). Thus, rather than only reflecting common reactions driven by cuts, sounds, and image properties, assessing collective brain responses through ISC analysis can also aid the investigation of common psychological processes engaged by movies.

Mounting evidence suggests that ISC is sensitive to the strength of message content, be it in terms of strong verbal arguments or the potential to provoke strong affective responses, or due to the relevance that content has for receivers. Heightened ISC across audience brains has been shown during powerful political speeches, and health messages that were evaluated as more effective also prompt higher ISC than weaker messages, particularly in mediofrontal brain regions (Imhof, Schmälzle, Renner, & Schupp, 2017; Schmälzle, Häcker, Honey, & Hasson, 2015). Together with several other studies (Dmochowski et al., 2014; Nummenmaa, Lahnakoski, & Glerean, 2018; Nummenmaa et al., 2014), this suggests that ISC analysis could be beneficial to investigate suspense, which has been intensively studied within media psychology (Vorderer et al., 2013; Zillmann, 1980) and has begun to attract interest from neuroscience (Bezdek et al., 2015; Lehne et al., 2015).

Suspense: Theoretical Perspectives and Empirical Approaches

Everyone who has seen a movie by Alfred Hitchcock knows how powerfully suspense can affect viewers. Suspense characterizes an affective state associated with conflict and uncertainty regarding an emotionally significant event that motivates future-oriented prediction and desire for a resolution (Lehne & Koelsch, 2015). To illustrate this, consider one of Hitchcock’s classic strategies, that is, to give the audience information that the characters lack, for example, by showing a villain hiding a bomb before the protagonists enter the scene. The audience’s affection for the protagonists and the uncertainty regarding whether the bomb will go off create a mix between fear and hope and a motivational tension that keeps viewers focused on the unfolding story events. In this way, suspense taps into processes that cut across the domains of cognition and emotion, which are often described as a dichotomy but actually are intimately interwoven (Pessoa, 2013).

These powerful effects of suspense make it an ideal context for studying engaged audiences. As argued earlier, the fact that the architecture and function of visual and auditory sensory systems are conserved across humans explains why their brains will respond similarly to the same visual and auditory events. Critically, a parallel argument can be made for suspense in that it taps into deeply rooted motivational and social mechanisms, such as the predisposition to empathize with others and fear for their well-being (Frith & Frith, 2012; Olsson, Nearing, & Phelps, 2007; Tamborini, 2011). Therefore, just like processing sensory aspects of a continuous movie will prompt similar processing across viewers, we can also expect that the tension produced by suspense should induce similar brain responses, although this would likely occur in different regions that will be discussed later (Adolphs & Anderson, 2018). However, in contrast to the sensory systems, the brain processes underlying suspense and other hot-cognition phenomena require more complicated conceptual content to become responsive. Movies are well-suited to advance this research because they can effectively evoke suspense in such a way that the time-locked content is identical in its presentation to all audience members, which in turn fulfills the prerequisite for ISC analysis to identify shared audience responses.

The Interface Between Media, Attention, and Motivation

It is clear that when a message can activate fundamental motivational systems that are shared across humans, this will affect how the message is encoded, processed, and stored (Fisher, Keene, Huskey, & Weber, 2018). For instance, the LC4MP (Lang, 2009), which views message processing as a continuously evolving process and is compatible with cognitive neuroscience, emphasizes the importance of motivation in understanding cognitive effects. The LC4MP’s conceptualization of motivation is based on the hypothesized dual motive systems in the brain, which organize appetitive and aversive behaviors (Cacioppo, Gardner, & Berntson, 1999; Lang, 2010). In brief, real-life stimuli that carry affective significance can activate core motive circuits, although it must be noted that this does not necessarily map onto the activity of single brain regions, and the systems are complex in ways that simple labels such as appetitive and aversive tend to mask. Importantly, mediated representations, such as an image showing somebody being attacked, can also engage these systems.

Previous studies from affective neuroscience reveal a prioritized processing of emotional content, which has been interpreted as motivated attention (Schupp, Flaisch, Stockburger, & Junghöfer, 2006; Schupp, Kirmse, Schmälzle, Flaisch, & Renner, 2016). Regionally, these effects have not only been reported for affective-motivational brain systems including mediofrontal regions, subcortical regions, and key hubs in the salience network such as the anterior cingulate cortex and the insula, but also as more distributed enhancements of sensory-perceptual processes (Pessoa, 2018; Schupp et al., 2007). Of note, attention here serves as an umbrella term for a collection of processes that can amplify or sustain responses for visual, auditory, visual, and higher-order representations (Chun, Golomb, & Turk-Browne, 2011; Sara & Bouret, 2012). The notion of motivated attention extends the cognitive notions of automatic/bottom–up or task-based/top–down selective attention in a way that is very compatible with media psychological research and more naturalistic approaches to the phenomenon (Fawcett, Risko, & Kingstone, 2015). Indeed, attention is naturally attracted to motivationally relevant stimuli, like when someone notices a murder on screen despite completing some separate task. However, it will be apparent to media scholars that work using static images and brief sounds to study the attention–motivation interface is less suited for narrative phenomena that build on continuous information. Moreover, although the motivated attention literature has demonstrated that motivationally relevant stimuli have very robust and thus intersubjectively consistent effects on brain activity, there has been no focus on audience-wide effects that are central to mass communication and media research.

The Current Study and Hypothesis

Suspense represents an inherently temporal experience that ebbs and flows throughout a movie as controlled by plot development (Zillmann, 1996). This necessitates a research approach that takes time into account, yet previous work has largely relied on static or retro- and introspective measures (Watt, 1994). The ISC approach has potential to unpack the reception process by providing spatially resolved and temporal information about how movies engage audiences at the neural level, yet theoretical connections with media psychology remain scarce (Dudai, 2012; Hasson et al., 2008; Schmälzle & Grall, in press). Furthermore, previous work on audience-wide ISC in response to motivational messages has focused on comparing different messages but rarely examined dynamic fluctuations of ISC within a message, such as during the more suspenseful or less suspenseful parts of a movie. Finally, although each viewer responds to suspense individually, the key processes should unfold similarly across viewers and promote a shared experience.

On the basis of these considerations, we studied how suspense modulates audience brain responses and how these relate over time to reports of suspense in audiences at large. We used a large dataset in which fMRI had been recorded while participants viewed a short movie by Alfred Hitchcock (Shafto et al., 2014), for which we obtained reports of suspense via continuous response measurement from an independent audience (Biocca, David, & West, 1994). We reasoned that if ISC reflects engagement with strong message content, and suspense similarly activates deeply rooted psychological processes across audience members (Lehne & Koelsch, 2015), then fluctuations in ISC across a movie should be linked to variations of reported suspense. This logic is laid out in Figure 1C. Note that both dynamic ISC as well as the time-varying strength of suspense reports represent group-level metrics; dynamic ISC indexes variations in audience alignment as events unfold, and CRM reports of suspense index the strength of the audience-wide experience at specific moments in the movie.

Accordingly, we hypothesized that dynamic ISC of fMRI data will be associated with continuously reported suspense (H1). We first focused on ISC dynamics throughout the brain (H1a) because the notion of motivated attention, which would comprise visual, auditory, and higher-conceptual activities to process the content, should be reflected in distributed ISC effects. Second, based on work on motivational brain systems, we expected that regions of the salience network, key in attentional prioritization and emotional processes (Menon, 2015; Roy, Shohamy, & Wager, 2012), would exhibit a particularly tight relationship between dynamic ISC and suspense reports at the level of aggregate audiences (H1b).

Sample and Procedure

Inter-Subject Correlation (ISC) Analysis

The analysis was carried out on the brain activity time series extracted from the Shen Parcellation (Shen, Tokoglu, Papademetris, & Constable, 2013), which consists of 268 regions from the entire brain (see Figure 2). From each of these regions, time series were extracted resulting in a matrix of 193 TRs × 268 regions × 494 viewers. This matrix was submitted to ISC analysis (Hasson et al., 2010; Hasson, Nir, Levy, Fuhrmann, & Malach, 2004). In brief, ISC assesses the correlations between fMRI time series from corresponding brain regions across different individuals exposed to the same stimulus (see Figure 1).

Shared Brain Responses During Movie Viewing: Static ISC

Figure 2 shows the results of the ISC analysis computed across the entire movie. As can be seen, movie viewing evokes shared brain responses (ISC). As expected, the similarity of brain activity time series across viewers is strongest in visual and auditory regions with lesser, but significant, correlations in frontal regions (p < 0.01, FDR corrected). The spatial distribution of inter-subject similarity matches with previous reports (Hasson et al., 2004; Lahnakoski et al., 2014; Schmälzle, Häcker, Renner, Honey, & Schupp, 2013) showing that ISC extends into higher-order regions involved in social cognition, salience processing, and executive control. Very similar results emerge when the same analysis is carried out on data from subgroups, such as for younger and older females or males (Figure 2), and the spatial pattern of results is highly similar in each subgroup (spatial correlations all r > 0.96, p < 0.01).

We also conducted ISC analyses on resting-state data in which participants did not view any common stimulus. This analysis confirmed that without the movie acting as an “audience-aligner,” the different brains exhibit no similar processes but idle at their own pace (Figure 2D).

Collective Audience Brain Alignment Over Time: Dynamic ISC

Next, to generate a time-resolved measure of how well the audience’s brain time series become more or less aligned as the movie unfolds, we computed dynamic ISC analysis. Specifically, for each region, we computed an ISC analysis within a given time window, recorded each result, and repeated the procedure with a shifted window. The results of this analysis are illustrated in Figure 3. Figure 3A shows the evolution of dynamic ISC averaged across the brain and Figure 3B focuses on the anterior cingulate cortex, a key node of the salience network.

To demonstrate the stability of this measure, we repeated the analysis for independent subgroups of the 494-person audience. Specifically, we randomly divided the audience into halves and ran dynamic ISCs for each. As shown by the shaded lines in Figure 3, the independent assessments confirm the stability of the results (whole brain: r = 0.9, p = 4.33e-111; ACC: r = 0.85; p = 2.32e-57). We also repeated this computationally intensive procedure via permutations (see supplementary analyses in the reproducibility package: https://github.com/nomcomm/CamCanSuspenseISC_JMP). All analyses confirmed the robustness of the dynamic ISC.

Ratings of Suspense (CRM)

To capture the degree of reported suspense over time, the continuous ratings of suspense were averaged across the group (Figure 4). Reported suspense varies over time, as shown by the 95% confidence interval around the mean for each timepoint. Combined visual inspection of the movie and reported suspense confirmed that the latter mirrors the narrative structure of the movie. For instance, peaks in reported suspense coincide with moments depicting imminent danger to innocent characters.

Relationship Between Dynamic ISC and Reported Suspense

The dynamic ISC analysis yields a time-resolved measure of the degree of collective brain similarity, which we theorized would respond to content-driven elements that promote engagement with the narrative. The CRM captures the extent to which an independent audience reports suspense over time, and we hypothesized that this should track with the collective alignment of brain responses. Supporting our prediction, fluctuations in dynamic ISC strength between the viewers whose brains are scanned are related to reports of experienced suspense in an independent audience, as indexed via cross-correlations. As seen in Figure 5A, this is the case for the whole-brain dynamic ISC; the dynamic ISC measure (averaged across the entire brain) takes a similar overall trajectory to the reported suspense (r = 0.47, p = 4.54e-12).

Whereas Figure 5A represents the dynamic ISC result averaged across the entire brain, regional analyses of dynamic ISC point to some specificity of this effect (Figures 5B–5D). The highest correlations between dynamic ISC and suspense ratings were observed for the mid-cingulate gyrus (r = 0.70, p = 2.21e-30), the angular gyrus (r = 0.71, p = 6.83e-31), and the right lateral prefrontal cortex (r = 0.79, p = 1.22e-42). Correlations between 0.5 < r < 0.7 (p < 0.01) emerged in the anterior cingulate, the bilateral anterior insulae, bilateral frontal cortices, the precuneus, medial prefrontal cortex, and the bilateral angular gyri. These regions are prominent parts of higher-order brain systems such as the saliency and executive networks, which are indicated by the green and blue outlines in Figure 5D, respectively. We confirmed that the reported relationship between dynamic ISC and reported suspense holds after regressing out the potential influence of motion (for both the results across the brain as well as for the regional analyses). We also confirmed these relationships in subsamples of test audiences with varying amounts of motion.

Of note, there are also regions in which dynamic ISC and reported suspense do not track with each other. These include the majority of the visual and auditory systems. Closer inspection revealed that these regions do not just exhibit a fixed level of dynamic ISC throughout the movie, but they show fluctuations over time. However, in this movie, these fluctuations in audience brain alignment do not correlate with the reported suspense experiences.

Integration Between Media Psychology and Neuroscience

The notion that motivated processing of conceptual content is central to media entertainment and strongly modulates cognitive responses has a long history in media psychology, as perhaps most prominently articulated by the LC4MP (Fisher et al., 2018; Lang, 2009). However, testing and advancing these ideas requires zooming in on relevant processes over time with a focus on motivation and affect, which are notoriously difficult to study despite being key modulators of executive processes. It is clear that neuroimaging has potential to advance this research, but prior work in cognitive neuroscience has predominantly focused on abstract cognitive processing and laboratory tasks that bear little resemblance to the powerful effects of media, particularly suspense. The field of affective neuroscience, in turn, has examined how affective images prompt motivated attention, but this work mainly employed static images with explicit perceptual but no narrative content (e.g., depictions of mutilations and erotica), which barely engage higher-order cognitive processing (Bradley, 2009; Lang, Bradley, & Cuthbert, 1997). The current study combines these lines of research in a mutually beneficial way; by using a conceptually powerful stimulus, we can engage higher-order systems involved in narrative processing, social cognition, and emotion in a very controlled manner to study their effects on individual brain activity and whole audiences. Although more work is needed, this approach is suited to study the audience response to dynamic media from an objective, neural perspective, and to link neural effects to content and subsequent responses (Dudai, 2012; Hasson et al., 2008; Schmälzle & Grall, in press).

Possible Generalization to Other Media Phenomena

A follow-up question is whether this approach can be generalized to other phenomena and viewing experiences. In terms of content types and genres, the argument we made for suspense as an “audience-aligner” could also be made for horror, comedy, tragedy, and action. For instance, a scene depicting a character’s sudden death is likely to prompt common responses across viewers no matter the genre. However, movies from different genres will likely recruit a mix of distinct and common brain systems to process the content and implement the motivational response. Our theorizing focuses on the strong audience response to motivationally relevant messages. This approach, to capture the “strength of the grip” (Hasson et al., 2008) a movie has on an audience, can be used across genres and even beyond movie viewing, such as effective speaking and health communication (Imhof et al., 2017; Schmälzle et al., 2015). Translational research has already begun to link neural data to outcomes such as memory, persuasion, and commercial success (Barnett & Cerf, 2017; Chen et al., 2017; Dmochowski et al., 2014; Ki, Kelly, & Parra, 2016; Pei, Schmälzle, Kranzler, O’Donnell, & Falk, 2019).

Strengths and Limitations

This study advances a novel quantitative way to link collective brain responses to media with audience effects, as measured via CRM, extending the seminal work by Lazarsfeld and Stanton on the Program Analyzer into the domain of neuroscientific response measurement (Levy, 1982). This general framework can easily be adapted to other movie genres, to other media types such as spoken or written narratives and music, and also to link audience brain effects back to their elicitors in media content. An additional strength of this study is the large brain imaging sample, which provides robust aggregate measures (Turner, Paul, Miller, & Barbey, 2018).

However, several limitations need to be acknowledged as well. With fMRI measures it is always important to consider subject motion (Power, Schlaggar, & Petersen, 2015), although it is not clear a priori how motion relates to suspense, as it might either cause people to move less owing to absorption or more owing to agitation during a particular scene. Empirically, we find that the relationship between dynamic ISC and suspense holds when controlling for the potential influence of motion, which bolsters our confidence in the data and suggests that synchronous movement does not account for these effects.

It should be noted that the “Bang! You’re Dead” episode has been used in previous studies (Dmochowski, Sajda, Dias, & Parra, 2012; Hasson et al., 2008; Naci, Cusack, Anello, & Owen, 2014), and some reported results are compatible with the notion that the degree of audience brain coupling tracks with reported suspense. The most closely related study is one that links time-resolved EEG-ISC to what is qualitatively described as “emotionally laden” moments (Dmochowski et al., 2012). However, this study used a slightly different version of the movie and a smaller audience. Similar findings have also been reported recently by the CamCAN team itself (Geerligs, Cam-CAN, & Campbell, 2018), who discuss high ISC in relation to “salient events” as defined by qualitative inspection of content. This study uses the same stimulus and dataset but focuses primarily on how age relates to idiosyncratic responding without considering the topic of suspense from a media psychological perspective. Overall, these prior results bolster our interpretation and suggest that the integration of multimodal measures with fine-grained analyses of movie content and experience sampling methods is promising and, in fact, already ongoing (Haufe et al., 2018).

The short duration of the movie in combination with the low temporal resolution of fMRI is a limitation. Although the movie is clearly long enough to present an engaging story, the slow nature of fMRI measurement poses an upper limit for our ability to resolve effects in time. Furthermore, time series analyses often face the issues of nonstationarity and a lack of independence between samples, and these issues become more acute when limited data are available. Conventional approaches to control for these issues are insufficient as they run the risk of interfering with the phenomenon of study. We thus opted to stay close to the measured data and refrain from statistical transformations, but these issues should be kept in mind when interpreting the findings. We also note that future analyses might modify inevitable choices we had to make, such as the window size for dynamic ISC analyses or potential lags regarding the temporal alignment of neural and subjective data. Solutions for overcoming these issues include collecting more data and employing other measures such as EEG (Imhof, Schmälzle, Renner, & Schupp, 2020). Fortunately, several large fMRI initiatives are currently scanning audiences while they are exposed to longer and more varied content (Dubois & Adolphs, 2016; Richardson, Lisandrelli, Riobueno-Naylor, & Saxe, 2018). The move toward large and freely available neuroimaging datasets including movie stimuli will make fMRI data easier to obtain for media scholars and create unique opportunities to expand the current research and address novel questions (Vanderwal et al., 2019). Going forward, it will also be important to integrate these databases with other methods, such as psychophysiology, to triangulate findings (Potter & Bolls, 2012).

Conclusion

To conclude, this study examined how a suspenseful movie evokes similar responses in the brains of audience members. We found that the strength of audience-wide brain similarity varied over time and tracked with reports of suspense. These findings contribute to our understanding of the mechanisms of suspense in the message reception process from content, to audience brains, to subsequent effects on subjective reports.