Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Next article
Original Communication

The Look of Fear from the Eyes Varies with the Dynamic Sequence of Facial Actions

Published Online:January 19, 2016https://doi.org/10.1024/1421-0185/a000166

Abstract

Abstract. Most research on the ability to interpret expressions from the eyes has utilized static information. This research investigates whether the dynamic sequence of facial actions in the eye region influences the judgments of perceivers. Dynamic fear expressions involving the eye region and eyebrows were created which systematically differed in the sequential occurrence of facial actions. Participants rated the intensity of sequential fear expressions, either in addition to a simultaneous, full-blown expression (Experiment 1) or in combination with different levels of eye gaze (Experiment 2). The results showed that the degree of attributed emotion and the appraisal ratings differed as a function of the sequence of facial expressions of fear, with direct gaze resulting in stronger subjective responses. The findings challenge current notions surrounding the study of static facial displays from the eyes and suggest that emotion perception is a dynamic process shaped by the time course of the facial actions of an expression. Possible implications for the field of affective computing and clinical research are discussed.

References

  • Adams, R. B., & Kleck, R. E. (2003). Perceived gaze direction and the processing of facial displays of emotion. Psychological Science, 14, 644–647. doi 10.1046/j.0956–7976.2003.psci_1479.x First citation in articleCrossref, Google Scholar

  • Adams, R. B., & Kleck, R. E. (2005). Effects of direct and averted gaze on the perception of facially communicated emotion. Emotion, 5, 3–11. doi 10.1037/1528-3542.5.1.3 First citation in articleCrossref, Google Scholar

  • Adolphs, R., Gosselin, F., Buchanan, T. W., Tranel, D., Schyns, P., & Damasio, A. R. (2005). A mechanism for impaired fear recognition after amygdala damage. Nature, 433(7021), 68–72. doi 10.1038/nature03086 First citation in articleCrossref, Google Scholar

  • Aue, T., Flykt, A., & Scherer, K. R. (2007). First evidence for differential and sequential efferent effects of stimulus relevance and goal conduciveness appraisal. Biological Psychology, 74, 347–357. doi 10.1016/j.biopsycho.2006.09.001 First citation in articleCrossref, Google Scholar

  • Back, E., Ropar, D., & Mitchell, P. (2007). Do the eyes have it? Inferring mental states from animated faces in autism. Child Development, 78, 397–411. doi 10.1111/j.1467-8624.2007.01005.x First citation in articleCrossref, Google Scholar

  • Baron-Cohen, S., Campbell, R., Karmiloff-Smith, A., Grant, J., & Walker, J. (1995). Are children with autism blind to the mentalistic significance of the eyes? British Journal of Developmental Psychology, 13, 379–398. doi 10.1111/j.2044-835X.1995.tb00687.x First citation in articleCrossref, Google Scholar

  • Baron-Cohen, S., & Cross, P. (1992). Reading the eyes: Evidence for the role of perception in the development of a theory of mind. Mind and Language, 7, 172–186. doi 10.1111/j.1468-0017.1992.tb00203.x First citation in articleCrossref, Google Scholar

  • Baron-Cohen, S., Golan, O., & Ashwin, E. (2009). Can emotion recognition be taught to children with autism spectrum conditions? Philosophical Transactions of the Philosophical Society B, 364, 3567–3574. doi 10.1098/rstb.2009.0191 First citation in articleCrossref, Google Scholar

  • Baron-Cohen, S., Jolliffe, T., Mortimore, C., & Robertson, M. (1997). Another advanced test of theory of mind: Evidence from very high functioning adults with autism or Asperger syndrome. Journal of Child Psychology and Psychiatry, 38, 813–822. doi 10.1111/j.1469-7610.1997.tb01599.x First citation in articleCrossref, Google Scholar

  • Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y., & Plumb, I. (2001). The Reading the Mind in the Eyes Test revised version: A study with normal adults, and adults with Asperger syndrome or high-functioning autism. Journal of Child Psychology and Psychiatry, 42, 241–251. doi 10.1111/1469-7610.00715 First citation in articleCrossref, Google Scholar

  • Bindemann, M., Burton, A. M., & Langton, S. R. H. (2008). How do eye gaze and facial expression interact? Visual Cognition, 16, 708–733. doi 10.1080/13506280701269318 First citation in articleCrossref, Google Scholar

  • Bombari, D., Schmid, P. C., Schmid Mast, M., Birri, S., Mast, F. W., & Lobmaier, J. S. (2013). Emotion recognition: The role of featural and configural face information. Quarterly Journal of Experimental Psychology, 66, 2426–2442. doi 10.1080/17470218.2013.789065 First citation in articleCrossref, Google Scholar

  • Calvo, M. G., & Fernández-Martin, A. (2013). Can the eyes reveal a person’s emotions? Biasing role of the mouth expression. Motivation and Emotion, 37, 202–211. doi 10.1007/s11031-012-9298-1 First citation in articleCrossref, Google Scholar

  • Delplanque, S., Grandjean, D., Chrea, C., Coppin, G., Aymard, L., Cayeux, I., ... & Scherer, K. R. (2009). Sequential unfolding of novelty and pleasantness appraisals of odors: Evidence from facial electromyography and autonomic reactions. Emotion, 9, 316–328. doi 10.1037/a0015369 First citation in articleCrossref, Google Scholar

  • Eisenbarth, H., & Alpers, G. W. (2011). Happy mouth and sad eyes: Scanning emotional facial expressions. Emotion, 11, 860–865. doi 10.1037/a0022758 First citation in articleCrossref, Google Scholar

  • Ekman, P. (1979). About brows: Emotional and conversational signals. In M. von CranachK. FoppaW. LepeniesD. PloogEds., Human ethology: Claims and limits of a new discipline (pp. 169–202). Cambridge, UK: Cambridge University Press. First citation in articleGoogle Scholar

  • Ekman, P. (1989). The argument and evidence about universals in facial expressions of emotion. In H. WagnerA. MansteadEds., Handbook of social psychophysiology (pp. 143–164). Chichester, UK: Wiley. First citation in articleGoogle Scholar

  • Ekman, P. (1992). An argument for basic emotions. Cognition and Emotion, 6, 169–200. doi 10.1080/02699939208411068 First citation in articleCrossref, Google Scholar

  • Ekman, P. (2003). Darwin, deception and facial expression. Annals of the New York Academy of Sciences, 1000, 205–221. doi 10.1196/annals.1280.010 First citation in articleCrossref, Google Scholar

  • Ekman, P., & Friesen, W. V. (1982). Felt, false and miserable smiles. Journal of Nonverbal Behavior, 6, 238–258. doi 10.1007/ BF00987191 First citation in articleCrossref, Google Scholar

  • Ekman, P., Friesen, W. V., & Hager, J. C. (2002). The Facial Action Coding System (2nd ed.). Salt Lake City, UT: Research Nexus eBook. First citation in articleGoogle Scholar

  • Ellsworth, P. C., & Scherer, K. R. (2003). Appraisal processes in emotion. In R. J. DavidsonK. R. SchererH. H. GoldsmithEds., Handbook of affective sciences (pp. 572–595). New York: Oxford University Press. First citation in articleGoogle Scholar

  • Fiorentini, C., Schmidt, S., & Viviani, P. (2012). The identification of unfolding facial expressions. Perception, 41, 532–555. doi 10.1068/p7052 First citation in articleCrossref, Google Scholar

  • Gepner, B., Deruelle, C., & Grynfeltt, S. (2001). Motion and emotion: A novel approach to the study of face processing by young autistic children. Journal of Autism and Developmental Disorders, 31, 37–45. doi 10.1023/A:1005609629218 First citation in articleCrossref, Google Scholar

  • Graham, R., & LaBar, K. (2007). Garner interference reveals dependencies between emotional expression and gaze in face perception. Emotion, 7, 296–313. doi 10.1037/1528-3542.7.2.296 First citation in articleCrossref, Google Scholar

  • Hobson, R. P. (1986). The autistic child’s appraisal of expressions of emotion. Journal of Child Psychology and Psychiatry, 27, 321–342. doi 10.1111/j.1469-7610.1986.tb01836.x First citation in articleCrossref, Google Scholar

  • Hopkins, I. M., Gower, M. W., Perez, T. A., Smith, D. S., Amthor, F. R., Wimsatt, F. C., & Biasini, F. J. (2011). Avatar assistant: Improving social skills in students with an ASD through a computer-based intervention. Journal of Autism and Developmental Disorders, 41, 1543–1555. doi 10.1007/s10803-011-1179-z First citation in articleCrossref, Google Scholar

  • Jack, R. E., Garrod, O. G. B., & Schyns, P. G. (2014). Dynamic facial expressions of emotions transmit an evolving hierarchy of signals over time. Current Biology, 24, 187–192. doi 10.1016/j.cub. 2013.11.064 First citation in articleCrossref, Google Scholar

  • Johnston, L., Miles, L., & McKinlay, A. (2008). A critical review of the Eyes Test as a measure of social-cognitive impairment. Australian Journal of Psychology, 60, 135–141. doi 10.1080/ 00049530701449521 First citation in articleCrossref, Google Scholar

  • Joseph, R. M., & Tanaka, J. (2003). Holistic and part-based face recognition in children with autism. Journal of Child Psychology and Psychiatry, 44, 529–542. doi 10.1111/1469-7610.00142 First citation in articleCrossref, Google Scholar

  • Krumhuber, E. G., Kappas, A., & Manstead, A. S. R. (2013). Effects of dynamic aspects of facial expressions: A review. Emotion Review, 5, 41–46. doi 10.1177/1754073912451349 First citation in articleCrossref, Google Scholar

  • Krumhuber, E., & Scherer, K. R. (2011). Affect bursts: Dynamic patterns of facial expression. Emotion, 11, 825–841. doi 10.1037/a0023856 First citation in articleCrossref, Google Scholar

  • Krumhuber, E., Tamarit, L., Roesch, E. B., & Scherer, K. R. (2012). FACSGen 2.0 animation software: Generating 3D FACS-valid facial expressions for emotion research. Emotion, 12, 351–363. doi 10.1037/a0026632 First citation in articleCrossref, Google Scholar

  • Lanctôt, N., & Hess, U. (2007). The timing of appraisals. Emotion, 7, 207–212. doi 10.1037/1528-3542.7.1.207 First citation in articleCrossref, Google Scholar

  • Langton, S. R. H, Watt, R. J., & Bruce, V. (2000). Do the eyes have it? Cues to the direction of social attention. Trends in Cognitive Sciences, 4(2), 50–59. doi 10.1016/S1364-6613(99)01436-9 First citation in articleCrossref, Google Scholar

  • Malatesta, L., Raouzaiou, A., Karpouzis, K., & Kollias, S. (2009). Towards modeling embodied conversational agent character profiles using appraisal theory predictions in expression synthesis. Applied Intelligence, 30, 58–64. doi 10.1007/s10489-007-0076-9 First citation in articleCrossref, Google Scholar

  • Milders, M., Hietanen, J. K., Leppänen, J. M., & Braun, M. (2011). Detection of emotional faces is modulated by the direction of eye gaze. Emotion, 11, 1456–1461. doi 10.1037/a0022901 First citation in articleCrossref, Google Scholar

  • Moore, D. G. (2001). Reassessing emotion recognition performance in people with mental retardation: A review. American Journal on Mental Retardation, 106, 481–502. First citation in articleCrossref, Google Scholar

  • Morris, J. S., deBonis, M., & Dolan, R. J. (2002). Human amygdala responses to fearful eyes. NeuroImage, 17, 214–222. doi 10.1006/nimg.2002.1220 First citation in articleCrossref, Google Scholar

  • N’Diaye, K., Sander, D., & Vuilleumier, P. (2009). Self-relevance processing in the human amygdala: Gaze direction, facial expression and emotion intensity. Emotion, 9, 798–806. doi 10.1037/a0017845 First citation in articleCrossref, Google Scholar

  • Niedenthal, P. M., Mermillod, M., Maringer, M., & Hess, U. (2010). The simulation of smiles (SIMS) model: Embodied simulation and the meaning of facial expression. Behavioral and Brain Sciences, 33, 417–433. doi 10.1017/S0140525X10000865 First citation in articleCrossref, Google Scholar

  • Niewiadomski, R., Hyniewska, S., & Pelachaud, C. (2009a). Evaluation of multimodal sequential expressions of emotions in ECA. In J. CohnA. NijholtM. PanticEds., Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction – ACII 2009 (pp. 635–641). Los Alamitos, CA: IEEE Computer Society Press. First citation in articleGoogle Scholar

  • Niewiadomski, R., Hyniewska, S., & Pelachaud, C. (2009b). Modeling emotional expressions as sequences of behaviors. In Z. RuttkayM. KippA. NijholtH. H. VilhjálmssonEds., Lecture Notes in Computer Science: Vol. 5773. Intelligent Virtual Agents: 9th International Conference, IVA 2009, Amsterdam, The Netherlands, September 14—16, 2009 Proceedings (pp. 316–322). Berlin, Germany: Springer. doi 10.1007/978-3-642-04380-2_34 First citation in articleGoogle Scholar

  • Nishio, S.Koyama, K.Nakmura, T. (1998). Temporal differences in eye and mouth movements classifying facial expressions of smiles. In Proceedings of the 3rd International Conference on Face & Gesture Recognition – FG ’98 (pp. 206–213). Nara, Japan: IEEE Computer Society Press. doi 10.1109/AFGR.1998.670950 First citation in articleGoogle Scholar

  • Nummenmaa, L., & Calder, A. J. (2009). Neural mechanisms of social attention. Trends in Cognitive Sciences, 13, 135–143. doi 10.1016/j.tics.2008.12.006 First citation in articleCrossref, Google Scholar

  • Oosterhof, N. N., & Todorov, A. (2008). The functional basis of face evaluation. Proceedings of the National Academy of Sciences of the USA, 105, 11087–11092. doi 10.1073/pnas.0805664105 First citation in articleCrossref, Google Scholar

  • Paleari, M., Grizard, A., & Lisetti, C. (2007). Adapting psychologically grounded facial emotional expressions to different anthropomorphic embodiment platforms. In D. WilsonG. SutcliffeEds., Proceedings of the 20th Florida Artificial Intelligence Research Society Conference (FLAIRS 2007) (pp. 565–570). Menlo Park, CA: AAAI Press. First citation in articleGoogle Scholar

  • Pelphrey, K. A., Sasson, N. J., Reznick, S. J., Paul, G., Goldman, B. D., & Piven, J. (2002). Visual scanning of faces in autism. Journal of Autism and Developmental Disorders, 32, 249–261. doi 10.1023/A:1016374617369 First citation in articleCrossref, Google Scholar

  • Pilowsky, I., Thornton, M., & Stokes, B. (1986). Towards the quantification of facial expressions with the use of a mathematic model of the face. In H. EllisM. A. JeevesF. NewcombeA. YoungEds., Aspects of face processing (pp. 340–348). Dordrecht, The Netherlands: Martinus Nijhoff. First citation in articleGoogle Scholar

  • Sander, D., Grandjean, D., Kaiser, S., Wehrle, T., & Scherer, K. R. (2007). Interaction effects of perceived gaze direction and dynamic facial expression: Evidence for appraisal theories of emotion. European Journal of Cognitive Psychology, 19, 470–480. doi 10.1080/09541440600757426 First citation in articleCrossref, Google Scholar

  • Scherer, K. R. (1984). On the nature and function of emotion: A component process approach. In K. R. SchererP. EkmanEds., Approaches to emotion (pp. 293–317). Hillsdale, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Scherer, K. R. (1999). On the sequential nature of appraisal processes: Indirect evidence from a recognition task. Cognition and Emotion, 13, 763–793. doi 10.1080/026999399379078 First citation in articleCrossref, Google Scholar

  • Scherer, K. R. (2001). Appraisal considered as a process of multilevel sequential checking. In K. R. SchererA. SchorrT. JohnstoneEds., Appraisal processes in emotion: Theory, methods, research (pp. 92–120). New York: Oxford University Press. First citation in articleGoogle Scholar

  • Scherer, K. R. (2009). The dynamic architecture of emotion: Evidence for the component process model. Cognition and Emotion, 23, 1307–1351. doi 10.1080/02699930902928969 First citation in articleCrossref, Google Scholar

  • Scherer, K. R., & Ellgring, H. (2007). Are facial expressions of emotion produced by categorical affect programs or dynamically driven by appraisal? Emotion, 7, 113–130. doi 10.1037/1528-3542.7.1.113 First citation in articleCrossref, Google Scholar

  • Senju, A., & Hasegawa, T. (2005). Direct gaze captures visuospatial attention. Visual Cognition, 12, 127–144. doi 10.1080/ 13506280444000157 First citation in articleCrossref, Google Scholar

  • Smith, M. L., Cottrell, G., Gosselin, F., & Schyns, P. G. (2005). Transmitting and decoding facial expressions of emotions. Psychological Science, 16, 184–189. doi 10.1111/j.0956-7976.2005.00801.x First citation in articleCrossref, Google Scholar

  • Smith, C. A., & Scott, H. S. (1997). A componential approach to the meaning of facial expressions. In J. A. RussellJ. M. Fernàndez-DolsEds., The psychology of facial expression (pp. 229–254). Cambridge, UK: Cambridge University Press. First citation in articleGoogle Scholar

  • Uono, S., Sato, W., & Toichi, M. (2009). Dynamic fearful expressions enhance gaze-triggered attention orienting in high and low anxiety individuals. Social Behavior and Personality, 37, 1313–1326. doi 10.2224/sbp.2009.37.10.1313 First citation in articleCrossref, Google Scholar

  • Vuilleumier, P. (2005). Staring fear in the face. Nature, 433, 22–23. doi 10.1038/433022a First citation in articleCrossref, Google Scholar

  • Wehrle, T., Kaiser, S., Schmidt, S., & Scherer, K. R. (2000). Studying the dynamics of emotional expression using synthesized facial muscle movements. Journal of Personality and Social Psychology, 78, 105–119. doi 10.1037/0022-3514.78.1.105 First citation in articleCrossref, Google Scholar

  • Whalen P. J., Kagan J., Cook, R. G., Davis, F. C., Kim, H., Polis, S., ... & Johnstone, T. (2004). Human amygdala responsivity to masked fearful eye whites. Science, 30, 5704–2061. doi 10.1126/science.1103617 First citation in articleGoogle Scholar

  • Wicker, B., Perrett, D. I., Baron-Cohen, S., & Decety, J. (2003). Being the target of another’s emotion: A PET study. Neuropsychologia, 41, 139–146. doi 10.1016/S0028-3932(02)00144-6 First citation in articleCrossref, Google Scholar

  • With, S., & Kaiser, S. (2011). Sequential patterning of facial actions in the production and perception of emotional expressions. Swiss Journal of Psychology, 70, 241–252. doi 10.1024/1421-0185/a000062 First citation in articleLink, Google Scholar