When Seeing a Dog Activates the Bark
Multisensory Generalization and Distinctiveness Effects
Abstract
The goal of the present study was to find evidence for a multisensory generalization effect (i.e., generalization from one sensory modality to another sensory modality). The authors used an innovative paradigm (adapted from Brunel, Labeye, Lesourd, & Versace, 2009) involving three phases: a learning phase, consisting in the categorization of geometrical shapes, which manipulated the rules of association between shapes and a sound feature, and two test phases. The first of these was designed to examine the priming effect of the geometrical shapes seen in the learning phase on target tones (i.e., priming task), while the aim of the second was to examine the probability of recognizing the previously learned geometrical shapes (i.e., recognition task). When a shape category was mostly presented with a sound during learning, all of the primes (including those not presented with a sound in the learning phase) enhanced target processing compared to a condition in which the primes were mostly seen without a sound during learning. A pattern of results consistent with this initial finding was also observed during recognition, with the participants being unable to pick out the shape seen without a sound during the learning phase. Experiment 1 revealed a multisensory generalization effect across the members of a category when the objects belonging to the same category share the same value on the shape dimension. However, a distinctiveness effect was observed when a salient feature distinguished the objects within the category (Experiment 2a vs. 2b).
References
1986). Varieties of perceptual independence. Psychological Review, 93, 154–179.
(2008). Grounded cognition. Annual Review of Psychology, 59, 617–645. doi: 10.1146/annurev.psych.59.103006.093639
(1995). Mental representations of movements. Brain potentials associated with imagination of hand movements. Electroencephalography and Clinical Neurophysiology, 96, 183–193.
(2007). Hedonic-specific activity in the piriform cortex during odor imagery mimics that during odor perception. Journal of Neurophysiology, 98, 3254–3262. doi: 10.1152/jn.00349.2007
(2009). The sensory nature of episodic memory: Sensory priming due to memory trace activation. Journal of Experimental Psychology: Learning, Memory and Cognition, 35, 1081–1088. doi: 10.1037/a0015537
(2010). The sensory nature of knowledge: Sensory priming effect in semantic categorization. The Quarterly Journal of Experimental Psychology, 63, 955–964. doi: 10.1080/17470210903134369
(2010). Memory and consciousness: Trace distinctiveness in memory retrievals. Consciousness & Cognition, 19, 926–937. doi: 10.1016/j.concog.2010.08.006
(2009). The sensory nature of knowledge generalization vs. specification mechanisms. In , Proceedings of the 31st Annual Conference of the Cognitive Science Society (pp. 2789–2794). Austin, TX: Cognitive Science Society.
(2010). When hearing the bark helps to identify the dog: Semantically-congruent sounds modulate the identification of masked pictures. Cognition, 114, 389–404. doi: 10.1016/j.cognition.2009.10.012
(2001). Category variability, exemplar similarity, and perceptual classification. Memory & Cognition, 29, 1165–1175.
(2010). Look but don’t touch: Tactile disadvantage in processing of modality-specific words. Cognition, 115, 1–9. doi: 10.1016/j.cognition.2009.10.005
(2007). Multisensory similarity and categorization of novel, three-dimensional objects. Neuropsychologia, 45, 484–495. doi: 10.1016/j.neuropsychologia.2006.02.009
(2009). Feature bindings are maintained in visual short-term memory without sustained focused attention. Experimental Psychology, 57, 108–116. doi: 10.1027/1618-3169/a000014
(1999). Auditory-visual integration during multisensory object recognition in humans: A behavior and electrophysiological study. Journal of Cognitive Neuroscience, 11, 473–490.
(2010). Embodiment as a unifying perspective for psychology. Wiley Interdisciplinary Review: Cognitive Science, 1, 586–596. doi: 10.1002/wcs.55
(1994). Influences of categorization on perceptual discrimination. Journal of Experimental Psychology: General, 123, 178–200.
(2000). Unitization during category learning. Journal of Experimental Psychology: Human Perception and Performance, 26, 86–112.
(1998). Reuniting perception and conception. Cognition, 65, 231–262.
(2008). Learning to see and conceive. In , The new cognitive sciences (pp. 163–188). Cambridge, MA: MIT Press.
(2001). Altering object representations through category learning. Cognition, 78, 27–43.
(1991). Perceptual and associative learning. New York, NY: Clarendon Press/Oxford University Press.
(1986). “Schema abstraction” in a multiple-trace model. Psychological Review, 95, 528–551.
(1998). Event files: Evidence for automatic integration of stimulus-response episodes. Visual Cognition, 5, 183–216.
(2010). Featural selective attention, exemplar representation, and the inverse base-rate effect. Psychonomic Bulletin & Review, 17, 637–643. doi: 10.3758/PBR.17.5.637
(2008). Activation and integration of motor components in a short-term priming paradigm. Acta Psychologica, 129, 108–111. doi: 10.1016/j.actpsy.2008.04.010
(2005). The role of multisensory memories in unisensory object discrimination. Cognitive Brain Research, 24, 326–334. doi: 10.1016/j.cogbrainres.2005.02.005
(2002). An instance theory of attention and memory. Psychological Review, 109, 376–400.
(1995). Discrete cortical regions associated with knowledge of color and knowledge of action. Science, 270, 102–105.
(1976). Hearing lips and seeing voices. Nature, 264, 746–748.
(2007). Hemodynamic responses in human multisensory and auditory association cortex to purely visual stimulation. BMC Neuroscience, 8, 14. doi: 10.1186/1471-2202-8-14
(2006). Modeling distinctiveness: Implications for general memory theory. In , Distinctiveness and memory (pp. 27–46). Oxford, NY: Oxford University Press.
(2002). Combining dimensions and features in similarity-based representations. In , Advances in neural information processing systems 15: Proceedings of the 2002 conference (pp. 67–74). Cambridge, MA: MIT Press.
(1986). Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology: General, 115, 39–57.
(1991). Tests of an exemplar model for relating perceptual classification and recognition memory. Journal of Experimental Psychology: Human Perception and Performance, 17, 3–27.
(1992). Similarity scaling and cognitive process models. Annual Review of Psychology, 43, 25–53.
(2003). A hybrid-similarity exemplar model for predicting distinctiveness effects in perceptual old-new recognition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29, 1194–1209.
(2004). Sensorimotor simulations underlie conceptual representations: Modality-specific effects of prior activation. Psychonomic Bulletin & Review, 11, 164–167.
(2010). Integration of perceptual information in word access. The Quarterly Journal of Experimental Psychology, 63, 81–107. doi: 10.1080/17470210902829563
(2011). Visual memory and visual perception: When memory improves visual search. Memory & Cognition, 39, 1094–1102. doi: 10.3758/s13421-011-0075-2
(1975). Family resemblances: Studies in the internal structure of the categories. Cognitive Psychology, 7, 573–605.
(2012). Contingency learning with evaluative stimuli. Experimental Psychology, 59, 175–182. doi: 10.1027/1618-3169/a000141
(2010). Contingency learning and unlearning in the blink of an eye: A resource dependent process. Consciousness & Cognition, 19, 235–250. doi: 10.1016/j.concog.2009.12.016
(2008). Multisensory identification of natural objects in a two-way crossmodal priming paradigm. Experimental Psychology, 55, 121–132. doi: 10.1027/1618-3169.55.2.121
(1987). Toward a universal law of generalization for psychological science. Science, 237, 1317.
(2012). The sensorimotor contribution to implicit memory, familiarity, and recollection. Journal of Experimental Psychology: General, 141, 260–281. doi: 10.1037/a0025658
(1977). Features of similarity. Psychological Review, 84, 327–352.
(2010). The perceptual nature of the cross-modal priming effect: Arguments in favor of a sensory-based conception of memory. Experimental Psychology, 57, 376–382. doi: 10.1027/1618-3169/a000045
(2008). Perceptual processing affects conceptual processing. Cognitive Science, 32, 579–590. doi: 10.1080/03640210802035365
(2009). The contents of long-term memory and the emergence of knowledge. The European Journal of Cognitive Psychology, 21, 522–560. doi: 10.1080/09541440801951844
(2000). Memory’s echo: Vivid remembering reactivates sensory-specific cortex. PNAS, 97, 11125–11129.
(