Lower Pitch Is Larger, Yet Falling Pitches Shrink
Interaction of Pitch Change and Size Change in Speeded Discrimination
Abstract
Experiments using diverse paradigms, including speeded discrimination, indicate that pitch and visually-perceived size interact perceptually, and that higher pitch is congruent with smaller size. While nearly all of these studies used static stimuli, here we examine the interaction of dynamic pitch and dynamic size, using Garner’s speeded discrimination paradigm. Experiment 1 examined the interaction of continuous rise/fall in pitch and increase/decrease in object size. Experiment 2 examined the interaction of static pitch and size (steady high/low pitches and large/small visual objects), using an identical procedure. Results indicate that static and dynamic auditory and visual stimuli interact in opposite ways. While for static stimuli (Experiment 2), higher pitch is congruent with smaller size (as suggested by earlier work), for dynamic stimuli (Experiment 1), ascending pitch is congruent with growing size, and descending pitch with shrinking size. In addition, while static stimuli (Experiment 2) exhibit both congruence and Garner effects, dynamic stimuli (Experiment 1) present congruence effects without Garner interference, a pattern that is not consistent with prevalent interpretations of Garner’s paradigm. Our interpretation of these results focuses on effects of within-trial changes on processing in dynamic tasks and on the association of changes in apparent size with implied changes in distance. Results suggest that static and dynamic stimuli can differ substantially in their cross-modal mappings, and may rely on different processing mechanisms.
References
2003). Motion onset captures attention. Psychological Science, 14, 427–432.
(1995). Congruence effects in classifying auditory stimuli: A review and a model. In , Fechner Day ‘95, Proceedings of the Eleventh Annual Meeting of the International Society for Psychophysics (pp. 145–150). Cassis, France: International Society for Psychophysics.
(1995). Visual-auditory interaction in speeded classification: Role of stimulus difference. Perception & Psychophysics, 57, 1151–1162.
(2009). Different time courses of Stroop and Garner effects in perception – An event-related potentials study. NeuroImage, 45, 1272–1288.
(2005). Is direction position? Position- and direction-based correspondence effects in tasks with moving stimuli. The Quarterly Journal of Experimental Psychology, 58a, 467–506.
(2013). How pitch and loudness shape musical space and motion: New findings and persisting questions. In , The Psychology of Music in Multimedia (pp. 161–187). Oxford, UK: Oxford University Press.
(in press ). Which way is more? Pitch, parametric scales, and the intricacies of cross-domain magnitude relationships. In , Festschrift for Eugene Narmour. New York, NY: Pendragon.2006). How music moves: Musical parameters and images of motion. Music Perception, 23, 221–247.
(2008). Louder is higher: Cross-modal interaction of loudness change and vertical motion in speeded classification. In , Proceedings of the 10th International Conference on Music Perception & Cognition, (ICMPC10). Adelaide, Australia: Causal Productions.
(2012 June) Garner’s paradigm and audiovisual correspondence in dynamic stimuli: Pitch and vertical direction. Paper presented at the International Multisensory Forum 2012 (IMRF 2012), Oxford, UK: IMRF. Abstract published in Seeing and Perceiving, (2012), 25, 70–71.
(2010). Natural cross-modal mappings between visual and auditory features. Journal of Vision, 10, 1–12.
(1984). Representational momentum. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10, 126–132.
(2006). Multisensory synesthetic interactions in the speeded classification of visual size. Perception & Psychophysics, 68, 1191–1203.
(1974). The processing of information and structure. Potomac, MD: Erlbaum.
(1987). Explorations of representational momentum. Cognitive Psychology, 19, 369–401.
(2008). Formal congruence between TELOP patterns and sound effects. Music Perception, 25, 429–448.
(2002). Hearing visual motion in depth. Nature, 416, 172–174.
(2004). Changing pitch induced visual motion illusion. Current Biology, 14, R990–R991.
(2000). Synesthesia. In , Varieties of anomalous experience: Phenomenological and scientific foundations (pp. 121–149). Washington, DC: American Psychological Association.
(2004). Cross-modal interactions in speeded classification. In , Handbook of Multisensory Processes (pp. 85–106). Cambridge, MA: MIT Press.
(1987). Perceiving similarity and comprehending metaphor. Monographs of the Society for Research in Child Development, 52, (No. 215).
(2000). Cross-modal interaction between vision and touch: The role of synesthetic correspondence. Perception, 29, 745–754.
(1990). Processes underlying dimensional interactions: Correspondences between linguistic and nonlinguistic dimensions. Memory & Cognition, 18, 477–495.
(1993). Early-holistic processing or dimensional similarity? Journal of Experimental Psychology: Human Perception and Performance, 19, 1114–1120.
(1987). Interaction between synesthetically corresponding dimensions. Journal of Experimental Psychology: General, 116, 323–336.
(2004). Do small white balls squeak? Pitch-object correspondences in young children. Cognitive, Affective, & Behavioral Neuroscience, 4, 133–136.
(1977). On the occurrence and significance of motivation-structural rules in some bird and mammal sounds. American Naturalist, 111, 855–869.
(1998). Perceptual bias for rising tones. Nature, 395, 123–124.
(1996). The Doppler illusion: The influence of dynamic intensity change on perceived pitch. Journal of Experimental Psychology: Human Perception and Performance, 22, 970–985.
(1994). The frequency code underlies the sound-symbolic use of voice pitch. In , Sound symbolism (pp. 325–347). Cambridge, UK: Cambridge University Press.
(2002). Garner and congruency effects in the speeded classification of bimodal signals. Journal of Experimental Psychology: Human Perception & Performance, 28, 755–775.
(1983). Global and local precedence: Selective attention in form and motion perception. Journal of Experimental Psychology: General, 112, 516–540.
(1973). Stimulus configuration in selective attention tasks. Perception & Psychophysics, 14, 565–569.
(1989). Attention and object perception. In , Object perception: Structure and process (pp. 53–89). Hillsdale, NJ: Erlbaum.
(2006). Dominance and the evolution of sexual dimorphism in human voice pitch. Evolution and Human Behavior, 27, 283–296.
(2003). Vocal communication of emotion: A review of research paradigms. Speech Communication, 40, 227–256.
(1999). Collisions between moving visual targets: What controls alternative ways of seeing an ambiguous display? Perception, 28, 415–432.
(1997). Sound alters visual motion perception. Nature, 385, 308.
(2001). Sensory modalities are not separate modalities: Plasticity and interactions. Current Opinion in Neurobiology, 11, 505–509.
(2011). Crossmodal correspondences: A tutorial review. Attention, Perception & Psychophysics, 43, 971–995.
(2003). Precise and full-range determination of two-dimensional equal loudness contours. Geneva, Switzerland: International Organization for Standardization (ISO).
(1995). Stroop can occur without Garner interference: Strategic and mandatory influences in multidimensional stimuli. Perception & Psychophysics, 57, 379–392.
(2000). Pitch and pitch change interact in auditory displays. Journal of Experimental Psychology: Applied, 6, 15–30.
(1984). Stroop interference based on the synaesthetic qualities of auditory pitch. Perception, 13, 75–81.
(2012). A common scheme for cross-sensory correspondences across stimulus domains. Perception, 41, 1186–92.
(2003). A theory of magnitude: Common cortical metrices of time, space and quality. Trends in Cognitive Sciences, 7, 483–488.
(