Sound Properties Associated With Equiluminant Colours

In: Multisensory Research
View More View Less
  • 1 School of Psychology, University of Sussex, Brighton, UK
  • | 2 Sackler Centre for Consciousness Science, University of Sussex, Brighton, UK
  • | 3 Allgemeine Psychologie, Justus-Liebig-Universität Gießen, Gießen, Germany

Purchase instant access (PDF download and unlimited online access):

€29.95$34.95

There is a widespread tendency to associate certain properties of sound with those of colour (e.g., higher pitches with lighter colours). Yet it is an open question how sound influences chroma or hue when properly controlling for lightness. To examine this, we asked participants to adjust physically equiluminant colours until they ‘went best’ with certain sounds. For pure tones, complex sine waves and vocal timbres, increases in frequency were associated with increases in chroma. Increasing the loudness of pure tones also increased chroma. Hue associations varied depending on the type of stimuli. In stimuli that involved only limited bands of frequencies (pure tones, vocal timbres), frequency correlated with hue, such that low frequencies gave blue hues and progressed to yellow hues at 800 Hz. Increasing the loudness of a pure tone was also associated with a shift from blue to yellow. However, for complex sounds that share the same bandwidth of frequencies (100–3200 Hz) but that vary in terms of which frequencies have the most power, all stimuli were associated with yellow hues. This suggests that the presence of high frequencies (above 800 Hz) consistently yields yellow hues. Overall we conclude that while pitch–chroma associations appear to flexibly re-apply themselves across a variety of contexts, frequencies above 800 Hz appear to produce yellow hues irrespective of context. These findings reveal new sound–colour correspondences previously obscured through not controlling for lightness. Findings are discussed in relation to understanding the underlying rules of cross-modal correspondences, synaesthesia, and optimising the sensory substitution of visual information through sound.

  • Afra P., Funke M., Matsuo F. (2009). Acquired auditory-visual synesthesia: a window to early cross-modal sensory interactions, Psychol. Res. Behav. Manag. 2, 3137.

    • Search Google Scholar
    • Export Citation
  • Álvaro L., Moreira H., Lillo J., Franklin A. (2015). Color preference in red–green dichromats, Proc. Natl Acad Sci. 112, 93169321.

  • Ayama M., Ikeda M. (1998). Brightness-to-luminance ratio of colored light in the entire chromaticity diagram, Color Res. Appl. 23, 274287.

    • Search Google Scholar
    • Export Citation
  • Batschelet E. (1981). Circular Statistics in Biology. Academic Press, London, UK.

  • Berens P. (2009). CircStat: a MATLAB toolbox for circular statistics, J. Stat. Softw. 31, 121.

  • Bien N., Ten Oever S., Goebel R., Sack A. T. (2012). The sound of size: crossmodal binding in pitch-size synesthesia: a combined TMS, EEG and psychophysics study, Neuroimage 59, 663672.

    • Search Google Scholar
    • Export Citation
  • Boersma P., Weenink D. (2012). Praat: doing phonetics by computer. Retrieved from http://www.praat.org/.

  • Bond B., Stevens S. (1969). Cross-modality matching of brightness to loudness by 5-year-olds, Percept. Psychophys. 6, 337339.

  • Braaten R. (1993). Synesthetic correspondence between visual location and auditory pitch in infants. Paper presented at the 34th Annual Meeting of the Psychonomic Society, Washington, DC, USA.

  • Brainard D. H. (1997). The psychophysics toolbox, Spat. Vis. 10, 433436.

  • Cohen Kadosh R., Terhune D. B. (2012). Redefining synaesthesia? Br. J. Psychol. 103, 2023.

  • Deroy O., Spence C. (2013). Why we are not all synesthetes (not even weakly so), Psychonom. Bull. Rev. 20, 643664.

  • Deroy O., Fasiello I., Hayward V., Auvray M. (2016). Differentiated audio-tactile correspondences in sighted and blind individuals, J. Exp. Psychol. Hum. Percept. Perform. 42, 12041214.

    • Search Google Scholar
    • Export Citation
  • Dolscheid S., Shayan S., Majid A., Casasanto D. (2013). The thickness of musical pitch psychophysical evidence for linguistic relativity, Psychol. Sci. 24, 613621.

    • Search Google Scholar
    • Export Citation
  • Eagleman D. M. (2012). Synaesthesia in its protean guises, Br. J. Psychol. 103, 1619.

  • Esterman M., Verstynen T., Ivry R. B., Robertson L. C. (2006). Coming unbound: disrupting automatic integration of synesthetic color and graphemes by transcranial magnetic stimulation of the right parietal lobe, J. Cogn. Neurosci. 18, 15701576.

    • Search Google Scholar
    • Export Citation
  • Evans K. K., Treisman A. (2010). Natural cross-modal mappings between visual and auditory features, J. Vis. 10, 6. DOI:10.1167/10.1.6.

  • Fairchild M. D. (1998). Color Appearance Models. Addison Wesley Longman, Reading, MA, USA.

  • Fisher R. A. (1921). On the ‘probable error’ of a coefficient of correlation deduced from a small sample, Metron 1, 332.

  • Fitch W. T. (1997). Vocal tract length and formant frequency dispersion correlate with body size in rhesus macaques, J. Acoust. Soc. Am. 102, 12131222.

    • Search Google Scholar
    • Export Citation
  • Fletcher H., Munson W. A. (1933). Loudness, its definition, measurement and calculation, Bell Syst. Techn. J. 12, 377430.

  • Forder L., He X., Witzel C., Franklin A. (2014). Speakers of different colour lexicons differ only in post-perceptual processing of colour, Perception (ECVP Abstract Suppl.) 43, 33.

    • Search Google Scholar
    • Export Citation
  • Fryer L., Freeman J., Pring L. (2014). Touching words is not enough: how visual experience influences haptic–auditory associations in the ‘Bouba–Kiki’ effect, Cognition 132, 164173.

    • Search Google Scholar
    • Export Citation
  • Giannakis K. (2001). Sound mosaics: a graphical user interface for sound synthesis based on audio-visual associations. Doctoral dissertation, Middlesex University, UK. Retrieved from http://eprints.mdx.ac.uk/6634/1/Konstantinos-sound_mosaics.phd.pdf.

  • Goodyear B. G., Menon R. S. (1998). Effect of luminance contrast on BOLD fMRI response in human primary visual areas, J. Neurophysiol. 79, 22042207.

    • Search Google Scholar
    • Export Citation
  • Hamilton-Fletcher G., Ward J. (2013). Representing colour through hearing and touch in sensory substitution devices, Multisens. Res. 26, 503532.

    • Search Google Scholar
    • Export Citation
  • Hamilton-Fletcher G., Wright T. D., Ward J. (2016a). Cross-modal correspondences enhance performance on a colour-to-sound sensory substitution device, Multisens. Res. 29, 337363.

    • Search Google Scholar
    • Export Citation
  • Hamilton-Fletcher G., Mengucci M., Medeiros F. (2016b). Synaestheatre: sonification of coloured objects in space, in: Proceedings of the International Conference on Live Interfaces 2016, Sussex, Brighton, UK, pp. 252–256.

  • Hamilton-Fletcher G., Obrist M., Watten P., Mengucci M., Ward J. (2016c). ‘I always wanted to see the night sky’: blind user preferences for sensory substitution devices, in: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, CA, USA, pp. 2162–2174.

  • Hupé J. M., Dojat M. (2015). A critical review of the neuroimaging literature on synesthesia, Front. Hum. Neurosci. 9, 103. DOI:10.3389/fnhum.2015.00103.

    • Search Google Scholar
    • Export Citation
  • ISO (2003). Acoustics — Normal Equal-Loudness Level Contours. International Organisation for Standardisation, Geneva, Switzerland.

  • Jewanski J. (2010). Color-tone analogies: a systematic presentation of the principles of correspondence, in: Audiovisuology: Compendium. A Multidisciplinary Survey of Audiovisual Culture, Daniels D., Naumann S., Thoben J. (Eds), pp.  338347. König, Köln, Germany.

    • Search Google Scholar
    • Export Citation
  • Jonas C., Spiller M. J., Hibbard P. (in press). Summation of visual attributes in auditory-visual crossmodal correspondences, Psychonom. Bull. Rev.

  • Ludwig V. U., Simner J. (2013). What colour does that feel? Tactile-visual mapping and the development of cross-modality, Cortex 49, 10891099.

    • Search Google Scholar
    • Export Citation
  • Ludwig V. U., Adachi I., Matsuzawa T. (2011). Visuoauditory mappings between high luminance and high pitch are shared by chimpanzees (Pan troglodytes) and humans, Proc. Natl. Acad. Sci. 108, 2066120665.

    • Search Google Scholar
    • Export Citation
  • Marks L. E. (1974). On associations of light and sound: the mediation of brightness, pitch, and loudness, Am. J. Psychol. 87, 173188.

    • Search Google Scholar
    • Export Citation
  • Marks L. E. (1987). On cross-modal similarity: auditory-visual interactions in speeded discrimination, J. Exp. Psychol. Hum. Percept. Perform. 13, 384394.

    • Search Google Scholar
    • Export Citation
  • Martino G., Marks L. E. (1999). Perceptual and linguistic interactions in speeded classification: tests of the semantic coding hypothesis, Perception 28, 903923.

    • Search Google Scholar
    • Export Citation
  • Melara R. D. (1989). Dimensional interaction between color and pitch, J. Exp. Psychol. Hum. Percept. Perform. 15, 6979.

  • Mondloch C. J., Maurer D. (2004). Do small white balls squeak? Pitch-object correspondences in young children, Cogn. Affect. Behav. Neurosci. 4, 133136.

    • Search Google Scholar
    • Export Citation
  • Moos A., Smith R., Miller S. R., Simmons D. R. (2014). Cross-modal associations in synaesthesia: vowel colours in the ear of the beholder, Iperception 5, 132142.

    • Search Google Scholar
    • Export Citation
  • Muggleton N., Tsakanikos E., Walsh V., Ward J. (2007). Disruption of synaesthesia following TMS of the right posterior parietal cortex, Neuropsychologia 45, 15821585.

    • Search Google Scholar
    • Export Citation
  • Mulert C., Jäger L., Propp S., Karch S., Störmann S., Pogarell O., Möller H.-J., Hegerl U. (2005). Sound level dependence of the primary auditory cortex: simultaneous measurement with 61-channel EEG and fMRI, Neuroimage 28, 4958.

    • Search Google Scholar
    • Export Citation
  • Nayatani Y. (1998). A colorimetric explanation of the Helmholtz–Kohlrausch effect, Color Res. Appl. 23, 374378.

  • Newton I. (1979 [1706]). Opticks: or a Treatise of the Reflections, Refractions, Inflections and Colours of Light. Courier Dover Publications, New York, NY, USA.

    • Search Google Scholar
    • Export Citation
  • Novich S., Cheng S., Eagleman D. M. (2011). Is synaesthesia one condition or many? A large-scale analysis reveals subgroups, J. Neuropsychol. 5, 353371.

    • Search Google Scholar
    • Export Citation
  • Orlandatou K. (2012). The role of pitch and timbre in the synaesthetic experience, in: Proceedings of the 12th International Conference on Music Perception and Cognition and the 8th Triennial Conference of the European Society for the Cognitive Sciences of Music, Thessaloniki, Greece, pp. 751–758.

  • Palmer S. E., Schloss K. B., Xu Z., Prado-León L. R. (2013). Music–color associations are mediated by emotion, Proc. Natl Acad. Sci. USA 110, 88368841.

    • Search Google Scholar
    • Export Citation
  • Parise C. V., Knorre K., Ernst M. O. (2014). Natural auditory scene statistics shapes human spatial hearing, Proc. Natl Acad Sci. USA 111, 61046108.

    • Search Google Scholar
    • Export Citation
  • Pelli D. G. (1997). The VideoToolbox software for visual psychophysics: transforming numbers into movies, Spat. Vis. 10, 437442.

  • Pridmore R. W. (2007). Effects of luminance, wavelength and purity on the color attributes: brief review with new data and perspectives, Color Res. Appl. 32, 208222.

    • Search Google Scholar
    • Export Citation
  • Root R. T., Ross S. (1965). Further validation of subjective scales for loudness and brightness by means of cross-modality matching, Am. J. Psychol. 78, 285289.

    • Search Google Scholar
    • Export Citation
  • Schloss K., Lai Y.-H., Witzel C. (2016). Yellow is no happier than blue when lightness and chroma are controlled, J. Vis. 16, 624. DOI:10.1167/16.12.624.

    • Search Google Scholar
    • Export Citation
  • Sebba R. (1991). Structural correspondence between music and color, Color Res. Appl. 16, 8188.

  • Sharpe L. T., Stockman A., Jägle H., Nathans J. (1999). Opsin genes, cone photopigments, color vision, and color blindness, in: Color Vision: from Genes to Perception, Gegenfurtner K. R., Sharpe L. T. (Eds), pp.  351. Cambridge University Press, Cambridge, UK.

    • Search Google Scholar
    • Export Citation
  • Simner J. (2012a). Defining synaesthesia, Br. J. Psychol. 103, 115.

  • Simner J. (2012b). Defining synaesthesia: a response to two excellent commentaries, Br. J. Psychol. 103, 2427.

  • Simner J. (2013). The ‘rules’ of synesthesia, in: Oxford Handbook of Synesthesia, Simner J., Hubbard E. (Eds), pp.  149164. Oxford University Press, Oxford, UK.

    • Search Google Scholar
    • Export Citation
  • Simner J., Ludwig V. U. (2012). The color of touch: a case of tactile-visual synaesthesia, Neurocase 18, 167180.

  • Simner J., Mulvenna C., Sagiv N., Tsakanikos E., Witherby S. A., Fraser C., Scott K., Ward J. (2006). Synaesthesia: the prevalence of atypical cross-modal experiences, Perception 35, 10241033.

    • Search Google Scholar
    • Export Citation
  • Simpson R. H., Quinn M., Ausubel D. P. (1956). Synesthesia in children: association of colors with pure tone frequencies, J. Genet. Psychol. 89, 95103.

    • Search Google Scholar
    • Export Citation
  • Spence C. (2011). Crossmodal correspondences: a tutorial review, Attent. Percept. Psychophys. 73, 971995.

  • Spence C., Deroy O. (2012). Crossmodal correspondences: innate or learned? Iperception 3, 316. DOI:10.1068/i0526ic.

  • Stevens J. C., Marks L. E. (1965). Cross-modality matching of brightness and loudness, Proc. Natl Acad. Sci. USA 54, 407411.

  • Thornley Head P. D. (2006). Synaesthesia: pitch-colour isomorphism in RGB-space? Cortex 42, 164174.

  • Walker P. (2012). Cross-sensory correspondences and cross talk between dimensions of connotative meaning: visual angularity is hard, high-pitched, and bright, Attent. Percept. Psychophys. 74, 17921809.

    • Search Google Scholar
    • Export Citation
  • Walker P., Bremner G., Mason U., Spring J., Mattock K., Slater A., Johnson S. (2010). Preverbal infants’ sensitivity to synaesthetic cross-modality correspondences, Psychol. Sci. 21, 2125.

    • Search Google Scholar
    • Export Citation
  • Walsh V. (2003). A theory of magnitude: common cortical metrics of time, space and quantity, Trends Cogn. Sci. 7, 483488.

  • Ward J. (2013). Synesthesia, Annu. Rev. Psychol. 64, 4975.

  • Ward J., Meijer P. (2010). Visual experiences in the blind induced by an auditory sensory substitution device, Consc. Cogn. 19, 492500.

    • Search Google Scholar
    • Export Citation
  • Ward J., Wright T. (2014). Sensory substitution as an artificially acquired synaesthesia, Neurosci. Biobehav. Rev. 41, 2635.

  • Ward J., Huckstep B., Tsakanikos E. (2006). Sound–colour synaesthesia: to what extent does it use cross-modal mechanisms common to us all? Cortex 42, 264280.

    • Search Google Scholar
    • Export Citation
  • Ward J., Moore S., Thompson-Lake D., Salih S., Beck B. (2008). The aesthetic appeal of auditory — visual synaesthetic perceptions in people without synaesthesia, Perception 37, 12851296.

    • Search Google Scholar
    • Export Citation
  • Witthoft N., Winawer J. (2006). Synesthetic colors determined by having colored refrigerator magnets in childhood, Cortex 42, 175183.

  • Witthoft N., Winawer J., Eagleman D. M. (2015). Prevalence of learned grapheme-color pairings in a large online sample of synesthetes, PloS One 10, e0118996. DOI:10.1371/journal.pone.0118996.

    • Search Google Scholar
    • Export Citation
  • Witzel C., Franklin A. (2014). Do focal colors look particularly ‘colorful’? J. Opt. Soc. Am. A Opt. Image Sci. Vis. 31, A365A374.

    • Search Google Scholar
    • Export Citation

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 617 177 18
Full Text Views 279 32 1
PDF Views & Downloads 49 25 1