Sound Properties Associated With Equiluminant Colours

in Multisensory Research
Restricted Access
Get Access to Full Text
Rent on DeepDyve

Have an Access Token?



Enter your access token to activate and access content online.

Please login and go to your personal user account to enter your access token.



Help

Have Institutional Access?



Access content through your institution. Any other coaching guidance?



Connect

There is a widespread tendency to associate certain properties of sound with those of colour (e.g., higher pitches with lighter colours). Yet it is an open question how sound influences chroma or hue when properly controlling for lightness. To examine this, we asked participants to adjust physically equiluminant colours until they ‘went best’ with certain sounds. For pure tones, complex sine waves and vocal timbres, increases in frequency were associated with increases in chroma. Increasing the loudness of pure tones also increased chroma. Hue associations varied depending on the type of stimuli. In stimuli that involved only limited bands of frequencies (pure tones, vocal timbres), frequency correlated with hue, such that low frequencies gave blue hues and progressed to yellow hues at 800 Hz. Increasing the loudness of a pure tone was also associated with a shift from blue to yellow. However, for complex sounds that share the same bandwidth of frequencies (100–3200 Hz) but that vary in terms of which frequencies have the most power, all stimuli were associated with yellow hues. This suggests that the presence of high frequencies (above 800 Hz) consistently yields yellow hues. Overall we conclude that while pitch–chroma associations appear to flexibly re-apply themselves across a variety of contexts, frequencies above 800 Hz appear to produce yellow hues irrespective of context. These findings reveal new sound–colour correspondences previously obscured through not controlling for lightness. Findings are discussed in relation to understanding the underlying rules of cross-modal correspondences, synaesthesia, and optimising the sensory substitution of visual information through sound.

Sound Properties Associated With Equiluminant Colours

in Multisensory Research

Sections

References

AfraP.FunkeM.MatsuoF. (2009). Acquired auditory-visual synesthesia: a window to early cross-modal sensory interactionsPsychol. Res. Behav. Manag. 23137.

ÁlvaroL.MoreiraH.LilloJ.FranklinA. (2015). Color preference in red–green dichromatsProc. Natl Acad Sci. 11293169321.

AyamaM.IkedaM. (1998). Brightness-to-luminance ratio of colored light in the entire chromaticity diagramColor Res. Appl. 23274287.

BatscheletE. (1981). Circular Statistics in Biology. Academic PressLondon, UK.

BerensP. (2009). CircStat: a MATLAB toolbox for circular statisticsJ. Stat. Softw. 31121.

BienN.Ten OeverS.GoebelR.SackA. T. (2012). The sound of size: crossmodal binding in pitch-size synesthesia: a combined TMS, EEG and psychophysics studyNeuroimage 59663672.

BoersmaP.WeeninkD. (2012). Praat: doing phonetics by computer. Retrieved from http://www.praat.org/.

BondB.StevensS. (1969). Cross-modality matching of brightness to loudness by 5-year-oldsPercept. Psychophys. 6337339.

BraatenR. (1993). Synesthetic correspondence between visual location and auditory pitch in infants. Paper presented at the 34th Annual Meeting of the Psychonomic Society Washington DC USA.

BrainardD. H. (1997). The psychophysics toolboxSpat. Vis. 10433436.

Cohen KadoshR.TerhuneD. B. (2012). Redefining synaesthesia? Br. J. Psychol. 1032023.

DeroyO.SpenceC. (2013). Why we are not all synesthetes (not even weakly so)Psychonom. Bull. Rev. 20643664.

DeroyO.FasielloI.HaywardV.AuvrayM. (2016). Differentiated audio-tactile correspondences in sighted and blind individualsJ. Exp. Psychol. Hum. Percept. Perform. 4212041214.

DolscheidS.ShayanS.MajidA.CasasantoD. (2013). The thickness of musical pitch psychophysical evidence for linguistic relativityPsychol. Sci. 24613621.

EaglemanD. M. (2012). Synaesthesia in its protean guisesBr. J. Psychol. 1031619.

EstermanM.VerstynenT.IvryR. B.RobertsonL. C. (2006). Coming unbound: disrupting automatic integration of synesthetic color and graphemes by transcranial magnetic stimulation of the right parietal lobeJ. Cogn. Neurosci. 1815701576.

EvansK. K.TreismanA. (2010). Natural cross-modal mappings between visual and auditory featuresJ. Vis. 106. DOI:10.1167/10.1.6.

FairchildM. D. (1998). Color Appearance Models. Addison Wesley LongmanReading, MA, USA.

FisherR. A. (1921). On the ‘probable error’ of a coefficient of correlation deduced from a small sampleMetron 1332.

FitchW. T. (1997). Vocal tract length and formant frequency dispersion correlate with body size in rhesus macaquesJ. Acoust. Soc. Am. 10212131222.

FletcherH.MunsonW. A. (1933). Loudness, its definition, measurement and calculationBell Syst. Techn. J. 12377430.

ForderL.HeX.WitzelC.FranklinA. (2014). Speakers of different colour lexicons differ only in post-perceptual processing of colourPerception (ECVP Abstract Suppl.) 4333.

FryerL.FreemanJ.PringL. (2014). Touching words is not enough: how visual experience influences haptic–auditory associations in the ‘Bouba–Kiki’ effectCognition 132164173.

GiannakisK. (2001). Sound mosaics: a graphical user interface for sound synthesis based on audio-visual associations. Doctoral dissertation Middlesex University UK. Retrieved from http://eprints.mdx.ac.uk/6634/1/Konstantinos-sound_mosaics.phd.pdf.

GoodyearB. G.MenonR. S. (1998). Effect of luminance contrast on BOLD fMRI response in human primary visual areasJ. Neurophysiol. 7922042207.

Hamilton-FletcherG.WardJ. (2013). Representing colour through hearing and touch in sensory substitution devicesMultisens. Res. 26503532.

Hamilton-FletcherG.WrightT. D.WardJ. (2016a). Cross-modal correspondences enhance performance on a colour-to-sound sensory substitution deviceMultisens. Res. 29337363.

Hamilton-FletcherG.MengucciM.MedeirosF. (2016b). Synaestheatre: sonification of coloured objects in space in: Proceedings of the International Conference on Live Interfaces 2016 Sussex Brighton UK pp. 252–256.

Hamilton-FletcherG.ObristM.WattenP.MengucciM.WardJ. (2016c). ‘I always wanted to see the night sky’: blind user preferences for sensory substitution devices in: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems San Jose CA USA pp. 2162–2174.

HupéJ. M.DojatM. (2015). A critical review of the neuroimaging literature on synesthesiaFront. Hum. Neurosci. 9103. DOI:10.3389/fnhum.2015.00103.

ISO (2003). Acoustics — Normal Equal-Loudness Level Contours. International Organisation for Standardisation Geneva Switzerland.

JewanskiJ. (2010). Color-tone analogies: a systematic presentation of the principles of correspondence in: Audiovisuology: Compendium. A Multidisciplinary Survey of Audiovisual CultureDanielsD.NaumannS.ThobenJ. (Eds) pp.  338347. König, Köln, Germany.

JonasC.SpillerM. J.HibbardP. (in press). Summation of visual attributes in auditory-visual crossmodal correspondences Psychonom. Bull. Rev.

LudwigV. U.SimnerJ. (2013). What colour does that feel? Tactile-visual mapping and the development of cross-modalityCortex 4910891099.

LudwigV. U.AdachiI.MatsuzawaT. (2011). Visuoauditory mappings between high luminance and high pitch are shared by chimpanzees (Pan troglodytes) and humansProc. Natl. Acad. Sci. 1082066120665.

MarksL. E. (1974). On associations of light and sound: the mediation of brightness, pitch, and loudnessAm. J. Psychol. 87173188.

MarksL. E. (1987). On cross-modal similarity: auditory-visual interactions in speeded discriminationJ. Exp. Psychol. Hum. Percept. Perform. 13384394.

MartinoG.MarksL. E. (1999). Perceptual and linguistic interactions in speeded classification: tests of the semantic coding hypothesisPerception 28903923.

MelaraR. D. (1989). Dimensional interaction between color and pitchJ. Exp. Psychol. Hum. Percept. Perform. 156979.

MondlochC. J.MaurerD. (2004). Do small white balls squeak? Pitch-object correspondences in young childrenCogn. Affect. Behav. Neurosci. 4133136.

MoosA.SmithR.MillerS. R.SimmonsD. R. (2014). Cross-modal associations in synaesthesia: vowel colours in the ear of the beholderIperception 5132142.

MuggletonN.TsakanikosE.WalshV.WardJ. (2007). Disruption of synaesthesia following TMS of the right posterior parietal cortexNeuropsychologia 4515821585.

MulertC.JägerL.ProppS.KarchS.StörmannS.PogarellO.MöllerH.-J.HegerlU. (2005). Sound level dependence of the primary auditory cortex: simultaneous measurement with 61-channel EEG and fMRINeuroimage 284958.

NayataniY. (1998). A colorimetric explanation of the Helmholtz–Kohlrausch effectColor Res. Appl. 23374378.

NewtonI. (1979 [1706]). Opticks: or a Treatise of the Reflections Refractions Inflections and Colours of Light. Courier Dover PublicationsNew York, NY, USA.

NovichS.ChengS.EaglemanD. M. (2011). Is synaesthesia one condition or many? A large-scale analysis reveals subgroupsJ. Neuropsychol. 5353371.

OrlandatouK. (2012). The role of pitch and timbre in the synaesthetic experience in: Proceedings of the 12th International Conference on Music Perception and Cognition and the 8th Triennial Conference of the European Society for the Cognitive Sciences of Music Thessaloniki Greece pp. 751–758.

PalmerS. E.SchlossK. B.XuZ.Prado-LeónL. R. (2013). Music–color associations are mediated by emotionProc. Natl Acad. Sci. USA 11088368841.

PariseC. V.KnorreK.ErnstM. O. (2014). Natural auditory scene statistics shapes human spatial hearingProc. Natl Acad Sci. USA 11161046108.

PelliD. G. (1997). The VideoToolbox software for visual psychophysics: transforming numbers into moviesSpat. Vis. 10437442.

PridmoreR. W. (2007). Effects of luminance, wavelength and purity on the color attributes: brief review with new data and perspectivesColor Res. Appl. 32208222.

RootR. T.RossS. (1965). Further validation of subjective scales for loudness and brightness by means of cross-modality matchingAm. J. Psychol. 78285289.

SchlossK.LaiY.-H.WitzelC. (2016). Yellow is no happier than blue when lightness and chroma are controlledJ. Vis. 16624. DOI:10.1167/16.12.624.

SebbaR. (1991). Structural correspondence between music and colorColor Res. Appl. 168188.

SharpeL. T.StockmanA.JägleH.NathansJ. (1999). Opsin genes, cone photopigments, color vision, and color blindness in: Color Vision: from Genes to PerceptionGegenfurtnerK. R.SharpeL. T. (Eds) pp.  351. Cambridge University PressCambridge, UK.

SimnerJ. (2012a). Defining synaesthesiaBr. J. Psychol. 103115.

SimnerJ. (2012b). Defining synaesthesia: a response to two excellent commentariesBr. J. Psychol. 1032427.

SimnerJ. (2013). The ‘rules’ of synesthesia in: Oxford Handbook of SynesthesiaSimnerJ.HubbardE. (Eds) pp.  149164. Oxford University PressOxford, UK.

SimnerJ.LudwigV. U. (2012). The color of touch: a case of tactile-visual synaesthesiaNeurocase 18167180.

SimnerJ.MulvennaC.SagivN.TsakanikosE.WitherbyS. A.FraserC.ScottK.WardJ. (2006). Synaesthesia: the prevalence of atypical cross-modal experiencesPerception 3510241033.

SimpsonR. H.QuinnM.AusubelD. P. (1956). Synesthesia in children: association of colors with pure tone frequenciesJ. Genet. Psychol. 8995103.

SpenceC. (2011). Crossmodal correspondences: a tutorial reviewAttent. Percept. Psychophys. 73971995.

SpenceC.DeroyO. (2012). Crossmodal correspondences: innate or learned? Iperception 3316. DOI:10.1068/i0526ic.

StevensJ. C.MarksL. E. (1965). Cross-modality matching of brightness and loudnessProc. Natl Acad. Sci. USA 54407411.

Thornley HeadP. D. (2006). Synaesthesia: pitch-colour isomorphism in RGB-space? Cortex 42164174.

WalkerP. (2012). Cross-sensory correspondences and cross talk between dimensions of connotative meaning: visual angularity is hard, high-pitched, and brightAttent. Percept. Psychophys. 7417921809.

WalkerP.BremnerG.MasonU.SpringJ.MattockK.SlaterA.JohnsonS. (2010). Preverbal infants’ sensitivity to synaesthetic cross-modality correspondencesPsychol. Sci. 212125.

WalshV. (2003). A theory of magnitude: common cortical metrics of time, space and quantityTrends Cogn. Sci. 7483488.

WardJ. (2013). SynesthesiaAnnu. Rev. Psychol. 644975.

WardJ.MeijerP. (2010). Visual experiences in the blind induced by an auditory sensory substitution deviceConsc. Cogn. 19492500.

WardJ.WrightT. (2014). Sensory substitution as an artificially acquired synaesthesiaNeurosci. Biobehav. Rev. 412635.

WardJ.HuckstepB.TsakanikosE. (2006). Sound–colour synaesthesia: to what extent does it use cross-modal mechanisms common to us all? Cortex 42264280.

WardJ.MooreS.Thompson-LakeD.SalihS.BeckB. (2008). The aesthetic appeal of auditory — visual synaesthetic perceptions in people without synaesthesiaPerception 3712851296.

WitthoftN.WinawerJ. (2006). Synesthetic colors determined by having colored refrigerator magnets in childhoodCortex 42175183.

WitthoftN.WinawerJ.EaglemanD. M. (2015). Prevalence of learned grapheme-color pairings in a large online sample of synesthetesPloS One 10e0118996. DOI:10.1371/journal.pone.0118996.

WitzelC.FranklinA. (2014). Do focal colors look particularly ‘colorful’? J. Opt. Soc. Am. A Opt. Image Sci. Vis. 31A365A374.

Figures

  • View in gallery

    Chroma adjustments for pure tones, complex sine waves and vocal timbres. Panel a illustrates the relationship between the frequency of pure tones for set 1 (grey diamonds) and set 2 (red squares) and chroma average adjustments. Panel b illustrates the same relationship, but with z-scored frequency to account for range effects within each stimulus set. Panel c illustrates results for pure tones that vary in loudness. Panel d illustrates the results for vocal frequency bands that vary in their average frequency. Panels e and f illustrate complex sine wave and vocal timbre sounds that vary in their ‘centre of gravity’. Key: p<0.05, ∗∗p<0.01, ∗∗∗p<0.001.

  • View in gallery

    Chroma adjustments for all stimulus sets in which the average frequency varied. Panel a illustrates the relationship between chroma and frequency (z-scored) across all stimulus sets in which the average frequency varied. Panel b illustrates the average chroma adjustments across all stimulus sets in which the average frequency varied. Key: ∗∗∗p<0.001.

  • View in gallery

    Change of hue with frequency, loudness, and ‘centre of gravity’. Statistics in the upper left corners report the correlation across all stimuli varying in frequency (pure tones set 1, set 2, vocal frequency bands) based on Fisher’s z-transform across individuals (panel a), different levels of pure tone loudness (panel b), and different levels of ‘centre of gravity’ (panel c). Key: ∗∗p<0.01, ∗∗∗p<0.001. Note that hue adjustments change linearly as a function of frequencies for pure tones and vocal frequency bands (panel a), and as a function of loudness (panel b); but stay almost constant for sounds that vary in their centre of gravity (panel c). A more detailed breakdown of hue for individual stimuli and sets can be found in Supplementary Figs S1 and S2.

Index Card

Content Metrics

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 30 30 20
Full Text Views 23 23 23
PDF Downloads 2 2 2
EPUB Downloads 0 0 0