[i] is Lighter and More Greenish Than [o]: Intrinsic Association Between Vowel Sounds and Colors

in Multisensory Research
Restricted Access
Get Access to Full Text
Rent on DeepDyve

Have an Access Token?



Enter your access token to activate and access content online.

Please login and go to your personal user account to enter your access token.



Help

Have Institutional Access?



Access content through your institution. Any other coaching guidance?



Connect

It has recently been reported in the synesthesia literature that graphemes sharing the same phonetic feature tend to induce similar synesthetic colors. In the present study, we investigated whether phonetic properties are associated with colors in a specific manner among the general population, even when other visual and linguistic features of graphemes are removed. To test this hypothesis, we presented vowel sounds synthesized by systematically manipulating the position of the tongue body’s center. Participants were asked to choose a color after hearing each sound. Results from the main experiment showed that lightness and chromaticity of matched colors exhibited systematic variations along the two axes of the position of the tongue body’s center. Some non-random associations between vowel sounds and colors remained effective with pitch and intensity of the sounds equalized in the control experiment, which suggests that other acoustic factors such as inherent pitch of vowels cannot solely account for the current results. Taken together, these results imply that the association between phonetic features and colors is not random, and this synesthesia-like association is shared by people in the general population.

Multisensory Research

A Journal of Scientific Research on All Aspects of Multisensory Processing

Sections

References

AsanoM.YokosawaK. (2011). Synesthetic colors are elicited by sound quality in Japanese synesthetes, Consc. Cogn. 20, 18161823.

AsanoM.YokosawaK. (2012). Synesthetic colors for Japanese late acquired graphemes, Consc. Cogn. 21, 983993.

BankierisK.SimnerJ. (2015). What is the link between synaesthesia and sound symbolism? Cognition 136, 186195.

BeauchampM. S.ArgallB. D.BodurkaJ.DuynJ. H.MartinA. (2004). Unraveling multisensory integration: patchy organization within human STS multisensory cortex, Nat. Neurosci. 7, 11901192.

BeeliG.EsslenM.JänckeL. (2007). Frequency correlates in grapheme–color synaesthesia, Psychol. Sci. 18, 788792.

BienN.Ten OeverS.GoebelR.SackA. T. (2012). The sound of size: crossmodal binding in pitch-size synesthesia: a combined TMS, EEG and psychophysics study, NeuroImage 59, 663672.

BoersmaP.WeeninkD. (2013). Praat: Doing phonetics by computer [Computer program]. Version 5.3.51. http://www.praat.org/. Retrieved 2 June 2013.

BrangD.RouwR.RamachandranV. S.CoulsonS. (2011). Similarly shaped letters evoke similar colors in grapheme–color synesthesia, Neuropsychologia 49, 13551358.

BrangD.WilliamsL. E.RamachandranV. S. (2012). Grapheme–color synesthetes show enhanced crossmodal processing between auditory and visual modalities, Cortex 48, 630637.

BrowmanC. P.GoldsteinL. M. (1986). Towards an articulatory phonology, Phonol. Yearb. 3, 219252.

Cohen KadoshR.HenikA. (2007). Can synaesthesia research inform cognitive science? Trends Cogn Sci. 11, 177184.

CuskleyC.KirbyS. (2013). Synaesthesia, cross-modality, and language evolution, in: Oxford Handbook of Synaesthesia, SimnerJ.HubbardE. M. (Eds), pp.  869907. Oxford University Press, Oxford, UK.

CuskleyC.SimnerJ.KirbyS. (2017). Phonological and orthographic influences in the bouba–kiki effect, Psychol. Res. 81, 119130.

DeroyO.SpenceC. (2013). Why we are not all synesthetes (not even weakly so), Psychonom. Bull. Rev. 20, 643664.

DixonM. J.SmilekD.DuffyP. L.ZannaM. P.MerikleP. M. (2006). The role of meaning in grapheme–colour synaesthesia, Cortex 42, 243252.

EaglemanD. M.KaganA. D.NelsonS. S.SagaramD.SarmaA. K. (2007). A standardized test battery for the study of synesthesia, J. Neurosci. Meth. 159, 139145.

FairbanksG.HouseA. S.StevensE. L. (1950). An experimental study of vowel intensities, J. Acoust. Soc. Am. 22, 457459.

FernayL.RebyD.WardJ. (2012). Visualized voices: a case study of audio-visual synesthesia, Neurocase 18, 5056.

GallaceA.SpenceC. (2006). Multisensory synesthetic interactions in the speeded classification of visual size, Percept. Psychophys. 68, 11911203.

HansonH. M.StevensK. N. (2002). A quasiarticulatory approach to controlling acoustic source parameters in a Klatt-type formant synthesizer using HLsyn, J. Acoust. Soc. Am. 112, 11581182.

HintonL.NicholsJ.OhalaJ. J. (2006). Sound Symbolism. Cambridge University Press, Cambridge, UK.

HondaK. (1983). Relationship between pitch control and vowel articulation, in: Haskins Laboratories Status Report on Speech Research, SR, 73, pp.  269282. Haskins Laboratories, New Haven, CT, USA.

HubbardT. L. (1996). Synesthesia-like mappings of lightness, pitch, and melodic interval, Am. J. Psychol. 109, 219238.

JacobsonR. (1962). Selected Writings I: Phonological Studies. Mouton, The Hague, The Netherlands.

KangM.-J.KimY.ShinJ.-Y.KimC.-Y. (2017). Graphemes sharing phonetic features tend to induce similar synesthetic colors, Front. Psychol. 8, 337. DOI:10.3389/fpsyg.2017.00337.

KarwoskiT. F.OdbertH. S. (1938). Color–music, Psychol. Monogr. 50, 160.

KimS.BlakeR.KimC.-Y. (2013). Is ‘Σ’ purple or green? Bistable grapheme–color synesthesia induced by ambiguous characters, Consc. Cogn. 22, 955964.

KimY.KimC.-Y. (2014). Correlation between grapheme frequency and synesthetic colors in color-graphemic synesthesia, Korean J. Cogn. Biol. Psychol. 26, 133149.

KöhlerW. (1929). Gestalt Psychology. Liveright, New York, NY, USA.

LewkowiczD. J.TurkewitzG. (1980). Cross-modal equivalence in early infancy: auditory–visual intensity matching, Dev. Psychol. 16, 597607.

MarksL. E. (1974). On associations of light and sound: the mediation of brightness, pitch, and loudness, Am. J. Psychol. 87, 173188.

MarksL. E. (1975). On colored-hearing synesthesia: cross-modal translations of sensory dimensions, Psychol. Bull. 82, 303331.

MarksL. E. (1989). On cross-modal similarity: the perceptual structure of pitch, loudness, and brightness, J. Exp. Psychol. Hum. Percept. Perform. 15, 586602.

MarksL. E.SzczesiulR.OhlottP. (1986). On the cross-modal perception of intensity, J. Exp. Psychol. Hum. Percept. Perform. 12, 517534.

MartinoG.MarksL. E. (2001). Synesthesia: strong and weak, Curr. Dir. Psychol. Sci. 10, 6165.

MaurerD.PathmanT.MondlochC. J. (2006). The shape of boubas: sound–shape correspondences in toddlers and adults, Dev. Sci. 9, 316322.

MelaraR. D. (1989). Dimensional interaction between color and pitch, J. Exp. Psychol. Hum. Percept. Perform. 15, 6979.

MermelsteinP. (1973). Articulatory model for the study of speech production, J. Acoust. Soc. Am. 53, 10701082.

MoosA.SmithR.MillerS. R.SimmonsD. R. (2014). Cross-modal associations in synaesthesia: vowel colours in the ear of the beholder, i-Perception 5, 132142.

NamH.GoldsteinL. M.GiuliviS.LevittA. G.WhalenD. H. (2013). Computational simulation of CV combination preferences in babbling, J. Phon. 41, 6377.

NewmanS. S. (1933). Further experiments in phonetic symbolism, Am. J. Psychol. 45, 5375.

NielsenA. K.RendallD. (2013). Parsing the role of consonants versus vowels in the classic Takete–Maluma phenomenon, Can. J. Exp. Psychol. 67, 153163.

NoesseltT.RiegerJ. W.SchoenfeldM. A.KanowskiM.HinrichsH.HeinzeH. J.DriverJ. (2007). Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices, J. Neurosci. 27, 1143111441.

NoppeneyU.JosephsO.HockingJ.PriceC. J.FristonK. J. (2007). The effect of prior visual information on recognition of speech and sounds, Cereb. Cortex 18, 598609.

OhalaJ. J. (1994). The frequency code underlies the sound symbolic use of voice pitch, in: Sound Symbolism, HintonL.NicholsJ.OhalaJ. J. (Eds), pp.  325347. Cambridge University Press, Cambridge, UK.

PalmerS. E.SchlossK. B.XuZ.Prado-LeónL. R. (2013). Music–color associations are mediated by emotion, Proc. Natl Acad. Sci. 110, 88368841.

PariseC. V.SpenceC. (2009). ‘When birds of a feather flock together’: Synesthetic correspondences modulate audiovisual integration in non-synesthetes, PLoS ONE 4, e5664. DOI:10.1371/journal.pone.0005664.

PeñaM.MehlerJ.NesporM. (2011). The role of audiovisual processing in early conceptual development, Psychol. Sci. 22, 14191421.

RamachandranV. S.HubbardE. M. (2001). Synaesthesia — a window into perception, thought and language, J. Consc. Stud. 8, 334.

RevillK. P.NamyL. L.DeFifeL. C.NygaardL. C. (2014). Cross-linguistic sound symbolism and crossmodal correspondence: evidence from fMRI and DTI, Brain Lang. 128, 1824.

RichA. N.BradshawJ. L.MattingleyJ. B. (2005). A systematic, large-scale study of synaesthesia: implications for the role of early experience in lexical–colour associations, Cognition 98, 5384.

RubinP. E.SaltzmanE.GoldsteinL. M.McGowanR. S.TiedeM. K.BrowmanC. P. (1996). CASY and extensions to the task-dynamic model, in: Proceedings of the 1st ESCA ETRW on Speech Production Modeling and 4th Speech Production Seminar, pp. 125–128. Autrans, Grenoble, France.

SapirE. (1929). A study in phonetic symbolism, J. Exp. Psychol. 12, 225239.

ShinE. H.KimC. Y. (2014). Both ‘’ and ‘’ are yellow: cross-linguistic investigation in search of the determinants of synesthetic color, Neuropsychologia 65, 2536.

SimnerJ. (2007). Beyond perception: synaesthesia as a psycholinguistic phenomenon, Trends Cogn. Sci. 11, 2329.

SimnerJ.LudwigV. U. (2012). The color of touch: a case of tactile–visual synaesthesia, Neurocase 18, 167180.

SimnerJ.WardJ.LanzM.JansariA.NoonanK.GloverL.OakleyD. A. (2005). Non-random associations of graphemes to colours in synaesthetic and non-synaesthetic populations, Cogn. Neuropsychol. 22, 10691085.

SimnerJ.MulvennaC.SagivN.TsakanikosE.WitherbyS. A.FraserC.WardJ. (2006). Synaesthesia: the prevalence of atypical cross-modal experiences, Perception 35, 10241033.

SimpsonR. H.QuinnM.AusubelD. P. (1956). Synesthesia in children: association of colors with pure tone frequencies, J. Gen. Psychol. 89, 95103.

SmilekD.CarriereJ. S.DixonM. J.MerikleP. M. (2007). Grapheme frequency and color luminance in grapheme–color synaesthesia, Psychol. Sci. 18, 793795.

SpectorF.MaurerD. (2013). Early sound symbolism for vowel sounds, i-Perception 4, 239241.

ThompsonP. D.EstesZ. (2011). Sound symbolic naming of novel objects is a graded function, Q. J. Exp. Psychol. (Hove) 64, 23922404.

WardJ.HuckstepB.TsakanikosE. (2006). Sound–colour synaesthesia: to what extent does it use cross-modal mechanisms common to us all? Cortex 42, 264280.

WatsonM. R.AkinsK. A.EnnsJ. T. (2012). Second-order mappings in grapheme–color synesthesia, Psychonom. Bull. Rev. 19, 211217.

WhalenD. H.LevittA. G. (1995). The universality of intrinsic F0 of vowels, J. Phon. 23(3), 349366.

WrembelM. (2009). On hearing colours — cross-modal associations in vowel perception in a non-synaesthetic population, Poznań Stud. Contemp. Linguist. 45, 595612.

Figures

  • Vocal tract representation of CASY with its articulator variables for generating vowel sounds. F: mandibular condyle, C: tongue body center, B: tongue blade, T: tongue tip, J: jaw. The jaw position is given by the angle (JA) from a horizontal line at the joint with the constant distance from F. The tongue body is represented as an imaginary circle with a fixed radius. The position of the tongue body is given by the position of the circle’s center, which is determined by the angle (CA) from the line F–J and the length (CL) of the line F–C. The tongue blade and tip are attached to the tongue body’s circle. The tongue body’s articulator variables (CA and CL) were modulated parametrically to manipulate vowel sounds. All the other articulatory variables were fixed, including the jaw position that affects mouth openness. The positions of the tongue body’s center used for generating the 42 auditory stimuli are superimposed on the outline of the vocal tract.

    View in gallery
  • Acoustic variations of the auditory stimuli in the main experiment generated by articulatory synthesis. Acoustic properties of the vowel stimuli were displayed as brightness of each circle. (A) pitch, (B) intensity, (C) the first (left) and the second (right) formants. Stimuli with higher intensities and frequencies are represented with lighter shades.

    View in gallery
  • Luminance (L) results. (A) The relationship between the tongue body’s position (height and frontness) and L values of the matched colors. The dark green and blue lines indicate results from the main and the control experiment, respectively. The bold lines indicate statistical significance (p<0.05, F-test). The shades indicate ±1 standard error of the mean (SEM). (B) The group mean L values for each of the 42 auditory stimuli based on the tongue body’s position are shown as the level of lightness. Stimuli matched with lighter colors (larger L) were shown in lighter shades.

    View in gallery
  • Chromaticity results along the green–red continuum (a). (A) The relationship between the tongue body’s position (height and frontness) and a values of the matched colors. Dark green and blue lines indicate results from the main and the control experiment, respectively. The bold lines indicate statistical significance (p<0.05, F-test). The shades denote ±1 SEM. (B) The group mean a values for each of the 42 auditory stimuli based on the tongue body’s position are represented as colors with reference to the green–red color axis. More reddish colors denote larger a values whereas more greenish colors denote smaller a values.

    View in gallery
  • Chromaticity results along the blue–yellow continuum (b). (A) The relationship between the tongue body’s position (height and frontness) and b values of the matched colors. The dark green and blue lines indicate results from the main and the control experiment, respectively. The bold lines indicate statistical significance (p<0.05, F-test). The shades denote ±1 SEM. (B) The group mean b values for each of the 42 auditory stimuli based on the tongue body’s position are represented as colors with reference to the blue–yellow color axis. More yellowish colors denote larger b values whereas more bluish colors denote smaller b values.

    View in gallery

Information

Content Metrics

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 20 20 19
Full Text Views 6 6 6
PDF Downloads 1 1 1
EPUB Downloads 0 0 0