Semantically Congruent Visual Information Can Improve Auditory Recognition Memory in Older Adults

in Multisensory Research
Restricted Access
Get Access to Full Text
Rent on DeepDyve

Have an Access Token?

Enter your access token to activate and access content online.

Please login and go to your personal user account to enter your access token.


Have Institutional Access?

Access content through your institution. Any other coaching guidance?


In the course of normal aging, memory functions show signs of impairment. Studies of memory in the elderly have previously focused on a single sensory modality, although multisensory encoding has been shown to improve memory performance in children and young adults. In this study, we investigated how audiovisual encoding affects auditory recognition memory in older (mean age 71 years) and younger (mean age 23 years) adults. Participants memorized auditory stimuli (sounds, spoken words) presented either alone or with semantically congruent visual stimuli (pictures, text) during encoding. Subsequent recognition memory performance of auditory stimuli was better for stimuli initially presented together with visual stimuli than for auditory stimuli presented alone during encoding. This facilitation was observed both in older and younger participants, while the overall memory performance was poorer in older participants. However, the pattern of facilitation was influenced by age. When encoding spoken words, the gain was greater for older adults. When encoding sounds, the gain was greater for younger adults. These findings show that semantically congruent audiovisual encoding improves memory performance in late adulthood, particularly for auditory verbal material.

Semantically Congruent Visual Information Can Improve Auditory Recognition Memory in Older Adults

in Multisensory Research



BoppK.VerhaeghenP. (2005). Aging and verbal memory span: a meta-analysisJ. Gerontol. B Psychol. Sci. Soc. Sci. 60223233.

CienkowskiK.CarneyA. (2002). Audio-visual speech perception and agingEar Hearing 23439449.

CohenM. A.HorowitzT. S.WolfeJ. M. (2009). Auditory recognition memory is inferior to visual recognition memoryProc. Natl Acad. Sci. USA 10660086010.

DiaconescuA.HasherL.McIntoshA. (2013). Visual dominance and multisensory integration changes with ageNeuroImage 65152166.

DiederichA.ColoniusH.SchomburgA. (2008). Assessing age-related multisensory enhancement with the time-window-of-integration modelNeuropsychologia 4625562562.

GordonM. S.AllenS. (2009). Audiovisual speech in older and younger adults: integrating a distorted visual signal with speech in noiseExp. Aging Res. 35202219.

GradyC.McInthosA.HorwitzB.MaisogJ.UngerleiderL.MentisM.PietriniP.SchapiroM.HaxbyJ. (1995). Age-related reductions in human recognition memory due to impaired encodingScience 269(5221) 218221.

GreenD. M.SwetsJ. (1966). Signal Detection Theory and Psychophysics. WileyNew York, NY, USA.

HeikkiläJ.TiippanaK. (2016). School-aged children can benefit from audiovisual semantic congruency during memory encodingExp. Brain Res. 23411991207.

HeikkiläJ.AlhoK.HyvönenH.TiippanaK. (2015). Audiovisual semantic congruency during encoding enhances memory performanceExp. Psychol. 62123130.

HeikkiläJ.AlhoK.TiippanaK. (in press a). Semantically congruent visual stimuli can improve auditory memoryMultisens. Res.

HeikkiläJ.AlhoK.TiippanaK. (in press b). Semantic congruency improves recognition memory performance for both audiovisual and visual stimuliMultisens. Res.

LaurientiP. J.BurdetteJ. H.MaldjianJ. A.WallaceM. T. (2006). Enhanced multisensory integration in older adultsNeurobiol. Aging 2711551163.

LehmannS.MurrayM. M. (2005). The role of multisensory memories in unisensory object discriminationCogn. Brain Res. 24326334.

LuoL.HendriksT.CraikF. (2007). Age differences in recollection: three patterns of enhanced encodingPsychol. Aging 22269280.

MacmillanN. A.CreelmanC. D. (2005). Detection Theory: a User’s Guide. Lawrence Erlbaum AssociatesNew York, NY, USA.

MaguinnessC.SettiA.BurkeK.KennyR. A.NewellF. (2011). The effect of combined sensory and semantic components on audio-visual speech perception in older adultsFront. Aging Neurosci. 319. DOI:10.3389/fnagi.2011.00019.

MillerJ. (1996). The sampling distribution of dPerception and Psychophysics 586572.

MoranZ.BachmanP.PhamP.ChoS.CannonT.ShamsL. (2013). Multisensory encoding improves auditory recognitionMultisens. Res. 26581592.

MurdockB. B. (1982). Recognition memory in: Handbook of Research Methods in Human Memory and CognitionPuffC. R. (Ed.) pp.  126. Academic Press Inc.London, UK.

MurrayM. M.MichelC. M.Grave de PeraltaR.OrtigueS.BrunetD.Gonzalez AndinoS.SchniderA. (2004). Rapid discrimination of visual and multisensory memories revealed by electrical neuroimagingNeuroImage 21125135.

OldS. R.Naveh-BenjaminM. (2008). Age-related changes in memory in: Handbook of Cognitive AgingHoferS. M.AlwinD. F. (Eds) pp.  151167. Sage PublicationsThousand Oaks, CA, USA.

ParkD.LautenschlagerG.HeddenT.DavidsonN.SmithA.SmithP. (2002). Models of visuospatial and verbal memory across the adult life spanPsychol. Aging 17826837.

PeifferA.MozolicJ.HugenschmidtC.LaurientiP. (2007). Age-related multisensory enhancement in a simple audiovisual detection taskNeuroReport 1810771081.

RönnlundM.NybergL.BäckmanL.NilssonL. (2005). Stability, growth, and decline in adult life span development of declarative memory: cross-sectional and longitudinal data from a population-based studyPsychol. Aging 20318.

SalthouseT. (1996). The processing speed theory of adult age differences in cognitionPsychol. Rev. 103403428.

SalthouseT. (2000). Aging and measures of processing speedBiol. Psychol. 543554.

SchneiderT. R.EngelA. K.DebenerS. (2008). Multisensory identification of natural objects in a two-way crossmodal priming paradigmExp. Psychol. 55121132.

SekiyamaK.SoshiT.SakamotoS. (2014). Enhanced audiovisual integration with aging in speech perception: a heightened McGurk effect in older adultsFront. Psychol. 5323. DOI:10.3389/fpsyg.2014.00323.

SettiA.BurkeK. E.KennyR.NewellF. N. (2013). Susceptibility to a multisensory speech illusion in older persons is driven by perceptual processesFron. Psychol. 4575. DOI:10.3389/fpsyg.2013.00575.

SommersM. S.Tye-MurrayN.SpeharB. (2005). Auditory-visual speech perception and auditory-visual enhancement in normal hearing younger and older adultsEar Hearing 26263275.

SpaniolJ.MaddenD.VossA. (2006). A diffusion model analysis of adult age differences in episodic and semantic long-term memory retrievalJ. Exp. Psychol. Learn. Mem. Cogn. 32101117.

StanislavH.TorodovN. (1999). Calculating signal detection theory measuresBehav. Res. Meth. Instrum. Comput. 31137149.

StevensonR. A.NelmsC. E.BaumS. H.ZurkovskyL.BarenseM. D.NewhouseP. A.WallaceM. T. (2015). Deficits in audiovisual speech perception in normal aging emerge at the level of whole-word recognitionNeurobiol. Aging 36283291.

TalsmaD.SenkowskiD.Soto-FaracoS.WoldorffM. G. (2010). The multifaceted interplay between attention and multisensory integrationTrends Cogn. Sci. 14400410.

ThelenA.TalsmaD.MurrayM. (2015). Single-trial multisensory memories affect later auditory and visual object discriminationCognition 138148160.

Tye-MurrayN.SommersM.SpeharB.MyersonJ.HaleS.RoseN. S. (2008). Auditory-visual discourse comprehension by older and young adults in favorable and unfavorable conditionsInt. J. Audiol. 47(Suppl. 2) S31S37.

UenoD.MasumotoK.SutaniK.IwakiS. (2015). Latency of modality-specific reactivation of auditory and visual information during episodic memory retrievalNeuroReport 26302308.

Van GervenP. W.GuerreiroM. J. (2016). Selective attention and sensory modality in aging: curses and blessingsFront. Hum. Neurosci. 10147. DOI:10.3389/fnhum.2016.00147.

WinnekeA. H.PhillipsN. A. (2011). Does audiovisual speech offer a fountain of youth for old ears? An event-related brain potential study of age differences in audiovisual speech perceptionPsychol. Aging 26427438.

Yuval-GreenbergS.DeouellL. Y. (2009). The dog’s meow: asymmetrical interaction in cross-modal object recognitionExp. Brain Res. 193603614.


  • View in gallery

    Recognition memory performance (d) for sounds and spoken words presented alone or together with semantically congruent pictures or written words during encoding in younger and older adults. Error bars represent standard error of the mean.

  • View in gallery

    Combined gain indices for sounds and spoken words in younger and older adults. Error bars represent standard error of the mean.


Content Metrics

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 33 33 22
Full Text Views 72 72 60
PDF Downloads 3 3 1
EPUB Downloads 0 0 0