Do Congruent Auditory Stimuli Facilitate Visual Search in Dynamic Environments? An Experimental Study Based on Multisensory Interaction
In: Multisensory ResearchAbstract
The purpose of this study was to investigate the cue congruency effect of auditory stimuli during visual search in dynamic environments. Twenty-eight participants were recruited to conduct a visual search experiment. The experiment applied auditory stimuli to understand whether they could facilitate visual search in different types of background. Additionally, target location and target orientation were manipulated to clarify their influences on visual search. Target location was related to horizontal visual search and target orientation was associated with visual search for an inverted target. The results regarding dynamic backgrounds reported that target-congruent auditory stimuli could speed up the visual search time. In addition, the cue congruency effect of auditory stimuli was critical for the center of the visual display but declined for the edge, indicating the inhibition of horizontal visual search behavior. Moreover, few improvements accompanying auditory stimuli were provided for the visual detection of non-inverted and inverted targets. The findings of this study suggested developing multisensory interaction with head-mounted displays, such as augmented reality glasses, in real life.
Purchase
Buy instant access (PDF download and unlimited online access):
Institutional Login
Log in with Open Athens, Shibboleth, or your institutional credentials
Personal login
Log in with your brill.com account
-
Allen, R., Mcgeorge, P., Pearson, D. and Milne, A. B. (2004). Attention and expertise in multiple target tracking, Appl. Cogn. Psychol. 18, 337–347. DOI:10.1002/acp.975.
-
Bagui, S. (1998). Reasons for increased learning using multimedia, J. Educ. Multimed. Hypermedia 7, 3–18.
-
Beck, M. R., Peterson, M. S., Boot, W. R., Vomela, M. and Kramer, A. F. (2006). Explicit memory for rejected distractors during visual search, Vis. Cogn. 14, 150–174. DOI:10.1080/13506280600574487.
-
Benoit, M. M., Raij, T., Lin, F.-H., Jääskeläinen, I. P. and Stufflebeam, S. (2010). Primary and multisensory cortical activity is correlated with audiovisual percepts, Hum. Brain. Mapp. 31, 526–538. DOI:10.1002/hbm.20884.
-
Brungart, D. S., Kruger, S. E., Kwiatkowski, T., Heil, T. and Cohen, J. (2019). The effect of walking on auditory localization, visual discrimination, and aurally aided visual search, Hum. Factors 61, 976–991. DOI:10.1177/0018720819831092.
-
Chapman, P. R. and Underwood, G. (1998). Visual search of driving situations: danger and experience, Perception 27, 951–964. DOI:10.1068/p270951.
-
Cunio, R. J., Dommett, D. and Houpt, J. (2019). Spatial auditory cueing for a dynamic three-dimensional virtual reality visual search task, Proc. Hum. Factors Ergonom. Soc. Annu. Meet. 63, 1766–1770. DOI:10.1177/1071181319631045.
-
Fetsch, C. R., DeAngelis, G. C. and Angelaki, D. E. (2013). Bridging the gap between theories of sensory cue integration and the physiology of multisensory neurons, Nat. Rev. Neurosci. 14, 429–442. DOI:10.1038/nrn3503.
-
Figueroa, J. C. M., Arellano, R. A. B. and Calinisan, J. M. E. (2018). A comparative study of virtual reality and 2D display methods in visual search in real scenes, in: Advances in Human Factors in Simulation and Modeling, AHFE 2017, D. Cassenti (Ed.), Advances in Intelligent Systems and Computing, vol. 591, pp. 366–377. Springer, Cham, Switzerland. DOI:10.1007/978-3-319-60591-3_33.
-
Gray, R., Spence, C., Ho, C. and Tan, H. Z. (2013). Efficient multimodal cuing of spatial attention, Proc. IEEE 101, 2113–2122. DOI:10.1109/JPROC.2012.2225811.
-
Horowitz, T. and Wolfe, J. (2003). Memory for rejected distractors in visual search?, Vis. Cogn. 10, 257–298. DOI:10.1080/13506280143000005.
-
Horowitz, T. S. and Wolfe, J. M. (1998). Visual search has no memory, Nature 394, 575–577. DOI:10.1038/29068.
-
Iordanescu, L., Guzman-Martinez, E., Grabowecky, M. and Suzuki, S. (2008). Characteristic sounds facilitate visual search, Psychon. Bull. Rev. 15, 548–554. https://dx.doi.org/10.3758%2Fpbr.15.3.548.
-
Iordanescu, L., Grabowecky, M. and Suzuki, S. (2011). Object-based auditory facilitation of visual search for pictures and words with frequent and rare targets, Acta Psychol. (Amst.) 137, 252–259. DOI:10.1016/j.actpsy.2010).07.017.
-
Knoeferle, K. M., Knoeferle, P., Velasco, C. and Spence, C. (2016). Multisensory brand search: how the meaning of sounds guides consumers’ visual attention, J. Exp. Psychol. Appl. 22, 196–210. DOI:10.1037/xap0000084.
-
Lau, S. T., Maracle, J., Coletta, D., Singh, G., Campos, J. and Pichora-Fuller, M. K. (2012). Auditory spatial attention in a complex acoustic environment while walking: investigation of dual-task performance, Can. Acoust. 40, 118–119. https://jcaa.caa-aca.ca/index.php/jcaa/issue/view/254.
-
Lundqvist, L.-M. and Eriksson, L. (2019). Age, cognitive load, and multimodal effects on driver response to directional warning, Appl. Ergon. 76, 147–154. DOI:10.1016/j.apergo.2019).01.002.
-
Marucci, M., Di Flumeri, G., Borghini, G., Sciaraffa, N., Scandola, M., Pavone, E. F., Babiloni, F., Betti, V. and Aricò, P. (2021). The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence, Sci. Rep. 11, 4831. DOI:10.1038/s41598-021-84196-8.
-
Mclntire, J. P., Havig, P. R., Watamaniuk, S. N. J. and Gilkey, R. H. (2010). Visual search performance with 3-D auditory cues: effects of motion, target location, and practice, Hum. Factors 52, 41–53. DOI:10.1177/0018720810368806.
-
Navarra, J., Soto-Faraco, S. and Spence, C. (2007). Adaptation to audiotactile asynchrony, Neurosci. Lett. 413, 72–76. DOI:10.1016/j.neulet.2006.11.027.
-
Noel, J.-P., Wallace, M. T., Orchard-Mills, E., Alais, D. and Van der Burg, E. (2015). True and perceived synchrony are preferentially associated with particular sensory pairings, Sci. Rep. 5, 17467. DOI:10.1038/srep17467.
-
Ojanpää, H., Näsänen, R. and Kojo, I. (2002). Eye movements in the visual search of word lists, Vision Res. 42, 1499–1512. DOI:10.1016/s0042-6989(02)00077-9.
-
Shams, L. and Seitz, A. R. (2008). Benefits of multisensory learning, Trends Cogn. Sci. 12, 411–417. DOI:10.1016/j.tics.2008).07.006.
-
Spence, C. J. and Driver, J. (1994). Covert spatial orienting in audition: exogenous and endogenous mechanisms, J. Exp. Psychol. Hum. Percept. Perform. 20, 555–574. DOI:10.1037/0096-1523.20.3.555.
-
Spence, C. J. and Driver, J. (1997). Audiovisual links in exogenous covert spatial orienting, Percept. Psychophys. 59, 1–22. DOI:10.3758/bf03206843.
-
Stein, B. E., Standford, T. R. and Rowland, B. A. (2014). Development of multisensory integration from the perspective of the individual neuron, Nat. Rev. Neurosci. 15, 520–535. DOI:10.1038/nrn3742.
-
Tombu, M. and Seiffert, A. E. (2008). Attentional costs in multiple-object tracking, Cognition 108, 1–25. https://dx.doi.org/10.1016%2Fj.cognition.2007).12.014.
-
Treisman, A. M. and Gelade, G. (1980). A feature-integration theory of attention, Cogn. Psychol. 12, 97–136. DOI:10.1016/0010-0285(80)90005-5.
-
Underwood, G., Chapman, P., Bowden, K. and Crundall, D. (2002). Visual search while driving: skill and awareness during inspection of the scene, Transp. Res. Part F Traffic Psychol. Behav. 5, 87–97. DOI:10.1016/S1369-8478(02)00008-6.
-
Van der Burg, E., Orchard-Mills, E. and Alais, D. (2015). Rapid temporal recalibration is unique to audiovisual stimuli, Exp. Brain Res. 233, 53–59. DOI:10.1007/s00221-014-4085-8.
-
van Ee, R., van Boxtel, J. J. A., Parker, A. L. and Alais, D. (2009). Multisensory congruency as a mechanism for attentional control over perceptual selection, J. Neurosci. 29, 11641–11649. DOI:10.1523/JNEUROSCI.0873-09.2009.
-
Vestner, T., Gray, K. L. H. and Cook, R. (2020). Why are social interactions found quickly in visual search tasks?, Cognition 200, 104270. DOI:10.1016/j.cognition.2020.104270.
-
Wagenmakers, E.-J., Love, J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Selker, R., Gronau, Q. F., Dropmann, D., Boutin, B., Meerhoff, F., Knight, P., Raj, A., van Kesteren, E.-J., van Doorn, J., Šmíra, M., Epskamp, S., Etz, A., Matzke, D., de Jong, T., van den Bergh, D., Sarafoglou, A., Steingroever, H., Derks, K., Rouder, J. N. and Morey, R. D. (2018). Bayesian inference for psychology. Part II: example applications with JASP, Psychon. Bull. Rev. 25, 58–76. DOI:10.3758/s13423-017-1323-7.
-
Weatherless, R. A., Fedele, P. D., Kehring, K. L. and Letowski, T. R. (2013). The effects of simulated hearing loss on speech recognition and walking navigation, Hum. Factors 55, 285–297. DOI:10.1177/0018720812453465.
-
Witten, I. B. and Knudsen, E. I. (2005). Why seeing is believing: merging auditory and visual worlds, Neuron 48, 489–496. DOI:10.1016/j.neuron.2005.10.020.
-
Wolfe, J. M., Yee, A. and Friedman-Hill, S. R. (1992). Curvature is a basic feature for visual search tasks, Perception 21, 465–480. DOI:10.1068/p210465.
| All Time | Past Year | Past 30 Days | |
|---|---|---|---|
| Abstract Views | 312 | 312 | 32 |
| Full Text Views | 20 | 20 | 6 |
| PDF Views & Downloads | 38 | 38 | 17 |
Do Congruent Auditory Stimuli Facilitate Visual Search in Dynamic Environments? An Experimental Study Based on Multisensory Interaction
In: Multisensory ResearchAbstract
The purpose of this study was to investigate the cue congruency effect of auditory stimuli during visual search in dynamic environments. Twenty-eight participants were recruited to conduct a visual search experiment. The experiment applied auditory stimuli to understand whether they could facilitate visual search in different types of background. Additionally, target location and target orientation were manipulated to clarify their influences on visual search. Target location was related to horizontal visual search and target orientation was associated with visual search for an inverted target. The results regarding dynamic backgrounds reported that target-congruent auditory stimuli could speed up the visual search time. In addition, the cue congruency effect of auditory stimuli was critical for the center of the visual display but declined for the edge, indicating the inhibition of horizontal visual search behavior. Moreover, few improvements accompanying auditory stimuli were provided for the visual detection of non-inverted and inverted targets. The findings of this study suggested developing multisensory interaction with head-mounted displays, such as augmented reality glasses, in real life.
| All Time | Past Year | Past 30 Days | |
|---|---|---|---|
| Abstract Views | 312 | 312 | 32 |
| Full Text Views | 20 | 20 | 6 |
| PDF Views & Downloads | 38 | 38 | 17 |