Influence of Sensory Conflict on Perceived Timing of Passive Rotation in Virtual Reality
In: Multisensory ResearchSearch for other papers by William Chung in
Current site
Google Scholar
Search for other papers by Michael Barnett-Cowan in
Current site
Google Scholar
Abstract
Integration of incoming sensory signals from multiple modalities is central in the determination of self-motion perception. With the emergence of consumer virtual reality (VR), it is becoming increasingly common to experience a mismatch in sensory feedback regarding motion when using immersive displays. In this study, we explored whether introducing various discrepancies between the vestibular and visual motion would influence the perceived timing of self-motion. Participants performed a series of temporal-order judgements between an auditory tone and a passive whole-body rotation on a motion platform accompanied by visual feedback using a virtual environment generated through a head-mounted display. Sensory conflict was induced by altering the speed and direction by which the movement of the visual scene updated relative to the observer’s physical rotation. There were no differences in perceived timing of the rotation without vision, with congruent visual feedback and when the speed of the updating of the visual motion was slower. However, the perceived timing was significantly further from zero when the direction of the visual motion was incongruent with the rotation. These findings demonstrate the potential interaction between visual and vestibular signals in the temporal perception of self-motion. Additionally, we recorded cybersickness ratings and found that sickness severity was significantly greater when visual motion was present and incongruent with the physical motion. This supports previous research regarding cybersickness and the sensory conflict theory, where a mismatch between the visual and vestibular signals may lead to a greater likelihood for the occurrence of sickness symptoms.
Purchase
Buy instant access (PDF download and unlimited online access):
Institutional Login
Log in with Open Athens, Shibboleth, or your institutional credentials
Personal login
Log in with your brill.com account
-
Acerbi, L., Dokka, K., Angelaki, D. E. and Ma, W. J. (2018). Bayesian comparison of explicit and implicit causal inference strategies in multisensory heading perception, PLoS Comput. Biol. 14, e1006110. DOI:10.1371/journal.pcbi.1006110.
-
Arcioni, B., Palmisano, S., Apthorp, D. and Kim, J. (2019). Postural stability predicts the likelihood of cybersickness in active HMD-based virtual reality, Displays 58, 3–11. DOI:10.1016/j.displa.2018.07.001.
-
Ash, A., Palmisano, S., Govan, D. G. and Kim, J. (2011). Display lag and gain effects on vection experienced by active observers, Aviat. Space Environ. Med. 82, 763–769. DOI:10.3357/ASEM.3026.2011.
-
Barnett-Cowan, M. and Harris, L. R. (2009). Perceived timing of vestibular stimulation relative to touch, light and sound, Exp. Brain Res. 198, 221–231. DOI:10.1007/s00221-009-1779-4.
-
Barnett-Cowan, M. and Harris, L. R. (2011). Temporal processing of active and passive head movement, Exp. Brain Res. 214, 27. DOI:10.1007/s00221-011-2802-0.
-
Barnett-Cowan, M., Raeder, S. M. and Bülthoff, H. H. (2012). Persistent perceptual delay for head movement onset relative to auditory stimuli of different durations and rise times, Exp. Brain Res. 220, 41–50. DOI:10.1007/s00221-012-3112-x.
-
Berger, D. R. and Bülthoff, H. H. (2009). The role of attention on the integration of visual and inertial cues, Exp. Brain Res. 198, 287–300. DOI:10.1007/s00221-009-1767-8.
-
Butler, J. S., Smith, S. T., Campos, J. L. and Bülthoff, H. H. (2010). Bayesian integration of visual and vestibular signals for heading, J. Vis. 10, 23. DOI:10.1167/10.11.23.
-
Butler, J. S., Campos, J. L. and Bülthoff, H. H. (2015). Optimal visual–vestibular integration under conditions of conflicting intersensory motion profiles, Exp. Brain Res. 233, 587–597. DOI:10.1007/s00221-014-4136-1.
-
Chang, N.-Y. N., Uchanski, R. M. and Hullar, T. E. (2012). Temporal integration of auditory and vestibular stimuli, Laryngoscope 122, 1379–1384. DOI:10.1002/lary.23329.
-
Chung, W. and Barnett-Cowan, M. (2017). Persistent perceptual delay for active head movement onset relative to sound onset with and without vision, Exp. Brain Res. 235, 3069–3079. DOI:10.1007/s00221-017-5026-0.
-
Correia Grácio, B. J., Bos, J. E., van Paassen, M. M. and Mulder, M. (2014). Perceptual scaling of visual and inertial cues, Exp. Brain Res. 232, 637–646. DOI:10.1007/s00221-013-3772-1.
-
Cuturi, L. F. and MacNeilage, P. R. (2014). Optic flow induces nonvisual self-motion aftereffects, Curr. Biol. 24, 2817–2821. DOI:10.1016/j.cub.2014.10.015.
-
Davis, S., Nesbitt, K. and Nalivaiko, E. (2015). Comparing the onset of cybersickness using the Oculus Rift and two virtual roller coasters, in: IE 2015: Proceedings of the 11th Australasian Conference on Interactive Entertainment, pp. 3–14.
-
de Winkel, K. N., Weesie, J., Werkhoven, P. J. and Groen, E. L. (2010). Integration of visual and inertial cues in perceived heading of self-motion, J. Vis. 10, 1. DOI:10.1167/10.12.1.
-
de Winkel, K. N., Soyka, F., Barnett-Cowan, M., Bülthoff, H. H., Groen, E. L. and Werkhoven, P. J. (2013). Integration of visual and inertial cues in the perception of angular self-motion, Exp. Brain Res. 231, 209–218. DOI:10.1007/s00221-013-3683-1.
-
de Winkel, K. N., Katliar, M. and Bülthoff, H. H. (2015). Forced fusion in multisensory heading estimation, PLoS ONE 10, e0127104. DOI:10.1371/journal.pone.0127104.
-
de Winkel, K. N., Katliar, M. and Bülthoff, H. H. (2017). Causal inference in multisensory heading estimation, PloS ONE 12, e0169676. DOI:10.1371/journal.pone.0169676.
-
Drugowitsch, J., DeAngelis, G. C., Klier, E. M., Angelaki, D. E. and Pouget, A. (2014). Optimal multisensory decision-making in a reaction-time task, Elife 3, e03005. DOI:10.7554/eLife.03005.
-
Elkin, L. A., Kay, M., Higgins, J. J. and Wobbrock, J. O. (2021). An aligned rank transform procedure for multifactor contrast tests, in: UIST’21: the 34th Annual ACM Symposium on User Interface Software and Technology (UIST 2021), pp. 754–768. DOI:10.1145/3472749.3474784.
-
Ernst, M. O. and Bülthoff, H. H. (2004). Merging the senses into a robust percept, Trends Cogn. Sci. 8, 162–169. DOI:10.1016/j.tics.2004.02.002.
-
Feng, J., Kim, J., Luu, W. and Palmisano, S. (2019). Method for estimating display lag in the Oculus Rift S and CV1, in: SA’19: SIGGRAPH Asia 2019 Posters, pp. 1–2. DOI:10.1145/3355056.3364590.
-
Fetsch, C. R., Turner, A. H., DeAngelis, G. C. and Angelaki, D. E. (2009). Dynamic reweighting of visual and vestibular cues during self-motion perception, J. Neurosci. 29, 15601–15612. DOI:10.1523/JNEUROSCI.2574-09.2009.
-
Fetsch, C. R., DeAngelis, G. C. and Angelaki, D. E. (2013). Bridging the gap between theories of sensory cue integration and the physiology of multisensory neurons, Nat. Rev. Neurosci. 14, 429–442. DOI:10.1038/nrn3503.
-
Gallagher, M. and Ferrè, E. R. (2018). Cybersickness: a multisensory integration perspective, Multisens. Res. 31, 645–674. DOI:10.1163/22134808-20181293.
-
Gallagher, M., Dowsett, R. and Ferrè, E. R. (2019). Vection in virtual reality modulates vestibular-evoked myogenic potentials, Eur. J. Neurosci. 50, 3557–3565. DOI:10.1111/ejn.14499.
-
Gallagher, M., Choi, R. and Ferrè, E. R. (2020). Multisensory interactions in virtual reality: optic flow reduces vestibular sensitivity, but only for congruent planes of motion, Multisens. Res. 33, 625–644. DOI:10.1163/22134808-20201487.
-
Garzorz, I. T. and MacNeilage, P. R. (2017). Visual-vestibular conflict detection depends on fixation, Curr. Biol. 27, 2856–2861. DOI:10.1016/j.cub.2017.08.011.
-
Grabherr, L., Nicoucar, K., Mast, F. W. and Merfeld, D. M. (2008). Vestibular thresholds for yaw rotation about an Earth–vertical axis as a function of frequency, Exp. Brain Res. 186, 677–681. DOI:10.1007/s00221-008-1350-8.
-
Greenlee, M. W., Frank, S. M., Kaliuzhna, M., Blanke, O., Bremmer, F., Churan, J., Cuturi, L. F., MacNeilage, P. R. and Smith, A. T. (2016). Multisensory integration in self motion perception, Multisens. Res. 29, 525–556. DOI:10.1163/22134808-00002527.
-
Heerspink, H., Berkouwer, W., Stroosma, O., van Paassen, R., Mulder, M. and Mulder, B. (2005). Evaluation of vestibular thresholds for motion detection in the SIMONA research simulator, in: AIAA Modeling and Simulation Technologies Conference and Exhibit, p. 6502. DOI:10.2514/6.2005-6502.
-
Holten, V. and MacNeilage, P. R. (2018). Optic flow detection is not influenced by visual-vestibular congruency, PLoS ONE 13, e0191693. DOI:10.1371/journal.pone.0191693.
-
Jaekl, P. M., Jenkin, M. R. and Harris, L. R. (2005). Perceiving a stable world during active rotational and translational head movements, Exp. Brain Res. 163, 388–399. DOI:10.1007/s00221-004-2191-8.
-
Jerald, J., Whitton, M. and Brooks, F. P. (2012). Scene–motion thresholds during head yaw for immersive virtual environments, ACM Trans. Appl. Percept. 9, 1–23. DOI:10.1145/2134203.2134207.
-
Kaliuzhna, M., Prsa, M., Gale, S., Lee, S. J. and Blanke, O. (2015). Learning to integrate contradictory multisensory self-motion cue pairings, J. Vis. 15, 10. DOI:10.1167/15.1.10.
-
Kaliuzhna, M., Ferrè, E. R., Herbelin, B., Blanke, O. and Haggard, P. (2016). Multisensory effects on somatosensation: a trimodal visuo-vestibular-tactile interaction, Sci. Rep. 6, 26301. DOI:10.1038/srep26301.
-
Kay, M., Elkin, L. A. and Wobbrock, J. O. (2021). Contrast tests with ART. ARTool [R] package vignette. Updated October 12, 2021.
-
Keshavarz, B. and Hecht, H. (2011). Validating an efficient method to quantify motion sickness, Hum. Factors 53, 415–426. DOI:10.1177/0018720811403736.
-
Keshavarz, B., Riecke, B. E., Hettinger, L. J. and Campos, J. L. (2015). Vection and visually induced motion sickness: how are they related?, Front. Psychol. 6, 472. DOI:10.3389/fpsyg.2015.00472.
-
Kim, J. and Palmisano, S. (2010). Visually mediated eye movements regulate the capture of optic flow in self-motion perception, Exp. Brain Res. 202, 355–361. DOI:10.1007/s00221-009-2137-2.
-
Kim, J., Chung, C. Y. L., Nakamura, S., Palmisano, S. and Khuu, S. K. (2015). The Oculus Rift: a cost-effective tool for studying visual-vestibular interactions in self-motion perception, Front. Psychol. 6, 248. DOI:10.3389/fpsyg.2015.00248.
-
Kim, J., Luu, W. and Palmisano, S. (2020). Multisensory integration and the experience of scene instability, presence and cybersickness in virtual environments, Comput. Hum. Behav. 113, 106484. DOI:10.1016/j.chb.2020.106484.
-
Lambrey, S. and Berthoz, A. (2003). Combination of conflicting visual and non-visual information for estimating actively performed body turns in virtual reality, Int. J. Psychophysiol. 50, 101–115. DOI:10.1016/S0167-8760(03)00127-2.
-
LaViola, J. J. (2000). A discussion of cybersickness in virtual environments, ACM Sigchi Bull. 32, 47–56. DOI:10.1145/333329.333344.
-
Moroz, M., Garzorz, I., Folmer, E. and MacNeilage, P. (2019). Sensitivity to visual speed modulation in head-mounted displays depends on fixation, Displays 58, 12–19. DOI:10.1016/j.displa.2018.09.001.
-
Oman, C. M. (1990). Motion sickness: a synthesis and evaluation of the sensory conflict theory, Can. J. Physiol. Pharmacol. 68, 294–303. DOI:10.1139/y90-044.
-
Palmisano, S., Mursic, R. and Kim, J. (2017). Vection and cybersickness generated by head-and-display motion in the Oculus Rift, Displays 46, 1–8. DOI:10.1016/j.displa.2016.11.001.
-
Paludan, A., Elbaek, J., Mortensen, M., Zobbe, M., Nilsson, N. C., Nordahl, R., Reng, L. and Serafin, S. (2016). Disguising rotational gain for redirected walking in virtual reality: effect of visual density, in: 2016 IEEE Virtual Reality (VR), pp. 259–260. DOI:10.1109/VR.2016.7504752.
-
Prsa, M., Gale, S. and Blanke, O. (2012). Self-motion leads to mandatory cue fusion across sensory modalities, J. Neurophysiol. 108, 2282–2291. DOI:10.1152/jn.00439.2012.
-
Ragan, E. D., Scerbo, S., Bacim, F. and Bowman, D. A. (2017). Amplified head rotation in virtual reality and the effects on 3d search, training transfer, and spatial orientation, IEEE Trans. Vis. Comput. Graph. 23, 1880–1895. DOI:10.1109/TVCG.2016.2601607.
-
Reason, J. T. (1978). Motion sickness adaptation: a neural mismatch model, J. R. Soc. Med. 71, 819–829.
-
Reason, J. T. and Brand, J. J. (1975). Motion Sickness. Academic Press, London, UK.
-
Riecke, B. E. and Schulte-Pelkum, J. (2013). Perceptual and cognitive factors for self-motion simulation in virtual environments: how can self-motion illusions (“vection”) be utilized?, in: Human Walking in Virtual Environments, F. Steinicke, Y. Visell, J. Campos and A. Lécuyer (Eds), pp. 27–54. Springer, New York, NY, USA. DOI:10.1007/978-1-4419-8432-6_2.
-
Sachgau, C., Chung, W. and Barnett-Cowan, M. (2018). Perceived timing of active head movement at different speeds, Neurosci. Lett. 687, 253–258. DOI:10.1016/j.neulet.2018.09.065.
-
Sanders, M. C., Chang, N.-Y. N., Hiss, M. M., Uchanski, R. M. and Hullar, T. E. (2011). Temporal binding of auditory and rotational stimuli, Exp. Brain Res. 210, 539–547. DOI:10.1007/s00221-011-2554-x.
-
Wobbrock, J. O., Findlater, L., Gergle, D. and Higgins, J. J. (2011). The aligned rank transform for nonparametric factorial analyses using only ANOVA procedures, in: CHI’11: Proceedings of the SIGGHI Conference on Human Factors in Computing Systems (CHI 2011), pp. 143–146. DOI:10.1145/1978942.1978963.
| All Time | Past Year | Past 30 Days | |
|---|---|---|---|
| Abstract Views | 1071 | 496 | 48 |
| Full Text Views | 67 | 29 | 1 |
| PDF Views & Downloads | 108 | 49 | 0 |
Influence of Sensory Conflict on Perceived Timing of Passive Rotation in Virtual Reality
In: Multisensory ResearchSearch for other papers by William Chung in
Current site
Google Scholar
PubMed
Search for other papers by Michael Barnett-Cowan in
Current site
Google Scholar
PubMed
Abstract
Integration of incoming sensory signals from multiple modalities is central in the determination of self-motion perception. With the emergence of consumer virtual reality (VR), it is becoming increasingly common to experience a mismatch in sensory feedback regarding motion when using immersive displays. In this study, we explored whether introducing various discrepancies between the vestibular and visual motion would influence the perceived timing of self-motion. Participants performed a series of temporal-order judgements between an auditory tone and a passive whole-body rotation on a motion platform accompanied by visual feedback using a virtual environment generated through a head-mounted display. Sensory conflict was induced by altering the speed and direction by which the movement of the visual scene updated relative to the observer’s physical rotation. There were no differences in perceived timing of the rotation without vision, with congruent visual feedback and when the speed of the updating of the visual motion was slower. However, the perceived timing was significantly further from zero when the direction of the visual motion was incongruent with the rotation. These findings demonstrate the potential interaction between visual and vestibular signals in the temporal perception of self-motion. Additionally, we recorded cybersickness ratings and found that sickness severity was significantly greater when visual motion was present and incongruent with the physical motion. This supports previous research regarding cybersickness and the sensory conflict theory, where a mismatch between the visual and vestibular signals may lead to a greater likelihood for the occurrence of sickness symptoms.
| All Time | Past Year | Past 30 Days | |
|---|---|---|---|
| Abstract Views | 1071 | 496 | 48 |
| Full Text Views | 67 | 29 | 1 |
| PDF Views & Downloads | 108 | 49 | 0 |