Studies of the illusory sense of self-motion elicited by a moving visual surround (‘vection’) have revealed key insights about how sensory information is integrated. Vection usually occurs after a delay of several seconds following visual motion onset, whereas self-motion in the natural environment is perceived immediately. It has been suggested that this latency relates to the sensory mismatch between visual and vestibular signals at motion onset. Here, we tested three techniques with the potential to reduce sensory mismatch in order to shorten vection onset latency: noisy galvanic vestibular stimulation (GVS) and bone conducted vibration (BCV) at the mastoid processes, and body vibration applied to the lower back. In Experiment 1, we examined vection latency for wide field visual rotations about the roll axis and applied a burst of stimulation at the start of visual motion. Both GVS and BCV reduced vection latency by two seconds compared to the control condition, whereas body vibration had no effect on latency. In Experiment 2, the visual stimulus rotated about the pitch, roll, or yaw axis and we found a similar facilitation of vection by both BCV and GVS in each case. In a control experiment, we confirmed that air-conducted sound administered through headphones was not sufficient to reduce vection onset latency. Together the results suggest that noisy vestibular stimulation facilitates vection, likely due to an upweighting of visual information caused by a reduction in vestibular sensory reliability.
Recent developments in graphics technologies permit real-time rendering of complex three-dimensional environments that approximate the natural world with a high degree of realism (see Scarfe and Glennerster, 2015, for a review). Virtual reality (VR) environments can be rendered to provide stereoscopic views with a veridical centre of projection at high refresh rates and with low motion-to-photon latency. These technological advancements continue at a rapid rate (e.g., Friston et al., 2016; Greer et al., 2016). Developments in this area are likely to be crucial for bridging the gap between real and artificial conditions that can impair the ability for participants to gather information and perform naturally in the virtual world (Riecke, 2011; Slater, 2009). Researchers have taken advantage of this emerging technology in myriad studies of human perception-action coupling in VR that appear to generalize well to real-world situations (Jain and Backus, 2010; Linkenauger et al., 2015; Vignais et al., 2015; also see Hardiess et al., 2015, and Wilson and Soranzo, 2015, for reviews).
Although these studies shine a new light on the process by which sensory information guides action, there are a number of physical differences and resulting perceptual discrepancies between the real world and the virtual worlds that are commonly used in studies of human perception and action. The physical differences between natural and VR conditions can be exemplified by the following: mismatches in VR displays between accommodation and vergence cues (e.g., Hoffman et al., 2008; Wann et al., 1995); latencies between self-motion and the response of the visual stimulus to these movements; spatial resolution issues; and mismatches between multimodal sensory cues (note this is not a comprehensive list). As a result, perception in VR differs in many ways from the way we perceive the real world. Examples include: consistent underestimation observed when evaluating egocentric distances in VR settings (Loomis and Knapp, 2003; Willemsen and Gooch, 2002); simulator sickness produced by VR tasks that are not nauseogenic in the real world (Sharples et al., 2008); the deforming effect of display latency on the visual 3D environment (Deering, 1992); and differences between self-motion perception in the virtual world compared with the natural environment (McCauley and Sharkey, 1992). The process of linking the various physical discrepancies between VR and real world settings with the perceptual or physiological differences observed between these conditions has been the focus of much attention. A number of hypotheses exist, for example, that link sensory mismatch in VR with self-motion perception and simulator sickness (for a review see Shupak and Gordon, 2006).
Given the growth and future potential for VR in studying naturalistic human behaviour, the characteristics of self-motion perception in VR are of particular importance. In the natural environment, active or passive movement of the body through space results in immediate perception of body motion (Dichgans and Brandt, 1978). The most often studied case involves visually evoked illusions of self-motion, known as ‘vection’, which emerged from a rich history of research that demonstrates the robust link between visual flow and control of bodily posture. Seminal work by Gibson (1950, 1966) outlined the basis for the specification of body movement through optic flow, and a series of contemporaries went on to provide compelling examples of the laws that Gibson described. Examples include the elegant demonstration by Lishman and Lee (1973) showing that a room that swayed around a stationary observer could give rise to illusory self-motion and coherent postural responses (also see Lee and Aronson, 1974; Lee and Lishman, 1975). Vection was also documented by Johansson (1977), where the phenomenon was called the ‘elevator illusion’ since illusory upwards self-motion was evoked by downward optic flow. A similar dependency between visual motion and perceived body movement was shown by Warren (1976), who used optic flow consisting of a simple dot display to show that observers perceived vection that felt similar to real movement.
Real self-motion is associated with cues from senses other than vision, including auditory, haptic, or proprioceptive signals. Self-motion illusions are facilitated by auditory (Väljamäe, 2009) and haptic movement cues (Riecke et al., 2008). Entirely non-visual illusions of self-motion have been demonstrated, including so-called ‘auditory vection’ produced by auditory cues that imply self-motion (Lackner, 1977; Väljamäe et al., 2004). Body stimulation in blindfolded participants is also sufficient to produce self-motion illusions, termed ‘haptokinetic vection’ for tactile stimulation that implies motion (informal observations were recorded by Dichgans and Brandt, 1978) and ‘arthrokinetic vection’ for tonic limb rotation (Brandt et al., 1977). Even ‘vestibular vection’ has been identified in cases where the vestibular organs are stimulated using a caloric method (Fasold et al., 2002).
The immediacy of self-motion produced by real motion is not observed in the case of vection. The latencies between the onset of visual motion and the establishment of a sense of self-motion typically range between one and ten seconds, depending on how the visual stimulus is rendered and presented. For example, vection tends to occur faster and feels stronger for roll rotation than for pitch rotations, and is experienced more quickly for pitch rotations than for yaw rotations, although this pattern can differ depending on the mode of presentation (Tanahashi et al., 2012; Ujike et al., 2004). The latency between visual motion onset and the impression of self-motion has the potential to drastically alter the way in which participants act in tasks involving VR.
Researchers have identified a potential cause for vection onset latency in the mismatch between visual and non-visual sensory cues at the onset of the motion stimulus (e.g., Flanagan et al., 2004; Israël and Warren, 2005; Wong and Frost, 1978, 1981). Perceiving the degree to which ones’ own body is moving mainly relies on an integration of visual and vestibular information (Angelaki et al., 2011; Israël and Warren, 2005). A variety of human psychophysics studies and macaque imaging studies, as summarized by Greenlee and colleagues (2016), show that self-motion perception is likely served by interconnected populations of neurons that respond to both visual and vestibular cues in particular.
Behavioural studies often probe this self-motion network by inducing vection in observers. The classic conditions required to induce vection involve the presentation of a large-field visual stimulus that specifies the direction and magnitude of virtual self-motion through optic flow. Given that the participant is stationary in space (often seated in a chair in front of a screen or computer monitor), the vestibular organs do not receive the corroborating activation that would arise if the participant actually started to move. Sensory mismatch occurs at visual motion onset, when the visual stimulus suggests acceleration into self-motion but no accelerations are detected by the vestibular organs (when the visual stimulus is moving at constant velocity, no mismatch occurs). Eventually an observer acquires sufficient evidence that they are likely to be in motion from the visual cues, and the feeling of vection takes hold (Israël and Warren, 2005). The relation between sensory mismatch and vection has been supported by research that shows that physical rotation of an observer at visual motion onset results in a decrease in vection onset latency, although this effect was only produced when visual and vestibular cues were coherent in their direction (Brandt et al., 1974; Riecke et al., 2006; Wong and Frost, 1981). Schulte-Pelkum (2007) confirmed the results of Wong and Frost (1981) where a vestibular ‘kick’ through body rotation caused a large reduction in latency for linear vection in a VE. On the other hand, conflicting visual and vestibular cues seem to suppress rather than enhance vection (Ash and Palmisano, 2012; Lackner and Teixeira, 1977; Young et al., 1973). These results support the idea that vection onset latency can be attributed to the delay required by the nervous system to acquire sufficient visual self-motion information in order to disregard the sensory mismatch.
The idea that visual-vestibular mismatch underlies vection onset latency gained further support from studies on vestibular dysfunction patients. Patients with a low vestibular sensitivity to a specific direction of head rotation show decreased vection latency for that direction compared to other directions (Wong and Frost, 1981). A similar negative relationship between vestibular threshold and vection onset latency was identified in a healthy population (Lepecq et al., 1999). This finding highlights the importance of cue uncertainty in guiding the decision about whether the body has moved or not. Participants with vestibular deficits appear to rely strongly upon visual motion signals to decide whether or not the body is likely to be in motion, given that they cannot rely on information provided by the vestibular sense.
Other research has shown that the use of galvanic vestibular stimulation (GVS) in VR can strongly influence the strength of vection experienced by users. This technique involves applying a small direct current to stimulate the vestibular system, normally via electrodes placed at the mastoid processes (Curthoys and MacDougall, 2012; Day and Fitzpatrick, 2005; Swaak and Oosterveld, 1975). Cress and colleagues (1997) showed that GVS can increase vection magnitude if applied during observation of a vection-inducing visual stimulus. As well, Lepecq and colleagues (2006) provided evidence that directional GVS can modify the perceived direction of illusory self-motion. This research is in line with the studies discussed above that show facilitation of vection when visually-consistent body rotation is applied at the onset of visual motion (Brandt et al., 1974; Riecke et al., 2006; Wong and Frost, 1981).
The research discussed above has shown that vection latency is shorter when the body is physically moved to corroborate visual motion cues. Likewise, GVS has been shown to influence vection if the stimulation is congruent with visual motion. On the other hand, several studies have indicated that in some cases sensory mismatch is irrelevant to the experience of vection, and may even lead to enhanced vection sensation (Ash and Palmisano, 2012; Palmisano and Keane, 2004; Palmisano and Kim, 2009; Palmisano et al., 2000, 2011). For example, introducing visually simulated viewpoint jitter into a visual stimulus can enhance vection despite introducing a significant degree of visual-vestibular conflict. These findings are intriguing as they appear to contradict a variety of the literature cited here, and the effects have been shown to be robust to changes in experimental methodology and instruction (Palmisano and Chan, 2004). It is possible that viewpoint jitter increases the compelling nature of vection by adding ecological validity to the visual signals (see Palmisano et al., 2011, for a discussion). However, viewpoint jitter that does not cause a sensory conflict is associated with stronger vection (Ash and Palmisano, 2012). The mixed evidence described here shows the need to further examine the relationship between vection and sensory mismatch.
In the current study, we wanted to test if noisy, non-directional vestibular signals which are neither congruent nor incongruent but mask the vestibular information transfer can facilitate quicker vection. The motivation for this study was based on principles of multisensory integration. The process of resolving self-motion is thought to be guided by sensory cues that are integrated based on their reliability (see Ernst and Banks, 2002; Landy et al., 1995). When two cues are in conflict an estimate of the reliability of the cues may be used in order to come to an appropriate decision. This theory predicts that manipulating the reliability of the vestibular sense by applying noise to it would cause the nervous system to disregard the conflicting vestibular signals at motion onset, and instead to assign more weight to visual motion cues. Consequently, noisy vestibular stimulation applied at visual motion onset is expected to lead to shorter vection latency.
To date, no study has tested the proposition that noisy stimulation of the vestibular system could decrease vection latency. We designed Experiment 1 to test this hypothesis, taking advantage of two types of noisy stimulation to the vestibular system — galvanic vestibular stimulation (GVS) and bone conducted vibration (BCV) applied at the mastoid processes. The effects of GVS on vection are well studied, as described above. BCV has been shown to affect the vestibular system (e.g., Curthoys et al., 2014), but our research marks the first test of the effects of BCV on vection. GVS remains the standard for vestibular stimulation in studies of vection and here we aimed to test if BCV has similar effects to GVS. It is important to note that there are several considerations involved when using GVS: GVS use requires significant technical expertise, and it is considered an invasive technique that is not appropriate for all users (Lenggenhager et al., 2008). On the other hand, BCV is easy to employ and non-invasive. We also included a condition where we applied a non-vestibular stimulation by vibrating the lower back using a cushion (body vibration; BV). Body vibration has been previously shown to enhance the perceived magnitude of vection, but its effect on vection latency is unclear (Riecke et al., 2008). In Experiment 1, we also included a control condition where no stimulation was applied. Our prediction was that noisy vestibular stimulation will reduce sensory mismatch through downweighting vestibular and upweighting visual cues, facilitating shorter vection onset latency as a result. We also predicted that body vibration, which does not involve stimulating the vestibular system, will not cause a reduction in vection onset latency compared to the control condition. While we only used visual motion about the roll axis in Experiment 1, we added pitch and yaw motion in Experiment 2. As an important control, we then conducted Experiment 3 to test if sound administered through headphones affects vection latency.
2. Experiment 1
Research has shown that vestibular stimulation that corroborates visual motion can influence vection. However, corroborating stimulation should not be required if the aim is simply to reduce the influence of conflicting vestibular motion signals. We designed Experiment 1 in order to assess the effect of noisy vestibular stimulation on vection onset latency. Our hypothesis was that noisy stimulation would reduce the influence of the vestibular sense in self-motion perception due to increased sensory uncertainty. Assuming Bayesian cue integration (Ernst and Banks, 2002; Greenlee et al., 2016; Landy et al., 1995), we posited that reduced reliability for vestibular signals will lead visual cues to dominate self-motion perception. As a consequence, we expected to observe a reduction in vection latency when noisy stimulation is applied to the vestibular system.
2.1. Material and Methods
We tested the effects of three methods of stimulation during the experiment. The techniques we used are outlined below.
2.1.1. Noisy Galvanic Vestibular Stimulation
A small direct current applied to the mastoids stimulates the vestibular system, producing postural and oculomotor responses (Curthoys and MacDougall, 2012; Day and Fitzpatrick, 2005; Swaak and Oosterveld, 1975). In particular, for a coherent bipolar GVS signal applied at the mastoid processes, participants tend to lean towards the anodal stimulus and away from the cathodal stimulus (Pavlik et al., 1999). The method causes an excitation of vestibular afferents on the cathodal side of stimulation, while an inhibition occurs on the anodal side (Goldberg et al., 1984). Alternatively, cathodal current can be applied at both mastoids with the anode being placed at C7 above the spine, resulting in a lean in the anterior-posterior direction (Pal et al., 2009). Stimulation can also be applied in the form of a signal that randomly varies in amplitude, termed stochastic or noisy GVS (Pal et al., 2009; Pavlik et al., 1999). GVS has been used often to explore the relationship between the vestibular sense and control of the body (see for example: Benson et al., 1986; Fitzpatrick et al., 1999; Goldberg et al., 1984; Nashner and Wolfson, 1974; St George and Fitzpatrick, 2011). Although there is some controversy regarding what GVS actually stimulates — either primarily otolithic neurons (Cohen et al., 2012), or semicircular canal afferent neurons (Fitzpatrick and Day, 2004; Reynolds and Osler, 2012) — the common consensus is that it stimulates a combination of both types of neuron (Curthoys and MacDougall, 2012; Day and Fitzpatrick, 2005; Kim and Curthoys, 2004; Wardman and Fitzpatrick, 2002).
Research has shown that the nervous system interprets noisy GVS differently depending on the magnitude and frequency of stimulation. In one study, low-current noisy GVS stimulation (<0.5 mA with a power spectrum) was observed to produce an increase in body stability for Parkinson’s patients (Pal et al., 2009). The authors have claimed that this stability increase occurred due to the principle of stochastic resonance. This theory states that signal detection can be improved by addition of a small amount of noise to the system, due to the fact that a summation of the noise and the true signal is more likely to breach the sensory threshold (Moss et al., 2004). Stochastic resonance has been termed a ‘counterintuitive’ phenomenon, because the addition of noise to a system will tend to lower signal detection (McDonnell and Abbott, 2009). We wished to avoid the possibility of improving signal detection using noise in our experiment, as we were primarily interested in assessing whether reduced sensory reliability can facilitate vection. The enhanced signal detection resulting from stochastic resonance depends upon low intensity stimulation of the vestibular organs, identified as <0.5 mA by Pal and colleagues (2009), while greater current will disrupt signal detection through increasing the signal-to-noise ratio. In order to ensure that we did not facilitate signal detection, we used current levels that were approximately four times greater than the upper limit of currents administered by Pal and colleagues (2009).
We used a Vestibulator (Draisey and Mullins, 2004; Good Vibrations Engineering, King City, ON, Canada) to electrically stimulate the vestibular system for one second per trial. In each trial we generated a unique zero-mean noise signal with a type power spectrum and maximum amplitude of ±1.96 milliamperes. The noise signal for that trial was then delivered at 40 Hz using a host program on the workstation. We prepared the skin over the left and right mastoid processes by gently rubbing the skin at the application site with alcohol using cotton pads, and attached 2 cm2 electrodes with conductive gel to each mastoid process. Participants of each experiment were not informed about the perceptual effects of GVS.
2.1.2. Bone Conducted Vibration
Bursts of bone conducted vibration (BCV) applied at either the forehead (Fz) or the mastoid processes stimulate the vestibular system by causing small linear accelerations of the utricle and the saccule (Curthoys et al., 2014; Sheykholeslami et al., 2001; Todd et al., 2000). The stimulation triggers large vestibular evoked myogenic potentials (VEMPs) in the sternocleidomastoid muscle (Sheykholeslami et al., 2000) and in the extraocular muscles (Rosengren et al., 2005). Predictable eye movements result from BCV stimulation (Cornell et al., 2009, 2015; Manzari et al., 2010). The effectiveness of stimulation depends on the frequency of vibration used, with research showing a well-defined frequency tuning range for BCV bursts whereby tones between 200 and 500 Hz produce the largest myogenic potentials (Sheykholeslami et al., 2001; Todd et al., 2000; Townsend and Cody, 1971).
We applied clinical bone vibrators (Radioear B71, New Eagle, PA, USA) to the left and right mastoid processes (Fig. 1A). The vibrators operated at a frequency of 500 Hz for one second per trial. The voltage signal used to drive the vibrators was generated with MATLAB and delivered using a sound card attached to a custom-built audio amplifier. An elasticated head-band secured the vibrators in place on the surface of the skin (Fig. 1B).
The degree to which BCV stimulates the vestibular sense depends on a multitude of factors, including the location of application, the size of the head of the participant, the shape of the skull, and the anatomy of the vestibular labyrinth for an individual participant (Blanks et al., 1975; Curthoys et al., 2009). It is therefore important to tailor the magnitude of stimulation for each participant so as to ensure comfortable but consistent vestibular stimulation. We selected a standard BCV magnitude based on the lowest magnitude of stimulation that reliably produced a small degree of oscillopsia in a separate group of participants that we had assessed in a pilot study. Before each experimental session we stimulated each participant with the standard magnitude of BCV, and asked if they could tolerate this magnitude. If they could not, we decreased the magnitude by small increments until it reached a tolerable level. Participants of each experiment were not informed about the perceptual effects of BCV.
2.1.3. Body Vibration
We used a vibrating cushion (Interactor Cushion, Aura Systems Inc., El Segundo, CA, USA) to apply vibration to the lower back of the participant at a frequency of 62.5 Hz for one second per trial. We selected this frequency as it produced the most intense sensation of vibration that we could achieve with our equipment. The vibrating cushion produced an audible sound at this frequency when activated. We were concerned that the additional auditory stimulation might have influenced whether or not participants experienced self-motion. To mitigate this influence, we ensured that participants wore noise-canceling headphones (Logitech Unreal Ears 6000) during trials in this condition.
We recruited 12 participants (10 were female) from a graduate student mailing list at Queen’s University. Each participant took part in an individual session lasting approximately one hour. We compensated each participant $10 per hour. Mean age was 23.08 (SD of 2.02). All had normal or corrected to normal vision. Each participant gave informed written consent before the study in accordance with the Declaration of Helsinki.
We created the visual stimulus using the OpenGL library of the Psychophysics Toolbox (3.0.10; Brainard, 1997) in MATLAB (Version 2011a). The stimulus consisted of a pattern of 300 white cubes (side length 4 cm) that were uniformly positioned in the environment at simulated distances of between 60–100 cm from the centre of the virtual space. Density of the cubes was 0.33 per square degree of visual angle. The centre of the virtual space was aligned with the viewpoint of the observer. When set into motion, the stimulus accelerated instantly to 30 degrees per second angular velocity. The stimulus rotated about a point at eye height straight ahead of the observer. In half of the trials the stimulus rotated clockwise and in the other half the stimulus rotated counter-clockwise.
We projected the virtual environment onto three large screens that surrounded the participant (see Fig. 2). The centre of projection was half way between the two lateral screens and 100 cm in front of the central screen at a height of 135 cm above the floor. Position and height of the chair was adjusted such that the observer’s eyes were located at the centre of projection. The projection subtended approximately 86 degrees of visual angle vertically, and approximately 285 degrees of visual angle horizontally. The cubes that comprised the stimulus subtended approximately 2.30–4.00 degrees of visual angle. We used short-throw projectors located above each screen to display the visual stimulus at 60 frames per second.
The experiment consisted of three stimulation conditions that we ran within-subjects: bone conducted vibration (BCV), galvanic vestibular stimulation (GVS), and body vibration (BV). The fourth condition, the control, consisted of no stimulation. We replicated each condition 10 times (five clockwise and five counter-clockwise stimulus rotations) resulting in a total of 40 trials. The experiment lasted approximately one hour including introduction and debriefing.
The design of the study was blocked by condition and we ordered the presentation of blocks using a Latin square design. The order of the trials within a block was randomized.
We measured latency of vection after visual motion onset by asking participants to press a button on a wireless remote control (Nintendo Wii Remote) when they experienced vection. The responses of participants were recorded in MATLAB using the WiiLab library (Brindza et al., 2009).
The participant entered the room and the experimenter seated him/her on a chair located 100 cm from the central projection screen. The height of the chair was adjusted to align the line of sight of the participant with the axis of stimulus rotation that was located 135 cm above the floor. The experimenter requested that the participant should rest their feet on the footrest that was attached to the chair at all times. Next the experimenter explained the task and described the perception of self-motion that could arise for the participant. The experimenter gave the participant the wireless remote control and instructed the participant that during trials they may experience the illusion that their body was in motion. The task was described as following: “Push the button on the remote if you feel the illusion of yourself moving. Hold down the button until the sensation ceases or until the trial ends.” Movement of the participants was not restrained by the use of a chinrest or a bite bar, but participants were asked not to move. Given the size of the apparatus, small deviations from the desired head position (in the axis of rotation) would only cause very minor distortion of the projection.
Each trial began with the presentation of a static frame of the visual stimulus. Next, the experimenter pressed a button on the keyboard to begin recording participant responses. Five seconds later, the visual stimulus began to move. At visual motion onset, stimulation was applied according to the condition. GVS, BCV, and BV were each applied for 1 s per trial at visual motion onset. If the participant pressed the button on the wireless remote to indicate vection onset, the stimulus continued to move for another 10 s before the trial ended. If the participant did not report vection at any time, the motion stimulus would terminate after 60 s, although this did not occur in any trials in our experiments. At the end of each trial the experimenter pressed a key on the keyboard to progress, and the experiment continued until all trials had been completed.
2.2. Experiment 1: Results and Discussion
Compared to the control condition, vection onset latency decreased by approximately 40% for both the BCV and GVS conditions, but did not decrease in the BV condition. A repeated-measures analysis of variance applied to the latency data showed that the type of stimulation influenced the latency of vection onset, , , . We conducted follow-up tests on the four types of stimulation using estimated marginal means. In both the GVS and BCV conditions, latencies were shorter than in the BV condition (GVS vs BV, ; BCV vs BV, ). These two conditions did not differ from each other (). Also, the BV condition did not differ from the control condition (). The latency data are plotted in Fig. 3.
We conclude that noisy vestibular stimulation facilitates vection. Our data support our hypothesis and indicate that sensory mismatch was reduced through noisy vestibular stimulation.
These results provide the first indication that noisy vestibular stimulation influences vection onset latency. As well, the findings mark the first time that the relatively non-invasive method of BCV has been shown to impart an influence on self-motion perception.
The findings of Experiment 1 suggest that adding noise to the vestibular system through the otoliths (BCV; Curthoys et al., 2014) or non-specific vestibular afferents (GVS; Curthoys and MacDougall, 2012) reduces vection onset latency. It could have been the case that noisy stimulation is effective for reducing roll vection latency, but might not have the same effect for pitch or yaw vection. We decided to replicate Experiment 1 for other types of circular vection to see if the effect of noisy stimulation depends on the visual rotation axis. To this end we conducted a second experiment where visual stimuli were rotated about the roll, pitch, and yaw axis.
3. Experiment 2
3.1. Material and Methods
We recruited 12 participants (nine were female) from a graduate student mailing list at Queen’s University. Each participant took part in an individual session lasting approximately one hour. We compensated each participant $10 per hour. Mean age was 23.75 (SD of 3.14). All had normal or corrected to normal vision. Each participant gave informed written consent before the study in accordance with the Declaration of Helsinki.
3.1.2. Stimuli, Design, and Procedure
The stimuli, overall design, and procedure were similar to those in Experiment 1. However, we added two rotation axes for the visual stimulus — yaw and pitch — and removed the condition where no stimulation was applied. Instead, we considered the body vibration condition to be a control condition given that it produced similar results to the control in Experiment 1. We ran each condition 10 times (five clockwise, five counter-clockwise visual rotations) to result in 90 trials. The experiment lasted approximately 1 h 30 min including introduction and debriefing.
3.2. Experiment 2: Results and Discussion
In Experiment 2 we obtained a similar pattern of data for pitch, roll, and yaw vection to that found in Experiment 1 for roll vection (Fig. 4). We ran a two-way repeated measures analysis of variance (ANOVA) on the within-subjects factors of axis (pitch, roll, and yaw) and stimulation (BCV, GVS, and BV). The analysis revealed a main effect of stimulation on vection latency, , , , a main effect of rotation axis, , , , and an interaction between the stimulation and rotation axis factors, , , .
We followed up the significant interaction by first testing for differences in vection latency between stimulation conditions separately for the pitch, roll, and yaw rotation axes. For pitch, there was a significant simple main effect of stimulation, , , , and follow up pairwise comparisons showed that BCV and GVS were associated with lower vection latency than BV (), but BCV and GVS were not different from each other ().
For roll, we also found a significant simple main effect of stimulation, , , , and follow up pairwise comparisons showed that BCV and GVS were associated with lower vection latency than BV ( and , respectively), but BCV and GVS were not different from each other ().
For yaw, again we found a significant simple main effect of stimulation, , , , and follow up pairwise comparisons showed the same pattern we had seen already for pitch and roll: BCV and GVS were associated with lower vection latency than BV (), but BCV and GVS were not different from each other ().
Next we conducted a simple main effect analysis on the effect of axis on vection latency separately for the BCV, GVS, and BV stimulation conditions.
For BCV, we found a significant simple main effect of axis, , , , and follow up pairwise comparisons showed that roll and pitch rotation was associated with significantly lower vection latency than yaw rotation (). Pitch and roll did not differ from each other ().
For GVS, we found a significant simple main effect of axis, , , , and follow up pairwise comparisons showed that roll rotation was associated with significantly lower vection latency than pitch rotation (), which was in turn associated with significantly lower vection latency than yaw rotation ().
For BV, we found a significant simple main effect of axis, , , , and follow up pairwise comparisons showed that roll rotation was associated with significantly lower vection latency than pitch rotation (), which was in turn associated with significantly lower vection latency than yaw rotation ().
The results of Experiment 2 indicated that BCV and GVS did not differ in terms of vection latency across rotation axes. This suggests that the effect of stimulation takes place through a mechanism shared by both BCV and GVS. We conclude that noisy stimulation reduced sensory reliability for the vestibular system. This reduces the weight of vestibular signals and increases the weight of vision, meaning that sensory mismatch could be resolved easily by ignoring unreliable vestibular cues.
In Experiment 2 we replicated the effect observed in Experiment 1 for pitch, roll, and yaw axes of rotation. Yaw rotation resulted in clearly higher vection latency on average than for the other two axes, and pitch trials tended to be associated with slightly increased vection latency compared to roll trials. In terms of these differences between visual rotation axes, our results are consistent with some previous findings (e.g., Ujike et al., 2004), but inconsistent with a study where no differences were observed between yaw and roll (Tanahashi et al., 2012). Although yaw rotation tended to produce longer vection latency than other axes, the latency we observed was similar or shorter to that found in other studies (see for example, Fushiki et al., 1999; Schulte-Pelkum, 2007). Vection latency did not differ between the pitch and roll trials for the BCV condition, but the latency for pitch and roll trials differed in the BV and GVS conditions: roll rotation was associated with lower vection latency than pitch rotation.
While effects of GVS on properties of vection have been documented before, our finding that BCV has similar effects to GVS on vection latency is entirely new. It is well documented that BCV stimulation affects the vestibular system and that evidence had motivated our use of BCV to influence vestibular sensory reliability. However, participants also perceive the vibrations in the auditable frequency range as an intense sound. In the final experiment we aimed to test whether the observed effects of BCV on vection were conveyed directly by the vestibular system, or whether they were caused by a startle reaction at visual motion onset. In our study, we considered that a startle reaction evoked by vestibular stimulation might have either distracted participants at visual motion onset or caused head movements that may have generated additional vestibular signals. To this end, we conducted Experiment 3. Here, we presented air-conducted sound at visual motion onset with the same frequency (500 Hz) and loudness as the BCV stimulus and compared the effects of sound, BCV, and BV on roll vection latency.
4. Experiment 3
4.1. Material and Methods
We recruited 13 participants (nine were female) from a graduate student mailing list at Queen’s University. Each participant took part in an individual session lasting approximately one hour. We compensated each participant $10 per hour. Mean age was 22.08 (SD of 2.36). All had normal or corrected to normal vision. Each participant gave informed written consent before the study in accordance with the Declaration of Helsinki.
4.1.2. Stimuli, Design, and Procedure
The stimuli, overall design, and procedure were similar to those in Experiments 1 and 2. However, we added a condition where we used headphones (Logitech Unreal Ears 6000) to present an auditory stimulus at visual motion onset. The stimulus was the same frequency as the BCV stimulus; a 500 Hz pure tone. Before the experiment we used the method of adjustment to match the loudness of the stimulus played through the headphones with the loudness of the BCV stimulus for each participant. First we presented the standard BCV magnitude, and if necessary adjusted the magnitude slightly until it became tolerable to the participant. Next we presented the auditory tone and asked the participant to adjust a dial with a computer mouse until the tone sounded equal in loudness to the BCV stimulus. Each adjustment made by the participant was accompanied by a sound, and participants were permitted to make as many adjustments as required to complete the task. All participants successfully identified a sound level at which the 500 Hz pure tone played through headphones sounded equal in loudness to the standard BCV stimulus.
We presented only visual stimuli rotating about the roll axis, and compared the vection onset latency for participants experiencing either BV at the same frequency as in Experiments 1 and 2 (that is, 62.5 Hz), a pure tone sound at 500 Hz, or BCV at 500 Hz, at the start of each trial. We ran each condition 10 times (five clockwise, five counter-clockwise visual rotations) to result in 30 trials. The experiment lasted approximately 45 minutes including introduction and debriefing.
4.2. Experiment 3: Results and Discussion
There was a main effect of stimulation type, , , . Follow up pairwise comparisons showed that BCV was associated with lower vection onset latency than either the sound or BV conditions (). The latency data obtained in the sound and BV conditions did not differ (). This suggests that the effect of BCV and GVS is likely to be driven by stimulation of the vestibular system, rather than by a more general startle response to the sound or electrical stimulation perceived during application of BCV or GVS. Data from Experiment 3 are plotted in Fig. 5.
5. General Discussion
Previous research has shown that vestibular signals at visual motion onset are sufficient to reduce the sensory mismatch that seems to drive vection onset latency. Our data show that even noisy, non-directional stimulation of the vestibular system can facilitate vection. Both vibrations and galvanic stimulation applied at the mastoid processes resulted in participants reporting they felt as if they were in motion more quickly than if they had received no vestibular stimulation. Specifically, the results of Experiment 1 revealed that BCV and noisy GVS facilitated circular vection about the roll axis. Body vibration, in contrast, had no effect. We replicated the effect of BCV and GVS on vection latency for yaw and pitch rotation axes in Experiment 2. Additionally, the results of Experiment 3 showed that air-conducted sound has no effect on vection latency which suggests that the effects of BCV and GVS are unlikely to be caused by a general startle response. Rather, the effects observed here are specific to vestibular stimulation. We note that the data show the potential of BCV as a non-invasive technique for probing, and even manipulating, human sensory integration.
Vection latency was reduced using two techniques that stimulate different targets. It is thought that BCV stimulates the otolith organs (e.g., Curthoys et al., 2014), while GVS stimulates non-specific vestibular afferents (e.g., Curthoys and MacDougall, 2012). The data suggest that both techniques reduce vection latency if applied when angular acceleration of the head, detected by vestibular canals, would be expected. The fact that both of these techniques proved effective supports the view that adding noise to the vestibular system reduces the reliance on vestibular cues for self-motion perception, regardless of the manner in which the noise is added. The finding may highlight the fact that neurons responsible for integrating multisensory self-motion cues maintain a joint reliability of otolith and semicircular canal information. Studies show that semicircular canals and otoliths have unique neural connectivity, but that cues from both converge relatively early in the vestibular pathways (Uchino and Kushiro, 2011; Uchino et al., 2000; Zhang et al., 2002). It seems likely that added vestibular noise affected reliability estimates upstream of this convergence.
Substantial differences exist in the sensory conflict for each of the rotation axes in our study, but the effect of stimulation was reasonably similar for each axis. For roll vection, there is an initial sensory conflict where no head acceleration is sensed, and a constant conflict where an expected tilt with respect to gravity is not obtained by the otolith organs. An initial conflict due to expected acceleration and a constant conflict due to expected head tilt also applies to pitch vection. For yaw vection, sensory conflict only emerges at motion onset: the head remains upright with respect to gravity during real yaw rotation. Despite these differences, the only axis-related difference we observed in Experiment 2 was that BCV produced similar latency for pitch and roll, whereas GVS led to lower vection for roll than for pitch rotation. Both BCV and GVS were related to higher vection latency for yaw rotation than for the other axes, while yaw vection latency did not differ between BCV and GVS. The constant mismatch related to expected head tilt for pitch and roll vection did not lead to increased latency in those conditions compared with yaw rotation, where no constant mismatch occurs. This result seems to provide evidence against the sensory mismatch theory, as yaw rotation should be associated with lower vection latency due to lower sensory conflict compared to pitch or roll. Indeed, it has been shown that vection latency for yaw is sometimes shorter than for other rotation axes, although visual display factors seem to impart an influence on the pattern of data across axes (Tanahashi et al., 2012). Future studies need to characterize the reasons for the variance in vection experiences across visual display setups.
Vestibular stimulation at visual motion onset reduced vection latency, but vection onset typically still occurred a number of seconds after stimulus offset. Theories of decision making such as diffusion drift might provide an explanation as to why the effect of stimulation was so enduring. In making a decision between two alternatives (such as ‘I am moving’ or ‘I am stationary’), evidence from sensory data is integrated over time in order to reach a threshold for one of the two decisions (Israël and Warren, 2005; Stone, 1960; also see Bitzer et al., 2014 for an equivalent approach derived from Bayesian statistics). In our study, we contend that noisy stimulation at visual motion onset reduced the sensory information that indicates the body is stationary. This would have reduced the amount of evidence required to reach a decision that the body is in fact moving. As such, vection would still be delayed beyond stimulus offset, but it would be experienced more quickly.
There are other possible explanations of the results obtained here, including possible effects of eye movements and head movements. Stimulation with GVS can produce oculomotor responses, although these are mostly suppressed when vision is present (Curthoys and MacDougall, 2012). Equally, a variety of small eye movements (around half a degree of visual angle) are known to occur as a result of BCV (Cornell et al., 2015). Despite the small size of these eye movements, it has been shown before that vection can be strengthened by small movements of the eye that are generated both actively (Kim and Palmisano, 2010a; Palmisano et al., 2012) and passively (Kim and Palmisano, 2010b; Palmisano et al., 2015). We did not measure eye movements and cannot speculate about their involvement. Similarly, we did not fix the head position of participants. Since we used high frequency vestibular stimulation, it is unlikely that postural responses would have occurred (Dakin et al., 2007; Pavlik et al., 1999). If head movements had played a role in our data we would have expected to see a facilitation of vection for the sound stimulus we used in Experiment 3, given that brief pulses of sound can evoke orienting head movements (Bickford et al., 1964; Colebatch et al., 1994). However, we did not observe a reduction in vection latency for sound alone. This supports the idea that incidental head movements are not responsible for the pattern of data obtained here. Since head and eye movements cannot be altogether ruled out as a factor here, a future study conducted with restrained head movement or eye tracking might prove useful.
Body vibrations have proven effective in terms of enhancing vection magnitude and slightly decreasing vection latency in a previous study (Riecke et al., 2008). Our results do not show support for this finding. The vection onset latency produced in our body vibration condition did not differ from that which we observed in the control condition. This discrepancy might be attributed to differences in the body vibration applied here and in the study by Riecke and colleagues; body vibration in our study was greater in magnitude than the ‘barely noticeable’ (p. 149) vibrations applied in Riecke’s experiment. We also used higher frequency vibrations (62.5 Hz vibrations, where Riecke and colleagues used 7 Hz vibration). It is possible that the low-magnitude stimulation used by Riecke and colleagues produced stochastic resonance which was unlikely to occur for our high-magnitude body vibration (Moss et al., 2004). Additionally, the difference in vibration frequency between the two studies could impact the likelihood of stimulating haptic receptors. A factor that we did not explore here is the potential for further reductions in vection onset latency through the use of multimodal stimulation at motion onset. As discussed above, cues from a range of senses can influence or induce vection. It is possible that the remaining vection latency we observed is related to cue conflict across multiple senses.
Our research marks the first comparison of BCV and GVS in vection research. For both of these stimulation types, we kept the characteristics of the signal constant across the experiment. This was intended as a first exploration of the effectiveness of the stimulation, and indeed we found both to influence vection latency. There are a number of additional stimulation characteristics that could prove important for facilitating vection. These include the frequency, latency, and magnitude of stimulation. Additional research that explores these factors will be vital for understanding the manner in which artificial sensory stimulation can influence multisensory integration in self-motion perception.
Given that sensory mismatch is suspected to underlie the symptoms of discomfort that some VR users experience (symptoms such as headache and nausea known as ‘simulator sickness’; Kennedy et al., 1993), noisy vestibular stimulation might provide significant benefits in these settings. Sensory mismatch has been strongly implicated in the etymology of motion sickness (Cheung et al., 1991; Shupak and Gordon, 2006). For example, Cheung and colleagues (1991) found that simulator sickness does not emerge for patients who have bilateral defects in the vestibular labyrinths, even when they are exposed to conditions that are nauseogenic for almost all healthy participants. Sensory mismatch reduction has been used to provide therapeutic effects for simulator sickness. Reed-Jones and colleagues (2007) showed that GVS applied during driving simulator use can reduce simulator sickness. Although this result shows that supplying expected vestibular signals is effective in reducing sickness, non-directional stimulation could prove even more appealing. A method based on noisy vestibular stimulation would be significantly easier to implement in practical settings, and would not be prone to some of the problems of using directional stimulation. For example, non-directional stimulation consistently produces a reason to disregard vestibular cues (or their absence), whereas for directional stimulation the lack of a precise one-to-one mapping between the applied and expected vestibular stimulation could affect performance drastically. Other important considerations include the risk factors involved in each stimulation technique. The use of GVS is not recommended for some populations (for example, pacemaker users and women in pregnancy) and can also produce symptoms of discomfort in normal healthy users (Lenggenhager et al., 2008). It is clear that an exigency exists for a non-invasive technique that can reduce simulator sickness. The BCV technique that we employed here can be used with all populations and causes no known adverse symptoms (Manzari et al., 2010). Further studies are needed to test the anti-nauseogenic properties of this technique when applied during VR exposure.
This research was sponsored by funding from the Canadian Natural Sciences and Engineering Research Council (NSERC) Collaborative Research and Training Experience Program (CREATE) and the Deutsche Forschungsgemeinschaft (DFG) International Research Training Group (IRTG) granted to S.W. and an NSERC Discovery Grant to N.F.T. The authors thank the members of the BioMotion Lab for assistance in the project.
BrindzaJ.SzwedaJ.LiaoQ.JiangY.StriegelA. (2009). WiiLab: bringing together the Nintendo Wiimote and MATLAB, in: 39th ASEE/IEEE Frontiers in Education Conference, San Antonio, TX, USA.
CurthoysI. S.ManzariL.SmuldersY. E.BurgessA. M. (2009). A review of the scientific basis and practical application of a new test of utricular function–ocular vestibular–evoked myogenic potentials to bone-conducted vibration, Acta Otorhinolaryngol. Ital. 29, 179–186.
CurthoysI. S.VulovicV.BurgessA. M.ManzariL.SokolicL.PogsonJ.RobinsM.MezeyL. E.GoonetillekeS.CornellE. D.MacDougallH. G. (2014). Neural basis of new clinical vestibular tests: otolithic neural responses to sound and vibration, Clin. Exp. Pharmacol. Physiol. 41, 371–380.
DichgansJ.BrandtT. (1978). Visual–vestibular interaction: effects on self-motion perception and postural control, in: Perception, HeldR.LeibowitzG. W.TeuberH.-I. (Eds), Handbook of Sensory Physiology, Vol. 8, pp. 755–804. Springer-Verlag, Berlin, Heidelberg, Germany, and New York, NY, USA.
DraiseyS.MullinsM. (2004). Human audio/vestibular system: data input channels for robotic force and moment sensor measurements, in: Proceedings of the Canadian Engineering Education Association, McGill University, Montréal, QC, Canada.
IsraëlI.WarrenW. H. (2005). Vestibular, proprioceptive, and visual influences on the perception of orientation and self-motion in humans, in: Head Direction Cells and the Neural Mechanisms of Spatial Orientation, WienerS. I.TaubeJ. S. (Eds), pp. 347–381. MIT Press, Cambridge, MA, USA.
PalmisanoS.KeaneS. K. (2004). Effects of visual jitter on visual–vestibular interaction during vection, in: Proceedings of the 39th Annual Australian Psychological Society Conference, Sydney, Australia, pp. 221–224.
Reed-JonesR. J.Reed-JonesJ. G.TrickL. M.VallisL. A. (2007). Can galvanic vestibular stimulation reduce simulator adaptation syndrome?, in: Proceedings of the 4th International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design, Stevenson, WA, USA, pp. 534–540.
RieckeB. E.Schulte-PelkumJ.CaniardF. (2006). Visually induced linear vection is enhanced by small physical accelerations, in: 7th International Multisensory Research Forum, Dublin, Ireland.
RieckeB. E.FeuereissenD.RieserJ. J. (2008). Auditory self-motion illusions (circular vection) can be facilitated by vibrations and the potential for actual motion, in: Proceedings of the 5th Symposium on Applied Perception in Graphics and Visualization, Los Angeles, CA, USA, pp. 147–154.
Schulte-PelkumJ. (2007). Perception of self-motion: vection experiments in multi-sensory virtual environments, PhD Thesis, Ruhr-Universität Bochum, Germany.
UjikeH.YokoiT.SaidaS. (2004). Effects of virtual body motion on visually-induced motion sickness, in: Engineering in Medicine and Biology Society, 26th Annual International Conference of the IEEE, San Francisco, CA, USA, pp. 2399–2402.
VäljamäeA.LarssonP.VästfjällD.KleinerM. (2004). Auditory presence, individualized head-related transfer functions, and illusory ego-motion in virtual environments, in: Proceedings of 7th International Conference on Presence, Limerick, Ireland, pp. 141–147.
WillemsenP.GoochA. A. (2002). Perceived egocentric distances in real, image-based, and traditional virtual environments, in: Proceedings of the IEEE Virtual Reality 2002 (VR02), Washington, DC, USA, pp. 275–276.