Abstract
Accurate perception of verticality is critical for postural maintenance and successful physical interaction with the world. Although previous research has examined the independent influences of body orientation and self-motion under well-controlled laboratory conditions, these factors are constantly changing and interacting in the real world. In this study, we examine the subjective haptic vertical in a real-world scenario. Here, we report a bias of verticality perception in a field experiment on the Hong Kong Peak Tram as participants traveled on a slope ranging from 6° to 26°. Mean subjective haptic vertical (SHV) increased with slope by as much as 15°, regardless of whether the eyes were open (Experiment 1) or closed (Experiment 2). Shifting the body pitch by a fixed degree in an effort to compensate for the mountain slope failed to reduce the verticality bias (Experiment 3). These manipulations separately rule out visual and vestibular inputs about absolute body pitch as contributors to our observed bias. Observations collected on a tram traveling on level ground (Experiment 4A) or in a static dental chair with a range of inclinations similar to those encountered on the mountain tram (Experiment 4B) showed no significant deviation of the subjective vertical from gravity. We conclude that the SHV error is due to a combination of large, dynamic body pitch and translational motion. These observations made in a real-world scenario represent an incentive to neuroscientists and aviation experts alike for studying perceived verticality under field conditions and raising awareness of dangerous misperceptions of verticality when body pitch and translational self-motion come together.
1. Background
Accurate perception of verticality enables us to maintain an upright posture and interact successfully with the world around us. The failure to accurately sense our body orientation relative to gravity may result in inappropriate movements or falls, as well as impair daily functions such as walking, climbing a staircase, and carrying a tray. In specialized tasks such as driving a car and piloting a plane, this failure could be disastrous.
Our perception of verticality is known to depend on the interaction between the vestibular, proprioceptive, somatosensory, and visual systems (Dyde et al., 2006; Zupan et al., 2002). This interaction is best demonstrated by several well-known illusions. In the Haunted Swing, for example, an observer sitting on a stationary swing perceives himself as rotating full circle, while in reality, the house by which he is enclosed rotates around him (Howard and Childerson, 1994; Metzger, 2006 (1936); Wood, 1895). In this case, the self-motion experienced by the observer is a result of a conflict between visual information on one hand and tactile, proprioceptive, and vestibular sensations on the other (Howard and Childerson, 1994; Ohmi, 1996). The tilted room (Shimojo, 2008) similarly demonstrates how the interplay between the visual, vestibular, and somatosensory reference systems misleads one’s perception of orientation by suggesting a false vertical. The misperception is so vivid that falling objects appear to violate gravity (e.g., a ball can be seen to float in space instead of falling straight down). As a result, an observer can experience three distinct verticality views in the same room: when lying down in a supine position, one experiences oneself as (i) sliding down on a tilted slope, or (ii) rotating clockwise or (iii) counter-clockwise, depending on the orientation of whichever surface is considered the ceiling (Shimojo, 2008). The error in the perceived orientation of one’s body comes from an attempt to rectify the tilted room to be aligned with the gravitational vertical. Similar illusions are well known in aviation, where the verticality perception of pilots is affected by extreme translational or rotational acceleration, leading to the oculogravic, somatogravic, and somatogyral illusions (Graybiel, 1952; Seidman et al., 1998; Wade and Curthoys, 1997).
Fortunately, in an artificial environment such as the laboratory, we are quite good at telling the true vertical even when our body is tilted (Howard, 1982). For example, in the study by Bortolami et al. (2006), blindfolded observers correctly indicated their subjective vertical haptically, using a gravity-neutral rod (subjective haptic vertical, SHV). Experimenters systematically rotated the observer’s seat about each of three axes (roll, pitch, yaw) and found, for example, an SHV error of less than 1° for a backward pitch of 26°, consistent with an earlier study by Schöne (1964). Observers slightly underestimated their actual body pitch (i.e., SHV estimates were biased toward the body). Similarly, small deviations from gravity were observed along the longitudinal (roll) and vertical (yaw while supine) axes (see also Bauermeister et al., 1964; Lackner et al., 2001; Miller et al., 1968). These results demonstrate that the perception of verticality based on the vestibular sense is quite robust for a wide range of body orientations.
These findings show that (a) subjective verticality perception in the laboratory based on body orientation cues exclusively is nevertheless mostly accurate, and (b) subjective verticality perception in an artificially created environment can be heavily biased by visual cues when they conflict with body orientation cues. Additionally, most laboratory studies test verticality perception in stationary observers, whereas everyday environments require observers to judge verticality when they are in motion (e.g., on a moving tram or flying a plane). However, little is known about the interaction between body orientation, visual cues, and translational motion in everyday environments. The current study fills this gap by investigating verticality perception under real-world conditions.
2. The Current Study
To examine subjective verticality perception in a natural environment that affords the manipulation of different sensory cues as in the laboratory, the current study utilizes a unique testing venue on the Hong Kong (HK) Peak Tram, Hong Kong. The funicular HK Peak Tram travels up and down a 1.4-km track with a maximum incline of 26°, at a maximum speed of 6 m/s, with passengers always facing uphill during both ascent and descent (fig. 1A). The inclination of the tram changes dynamically along the track, effectively manipulating the body orientation cues received by the observers, different from static pitch manipulation in the laboratory. Notably, passengers sit perpendicular to the slope of the mountain, rather than upright with respect to gravity. The interior tram compartment provides visual cues that are aligned with the slope of the mountain such as window frames and lamp fixtures (fig. 1B), whereas the visual environment outside the tram (such as trees, buildings, and the horizon) provides visual cues informing the observer about the veridical vertical and horizontal. The interaction between interior and exterior visual cues can lead to illusions in spatial orientation such as experienced by pilots during low-level flights (Patterson et al., 2013). Most importantly, the tram is not static but moves for an average duration of five minutes per ride, permitting the study of how body orientation cues and different kinds of visual cues dynamically interact with each other to determine subjective verticality perception when the observer is in translational motion (see Note 1).
The exterior (A) and interior (B) of the Hong Kong Peak Tram at day time. (C) Passengers experience a visual tilt illusion during their ride on the Hong Kong Peak Tram.
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
The exterior (A) and interior (B) of the Hong Kong Peak Tram at day time. (C) Passengers experience a visual tilt illusion during their ride on the Hong Kong Peak Tram.
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
The exterior (A) and interior (B) of the Hong Kong Peak Tram at day time. (C) Passengers experience a visual tilt illusion during their ride on the Hong Kong Peak Tram.
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
This kind of environment has been previously shown to be a venue for studying a related perceptual phenomenon: as passengers look out from the window of the tram, nearby high rises appear to ‘lean’ toward the mountain (i.e., uphill) by as much as 30° (Tseng et al., 2013; fig. 1C). This phenomenon has been called the HK Peak Tram illusion. The factors underlying this illusion have, in part, been elucidated (Tseng et al., 2013), but the mechanism behind them remains elusive. We here suggest that the perceived tilt may be attributable, in part, to an error signal from the subjective vertical. This assumption owes to the following similarities: (1) the strength of the HK Peak Tram illusion can be modified by multiple sensory inputs from the visual, vestibular, proprioceptive, and tactile modalities, which are the same senses known to also contribute to our subjective verticality perception (Angelaki et al., 2009). (2) The various sensory inputs have been shown to affect the HK Peak Tram illusion jointly in a nonlinear fashion similar to what is known about subjective verticality perception (Dyde et al., 2006; Howard, 1986).
We here ask whether a systematic misperception of the subjective vertical occurs when one is seated in a reclining position on the moving mountain tram. If so, quantifying this verticality misperception under different sensory conditions will allow us to establish real-world conditions that can affect perception of verticality. Additionally, this misperception of verticality might also account for the Peak Tram Illusion. This bias in subjective vertical perception is expected to increase linearly with body pitch from the mountain slope, similar to what was found for the magnitude of the HK Peak Tram illusion (Tseng et al., 2013). Such a false perception could be responsible for the perceived tilt of the nearby buildings.
To test the hypothesis of a biased verticality perception on the moving mountain tram, we conducted a field study where observers set their SHV on the HK Peak Tram when traveling up and down the mountain (Exp. 1). In further experiments, we manipulated several possible modulating factors including visual verticality cues (Exp. 2), vestibular, proprioceptive, and tactile cues (Exp. 3), and translational motion cues (Exp. 4A and 4B). Studying this potential misperception of verticality in the field allows us to isolate factors that are known to be crucial to the experience of verticality. At the same time, it may give us a chance to unravel the mechanism underlying the HK Peak Tram illusion, a real-world observation that has yet to be fully understood.
3. Experiment 1. Subjective Verticality Measured on a Mountain Tram
In Experiment 1, we asked participants to report their SHV, the perceived direction of gravity, by setting a gravity-neutral rod. Simultaneously, we determined the body pitch angle of the participants from the slope of the tram on the mountain. The purpose of this experiment was to examine whether participants experienced a bias in SHV in the pitch plane, and, if so, to characterize the relationship between this bias and the body pitch. To allow for a fair comparison with the nighttime HK Peak Tram illusion, nighttime SHV measurements were taken under conditions comparable to those used by Tseng et al. (2013). During the 2013 experiment, participants looked straight ahead while resting their back and head against the last row of the compartment seat, which was bolted down to the bottom floor of the tram. Because participants performed a visual matching task to quantify the perceived building tilt along the tramway, they sat at the right window seat and look straight to ensure a clear view of the buildings (the left side is the mountain view, fig. 1). The seating arrangement is similar in the current study, except that our participants sat at the aisle seat, rather than the window seat, so that they had space to hold the rod to set their subjective haptic vertical. Other details about the measurement details are included in section 3.2: SHV Measurement Procedure below.
3.1. Method
3.1.1. Testing Venue
3.1.1.1. Apparatus.
Gravity-neutral SHV rod. A hollow plastic rod, 105 cm long and 3.6 cm thick, was fastened to a wooden stand made of a long board (19.5 cm × 34 cm × 4 cm) glued at right angles to a narrow base (40 cm × 20 cm × 4 cm) (fig. 2A). An axle penetrating the upper section of the wooden stand as well as the middle of the rod held the rod in place. The stand was fixed to a bench on the tram with two clamps (fig. 2A) on the right-hand side of the subject. The rod was solely supported by the wooden stand and did not move freely under gravity, hence being gravity-neutral. The orientation of the rod could be adjusted with minimal force by the observer to match his/her perceived direction of subjective gravity in the pitch plane.
The devices for measuring the haptic subjective vertical and the slope of the mountain. (A) A gravity-free device comprising a plastic rod and wooden stand was firmly clamped to a stable surface in the laboratory. During the experiment, this device was clamped to the chair and placed alongside the observer on the Peak Tram (not shown here). Observers set the rod to match their subjective vertical in the pitch plane. (B) An Apple iOS TiltMeter App measured the tilt of the rod. (C). A rotary pitch with an incorporated angle meter, rotating scale, and bubble vial system was used to measure the slope of the mountain.
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
The devices for measuring the haptic subjective vertical and the slope of the mountain. (A) A gravity-free device comprising a plastic rod and wooden stand was firmly clamped to a stable surface in the laboratory. During the experiment, this device was clamped to the chair and placed alongside the observer on the Peak Tram (not shown here). Observers set the rod to match their subjective vertical in the pitch plane. (B) An Apple iOS TiltMeter App measured the tilt of the rod. (C). A rotary pitch with an incorporated angle meter, rotating scale, and bubble vial system was used to measure the slope of the mountain.
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
The devices for measuring the haptic subjective vertical and the slope of the mountain. (A) A gravity-free device comprising a plastic rod and wooden stand was firmly clamped to a stable surface in the laboratory. During the experiment, this device was clamped to the chair and placed alongside the observer on the Peak Tram (not shown here). Observers set the rod to match their subjective vertical in the pitch plane. (B) An Apple iOS TiltMeter App measured the tilt of the rod. (C). A rotary pitch with an incorporated angle meter, rotating scale, and bubble vial system was used to measure the slope of the mountain.
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
Apple iOS device with measurement apps. A phone case holding a digital device (Apple iPhone or iPod Touch) was attached to the other end of the axle such that the rod and the digital device rotated in unison. We used an app installed in the digital device (TiltMeter or iHandy Level) to measure the SHV (fig. 2B). When the rod was kept truly vertical with respect to gravity, the App indicated that the tilt was at 0°. When the observer pushed the bottom of the bar in the sagittal plane as in fig. 2A, the device showed a value simultaneously (fig. 2B). The digital device was calibrated at the beginning of each experimental session using a plumb line and regularly rechecked throughout the experiment. Its readings were also compared with a rotary pitch (see the next paragraph), and the readings were always identical. The apparent tilt in the HK Peak Tram illusion (Tseng et al., 2013) is away from the observer, thus negative, whereas the observed change of the subjective vertical is toward the observer, thus positive.
Rotary Pitch. An angle meter with a rotating scale and a bubble vial system (i.e., rotary pitch) was positioned on the window frame of the tram to measure the slope of the mountain (fig. 2C). A human coder concurrently adjusted the bubble to the middle of the vial and read out the angle from the meter. The slope of the mountain was used in our analyses as the objective measure of observers’ body pitch angle.
3.2. SHV Measurement Procedure
Observers sat on the last row of the tram during the experiment. They sat on a bench bolted to the floor of the tram separated from the tram exit by a glass divider and were instructed to look in the direction of tram motion with their head resting against the back of the bench (i.e., keeping their head’s longitudinal axis perpendicular to the tram floor, and thus the mountain slope). The seating position allowed our participants to sit fully upright against the back and head rest. They grasped the rod with their right hand and adjusted it by moving it one way or the other until they felt that it was vertical. Settings typically were made swiftly. Observers verbally indicated when they were done after each setting. An experimenter sitting next to the observer recorded the angle relative to gravity with the TiltMeter while a coder sitting in front of the observer simultaneously recorded the slope of travel using the rotary pitch. The deviation between the angle of the rod and the gravitational vertical was designated as the error of the subjective vertical (β in fig. 3A), with a positive sign indicating that the SHV was shifted toward the reclining body axis (underestimation of body pitch). Before each new estimate, the rod was reset to a new arbitrary position. Widely differing resets were used to minimize any bias. SHV measurements were taken after dusk as in the original report of the HK Peak Tram illusion (Tseng et al., 2013). Each observer made at least 20 estimates of SHV during a single tram ride, and provided SHV measurements for at least one round trip, resulting in an average of 75 estimates for each observer and condition. The number of SHV measurements varied with the mountain slope with a median of 14 measurements taken for each slope across observers.
Experimental Conditions (illustrated by the boxes A–D) and Average Results (curves in E–G) of Experiments 1-3. Observers provided subjective haptic vertical (SHV) settings on the HK Peak Tram while the tram was ascending or descending the mountain with their eyes open at night (A), eyes open during the day (B), eyes closed (C), or eyes closed with a back wedge inserted to shift their body position by a constant amount of 18.7° (D). For illustrative purposes, the mean SHV error, i.e., the deviation of the SHV (β) from the gravitational vertical (g in panel A) averaged across observers at each body pitch value, was plotted as a function of body pitch, measured by mountain slope. Each data point includes 27 measurements on average. The data from ascending/descending trips are collapsed in this graph as well. (E) Results from Experiment 1: SHV error (
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
Experimental Conditions (illustrated by the boxes A–D) and Average Results (curves in E–G) of Experiments 1-3. Observers provided subjective haptic vertical (SHV) settings on the HK Peak Tram while the tram was ascending or descending the mountain with their eyes open at night (A), eyes open during the day (B), eyes closed (C), or eyes closed with a back wedge inserted to shift their body position by a constant amount of 18.7° (D). For illustrative purposes, the mean SHV error, i.e., the deviation of the SHV (β) from the gravitational vertical (g in panel A) averaged across observers at each body pitch value, was plotted as a function of body pitch, measured by mountain slope. Each data point includes 27 measurements on average. The data from ascending/descending trips are collapsed in this graph as well. (E) Results from Experiment 1: SHV error (
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
Experimental Conditions (illustrated by the boxes A–D) and Average Results (curves in E–G) of Experiments 1-3. Observers provided subjective haptic vertical (SHV) settings on the HK Peak Tram while the tram was ascending or descending the mountain with their eyes open at night (A), eyes open during the day (B), eyes closed (C), or eyes closed with a back wedge inserted to shift their body position by a constant amount of 18.7° (D). For illustrative purposes, the mean SHV error, i.e., the deviation of the SHV (β) from the gravitational vertical (g in panel A) averaged across observers at each body pitch value, was plotted as a function of body pitch, measured by mountain slope. Each data point includes 27 measurements on average. The data from ascending/descending trips are collapsed in this graph as well. (E) Results from Experiment 1: SHV error (
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
3.3. Participants
Four of the authors (CH, DC, LS, MO) and four naïve observers (AC, BC, JZ, LF), aged 21 to 73 years (two female), participated in this experiment. All had normal or corrected-to-normal vision and had no known health problems related to sensory performance. All participants signed consent forms and volunteered to participate in the experiments.
3.4. Analysis
A mixed-effects linear regression modeling was performed using R Studio (Version 3.4.3, R Core Team, 2017), R packages lme4 (Kuznetsova et al., 2017), and lmerTest (Bates et al., 2015). To estimate the fixed effect of body pitch and motion direction (ascending vs descending) on the SHV error, the model was specified with the following formula: SHV error ∼ 1 + Body Pitch + Motion Direction + (1 | Observer), using a total of 596 SHV measurements from eight observers. The last term of the formula specifies that observers are included as a random effect for the intercept, meaning each observer could have a different baseline SHV error when there was zero body pitch (intercept). Ultimately, we estimated the fixed effects of body pitch beyond interobserver differences. We here report the estimated fixed effect of body pitch [slope, 95% confidence interval (CI, bootstrapped from 1000 simulations), and t-test statistical significance with Satterthwaite’s method provided by lmerTest]. To indicate model fit, we also report the variance explained by both fixed and random effects, as conditional R2 (Nakagawa and Schielzeth, 2013; Nakagawa et al., 2017) estimated by package MuMIn (Bartoń, 2009); the higher the conditional R2, the better the model fits the data. The data from four authors and four naïve observers were collapsed, as the observer status (author vs naïve) did not modulate the influence of mountain slope on SHV error [slope change = 0.04, 95% CI (−0.06, 0.14),
We performed the same analysis on the illusory tilt effect (values adopted from Tseng et al., 2013) specified with the following formula: Illusion ∼ 1 + Body Pitch + (1 | Observer), using a total of 240 observations from four observers.
3.5. Results and Discussion
We plot the SHV error as a function of body pitch in fig. 3E (solid line). If the subjective vertical settings were consistent with gravity, the resulting plot should be a horizontal line running through the origin; that is, it should be independent of the slope of travel. Instead, our result showed that the SHV error increased linearly with increasing body pitch, at a slower rate than that of the tilt illusion. This is confirmed by model fitting (conditional R2 = 0.74) with eight observers’ data. We found the following characteristics: (1) the mean SHV error (β in fig. 3E, solid line) increased linearly with increasing body pitch [slope = 0.52, 95% CI (0.36, 0.56),
Note that in comparison to studies using a comparable backward pitch of 26° under controlled laboratory conditions (see fig. 4 in Bortolami et al., 2006; Citek and Ebenholtz, 1996; Miller, 1962), the error of the subjective vertical in our observers (fig. 3E solid line) was greater by over 10°. Additionally, to evaluate the temporal variability of SHV error within participant, we computed an absolute difference score in SHV error for each pair of consecutive measurements by each participant. A small (close to zero) difference score indicates small variability of SHV error across consecutive measurements. We found that the median difference score for each participant varied between 2.5° and 4.7° (group mean = 2.7°, SD = 0.6°), indicating a small variability of SHV error.
The SHV has been shown to depend primarily on three modalities, each of which may produce an erroneous signal on a tram traveling up or down a mountain. (1) The visual system: the eyes signal body orientation and body motion relative to the environment from the motion of bypassing trees and buildings. (2) The vestibular system: the otoliths in the inner ear weighing down on the hair cells signal our postural inclination relative to gravity, as well as our body’s linear acceleration/ deceleration induced by translational motion. (3) The somatosensory system: the stretch- and mechano-receptors on the skin and in the joints signal the pressure exerted by the seat onto our back, buttocks, and thighs and the pressure from the floor on our feet in accordance with the inclination of the body relative to gravity. To find out how much these sensory modalities contribute to the SHV, we examined each of them in isolation.
4. Experiment 2. Visual Cues for the Subjective Vertical
In this experiment, we examined how visual orientation cues from the environment affect observers’ SHV on the HK Peak Tram. Two kinds of visual information are relevant to our subjective vertical perception. The first type is optostatic information from fixtures that are known to be vertical and horizontal (e.g., buildings, the horizon, floors, and ceilings); if the retinal image of these fixtures is tilted, it will likely be caused by head tilt rather than by the optostatic objects. The second kind, optokinetic information, comes from the movement of the visual scene on the retina generated by our own movements (e.g., walking, running, bending, or turning); the resulting retinal flow patterns (i.e., optical flow) support our spatial orientation and navigation. A large flow field can induce self-motion perception (vection) while the observers are stationary, and the perceived tilt is in the direction opposite to the visual flow (Dichgans et al., 1972). Optic flow provides information about the speed and movement direction of the tram (Gibson, 1950). Both kinds of visual cues might inform observers’ body inclination and contribute toward SHV error.
Most laboratory research on the SHV was conducted with blindfolded observers or in complete darkness to exclude influences from visual cues (Citek and Ebenholtz, 1996; Miller, 1962; Wade and Curthoys, 1997). In comparison, the environment seen from the HK Peak Tram is replete with visual cues. The affordance of visual cues is especially relevant in Experiment 1, where observers provided SHV settings at night (fig. 3A), when external orientation cues about the true vertical were visible but reduced, and interior cues inside the tram were thus emphasized. For comparison, we designated two additional conditions with varying degrees of visual cues: eyes open at daytime (fig. 3B), affording peripheral glimpses of the true vertical in the outside world; and eyes closed (fig. 3C), resulting in a total absence of visual cues. In this experiment, the two additional conditions were tested in a random sequence, with each observer providing a total of 50 data points from at least one round trip for each condition. Data from these conditions were compared against those in Experiment 1 to establish the effects of visual cues.
Any difference between the results obtained under the conditions in fig. 3A and fig. 3B would reveal the relative influence of conflicting interior and exterior visual orientation cues on the subjective vertical, while the results obtained under the condition in fig. 3C would eliminate the influence of vision altogether in determining the subjective vertical.
4.1. Method
4.1.1. Participants, Apparatus, and Procedure
The same eight observers from Experiment 1 participated in all three conditions. The procedures for measuring the SHV were the same as in Experiment 1. Nighttime trials were conducted after dusk; in combination with window reflections of the tram interior, exterior cues were effectively reduced. This condition was the same as in Experiment 1, so we used the data from this experiment here for a direct comparison. Daytime trials occurred during the afternoon hours, and hence environmental cues, indicating the true vertical, could readily be noticed in the peripheral field of vision outside the tram. In trials with eyes closed, observers shut their eyes before the beginning of the tram ride until after completing the data collection for a single journey. Our participants were all well-trained and had sufficient practice before data collection. They could not be blindfolded due to safety concerns in case of an emergency.
Analysis. As in Experiment 1, we tested the effect of body pitch and visual cues on the SHV error using a mixed-effects linear modeling. To estimate the fixed effect of body pitch and visual conditions on the SHV error, the model was specified with the following formula: SHV error ∼ 1 + Body Pitch * Condition * Motion Direction + (1 | Observer) for each comparison (night, open-eye (reference) vs day, open-eye; night, open-eye (reference) vs closed-eye). If changing visual cues reduces or eliminates the SHV error, we expect to see a significant interaction between body pitch and visual condition, where the slope of the SHV error when plotted against body pitch should be reduced by removing the availability of visual cues. For each model, we report the model fit (conditional R2) and the estimated effect of body pitch, visual condition, and their interaction (the intercept and slope estimate, 95% CI, and t-test statistical significance).
4.2. Results and Discussion
The SHV error obtained under the three visual conditions (eyes open at night, eyes open at day, eyes closed at day) are plotted in fig. 3F as a function of body pitch. Results for eyes open at night were taken from fig. 3E. We find a linear increase in SHV error with increasing mountain slope for all three conditions in seven of eight observers (see supplementary fig. S2 for regression lines separated by observer), confirming that our perception of verticality is increasingly compromised as the inclination of the tram compartment and the observer deviate progressively from the true vertical, regardless of the visual input.
4.2.1. Comparing Between Daytime and Nighttime Measurements
In daylight, the SHV error at the intercept (zero body pitch) was −0.9°, a magnitude reduction of 2.7° [95% CI (1.0°, 4.4°)], compared to the SHV error (−3.6°) at nighttime (reference condition),
4.2.2. Comparing Between Eyes-Closed and (Eyes-Open) Nighttime Measurements
By contrast, removing all visual cues by asking participants to report their SHV with their eyes closed changed the slope of the SHV error when plotted against body pitch, indicated by a small and significant slope change of −0.17 [95% CI (−0.29, −0.04),
Our results show that tram-exterior visual cues, afforded by day vs nighttime measurements, induce a systematic reduction of the SHV error that does not interact with body pitch, whereas tram-interior visual cues, more prominent in nighttime measurements and lacking in the eyes-closed condition, interact with body pitch to affect verticality perception. Taken together, these results suggest potentially different roles of different visual reference frames on verticality perception (world-centered for tram-exterior visual cues, vs body-centered for tram-interior visual cues). Nonetheless, the SHV error persisted when visual cues were absent. Visual cues, or the HK Peak Tram illusion, may thus be ruled out as a primary cause of the linear increase of SHV with body pitch.
5. Experiment 3. Vestibular Cues for the Subjective Vertical
In Experiment 2, the increase of SHV error with increasing pitch (shown in fig. 3G) was found even with eyes closed, when visual information was completely absent, which implies that visual input may be ruled out as a primary source of information for setting the SHV. Here we study to what extent vestibular information can account for the obtained results. On the HK Peak Tram, observers’ body pitch during a tram ride changed dynamically by the mountain slope. Thus, vestibular inputs informing the observer about his/her momentary body inclination (absolute body pitch) and changes in body inclination (dynamic body pitch) might be at play. Ideally, a vertical posture (by a balance board that dynamically compensates for the mountain slope) would be desirable to cancel out any vestibular contribution, but such a device was not available to us. Instead, we added a constant shift of body pitch of 18.7°, the value of which represents the average slope on the mountain. This manipulation allowed us to alter absolute body pitch while maintaining the dynamic changes of body pitch from one moment to the next due to the mountain slope. If this manipulation altered the SHV error, it might indicate that the previously observed SHV was, in part, modulated by the vestibular system encoding absolute body pitch. If SHV error remained unaffected, it would indicate that SHV needs to be attributed to factors other than the absolute body pitch.
To accomplish this, we inserted a wedge behind observers’ back during the tram ride (fig. 3D) and measured the SHV error with that obtained without the wedge in Experiment 2 (fig. 3C). Observers had their eyes closed in both cases to exclude visual influence.
5.1. Method
5.1.1. Participants, Apparatus, Procedure and Analysis
Seven of the eight observers (except author LS) in Experiment 2 participated in all conditions of this experiment (two female). The same procedure as before was used for measuring SHV. A Styrofoam wedge of 18.7°, 100 cm tall, was inserted behind the observers’ back. Observers were instructed to keep their heads against the wedge. In this way, their posture was shifted from a backward pitch ranging from 6° to 26° to a pitch ranging from −12.7° (forward pitch) to 7.3° (backward pitch) over the length of the ride. Again, mixed-effects linear modeling was performed to estimate the effect of shifting the backward pitch on the SHV error: SHV error ∼ 1 + Mountain Slope * Condition * Motion Direction + (1 | Observer).
5.2. Results and Discussion
The SHV error measured with and without a wedge placed behind the back is plotted in fig. 3G as a function of mountain slope (instead of body pitch) for easier visual comparison across experiments. Mean data collapsed from seven observers (supplementary fig. S3 for data separated by observers) show that there is no difference between the results for the two experimental conditions by visual inspection, which is confirmed by mixed-effects linear modeling.
As in previous experiments, the SHV error increased linearly with mountain slope at a slope of 0.38 [95% CI (0.30, 0.47),
These results show that shifting body pitch by a constant amount did not affect the SHV error. It is also important to note that the wedge does not only alter head position signaled by vestibular cues, but also body position which can be sensed by extravestibular factors such as somatosensory graviceptive cues from the trunk and proprioceptive cues from the hips and spine. Nonetheless, all cues were concurrently shifted by the wedge. Vestibular cues (and other nonvisual extravestibular cues) informing us about the absolute body inclination relative to gravity may thus be ruled out as a major factor of the linear increase of SHV with body pitch.
6. Experiment 4. Translational Motion Cues for the Subjective Vertical
Our measurements on the HK Peak Tram were all taken while the tram was in motion, whereas measurements in the laboratory are typically conducted in a static environment with the observer at rest. We are unaware of any study of subjective verticality during translational motion in everyday life. However, the available investigations suggest that translational motion may bias the subjective vertical through optic flow (Bourrelly et al., 2010), as well as by interfering with and reducing the effect of somatosensory and vestibular information (Dichgans and Brandt, 1978; Mergner and Rosemeier, 1998). Because we asked participants to close their eyes in Experiment 3, any visual influence is unlikely to be of concern here. In terms of vestibular influence, given that the otoliths respond identically to speed changes in translational motion and changes in head orientation relative to gravity (Angelaki and Yakusheva, 2009; Angelaki et al., 2004; Dickman, Angelaki and Correia, 1991), changes in translational motion of the Peak Tram might be falsely attributed to head and body tilt, amplifying the SHV error, especially with more variable otolith signals when body pitch is high (Alberts et al., 2016; Clemens et al., 2011). If this were the case, we would expect the SHV error to be jointly influenced by translational motion and body pitch. Specifically, we hypothesized that SHV error is larger when the observer experiences translational motion and is inclined at a larger body pitch. We tested this hypothesis in this experiment, by comparing observers’ SHV measurements under six conditions, four of which were tested when body pitch was small (5–8°) and two of which were tested when body pitch was large (∼26°). Table 1 is a summary of our testing conditions.
Experiment 4: testing condition summary
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
Experiment 4: testing condition summary
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
Experiment 4: testing condition summary
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
For the four conditions with small body pitch, observers were asked to report their SHV when (1) the HK Peak Tram stopped at the two terminal stations at a slope of 6°, and (2) when they were seated in a dental chair inclined by 6°. In both conditions (1) and (2), observers thus were leaning backward with a small pitch (no motion, constant body pitch). In addition, results from these two conditions were compared with the SHV recorded in conditions (3) when the Peak Tram moved up the mountain at a slope of 5–8° (translational motion, dynamically changing body pitch) and (4) on a double-decker street tram in Hong Kong (fig. 4, popularly called Ding Ding Tram), moving along on a horizontal track (translational motion, no change in body pitch), when participants sat upright with a 6° body pitch. If translational motion induces SHV biases, we expected to see a smaller SHV error for a static observer (e.g., condition 1, 2) than for a moving observer either on the Peak Tram (condition 3) or the street tram (condition 4). If the dynamically changing body pitch induces a SHV bias, we expected to see a larger SHV error on the Peak Tram than on the double-decker tram.
The exterior (A, B) and interior (C) of a horizontal moving double-decker street tram in Hong Kong.
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
The exterior (A, B) and interior (C) of a horizontal moving double-decker street tram in Hong Kong.
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
The exterior (A, B) and interior (C) of a horizontal moving double-decker street tram in Hong Kong.
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
To investigate the effect of translational motion when body pitch is large, we also seated participants (5) in a dental chair inclined by 26° (no motion) and compared their SHV settings with those collected, and (6) on the moving Peak Tram at similar slopes of 26° (translational motion) in our previous experiments. We expected to see a larger SHV error when the observer reclines at a large slope on the moving tram.
6.1. Participants, Apparatus and Procedure
6.1.1. On a Stopped HK Peak Tram (5–8° of Body Pitch)
The same eight participants from Experiment 1 were asked to make SHV settings at each of the two terminuses when the tram was not moving (fig. 5A). A total of ten such settings were made by each observer with eyes closed. The mountain slope in both locations was 5–8°.
Experimental Conditions (A–D) and Results (E, F) of Experiment 4. Subjective haptic vertical (SHV) measurements were taken when observers sat in a stopped HK Peak Tram (A), a horizontally moving street tram (B), or a static dental chair inclined at a small (5-8° in C) or large pitch inclination (∼26° in D). Observers’ eyes were closed. Each participant made at least 10 measurements in each condition. Mean SHV error of each observer (open circle), mean SHV error across observers (column height), and the standard error of the mean across observers (error bar) were plotted in (E), separately for small pitch vs large pitch. Data in (F) were selected from (E), highlighting the interaction between body pitch inclination (small vs large) and translational motion (static vs moving). The data for the moving Peak Tram were taken from Experiment 2 (eyes closed). Note that only four observers completed all the conditions and that CH, DC, LS and MO are authors of this paper.
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
Experimental Conditions (A–D) and Results (E, F) of Experiment 4. Subjective haptic vertical (SHV) measurements were taken when observers sat in a stopped HK Peak Tram (A), a horizontally moving street tram (B), or a static dental chair inclined at a small (5-8° in C) or large pitch inclination (∼26° in D). Observers’ eyes were closed. Each participant made at least 10 measurements in each condition. Mean SHV error of each observer (open circle), mean SHV error across observers (column height), and the standard error of the mean across observers (error bar) were plotted in (E), separately for small pitch vs large pitch. Data in (F) were selected from (E), highlighting the interaction between body pitch inclination (small vs large) and translational motion (static vs moving). The data for the moving Peak Tram were taken from Experiment 2 (eyes closed). Note that only four observers completed all the conditions and that CH, DC, LS and MO are authors of this paper.
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
Experimental Conditions (A–D) and Results (E, F) of Experiment 4. Subjective haptic vertical (SHV) measurements were taken when observers sat in a stopped HK Peak Tram (A), a horizontally moving street tram (B), or a static dental chair inclined at a small (5-8° in C) or large pitch inclination (∼26° in D). Observers’ eyes were closed. Each participant made at least 10 measurements in each condition. Mean SHV error of each observer (open circle), mean SHV error across observers (column height), and the standard error of the mean across observers (error bar) were plotted in (E), separately for small pitch vs large pitch. Data in (F) were selected from (E), highlighting the interaction between body pitch inclination (small vs large) and translational motion (static vs moving). The data for the moving Peak Tram were taken from Experiment 2 (eyes closed). Note that only four observers completed all the conditions and that CH, DC, LS and MO are authors of this paper.
Citation: Multisensory Research 36, 1 (2023) ; 10.1163/22134808-bja10086
6.1.2. On a Dental Chair (∼6° or ∼26° of Body Pitch)
The same six observers who provided SHV measurements on the street tram sat in an electromechanical dentist’s chair with the back of the seat adjustable relative to gravity. In the Recumbent Sitting condition (fig. 5D), the chair and observer were inclined by 26°. Wedges were placed under observers’ buttocks and feet to match the seated position on the Peak Tram when it was traveling on the steepest slope (fig. 3A). Observers made settings with their eyes closed.
To produce a body pitch of ∼6° (comparable to the shallow sections of the moving tram), we placed a wedge of 18.7° behind the observer’s back, when the chair was inclined by 24° (Fig. 5C), to match the body pitch to the observer’s posture on the Peak Tram.
The height of the chair was set so that observers’ feet rested comfortably on the ground or a foot wedge. The SHV measurement procedure was the same as in the previous experiments. Observers made 10–14 settings of their SHV in both conditions.
6.1.3. On a Horizontally Moving Double-Decker Street Tram (∼6° of Body Pitch)
Four of the observers from Experiment 1 plus two untrained observers (four females total) were also tested on a double-decker streetcar running on level ground at an average speed of 2.7 m/s and a maximum speed of 5.4 m/s (figs 4, 5B). The back support of the passenger seat on this tram was tilted backward by 6°, thus being comparable to the seat on the stopped Peak Tram. Participants used the same procedure as in previous experiments. The SHV rod was clamped securely to the seat on the right of the subject. Settings were made with eyes closed. The speed of the street tram was not controlled. However, the 20 measurements were made when the tram was moving at a relatively constant speed. No measurements were taken when the tram accelerated (e.g., started) or decelerated (e.g., entered curves or stopped).
6.1.4. On the HK Peak Tram (∼6° or ∼26° of Body Pitch)
These were measurement data obtained from Experiment 2 (eyes closed) on the moving Peak Tram (fig. 3C, D), which travels up and down a 1.4-km track with a maximum inclination of 26°, at a maximum speed of 6 m/s (fig. 1A). The commercial tram speed was generally kept constant (to avoid motion sickness) except when it entered stations and turns (Powell and Palacin, 2015). Our experiments were conducted during its business hours while other passengers were on board, so we did not have much control over the speed. The average ride took 5 minutes.
6.2. Results
6.2.1. The Effect of Translational Motion When Body Pitch Is Small
Figure 5E plots mean SHV errors across observers for a small body inclination (5–8° on the Peak Tram, or approximately 6° on the street tram and dental chair). Average SHV errors on the dental chair and on the stationary Peak Tram were −0.6° [95% CI (−3.7°, 2.4°)] and 1.6° [95% CI (−1.2°, 4.5°)], respectively. Similarly, average errors on the moving Peak Tram and the horizontally moving street tram were 0.8° [95% CI (−4.5°, 6.1°)] and 1.5° [95% CI (−0.1°, 3.0°)]. All these differences seem small and are not statistically significant, as discussed below.
To estimate the effect of translational motion on a shallow slope across testing venues, we fitted raw data using mixed-effects linear regression model according to this formula: SHV error ∼ 1 + Motion Condition (moving vs stationary) + (1 | Observer) + (1 | Test Venue). The final model (conditional R2 = 0.62) was fitted with 405 observations from 10 observers and from three test venues (Peak Tram, horizontal tram, dental chair). We did not find an effect of motion condition [change estimate = −0.6° 95% CI (−1.7°, 0.3°),
Thus, motion by itself does not seem to affect SHV error at small values of body pitch. Neither does dynamic body pitch, as evidenced by the observation that SHV errors were similar on a moving tram with dynamic body pitch (Peak Tram) vs constant body pitch (Ding Ding Tram). This result suggests that the role of motion or dynamic body pitch might be too small to be noticed at small values of body pitch.
6.2.2. The Interaction of Translational Motion and Body Pitch
Figure 5E shows an interaction between translational motion and body pitch. When body pitch angle was small, stationary and moving observers reported comparable SHV errors, an average of −0.6° [95% CI (−3.7°, 2.4°)] and 0.8° [95% CI (−4.5°, 6.1°)], respectively. By comparison, fig. 5F shows that the difference between the SHV error for the static and moving observer obtained for large values of body pitch is substantial [
To statistically test for this interaction between translational motion and body pitch, we fitted raw data using mixed-effects linear regression model according to this formula: SHV error ∼ 1 + Body Pitch Condition (small vs large body pitch) * Motion Condition (moving vs stationary) + (1 | Observer). Our goal is to estimate SHV error as a result of mixed effects from body pitch and motion condition, with the intercept variable according to each tested observer. Our model was fitted with 427 observations from 10 observers (conditional R2 = 0.58). We found a significant interaction between body pitch and translational motion as shown in fig. 5F: when the observer was stationary, SHV error was reduced by 4.4° [95% CI (−5.5°, −3.2°)] when body pitch increased from small to large (
In summary, the SHV error was especially large when the Peak Tram was moving and the body pitch was large. This finding suggests an interaction between body inclination and body motion in determining verticality. Note that when body pitch was large, the direction of the average SHV error changed from negative (biased away from the observer) on the dental chair to positive (biased toward the observer) on the moving Peak Tram. This sign change was obtained for four observers who completed both conditions when the body pitch was large.
7. General Discussion
Our study examined verticality perception on the HK Peak Tram in light of the interaction between various environmental and sensory factors. We tested the deviation of the SHV from gravity at various slopes and with select sensory inputs. Based on previous findings in the literature (Bortolami et al., 2006; Schöne, 1964), we expected that the SHV would not be different from the objective vertical (i.e., gravity) under these conditions. Yet, contrary to expectation, we found a systematic SHV error on the HK Peak Tram, increasing linearly with increasing body pitch in accordance with the changing mountain slope (Exp. 1). This error prevailed even when observers closed their eyes (Exp. 2), ruling out visual cues as the cause of this error. A wedge behind observers’ back (Exp. 3), inserted to change body posture, produced the same result, ruling out body pitch as a factor for the SHV. Instead, we found an interaction between body pitch and translational body motion (Exp. 4) on SHV perception. These results suggest a possible interaction of various sensory cues affecting verticality perception on an everyday moving vehicle.
7.1. Deviation of SHV versus HK Peak Tram Illusion
The deviation of SHV from gravity on the HK Peak Tram persists across experiments. The linear relationship between body pitch and SHV error was replicated in most observers, naïve and non-naïve (fig. 3). Because the experimenter reset the gravity-neutral rod to a new arbitrary position prior to each measurement, observers could not rely on previous settings. The small temporal variance (see section 3.5: Results and Discussion of Exp. 1), therefore speaks for the validity of the data. The SHV error was slightly larger during descending than ascending trips (see statistical results of Exp. 1–3). This effect might be attributable to differences in processing backward (experienced during descending trips) vs forward (experienced during ascending trips) translational motion.
The direction of the observed SHV deviation is consistent with the HK Peak Tram illusion, another example of verticality misperception on the same test venue (Tseng et al., 2013). In the Peak Tram Illusion, observers perceived the skyscrapers next to the tram leaning toward the mountain, i.e., away from the true vertical, implying an overestimation of body tilt. In the current study, observers made complementary errors when setting their SHV, i.e., biased toward the observer, possibly reflecting an underestimation, similar to the so-called A-Effect or SV bias toward the body axis described by Aubert (1861). These two examples of verticality deviation on the HK Peak Tram constitute two sides of the same coherent story. As the SHV becomes biased from the true vertical and toward the observer, any vertically oriented structures (i.e., skyscrapers) would in turn be falsely perceived as tilted away in the opposite direction. However, compared to the latter the magnitude of the error is twice as large in the former and thus cannot fully account for it.
Compared to the HK Peak Tram illusion, which amounted to as much as 30°, the observed SHV error in this study can at best account for only half of the perceived tilt found in the illusion. This difference suggests that additional factors are involved. On the other hand, when compared to the near-zero deviation of the SHV found in laboratory studies (e.g., Bortolami et al., 2006; Schöne, 1964) for a similar body pitch, the SHV error measured by us under field conditions is far larger. How can we explain this difference?
7.2. A Possible Role of the Otoliths in Reading out Translational Motion as Pitch
To explain the observed SHV error in this study, we propose that sensory signals from translational motion of the mountain tram in conjunction with the mountain slope may have combined into a dynamic error of subjective verticality. In a field study, conditions can never be fully controlled. Therefore, in addition to the changing mountain slope, the speed of the tram along the ride was not uniform but changed noticeably throughout the entire ride. The hair cells of the otolith organs in the inner ear would therefore signal a continuous succession of speed changes and body pitch changes relative to gravity. A failure to disambiguate inertial force caused by translational acceleration/deceleration and gravitational force caused by the mountain slope could have resulted in an amplified error signal for the SHV.
This explanation is plausible when one considers the similarities between the SHV error observed in our study and the somatogravic illusion, attributed to the failure of the otolith system to disambiguate between tilt and translation (i.e., the tilt/translation ambiguity; Angelaki and Yakusheva, 2009). It is well known that pilots often perceive an exaggerated nose-up tilt during take-off with linear acceleration (Sipes and Lessard, 2000), especially in fog and when flying through clouds. In the laboratory, the somatogravic illusion can be induced by translational acceleration (Graybiel, 1952; Merfeld et al., 2005). For example, in a classic study, Graybiel (1952) reported that observers in a centrifuge, built to spin a human observer in a gondola to induce acceleration, experienced a change in body pitch (i.e., somatogravic illusion) and an upward movement of a stationary object (oculogravic illusion). An acceleration force of 0.5–0.6 g, if applied in a centrifuge in a sustained manner, could generate a 5° (Clément et al., 2001) to ∼15° (Eriksson et al., 2008) perceived backward pitch, which is in the range of the observed SHV error. However, it should be noted that acceleration on public ground transport and in the present conditions is unlikely to exceed 0.1 g for more than a few seconds when starting and stopping (De Graaf and Van Weperen, 1997; Hoberock, 1976). But given that otolith signals become more variable with body tilt (Alberts et al., 2016; Clemens et al., 2011), it is possible that biases in verticality perception induced by even minor linear acceleration might become more apparent when body tilt is large, and could explain the linear relationship between SHV error and body pitch observed in our study.
Nevertheless, there are still a lot of open questions as to how self-motion interplays with verticality perception. For one, a systematic investigation will be helpful to characterize the boundary and constraints of the interplay between graviception and motion. Most available studies use a motion stimulator or visual optical flow to stimulate a sensory modality (e.g. vestibular in a motion stimulator, visual in optical flow) to create self-motion (vection) in a lab environment while all other senses are either weak or absent (see Cuturi, 2022 for a review). However, our final percept of verticality is a multisensory decision after our brain weighs the consistency and redundancy from all involved modalities (De Vrijer et al., 2008). Our discovery in this series of field studies is a rare case where all acquired senses provide coherent self-motion (linear acceleration + inclination) from proprioceptive, vestibular, haptic, and visual modalities. A parametric study is desirable to complete the missing pieces of this puzzle.
7.3. The Role of Other Sensory Cues
Compared to laboratory settings, the Peak Tram track is replete with multiple non-otolith sensory cues important to verticality perception. Here we describe the potential influence of these additional cues on the SHV error.
7.3.1. Visual Cues
Previous studies in aviation pilots have found that visual cues such as distant shore or ridgelines can serve as false horizons, resulting in the illusion that a level aircraft is nosing up (Patterson et al., 2013), thereby confirming a role of visual cues on verticality biases. However, our experiments show that the availability of visual cues played a small, if any, role in affecting the SHV error on the Peak Tram (Exp. 2). This is not surprising as tram-interior cues such as lamp fixtures and benches inside the tram do not change in relation to the observer regardless of the changing mountain slope. Thus, they are uninformative for tilt/translation disambiguation. In contrast, tram-exterior cues such as buildings and trees might convey information about changes in body pitch due to translational speed. Measurements during daytime did indeed reduce the SHV error at the intercept but did not reduce the SHV error at larger body pitch angles. This suggests that they are not critical here.
7.3.2. Semi-Circular Canal Cues
With respect to a possible involvement of the semicircular canals (Angelaki and Yakusheva, 2009), only the initial change when the tram picked up speed constituted a signal, whereas the dynamic changes in slope or speed during the ride were a lot smaller than the changes used in laboratory studies to investigate the role of dynamic tilt on verticality perception (e.g., 180°/s in Jaggi-Schwarz and Hess, 2003; 90°/s in Pavlou et al., 2003). Therefore, their effect on the SHV may be small if any.
7.3.3. Tactile and Proprioceptive Cues
Tactile and proprioceptive inputs from the cutaneous and musculo-skeletal mechanoreceptors known to contribute to the oculogravic illusion (Clark and Graybiel, 1966) did not help subjective verticality perception on the HK Peak Tram either. This also applies to their role for the perception of translational motion (Gianna et al., 1996) and the subjective vertical (Alberts et al., 2016; Angelaki and Laurens, 2020). During a tram ride, proprioceptive signals from the feet resting on the tilted floor as well as tactile pressure on the thighs and buttocks from the tram seat varied according to slope and speed but did not eliminate the SHV error.
7.3.4. Integration of Multiple Cues
Importantly, observers in our experiment kept their head level with the mountain slope, implying that the vestibular input from the otoliths and the proprioceptive/ somatosensory input from the trunk were synergistic. Given this synergy, the observed SHV error is unlikely to have been caused by suboptimal integration of sensory information, as shown in studies using inconsistent sensory cues to demonstrate sensory dominance and differences in reference frames (Clemens et al., 2011; Fraser et al., 2015; Harris et al., 2017). The neural and computational mechanisms underlying the SHV error remain to be addressed by future studies, combining different sensory inputs. Previous studies looked at the interaction between the visual and vestibular systems (Dockheer et al., 2018; Fraser et al., 2015; Schuler et al., 2010). Our findings under field conditions suggest that other sensory inputs need to be incorporated to reflect the rich interaction of everyday stimuli.
7.4. Limitations and Future Directions
To test for the role of dynamic pitch and translational motion, one would have to measure the SHV on a mountain tram that adjusts itself continuously to the varying slope so that observers’ position is always aligned with gravity. Under such conditions, we would expect the vestibular, cutaneous and proprioceptive inputs to be constant and the SHV close to gravity. The Hungerburg Funicular going from Innsbruck to the Nordkette mountain range is an example of such an adjustment.
8. Conclusion
Our study provides behavioral evidence that observers’ perception of haptic verticality in a natural environment deviates from gravity due to changing body pitch and speed of translational motion. The combination of both factors is critical. However, exactly how translational motion interacts with body pitch to affect verticality perception remains to be seen.
The active research field on perceived body orientation has long ignored the combination of both factors although they were extensively studied independently. Our findings of a misperception of verticality in a real-world scenario involving self-motion of the observer emphasizes the need for a systematic investigation to understand this long-overlooked factor. Future research of verticality perception in the field should also have implications for a better understanding of cross-sensory interaction, the perception of body sensations, as well as practical consequences in the realm of aviation and clinical application.
Tseng and Chow contributed equally.
Corresponding author; e-mail: CH_Tseng@alumni.uci.edu
Acknowledgements
We are grateful to the research assistants and students who participated in the experiments, and to Jason Carlow who helped us to make our measurement device. We wish to thank the Hong Kong Peak Tram Co. which made this field study possible. We also thank the following researchers for their invaluable insights and discussion on this project: Antun Pejak, Drs. Mark Greenlee, Jean Laurens, Gregory DeAngelis, and Marc Ernst. The University Human Research Ethics Committee at The University of Hong Kong approved all the experimental procedures. All methods were performed in accordance with the principles expressed in the Declaration of Helsinki as regards the treatment of participants. The data sets and the analysis scripts used for this manuscript are available on Open Science Framework via this link: https://osf.io/hr73x/?view_only=2fdb1fe91a1c43d89114e5de2ea29421. CT and KS are supported by the Cooperative Research Project Program from the Research Institute of Electrical Communication at Tohoku University. HC was supported by a Postdoctoral Fellowship from the Canadian Institutes of Health Research, and a Research Trainee Award from the Michael Smith Foundation for Health Research held at the University of British Columbia, Vancouver, Canada. LS would like to thank the support from the Serena Yang Foundation for his visitorship. KS is supported by a JSPS Grant-in-Aid for Scientific Research (C) Grant Number 17K04498, 21K03145, (A) 21H04426. CT, MO, HC and LS designed and executed the study, analyzed the data, and wrote the paper. All authors wrote the paper and approved the final version of the manuscript.
Supplementary Material
Supplementary material is available online at: https://doi.org/10.6084/m9.figshare.21610407
Note
- 1. To watch the observers’ experience from the HK Peak Tram, here are some videos available on the web: (1) is.gd/aicGvz, (2) is.gd/6cCOcq, (3) is.gd/DWLxXY.
References
Alberts, B. B. G. T., Selen, L. P. J., Bertolini, G., Straumann, D., Medendorp, W. P. and Tarnutzer, A. A. (2016). Dissociating vestibular and somatosensory contributions to spatial orientation, J. Neurophysiol. 116, 30–40. DOI:10.1152/jn.00056.2016.
Angelaki, D. E. and Laurens, J. (2020). Time course of sensory substitution for gravity sensing in visual vertical orientation perception following complete vestibular loss, eNeuro 7. DOI:10.1523/ENEURO.0021-20.2020.
Angelaki, D. E. and Yakusheva, T. A. (2009). How vestibular neurons solve the tilt/translation ambiguity, Ann. N. Y. Acad. Sci. 1164, 19–28. DOI:10.1111/j.1749-6632.2009.03939.x.
Angelaki, D. E., Shaikh, A. G., Green, A. M. and Dickman, J. D. (2004). Neurons compute internal models of the physical laws of motion, Nature 430, 560–564. DOI:10.1038/nature02754.
Angelaki, D. E., Gu, Y. and DeAngelis, G. C. (2009). Multisensory integration: psychophysics, neurophysiology, and computation, Curr. Opin. Neurobiol. 19, 452–458. DOI:10.1016/j.conb.2009.06.008.
Aubert, H. (1861). Eine scheinbare bedeutende Drehung von Objecten bei Neigung des Kopfes nach rechts oder links, Arch. Pathol. Anat. Physiol. Klin Med. 20, 381–393. DOI:10.1007/BF02355256.
Bartoń, K. (2009). MuMIn: Multi-Model Inference. R package version 0.12.0. https://cran.r-project.org/web/packages/MuMIn/index.html.
Bates, D., Maechler, M., Bolker, B. and Walker, S. (2015). Fitting linear mixed-effects models using lme4, J. Stat. Softw. 67, 1–48. DOI:10.18637/jss.v067.i01.
Bauermeister, M., Werner, H. and Wapner, S. (1964). The effect of body tilt on tactual-kinesthetic perception of verticality, Am. J. Psychol. 77, 451–456. DOI:10.2307/1421016.
Bortolami, S. B., Pierobon, A., DiZio, P. and Lackner, J. R. (2006). Localization of the subjective vertical during roll, pitch, and recumbent yaw body tilt, Exp. Brain Res. 173, 364–373. DOI:10.1007/s00221-006-0385-y.
Bourrelly, A., Vercher, J.-L. and Bringoux (2010). Pitch body orientation influences the perception of self-motion direction induced by optic flow, Neurosci. Lett. 482, 193–197. DOI:10.1016/j.neulet.2010.07.028.
Citek, K. and Ebenholtz, S. M. (1996). Vertical and horizontal eye displacement during static pitch and roll postures, J. Vestib. Res. 6, 213–228. DOI:10.3233/VES-1996-6306.
Clark, B. and Graybiel, A. (1966). Perception of the visual horizontal in normal and labyrinthine defective observers during prolonged rotation, Am. J. Psychol. 79, 608–612. DOI:10.2307/1421298.
Clemens, I. A. H., De Vrijer, M., Selen, L. P. J., Van Gisbergen, J. A. M. and Medendorp, W. P. (2011). Multisensory processing in spatial orientation: an inverse probabilistic approach, J. Neurosci. 31, 5365–5377. DOI:10.1523/jneurosci.6472-10.2011.
Clément, G., Moore, S. T., Raphan, T. and Cohen, B. (2001). Perception of tilt (somatogravic illusion) in response to sustained linear acceleration during space flight, Exp. Brain Res. 138, 410–418. DOI:10.1007/s002210100706.
Cuturi, L. F. (2022). Perceptual biases as the side effect of a multisensory adaptive system: insights from verticality and self-motion perception, Vision 6, 53. DOI:10.3390/vision6030053.
De Graaf, B. and Van Weperen, W. (1997). The retention of blance: an exploratory study into the limits of acceleration the human body can withstand without losing equilibrium, Hum. Factors 39, 111–118. DOI:10.1518/00187209777894061.
De Vrijer, M., Medendorp, W. P. and Van Gisbergen, J. A. M. (2008). Shared computational mechanism for tilt compensation accounts for biased verticality percepts in motion and pattern vision, J. Neurophysiol. 99, 915–930. DOI:10.1152/jn.00921.2007.
Dichgans, J. and Brandt, T. (1978). Visual–vestibular interaction: effects on self-motion perception and postural control, in: Perception. Handbook of Sensory Physiology, vol. 8, R. Held, H. W. Leibowitz and H. L. Teuber (Eds), pp. 755–804. Springer-Verlag, Berlin, Germany. DOI:10.1007/978-3-642-46354-9_25.
Dichgans, J., Held, R., Young, L. R. and Brandt, T. (1972). Moving visual scenes influence the apparent direction of gravity, Science (New York, N.Y.) 178, 1217–1219. DOI:10.1126/science.178.4066.1217.
Dickman, J. D., Angelaki, D. E. and Correia, M. J. (1991). Response properties of gerbil otolith afferents to small angle pitch and roll tilts, Brain Res. 556, 303–310. DOI:10.1016/0006-8993(91)90320-U.
Dockheer, K. M., Bockisch, A. J. and Tarnutzer, A. A. (2018). Effects of optokinetic stimulation on verticality perception are much larger for vision-based paradigms than for vision-independent paradigms, Front. Neurol. 9, 323. DOI:10.3389/fneur.2018.00323.
Dyde, R. T., Jenkin, M. R. and Harris, L. R. (2006). The subjective visual vertical and the perceptual upright, Exp. Brain Res. 173, 612–622. DOI:10.1007/s00221-006-0405-y.
Eriksson, L., von Hofsten, C., Tribukait, A., Eiken, O., Andersson, P. and Hedström, J. (2008). Visual flow scene effects on the somatogravic illusion in non-pilots, Aviat. Space Environ. Med. 79, 860–866. DOI:10.3357/ASEM.2264.2008.
Fraser, L. E., Makooie, B. and Harris, L. R. (2015). The subjective visual vertical and the subjective haptic vertical access different gravity estimates, PLoS ONE 10, e0145528. DOI:10.1371/journal.pone.0145528.
Gianna, C., Heimbrand, S. and Gresty, M. (1996). Thresholds for detection of motion direction during passive lateral whole-body acceleration in normal subjects and patients with bilateral loss of labyrinthine function, Brain Res. Bull. 40, 443–447. DOI:10.1016/0361-9230(96)00140-2.
Gibson, J. J. (1950). The Perception of the Visual World. Houghton Mifflin, Boston, MA, USA.
Graybiel, A. (1952). Oculogravic illusion, AMA Arch. Ophthalmol. 48, 605–615. DOI:10.1001/archopht.1952.00920010616007.
Harris, L. R., Sakurai, K. and Beaudot, W. H. A. (2017). Tactile flow overrides other cues to self motion, Scientific Reports 7, 1059. DOI:10.1038/s41598-017-01111-w.
Hoberock, L. L. (1976). A Survey of Longitudinal Acceleration Comfort Studies in Ground Transportation Vehicles. Research Report, 40, Council for Advanced Transportation Studies, Austin, TX, USA.
Howard, I. P. (1982). Human Visual Orientation. John Wiley and Sons, Chichester, UK.
Howard, I. P. (1986). The perception of posture, self motion, and the visual vertical, in: Handbook of Perception and Human Performance, vol. 1, K. R. Boff, L. Kaufman and J. P. Thomas (Eds), Sensory Processes and Perception, pp. 18-1–18-62. Wiley-Interscience, New York, NY, USA.
Howard, I. P. and Childerson, L. (1994). The contribution of motion, the visual frame, and visual polarity to sensations of body tilt’, Perception 23, 753–762. DOI:10.1068/p230753.
Jaggi-Schwarz, K. and Hess, B. J. M. (2003). Influence of dynamic tilts on the perception of Earth-vertical, Exp. Brain Res. 149, 340–350. DOI:10.1007/s00221-002-1343-y.
Kuznetsova, A., Brockhoff, P. B. and Christensen, R. H. B. (2017). lmerTest package: tests in linear mixed effects models, J. Stat. Softw. 82, 1–26. DOI:10.18637/jss.v082.i13.
Lackner, J. R., Rabin, E. and DiZio, P. (2001). Stabilization of posture by precision touch of the index finger with rigid and flexible filaments, Exp. Brain Res. 139, 454–464. DOI:10.1007/s002210100775.
Merfeld, D. M., Park, S., Gianna-Poulin, C., Black, F. O. and Wood, S. (2005). Vestibular perception and action employ qualitatively different mechanisms. I. Frequency response of VOR and perceptual responses during Translation and Tilt, J. Neurophysiol. 94, 186–198. DOI:10.1152/jn.00904.2004.
Mergner, T. and Rosemeier, T. (1998). Interaction of vestibular, somatosensory and visual signals for postural control and motion perception under terrestrial and microgravity conditions — a conceptual model, Brain Res. Rev. 28, 118–135. DOI:10.1016/S0165-0173(98)00032-0.
Metzger, W. (2006). Laws of Seeing. MIT Press, Cambridge, MA, USA. Original German edition 1936.
Miller, E. F. (1962). Counterrolling of the human eyes produced by head tilt with respect to gravity, Acta Otolaryngol. 54, 479–501. DOI:10.3109/00016486209126967.
Miller 2nd, E. F., Fregly, A. R. and Graybiel, A. (1968). Visual horizontal-perception in relation to otolith-function, Am. J. Psychol. 81, 488–496.
Nakagawa, S. and Schielzeth, H. (2013). A general and simple method for obtaining R2 from generalized linear mixed-effects models, Methods Ecol. Evol. 4, 133–142. DOI:10.1111/j.2041-210x.2012.00261.x.
Nakagawa, S., Johnson, P. C. D. and Schielzeth, H. (2017). The coefficient of determination R2 and intra-class correlation coefficient from generalized linear mixed-effects models revisited and expanded, J. R. Soc. Interface 14, 20170213. DOI:10.1098/rsif.2017.0213.
Ohmi, M. (1996). Egocentric perception through interaction among many sensory systems, Brain Res. Cogn. Brain Res. 5, 87–96. DOI:10.1016/S0926-6410(96)00044-4.
Patterson, F. R., Arnold, R. D. and Williams, H. P. (2013). Visual perspective illusions as aviation mishap causal factors, in: 17th International Symposium on Aviation Psychology, Dayton, OH, USA, pp. 512–517.
Pavlou, M., Wijnberg, N., Faldon, M. E. and Bronstein, A. M. (2003). Effect of semicircular canal stimulation on the perception of the visual vertical, J. Neurophysiol. 90, 622–630. DOI:10.1152/jn.00960.2002.
Powell, J. P. and Palacín, R. (2015). Passenger stability within moving railway vehicles: limits on maximum longitudinal acceleration, Urban Rail Transit 1, 95–103. DOI:10.1007/s40864-015-0012-y.
Schöne, H. (1964). On the role of gravity in human spatial orientation, Aerosp. Med. 35, 764–772.
Schuler, J. R., Bockisch, C. J., Straumann, D. and Tarnutzer, A. A. (2010). Precision and accuracy of the subjective haptic vertical in the roll plane, BMC Neurosci. 11, 83. DOI:10.1186/1471-2202-11-83.
Seidman, S. H., Telford, L. and Paige, G. D. (1998). Tilt perception during dynamic linear acceleration, Exp. Brain Res. 119, 307–314. DOI:10.1007/s002210050346.
Shimojo, S. (2008). Self and the world: large scale installations at science museums, Spat. Vis. 21, 337–346. DOI:10.1163/156856808784532536.
Sipes, W. E. and Lessard, C. S. (2000). A spatial disorientation survey of experienced instructor pilots, IEEE Eng. Med. Biol. Mag. 19, 35–42. DOI:10.1109/51.827403.
Tseng, C.-h., Chow, H. M. and Spillmann, L. (2013). Falling skyscrapers: when cross-modal perception of verticality fails, Psychological Science 24, 1341–1347. DOI:10.1177/0956797612472344.
Wade, S. W. and Curthoys, I. S. (1997). The effect of ocular torsional position on perception of the roll-tilt of visual stimuli, Vision Res. 37, 1071–1078. DOI:10.1016/S0042-6989(96)00252-0.
Wood, R. W. (1895). The ‘Haunted Swing’ illusion, Psychol. Rev. 2, 277–278. DOI:10.1037/h007333.
Zupan, L. H., Merfeld, D. M. and Darlot, C. (2002). Using sensory weighting to model the influence of canal, otolith and visual cues on spatial orientation and eye movements, Biol. Cybern. 86, 209–230. DOI:10.1007/s00422-001-0290-1.