Toward virtual stair walking

(1)

https://doi.org/10.1007/s00371-021-02179-2 O R I G I N A L A R T I C L E

Toward virtual stair walking

MinYeong Seo¹·HyeongYeop Kang¹

Accepted: 26 May 2021 / Published online: 8 June 2021

Abstract

This paper presents a motion remapping-based locomotion technique. Our technique can provide a realistic sensation of climbing and descending stairs when users navigate the virtual environment on foot. The main contribution is to provide users a realistic experience of walking up and down virtual stairs while in reality, they are walking on a flat surface. When a user lifts their real foot, our technique controls the position of virtual foot in order to match the timing of real foot touching the floor with that of virtual foot touching the stairs. The avatar’s head and waist are also controlled to mimic the height change movements of stair walking. To achieve this, we collected the actual motion data beforehand and then designed our locomotion technique using the data. Then, we conducted an experiment and an application test. In the experiment, we identified how much visual gain should be applied to foot motion to induce a realistic sensation of stair walking. The results demonstrated that applying visual gains of 1.193 and 0.822 to motions of climbing and descending the stairs were accepted as the most realistic, respectively. In the application test, we investigated whether the proposed technique successfully increases the user’s perceived presence and provides a positive user experience. The results demonstrated that the user’s perceived presence was significantly enhanced when we applied visual gains. The results also showed that participants felt as if they were walking on the stairs in the virtual environment without experiencing discomfort or postural instability. As the proposed technique only needs visual cue control, we expect that it can easily be applied to commercial applications .

Keywords Virtual reality·Locomotion·Visual gain

1 Introduction

The rapid evolution of commercial head-mounted displays (HMDs) and virtual reality (VR) technologies allows users to walk through an immersive virtual environment. This is generally achieved by mapping the users’ position and posture in the real world to the virtual avatar. However, the range of the user’s movement in the virtual environment is often restricted because the corresponding physical play area has a limited range. To solve this problem, several studies have proposed locomotion devices that prevent the displacement of a walking user in the real world [8,9,13,19,21,46,47].

Some other studies have suggested several techniques that manipulate a user’s perceived self-motion. For example, the redirected walking technique [31,32,41,42,49] can scale user’s displacements and rotations when mapping the real

B

HyeongYeop Kang siamiz@khu.ac.kr

1 Deptartment of Software Convergence, Kyung Hee University, 1732, Deogyeong-daero, Giheung-gu, Yongin-si, Gyeonggi-do, South Korea

world motions to the virtual world motions. By controlling the amount of scale, the redirected walking technique allows users to explore a horizontally large virtual environment even when they are in a small play area. However, enabling a user to walk vertically large virtual environments, such as stair walking, in a flat play area has not received much attention. Although several hardware interfaces [19,20,44] have been proposed to simulate stair walking motions, they are often bulky and expensive. Some other studies have proposed the installation of small bumps on the floor to provide the haptic stimuli of the edge of the stair in the virtual environment [2,34,35]. However, the constraint that the real bump must be physically placed exactly under the user’s feet inevitably restricts the design of the virtual world and the user’s locomotion.

This study aims to provide a realistic sensation of stair walking in the virtual environment while allowing VR users to walk on a flat surface without additional installments. To this end, we first collected the actual motion data of walking up and down physical staircases. Based on the collected data, we formulated equations for remapping the user’s physical

(2)

motion to the virtual avatar, by controlling the positions of the virtual avatar’s head, waist, and feet. Then, we employed the concept of visual gain in our equation and conducted an experiment to identify the appropriate value of visual gain that can provide the most natural sensation of stair walking. Lastly, we conducted an application test to investigate whether the proposed technique successfully increases the user’s perceived presence and provides a positive user experience. The proposed method does not require additional hardware and can thus be easily adapted to commercial applications.

2 Related work

Methods of allowing users to locomote horizontally in immersive virtual environments have been widely studied.

Conversely, only a few studies have focused on enabling users to locomote vertically, such as stair walking, in virtual environments. Since the surface of the physical play area is usually flat, it is difficult to provide users with a realistic sensation of virtual stair walking. For this reason, many VR applications have used virtual portals [11,17,48], teleportation [5,10], or flying interface [50–52], instead of allowing users to walk on virtual stairs. However, such alternatives inevitably diminish the user’s perceived sense of presence because users perceive a higher sense of presence when they locomote by real walking [29,53].

Several studies have tried to resolve such a problem by using hardware interfaces to simulate vertical movement.

Sarcos Treadport [18] and Ground Surface Simulator [38]

can provide a feeling of climbing slopes by tilting the treadmill. The Gait Master [20] enables users to climb or descend virtual stairs while maintaining users’ physical position. The CirculaFloor locomotion interface [19] exploits movable tiles to realize omnidirectional walking. Level-Ups [44] simulate the user’s stair steps in the virtual environment by controlling stilts. A cable-based suspension system [23,26] allows users to jump with their own feet, enabling jump onto the building or stair. However, such hardware interfaces are expensive and expose users to injury risk.

As a simple and low-cost alternative solution to this problem, approaches manipulating user’s sensory modalities in a virtual environment have gained research attention. Marchal et al. [30] modify the motion of the user’s virtual viewpoint to simulate the sensation of walking up and down in virtual environments. However, the proposed manipulation technique was too simple to induce a strong sensation of ascending and descending. Nordahl et al. [36,39] found that providing haptic stimulation to the feet can induce stronger vertical illusory self-motion. Lai et al. [28] use arm and leg gestures of the user to induce a strong sensation of climbing a ladder.

Nagao et al. [34,35] exploit visuo-haptic interaction to sim-

ulate virtual stairs. Since the haptic and visual sensations are presented simultaneously, users are able to experience a realistic sensation of walking up and down the stairs. However, it has several limitations such as having to know the dimensions of virtual stairs in advance and requiring external haptic devices which are often inaccessible to general users.

In this study, we focus on the virtual stair walking. Our goal is to provide a realistic sensation of climbing and descending virtual stairs to users, while the users are walking on a flat surface in the physical play area. To achieve this without external haptic devices, we exploit the fact that visual cues often overwhelm the other modalities [7,12,40]. Therefore, we dynamically remap the user’s physical walking motion on a flat surface to the avatar’s virtual stair walking motion on staircases. To achieve this, we collected actual motion-data of stair walking to formulate remapping functions. During the remapping, we focus on matching the timing of real foot touching the floor with that of virtual foot touching the virtual stair. As discussed earlier [1,3,27,54], this coincidence between visual and physical stimuli provides a believable sensation to users even though there are some differences between visual and physical motions. Then, we additionally exploit the concept of visual gain which changes the mapping ratio between visual and physical motions. Through the application test, we found that the proper amount of visual gain successfully induces stronger illusory self-motion, as discussed in the previous studies [6,14,22,43].

3 Motion data acquisition

To formulate the realistic remapping functions, we first recorded the user’s movement using HMD, and trackers.

Similar to previous VR studies [23,26], we used the HTC VIVE Pro headset, two VIVE controllers, and three VIVE trackers. The controllers were held in both hands. One tracker was fixed around the waist with a strap, while the other two were attached to the shoes. See Fig.1a. The HMD provided a resolution of 1440 ×1600 pixels per eye with a refresh rate of 90 Hz and a field of view of 110^◦. During the data acquisition, users were provided the real-world scene cap- tured through an HMD’s front camera with a resolution of 640×480 pixels.

We recruited 12 volunteers (5 females and 7 males) aged between 21 and 27. We measured two types of motion data:

walking up and walking down the stairs. Each volunteer was asked to walk up or down the stairs, and the order of walking direction was counterbalanced between the volunteers. As each direction was repeated 12 times, each volunteer completed 24 times of walking the stairs. The entire stairs consist of three stairsteps, and each step has a rise of 12 cm and a tread of 30 cm, as shown in Fig. 1b. During walking, the

(3)

Fig. 1 Setup for motion-data acquisition.aThe user’s head, waist, and feet motions are tracked by the VIVE headset and trackers.bStair walking motions are recorded

head, waist, and feet motions were tracked and recorded. To prevent accidents, a staff stood near the stairs.

After data acquisition, we analyzed the motion data.

Owing to the software error, 19 of the 288 trials were removed from the analysis. We first measured the duration of flight (DF) and height difference of the foot between before taking off and reaching the peak during a step (PEAK). In the walking-up motion, the average (μ) and standard deviation (σ) of the DF wereμ =0.80 s andσ =0.27, while those of the PEAK wereμ=27 cm andσ =0.79. Similarly, for the walking-down motion, we measured the DF (μ=0.81 s andσ =0.17) and PEAK (μ=9 cm andσ =0.06).

Then, we computed the frequently observed trajectory by using collected motion data, as shown in Fig.2. Based on this, we subsequently analyzed each trajectory of the head, waist, and feet. We observed that the trajectories of the head and waist were similar. We also observed that the heights of the head and waist did not change during the first step of walking up motions and during the last step of walking down motions.

In contrast, we observed that the trajectories of the feet were a parabolic motion, which significantly differed from those of head and waist. The foot trajectories are also different from the foot motion model designed by Nagao et.al. [35] which is based on the sigmoid function.

4 Motion remapping

Our goal is to make users perceive as if they are walking up and down the virtual stairs, while they are walking on a flat surface in the physical play area. To achieve this, we dynamically remap the user’s physical walking motions to the avatar’s virtual stair walking motion. This is achieved by controlling the height of the virtual avatar’s head, waist,

0 0.5 1 1.5 2 2.5 3

time (s) 0

0.5 1 1.5 2 2.5

height (m)

Walking down the stairs

R_H R_W R_L R_R

0 1 2 3 4

time (s) 0

0.5 1 1.5 2 2.5

height (m)

Walking up the stairs

R_H R_W R_L R_R

Fig. 2 Frequently observed trajectory. Graph shows the recorded data of head (RH), waist (RW), left foot (RL), and right foot (RR) motions and feet. Our technique aims to flexibly control the motions of the virtual avatar by reflecting the user’s current physical walking motions.

Based on the collected actual motion data, we designed two types of remapping functions: foot and head–waist remapping functions. With the foot remapping function, we dynamically control the height of the virtual foot to match the timing of real foot touching the floor with that of virtual foot touching the stairs. With the head–waist remapping function, we dynamically control the height of the virtual head and waist to mimic the height changes of stair walking motions.

4.1 Foot remapping

When climbing or descending the virtual stairs, a significant difference exists between the virtual foot and real foot motions because virtual stairs change the height of the virtual foot, while a flat surface does not change the height of the real foot. During the ascending motion of foot, no difference exists between the virtual foot and real foot motions. Dur- ing the descending motion of foot, conversely, a noticeable deviation exists because the virtual foot touches the stairs

(4)

and stops before the real foot touches the floor. Then, users may suffer from sensory conflicts and the sense of presence is diminished.

To resolve such a mismatch, we dynamically control the height of the virtual foot, while the foot is off the floor. It is difficult to control ascending motion of foot because we do not know how much the user will lift the foot. In contrast, in descending motion of the foot, we know how much the user is going to lower the foot. Therefore, our technique mainly controls the virtual foot in the descending motion. The height of the virtual foot at timesteptis determined as follows:

H_v^f(t)=

⎧⎨

⎩

Hv^f(t−1)+αHr^f(t), ift≤ttop

Hv^f(t−1)+ ^H^v^f⁽^t^top_H^)−ωf ^H^v^s⁽^t^top⁾⁻

r(t_top)− Hr^f(t), otherwise (1) where

Hv^f(t)the height of the virtual foot att Hr^f(t)the height of the real foot att Hr^f(t)the velocity of the real foot att H_v^s(t)the height of the virtual stair att

ωdirection value (1 if walking up, -1 otherwise) ttopthe time when the foot reaches the top initial height of the tracker from the ground αa visual gain parameter.

In the foot remapping function presented in Equation1, ttopdivides the parabolic motion into ascent and descent parts.

As the remapping function is applied differently between ascent and descent parts, the determination of the appropriate value ofttopduring the dynamic remapping is a critical topic of concern. In our technique, ttop is dynamically updated witht once for each step when the following three criteria are satisfied:

• ^H^r^f⁽^t⁻³⁾⁺^H^r^f⁽^t⁻²₄⁾⁺^H^r^f⁽^t⁻¹⁾⁺^H^r^f⁽^t⁾ ≤0.01m/s

• Hr^f(t) > ^h₃^s

• t>t0

wherehsis the rise of each stairstep andt0is the most recent time the foot starts to ascend. The criteria were empirically determined. In the internal test, the proposed technique successfully determined the appropriate value ofttopduring the step, as illustrated in Fig.3.

Applying the proposed technique, users can walk on the VR stairs without sensory conflict. However, it would be insufficient for inducing the realistic sensation of stair walking because the user’s physical movement is still a motion of walking on a flat floor. To provide a more realistic experience, we exploited the concept of visual gain [14,22,37,56], which changes the mapping ratio between visual and physical motions. This ratio can be adjusted by the value ofα.

0 1 2 3 4 5 6

time (s) 0

0.1 0.2 0.3 0.4 0.5

height (m)

H^f_r

0 1 2 3 4 5 6

time (s) -1

-0.5 0 0.5 1

velocity (m/s)

H^f_r

t_top

(a)

(b)

Fig. 3 During the parabolic foot motion, the time the foot reaches the top,t_top, is determined on-the-fly.aHeight of the user’s real foot.b Velocity of the user’s real foot

Figure4illustrates the gain-applied trajectories of a virtual foot in walking up and down the virtual stairs by using our technique. The determination of the appropriate value ofα is discussed in Sect.5.

4.2 Head and waist remapping

When the user’s real foot starts to ascend, the remapping function begins to control the height of the virtual avatar’s head. During the remapping, the height of the virtual avatar’s head attis calculated as follows:

H_v^h(t)=

⎧⎨

⎩

H_v^h(t−1)+H_r^h(t) ift−t0≥T H_v^h(t0)+H_r^h(t)−H_r^h(t0)+ ^ω^h^s

1+e^−aT(t−t0−δ), otherwise (2) where

H_v^h(t)= the height of the virtual head att H_r^h(t)= the height of the real head att

ω= direction value (1 if walking up, -1 otherwise) hs= the rise of each stairstep

T = transition time (0.8 in our implementation) a= transitional stiffness (12 in our implementation) t0= the time the foot starts to ascend

δ= shift parameter (T⁵⁻₁₀²^ω in our implementation) In head remapping function presented in Equation2,t0is set by default to−T and dynamically updated with timet when either the left or right foot starts to ascend. The equation is built upon a sigmoid function, and its transition duration

(5)

0 1 2 3 4 5 6 time (s)

-0.1 0 0.1 0.2 0.3 0.4

height (m)

Walking up the stairs H^f_v ( = 1.2)

H^f_v ( = 1.0) H^f_v ( = 0.8) H^f_r

0 1 2 3 4 5 6

time (s) -0.1

0 0.1 0.2 0.3 0.4

height (m)

Walking down the stairs H^f_v ( = 1.2)

H^f_v ( = 1.0) H^f_v ( = 0.8) H^f_r

Fig. 4 Trajectory of the virtual foot(H_v^f)when the user walks up and down a 12cm high virtual staircase with different visual gains (α = 0.8,1.0,1.2)

and stiffness are adjusted byT anda, respectively. Figure5 shows that the trajectory of the virtual head(H_v^h)successfully mimics the head motion of frequently observed trajectory (RH). Note that the head remapping does not begin with the first step of walking up motions and the last step of walking down motions as observed in the actual motion data.

As discussed earlier, no significant difference exists between the trajectories of the head and waist in walking up and down the stairs. Therefore, we apply the same remapping function to the head and waist. In our internal test, the proposed head and waist remapping functions successfully provided a realistic sensation of walking up and down the stairs.

5 Experiment for visual gain

As visual cues often affect other sensory cues, we exploited visual gain to induce a stronger vertical illusory self-motion and provide a more compelling experience of stair walking.

Initially, we aimed to apply the visual gain to both the foot and head–waist remapping functions. However, the height of the real waist and head is mostly constant during walking on a floor so that there is no room for applying visual gain to motions of waist and head. Therefore, we only applied the visual gain to the foot motions.

Through the internal test, we assumed two hypotheses regarding the determination of the visual gain parameter (α):

0 1 2 3 4

time (s) 0

0.5 1 1.5 2 2.5 3

height (m)

H^h_v H^h_r R_H R_L R_R

0 0.5 1 1.5 2 2.5 3

time (s) 0

0.5 1 1.5 2 2.5 3

height (m)

H^h_v H^h_r R_H R_L R_R

Fig. 5 Simulation results of the head remapping. We assume that a user walks on a flat surface so that the height of the real head(H_r^h)shows a constant height. Then, we can generate the height of the virtual head (H_v^h)by using the left foot motion(R_L)and right foot motion(R_R)of the frequently observed trajectory. As shown in the figure, generated (H_v^h)shows the similar trajectory of the head motion of the frequently observed trajectory(R_H)

1. H1:α >1 would induce stronger self-motion thanα= 1 because the former makes virtual foot motion more noticeable.

2. H2:α <1 would induce stronger self-motion thanα= 1 because the former makes users move the virtual foot with greater physical motion.

In the test, we also observed that applying the overly exceeding or lowering gain generated exaggerated trajectories which were perceived as unrealistic, as discussed in previous studies [4,55]. Taking this into account, we conducted an experiment to test our hypotheses and identify how much visual gain should be applied to enhance the user’s perceived self-motion.

(6)

5.1 Participants

Twenty participants (14 males and 6 females) were recruited for the experiment. The mean (μ) and standard deviation (σ) of height were calculated asμ=169.2 cm andσ =7.46, and those of age were calculated asμ = 22.3 years and σ = 1.49, respectively. All participants were undergraduate or graduate students, with normal or corrected-to-normal vision. Three participants had never experienced a VR HMD, but they knew about the VR technology. Each participant was paid 15 USD for participation.

5.2 Method and procedure

The virtual environment and corresponding physical play area used for the experiment are illustrated in Fig.6a and b, respectively. We used an enclosed environment to increase the user’s perceived self-motion [16,24]. The user’s virtual avatar was rendered during the experiment and animated by the inverse kinematics technique. We expected that this enhances the user’s body-ownership and perceived self- motion [2,33].

In the experiment, participants were asked to either walk up or down the virtual stairs from the stairstep marked with red footprints to the stairstep marked with blue footprints.

Initially, each stairstep was 12 cm with a tread of 30 cm.

However, through the internal test, we have found that some users have a habit of not lifting the foot high enough when walking down the stairs. This reduced the duration of flight (DF) and increased the amount of height change in each timestep, increasing the probability that the user would feel that their body is being controlled strangely. The only solution was to increase DF somehow. To make users lift their foot higher unconsciously, we attached a 6-cm-high bump to the front of the stairs, as shown in Fig.7.

The experiment was run with one experimenter and one safety guard. On arrival, each participant was asked to fill in a consent form, a demographics questionnaire, and the Simulator Sickness Questionnaire (SSQ) [25]. Each participant subsequently underwent 30 min of the training session, which was composed of (1) 10-min of free movement in virtual environment, (2) 10-min of walking up the virtual stairs by applying our technique, and (3) 10-min of walking down the stairs by applying our technique. During the training,α was set at 1.

After the training session, each participant was asked to either walk up or down the four steps of the virtual stairs repeatedly with variedα. The tests for walking up and down the stairs were conducted separately, and the order of the test direction was counterbalanced between participants. Based on the observation from the internal test, we decided to test seven values ofαin the range of[0.4,1.6]with steps of 0.2 for both directions. For each direction, a test block was composed

Fig. 6 Experiment for visual gain.a Virtual environment setting.b Physical play area used for the experiment. The dimensions were about 4m×4m×3m

Fig. 7 A 6-cm-high bump is attached to the front of the virtual stairs to induce users to lift their foot higher.aInitial design.bBump-attached design

of seven trials of stair walking, and each participant went through 10 blocks. Seven trials were defined by seven distinct αvalues, and their order was counterbalanced between each block. A 3-min break was provided between two consecutive blocks.

At the start of each trial, the experimenter placed the user’s virtual avatar in the center of the stairs marked with red footprints and rotated the avatar so that the forward movement in the real world was the test direction. Subsequently, the experimenter asked the participant to wear the HMD and walk to the step marked with blue footprints. Then, the participant walked forward in the physical play area to walk on the virtual stairs, as shown in Fig.6b.

After completing each trial, the HMD screen faded to black and each participant was asked to choose whether the perceived sensation of stair walking was realistic or unrealistic and, if possible, to share the reason why they choose that answer. Next, the experimenter guided the participant to return to the initial position of the physical play area and conducted the next trial. In this experiment, each participant completed 140 trials for about 90 min. At the end of the experiment, each participant was asked to fill out the SSQ again.

5.3 Result

The experimental results are presented in Fig.8, where the x-axis shows the applied visual gain,α, and the y-axis shows the probability that participants evaluate the virtual stair walking as realistic. Similar to analysis of previous

(7)

0.4 0.6 0.8 1 1.2 1.4 1.6 visual gain

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

probability of realistic responses

0.4 0.6 0.8 1 1.2 1.4 1.6

visual gain 0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

probability of realistic responses

μ＋σ μ

μ－σ

μ＋σ μ

μ－σ

Fig. 8 Analysis results of the user responses in the experiment

study [24], the data are fitted with the Gaussian function of the form,y=ae⁻⁽

x−b)2

2c2 , with real numbersa,b, andc. In each direction, meanμof the fitted function was taken as the most realistic gain. We defined the range of acceptable gains byμ-σandμ+σ, whereσdenotes the standard deviation of fitted function. When walking up the stairs, the most realistic gain and range of the acceptable gains were 1.193 and [0.802,1.583], respectively. When walking down the stairs, the most realistic gain and the range of acceptable gain were 0.822 and[0.392,1.253], respectively.

During the experiment, we tracked and recorded the position data of the user’s real foot. Figure9illustrates the average PEAK of the real foot,Hr^f(ttop), in each of the seven types of gain-applied stair walking motions. The average PEAK of the real foot is 41 cm in walking up motion and 32 cm in walking down motion.

We compared the pre- and post-SSQs to investigate whether the experiment induced simulator sickness to participants. The Shapiro–Wilk and Kolmogorov–Smirnov tests at 5% revealed that both the pre- and post-SSQs were not normally distributed. Therefore, we conducted Wilcoxon signed-rank tests and found that no significant difference (Z = −0.59;p = 0.56) exists between pre- (μ = 0.68 andσ =0.63) and post-SSQs (μ=0.77 andσ =0.67).

0.4 0.6 0.8 1 1.2 1.4 1.6 visual gain 0.2

0.3 0.4 0.5

PEAK of real-foot (m)

0.4 0.6 0.8 1 1.2 1.4 1.6 visual gain 0.2

0.3 0.4 0.5

PEAK of real-foot (m)

Fig. 9 Height difference of the real foot between before taking off and reaching the peak during a step; PEAK

5.4 Discussion

During the experiment, no participants reported that they felt latency between physical and virtual movements. Nine- teen participants explicitly mentioned that they felt stronger self-motion of stair walking as the virtual foot showed more noticeable movement (α >1). Furthermore, 18 participants mentioned that they also felt stronger self-motion as their physical motion became more noticeable (α <1). These com- ments were obtained regardless of when walking up or down the stairs. Therefore, we speculated that both hypotheses H1 and H2 were correct. In other words, the stronger self-motion can be induced by either increased virtual foot or real foot motions.

However, it seemed that the appropriate α depends on the walking direction. We speculated that this is because the amount of contribution of visual cues and physical cues to induce realistic self-motion depends on the direction of stair walking motions. As shown in Fig.9, the PEAK of the real foot tended to decrease as α increased, which means that the visual cues and physical cues are in inverse proportion to each other. When walking up the stairs, the most realistic gain was greater than 1. Therefore, we speculated that increasing the visual cues and decreasing the physical cues can provide a more realistic experience when walking up the stairs. In contrast, the most realistic gain was less than 1 when walking down the stairs. Accordingly, we speculated that decreasing the visual cues and increasing the physical cues can provide a more realistic experience when walking down the stairs.

Participants felt uncomfortable when overly increased or decreased visual gains were applied. When the visual discrepancy between the virtual foot and real foot motions increased over a certain threshold, participants had diffi- culty in controlling the foot, lost their body-ownership, and experienced unstable walk. Furthermore, participants often reported that their avatar’s knees were bent too much when applying overly increased visual gains, resulting in loss of avatar control.

As we observed in the motion data described in Sect.3, the average PEAK of the real foot when walking up the stairs was higher than that when walking down the stairs. A similar ten-

(8)

dency was also observed in this experiment. Interestingly, the average PEAK in the experiment was much greater than that in the actual motion data described in Section3. The average PEAK increased by 50.8% when walking up the stairs (27 to 41 cm), while it increased by 377.8% when walking down the stairs (9 – 34 cm). In both cases, the amount of increase was larger than the size of the bump (6 cm). We speculate that this is due to the lack of sensory modalities. Humans are known to accept multiple sources of sensory information and take a weighted average across the sensory signals [15]. How- ever, participants were provided insufficient physical cues of stair walking as they merely walked the flat surface. To com- pensate for this, the participants seemed to exaggerate their physical motion more than usual.

6 Application test

In the experiment for visual gain, the most realistic visual gain and the range of acceptable visual gains were identified for each direction of stair walking. By applying the most realistic gains to our locomotion technique, we expect that our technique can provide a realistic stair walking experience and increase the user’s perceived presence. To validate our expectations, we conducted the application test which is a comparative within-subject test with four types of test conditions for each direction, as shown in Table1. In the gain applied conditions, we applied the most realistic gains to our locomotion technique. The major objective of this test is to investigate whether our locomotion technique increases the user’s perceived presence and whether our technique provides either a positive or negative user experience.

6.1 Participants

Twenty-four participants (17 males and 7 females) were newly recruited for this experiment. The mean and standard deviation of height wereμ = 170.06 cm and σ = 7.36, and those of age wereμ = 22.17 years and σ = 1.58, respectively. All participants were undergraduate or graduate students with normal or corrected-to-normal vision. Four participants had never experienced a VR HMD, but they knew about the VR technology. Each participant was paid 5 USD for participation.

6.2 Method and procedure

The virtual environment and corresponding physical play area used for the application test are illustrated in Figure10- (a) and -(b), respectively. As in the experiment for visual gain, we used an enclosed environment and rendered virtual self-avatar to increase the user’s perceived self-motion. The

Fig. 10 Application test.aThe virtual environment used for the test.b The corresponding physical play area used for the test

dimension of the stairstep used in this test was also the same as in the experiment for visual gain.

The experiment was run by an experimenter with one safety guard. On arrival, each participant was asked to fill in a consent form, a demographics questionnaire, and the SSQ. Subsequently, each participant underwent 30 min of the training session, as in the experiment for visual gain.

After the training session, each participant was asked to start the main test composed of four test conditions presented in Table1, and the order of the test conditions was counterbalanced between participants. The main test was conducted individually for each of the climbing and descending the stairs, and the order of the test direction was counterbalanced between participants.

At the start of each case, the experimenter placed the user’s virtual avatar on the red footprints marked upstairs or down- stairs depending on the current test direction. Subsequently, the experimenter asked the participant to wear the HMD and perform the given task. The task involved picking up a virtual book and then moving it to a shelf on another floor, as shown in Fig.10a and b. When the participant completed the given task, the HMD screen faded to black, and the experimenter guided the participant to return to the initial position of the physical play area. After performing each case twice in succession, we asked participants to fill in the Igroup Presence Questionnaire (IPQ) [45] as well as a subjective evaluation questionnaire (SEQ), as shown in Table2. Both questionnaires were answered on a 7-point Likert-scale. Sub- sequently, we conducted an open-ended interview to gather the participant’s thoughts, feelings, experiences, and pref- erences. In the application test, each participant completed the given task 16 times (4 test conditions×2 directions× 2 times repetition) for about 30 min. At the end of the test, each participant was asked to fill in the SSQ again.

6.3 Result

The IPQ uses four subscales: general presence (G), spatial presence (SP), involvement (INV), and experienced real- ism (REAL). The analysis result of IPQ data is presented in Figure11. The Shapiro–Wilk and Kolmogorov–Smirnov tests revealed that not all scores of factors were normally distributed. Therefore, we performed a Friedman test. For

(9)

Table 1 Four types of test conditions in the application test

Without Gain applied Gain applied

Without bump Control condition (CC)α=1 Gain-applied condition (GC)α=1.193 for walking up α=0.822 for walking down

Bump Bump-applied condition (BC)α=1 Gain and bump-applied condition (GBC)α=1.193 for walking upα=0.822 for walking down

Table 2 Subjective evaluation questionnaire

Category Question

Aesthetic The shape of the stairstep looks pleasant aesthetically.

Discomfort The shape of the stairstep is discomfort to walk.

Stability I could walk the virtual stairs stably.

Reality This technique provides a realistic sensation of stair walking.

Utility It seems that this technique will be adopted by many other VR applications and improve my future VR experience.

walking up the stairs, the test revealed significant differences between G (X²(3) = 24.07,p < 0.001), SP (X²(3) = 18.58,p<0.001), and REAL (X²(3)=24.40,p<0.001).

As a post hoc analysis, we conducted Wilcoxon signed-rank tests with a Bonferroni correction. In terms of G, significant differences were observed between CC and GC (Z =

−2.65;p<0.008), CC and GBC (Z = −3.27;p<0.001), GC and BC (Z = −2.95;p < 0.008), and BC and GBC (Z = −3.36;p <0.001). In terms of SP, significant differences were observed between CC and GC (Z = −2.67;p<

0.008), CC and GBC (Z = −3.13;p < 0.008), and BC and GBC (Z = −3.45;p < 0.001). In terms of REAL, significant differences were observed between CC and GC (Z = −2.67;p <0.008), CC and GBC (Z = −3.13;p <

0.008), and BC and GBC (Z = −3.45;p < 0.001). For walking down the stairs, the test revealed significant differences between G (X²(3)=13.95,p<0.05), SP (X²(3)= 12.58,p <0.01), and REAL (X²(3)=18.14,p <0.001) factors. Post hoc analysis was conducted using Wilcoxon signed-rank tests with a Bonferroni correction. In terms of G, significant differences were observed between CC and GC (Z = −2.82;p <0.008), CC and GBC (Z = −3.42;p <

0.001), and BC and GBC (Z = −4.13;p < 0.001). In terms of SP, significant differences were observed between CC and GBC (Z = −2.68;p <0.008), and BC and GBC (Z = −2.66;p < 0.008). In terms of REAL, significant differences were observed between CC and GC (Z =

−3.04;p<0.008), CC and GBC (Z = −3.11;p<0.008), and BC and GBC (Z = −3.51;p<0.001).

The responses to the SEQ are shown in Fig. 12. The Shapiro–Wilk and Kolmogorov–Smirnov tests revealed that not all responses to each question were normal distribution.

Fig. 11 IPQ results in the application test. Every question was answered on a 7-point Likert scale (1: strongly disagree, 7: strongly agree).

The graph plots the median (-), mean (x), interquartile ranges, and maximum/minimum values (whiskers). The square brackets indicate significant differences(∗ ∗p<0.01,∗ ∗ ∗p<0.001)

Therefore, we performed the Friedman test. For walking up the stairs, the test revealed significant differences between reality (X²(3) = 18.17,p < 0.001) and utility questions (X²(3)=27.91,p <0.001). As a post hoc test, Wilcoxon signed-rank tests were conducted with a Bonferroni correction. In terms of reality, significant differences were observed between CC and GC (Z = −2.78;p<0.008), CC and GBC (Z = −3.52;p < 0.001), GC and BC (Z = −2.85;p <

0.008), and BC and GBC (Z = −3.23;p<0.001). In terms of utility, significant differences were observed between CC and GC (Z = −3.55;p < 0.001), CC and GBC (Z =

−3.58;p <0.001), GC and BC (Z = −3.16;p <0.008), and BC and GBC (Z = −3.29;p < 0.001). For walking down the stairs, the test revealed significant differences between reality (X²(3) = 24.92,p < 0.001) and utility questions (X²(3)=23.71,p <0.001). As a post hoc test, Wilcoxon signed-rank tests were conducted with a Bonfer-

(10)

Fig. 12 Responses to the subjective evaluation questionnaire. Every question was answered on a 7-point Likert scale (1: strongly disagree, 7: strongly agree). The graph plots the median (-), mean (x), interquartile ranges, and maximum/minimum values (whiskers). The square brackets indicate significant differences(∗ ∗p<0.01,∗ ∗ ∗p<0.001) roni correction. In terms of reality, significant differences were observed between CC and GC (Z = −3.54;p <

0.001), CC and GBC (Z = −3.32;p < 0.001), and BC and GBC (Z = −3.21;p <0.001). In terms of utility, significant differences were observed between CC and GC (Z =

−3.49;p<0.001), CC and GBC (Z = −2.93;p<0.008), GC and BC (Z = −3.13;p < 0.008), and BC and GBC (Z = −2.86;p<0.008).

The pre- and post-SSQs were compared. The Shapiro- Wilk and Kolmogorov-Smirnov tests revealed that both the SSQs were not normally distributed. Therefore, we conducted Wilcoxon signed-rank tests and found that there was no significant difference (Z = −0.653;p=0.516) between pre- (μ = 0.44 and σ = 0.45) and post-SSQs (μ = 0.52 andσ =0.57).

6.4 Discussion

The IPQ analysis revealed that the sole use of our locomotion technique without applying visual gain and bump can provide plausible sense of presence when walking up (G=4.09) and down (G=4.43) the virtual stairs. It also revealed that applying the appropriate visual gain to our locomotion technique can increase user’s sense of presence both in walking up (G=4.61) and down (G=5.00) the virtual stairs. On the other hand, attaching the bump to the virtual stairs did not significantly affect the user’s sense of presence. Although

we did not use virtual bump to make users lift their foot higher, our technique seems to successfully induce stronger self-motion.

There was a concern that attaching bumps to the stairs makes stairs aesthetically unpleasant and diminishes the user’s sense of presence. However, the IPQ results showed no significant reduction in the user’s presence when attaching bumps. Furthermore, the average scores of the aesthetic question in the case of using bumps (BC and GBC) were not significantly different from those in the case without bumps (CC and GC). In our field observations, users generally accepted the bump as some kind of stair design.

Given that users walked on a flat physical surface while walking on virtual stairs, they may feel the discrepancy between physical and virtual motion. Therefore, we investigated whether our technique leads to physical discomfort or unstable walking. However, the average scores of the discomfort question in all test conditions were about 2 and the average scores of the stability question in all test conditions were about 5.

Most of the participants replied that the proposed technique provided the realistic sensation of climbing and descending staircases. In particular, there were many com- ments that the movements of the head and waist were surprisingly natural. Regarding leg motions, participants generally agreed that applying the visual gains makes their stair walking sensations clearer and more noticeable. One participant suggested a penalty system in which users fall off the stairs if they fail to move their feet correctly when walking the stairs. He said this would make the users climb and descend the stairs cautiously as in real life, which would make their sensation stronger.

Attaching bumps to virtual stairs does not seem to have a significant effect on making the feeling of walking the stairs realistic. However, since we observed that attaching bump to the virtual stairs significantly changed the PEEK in the experiment for visual gain, it can be used for other purposes. For example, some users may have a walking habit of dragging their feet instead of raising their feet when walking down the stairs. Then, the ascending phase of the foot will disappear and descending phase of the foot will be shortened. As discussed earlier, this has a negative impact when applying our locomotion technique. By using a bump to make users lift their foot, such cases can be minimized.

7 Conclusion

In this paper, we proposed a motion remapping-based locomotion technique to provide the sensation of virtual stair walking. Our technique dynamically remaps the user’s walking motion on a flat surface to the avatar’s walking motion on staircases. With this technique, we conducted an experiment

(11)

and an application test. In the experiment, we investigated how much visual gain should be applied to foot motion to induce a realistic sensation of stair walking. As a result, we found that applying visual gains of 1.193 and 0.822 to motions of climbing and descending the stairs were accepted as the most realistic, respectively. By using this result, we conducted an application test. Through the test, we observed that sole use of our technique without applying visual gain and bump successfully provides the realistic sensation of walking the stairs in the virtual environment.

We also observed that application of a natural visual gain can enhance the user’s sense of presence without causing discomfort or postural instability.

Our technology can be improved further in many directions. Combining our technology with the redirected walking technique is a promising improvement. During the experiment, we used a few stairsteps owing to the limited physical play area. We expect more stairsteps, and complex virtual environment can be used if appropriate translational and rota- tional gains are applied to our technique. Our technique can also be improved to enable VR users to walk not only the staircases but also terrain with small elevation differences.

In our remapping equation, the height difference between consecutive staircases is a key factor to make differences between virtual and physical motions. If the factor becomes the height difference in the terrain surface, our technique can provide the locomotion experience on the uneven terrain surface. Implementing the penalty system that makes users fall off the stairs when they incorrectly move their feet also seems interesting extension. Through this penalty system, it is expected that users can naturally learn the walking posture required by our locomotion technique.

We envision that our technique can be easily applied to many other VR applications because it only needs visual cue control. By using this technique, VR users can enjoy vari- ous activities that can occur on the staircase, such as sudden battles on the staircases. This technique can also be used for VR tourism or VR rehabilitation treatment as it improves the overall locomotion experience by allowing virtual stair walking.

Declarations

Conflict of interest The authors declare that they have no conflict of interest. This research was supported by Basic Science Research Pro- gram through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2020R1F1A1076528).

References

1. Abtahi, P., Follmer, S.: Visuo-haptic illusions for improving the perceived performance of shape displays. In: Proceedings of the

2018 CHI Conference on Human Factors in Computing Systems, pp. 1–13 (2018)

2. Asjad, N.S., Adams, H., Paris, R., Bodenheimer, B.: Perception of height in virtual reality: a study of climbing stairs. In: Proceed- ings of the 15th ACM Symposium on Applied Perception, pp. 1–8 (2018)

3. Azmandian, M., Hancock, M., Benko, H., Ofek, E., Wilson, A.D.:

Haptic retargeting: Dynamic repurposing of passive haptics for enhanced virtual reality experiences. In: Proceedings of the 2016 chi conference on human factors in computing systems, pp. 1968–

1979 (2016)

4. Bergström, J., Mottelson, A., Knibbe, J.: Resized grasping in vr:

Estimating thresholds for object discrimination. In: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, pp. 1175–1183 (2019)

5. Bhandari, J., MacNeilage, P.R., Folmer, E.: Teleportation without spatial disorientation using optical flow cues. In: Graphics interface, pp. 162–167 (2018)

6. Biocca, F., Kim, J., Choi, Y.: Visual touch in virtual environments:

an exploratory study of presence, multimodal interfaces, and cross- modal sensory illusions. Presence Teleop. Virtual Environ.10(3), 247–265 (2001)

7. Botvinick, M., Cohen, J.: Rubber hands ‘feel’touch that eyes see.

Nature391(6669), 756–756 (1998)

8. Bouguila, L., Sato, M.: Virtual locomotion system for large-scale virtual environment. In: Proceedings IEEE Virtual Reality 2002, pp. 291–292. IEEE (2002)

9. Bouguila, L., Sato, M., Hasegawa, S., Naoki, H., Matsumoto, N., Toyama, A., Ezzine, J., Maghrebi, D.: A new step-in-place locomotion interface for virtual environment with large display system.

In: ACM SIGGRAPH 2002 conference abstracts and applications, pp. 63–63 (2002)

10. Bozgeyikli, E., Raij, A., Katkoori, S., Dubey, R.: Point & teleport locomotion technique for virtual reality. In: Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play, pp.

205–216 (2016)

11. Bruder, G., Steinicke, F., Hinrichs, K.H.: Arch-explore: A natural user interface for immersive architectural walkthroughs. In: 2009 IEEE Symposium on 3D User Interfaces, pp. 75–82. IEEE (2009) 12. Burns, E., Razzaque, S., Panter, A.T., Whitton, M.C., McCallus, M.R., Brooks, F.P.: The hand is slower than the eye: A quantitative exploration of visual dominance over proprioception. In: IEEE Pro- ceedings. VR 2005. Virtual Reality, 2005., pp. 3–10. IEEE (2005) 13. Darken, R.P., Cockayne, W.R., Carmein, D.: The omni-directional treadmill: a locomotion device for virtual worlds. In: Proceedings of the 10th annual ACM symposium on User interface software and technology, pp. 213–221 (1997)

14. Dominjon, L., Lécuyer, A., Burkhardt, J.M., Richard, P., Richir, S.: Influence of control/display ratio on the perception of mass of manipulated objects in virtual environments. In: IEEE Proceedings.

VR 2005. Virtual Reality, 2005., pp. 19–25. IEEE (2005) 15. Ernst, M.O.: A bayesian view on multimodal cue integration.

Human Body Percept. Inside Out131, 105–131 (2006)

16. Festl, F., Recktenwald, F., Yuan, C., Mallot, H.A.: Detection of lin- ear ego-acceleration from optic flow. J. Vision12(7), 10–10 (2012) 17. Freitag, S., Rausch, D., Kuhlen, T.: Reorientation in virtual environments using interactive portals. In: 2014 IEEE symposium on 3D user interfaces (3DUI), pp. 119–122. IEEE (2014)

18. Hollerbach, J.M., Xu, Y., Christensen, R., Jacobsen, S.C., et al.:

Design specifications for the second generation sarcos treadport locomotion interface. In: Haptics Symposium, Proc. ASME Dynamic Systems and Control Division, vol. 69, pp. 1293–1298 (2000)

19. Iwata, H., Yano, H., Fukushima, H., Noma, H.: Circulafloor [locomotion interface]. IEEE Computer Gr. Appl.25(1), 64–67 (2005)