Kortum P. (ed.) HCI Beyond the GUI. Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces
Подождите немного. Документ загружается.
FIGURE
4.13
Konami Dance Dance Revolution sensor pad.
The game player presses switches with her feet (and sometimes hands) to
coordinate with on-screen dance steps. (Image courtesy of the Department of
Computer Science, UNC, Chapel Hill.)
FIGURE
4.14
Surround-screen display.
The three-wall-plus-floor projection system is seen (
left
) with the projectors off
and (
right
) displaying a scene with a virtual human on the left and a real human
user on the right. (Courtesy of University College London, Department of
Computer Science.)
4 Locomotion Interfaces
122
In addition to visual displays, there are displays for other senses. As mentioned
earlier, people can get the sensation that they are moving from auditory cues alone,
without any visuals (Lackner, 1977). While we have seen one virtual-environment
system that uses only auditory displays (from AuSIM), most systems use auditory
displays as a supplement to the primary visual display. Hearing your own footsteps
is one cue that helps you remain oriented during movement, and subtle echoes tell
you much about the size of the room you are in and your position in it.
Another display used in whole-body locomotion interfaces is a motion plat-
form—the user is sitting or standing on a platform that is physically moved under
computer control. Some platforms tilt along one or more axes, whereas others tilt
and translate. Motion platforms are most commonly used in flight simulators. The
deck of the ship in Disney’s Pirates of the Caribbean ride has a motion platform as
well as a surround screen visual display (Schell & Shochet, 2001). The motion plat-
form tilts the deck so that there is a match between the virtual waves the user sees
and the real rolling of the ship’s deck (Figure 4.15).
FIGURE
4.15
Disney’s Pirates of the Caribbean ride.
This ride has surround screens (in the distance) and a motion platform (in the
shape of a pirate ship’s deck). Players aim the cannons and steer the ship using the
physical props. (Courtesy of Disney.)
4.2 Technology of the Interface
123
Finally, a motorized treadmill, where the computer system continually
adjusts the speed and direction of the moving belt in response to the user’s inputs
and position, can also be considered as a kind of motion display. The University of
Utah Treadport (Figure 4.2) combines several kinds of displays into a single loco-
motion interface: It has a surround screen visual display, a motorized treadmill,
a motion platform that tilts the entire treadmill up and down (to simulate differ-
ent slopes), and a force display that pushes and pulls a harness worn by the user
(to simulate momentum).
4.3
CURRENT IMPLEMENTATIONS
OF THE INTERFACE
This section discusses the current state of the art of locomotion interfaces. For all
of the examples described, at least one prototype has been built. Unfortunately,
several of them no longer exist (as far as we know).
4.3.1 Flying and Leaning
The most common locomotion technique in virtual-reality systems is flying using
a joystick or some other hand controller. When the user pushes a joystick or
presses a button, she moves forward in the virtual scene. She can still move about
locally by leaning or taking a real step in any direction, assuming her head is
tracked by the VE system. The effect is similar to that of walking about on a
moving flatbed truck or flying carpet (Robinett & Holloway, 1992). When the user
presses the button, the truck moves forward in the virtual scene and the user can
simultaneously move about on the truck bed.
There are significant variations in how flying is implemented. Pushing the
joystick or the button signals that the user intends to move. The direction of
motion can be established in various ways. In some VE systems, the user moves
in the direction her head (or nose) is pointing (
head directed
). Other interfaces
move the user in the direction in which she is pointing with her hand controller.
Still others interpret forward using a vehicle metaphor: Forward is toward the
center-front wall of the multiwall projection system.
Similar to flying, leaning techniques move the user in the virtual scene in the
direction in which she is leaning (LaViola Jr. et al., 2001; Peterson et al., 1998). Most
leaning interfaces also control speed—the farther the user leans, the faster she moves.
Leaning has the advantage of not requiring a hand controller, leaving the hands free.
4.3.2 Simulated Walking and Walking
Locomotion interfaces that require users to make walking motions with their legs
are attractive because of their similarity to real walking. Two primary methods
4 Locomotion Interfaces
124
allow a person to feel he is walking but keep him in a restricted space: treadmills
and walking-in-place.
Treadmills and Treadmill-Like Devices
There are a number of techniques that simulate the physical act of walking with
treadmills (Brooks 1998; Hollerbach et al., 2000). As users walk forward on the
treadmill, they move forward in the virtual scene. Using a passive treadmill
(i.e., without a motor) requires a physical effort beyond that of walking in the real
world because of the mass and friction of the belt and rollers. Motorized treadmills
address this problem and still allow the user to change speed. Motorized treadmills
sense the user’s position on the belt and vary the belt motor speed to keep the
user centered. The disadvantage of motorized treadmills is the noise and potential
for injury—many virtual-locomotion treadmills have the users wear a safety tether
to prevent them from falling. All of the treadmills discussed thus far have the lim-
itation that the treadmill has a preferred orientation; it is difficult, disorienting,
and often impossible to turn on the spot in the virtual scene. The University of
North Carolina’s treadmill (Brooks, 1988), for example, had handlebars to steer
like a bicycle.
To allow turning on the spot, several groups have developed 2D treadmills
(Darken, Cockayne, & Carmein, 1997; Iwata, 1999) or omnidirectional treadmills
(Virtual Devices) where the user can walk in any direction on the ground plane.
As can be seen in Figure 4.16, these devices are large, noisy, and mechanically
complex. Darken and colleagues (1997) commented about the omnidirectional
treadmill that it was hard to change directions while running because it felt like
“slippery ice.”
Another approach to 2D treadmill-like locomotion is having the user
contained inside a very large sphere (akin to a human-sized “hamster ball”) that
is held in place while it rotates. There are at least two implementations of
this idea—the VirtuSphere and the CyberSphere. The VirtuSphere (Figure 4.17)
is passive, whereas the CyberSphere is motorized. As of this writing, several Virtu-
Spheres have been installed, but we are unaware of any working CyberSpheres
having been built.
Walking Interfaces
When using a walking-in-place (WIP) interface, the user makes walking or step-
ping motions, that is, lifting the legs, but stays on the same spot physically. The
locomotion interface detects this motion and moves her forward in the virtual
scene (Slater, Usoh, & Steed, 1995; Templeman, Denbrook, & Sibert, 1999; Usoh
et al., 1999). Like flying, WIP does not require a large lab or tracking space.
Like a treadmill, the user makes motions that are similar to real walking. Virtual-
reality system users who walk-in-place have a greater sense of presence than
those who fly with a joystick or hand controller (Slater et al., 1995; Usoh et al.,
4.3 Current Implementations of the Interface
125
FIGURE
4.16
2D treadmills.
Left:
The Torus treadmill, which consists of many small one-dimensional
treadmills connected together such that they become the links of a giant treadmill.
Center:
When the small treadmills are rotating at the same speed and the large
treadmill is stopped, the user walks in the direction of the small treadmills.
Right:
When the small treadmills are stopped but the large one is moving, the user
walks in the direction of the large treadmill.
Source:
From Iwata (1999);
#
Nivoo
Iwata.
FIGURE
4.17
Transparent version of the VirtuSphere (standard versions are made of a
mesh).
The users are wearing wireless headsets that provide visual feedback while
moving. (Courtesy of VirtuSphere, Inc.)
4 Locomotion Interfaces
126
1999). Walking-in-place is the subject of this chapter’s case study and is explored
in greater detail there.
If the virtual scene is the same size as or smaller than the tracked space, then
real walking is feasible. Here the user’s movement in the virtual scene corre-
sponds exactly to her movement in the lab: If she walks 5 m in the lab, she also
walks 5 m in the virtual scene. When virtual spaces grow very large, it is sometime
necessary to augment a real-walking interface with a magical interface.
4.3.3 Magical and Unnatural Interfaces
When the virtual scene is very large and the task requires the user to visit
faraway places, mundane interfaces may be augmented with
magical
interface
techniques that allow the user to quickly, or even instantaneously, move over
long distances or to a completely different scene. Current techniques include
teleportation
to and from special locations within the scene, going through
special portals
that take the user to a new scene, or a special locomotion mode
that switches the user’s pace from a normal walking speed to a
hyperdrive
mode.
A concern with magical locomotion techniques is that the user may become
disoriented when the scene changes rapidly. Alerting the user that something
special is going to happen—for instance,byrequiringhertogotoaspecialplace
or through a doorway—prepares her for an abrupt change of location. Gradually
speeding up and slowing down at the start and end of a high-speed move is less
disturbing to users than abrupt speed changes. A lovely example of context-
preserving movement over great distance is the Google Earth mapping service:
The viewpoint zooms away unt il the whole Earth is visible, the Earth then
rotates until the new target location is centered, and then the viewpoi nt zooms
back in.
The Digital ArtForms SmartScene Visualizer product
3
and Mine
´
(1997) have
demonstrated techniques where the user can grab the virtual scene and move it
toward himself. By repeatedly grabbing points in the virtual scene and pulling
them in, the user can locomote from one place to another. Even less like the real
world (but no less fun), Stoakley’s Worlds-in-Miniature technique has the user
manipulate a handheld, dollhouse–sized model of the virtual scene. The user
moves in the virtual scene by moving a doll (representing herself) to the desired
location in the miniature virtual scene. Then the miniature virtual scene grows
to become human scale and the user finds herself in the desired new location in
the virtual scene (Stoakley, Conway, & Pausch, 1995). Figure 4.18 illustrates what
the user sees with the Worlds-in-Miniature technique.
3
SmartScene was originally developed by MultiGen.
4.3 Current Implementations of the Interface
127
4.4
HUMAN FACTORS OF THE INTERFACE
Section 4.1.1 showed that humans rely on a complex combination of sensory input
to plan and control their movement. The sensory complexity is there whether the
motion is controlling a real or a virtual vehicle or controlling real or virtual move-
ment on foot. In this section, we again focus on locomotion on foot, and will dis-
cuss factors that make the interface natural (or at least intuitive) to use and
elements of system design that minimize the risk of simulator sickness.
4.4.1 Making the Interface Feel Natural
We are a long way from being able to place sensors on users that even approxi-
mate the number, variety, and sensitivity of the sensory inputs that control natu-
ral human movement. The sensor inputs to the locomotion interface are
impoverished when compared to the inputs that control real movement. The job
FIGURE
4.18
Worlds-in-Miniature interface as seen by the user.
The small room with the checkerboard floor (in the foreground) is held in the
user’s hand (invisible here), and is identical to the full-scale room in which the
user is standing (background). To move herself in the full-scale room, she moves a
tiny doll (not pictured) in the miniature handheld room.
Source:
Adapted from
Stoakley et al. (1995);
#
Randy Pausch.
4 Locomotion Interfaces
128
of the interface designer, then, is to make trade-offs to maximize the correspondence
of virtual locomotion to real locomotion in terms of the movements the user makes
and
the movements the user perceives through feedback to her various senses.
If you are designing a locomotion interface, you have to decide what motion
by which part of the user’s body will generate a particular type of locomotion. Fac-
tors in this choice are not only how like natural motion it is, but also whether this
motion will stimulate other senses—in particular the vestibular system—in the
same way that natural motion would.
First, as the interface designer, you have to choose one or more tracker sys-
tems, decide what parts of the body to track, and determine how to interpret those
tracker readings. Natural forward motion is relatively easy to accomplish in a
walking-in-place interface with only a few sensor input signals, such as if trackers
on the knees show the feet are going up and down, then move forward. If the design
criteria require that the user be able to move sideways and backward while looking
forward, it becomes much more difficult to define a natural motion that can be
detected from the output of only two sensors. The Gaiter system enabled sideways
and backward motion by having the user make distinctive gestures with her legs
(Templeman et al., 1999). The user swings her leg sideways from the hip to move
sideways and kicks her heels up behind her when stepping to move backward.
In fact, users can move in any direction by extending their legs in the desired direc-
tion. While these movements are not the way we normally move sideways or back-
ward, the gestures are somewhat consistent with the intended direction of motion.
Walking-in-place provides many of the same sensory cues as really walking,
except that the user experiences, and senses, none of the physical forces asso-
ciated with forward momentum. The part of the interface most critical for making
feedback natural are the algorithms executed in the function box in Figure 4.9
labeled “Interpret signals and convert to D POV.” To date, almost all of these algo-
rithms have been very simple—detect a step, move forward a step. A few inter-
faces have begun to improve the naturalness of the visual feedback with a more
complex model that reflects, frame to frame, the subtle variations in velocity that
occur in natural human walking.
4.4.2 Higher-Fidelity Reproduction of Human Gait
When we walk through the real world, the position of our viewpoint changes
continuously, but not at a constant speed. Our speed goes through cycles of accel-
eration and deceleration as we step. The graph in Figure 4.19 (
right
) shows how
our speed varies: There is a dominant pattern of speeding up as we start followed
by a
rhythmic phase
of near constant speed that lasts until we slow down to stop. The
smaller perturbations in speed are the small accelerations and decelerations that
occur as we take steps (Inman, 1981). Ideally, if we measured and plotted the speed
for users of our virtual-locomotion interface, it would look much like this figure.
The ideal is difficult to achieve, but our approximations are getting better.
4.4 Human Factors of the Interface
129
We have experimented with three methods of allocating movement to the
change in eye position per frame: Insert the entire step length in one frame,
divide the step length into uniform segments and insert the parts over several
frames, and divide the step length into nonuniform segments and insert the seg-
ments over several frames (Figure 4.19,
left
). In informal testing, users preferred
the last method. Note that even the last waveform only crudely approximates the
pattern of the rhythmic phase of real human walking.
4.4.3 Avoiding Simulator Sickness
Locomotion interfaces, just like flight simulators, have the risk of making the user
sick. After exposure to virtual environments, some percentage of users report some
sickness symptoms: dry mouth, nausea, dizziness, visual aftereffects (flashbacks),
pallor, sweating, ataxia (loss of balance), and even vomiting. VE practitioners com-
monly refer to this phenomenon as simulator sickness, or
cybersickness
. The occur-
rence of these symptoms varies widely from person to person and among VE
systems. Twenty to forty percent of military pilots suffer from simulator sickness,
depending on the simulator (Kolasinski, 1995).
Velocity
Time (frames)
(a)
(c)
Time (frames)
Velocity
(b)
Time (frames)
Velocity
Time
Average speed during
rhythmic phase
Speed
Development
phase
Rhythmic
phase
Decay
phase
FIGURE
4.19
Walking viewpoint and start-up speed variations.
Left:
Three methods of moving the viewpoint over the course of a few virtual
steps. (a) All change in eye position is made at a single instant—user appears to
snap forward with each step. This makes the motion seem jerky. (b) The change is
distributed uniformly over several frames (a few hundred msec). Users found the
glide forward/stop/glide forward/stop nature of this technique very disturbing.
(c) Change is distributed over several frames, with the amount decreasing over
time. Velocity is highest immediately after the foot strike, and then it decays.
Right:
Speed variations as a person starts walking, walks at a steady pace, and
stops. Note the small variations in speed that are associated with each step, even
during the rhythmic phase.
Source:
Adapted from Inman et al. (1981).
4 Locomotion Interfaces
130
Consequences of Simulator Sickness
Sometimes simulator sickness symptoms linger for days after using a locomotion
interface, affect motor control and coordination (Draper, 1998), and can impair a
person’s performance in the real world (e.g., while driving a car or flying a plane).
Kennedy and Lilienthal (1995) and others (Cobb & Nichols, 1998) have proposed
quantifying the severity of simulator sickness as a mapping (using measures of a
pilot’s ability to keep balance) to blood alcohol level—the legal metric of alcohol
intoxication used to determine if a person is fit to drive.
Factors That Aggravate Simulator Sickness
It is useful to consider how characteristics of the user and of the system affect the
level of sickness suffered. The factors can be divided into those of the individual
user and those of the system or simulator. Qualities that correlate with decreased
susceptibility to simulator sickness include being in good health (hangovers, med-
ications, stress, illness, and fatigue all increase sickness) and previous experience
with simulators or locomotion.
Table 4.2 lists the characteristics of systems divided into two categories. The
characteristics in the upper list are technical shortcomings of the equipment.
As technologies improve, one expects these shortcomings to be reduced and
TABLE 4.2 Qualities of VE Systems and Flight Simulators That Increase
Simulator Sickness
Equipment Shortcomings
F Tracker inaccuracies (temporal and spatial)
F Low update rate
F High latency/lag
F Mismatches between display parameters and image generator’s
parameters (e.g., incorrect FOV setting)
F Display flicker
F Headset weight
Desirable Functions of System
F Stereo display
F Long exposure duration
F Wide FOV display
F Free head movement
F Viewpoint motion controlled by someone other than the viewer
F High rates of simulated linear or rotational acceleration
Source:
Adapted from Kolasinski (1995).
4.4 Human Factors of the Interface
131