Kortum P. (ed.) HCI Beyond the GUI. Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces
Подождите немного. Документ загружается.
If all devices are not quickly updated as soon as a change to an address has been
made, and the user tries to use the address on another device, then the subsequent
selection of that address may not be accurate. Additional confusion may occur,
depending on whether the user recognizes that the update has not yet occurred.
Consequently, it is important that information be updated in a timely manner so a
user can trust that changes made using one device will be available on all devices.
Confusion about data or application behavior because of changes made in
other UIs can happen in many situations. For instance, in a voice mail application
that can be reached over the Web or the phone, one member of a household might
listen to a message streamed over the Web and then delete it. At about the same
time another member of the household is checking messages from her cell phone
and hears that there is a new message. However, when she chooses to listen to the
message, there will be no message to play because it has been deleted.
Typically, the precedence for changes to an account or account data from differ-
ent UIs is whichever is the most recent. So, in the previous example, usually the
household member who gave the command to delete or save the message first would
establish the precedence for this specific set of information. However, other criteria
for precedence could be established based on user needs and contexts of use for the
application. For example, if there are master and secondary accounts, changes made
by the master user would override changes by the secondary account users.
Preventing data corruption or lost work when accessing the same data from
more than one UI can be critical. There are few situations more disheartening and
frustrating for the typical user who has just spent time and effort accomplishing a
task, only to see it lost. Worse yet is when a user thinks he had successfully finished
a task, than finding out later that he did not. For example, a case worker in the field
calls into an automated system to make updates to a patient’s file immediately after
an in-home visit. She makes changes and hangs up, believing that the file has been
updated. Meanwhile, her procrastinating coworker is at their office computer and
just getting around to completing updates from the previous week’s visit to the same
patient. The coworker’s changes are made later and overwrite the changes of the
first case worker. In this case, a lockout system to prevent accessing the same record
at the same time could prevent this scenario, or an application that could track a
history of changes and the individuals who made them could help.
Although most of these scenarios have been about preventing negative conse-
quences of a user or users accessing the application through multiple UIs, MMUIs
can be beneficial, if taken advantage of, for the appropriate tasks. A user may
want the ability to repeatedly switch back and forth between the UI modalities
to more quickly or easily accomplish a task. For example, although the primary
method for setting up a voice mail application is a setup wizard on a website, it
may be much more convenient for the user to record greetings using a phone
interface at the appropriate time. In this case, accomplishing the task (setting up
the voice mail interface) uses the appropriate interface modality to facilitate the
components of the task.
11 Multimode Interfaces: Two or More Interfaces
362
Another major technological consideration for MMUI applications is the
underlying software and systems that enable the UI. Although not directly related
to UI design, the supporting infrastructure can have a major impact on the quality
of user interaction with the application. This is a practical consideration for any UI
design, but it is even more important for MMUIs. This is because there is a far
greater possibility for a disjointed user experience if an MMUI is created by stitch-
ing together several independently developed UIs to create the multimodal appli-
cation. If the requirement to develop the necessary infrastructure to support
MMUIs is adhered to, then it is likely that the resulting MMUIs will be character-
ized by descriptions like “seamless,” “fully integrated,” and “converged.” If not,
then these terms will only be buzzwords for applications that will ultimately
disappoint users. Consequently, good MMUI design is inextricably linked to
understanding the ability of the underlying systems to aggregate, store, and
provide data to multiple UIs in a consistent, timely, and device-independent
manner.
To help understand these complex relationships among users, contexts of use,
multiple UIs, and underlying infrastructure, Figure 11.1 depicts an example of
how an application that uses multiple UIs might be implemented.
The uniqueness of designing an MMUI is factored in from the beginning with
consideration of the user (although sometimes there are multiple users) and their
different contexts of use and tasks. Because of this, there may be a need to interact
with multiple devices and, consequently, multiple UIs, although some contexts of
use may limit such interaction. These devices interact with various software appli-
cations to provide service functionality. Depending on the implementation, the
same application may support only one device/UI, but it could support more than
one. If there are multiple applications, their ability to communicate with each
other is a factor in the overall level of integration that the user experiences.
Finally, applications network with back-end systems (e.g., databases) to carry
out user requests. Again, some implementations may use multiple back-end sys-
tems, and their ability to synchronize data also will have an impact on the user.
11.3
CURRENT IMPLEMENTATIONS
OF THE INTERFACE
The prevalence of MMUIs is much less than single-mode UIs. Seffah, Forbrig, and
Javahery (2004) suggest that this may be due to the “difficulty of integrating the
overwhelming number of technological, psychological, and sociological factors”
needed to create a good application. But while not as prevalent as single-modality
UIs, the use of more than one interface modality to complete the same task or set
of tasks is more common than one might think. For example, checking on the
showtimes for a movie can easily be done over the phone (using speech or even
11.3 Current Implementations of the Interface
363
touch-tone), on the Web, and using a small-screen device like a cell phone or PDA.
In fact, information retrieval is currently one of the most common uses of MMUIs.
11.3.1 Information Portals
Web portals, such as Yahoo!, are hubs on the Internet that collect and organize
some of the vast array of information sources and streams available to web users
(Figure 11.2). Over the phone, voice portals perform much the same function,
offering users the ability to hear information about news, stocks, weather, sports
scores, movie listings, phone number lookup, and information on other topics
that a user would want to know at any time or place. The latter motivation is
Context-of-use 1 Context-of-use 2
User 1User 1
User interface 1 User interface 2 User interface 3
Device 3Device 2Device 1
Service application 1 Service application 2
Back-end system 2Back-end system 1
Integration
S
y
nchronization
User 2
FIGURE
11.1
Example of an MMUI.
Interrelationship among users, contexts of use, multiple UIs, and the underlying
infrastructure for an example MMUI.
11 Multimode Interfaces: Two or More Interfaces
364
one of the reasons that voice portals have traditionally been associated with wire-
less phone services. However, the lower information-processing bandwidth over
the phone generally lends itself to far fewer topics in voice portals, as compared
to Web portals.
11.3.2 Automated Self-Help
Although some companies, such as Amazon.com, have established their business
with little or no direct human interaction, many traditional companies with major
call centers have been substituting automated self-help for the traditional care
provided by customer service representatives in order to reduce company costs.
The relatively high cost of providing live-agent call center support for a wide array
of products and services, especially in a 24/7 environment, has resulted in the use
of automated systems to provide support for tasks related to billing, payments,
technical support, account inquiries, and sales.
FIGURE
11.2
Traditional information portal for the Web.
The portal organizes disparate sources of information for the user. (Courtesy
Yahoo! Inc.)
11.3 Current Implementations of the Interface
365
Many of the early automated self-help applications were touch-tone interactive
voice response (IVR) systems (see Chapter 7 for more details on designing this kind
of application). The reason for this, simply, was that customers were already calling
and the simplest way to engage them in an automated interaction was to handle
their task over the phone. Nowadays, many of these tasks are also being integrated
into the company’s website. Customers can now call or visit a website to pay a bill,
order products and services, get technical support, and do other routine tasks.
11.3.3 Address Books
Users have address books in myriad locations in the electronic world. They have them
in a GUI environment within their e-mail programs and instant messengers. Some are
in cell phones for voice dialing and text messaging. Others are in online accounts on
the Web for shopping. The diverse tasks that benefit from having an address book
have driven the need to have a single, synchronized address book that can be accessed
from almost any end point: computer, mobile device, telephone, or TV.
11.3.4 Message Mailboxes
Handling messages, such as e-mail, short message service (SMS) text, instant mes-
sages (IMs), voice mails, and faxes, has become a part of everyday life. Messages allow
people to keep in contact even when they are not available at that moment. This need
for ubiquity, combined with very different origins for each type of message, has led to
the use of MMUIs to better handle and, more importantly, expand their usefulness by
giving users more flexibility in how and when they message other people.
11.3.5 Remote Access to Services
In the business setting many network managers have to leave their desks to trou-
bleshoot, but need to take their access to services with them. Others in business
need mobility in order to do their jobs, but need access to the same resources that
they have in their office. And, increasingly, home automation is providing more
capabilities for the home that can be remotely controlled and/or monitored. In
all of these cases, the user has a normal environment in which a robust set of
applications or services is available to them, but also need to move away from a
wired device. This has resulted in a corresponding need to remotely access these
services. The usual medium of interaction, typically a GUI or full-screen website,
is not portable in these cases and so other interfaces, such as phone-based IVRs or
small-screen devices, become a good alternative.
Figure 11.3(a) shows a common task—checking the weather radar—as a user
might see it when using a PC with a broadband connection. Figure 11.3(b) shows
what might be seen when using a mobile wireless device for the same task.
11 Multimode Interfaces: Two or More Interfaces
366
FIGURE
11.3
Checking the weather radar.
(a) Using a PC with a broadband connection. The weather radar from the National
Weather Service on the website has numerous options and a large display.
(b) Using a mobile wireless device. The weather radar from the National Weather
Service for mobile devices has few options and a small display. (Courtesy NOAA/NWS.)
11.3.6 Applications of the Interface to Universal
Design and Accessibility
It is worth taking some time to discuss the i mpact of MMUIs on people with
disabilities. Havin g a disabilit y, whether inherent (e.g., vision or he aring difficul-
ties, dyslexia, etc.) or due to one’s circumstances (e.g ., inabi lit y to see secondary
tasks while driving, to operate small controls while wearing gloves, etc.), can
present barriers to the use of computer technology. However, computer technol-
ogy can be one of the great enablers for people of all abilities. When the design
has considered the full range of user capabilities and limitations, this is called
universal design
.Andthedegreetowhichsomeonecanusethedesignispart
of accessibility.
A major tenet of universal design and accessibility i s flexibility o f user
input/ output (ANSI/HFES, 2005). The distinguishing characteris tic of MM UIs
is that, by their very nature, they offer multiple input/output modalities to the
user.
One good example of MMUIs helping to achieve a more universal design is
the use of touch-tone commands in conjunction with a speech-enabled UI (Fig-
ure 11.4) . Whether offered concurrently as a backup (although this is more of
a combined interface, as discussed in Chapter 12) or as an optional setting, offer-
ing these t wo UI technologies can greatly improve the performance for a wide
range of users.
The use of speech recognition helps those with a cognitive disability who
would have trouble remembering the mapping of touch-tone options played in a
menu, while another user with limited use of the hands (either because they have
<says nothing>
Caller
Okay, which are you calling about: your bill, an order, or technical
support?
System
Caller
<presses the 3 key>
System
I’m sorry, I didn’t hear. If you’re calling about your bill, say “my
bill” or press one. If it’s about an order, say “order” or press two.
Otherwise, say “technical support” or press three.
FIGURE
11.4
MMUI example.
This is a brief caller exchange with an automated system that demonstrates the
use of both speech and touch-tone input options.
11 Multimode Interfaces: Two or More Interfaces
368
a disability or are wearing gloves or holding something) can use speech to provide
the input. In contrast, offering touch-tone input as an option helps a user with a
speech impairment or a difficult accent, and provides other users who are in a
noisy environment or have a need for privacy, the ability to respond to the appli-
cation. Offering both modes of interaction enables as many users as possible to
interact with the application in the manner that works best for them.
While MMUIs should not be thought of as a means to circumvent making each
UI accessible, the inclusion of multiple modalities for interaction greatly increases
the flexibility for users to interact in ways that match their particular capabilities
and limitations.
11.4
HUMAN FACTORS DESIGN OF THE INTERFACE
Beginning to design an application that uses more than one UI modality is based
on the assumption that more than one is needed. Often this is based on a belief
that because it
can
be done it
should
be done. As with the choice of an interface
modality and technology for any application, the expected users and their needs,
tasks, and contexts of use first need to be identified and evaluated.
11.4.1 Research Requirements before Choosing
the Interface Modality
The user-centered design process begins with users, not technology. This is the
core belief of user-centered design, but it is mentioned here because when dealing
with the myriad UI technologies and the complex technological infrastructure
needed for MMUIs, it is even more likely that this point will be lost.
The first step in the design of MMUIs is the same as designing any other UI:
Define the user population. This process defines the characteristics of the poten-
tial users, including their capabilities and limitations and the environments in
which the application is expected to be used. This process includes considering
people with disabilities. Once the target users have been defined, then user
research can begin.
While doing the usual needs and contextual analysis, be aware that some
user needs may be specific to different environments in which the application
will be used—for example, checking e-mail f rom the office while at one’s desk
and checking e-mail on a crowded subway to and from the office. Some needs
may be based on the goals that users might want to accomplish using the pro-
posed application.
11.4 Human Factors Design of the Interface
369
Once the needs are well defined and understood, application features can
be added, removed, or refined. Matching these features with user needs cre -
ates a s et of common goals for the user and the application. Next, the usual
task analysis should be performed to determine how users accomplish these
goals and to look for opportunities to improve existing methods. The task anal-
ysis should also be used to determine which goals can or cannot be accom-
plished using specific interface modalities. For e xample, setting up a group
of address book contacts having a common pr operty, such as your daugh ter’s
soccer team, may be relatively easy to do with a visual interface, keyboard,
and mouse, but very difficult or impossible to do using a telephone keypad.
However, it may be possible to edit a team member’s e-mail address using
speech recognition.
There is some research into the area of development considerations particular
to MMUIs. For example, Kray, Wasinger, and Kortuem (2004) identify a taxonomy
of management (e.g., conflict resolution among users and/or tasks), technical
(e.g., synchronization), and social issues (e.g., privacy), but note that “they are
often hard to classify as due to overlapping categories.” Seffah, Forbrig, and
Javahery (2004) discuss the use of various models for development, including
task model–based development, UI pattern–driven development, and device-
independent development languages. They found that pattern models do help
with understanding the full complexity of tasks and the interrelationships parti-
cular to MMUIs. To assist design, Paterno (2004) describes an authoring environ-
ment for MMUIs. Some of its key components are grouping, relation, ordering,
and hierarchy.
While these techniques are not peculiar to MMUIs, the overriding theme is
that the design and development of applications using MMUIs are complex and
require a planned, methodical approach. To prevent becoming overwhelmed
when designing and developing an MMUI, the technique used is less important
than whether it is thorough.
Critically evaluating these points will provide a basis for an informed decision
about what modality and associated technology can best enable the users to
perform their tasks in the expected situations.
11.4.2 Choosing One or More User Interface
Modalities
Once background research and analysis have been conducted, then it is appropri-
ate to select of one or more UI modalities. The questions in Table 11.1 help guide
the decision of whether multiple UIs are needed, although they do not make up an
exhaustive list.
If the conclusion is that multiple modalities are warranted, then choosing
which modalities to use is the next step.
11 Multimode Interfaces: Two or More Interfaces
370
TABLE 11.1 Questions to Aid in Choosing an MMUI
Evaluation Question Explanation Example
Can more than one modality be
cost-justified?
Designing and developing more
than one user interface is
costly. Can the added expense
be balanced with a greater
customer base, greater use of
the service, increased
customer retention, and so
forth? (See Bias and Mayhew
[2005].)
Some customers want to pay
their bill over the Web, but
others call to pay their bill and
could use an IVR. Offering
both will automate more bill
payments overall, saving
money and increasing revenue
collection.
Can all of the tasks be
performed in only one
modality?
Based on the task analysis, can
all of the tasks be adequately
performed using only one
modality? If so, there may not
be a compelling reason to use
multiple modalities.
All voice mail tasks can be done
over the phone using touch-
tones.
Would more than one interface
modality directly fulfill user
needs?
Sometimes one user interface
modality cannot support all of
the needs and tasks of the user
population.
Sending a long e-mail in festive
colors to an entire list of party
guests with numerous details
and a map may call for a
website or GUI. Alternatively,
sending a quick e-mail to a
friend letting them know you
are stuck in traffic and will be
15 minutes late may call for
something mobile and simple,
like a cell phone or PDA UI.
Do the contexts of use and/or
environments necessitate
different modalities?
When users perform the same
task (or set of tasks) in
different situations they may
not have the same needs, so
the same modality may not
work, depending on where
they are and what they are
doing.
A tool to help customers resolve
problems with their Internet
service offers both web-based
help for those who can still
access the Internet and over-
the-phone help for those
whose Internet connections
are totally down.
Will the new application’s
functionality be composed of
myriad tasks currently
performed by different
modalities?
Are the user needs and tasks
currently handled by different
applications using different
modalities that users will not
easily give up?
An application that provides
movie listings uses a web user
interface and a telephone user
interface because some users
have always used one of these
and do not want to be forced to
change.
(
Continued
)
11.4 Human Factors Design of the Interface
371