Kortum P. (ed.) HCI Beyond the GUI. Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces

Подождите немного. Документ загружается.

6.2.2 Some Historical Notes on Speech Recognition

Historically, speech recognition systems were constrained by the lack of proces-

sing speed and capacity. Speech recognition is computationally intensive, and as

recently as the 1990s, the way speech systems functioned was limited by lack of

processing power. (See Baber and Noyes [1993] and Lea [1980] for illuminating dis-

cussions of the future of speech technology written in the recent past.) One way

of dealing with processing limitations was to restrict recognition to the utterances

of a single individual. These

speaker-dependent

systems are built to recognize the

speech of an individual, and thus require training data (in the form of spoken

utterances) from this individual in order to function.

In contrast,

speaker-independent

systems do not require the user to provide

training data before being recognized. Instead, speaker-independent recognition

proceeds from a set of training data from a large number of individuals who

may or may not be users of the system. The advantage of speaker-dependent sys-

tems was that they could support larger vocabularies than speaker-independent

ones, given a fixed amount of processing power. If processing power is limited,

there is a direct trade-off between the number of words in a system’s vocabulary

and the number of different voices that can be recognized. Today, this distinction

has all but disappeared, and the vast majority of commercially deployed speech

systems are speaker independent and recognize a limited number of items

(although this number may be in the thousands) for any recognition state.

One important counterexample of speaker dependence in use currently is

speaker verification systems (Campbell, 1997; Furui, 1996). These systems are

trained to identify users by the unique characteristics of their voices. The user’s

voice therefore becomes a method of authentication that can be used alongside

less reliable authenticators like passwords or user IDs. For speaker verification,

users are enrolled (typically over the phone) by speaking a particular utterance

multiple times. Representations of these utterances (sometimes called

voice

prints

) are stored by the organization deploying the verification system and are

used as one method of authenticating the user when she next calls in to the

system. Speaker verification has been shown to be robust and difficult to fool,

and is advantageous because users no longer need to remember passwords.

Instead, by simply speaking a piece of identifying information such as an account

number, the system is able to both recognize the data being spoken and recognize

the voice itself. There are many possible combinations of data and voice prints

possible to provide different levels of security for speech applications.

Another historical distinction is between systems that recognize isolated

words and those that recognize connected (or continuous) speech. Previously, rec-

ognition was much more accurate for isolated words because isolated words elim-

inate the need to segment the incoming speech signal into words. It can be

notoriously difficult to accurately locate word boundaries in connected speech,

and eliminating this distinction significantly improved recognition accuracy. The

6 Voice User Interfaces

212

majority of speech systems in use today still rely on isolated word recognition

(although the systems can now recognize longer phrases, not just words). Statisti-

cal language models, which are trained on continuous speech, offer the nearest

approximation to continuous speech recognition in commercial use today. For

an excellent history of speech technologies, see appendix A in Weinshenk and

Barker (2000).

6.3

CURRENT IMPLEMENTATIONS OF

THE INTERFACE: ON THE PHONE

Voice user interfaces are overwhelmingly encountered over the phone. Speech

recognition technology has been adopted in the call center as the successor to

touch-tone IVR systems. Speech has a number of inherent advantages over

touch-tone. Allowing users to speak their responses rather than key them in on

the telephone keyboard makes the interaction easier in eyes-busy, hands-busy

situations. The prevalence of cordless and mobile phones also gives an advantage

to speech input over touch-tone. Speech does not require that users pull the hand-

set away from their ear, press a key, and then return the phone to the ear

repeatedly.

Speech also enables functionality that is awkward or impossible using touch-

tone. Entering names, cities, fund names, or any string that contains both num-

bers and digits is very difficult in touch-tone, which requires users to either spell

words on the telephone keypad or choose from very long lists of items. Speech

recognition simplifies tasks such as collecting name and address, transferring

funds between accounts, and making travel reservations. Speech also allows for

shallower menu hierarchies, as the interaction is no longer constrained by the lim-

its of the 12-digit telephone keypad. VUIs have grown and matured in the tele-

phony world, and much of VUI design is totally phone-centric. Salespeople also

pitch speech systems as more “natural” than touch-tone, since we all know how

to talk. (I sincerely hope that this chapter will convince the reader of the fallacy

of this sales pitch.)

Voice user interfaces can be found outside the call center, although in much

smaller numbers. One application that is rapidly becoming more widespread is

speech-enabled in-vehicle navigation systems. These systems allow the driver to

control the navigation system and various features of the vehicle (sound system,

air conditioning) through voice commands. These vehicle-based systems are mul-

timodal in that there is a screen available throughout the interaction. This makes

the speech part of the user interface somewhat less rich than what is seen in tele-

phony applications.

Speech recognition on the personal computer desktop actually preceded wide-

spread use of speech in the call center, but desktop speech recognition use has yet

6.3 Current Implementations of the Interface: On the Phone

213

to become the norm for the vast majority of users. Dictation programs have been

available for decades and some users are committed to them. These programs are

built to recognize free-form input from a single user rather than a small grammar

of words spoken by any user. Because of this, dictation programs must be trained

to the voice of the user, which entails an initial time investment before the program

is usable. Dictation programs can work quite well, but suffer from a poor reputation

of being time-consuming and inaccurate. Less commonly used (though now widely

available through Windows Vista) are desktop command and control programs.

With these programs, users can speak commands to perform the same tasks they

usually accomplish with keyboard and mouse. Again, the voice component of the

interface here is rather minimal—the user has a set of commands that can be spo-

ken. There is no “conversation” as there is in telephone systems.

Speech is also found is a number of tiny niche markets. There are dedicated

speech recognition systems in fields such as medical dictation in which there is a

complex but limited specialized vocabulary. Speech systems are also used by workers

in warehouses to assist with inventory and order fulfillment tasks. Although these

niche applications are on the fringe of VUI today, they are likely to proliferate in com-

ing years. Niche markets are conducive to speech technology because a niche is a

limited domain that constrains the vocabulary (thus enabling smaller grammars

and better recognition performance), and because niche users tend to be more will-

ing to live with limitations that the general population might not tolerate. Niche users

may be willing to wear headsets or other devices to enable speech recognition, and

may be amenable to having to learn how to use the system (as opposed to the general

population, who tend to demand immediate ease of use for speech technologies.)

6.4

HUMAN FACTORS DESIGN OF THE INTERFACE

Voice user interfaces often suffer from a lack of human factors involvement dur-

ing design and development. The reasons for this are logistical and technical:

Speech systems are most often deployed in a telephony environment, typically

in the call center, and VUIs rely on a different technology than GUIs and thus

require different developer skill sets. Speech system developers are often not

accustomed to working with human factors professionals and may have difficulty

adjusting to new procedures and timelines imposed by user-centered design prac-

tices. In large organizations, there may be an existing staff of human factors pro-

fessionals working on website and software issues. However, the call center is not

the typical realm for human factors specialists, and IVR managers may be

unaware of the human factors skills available to them. Consequently, VUI design

often proceeds without any human factors involvement at all. Sadly, even when

human factors folks are involved in speech projects, most lack expertise in speech

technologies and human conversational behavior and thus do not contribute as

much as they do in GUI projects.

6 Voice User Interfaces

214

In the vast majority of cases, the sole representative of human factors on

speech system projects is the VUI designer (VUID). Much like the early human

factors pioneers, VUIDs are a heterogeneous group made up of ex-developers, way-

ward linguists, telephony managers, assorted psychologists, and many others with

little or no relevant formal training in speech technology or human factors. There

are currently no formal degree programs specifically in VUI design; however,

Auburn University has an engineering program with a strong VUI emphasis under

the direction of Juan Gilbert. What VUIDs may lack in formal human factors train-

ing, most make up for in their enthusiasm to bring strong user-centered design

practices to speech projects. To get a sense of the VUID community, consider

visiting the VUIDs Yahoo group at

http://tech.groups.yahoo.com/group/vuids

Strong support for human factors in speech technology systems has come from

AVIOS, the Applied Voice Input Output Society. AVIOS is a nonprofit membership

organization for speech technology professionals that has been influential for over

20 years. AVIOS often sponsors speech technology conferences and previously had

editorial control of the

International Journal of Speech Technology

, an important

source of rigorous technical and user interface information for speech systems.

More so than in the GUI world, VUIDs need a mix of both technical and

design skills. GUI interactions occur using a mouse, keyboard, or some other pre-

cise input device, and GUI designers can reasonably assume that user input will

be interpreted correctly by the system. In speech-enabled applications, well-

formed user input will be misinterpreted in a significant proportion of cases. Most

current speech recognition algorithms boast near 100 percent recognition accu-

racy, but in practice, error rates of 10 to 20 percent or more at a given recognition

state are not uncommon. Therefore, the VUI designer must understand the limita-

tions of the technology and design using strategies that minimize technology-level

errors. VUIDs also serve as the voice of the user in speech projects, much as other

human factors professionals do in the GUI world. On many projects, the VUI

designer is working alone without any other human factors support, so it is vital

to have firm guidelines for appropriate user-centered design processes, as well

as guerrilla techniques to fall back on when the ideal is not possible. The ability

to justify and evangelize for user-centered design (UCD) processes is vital for

VUIDs who are often the first exposure project members have to human factors.

6.4.1 User-Centered Design for Speech Projects

User-centered design practice for speech technologies is similar but not identical

to established UCD techniques. At a general level, the process is similar: Gather

information from users and the business, design a preliminary interface, test with

real users, modify the design, and iterate. There are different twists at each stage,

however, because of the unique position of speech technologies.

During requirements gathering, VUI designers have unique sources of infor-

mation relevant to the voice channel. Since most speech technology is deployed

6.4 Human Factors Design of the Interface

215

over the phone in call centers, call center agents and managers can provide valu-

able background on the user’s state of mind when calling, terminology used and

understood by callers, and nuances of calling patterns that may not be obvious

from other sources (e.g., users who call during business hours tend to be more

impatient than users who call later in the day). In call centers with existing IVRs,

usage statistics can reveal frequency of use for various functions, patterns of use

over time, and points at which users tend to abandon self-service in favor of live

agents. Existing IVRs also offer the opportunity to listen to real calls so that

designers may understand the current user experience and thus gain insight on

how to improve it.

Another data point in the requirements process is the details of the particular

speech recognition system being used on a project. While all recognition engines

have the same basic capabilities, there are a multitude of differences in systems

sold by different vendors that can affect the VUI. Understanding what speech

technology is available and how to put it to best use is a foundation for designing

an effective VUI.

The process of VUI design is similar to GUI design except, of course, for the

modality of the presentation layer of the interface. In spite of being auditory, VUIs

need to be documented visually. Most VUI designers rely on a combination of

flowcharts to represent the logical flow of the application (in a program such as

Visio) and text documents to specify the prompts and detailed behavior at each

recognition state. This combined call-flow–plus–text-document arrangement is

suboptimal and difficult to maintain, but most VUI designers have no better

alternative. There is no tool in wide use that captures both call flow and detailed

specification information in a single program.

6.4.2 How Much Speech Should We Use?

One of the most important questions for VUI designers is how to use speech tech-

nology smartly to achieve business goals and meet user needs. The aim is to avoid

using speech technology for its own sake, and instead look for opportunities in

which speech provides distinct value to the user and to the business. These oppor-

tunities arise under several conditions:

F The current technology underaddresses user or business needs, such as typing

city names using the numerical telephone keypad.

F The typical context of use is an eyes-busy, hands-busy situation, such as any

application where users are likely to be mobile.

F Users have a disability that limits their ability to use their hands. (See the

United States Access Board, Section 508 , and the Web Accessibility Initiative of

the W3C for further recommendations for speech and accessibility.)

F The wait time to speak to a live representative is unacceptable to the users for

their task.

6 Voice User Interfaces

216

Some factors weigh against using speech. Many users have concerns about pri-

vacy and may not be willing to speak certain information aloud (e.g., passwords,

Social Security numbers, account numbers, medical data, other financial informa-

tion). If you choose to speech-enable collecting this type of sensitive information,

be sure to allow users an alternate method to enter their information. Speech is also

a poor fit for collecting unconstrained alphanumeric sequences because of the diffi-

culty of recognizing sequences of variable lengths composed of letters and numbers

that are notoriously confusable. Various noisy acoustic environments also make

speech a less desirable solution. In many other situations, speech works reasonably

well for the task at hand, but may not offer any real benefits over another technology

(like touch-tone input). In these scenarios, the case for using speech is more about

the ability of speech interfaces to generate positive customer experience for users.

When it is clear that speech recognition technology is a good fit for the task,

there is still the choice of the type of dialog and grammar to use. For telephone-based

speech systems, a directed dialog approach is often appropriate. Directed dialog refers

to the class of VUIs in which the system offers users the choice among a finite set of

options, and those options are recognized against a standard grammar defined in

advance. Directed dialog can provide effective and satisfying interactions for wide

variety of routine information retrieval tasks (e.g., getting an account balance, check-

ing flight status) and for many transactions such as placing an order or changing an

address. In the speech industry press, directed dialog is sometimes portrayed as

unnatural or inferior to so-called “natural-language” systems.

Natural language is an imprecise term that is used differently by each author.

A common interpretation for “natural language” is the sort of minimally con-

strained responses possible when using a statistical language model. As explained

above, SLMs are simply a more expansive grammar, not true language under-

standing, and can be costly to implement. And because most users are unaccus-

tomed to speaking naturally to automated systems, they do not know how to

best form their responses, and are thus uncomfortable with these more “natural”

interfaces. To the contrary, my experience shows that well-designed directed

dialog interfaces are often the most successful and comfortable for users.

6.5

TECHNIQUES FOR TESTING THE INTERFACE

Collecting data from end users of speech-enabled applications in usability tests is a

vital step in any speech project, as it is for any automated system. Usability testing

for speech-enabled applications should begin at the earliest stages of a project in

order to be maximally effective. Early user input allows the designer to make

changes when it is still relatively quick and inexpensive to make them. Moreover,

subjective data from end users is an important factor in interpreting objective

measures of the technical performance of a speech application. In isolation, mea-

sures such as recognition rate or call containment do not give organizations

6.5 Techniques for Testing the Interfa ce

217

a meaningful way of measuring the overall success of a speech application. Only

the combination of objective performance measures (such as recognition rate)

coupled with subjective measures of performance (such as perceived frequency

of being misunderstood by the system) allows us to know if a speech-enabled

application is good enough (Hura, 2007a).

As in any usability project, there is a trade-off between how early you test and

how representative the collected data are. There are direct correlates in speech to

many of the testing techniques that are used in GUI projects. For formal VUI usability

tests, there is a choice between using actual speech recognition during testing and

mimicking it. This is a significant decision because fully functional speech prototypes

are typically coded by developers (rather than usability specialists) and thus usually

occur later in the project life cycle. The alternative is a so-called Wizard of Oz test in

which no application has been developed at all. Instead, a usability specialist (the “wiz-

ard”) supplies speech recognition by listening to user input and then playing sound

files of prompts to mimic the behavior of the speech application (Hura, Polkosky, &

Gilbert, 2006). There are benefits and downfalls to each type of testing.

Wizard of Oz (WOZ) testing is often discussed as being a “discount” usability

method equivalent to prototype testing, but no coding is required. Although there

is no coding, the benefit is exaggerated in relation to the overall amount of work to

run a WOZ test which is equivalent to a prototype test. WOZ testing requires a

complete VUI design, professionally recorded prompts, a system for playing

prompts over the phone, and the full-time involvement of two human factors pro-

fessionals for every test session (one to act as the wizard and one to facilitate test

sessions—both demanding jobs).

It is certainly possible to run WOZ tests that compromise on one or more of these

factors, but the test is then not a true substitute for prototype testing. For example, if

you choose to have the wizard simply speak the prompts rather than using recorded

prompts, there are multiple disadvantages: First, you lose the ability to gather feed-

back about the voice talent; second, the prompt will be spoken differently every time

it is uttered, which introduces unwelcome variability into the testing process.

Because so much of the user experience is bound up in the minute details of the

prompts, this compromise is hardly worthwhile in many cases. In spite of the inten-

sive preparation required by the usability team, WOZ tests are typically conducted

earlier in the project than tests using functional speech recognition prototypes.

Testing with a functioning speech prototype is the gold standard for assessing

VUI usability (Hura, 2003). Prototypes give a realistic view of the flow of the auto-

mated conversation, and the data that can be gathered in a prototype usability test

are an order of magnitude more realistic than the data from a WOZ test. The

timing of the interaction is also more realistic in prototype testing. It is notori-

ously difficult in a WOZ test to accurately simulate the response latency of a

speech application, and wizards typically respond more quickly than the system

in some situations and more slowly in others.

One later-stage technique that is underused in VUI testing is field tests, in

which users interact with a VUI from their typical usage environment. Field tests

6 Voice User Interfaces

218

are logistically complicated because telephone-based systems can be used any-

where and while in motion (as opposed to computers, which are stationary when

used). Additionally, because many commercial speech applications are used

rarely by a given individual, it can be difficult to arrange to be present at the time

of use. Instead, most VUI projects rely on live call monitoring, or call recording, in

which usability specialists can listen to users as they have real-life interactions

with a VUI. These techniques do not allow the designer to give the user a set of

tasks to complete (hence we can never be totally sure of a user’s motivation in

the interaction), and there is generally no ability to speak to users directly to

understand reactions to their experience following an interaction (although

surveys are sometimes administered at this time).

A final consideration for VUI usability testing is logistics: VUI usability testing

can take place with the user and facilitator in the same location or remotely over

the phone. The technology is readily available to allow users to interact with a

speech application while usability specialists (and others) listen in, all over a stan-

dard telephone connection. The benefits of over-the-phone testing are that it saves

the considerable expense of securing a testing facility, and the time and cost asso-

ciated with having participants travel to this facility. The downfalls all relate to the

fact that we can only listen to the interaction, not watch it, which deprives us of

cues that users give through their facial expressions, postures, and body language.

In-person usability testing for VUIs allows us to watch and listen to interac-

tions, and therefore record both audio and video of test sessions. Building rapport

with a user is easier when you are physically in the same room with one another,

which encourages users to give uncensored responses during interviews. The

downfalls of in-person testing are the time and cost associated with bringing a

group of participants to one location for testing. In-person testing also limits test

participants to those who are willing to travel to the test location, as opposed to

over-the-phone testing where remote users can be included easily.

One final VUI testing method is tuning. Tuning is the process of optimizing

performance of a speech application overall. The focus in tuning is very broad:

Everything from low-level recognition parameters to prompts is evaluated. Tuning

involves recording live user interactions with a speech system—several hundred

interactions are a typical analysis set for a round of tuning. Recorded calls are tran-

scribed and transcriptions are used as a basis for determining recognition accuracy

in the application overall and state by state. Utterances are categorized as in gram-

mar or out of grammar and as correctly recognized, incorrectly recognized (or false

accept in which an out-of-grammar utterance is mistakenly recognized as an

in-grammar utterance that fails to be recognized), or correctly rejected. Along with

computing statistics, tuning also involves listening to as many user calls as possible.

Tuning calls are sometimes the only opportunity that VUI designers have to

observe their design in action and determine whether conclusions from usability test-

ing hold in the real world. Based on the computed statistics and insights gained from

monitoring calls, designers can identify issues and propose solutions. Tuning offers

the opportunity to test some types of solutions before they are put into practice. It is

6.5 Techniques for Testing the Interfa ce

219

often possible to modify a grammar, run the recorded speech samples through, and

observe the results. If tuning data are interpreted narrowly (applying results only to

specific dialog states rather than overall across the application) and used judi-

ciously—without making too many assumptions about what users are thinking—it

is possible to come to some conclusions about usability from tuning data.

There are few established metrics of VUI usability available or in wide use.

Gupta and Gilbert (2005) propose the holistic usability metric, which allows the

tester to apply weights to factors judged to be most important for individual inter-

faces. Because each tester may assign weights differently, this metric can be tricky

to interpret universally, but can be quite valuable for testing one application over

time or a set of similar applications using the same version of the metric. Polkosky

(2005) has constructed an opinion metric specific to VUIs for collecting subjective

user data. Her research shows that users perceive the behavior of a VUI as evi-

dence of higher-order factors, such as the overall level of customer service orien-

tation within an organization. This metric is valuable in that it addresses issues

beyond pure usability, such as desirability and customer satisfaction, and it has

wide implications for exactly which factors in a VUI influence user opinions.

Neither of these VUI metrics has been put to widescale use in the industry.

Instead, organizations tend to rely on home-grown surveys or customer satisfac-

tion surveys designed for Web or other media, with a few minor adjustments for

speech, to evaluate the quality and usability of speech-enabled systems. The items

on such surveys are rarely vetted for validity or reliability, and as such the data

gathered from them are less than ideal. One notable exception to this pattern is

the services offered by Vocal Laboratories, which offers professionally designed

surveys and the ability to quickly collect data from very large populations of users.

6.6

DESIGN GUIDELINES

There are a number of references that provide an overall philosophy of VUI design,

including Cohen, Giangola, and Balogh (2004), Ballentine (2001), Kotelly (2003), and

Larson et al. (2005). The guidelines presented here are intended not as all-inclusive,

but rather as a collection of practical tips for writing effective prompts. The data sup-

porting these tips derive from observations of hundreds of users interacting with

dozens of speech-enabled IVR applications during usability tests.

6.6.1 Do Not Worry So Much about the Number

of Menu Options

George Miller’s 50-year-old research on short-term memory (1956) is often portrayed

as the final word on the permissible number of auditorily presented menu options.

The thinking here is that it is primarily the limits of auditory short-term memory that

6 Voice User Interfaces

220

determine the usability of a voice menu. There is a kernel of truth here—it is cer-

tainly possible to overload users with too much auditory information—but the strict

limit of seven-plus-or-minus-two simply does not hold for VUI menus (Hura,

2007b). The flaw in applying Miller verbatim is that using a menu is not a recall task

but rather a selection task. Users should never need to hold an entire VUI menu in

memory in order to make their selection. The goal is to have them evaluate and accept

menu options one by one or, at most, remember a few likely candidates.

6.6.2 Present Menu Options in a Way That Makes

Them Comprehensible and Easy to Retain

Especially in menus with more than a small handful of options, the best way to enable

successful selection is to ensure that the overall set of menu options is

descriptive

and

distinct

Descriptive

menu options give users the immediate sense of “I know what

that is!” This enables users to evaluate and discard nondesired options without holding

them in memory. Similarly, users will immediately know the differences between

distinct

menu options, which eliminates the need to remember the entire list while

making their menu selection. Current short-term memory theorists agree that there

are many factors beyond the number of items that affect auditory short-term memory,

such as word length, frequency, familiarity, and inter-item discriminability (i.e., how

distinct items on a list are from one another [Nairne, 2002]). The importance of

descriptiveness is supported by recent research in human–computer interaction

showing that meaningful menu option labeling is more important than information

architecture for users (Resnick & Sanchez, 2004).

6.6.3 Do Not Let Persona Get in the Way

of Effectiveness

Users will always attribute a persona to a VUI, but persona should always be subor-

dinate to effectiveness. Some designers argue that a VUI should never say anything

that a person would not say, but following this rule will often lead the VUI designer

to write prompts that encourage the user to overestimate the abilities of the speech

system. Instead, focus on crafting prompts that lead users to give appropriate

responses. It is far better to be somewhat unnatural than to take a user into an error

state for the sake of a nice-sounding prompt (Klie, 2007; Rolandi, 2007).

6.6.4 Consider Error Sources When Writing

Error-Handling Prompts

Users find themselves in error conditions for a variety of reasons, many of

them unrelated to the VUI itself. There are distractions in the environment

6.6 Design Guidelines

221