Kortum P. (ed.) HCI Beyond the GUI. Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces

Подождите немного. Документ загружается.

media sharing, and increased software control found under the surface. The

number of information appliances targeted toward all areas of small-screen

technology will continue to blossom over time.

The trend to larger, higher-resolution displays will also continue. Interfaces

that use vector graphics to scale to any resolution will become more common.

Flexible OLED displays will mean that big screens will become portable and use

less battery power than LCDs while displaying richer data and media. Figure 10.21

shows two concept devices using rollable displays. In addition, eyewear enabling

users to privately view a big-screen-like display has already become available. In

2005, Orange SA launched video eyewear with a Samsung phone enabling users

to view TV, movies, photos, and broadband Internet content with a big-screen

viewing effect (DigitalCamera@101review, 2005). The small-screen device will

continue to catch up to the PC in terms of CPU speed and functionality, and

mobile devices will run an increasingly complex variety of software.

In the medium-term future, displaying information on a traditional screen

will be eclipsed by new display technologies. The problem with current larger-

but-portable screen technology is that no matter how small individual components

can be made, the screen will always limit the reduction in size of the form factor.

People want small devices that are portable. Making the display external to the

device removes this restriction. Three interesting technologies are projection

displays, heads-up displays, and open-air holographic-like displays. Projection

technology means that the screen itself can be as large as the user requires, but

the device itself is still very small and thus highly portable. Heads-up displays

use an external device to display important information to the user. The open-

air holographic-like display projects an image in the air just above it. One example

of this is the heliodisplay developed by IO2 Technology that is capable of project-

ing an interactive two-dimensional image (Genuth, 2006). All of these display

technologies will see further refinement in coming years.

FIGURE

10.21

Concept devices using rollable displays.

The concept at right shows device open and closed.

Source:

From De Rossi (2005);

courtesy Polymer Vision.

10 Small-Screen Interfaces

352

As demonstrated in Section 10.2, the paramount challenge in small-screen device

design is not output but user input. The primary methods of input on a personal com-

puter (the mouse and keyboard) do not translate well into the realm of mobile devices.

In the next 10 years, three areas of research will dominate small-device input: spatial

interfaces, eye-tracking interfaces, and natural-language comprehension.

A spatial interface is an input technology that is aware of device movement.

As the device (or a dedicated input device) is moved, the interface responds

appropriately. Spatial interfaces are easy to use and well understood even by nov-

ice users. Typically, a spatial interface works in three dimensions, allowing the

input device to be moved anywhere in space. The sudden-motion sensor built in

to Apple laptops (originally designed to prevent damage to the hard disk if

dropped) has already been co-opted as a spatial input. Enterprising engineers have

already built software for the motion sensor to scroll windows and play games

(Singh, 2006). Dedicated spatial-interface devices like the Wii Remote included

with the Nintendo Wii provide a completely new way to interact with games by

including a gyroscope to track hand movement. Motion-sensing technology is a

practical input methodology for small devices because the hardware required

can be made very small and the interaction is natural for users.

Another technology with considerable potential requires even less user effort

than a spatial interface. Eye tracking (sometimes called a gaze interface) monitors

the movement of the eye across a visual display. The tracked movement of the

eye is translated into input within the interface. Eye tracking has been tradition-

ally used in perception and advertising studies but shows promise for visual

search tasks such as exploring images and for software navigation. The downside

of eye-tracking interfaces is that they require some training to use effectively.

However, eye-tracking interfaces can be made small enough for small-screen

devices and are also appealing from an accessibility standpoint.

A technology with great potential, both from an ease-of-use and an accessibil-

ity perspective, is natural-language recognition and comprehension. As discussed

in Section 10.2, some mobile phones already incorporate rudimentary language

recognition with limited success. As linguists better understand how to teach com-

puters semantic and grammatical understanding, language interfaces will improve

to the point that they are a powerful input modality (Yates, Etzioni, & Weld, 2003).

Already technologies such as IBM’s Via Voice for the PC are approaching func-

tional everyday use, and it is only a matter of time before the technology is found

in smaller devices.

Even further in the future, the quintessential human–computer interface will

communicate directly with the brain. While virtual reality pumped directly into

the brain (like the interface depicted in the movie

The Matrix

) may be many years

away, elementary brain input interfaces are already being studied in labs such as

the University of British Columbia’s Brain–Computer Interface lab (Birch, 2007).

Clearly a direct-to-brain interface involves almost no effort to control input or

output data, and is accessible to anyone with or without physical disabilities.

The brain interface truly represents the panacea of computing interfaces.

10.8 Future Trends

353

REFERENCES

37signals. (2006). Getting Real; available at

http://gettingreal.37signals.com/toc.php

Alibaba.com. (2007). Car Parking Sensor System with LED Display; available at

http://

cyberfair.trustpass.alibaba.com/product/10885882/Car_Parking_Sensor_System_With_

LED_Display.html

Allaboutsymbian.com. (2007). Nokia e61 Contacts Application; available at

http://allabout-

symbian.com/images/devices/e61/nokaie61_review2_07.JPG

American National Standards Institute (ANSI)/Human Factors and Ergonomics Society

(HFES) 100-1988. (1988).

American National Standard for Human Factors Engineering

of Visual Display Terminal Workstations

. Santa Monica: Human Factors and Ergonomics

Society.

Apple. (2007). Sound and Hearing; available at

http://www.apple.com/sound/

AppleInsider.com. (2006). iPod: How big can it get?

AppleInsider.com

, May—

http://www.

appleinsider.com/article.php?id=1770

Av.it.pt. (2007). Tektronix TDS 3052B; available at

http://www.av.it.pt/wcas/images/

PIC0014.JPG

Baecker, R., Small, I., & Mander, R. (1991). Bringing icons to life.

Proceedings of the SIGCHI

Conference on Human Factors in Computing Systems: Reaching through Technology

New York: ACM Press, 1–6.

Baudish, P., Xie, X., Wang, C., & Ma, W. (2004). Collapse-to-zoom: Viewing Web pages on

small screen devices by interactively removing irrelevant content.

Proceedings of Sev-

enth Annual ACM Symposium on User Interface Software and Technology

. New York:

ACM Press, 91–94.

Bernard, M. L. (2002a). Examining a metric for predicting the accessibility of information

within hypertext structures. PhD diss. Wichita State University.

Bernard, M. L. (2002b). Examining the effects of hypertext shape on user performance.

Usabil-

ity News

4.2—

http://psychology.wichita.edu/surl/usabilitynews/42/hypertext.htm

Blackwell, H. R. (1946). Contrast thresholds of the human eye.

Journal of the Optical Soci-

ety of America

36:624.

Birch, G. (2000). The Brain–Computer Interface Project; available at

http://www.ece.ubc.

ca/~garyb/BCI.htm

Bohan, M. (2000). Entering text into hand-held devices: Comparing two soft keyboards.

Usability News

2.1; available at

http://psychology.wichita.edu/surl/usabilitynews/2W/

softkeyboards.htm

Board of Standards Review (BSR)/HFES 100. (2006).

Human Factors Engineering of Com-

puter Workstations: Second Canvass Review, 2006

. Santa Monica, CA: Human Factors

and Ergonomics Society.

Buyukkokten, O., Garcia-Molina, H., & Paepcke, A. (2001).

Seeing the Whole in Parts: Text

Summarization for Web Browsing on Handheld Devices

. Digital Library Project (InfoLab),

Stanford University. Available at

http://www10.org/cdrom/papers/594/

Cambridgeincolour.com. (2007). Basics of Digital Camera Pixels; available at

http://www.

cambridgeincolour.com/tutorials/digital-camera-pixel.htm

Chapanis, A. (1949). How we see: A summary of basic principles. In

Human Factors in

Undersea Warfare

. Washington, DC: National Research Council, 12.

10 Small-Screen Interfaces

354

Cooke, N. J. (1994). Varieties of knowledge elicitation techniques.

International Journal of

Human–Computer Studies

41:801–49.

De Rossi, L. C. (2005). Portable rollable displays. Foldable screens that fit in your pocket;

available at

http://www.masternewmedia.org/video_display/portable_screens/rollable_

portable_screens_prototypes_first_looks_20050712.htm

DigitalCamera@101reviews. (2005). Big screen viewing effect for mobile phone videos;

available at

http://digitalcamera.101reviews.com/news/big-screen-viewing-effect-for-

mobile-phone-videos

Dix, D. A. (1995). Acceleration and toolbars: Learning from the menu.

Proceedings of

People and Computers IX (Adjunct): British Computer Society Conference on Human

Computer Interaction

. London: British Computer Society, 138–43.

Duchnicky, R. L., & Kolers, P. A. (1983). Readability of text scrolled on visual display term-

inals as a function of window size.

Human Factors

25(6):683–92.

Dumas, J. S., & Redish, J. C. (1994).

A Practical Guide to Usability Testing

. Norwood, NJ:

Ablex.

Elkerton, J., & Williges, R. (1984). Information retrieval strategies in a file search environ-

ment.

Human Factors

26(2):171–84.

Enablemart.com. (2007). Hearing; available at

http://www.enablemart.com/productGroup-

Detail.aspx?store=10&dept=15&group=29

Engadget.com. (2005). Morse Code Trumps SMS in Head-to-Head Speed Texting Combat.

May 26—

http://www.engadget.com/2005/05/06/morse-code-trumps-sms-in-head-to-head-

speed-texting-combat/

Engadget.com. (2004). Watch this Wednesday: The Samsung GPRS Class 10 Watch Phone.

November 17—

http://www.engadget.com/2004/11/17/watch-this-wednesday-the-samsung-

gprs-class–10-watch-phone/

Genuth, I. (2006). Heliodisplay—Floating Free-Space Interactive Display; available at

http://www.tfot.info/content/view/101/

Geven, A., Sefelin, R., & Tscheligi, M. (2006). Depth and breadth away from the desktop—

the optimal information hierarchy for mobile use. In

Proceedings of Eighth Conference

on Human–Computer Interaction with Mobile Devices and Services

. New York: ACM

Press, 157–64.

Gohring, N. (2006). Mobile phone sales topped 800 million in 2005, says IDC.

InfoWorld

—

http://www.infoworld.com/article/06/01/27/74861_HNmobilephonesales_1.html

Grether, W. F., & Baker, C. A. (1972). Visual presentation of information. In Van Cott, H. P.,

& Kinkade, R. G., eds.,

Human Engineering Guide to Equipment Design

. New York: John

Wiley & Sons, 41–121.

Gutwin, C., & Fedak, C. (2004). Interacting with big interfaces on small screens: A compari-

son of fisheye, zoom, and panning techniques.

Proceedings of 2004 Conference on Gra-

phics Interfaces

. Waterloo, Ontario, Canada, 145–52.

Gwmicro.com. (2007). GW Micro—Small Talk Ultra; available at

http://www.gwmicro.com/

Small-Talk_Ultra/

Hayhoe, D. (1990). Sorting-based menu categories.

International Journal of Man–Machine

Studies

33:677–705.

Isaacs, E., & Walendowski, A. (2002).

Design from Both Sides of the Screen

. Indianapolis:

New Riders.

Jones, M., & Marsden, G. (2006).

Mobile Interaction Design

. London: John Wiley & Sons.

References

355

Lam, H., & Baudisch, P. (2005). Summary thumbnails: Readable overview for small screen

web browsers.

Proceedings of SIGCHI Conference on Human Factors in Computing Sys-

tems

. New York: ACM Press, 681–90.

Lee, E. S., & Raymond, D. R. (1993). Menu-driven systems. In Kent, A., & Williams, J., eds.,

The Encyclopedia of Microcomputers

11:101–27.

Lewis, C., & Reiman, J. (1993). Task-Centered User Interface Design; available at

http://

hcibib.org/tcuid/index.html

Logitech.com. (2007). HarmonyW 670 Advanced Universal Remote; available at

http://

www.logitech.com/index.cfm/products/detailsharmony/US/EN,CRID=2084,

CONTENTID=12701

MacGregor, J. N. (1992). A comparison of the effects of icons and descriptors in videotext

menu retrieval.

International Journal of Man–Machine Studies

37:767–77.

Marics, M. A., & Engelbeck, G. (1997). Designing voice menu applications for telephones. In

Helander, M., Landauer, T. K., & Prabhu, P., eds.,

Handbook of Human–Computer Inter-

action

. Amsterdam: Elsevier Science, 1085–102.

Marshall, C. C., & Ruotolo, C. (2002). Reading-in-the-small: A study of reading on small form

factor devices. In

Proceedings of Second ACM/IEEE-CS Joint Conference on Digital

Libraries

. New York: ACM Press, 56–64.

MiniFonts.com. (2006). Home page.

http://www.minifonts.com/

Muto, W. H., & Maddox, M. E. (2007, in press). Visual displays. In Weinger, M. B., ed.,

Hand-

book on Human Factors in Medical Device Design.

Hillsdale, NJ: Erlbaum.

Nielsen, J. (1993).

Usability Engineering

. San Francisco: Morgan Kaufmann.

Noldus. (2007). Mobile device camera; see

http://www.noldus.com/site/doc200402054

Norman, D. (1999).

The Invisible Computer

. Cambridge, MA: MIT Press.

Norman, D. (1988).

The Design of Everyday Things

. New York: Basic Books.

Norman, K. L., & Chin, J. P. (1988). The effect of tree structure on search performance in a

hierarchical menu selection system.

Behaviour and Information Technology

7:51–65.

Nystedt, D. (2006). Mobile subscribers t o reach 2.6B this year.

InfoWorld

—

http://www.

infoworld.com/article/06/ 11/10/HNmob ilesubsc ribers_1.html

Onetouch.ca. (2007). OneTouch Ultra; available at

http://www.onetouch.ca/english/

product_detail.asp?gr=0&cat=1&pid=47

Paap, K. R., & Cooke, N. J. (1997). Design of menus. In Helander, M., Landauer, T. K., &

Prabhu, P., eds.,

Handbook of Human–Computer Interaction

. Amsterdam: Elsevier Sci-

ence, 533–72.

Research in Motion. (2003). Blackberry Wireless Handheld User Interface Style Guide; available

http://www.blackberry.com/developers/na/java/doc/bbjde/BlackBerry_Wireless_Hand-

held_User_Interface_Style_Guide.pdf

Roeder-johnson.com. (2007). Projection Keyboard PDA; available at

http://www.roeder-

johnson.com/RJDocs/CAProjectionKeyboardPDA.jpg

Roske-Hofstrand, R. J., & Paap, K. R. (1986). Cognitive networks as a guide to menu organi-

zation: An application in the automated cockpit.

Ergonomics

29(11):1301–12.

Rubin, J. (1994).

Handbook of Usability Testing: How to Plan, Design, and Conduct Effec-

tive Tests

. New York: John Wiley & Sons.

Rugg, G., & McGeorge, P. (1997). The sorting techniques: A tutorial paper on card sorts, pic-

ture sorts, and item sorts.

Expert Systems

14(2):80–93.

10 Small-Screen Interfaces

356

S60.com. (2007). Nokia 95; available at

http://www.s60.com/life/s60phones/displayDevice-

Overview.do?deviceId=1065

Sanders, M. S., & McCormick, E. J. (1993).

Human Factors in Engineering and Design

. New

York: McGraw-Hill.

Sarker, S., & Wells, J. D. (2003). Understanding mobile handheld device use and adoption.

Communications of the ACM

46(12):35–40.

Schiffman, H. R. (1990).

Sensation and Perception: An Integrated Approach

. New York:

John Wiley & Sons.

Schusteritsch, R., Wei, C., & LaRosa, M. (2007). Towards the perfect infrastructure for

usability testing on mobile devices.

Proceedings of Conference on Human Factors in

Computing Systems. CHI ’07 Extended Abstracts on Human Factors in Computing Sys-

tems.

New York: ACM Press, 1839–44.

Singh, A. (2006). The Apple Motion Sensor as a Human Interface Device; available at

http://www.kernelthread.com/software/ams2hid/ams2hid.html

Tognazzini, B. (1992).

Tog on Interface

. Reading, MA: Addison-Wesley.

U.S. Department of Defense. (1999).

Department of Defense Design Criteria Standard:

Human Engineering

. MIL-STD-1472F. Washington, DC: U.S. DOD.

Wikimedia.org. (2007). Motorola RAZR Keypad; available at

http://upload.wikimedia.org/

wikipedia/en/thumb/1/10/RazrKeypad.jpg/180px-RazrKeypad.jpg

Wikipedia.org. (2007). Vacuum fluorescent display; available at

http://en.wikipedia.org/

wiki/Vacuum_fluorescent_display

Williams, M., & Cowley, S. (2006). Update: PC market achieved double-digit growth in 2005.

Info-

World

;availableat

http://www.infoworld.com/article/06/01/19/74317_HNgrowthin2005_1.

html

Wobbrock, J. O., Forlizzi, J., Hudson, S. E., & Myers, B. A. (2002). WebThumb: Interaction

techniques for small-screen browsers.

Proceedings of Fifteenth Annual ACM Symposium

on User Interface Software and Technology

. New York: ACM Press, 205–08.

Wright, S. L., & Samei, E. (2004). Liquid-crystal displays for medical imaging: A discussion of

monochrome versus color. Medical Imaging 2004: Visualization, Image-Guided Proce-

dures, and Display.

Proceedings of SPIE

5367:444–55.

Yates, A., Etzioni, R., & Weld, D. (2003). A reliable natural language interface to household

appliances. In

Proceedings of Eighth International Conference on Intelligent User Inter-

faces

. New York: ACM Press, 189–96.

Zwick, C., Burkhard, S., & Ku

hl, K. (2005).

Designing for Small Screens

. Switzerland: AVA.

References

357

This page intentionally left blank

CHAPTER

Multimode Interfaces: Two

or More Interfaces to Accomplish

the Same Task

Aaron W. Bangor, James T. Miller

11.1

NATURE OF THE INTERFACE

Today, technology is providing consumer and business users with an increasing

array of devices and software applications that can be used to organize, entertain,

and inform. Because these technology solutions have become more ubiquitous,

flexible, and capable, it is reasonable to ask if the user interface (UI) that allows

the user to take advantage of the technology has kept pace. A couple of scenarios

may help illustrate this.

In the first scenario, let us suppose that you are riding the bus home from

work and, just as Murphy’s Law might have predicted, you get stuck in traffic

and will not be home until much later than you expected. You will, in fact, be

so late that you are going to miss your favorite TV show. You did not set it up to

record because you thought you would be home in plenty of time. No one is home

to record it for you, so you now wish you had a way to record the show, such as

calling your digital video recorder (DVR) from your cell phone to set up the

recording.

Another scenario is that your best friend just had a big argument with her

roommate and you want to call her to give her some support. She has moved to

another place and emailed you her new phone number. You know you changed

the phone number when you got a message from her, but you cannot remember

where you changed it—in your e-mail address book, your cell phone speed dial,

your voice dial list on your home phone, and/or your PDA. And you definitely

do not want to call the old number and speak with the former roommate.

The first scenario points to users’ need to be able to manage their services

using a number of different devices, depending on the situation. The second sce-

nario illustrates the frustrations that many users can experience when these

needs are met, but without forethought about how the multiple devices that pro-

vide the same functionality might work together.

That is the topic of this chapter—how to design applications that provide users

with more than one UI modality, so that they can interact with the same application

and do the same tasks with whatever device and interface are most convenient

at the time. For example, in the first scenario the visual on-screen interface for

the home television is sufficient for the most common user situations. However,

another visual interface accessed by a computer, a wireless phone, or PDA would

be useful in other situations. It might even be better to provide an auditory

interface via a phone so that the user would not be forced to enter the relevant

information but could just speak it.

In the second scenario, the multiple modalities are already present, but the

design of each device was done without considering that the device—and the

information used by the device—might also apply to several other devices used

to accomplish the same tasks. Ideally, the information common to all of these

devices should be available to all of them so that a change in the information for

one device would result in a change for all.

The nature of the individual UIs that are a part of multimode UIs (MMUIs) is,

for the most part, not different from the other UI modalities discussed in this

book. And much has been researched and written about single-user, single-mode

UIs compared to MMUIs (Kray, Wasinger, & Kortuem, 2004). However, the focus

of this chapter is on the unique nature of designing UIs for the same application

where the user is expected to use multiple modalities to interact with the same

service and accomplish the same tasks. Research requirements, criteria to con-

sider when electing to create multiple modalities, and design and testing methods

are included in this chapter.

As a start, Seffah, Forbrig, and Javahery (2004) provide the following ele-

ments that define an MMUI:

F Allows a single or a group of users to interact with server-side services and

information using different interaction/UI styles.

F Allows an individual or a group to achieve a sequence of interrelated tasks

using multiple devices.

F Presents features and information that behave the same across platforms,

even though each platform/device has its specific look and feel.

F Feels like a variation of a single interface for different devices with the

same capabilities.

Note that these elements are technology independent and focus on a user—or

group of users—accomplishing tasks, although there may be variations on how

this is implemented, depending on the exact device and UI technology. That is

not to say that an MMUI is implemented without considering the specific UI tech-

nologies available (Paterno, 2004).

11 Multimode Interfaces: Two or More Interfaces

360

Typically, an MMUI is implemented using visual and/or auditory UIs. Other

chapters of this book cover these in more detail, including speech (Chapter 6),

touch-tone (Chapter 7), and small-screen visual displays (Chapter 10). Chapter 12

also has useful information on designing for more than one modality, although it

covers integrating more than one modality into the same UI, rather than

providing multiple, yet separate, modalities. There are numerous sources avail-

able for how to design visual user interfaces. Please see the end of this chapter

for general references regarding graphical UIs (GUIs) (Carroll, 2003; ISO, 1998;

Thatcher et al., 2006; Weinschenk, Jamar, & Yeo, 1997).

11.2

TECHNOLOGY OF THE INTERFACE

Any UI technology can be a candidate for an MMUI. Generally some combination

of graphical, web, touch-tone, speech recognition, and small-screen visual UIs is

considered when designing MMUIs. It is possible that other modalities, such as

tactile, olfactory, and gustatory, could be used, but these generally have not been

considered because they lack the powerful input/output capabilities that the

visual and auditory technologies have.

Other chapters of this book discuss the technologies used in nonvisual UI

technologies and many other works can provide the same for visual UIs. How-

ever, there are still some important technology considerations for MMUIs.

First, it should be understood that with MMUIs there is a greatly increased

chance of simultaneous interaction between application and user or multiple

users on the same account—either intentionally or unintentionally. While these

situations occur with single-modality UIs, greater attention needs to be given to

them in the case of MMUIs because of the multiple access methods.

Major concerns in these scenarios follow:

F Ensure accurate presentation of information to the user because of updates

made in other UIs.

F Prevent user confusion about displayed information or application behavior

because of changes made through other UIs that changed the state of the

information or application.

F Establish the precedence given to changes made to the account or account

data from the different UIs.

F Prevent data corruption or lost work when accessing the same data from more

than one UI.

F Allow the user to quickly switch between UIs to accomplish a task or a series

of tasks, if it is appropriate to the task(s).

As an example of the first consideration, think of a situation in which changes to an

address book on one device are propagated to other devices that have address books.

11.2 Technology of the Interface

361