Kortum P. (ed.) HCI Beyond the GUI. Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces

Подождите немного. Документ загружается.

9.1.2 Why Is a Taste Interface Difficult?

Taste is the last frontier of virtual reality. Taste is very difficult to display because

it is a multimodal sensation comprising chemical substance, sound, smell, and

haptic sensations. The literature on visual and auditory displays is extensive.

Smell display is not common, but smell can easily be displayed using a vaporizer.

Although not as extensive as visual and auditory displays, the literature on olfac-

tory displays is growing (e.g., Nakamoto et al., 1994; Davide et al., 2001).

Taste perceived by the tongue can be measured using a biological membrane

sensor (Toko et al., 1994, 1998). The sensor measures the chemical substance of

the five basic tastes. Any arbitrary taste can easily be synthesized from the five

tastes based on sensor data.

Another important element in taste is food texture. Measurement of biting

force has been studied. A multipoint force sensor has been used to measure force

distribution on the teeth (Khoyama et al., 2001, 2002). The sensory properties of

the texture of real food have also been studied (Szczesniak, 1963, 2002).

Dental schools have been working on the training of patients to chew prop-

erly. A master–slave manipulator for mastication has been developed (Takanobu

et al., 2002). A robot manipulated by the doctor applies appropriate force to the

patient’s teeth.

Haptic interfaces for hands or fingers are the focus of major research efforts

in virtual reality. In contrast, research into haptic displays for biting is very rare.

9.2

TECHNOLOGY OF THE INTERFACE

There have been no interactive techniques developed to date that can display

taste. The Food Simulator Project was an effort to develop just such an interface.

9.2.1 Existing Techniques for Displaying Taste

The traditional method for displaying taste is the use of filter paper discs, such as for

patients with taste disorders (Tsuruoka, 2003). A small filter paper disc is soaked

with a chemical (taste) substance and put on the patient’s tongue. Filter paper

discs with various density of the chemical substance are used to test taste disorders.

However, there has been no technique to display taste for human–computer

interaction.

9.2.2 Food Simulator Project: A Taste Interface

Challenge

In 2001, I launched a project called the “Food Simulator” to develop an interface

device that presents food texture as well as taste. The unsolved problem in taste

9 Taste Interfaces

292

display is a haptics issue. The first goal of the project was developing a haptic

interface for biting. The device should be suitably shaped for placing in the

mouth, and be effectively controlled to simulate food texture. The second goal

was to present a multimodal sensation to the user. To this end, biting sounds

and chemical tastes should be integrated with the haptic interface.

To achieve these goals, we developed a haptic interface to present the biting

force. The Food Simulator generates a force simulating the previously captured

force profiles of an individual biting real food. A film-like force sensor is used to

measure biting force associated with real food. A force sensor is installed in the

Food Simulator and the device is actuated using force control methods.

The Food Simulator is integrated with auditory and chemical sensations asso-

ciated with taste. The sound of biting is captured by a bone vibration microphone.

The sound is then replayed using a bone vibration speaker synchronized with the

biting action. The chemical sensation of taste is produced using an injection

pump, with a tube installed at the end effecter.

9.3

CURRENT IMPLEMENTATIONS

OF THE INTERFACE

The Food Simulator employs mechanical linkages to apply biting force to the

teeth. It is integrated with sound, vibration, and chemical taste.

9.3.1 Haptic Device in a Food Simulator

The haptic device is composed of a one degree-of-freedom (DOF) mechanism

that is designed to fit in the user’s mouth. The device is aimed at applying a force

representing the first bite. Chewing is not addressed in this prototype. Thus, the

device is designed to apply force in a direction normal to the teeth. The configu-

ration of the mechanical linkage takes into consideration the jaw structure. The

shape of the linkage enables the application of force to the back teeth. The width

of the end effecter is 12 mm, and the device applies force to two or three teeth.

Regarding hygiene, the end effecter includes a disposable cover of cloth and

rubber. Figure 9.1 shows the overall view of the apparatus.

The haptic device is composed of a ones DOF mechanism that employs four

linkages. Figure 9.2 illustrates its mechanical configuration. The linkages are

driven by a DC servo motor (MAXON Motor, RE25). The user bites the end effec-

ter of the device. The working angle of the end effecter is 35 degrees, and the max-

imum force applied to the teeth is 135 N. A force sensor that detects force applied

by the user’s teeth is attached to the end effecter. The device is controlled by a PC

(Pentium 4, 2 GHz). The update rate of force control is 1,700 Hz, which is suffi-

cient for controlling a haptic interface.

9.3 Current Implementations of the Interface

293

9.3.2 Multisensory Display

The second goal of the Food Simulator project was to present multimodal sensa-

tion. We also tried to integrate a multimodal display with the haptic interface.

Sound

Sound is closely related to biting action. Masti cation of a hard food generates

sound. This sound is perceived by vibration, mostly in the jawbone. Sounds of

biting real food were recorded using a bone vibration microphone. Figure 9.3

shows a recording session. The bone vibration microphone was inserted into

the ear. An accelerometer, installed in the microphone, picked up vibration of

the jawbone.

FIGURE

9.1

Overall view of the apparatus.

The 1-DOF haptic device is a core element of the system.

Motor

Force

sensor

Biting

force

FIGURE

9.2

Mechanical configuration of the haptic device.

Four linkages are employed in the device, which is designed to fit into

the mouth.

9 Taste Interfaces

294

FIGURE

9.3

Measuring biting sound.

An accelerometer picks up vibration of the jawbone while biting real food.

FIGURE

9.4

Bone vibration speaker.

The recorded sound is synchronized with the biting action.

The recorded sound was then displayed using a bone vibration speaker.

Figure 9.4 shows the speaker that generated vibrations in the jawbone. The sound

was synchronized with the biting action. For example, the sound of a virtual

cracker was displayed at the beginning of the second stage of the force control

shown in Figure 9.7 (see page 298).

9.3 Current Implementations of the Interface

295

Chemical Taste

Chemical sensations perceived by the tongue contribute greatly to the sense of taste.

An arbitrary taste can be synthesized from the five basic tastes. Table 9.1 summarizes

common chemical substances for the basic tastes.

The chemical sensation of taste was presented by injecting a small amount of

liquid into the mouth. A tube was attached to the end effecter of the haptic inter-

face. Figure 9.5 shows the tube at the top end of the linkage. The liquid was trans-

ferred using an injection pump (Nichiryo, DDU-5000). Figure 9.6 shows an overall

view of the injection pump. Injection of the liquid was synchronized with the

biting action. The pump provided 0.5 ml of solution for each bite.

Vision and Smell

Visual and olfactory sensations of food occur independently of the biting action. Thus,

the Food Simulator prototype did not support these sensations. However, head-

mounted displays can provide visualization of food. Also, smell can be displayed using

a vaporizer. These displays could easily be integrated with the haptic interface.

TABLE 9.1 Chemical Substances Used in Synthesis of Five Basic Tastes

Basic Taste Chemical Substance

Sweet Sucrose

Sour Tartaric acid

Salty Sodium chloride

Bitter Quinine sulfate

Umami Sodium glutamate

FIGURE

9.5

Tube for injection of chemical taste.

A small amount of liquid is injected from the top end of the tube into

the mouth.

9 Taste Interfaces

296

9.4

HUMAN FACTORS DESIGN OF THE INTERFACE

The Food Simulator is designed to generate biting force according to that of real

food. There are a number of human factors issues that must be considered when

designing a taste interface.

9.4.1 Measurement of Biting Force of Real Food

The Food Simulator generates force, simulating the force recorded from an indi-

vidual biting real food. A film-like force sensor (FlexiForce, Kamata Industries)

is used to measure this biting force. The sensor has a thickness of 0.1 mm and a

circular sensitive area of 9.5 mm in diameter. The sensing range is 0 to 110, with

the maximum sampling rate of 5,760 Hz. Figure 9.7 shows an overall view of the

sensor. Figure 9.8 shows a view of the participant in the experiment, who is biting

a cracker and the FlexiForce sensor.

Figure 9.9 shows measured biting force associated with a real cracker. Two peaks

appeared in the profile of the measured force. The first peak represents destruction of

the hard surface of the cracker. The second peak represents destruction of its internal

FIGURE

9.6

Injection pump.

The injection of the liquid is synchronized with the biting action. The pump

provides 0.5 ml of solution for each bite.

9.4 Human Factors Design of the Interface

297

FIGURE

9.8

Force sensor and real food.

The participant is biting real food along with the force sensor.

[N]

[s]

0.1 0.2 0.3 0.4 0.5

FIGURE

9.9

Measured force of a cracker.

The first peak represents destruction of the hard surface of the cracker.

The second peak represents destruction of its internal structure.

FIGURE

9.7

Film-like force sensor.

The sensor is placed in the mouth with real food to measure biting force.

9 Taste Interfaces

298

structure. Figure 9.10 shows the measured biting force associated with real cheese.

The slope observed in the figure represents the elastic deformation of the cheese.

9.4.2 Method of Generating Biting Force

Two food types are simulated: crackers and cheese.

Crackers

The force control of the device has two stages (Figure 9.11). First, the device applies

force to maintain its position. This process represents the hard surface of the virtual

cracker. When the biting force exceeds the first peak, the force control moves to the

second stage. The device generates the same force as that measured from the real

cracker. This stage is subject to open-loop control. Destruction of the internal struc-

ture of the virtual cracker is simulated by the second stage.

Figure 9.12 shows the measured force of the virtual cracker displayed by the

Food Simulator. The profile of the measured force has two peaks, as in the real

cracker. However, the shape of the profile is different from that associated with

the real cracker. This difference seems to be caused by the open-loop control.

Cheese

Figure 9.13 shows two stages of force control for biting real cheese. In order to sim-

ulate elastic deformation of the cheese, the spring constant of the cheese was esti-

mated from the collected data. The device generates force according to this spring

constant to display the elasticity of the virtual cheese. When the biting force exceeds

the peak force, the control enters the second stage. The device generates the same

[N]

[s]

0.2 0.4 0.6 0.8 1 1.2

FIGURE

9.10

Measured force of cheese.

The slope represents the elastic deformation of the cheese.

9.4 Human Factors Design of the Interface

299

[N]

[s]

Stage 1 Stage 2

0.1 0.2 0.3 0.4 0.5

FIGURE

9.11

Method of simulation of a cracker.

First, the device applies force to maintain its position. When the biting force

exceeds the first peak, the device generates the same force as that measured

from the real cracker.

[N]

[s]

0.2 0.4 0.6 0.8

FIGURE

9.12

Measured force of a virtual cracker.

Two peaks are observed.

9 Taste Interfaces

300

[N]

[s]

Spring constant

Stage 1 Stage 2

0.2 0.4 0.6 0.8 1 1.2

FIGURE

9.13

Simulation method for cheese.

The device generates force according to the spring constant of the real cheese.

[N]

[s]

0.2 0.4 0.6 0.8 1 1.41.2

FIGURE

9.14

Measured force of virtual cheese.

The profile resembles the measurements for real cheese.

force as that measured from the real cheese. This stage is subject to open-loop

control. Destruction of the virtual cheese is simulated by the second stage.

Figure 9.14 shows the measured force of the virtual cheese displayed by

the Food Simulator. The profile of the measured force is similar to the real one.

9.4 Human Factors Design of the Interface

301