Bunt H., Beun R.-J., Borghuis T. (eds.) Multimodal Human-Computer Communication. Systems, Techniques, and Experiments

Подождите немного. Документ загружается.

Lecture Notes in Artificial Intelligence

Subseries of Lecture Notes in Computer Science

Edited by J. G. Carbonell and J. Siekmann

1374

Lecture Notes in Computer Science

Edited by G. Goos, J. Hartmanis and J. van Leeuwen

Harry Bunt Robbert-Jan Beun

Tijn Borghuis (Eds.)

Multimodal

Human-Computer

Communication

Systems, Techniques,

and Experiments

Springer

Series Editors

Jaime G. Carbonell, Carnegie Mellon Universit3r Pittsburgh, PA, USA

J6rg

Siekmann, University of Saarland, Saarbriicken, Germany

Volume Editors

Harry Bunt

Tilburg University

Warandelaan 2, 5000 LE Tilburg, The Netherlands

E-mail: bunt@kub.nl

Robbert-Jan Beun

Center for Research on User-System Interaction (IPO)

P.O. Box 513, 5600 MB Eindhoven,The Netherlands

E-mail: rjbeun@ipo.tue.nl

Tijn Borghuis

Eindhoven University of Technology

P.O. Box 513, 5600 MB Eindhoven,The Netherlands

E-mail: tijn@win.tue.nl

Cataloging-in-Publication Data applied for

Die Deutsche Bibliothek - CIP-Einheitsaufnahme

Muitimodal human computer communication :

systems,

techniques, and experiments / Harry Bunt ... (ed.). - Berlin ;

Heidelberg

; New York ; Barcelona ; Budapest ;

Hong Kong

;

London ; Milan

; Paris ; Santa Clara ; Singapore ; Tokyo :

Springer,

1998

(Lecture notes in computer science ; Vol. 1374 : Lecture notes in

artificial intelligence)

ISBN 3-540-64380-X

CR Subject Classification (1991): 1.2, H.5.1-2, 1.3.6, D.2.2, K.4.2

ISSN 0302-9743

ISBN 3-540-64380-X Springer-Verlag Berlin Heidelberg New York

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is

concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting,

reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publicauon

or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965,

in its current version, and permission for use must always be obtained from Springer -Verlag. Violations are

liable for prosecution under the German Copyright Law.

9 Springer-Verlag Berlin Heidelberg 1998

Printed in Germany

Typesetting: Camera ready by author

SPIN 10631926 06/3142 - 5 4 3 2 1 0 Printed on acid-free paper

Preface

This volume contains revised versions of seventeen selected papers from the First

International Conference on Cooperative Multimodal Communication (CMC/95),

held in Eindhoven, the Netherlands, in May 1995. This was the first conference

in a series, of which the second one was held in Tilburg, The Netherlands, in

January 1998. Three of these papers were presented by invited speakers; those

by Mark Maybury, Bonnie Webber, and Kent Wittenburg. From the submitted

papers that were accepted by the CMC/95 program committee, thirteen were

selected for publication in this volume, after revision.

We thank the program committee for their excellent and timely feedback to

authors of submitted papers, and at a later stage for advising on the contents

of this volume and for providing additional suggestions for improving the se-

lected contributions. The program committee consisted of Norman Badler, Harry

Bunt, Jeroen Groenendijk, Walther von Hahn, Dieter Huber, Hans Kamp, John

Lee, Joseph Mariani, Mark Maybury, Paul Mc Kevitt, Rob Nederpelt, Kees van

Overveld, Ray Perranlt, Donia Scott, Wolfgang Wahlster, Bonnie Webber, and

Kent Wittenburg. We thank the Royal Dutch Academy of Sciences (KNAW)

and the Organization for Cooperation among Universities in Brabant (SOBU)

for their grants that made the conference possible.

January 1998

Harry Bunt

Robbert-Jan Beun

Tijn Borghuis

Table of Contents

Issues in Multimodal Human-Computer Communication

Harry Bunt

Part I: Systems

Towards Cooperative Multimedia Interaction

Mark T. Maybury

Multimodal Cooperation with the DenK System

Harry Bunt, Rend Ahn, Robbert-Jan Beun, Tijn Borghuis and

Kees van Overveld

Synthesizing Cooperative Conversation

Catherine Pelachaud, Justine CasseU, Norman Badler, Mark Steedman,

Scott Prevost and Matthew Stone

Instructing Animated Agents: Viewing Language in Behavioral

Terms

Bonnie Webber

Modeling and Processing of Oral and Tactile Activities in the

GEORAL System

Jacques Siroux, Marc Guyomard, Franck Multon and

Christophe Remondeau

Multimodal Maps: An Agent-Based Approach

Adam Cheyer and Luc Julia

Using Cooperative Agents to Plan Multimodal Presentations

Yi Han and Ingrid Zukerman

101

111

122

Part II: Techniques

Developing Multimodal Interfaces: A Theoretical Framework

and Guided Propagation Networks

Jean-Claude Martin, Remko Veldman and Dominique Bdroule

158

VIII

Cooperation between Reactive 3D Objects and a Multimodal

X Window Kernel for CAD

Patrick Bourdot, Mike Krus and Rachid Gherbi

A Multimedia Interface for Circuit Board Assembly

Fergal McCaffery, Michael Mc Tear and Maureen Murphy

Visual Language Parsing: If I Had a Hammer...

Kent Wittenburg

Anaphora in Multimodal Discourse

John Lee and Keith Stenning

188

213

231

250

Part III: Experiments

Speakers' Responses to Requests for Repetition in a

Multimedia Language Processing Environment

Laurel Fais, Kyung-ho Loke-Kim and Young-Duk Park

Object Reference in Task-Oriented Keyboard Dialogues

Anita Cremers

Referent Identification Requests in Multi-Modal Dialogs

Tsuneaki Kato and Yukiko L Nakano

Studies into Full Integration of Language and Action

Carla Huls and Edwin Bos

The Role of Multimodal Communication in Cooperation:

The Cases of Air Traffic Control

Marie-Christine Bressolle, Bruno Pavard and Marcel Leroux

264

279

294

312

326

Author Index 345

Lecture Notes in Artificial Intelligence

Subseries of Lecture Notes in Computer Science

Edited by J. G. Carbonell and J. Siekmann

1374

Lecture Notes in Computer Science

Edited by G. Goos, J. Hartmanis and J. van Leeuwen

Harry Bunt Robbert-Jan Beun

Tijn Borghuis (Eds.)

Multimodal

Human-Computer

Communication

Systems, Techniques,

and Experiments

Springer

Series Editors

Jaime G. Carbonell, Carnegie Mellon Universit3r Pittsburgh, PA, USA

J6rg

Siekmann, University of Saarland, Saarbriicken, Germany

Volume Editors

Harry Bunt

Tilburg University

Warandelaan 2, 5000 LE Tilburg, The Netherlands

E-mail: bunt@kub.nl

Robbert-Jan Beun

Center for Research on User-System Interaction (IPO)

P.O. Box 513, 5600 MB Eindhoven,The Netherlands

E-mail: rjbeun@ipo.tue.nl

Tijn Borghuis

Eindhoven University of Technology

P.O. Box 513, 5600 MB Eindhoven,The Netherlands

E-mail: tijn@win.tue.nl

Cataloging-in-Publication Data applied for

Die Deutsche Bibliothek - CIP-Einheitsaufnahme

Muitimodal human computer communication :

systems,

techniques, and experiments / Harry Bunt ... (ed.). - Berlin ;

Heidelberg

; New York ; Barcelona ; Budapest ;

Hong Kong

;

London ; Milan

; Paris ; Santa Clara ; Singapore ; Tokyo :

Springer,

1998

(Lecture notes in computer science ; Vol. 1374 : Lecture notes in

artificial intelligence)

ISBN 3-540-64380-X

CR Subject Classification (1991): 1.2, H.5.1-2, 1.3.6, D.2.2, K.4.2

ISSN 0302-9743

ISBN 3-540-64380-X Springer-Verlag Berlin Heidelberg New York

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is

concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting,

reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publicauon

or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965,

in its current version, and permission for use must always be obtained from Springer -Verlag. Violations are

liable for prosecution under the German Copyright Law.

9 Springer-Verlag Berlin Heidelberg 1998

Printed in Germany

Typesetting: Camera ready by author

SPIN 10631926 06/3142 - 5 4 3 2 1 0 Printed on acid-free paper