Multimodality in Language and Speech Systems

This book covers the topic of multimodality from a large number of different perspectives and provides the advanced student/researcher with a current survey of theories of multimodal communication between people as well as reviewing many aspects of multimodal input/output in technical systems. Chapters dealing with human-human multimodal communication include speech-gesture systems, semiotics of gesture, structure and functions of face-to-face communication, emotional relations and intercultural variation, and human-human communication which is mediated by computer for the handicapped. Chapters dealing with human-machine communication and interfaces cover the technology and science of creating talking faces, technology and methods for the development of animated interface agents in intelligent multimedia systems, and the integration of multimodal input and output in the computer interface. The book also covers computer processing and understanding of signal and symbol input from speech, text, and visual images.