Verbmobil: Foundations of Speech-to-Speech Translation

Verbmobil is the result of eight years of intensive research in a large speech-to-speech translation project, executed by a consortium comprising nineteen academic and four industrial partners. The system that was developed by more than 100 researchers and engineers handles dialogs in three business-oriented domains, with translation between three languages: German, English, and Japanese. Verbmobil deals with spontaneous speech, which includes realistic repair phenomena, and uses deep semantic analysis to recognize a speaker's slips and to translate what he tried to say rather than what he actually said.
After more than 400 single publications, this book gives the first comprehensive overview of the results of this unique and seminal project in human language technology. The theoretical and practical foundations from signal processing to language understanding are described in detail. The system anatomy is presented and the components for multilingual and robust speech processing, prosodic analysis, parsing, semantic analysis, discourse understanding, translation, generation and speech synthesis are discussed.
Contributions by leading scientists in speech and language technology look at the component technologies that make Verbmobil the most advanced speech-to-speech translation system worldwide and a landmark project in the history of natural language processing.

Auteur

Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster is the Director and CEO of the German Research Center for Artificial Intelligence (DFKI GmbH) and a Professor of Computer Science at the Universität des Saarlandes, Saarbrücken. In 2000, he was coopted as a Professor of Computational Linguistics at the same university. In addition, he is the Head of the Intelligent User Interfaces Lab at DFKI.

He was the Scientific Director of the Verbmobil consortium on spontaneous speech translation (1993-2000), the SmartKom consortium on multimodal dialog systems (1999-2003), and the SmartWeb consortium on mobile multimodal access to semantic web services (2004-2008).

He has authored more than 150 technical papers and 6 books on language technology and intelligent user interfaces, and edited many books, among them the Springer titles "Verbmobil: Foundations of Speech-to-Speech Translation", "SmartKom: Foundations of Multimodal Dialogue Systems", and "SemProM: Foundations of Semantic Product Memories for the Internet of Things". His research includes multimodal and perceptive user interfaces, user modeling, embodied conversational agents, smart navigation systems, semantic web services, and resource-adaptive cognitive technologies.

He was awarded the Deutscher Zukunftspreis (German Future Award) in 2001 and he received the Federal Cross of Merit, First Class, the Officer's Cross of the Order of Merit of the Federal Republic of Germany in 2006.

He is the Chief Academic Advisor for Information and Communication Technologies (ICT) in the Research Union of the German Government, he is a member or chair of many key international scientific advisory and governance boards, including the National Institute of Informatics (NII) of Japan, the jury for the European ICT Prize of the European Commission, the Singapore Agency for Science, Technology and Research, the SAP Next Business & Technology Advisory Board, the Center for Advanced Security Research in Darmstadt, the Intel Visual Computing Institute, the Fondazione Bruno Kessler, and the Future Internet Public-Private Partnership of the European Commission.

He is a member of many academies, including the German Academy of Sciences and Literature, Mainz, the Royal Swedish Academy of Sciences, Stockholm, the German Academy of Natural Scientists Leopoldina, Acatech, the Council for Engineering Sciences of the Union of German Academies of Science and Humanities, and the Berlin-Brandenburg Academy of Sciences and Humanities.

He has chaired and given invited talks at the key international conferences in computational linguistics and artificial intelligence, he is a board member of the key AI journals, and he is a Fellow of the Association for the Advancement of Artificial Intelligence (AAAI), the European Coordinating Committee for Artificial Intelligence (ECCAI), and the Gesellschaft für Informatik (GI).

Texte du rabat

In 1992 it seemed very difficult to answer the question whether it would be possible to develop a portable system for the automatic recognition and translation of spon­ taneous speech. Previous research work on speech processing had focused on read speech only and international projects aimed at automated text translation had just been terminated without achieving their objectives. Within this context, the German Federal Ministry of Education and Research (BMBF) made a careful analysis of all national and international research projects conducted in the field of speech and language technology before deciding to launch an eight-year basic-research lead project in which research groups were to cooperate in an interdisciplinary and international effort covering the disciplines of computer science, computational linguistics, translation science, signal processing, communi­ cation science and artificial intelligence. At some point, the project comprised up to 135 work packages with up to 33 research groups working on these packages. The project was controlled by means of a network plan. Every two years the project sit­ uation was assessed and the project goals were updated. An international scientific advisory board provided advice for BMBF. A new scientific approach was chosen for this project: coping with the com­ plexity of spontaneous speech with all its pertinent phenomena such as ambiguities, self-corrections, hesitations and disfluencies took precedence over the intended lex­ icon size. Another important aspect was that prosodic information was exploited at all processing stages.

Contenu

Mobile Speech-to-Speech Translation of Spontaneous Dialogs: An Overview of the Final Verbmobil System.- Facts and Figures about the Verbmobil Project.- Multilingual Speech Recognition.- Robust Recognition of Spontaneous Speech.- Fast Search for Large Vocabulary Speech Recognition.- Capturing Long Range Correlations Using Log-Linear Language Models.- Data Driven Generation of Pronunciation Dictionaries.- The Prosody Module.- The Recognition of Emotion.- Processing Self-Corrections in a Speech-to-Speech System.- Integrated Shallow Linguistic Processing.- Probabilistic LR-Parsing with Symbolic Postprocessing.- Robust Chunk Parsing for Spontaneous Speech.- Verbmobil Interface Terms (VITs).- Semantic Construction.- Deep Linguistic Analysis with HPSG.- HPSG Analysis of German.- HPSG Analysis of English.- HPSG Analysis of Japanese.- Efficient and Robust Parsing of Word Hypotheses Graphs.- Speech Lexica and Consistent Multilingual Vocabularies.- Combining Analyses from Various Parsers.- Robust Semantic Processing of Spoken Language.- Discourse and Dialog Semantics for Translation.- Multilingual Semantic Databases.- Semantic-Based Transfer.- Statistical Methods for Machine Translation.- Adapting a Large Scale MT System for Spoken Language.- Example-Based Incremental Synchronous Interpretation.- Example-Based Machine Translation with Templates.- Robust Content Extraction for Translation and Dialog Processing.- Modeling Negotiation Dialogs.- Dialog Processing.- Contextual Disambiguation.- The Verbmobil Generation Component VM-GECO.- The Application of HPSG-to-TAG Compilation Techniques.- Generating Multilingual Dialog Summaries and Minutes.- Speech Synthesis Using Multilevel Selection and Concatenation of Units from Large Speech Corpora.- Verbmobil Data Collection and Annotation.- The Tübingen Treebanks for Spoken German, English, and Japanese.- Multilingual Verbmobil-Dialogs: Experiments, Data Collection and Data Analysis.- Speech Recognition Performance Assessment.- Speech Synthesis Quality Assessment.- Fro…