History of Robotics Research and Development of Japan2013Integration, Intelligence, etc.Real-world Robust Microphone Array Processing for Robot Audition

Keisuke Nakamura	Honda Research Institute Japan Co., Ltd.
Hiroki Miura	Tokyo Institute of Technology
Takami Yoshida	Tokyo Institute of Technology
Yoshiaki Bando	Kyoto University
Takeshi Mizumoto	Honda Research Institute Japan Co., Ltd.
Kazuhiro Nakadai	Honda Research Institute Japan Co., Ltd.
Hiroshi G. Okuno	Kyoto University

Automatic Speech Recognition (ASR) by a robot is one of the most common human-robot interactions and is essential for a robot working in a human environment. Different from ASR, conducted via a close-talking microphone as observed in mobile phone applications, ASR by a robot has to use the robot's own microphones, and the location, timing, and number of speech sources are unknown. Moreover, because a robot has its own ego-noise (self-generated noise), the signal-to-noise ratio (SNR) of the speech is low in most of the cases. To achieve ASR in a highly dynamic and noisy environment, robot audition [1] has been studied for a decade. Figure 1 shows the overview of the robot audition system. In addition to the signal processing for ASR by a close-talking microphone, the robot audition system employs microphone array processing such as sound source localization (SSL) and sound source separation (SSS) to recognize an unknown number of individual sources of speech conducted simultaneously. Because SSL estimates the location, timing, and number of speech sources, and SSS separates the speech from each source, an improvement in the performances of SSL and SSS is essential. The microphone array processing generally has two assumptions; geometry information on a microphone array or time-consuming measurements of transfer functions between a microphone array and a sound source is necessary, and a fully synchronous microphones should be used. To solve these issues, we proposed in [2] an online framework for microphone array calibration by combining simultaneous localization and mapping (SLAM), and beamforming (See Figure 2). SLAM simultaneously calibrates locations of microphones and a sound source, and clocks differences between microphones every time a microphone array observes a sound event such as hand claps. In [3], we developed a self-calibration microphone array method using self-emitted sounds coming from the robot. In [4], we considered the practicality of using current robot audition systems in a highly dynamic and noisy environment and recognized that they have three problems: they are not adequately noise-robust for ego-noise, their resolution is not sufficiently high, and they exhibit difficulties during real-time implementation. To solve these three issues, we proposed in [4] SSL based on the spatial whitening of ego-noise using generalized singular value decomposition, the super-resolution SSL and SSS based on hybrid transfer function interpolation, and the optimal hierarchical SSL based on a coarse-to-fine approach (See Figure 3). Applying these methods to real robots, we have achieved the simultaneous speech recognition robot and SSL using a flying quadrocopter in an outdoor environment. 2th RSJ Advanced Robotics Best Paper Award in 2014. IROS 2011 Best Paper Award Finalist in 2011. IEEE Robotics and Automation Society Japan Chapter Young Award in 2013.

ロボット聴覚システムと課題 — Robot Audition System and its Issues

非同期マイクロホンアレイのオンラインキャリブレーション — Online Calibration of an Asynchronous Microphone Array

雑音ロバストな実時間超解像ロボット聴覚システム — Real-time Noise-robust Super-resolution Robot Audition System

記事はありません。

Related Article