SEARCH
MENU

History of Robotics Research and Development of Japan2000Integration, Intelligence, etc.Robot Audition: Research Issues and Current Status

Kazuhiro NakadaiJapan Science and Technology Corp.
Tino LourensJapan Science and Technology Corp.
Hiroshi G. OkunoKyoto University
Hiroaki KitanoSonComputer Science Lab./Japan Science and Technology Corp.
Ken-ichi HidaiJapan Science and Technology Corp.
Auditory processing for a robot has not been discussed intensively compared to robot vision. The article for AAAI-2000 entitled "Active Audition for Humanoid" proposed robot audition as a new research field in robotics and discussed five issues to realize robot audition as follows: * acoustical effect of a robot's body * acoustical effect by a robot's motion * perception and understanding of a general sound including speech, music and so on * perception and understanding of sound in an unknown environment * acquisition of auditory and visual information using an active motion In particular, most robots took a strategy so called "stop-perceive-act", which is a strong restriction to be applied to the real world. To solve this problem, and to realize Computational Auditory Scene Analysis (CASA) by a robot, we claimed that active perception which make a robot move for better perception is necessary, and thus, active audition was proposed to listen to sound even while a robot is in motion. Active audition is a framework to control parameters such as microphone's positions, rotation of a robot's body  with microphones, and microphone's directivity for better audition. On the other hand, it generates inevitable noise due to active motion. Even when the power of generated noise is small, the microphones capture it with high power because the noise source is closer to microphones than a target sound source. Therefore, we proposed two methods to solve this. One is internal noise suppression using acoustic embodiment which separates robot's internal and external spaces with a robot cover. The other is binaural sound source localization based on the auditory epipolar geometry which does not require the head related transfer function of a robot. Preliminary results with the upper-torso humanoid robot SIG showed that both methods worked properly when robot's motion noise exists. International Conference on Information Society in the 21st Century: Emerging Technologies and New Challenges (IS2000) Best Paper Award in 2000. International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems (IEA/AIE-2001) Best Paper Award in2001. IEEE/RSJ IROS 2001 Best Paper Award Finalist in 2002.
ヒューマノイドロボットSIG
humanoid robot SIG
ロボット内外を分ける2組のマイクロホン
two pairs of microphones for acoustic embodiment
SIG内部構成
SIG internal structure

Movies

photo
Audio-visual human tracking
photo
Receptioninst robot integrated with sound localization, sound separation, automatic speech recognition and face recognition.
photo
Simultaneous speech recognition empowered by active audition (asking again)
photo
Audio-visual simultaneous speech recognition

Correspondence papers

K. Nakadai, T. Lourens, H. G. Okuno, and H. Kitano：Active Audition for Humanoid

Proc. of the Seventeenth National Conference on Artificial Intelligence (AAAI-2000), pp.832-839, 2000.

H. G. Okuno, K. Nakadai, Ken-ichi Hidai, H. Mizoguchi, H. Kitano：Human-Robot Non-Verbal Interaction Empowered by Real-Time Auditory and Visual Multiple-Talker Tracking

Advanced Robotics, Vol. 17, No. 2, pp. 115-130, 2003.

Kazuhiro Nakadai, Ken-ichi Hidai, Hiroshi Mizoguchi, Hiroshi G. Okuno, and Hiroaki Kitano：Real-Time Human Tracking by Audio-Visual Integration for Humanoids - Integration of Active Audition and Face Recognition -

Journal of the Robotics Society of Japan, Vol. 21, No. 5, pp. 57-65, 2003 (in Japanese).

Related papers

[1] Kazuhiro Nakadai, Tino Lourens, Hiroshi G. Okuno, Hiroaki Kitano: Humanoid Active Audition System Improved by The Cover Acoustics. In Mizoguchi, R. and Slaney, J. (eds) PRICAI-2000 Topics in Artificial Intelligence (Sixth Pacific Rim International Conference on Artificial Intelligence), 544-554, Lecture Notes in Artificial Intelligence No. 1886, Springer-Verlag, Melborne, Aug. 2000.
[2] Kazuhiro Nakadai, Hiroshi G. Okuno, Tino Lourens, Hiroaki Kitano. Humanoid Active Audition System. Proc. of First IEEE-RAS International Conference on Humanoid Robots (Humanoids2000), Cambridge, Sep. 2000.
[3] Kazuhiro Nakadai, Tatsuya Matsui, Hiroshi G. Okuno, Hiroaki Kitano: Active Audition System and Humanoid Exterior Design. Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS-2000), 1453-1461, Takamatsu, Nov. 2000.
[4] Hiroshi G. Okuno, Kazuhiro Nakadai, Tino Lourens, Hiroaki Kitano: Sound and Visual Tracking for Humanoid, Proc. of 2000 International Conference on Information Society in the 21st Century: Emerging Technologies and New Challenges (IS2000), 254-261, Aizu-Wakamatsu, Nov. 2000. (BEST PAPER AWARD)
[5] Hiroshi G. Okuno, Kazuhiro Nakadai, Lourens, Tino, Hiroaki Kitano: Sound and Visual Tracking for Humanoid Robot. Proc. of 17th International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems (IEA/AIE-2001), Lecture Notes in Artificial Intelligence, No. 2070, Springer-Verlag. Budapest, Hungary, June 2001. (BEST PAPER AWARD)
[6] Kazuhiro Nakadai, Ken-ichi Hidai, Hiroshi Mizoguchi, Hiroshi G. Okuno, Hiroaki Kitano: Real-Time Auditory and Visual Multiple-Object Tracking for Robots. Proc. of 17th International Joint Conference on Artificial Intelligence (IJCAI-01), 1425-1432, Seattle, Aug. 2001. (電気通信普及財団奨励賞受賞)
[7] Hiroshi G. Okuno, Kazuhiro Nakadai, Tino Lourens, Hiroaki Kitano: Human-Robot Interaction Through Real-Time Auditory and Visual Multiple-Talker Tracking Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS-2001), IEEE, Maui, Hawaii, Oct. 2001. (BEST PAPER Nomination Finalist)
[8] 中臺 一博, 奥乃 博, 北野 宏明: ヒューマノイドにおける聴覚機能の課題とアクティブオーディションによる音源定位, 人工知能学会論文誌, Vol. 18, No. 2-F, pp.104-113, Mar. 2003.
[9] K. Nakadai, D. Matsuura, H. G. Okuno, H. Tsujino: Improvement of Recognition of Simultaneous Speech Signals Using AV Integration and Scattering Theory for Humanoid Robots, Speech Communication, Vol.44, pp. 97-112, Elsevier, Oct. 2004.

Related Article

記事はありません。