SEARCH
MENU

History of Robotics Research and Development of Japan2011Integration, Intelligence, etc.Self-Regulation Mechanism: A Principle for Continual Autonomous Learning in Open-Ended Environments

Yukiko HoshinoSony Corporation(currently Kawada Robotics Corporation)
Kenta KawamotoSony Corporation
Kuniaki NodaSony Corporation(currently Waseda University)
Kohtaro SabeSony Corporation
and autonomous learning are key features for a developmental agent in open-ended environments. This paper presents a mechanism of self-regulated learning to realize them. Considering the fact that learning progresses only when the learner is exposed to appropriate level of uncertainty, we propose that an agent's learning process be guided by the following two metacognitive strategies throughout its development: (a) Switch of behavioral strategies to regulate the level of expected uncertainty, and (b) Switch of learning strategies in accordance with the current subjective uncertainty. With this mechanism, we demonstrate efficient and stable online learning of a maze where only local perception is provided: the agent autonomously explores an environment of significant-scale, and self-develops an internal model that properly describes the hidden structure behind its experience. 26th RSJ Best Paper Award in 2012.
不確実性制御行動(a)不確実性と学習効率の関係 (b)不確実性軸とフロー理論のスキル-チャレンジ平面のマッピング
Rough sketches of uncertainty regulating behaviors. (a)Relation between uncertainty and learning efficiency. (b)Uncertainty-axis mapped on the skill-challenge plane of the Flow theory
自律発達エージェントのシステム構成図
(a) 迷路状の環境(b) 学習したHMMの内部状態（人がわかりやすいようにエージェントは知らない位置情報を用いて作図）（円:内部状態　矢印：内部状態間の遷移) (c) ５つのアクションシンボル (d) 16個の観測シンボル
(a) Maze-like environment. (b) Internal states of the
learned HMM, arranged using the position information
which is hidden to the agent. (Circle: internal state.
Arrow: transition between internal states.) (c) 5 action
symbols. (d) 16 observation symbols

Movies

photo
Stable and efficient autonomous online learning of a large-scale POMDP environment

Correspondence papers

Tarou Kougaku, Ichirou Rika, and Jirou Kikai：Self-Regulation Mechanism: A Principle for Continual Autonomous Learning in Open-Ended Environments

Journal of the Robotics Society of Japan, Vol. 29, No. 1, pp. 77-88, 2011 (in Japanese).

Related papers

[1]Kenta KAWAMOTO, Yukiko HOSHINO, Kuniaki NODA, Kohtaro SABE: "Self-regulation mechanism for continual autonomous learning in open-ended environments", In Proceedings of International Conference on Epigenetic Robotics (EpiRob 2009), pp. 73-80, 2009.

[2]K. Sabe, K. Kawamoto, H. Suzuki, K. Minamino and Kenichi Hidai: “Reward-free Learning using Sparsely-connected Hidden Markov Models and Local Controllers", The 9th International Conference on Epigenetic Robotics, 2009.

[3]Kenta KAWAMOTO, Kuniaki NODA, Takashi HASUO, Kotaro SABE: "Development of object manipulation through self-exploratory visuomotor experience", In Proceedings of IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob 2011), 2011.

[4]Kuniaki NODA, Kenta KAWAMOTO, Takashi HASUO, Kotaro SABE: "A generative model for developmental understanding of visuomotor experience", In Proceedings of IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob 2011), 2011.

[5]L. Chrisman: “Reinforcement Learning with Perceptual Aliasing: The Perceptual Distinctions Approach”, Proceedings of the Tenth National Conference on Artificial Intelligence, pp.183–188. AAAI Press, 1992.

[6]P. R. Cavalin, R. Sabourin, C. Y. Suen and A. S. Britto Jr.:“Evaluation of Incremental Learning Algorithms for An HMMBased Handwritten Isolated Digits Recognizer”, Proceedings of The 11th International Conference on Frontiers in Handwriting Recognition, pp.1–6, 2008.

Related Article

記事はありません。