![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
This work presents a novel learning method in the context of embodied artificial intelligence and self-organization, which has as few assumptions and restrictions as possible about the world and the underlying model. The learning rule is derived from the principle of maximizing the predictive information in the sensorimotor loop. It is evaluated on robot chains of varying length with individually controlled, noncommunicating segments. The comparison of the results shows that maximizing the predictive information per wheel leads to a higher coordinated behavior of the physically connected robots compared with a maximization per robot. Another focus of this article is the analysis of the effect of the robot chain length on the overall behavior of the robots. It will be shown that longer chains with less capable controllers outperform those of shorter length and more complex controllers. The reason is found and discussed in the information-geometric interpretation of the learning process.
References
| Amari, S. ( 1998). Natural gradient works efficiently in learning. Neural Computation, 10(2), 251-276. Google Scholar | Crossref | ISI | |
| Ay, N. , Bertschinger, N. , Der, R. , Güttler, F. , & Olbrich, E. ( 2008). Predictive information and explorative behavior of autonomous robots. The European Physical Journal B - Condensed Matter and Complex Systems, 63(3), 329- 339. Google Scholar | Crossref | |
| Barto, A.G. , Singh, S. , & Chentanez, N. ( 2004). Intrinsically motivated learning of hierarchical collections of skills. In J. Triesch & T. Jebara (Eds.), In Proceedings of 3rd International Conference on Development and Learning (pp. 112-119). San Diego, CA. Google Scholar | |
| Benveniste, A. , Priouret, P. , & Métivier, M. (1990). Adaptive algorithms and stochastic approximations. New York: Springer-Verlag. Google Scholar | |
| Bertschinger, N. ( 2008). An information theoretic perspective on cognitive systems: Memory and autonomy. Unpublished doctoral dissertation, University of Leipzig. Google Scholar | |
| Bialek, W. , Nemenman, I. , & Tishby, N. ( 2001). Predictability, complexity, and learning. Neural Computation, 13(11), 2409-2463. Google Scholar | Crossref | Medline | ISI | |
| Brooks, R.A. ( 1986). A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation, 2(1), 14-23. Google Scholar | Crossref | ISI | |
| Brooks, R.A. ( 1991). Intelligence without reason. In J. Myopoulos & R. Reiter (Eds.), Proceedings of the 12th International Joint Conference on Artificial Intelligence (IJCAI-91) (pp. 569-595). San Mateo, CA: Morgan Kaufmann publishers. Google Scholar | |
| Clark, A. ( 1996). Being there: Putting brain, body, and world together again. Cambridge, MA: MIT Press . Google Scholar | |
| Cliff, D. ( 1990). Computational neuroethology: a provisional manifesto . In From Animals to Animats: Proceedings of the 1st International Conference on Simulation of Adaptive Behavior (pp. 29-39). Cambridge, MA: MIT Press. Google Scholar | |
| Cover, T.M. , & Thomas, J.A. ( 2006). Elements of information theory (2 nd ed.). New Jersey: Wiley. Google Scholar | |
| Crutchfield, J.P. , & Young, K. ( 1989). Inferring statistical complexity. Physical Review Letters, 63(2), 105- 108. Google Scholar | Crossref | Medline | ISI | |
| Der, R. ( 2001). Self-organized acquisition of situated behavior. Theory in Biosciences, 120, 179-187. Google Scholar | Crossref | ISI | |
| Der, R. , Güttler, F. , & Ay, N. ( 2008). Predictive information and emergent cooperativity in a chain of mobile robots. In S. Bullock, J. Noble, R. Watson, & M. A. Bedau (Eds.), In Artifical Life XI. Proceedings of the Eleventh International Conference on the Simulation and Synthesis of Living Systems (pp. 166-172). Cambridge, MA: MIT Press. Google Scholar | |
| Der, R. , & Liebscher, R. ( 2002). True autonomy from self-organized adaptivity. In R. Damper & D. Ciff (Eds.) In Proceedings of the EPSRC/BBSRC International Workshop on Biologically-Inspired Robotics: The Legacy of Grey Walter. (pp. 134-141). Southampton, UK: University of Southampton. Google Scholar | |
| Di Paolo, E.A. ( 2000). Homeostatic adaptation to inversion of the visual field and other sensorimotor disruptions. In J.-A. Meyer, A. Berthoz, H. Floreano, D. Roitblat , & S. Wilson (Eds.), From Animals to Animats 6. Proceedings of the VI International Conference on Simulation of Adaptive Behavior. (pp. 440-449) Cambridge, MA: MIT Press. Google Scholar | |
| Förster, H. von. ( 1993). Wissen und Gewissen : Versuch einer Brücke (1 . Aufl. ed.; S. J. Schmidt , Ed.). Frankfurt am Main, D: Suhrkamp. Google Scholar | |
| Grassberger, P. ( 1986). Toward a quantitative theory of self-generated complexity . International Journal of Theoretical Physics, 25(9), 907-938. Google Scholar | Crossref | ISI | |
| Hofbauer, J. , & Sigmund, K. ( 2003). Evolutionary game dynamics. Bulletin of the American Mathematical Society, 40, 479-519. Google Scholar | Crossref | ISI | |
| Kakade, S. ( 2002). A natural policy gradient. Advances in neural information processing systems, 14 (pp. 1531-1538). Cambridge, MA: MIT Press. Google Scholar | |
| Kaplan, F. , & Oudeyer, P.-Y. (2004). Maximizing learning progress: An internal reward system for development. Embodied artificial intelligence (pp. 259-270). Berlin: Springer. Google Scholar | |
| Laughlin, S. ( 1981). A simple coding procedure enhances a neuron’s information capacity. Zeitschrift fr Naturforschung C, 36(9-10), 910-912. Google Scholar | |
| Linsker, R. ( 1988). Self-organization in a perceptual network. IEEE Computers, 88, 105-117. Google Scholar | |
| Lungarella, M. , & Sporns, O. ( 2005). Information self-structuring: Key principle for learning and development. In IEEE Proceedings of the 4th International Conference on Development and Learning (pp. 25-30). San Diego, CA: IEEE Press. Google Scholar | |
| Meltzoff, A. , & Moore, M.K. ( 1997). Explaining facial imitation: A theoretical model. Early Development and Parenting, 6, 179-192. Google Scholar | Crossref | Medline | |
| Mondada, F. , Franzi, E. , & Ienne, P. ( 1993). Mobile robot miniaturization: A tool for investigation in control algorithms. In Proceedings of the 3rd International Symposium on Experimental Robotics (pp. 501-513). Berlin: Springer Verlag. Google Scholar | |
| Nolfi, S. , & Floreano, D. ( 2000). Evolutionary robotics. Cambridge, MA: MIT Press. Google Scholar | |
| Oudeyer, P.-Y. , Kaplan, F. , & Hafner, V. ( 2007). Intrinsic motivation systems for autonomous mental development . IEEE Transactions on Evolutionary Computation, 11(2), 265-286. Google Scholar | Crossref | ISI | |
| Peters, J. , Vijayakumar, S. , & Schaal, S. ( 2005). Natural actor-critic. In Proceedings of the 16th European Conference on Machine Learning (ECML 2005) (pp. 280-291). Berlin: Springer. Google Scholar | |
| Pfeifer, R. , & Bongard, J.C. ( 2006). How the body shapes the way we think: A new view of intelligence . Cambridge, MA: MIT Press (Bradford Books). Google Scholar | |
| Pfeifer, R. , Lungarella, M. , & Iida, F. ( 2007). Self-organization, embodiment, and biologically inspired robotics. Science, 318(5853), 1088-1093. Google Scholar | Crossref | Medline | ISI | |
| Polani, D. , Nehaniv, C. , Martinetz, T. , & Kim, J.T. ( 2006). Relevant information in optimized persistence vs. progeny strategies. In L. M. Rocha , M. Bedau , D. Floreano , R. Goldstone , A. Vespignani , & L. Yaeger (Eds.), Proceedings of Artificial Life X. (pp. 337-343). Cambridge, MA: MIT Press. Google Scholar | |
| Porr, B. ( 2003). Sequence-learning in a self-referential closed-loop behavioural system. Phd thesis, Faculty of Human Sciences, Department of Psychology. University of Stir-ling. Google Scholar | |
| Schmidhuber, J. ( 1990). A possibility for implementing curiosity and boredom in model-building neural controllers. In From Animals to Animats: Proceedings of the 1st International Conference on Simulation of Adaptive Behavior (pp.222-227). Cambridge, MA: MIT Press. Google Scholar | |
| Schmidhuber, J. ( 2009). Driven by compression progress: A simple principle explains essential aspects of subjective beauty, novelty, surprise, interestingness, attention, curiosity, creativity, art, science, music, jokes. Anticipatory Behavior in Adaptive Learning Systems (pp. 48-76). Berlin : Springer. Google Scholar | |
| Shannon, C.E. ( 1948). A mathematical theory of communication. Bell System Technical Journal, 27, 379-423. Google Scholar | Crossref | |
| Steels, L. ( 2004). The autotelic principle. Embodied artificial intelligence (pp. 231-242). Berlin: Springer. Google Scholar | |
| Steels, L. , & Wellens, P. ( 2007). Scaffolding language emergence using the autotelic principle . In H. A. Abbass, M, Bedau, S. Nolfi & J. Wiles (Eds.), in Proceedings of the 1st IEEE Symposium on Artificial Life (pp. 325-332). Honolulu, HI: IEEE Press. Google Scholar | |
| Still, S. ( 2009). Information-theoretic approach to interactive learning . EPL, 85(2), 28005. Google Scholar | Crossref | ISI | |
| Storck, J. , Hochreiter, S. , & Schmidhuber, J. (1995). Reinforcement driven information acquisition in non-deterministic environments. In F. Fogelman-Soulíe, J. C. Rault , P. Gallinari & G. Dreyfus (Eds.), in Proceedings of the International Conference on Artificial Neural Networks (pp. 159- 164). Paris: EC2 & Cie. Google Scholar | |
| Thelen, E. , & Smith, L.B. ( 1996). A dynamic systems approach to the development of cognition and action. Cambridge, MA: MIT Press . Google Scholar | |
| Uexkuell, J. von. ( 1957). A stroll through the worlds of animals and men. In C. H. Schiller (Ed.), Instinctive behavior (pp. 5-80). New York: International Universities Press. (Original work published 1934) Google Scholar | |
| Williams, T. , & Kelley, C. ( 2009). gnuplot 4.2.6. http://www.gnuplot.info . Google Scholar | |
| Zahedi, K. & Paseman, F. ( 2007). Adaptive behavior control with self-regulating neurons . In M. Lungarella , F. Iida , J. Bongard , & R. Pfeifer (Eds.), (pp. 196-205). Berlin Heidelberg : Springer. Google Scholar | |
| Zahedi, K. , von Twickel, A. , & Pasemann, F. ( 2008). Yars: A physical 3d simulator for evolving controllers for real robots. In S. Carpin et al. (Eds.), Simpar 2008 (pp.71-82). Berlin: Springer. Google Scholar |
