Elamur Research⌕
☰
home / papers / 2012 / Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots

Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots

Paper · 2012 · Adrien Baranes and Pierre-Yves Oudeyer

Summary

For robots with many redundant degrees of freedom, it is difficult to learn an inverse model, that is, to find which motion parameters produce a desired effect in the task space. There are two reasons for this: the spaces are large and continuous, and each attempt takes real physical time. The authors propose the SAGG-RIAC architecture. Rather than trying random motor commands, the robot itself selects goals in a low-dimensional task space. It picks goals from regions where its competence improves fastest (competence progress). Local optimization then finds policy parameters for the selected goal, and regression generalizes the acquired mappings to new tasks. The method was tested on three setups: a redundant robotic arm, a quadruped robot, and an arm holding a flexible fishing rod. Exploring the task space proved considerably more efficient than exploring the actuator space. Progress-based goal selection statistically significantly outperformed random selection and standard variants of active motor babbling. In addition, the robot itself identifies which regions are unreachable for it. The work is important for autonomous learning and developmental robotics because it shows how to conserve a limited budget of trials when learning on real hardware.

Key points

  • SAGG-RIAC (Self-Adaptive Goal Generation – Robust Intelligent Adaptive Curiosity) combines two levels of active learning. At the upper level, the robot generates goals in the task space; at the lower level, it learns the motor policy parameters that lead to the selected goal.
  • The interest of a goal is estimated empirically: by how much previous attempts to reach similar goals have improved performance. As a result, learning naturally progresses from simple tasks to more complex ones.
  • Three experimental setups: inverse kinematics of a highly redundant arm; omnidirectional locomotion of a quadruped robot with motor primitives; control of a fishing rod with a flexible line.
  • The method was compared with random goal selection and with several active motor babbling methods in the action space. SAGG-RIAC proved more efficient, and its advantage over random selection is statistically significant.
  • Limitations: the task space must be low-dimensional and predefined. As directions for future work, the authors name automatic discovery of task spaces, social guidance of learning, and constraints inspired by infant maturation (maturational constraints).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Institut national de recherche en sciences et technologies du numériqueLe Chesnay-Rocquencourt · France90 works · 26,097 citations

Also on the byline: École Nationale Supérieure de Techniques Avancées · École Nationale Supérieure de Techniques Avancées Paris.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksEngineering · 206 worksControl and Systems Engineering · 72 worksBiomedical Engineering · 2 works

Cite

Cite as Adrien Baranes and Pierre-Yves Oudeyer. 2012. Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots. arXiv:1301.4862 · Baranes, A., Oudeyer, P-Y. (2013) Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots, Robotics and Autonomous Systems, 61(1), pp. 49-73.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1016/j.robot.2012.05.008 · arXiv 1301.4862

@misc{baranes2012active,
  title = {{Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots}},
  author = {Adrien Baranes and Pierre-Yves Oudeyer},
  year = {2012},
  doi = {10.1016/j.robot.2012.05.008},
  eprint = {1301.4862},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1016/j.robot.2012.05.008}
}

References

131 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. P. Abbeel and A. Ng. Apprenticeship learning via inverse reinforcement learning. In Proceedings of the 21st International Conference on Machine Learning (2004), pages 1–8. ACM Press, New York, 2004. find
  2. A. Albert. Regression and the moore-penrose pseudo inverse. In Mathematics in science and engineering. Academic Press, Inc., 1972. find
  3. H. R. Arkes and J. P. Garske. Optimal level theories. In Psychological theories of motivation, volume 2, pages 172–195. 1982. find
  4. S. Arya, D. M. Mount, N.S. Netanyahu, R. Silverman, and A.Y. Wu. An optimal algorithm for approximate nearest neighbor searching fixed dimensions. Journal of the ACM (JACM), 45(6):891–923, November 1998. find
  5. M. Asada, K. Hosoda, Y. Kuniyoshi, H. Ishiguro, T. Inui, Y. Yoshikawa, M. Ogino, and C. Yoshida. Cognitive developmental robotics: A survey. IEEE Trans. Autonomous Mental Development, 1(1), 2009. find
  6. B. Bakker and J. Schmidhuber. Hierarchical reinforcement learning based on subgoal discovery and subpolicy specialization. In Proc. 8th Conf. on Intelligent Autonomous Systems (2004), 2004. find
  7. G. Baldassare. What are intrinsic motivations? a biological perspective. In Proceeding of the IEEE ICDL-EpiRob Joint Conference, 2011. find
  8. A. Baranes and P-Y. Oudeyer. Riac: Robust intrinsically motivated exploration and active learning. IEEE Trans. on Auto. Ment. Dev., 1(3):155– 169, 2009. find
  9. A. Baranes and P. Y. Oudeyer. Intrinsically motivated goal exploration for active motor learning in robots: A case study. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Taipei, Taiwan, 2010. find
  10. A. Baranes and P-Y. Oudeyer. The interaction of maturational constraints and intrinsic motivations in active motor development. In Proceedings of ICDL-EpiRob 2011, 2011. find
  11. A. Barto and O. Simsek. Intrinsic motivation for reinforcement learning systems. In CT New Haven, editor, Proceedings of the Thirteenth Yale Workshop on Adaptive and Learning Systems (2005), 2005. find
  12. A Barto, S Singh, and N. Chenatez. Intrinsically motivated learning of hierarchical collections of skills. In Proc. 3rd Int. Conf. Dvp. Learn., pages 112–119, San Diego, CA, 2004. find
  13. D. Berlyne. Conflict, Arousal and Curiosity. McGraw-Hill, 1960. find
  14. D. Berlyne. Curiosity and exploration. Science, 153(3731):25–33, 1966. find
  15. M. Berniker, A. Jarc, E. Bizzi, and M.C. Tresch. Simplified and effective motor control based on muscle synergies to exploit musculoskeletal dynamics. Proceedings of the National Academy of Sciences of the United States of America (PNAS), 106(18):7601–7606, May 2009. find
  16. N Bernstein. The Coordination and Regulation of Movements. Pergamon, 1967. find
  17. N. E. Berthier, R.K. Clifton, D.D. McCall, and D.J. Robin. Proximodistal structure of early reaching in human infants. Exp Brain Res, 1999. find
  18. Aude Billard, Sylvain Calinon, Rudiger Dillmann, and Stefan Schaal. Robot programming by demonstration. In B. Siciliano and O. Khatib, editors, Handbook of Robotics, pages 1371–1394. Springer, 2008. find
  19. C.M. Bishop. Pattern recognition and machine learning. In Information Science and Statistics. Springer, 2007. find
  20. S. Bitzer and S. Vijayakumar. Latent spaces for dynamic movement primitives. In Proceedings of IEEE/RAS International Conference on Humanoid Robots, 2009. find
  21. D. Blank, D. Kumar, L. Meeden, and J. Marshall. Bringing up robot: Fundamental mechanisms for creating a self-motivated, self-organizing architecture. Cybernetics and Systems, 36(2), 2005. find
  22. Josh C. Bongard. Morphological change in machines accelerates the evolution of robust behavior. Proceedigns of the National Academy of Sciences of the United States of America (PNAS), January 2010. find
  23. D. Bullock, S. Grossberg, and F.H. Guenther. A self-organizing neural model of motor equivalent reaching and tool use by a multijoint arm. Journal of Cognitive Neuroscience, 5(4):408–435, 1993. find
  24. S. Calinon and A. Billard. Statistial learning by imitation of competing constraints in joint space and task space. Advanced Robotics, 23(15):2059– 2076, 2009. find
  25. R. Cantin-Martinez, M. Lopes, and L. Montesano. Body schema acquisition through active learning. In EEE - International Conference on Robotics and Automation (ICRA), Anchorage, Alaska, USA, 2010. find
  26. T. Cederborg, M. Li, A. Baranes, and P-Y. Oudeyer. Incremental local inline gaussian mixture regression for imitation learning of multiple tasks. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (2010), Taipei, Taiwan, 2010. find
  27. S. Chernova and M. Veloso. Interactive policy learning through confidence-based autonomy. J. Artificial Intelligence Research, 34:1–25, 2009. find
  28. S Chiaverini. Singularity-robust task-priority redundancy resolution for real-time kinematic control of robot manipulators. IEEE Transactions on Robotics and Automation, 13(3):398–410, 1997. find
  29. D. Cohn, L. Atlas, and R. Ladner. Improving generalization with active learning. Mach. Learn., 15(2):201–221, 1994. find
  30. David A. Cohn, Zoubin Ghahramani, and Michael I. Jordan. Active learning with statistical models. J Artificial Intelligence Research, 4:129–145, 1996. find
  31. M. Csikszentmihalyi. Creativity-Flow and the Psychology of Discovery and Invention. Harper Perennial, New York, 1996. find
  32. S. Dasgupta. Analysis of a greedy active learning strategy. Adv. Neural Inform. Process. Systems, 17, 2004. find
  33. J. Dattorro. Convex Optimization and Euclidean Distance Geometry. Me-boo Publishing USA, 2011. find
  34. A. D’Avella, P. Saltiel, and E. Bizzi. Combinations of muscle synergies in the construction of a natural motor behavior. Nature neuroscience, 6(3):300–308, march 2003. find
  35. E.L. Deci and M. Ryan. Intrinsic Motivation and self-determination in human behavior. Plenum Press, New York, 1985. find
  36. F. Delcomyn. Neural basis for rhythmic behaviour in animals. Science, 210:492–498, 1980. find
  37. T. B. Dichter, N. A. Busch, and D. E. Knauf. Mastery motivation: Appropriate tasks for toddlers. Infant Behavior and Development, 20(4):545– 548, 1997. find
  38. G.L. Drescher. Made-Up Minds: A Constructivist Approach to Artificial Intelligence. MIT Press, 1991. find
  39. J. Elman. Learning and development in neural networks: The importance of starting small. Cognition, 48:71–99, 1993. find
  40. I. Fasel, A. Wilt, N. Mafi, and C. T. Morrison. Intrinsically motivated information foraging. In Proceedings of the IEEE 9th International Conference on Development and Learning (2010), 2010. find
  41. V. Fedorov. Theory of Optimal Experiment. Academic Press, Inc., New York, NY, 1972. find
  42. R. M. French, M. Mermillod, P. C. Quinn, A. Chauvin, and D. Mareschal. The importance of starting blurry: Simulating improved basic-level category learning in infants due to weak visual acuity. In LEA, editor, Proceedings of the 24th Annual Conference of the Cognitive Science Society (2002), pages 3… find
  43. Y. Freund, H.S. Seung, E. Shamir, and N. Tishby. Selective sampling using the query by committee algorithm. Machine Learning, 28(2-3):133–168, 1997. find
  44. S. Grillner and P. Wallen. Central pattern generators for locomotion, with special reference to vertebrates. Annual Review of Neuroscience, 8:233– 261, march 1985. find
  45. N. Hansen and A. Ostermeier. Completely derandomized self- adaptation in evolution strategies. Evolutionary Computation, 9(2):159–195, 2001. find
  46. S. Hart and R. Grupen. Intrinsically motivated hierarchical manipulation. In Proceedings of the IEEE Conference on Robots and Automation (2008), 2008. find
  47. S. Hart, S. Sen, and R. Grupen. Generalization and transfer in robot control. In Lund Univeristy Cognitive Studies, editor, Proc. Of the 8th International Conference On Epigenetic Robotics (2008), University of Sussex, 2008. find
  48. X. Huang and J. Weng. Novelty and reinforcement learning in the value system of developmental robots. In C. Prince, Y. Demiris, Y. Marom, H. Kozima, and C. Balkenius, editors, Proc 2nd Int. Workshop Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems, volume 94, pages 47–55. Lund… find
  49. A.J. Ijspeert. Central pattern generators for locomotion control in animals and robots: A review. Neural Networks, 21(4):642–653, 2008. find
  50. N. Jetchev and M. Toussaint. Task space retrieval using inverse feedback control. In L. Getoor and T. Scheffer, editors, International Conference on Machine Learning (ICML-11), volume 28, pages 449–456, New York, NY, USA, 2011. find
  51. D. Jones, M. Schonlau, and W. Welch. Efficient global optimization of expensive black-box functions. Global Optimization, 13(4):455–492, 1998. find
  52. S. Kakade and P. Dayan. Dopamine: Generalization and bonuses. Neural Networks, 15(4-6):549–59, 2002. find
  53. A. Kapoor, K. Grauman, R. Urtasun, and T. Darrell. Active learning with gaussian processes for object categorization. In Proceeding of the IEEE 11th Int. Conf. Comput. Vis. (2007), Crete, Greece, 2007. find
  54. J. Klein. Breve: a 3d environment for the simulation of decentralized systems and artificial life. In MIT Press, editor, Proceeding of the eighth international conference on artificial life (2003), 2003. find
  55. J. Kober, E. Oztop, and J. Peters. Reinforcement learning to adjust robot movements to new situations. In Proceedings of Robotics: Science and Systems, 2010. find
  56. A. Krause and C. Guestrin. Nonmyopic active learning of gaussian processes: an exploration-exploitation approach. In 24th international conference on Machine learning, 2007. find
  57. A. Krause, A. Singh, and C. Guestrin. Near-optimal sensor placements in gaussian processes: Theory, efficientalgorithms and empirical studies. Journal of Machine Learning Research, 9:235–284, 2008. find
  58. W.A. Lee. Neuromotor synergies as a basis for coordinated intentional action. J. Mot. Behav., 16:135–170, 1984. find
  59. M. Lopes, F. Melo, B. Kenward, and J. Santos-Victor. A computational model of social-learning mechanisms. Adaptive Behavior, 467(17), 2009. find
  60. M. Lopes, f. Melo, and J. Santos-Victor. Abstraction levels for robotic imitation: Overview and computational approaches. In From Motor Learning to Interaction Learning in Robots. SpringerLink, 2010. find
  61. M. Lopes, F. S. Melo, and L. Montesano. Active learning for reward estimation in inverse reinforcement learning. In European Conference on Machine Learning (ECML/PKDD), Bled, Slovenia, 2009. find
  62. M. Lopes and P-Y. Oudeyer. Active learning and intrinsically motivated exploration in robots: Advances and challenges (guest editorial). IEEE Transactions on Autonomous Mental Development, 2(2):65–69, 2010. find
  63. M. Lopes and J. Santos-Victor. A developmental roadmap for learning by imitation in robots. IEEE Transactions in Systems Man and Cybernetic, Part B: Cybernetics, 37(2), 2007. find
  64. M. Luciw, V. Graziano, M. Ring, and J. Schmidhuber. Artificial curiosity with planning for autonomous perceptual and cognitive development. In Proceeding of the First IEEE ICDL-EpiRob Joint Conference (2011), 2011. find
  65. J. Marshall, D. Blank, and L. Meeden. An emergent framework for self-motivation in developmental robotics. In Proc. 3rd Int. Conf. Development Learn. (2004), pages 104–111, San Diego, CA, 2004. find
  66. Harold Martinez, Max Lungarella, and Rolf Pfeifer. On the influence of sensor morphology on eye motion coordination. In Proc. of the IEEE 9th International Conference on Development and Learning (2010), pages 238 –243, August 2010. find
  67. R. Martinez-Cantin, N. de Freitas, E. Brochu, J. Castellanos, and A. Doucet. A Bayesian exploration-exploitation approach for optimal online sensing and planning with a visually guided mobile robot. Autonomous Robots - Special Issue on Robot Learning, Part B, 2009. find
  68. Kathryn Merrick and Mary Lou Maher. Motivated learning from interesting events: Adaptive, multitask learning agents for complex environments. Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems, 17(1):7–27, 2009. find
  69. J. Mugan and B. Kuipers. Autonomously learning an action hierarchy using a learned qualitative state representation. In Proceedings of the International Joint Conference on Artificial Intelligence (2009), 2009. find
  70. M. Muja and D.G. Lowe. Fast approximate nearest neighbors with automatic algorithm. In International Conference on Computer Vision Theory and Applications (2009), 2009. find
  71. M. Nguyen, A. Baranes, and P-Y. Oudeyer. Bootstrapping intrinsically motivated learning with human demonstrations. In proceedings of the IEEE International Conference on Development and Learning, Frankfurt, Germany, 2011. find
  72. D. Nguyen-Tuong and J. Peters. Model learning for robot control: A survey. Cognitive Processing, 12(4):319–340, 2011. find
  73. J. Nishii, Y. Uno, and R. Suzuki. Mathematical models for the swimming pattern of a lamprey. In Biological Cybernetics, volume 72. Springer, 1994. find
  74. P. Oudeyer and F. Kaplan. How can we define intrinsic motivations ? In Proc. Of the 8th Conf. On Epigenetic Robotics (2008), 2008. find
  75. P. Oudeyer, F. Kaplan, V. Hafner, and A. Whyte. The playground experiment: Task-independent development of a curious robot. In Proceedings of the AAAI Spring Symposium on Developmental Robotics, pages 42–47, 2005. find
  76. P.-Y. Oudeyer and F. Kaplan. What is intrinsic motivation? a typology of computational approaches. Frontiers of Neurorobotics, page 1:6, 2007. find
  77. P-Y. Oudeyer, F. Kaplan, and V. Hafner. Intrinsic motivation systems for autonomous mental development. IEEE Transactions on Evolutionary Computation, 11(2):pp. 265–286, 2007. find
  78. J. Peters and S. Schaal. Natural actor critic. Neurocomputing, (7-9):1180– 1190, 2008. find
  79. J. Peters and S. Schaal. reinforcement learning of motor skills with policy gradients. (4):682–97, 2008. find
  80. J. Peters, S. Vijayakumar, and S. Schaal. Reinforcement learning for humanoid robotics. In Third IEEE-RAS International Conference on Humanoid Robots (2003), 2003. find
  81. I. Poupyrev, H. Kato, and M. Billinghurst. Artoolkit user manual, version 2.33. Technical report, University of Washington, 2000. find
  82. R. E. Redding, G. A. Morgan, and R. J. Harmon. Mastery motivation in infants and toddlers: Is it greatest when tasks are moderately challenging? Infant Behavior and Development, 11(4):419–430, 1988. find
  83. P. Redgrave and K. Gurney. The short-latency dopamine signal: A role in discovering novel actions? Nat. Rev. Neurosci., 7(12):967–75, Nov 2006. find
  84. M. Riedmiller, T. Gabel, R. Hafner, and S. Lange. Reinforcement learning for robot soccer. Autonomous Robot, 27:55–73, 2009. find
  85. M. Rolf, J. Steil, and M. Gienger. Goal babbling permits direct learning of inverse kinematics. IEEE Trans. Autonomous Mental Development, 2(3):216–229, 09/2010 2010. find
  86. M. Rolf, J. Steil, and M. Gienger. Online goal babbling for rapid bootstrapping of inverse models in high dimensions. In Proceeding of the IEEE ICDL-EpiRob Joint Conference (2011), 2011. find
  87. L. Ronnquist and C. von Hofsten. Neonatal finger and arm movements as determined by a social and an object context. Early Develop. Parent., 3(2):81–94, 1994. find
  88. P. Rouanet, P-Y. Oudeyer, and D. Filliat. An integrated system for teaching new visually grounded words to a robot for non-expert users using a mobile device. In Proceedings of IEEE-RAS International Conference on Humanoid Robots (HUMANOIDS 2010), Paris, France, 2009. find
  89. N. Roy and A. McCallum. Towards optimal active learning through sampling estimation of error reduction. In Proc. 18th Int. Conf. Mach. Learn. (2001), volume 1, pages 143–160, 2001. find
  90. Richard M. Ryan and Edward L. Deci. Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25(1):54 – 67, 2000. find
  91. C. Salaun, V. Padois, and O. Sigaud. Learning forward models for the operational space control of redundant robots. In From Motor Learning to Interaction Learning in Robots, volume 264, pages 169–192. Springer, 2010. find
  92. S. Schaal and C. G. Atkeson. robot juggling: an implementation of memory-based learning. Control systems magazine, pages 57–71, 1994. find
  93. A. Schein and L.H. Ungar. Active learning for logistic regression: An evaluation. Machine Learning, 68:235–265, 2007. find
  94. M. Schembri, M. Mirolli, and Baldassarre G. Evolution and learning in an intrinsically motivated reinforcement learning robot. In Springer, editor, Proceedings of the 9th European Conference on Artificial Life (2007), pages 294–333, Berlin, 2007. find
  95. M. Schembri, M. Mirolli, and Baldassarre G. Evolving childhood’s length and learning parameters in an intrinsically motivated reinforcement learning robot. In Proceedings of the Seventh International Conference on Epigenetic Robotics. Lund University Cognitive Studies, 2007. find
  96. J. Schmidhuber. Curious model-building control systems. In Proc. Int. Joint Conf. Neural Netw. (1991), volume 2, pages 1458–1463, 1991. find
  97. J. Schmidhuber. A possibility for implementing curiosity and boredom in model-building neural controllers. In J. A. Meyer and S. W. Wilson, editors, Proc. SAB’91, pages 222–227, 1991. find
  98. J. Schmidhuber. Artificial curiosity based on discovering novel algorithmic predictability through coevolution. In P. Angeline, Z. Michalewicz, M. Schoenauer, X. Yao, and Z. Zalzala, editors, Congress on Evolutionary Computation, pages 1612–1618, Piscataway, NJ, 1999. IEEE Press. find
  99. J. Schmidhuber. Exploring the Predictable, pages 579–612. Springer, 2002. find
  100. J. Schmidhuber. Optimal artificial curiosity, developmental robotics, creativity, music, and the fine arts. Connection Science, 18(2), 2006. find
  101. J. Schmidhuber. Formal theory of creativity, fun, and intrinsic motivation. IEEE Transaction on Autonomous Mental Development, 2(3):230– 247, 2010. find
  102. J. Schmidhuber. Powerplay: Training an increasingly general problem solver by continually searching for the simplest still unsolvable problem. In Report arXiv:1112.5309, 2011. — [arXiv:1112.5309](https://arxiv.org/abs/1112.5309) find
  103. G. Schohn and D. Cohn. Less is more: Active learning with support vector machines. Proceedings of the Seventeenth International Conference on Machine Learning, 2000. find
  104. W. Schultz, P. Dayan, and P. Montague. A neural substrate of prediction and reward. Science, 275:1593–1599, 1997. find
  105. B. Settles. Active learning literature survey. CS Tech. Rep. 1648, Univ. Wisconsin-Madison, Madison, WI, 2009.
  106. Siciliano and Khatib. Handbook of Robotics. Springer, 2008. find
  107. O. Sigaud, C. Salaun, and V. Padois. On-line regression algorithms for learning mechanical models of robots: a survey. Robotics and Autonomous System, 59(12):1115–1129, July 2011. find
  108. B. Castro Da Silva, G. Konidaris, and A. Barto. Learning parameterized skills. In Proceedings of International Conference of Machine Learning, 2012. find
  109. Ö. Şimşek and A.G. Barto. An intrinsic reward mechanism for efficient exploration. In Proceedings of the Twenty-Third International Conference on Machine Learning (2006), 2006. find
  110. S. Singh, R.L. Lewis, A.G. Barto, and J. Sorg. Instrinsically motivated reinforcement learning: An evolutionary perspective. IEEE Transactions on Autonomous Mental Development (IEEE TAMD), 2(2):70–82, 2010. find
  111. A. Stout and A Barto. Competence progress intrinsic motivation. In Proceedings of the International Conference on Development and Learning (2010), 2010. find
  112. F. Stulp and O. Sigaud. Path integral policy improvement with covariance matrix adaptation. In Proceedings of International Conference of Machine Learning, 2012. find
  113. F. Stulp, E. Theodorou, M. Kalakrishnan, P. Pastor, L. Righetti, and S. Schaal. Learning motion primitive goals for robust manipulation. In Int. Conference on Intelligent Robots and Systems (IROS),, 2011. find
  114. Richard S. Sutton and Andrew G. Barto. Reinforcement learning: An introduction, 1998. find
  115. R.S. Sutton. Integrated architectures for learning, planning, and reacting based on approximating integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In Proceedings of the International Machine Learning Conference, pages 212–218, 1990. find
  116. R.S. Sutton, D. Precup, and S. Singh. Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 1123(181-211), 1999. find
  117. E Theodorou, J Peters, and S. Schaal. Reinforcement learning for optimal control of arm movements. In Abstracts of the 37st meeting of the society of neuroscience (2007), 2007. find
  118. S. Thrun. Exploration in active learning. In M Arbib, editor, Handbook of Brain Science and Neural Networks, Cambridge, MA, 1995. MIT Press. find
  119. S. Thrun and K. Moller. Active exploration in dynamic environments. In Proceedings of Advances of Neural Information Processing Systems, 1992. find
  120. L. Ting and J. McKay. Neuromechanics of muscle synergies for posture and movement. Curr. Opin. Neubiol., 7:622–628, 2007. find
  121. Marc Toussaint and Amos Storkey. Probabilistic inference for solving discrete and continuous state markov decision processes. In Proceedings of the 23rd international conference on Machine learning, ICML ’06, pages 945–952, New York, NY, USA, 2006. ACM. find
  122. G. Turkewitz and P.A. Kenny. The role of developmental limitations of sensory input on sensory/perceptual organization. J Dev Behav. Pediatr., 6(5):302–6, 1985. find
  123. A. van der Meer. Keeping the arm in the limelight: Advanced visual control of arm movements in neonates. Eur. J. Paediatric Neurol, 1(4):103– 108, 1997. find
  124. A. van der Meer, F. van der Weel, and D. Lee. The functional significance of arm movements in neonates. Science, 267(5198):693–695, 1995. find
  125. S. Vijayakumar, A. D’Souza, and S. Schaal. Incremental online learning in high dimensions. Neural Computation, 17(12):2602–2634, 2005. find
  126. C. von Hofsen. An action perspective on motor an action perspective on motor development. TRENDS in Cognitive Science, 8(6), 2004. find
  127. L.S. Vygotsky. Mind and society: The development of higher mental processes. Cambridge, MA: Harvard University Press, 1978. find
  128. J. Weng. Developmental robotics: Theory and experiments. Int. J. Humanoid Robotics, 1(2):199–236, 2004. find
  129. J. Weng, J. McClelland, A. Pentland, O. Sporns, I. Stockman, M. Sur, and E. Thelen. Autonomous mental development by robots and animals. Science, 291(599-600), 2001. find
  130. R. White. Motivation reconsidered: The concept of competence. Psychol. Rev., 66:297–333, 1959. find
  131. S. Whitehead. A study of cooperative mechanisms for faster reinfocement learning. Technical Report 365, Univ. Rochester, Rochester, NY, 1991. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/baranes-2013-active-learning-inverse-models-intrinsically/