This project, carried out in collaboration with Stefan Schaal (USC) and Jun Nakanishi (ATR), aims at developing controllers for learning by imitation with a humanoid robot. The controllers are based on nonlinear dynamical systems, and use locally weighted regression techniques to learn complex, discrete or rhythmic, movements demonstrated by a human subject. These controllers can be considered to be discrete or rhythmic pattern generators which can replay and modulate the learned movements, while being robust against perturbations. The controllers have been tested to learn a series of movements (e.g. tennis swings and drumming movements) with a humanoid robot.
The controllers are also used in our Virtual Trainer project (collaboration with Carolee Winstein, Department of Biokinesiology and Physical Therapy, USC) which aims at using a dynamic humanoid simulation for demonstrating and supervising rehabilitation exercises in stroke-patients. The system will be able 1) to create a database of exercises through the recording of movements shown by a physiotherapist, 2) to demonstrate selected exercises to the patient, 3) to monitor how well the patient is performing the exercise, 4) to point to errors and suggest corrections, and 5) to gradually increase the difficulty of the exercises depending on progress, following a rehabilitation program as defined by the physiotherapist. The system therefore relies on a process of demonstration and imitation which can also find applications in humanoid robotics.
The humanoid robotics part of the project was done in collaboration with Mitsuo Kawato and the Cyber Human Project group at the ATR, Human Information Science Laboratories.
People involved: Ludovic Righetti, Sarah Dégallier, Auke Ijspeert
Sample animations/movies
Demo of robot’s compliance: 
courtesy of Dr Shibata,
Discrete Control Policies:
Pointing movement: demonstration 
, imitation
Tennis forehand: demonstration , imitation
Tennis backhand: demonstration , imitation
Movement from the Wolf Motor Function Test learned by the trajectory formation system in the humanoid robot simulation (cf. IROS2001 paper): Animation
Rhythmic Control Policies:
set of demonstrations-imitations 
Figure 8: demonstration , imitation
Square figure: demonstration , imitation, modulation of anchor , imitation, variety of modulations
Drumming: demonstration , imitation, with frequency modulation
Drumming: demonstration , imitation with amplitude modulation and perturbation
Drumming: demonstration , imitation, with frequency modulation for each arm
Related student projects
Control and Synchronization with Nonlinear Dynamical Systems for an application to Humanoid Robotics , Ludovic Righetti (Diploma project 2003-2004)
Related publications
A.J. Ijspeert, J. Nakanishi, and S. Schaal. Learning attractor landscapes for learning motor primitives. In S. Becker, S. Thrun, and K. Obermayer, editors, Advances in Neural Information Processing Systems 15 (NIPS2002), pages 1547-1554, 2003. [ bib | .pdf ]
A.J. Ijspeert, J. Nakanishi, and S. Schaal. Learning rhythmic movements by demonstration using nonlinear oscillators. In Proceedings of the IEEE/RSJ Int. Conference on Intelligent Robots and Systems (IROS2002), pages 958-963, 2002. [ bib | .pdf ]
A.J. Ijspeert, J. Nakanishi, and S. Schaal. Movement imitation with nonlinear dynamical systems in humanoid robots. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA2002), pages 1398-1403, 2002. (received the ICRA2002 best paper award). [ bib | .pdf ]