imitation

Movement Imitation: Linking Perception and Action: 

Movement Imitation: Linking Perception and Action Advanced Topics in Computer Vision, 2004 Lior Noy Department of Computer Science and Applied Mathematics Weizmann Institute of Science

Movement Imitation - Example: 

Movement Imitation - Example

Slide3: 

semantic world (objects, actions) realm of raw-data (pixels, muscles activation) Action Perception Imitation: Linking Perception and Action Imitation

Outline: 

Outline 2. Programming By Demonstration 1. Movement Imitation 3. Robotic Movement Imitation Primitives Based Approach (Mataric’) Real Time Tracking (“mirror-game”) (Ude et al.) 4. Direct Perception and Imitation

A Variety of Probes into Imitation: 

A Variety of Probes into Imitation Imitation Ethology Cognitive psychology Developmental psychology Neurophysiology Human Brain Imaging Robotics

Slide6: 

Possible Questions In Imitation Research What is the content of imitation? How perceptions are transformed to actions? What are the processes of learning by imitation? How to evaluate imitation? What is a “good” imitation? How does the ability to imitate develop?

Evaluating ImitationRobot Following in a Hilly Environment: 

Evaluating Imitation Robot Following in a Hilly Environment

Evaluating Imitation: 

Evaluating Imitation

Slide9: 

Evaluating Imitation

Slide10: 

Programming By Demonstration (PbD) Methods to program a robot Human Programming Reinforcement Learning Programming by Demonstration

Programming By Demonstration (PbD) Applications: 

Programming By Demonstration (PbD) Applications Navigation Locomotion Playing air-hockey Manipulating blocks Balancing a pole Hitting a tennis-serve Grasping unfamiliar objects Imitating dancing movement

PbD – Application Example: 

PbD – Application Example The “Golden Maze”

PbD – Application Example: 

PbD – Application Example Playing Air-Hockey

Slide14: 

PbD – Application Example Box Manipulations

Slide15: 

Three Approaches for PbD Symbolic Control-Based Statistical

Slide16: 

Symbolic Approach for PbD Analyze observed actions in terms of sub-goals Match actions needed to fulfill these sub-goals Create a symbolic description of the environment ( ”object A is above object B” ) Learn a series of symbolic if-then rules ( ”if object A is above object B then grasp-object[ object B ]” )

Example: Symbolic Approach for PbD: 

Example: Symbolic Approach for PbD (Kunyushi et al., 1994) … but how do you symbolically describes “hitting a tennis serve”?

Slide18: 

Control-Based Approach for PbD No symbolic parsing of perceived actions Assume a pre-defined control policy Acquire needed parameters from observation

Slide19: 

Control-Based Approach Inverse Models Sometime assume known inverse models (converting desired effect to needed commands)

Example: Control-Based Approach for PbD: 

Example: Control-Based Approach for PbD (Schaal, 2003) Tennis movie

Statistical Approach for PbD: 

Statistical Approach for PbD No prior assumption on used control policy Statistically match perception and action Can this be done? More on this later…

Example: Statistical Approach for PbD: 

Example: Statistical Approach for PbD (Asada, 1995)

Slide23: 

Example: Statistical Approach for PbD Learning: Perform random action A(i) Record resulted optical flow f(i) Compute principal-component p1(i), p2(i) Learn the connection A(i) – {p1(i), p2(i)}

Outline: 

Outline 2. Programming By Demonstration 1. Movement Imitation 3. Robotic Movement Imitation Primitives Based Approach (Mataric’) Real Time Tracking (“mirror-game”) (Ude et al.) 4. Direct Perception and Imitation

PbD for Movement ImitationPre-Cursor 1: Cartoons Retargeting : 

PbD for Movement Imitation Pre-Cursor 1: Cartoons Retargeting (Bregler et al., 2002)

Slide26: 

Cartoons Retargeting Two Types of Deformations Affine deformation Key shape deformation

Slide27: 

Cartoons Retargeting Affine Deformations Affine parameters

Slide28: 

Cartoons Retargeting Affine Deformations

Slide29: 

Cartoons Retargeting Key Shape Deformations Sk are the key shapes

Slide30: 

Cartoons Retargeting Key Shape Deformations

Slide31: 

Cartoons Retargeting - Results More on: http://www.cs.weizmann.ac.il/~hassner/cv03/ “Animating human motion”, Speakers : Simon Adar, Yoram Atir

PbD for Movement ImitationPre-Cursor 2: Guided Movement Synthesis: 

PbD for Movement Imitation Pre-Cursor 2: Guided Movement Synthesis (Zelnik-Manor, Hassner & Irani, 2004)

Event-Based Analysis of Video: 

Event-Based Analysis of Video (Zelnik-Manor & Irani, 2001)

Guided Movement Synthesis(a.k.a. “Movement Imitation”?): 

Guided Movement Synthesis (a.k.a. “Movement Imitation”?)

Slide35: 

PbD for Movement Imitation Pre-Cursor 2: Movement Synthesis

PbD for Movement Imitation Case Study: Primitive-Based Approach : 

PbD for Movement Imitation Case Study: Primitive-Based Approach The Problem: How to convert visual input to motor output? A Possible Solution: Use a common, sparse representation: sensory-motor primitives. … but what primitives to use?

Movement Imitation Using Sensory-Motor Primitives: 

Movement Imitation Using Sensory-Motor Primitives Motor primitives: Sequences of action that accomplish a complete goal-directed behavior. Examples: 1. Move hand in “straight line”, “parabola” (Felix…). 2. Perform “grasping”, “a tennis serve”.

Slide38: 

Imitation Learning Using Sensory-Motor Primitives (Schaal, Ijspeert & Billard, 2003)

Slide39: 

Inspiration for Using Sensory-Motor Primitives (Rizzolatti et al., 2002; Gallese et al. 1996) Evidence for: Coding of goal-directed actions. Shared representations of perception and action. Example – Mirror Neurons.

Movement Imitation Using Sensory-Motor Primitives: 

Movement Imitation Using Sensory-Motor Primitives (Mataric’,1998) General Principles: Selective attention focusing on end-points movements. Sensory-motor primitives as integrative representation. Learning new skills as compositions of primitives. Experimental test-beds.

What Sensory-Motor Primitives to Use?: 

What Sensory-Motor Primitives to Use? Primitives Innate Pre-defined control policies (e.g., central pattern generators) Learned Un-supervised clustering (using PCA, Isomap ) Joints Space (“motor space”) End-Points Space (“visual space”)

“Experiment in Imitation Using Perceptuo-Motor Primitives”, (Weber, Jenkins & Mataric’,2001): 

“Experiment in Imitation Using Perceptuo-Motor Primitives”, (Weber, Jenkins & Mataric’,2001) Extract hand (end-point) movements. Perform Vector-Quantization to get invariant representation.

Slide43: 

Classify movement to primitives (line, arc, circle). Group adjacent similar primitives. “Experiment in Imitation Using Perceptuo-Motor Primitives”

Slide44: 

Determine primitives parameters. Project to ego-centric space. “Experiment in Imitation Using Perceptuo-Motor Primitives”

Slide45: 

(Weber, Jenkins & Mataric’,2001) “Experiment in Imitation Using Perceptuo-Motor Primitives”

PbD for Movement Imitation Case Study: Real-Time Tracker: 

PbD for Movement Imitation Case Study: Real-Time Tracker (Ude et al.,2001) The Goal: Mimic movements in real-time The Problem: Large amount of data to process (6 MB/Sec) Need “continuous success” The Solution: Probabilistic approach to prevent excessive data interactions

“Real-Time Visual System for Interaction with Humanoid Robot” (Ude, Shibata & Atkeson, 2001): 

“Real-Time Visual System for Interaction with Humanoid Robot” (Ude, Shibata & Atkeson, 2001) Estimate positions of tracked “blobs” in the image Compute 3D coordinates of tracked objects using stereo Transform into via-points for robot hand trajectory Compute motor commands from desired trajectory

Real-Time Tracker Tracking “Blobs” In a Bayesian Setting : 

Real-Time Tracker Tracking “Blobs” In a Bayesian Setting probability for the pixel at location u to have Intensity Iu Given the process k a-priori probability for process k

Real-Time TrackingMinimize Log-Likelihood: 

Real-Time Tracking Minimize Log-Likelihood overall probability to observe image I Goal: determine the parameters that are most likely to produce this image – Maximal Likelihood Problem. computationally easier to minimize the negative log likelihood

Real-Time TrackingMinimize Log-Likelihood: 

Find minimum (using Lagrange Multipliers) and get: probability that pixel u stems from process l Real-Time Tracking Minimize Log-Likelihood

Slide51: 

Real-Time Tracking Find Probabilities Parameters The above equations are solved iteratively by the Expectation-Minimization (EM) algorithm Expectation stage: compute Pu,l using the current estimate for Ө and ω. Minimization stage: compute new Ө and ω assuming Pu,l are constant. from probabilities of pixels to belong to a certain process (e.g. – the human hand) …

“Real-Time Visual System for Interaction with Humanoid Robot”: 

“Real-Time Visual System for Interaction with Humanoid Robot” … to object locations

Real-Time TrackingGeneral Stages: 

Real-Time Tracking General Stages Estimate positions of tracked “blobs” in the image Compute 3D coordinates of tracked objects using stereo Transform into via-points for robot hand trajectory Compute motor commands from desired trajectory

Real-Time TrackingEstimate Trajectories with B-splines: 

Real-Time Tracking Estimate Trajectories with B-splines

Real-Time Tracking - Results: 

Real-Time Tracking - Results Robot Compliance Movie

References: 

References “Vision-Based Robot Learning for Behavior Acquisition” M. Asada, T. Nakamura, and K. Hosoda. Proc. of IEEE International Conference on Intelligent Robots And Systems 1995 (IROS '95) Workshop on Vision for Robots, pp.110-115, 1995. “Turning to the masters: Motion capturing cartoons” Bregler C, Loeb L, Chuang E, Deshpande H ACM TRANSACTIONS ON GRAPHICS 21 (3): 399-407 JUL 2002 “Movement, activity and action: The role of knowledge in the perception of motion” Bobick AF PHILOSOPHICAL TRANSACTIONS OF THE ROYAL SOCIETY OF LONDON SERIES B-BIOLOGICAL SCIENCES 352 (1358): 1257-1265 AUG 29 1997 “Challenges in Building Robots That Imitate People”, Breazeal C. and Scassellati B, in "Imitation in Animals and Artifacts", Kerstin Dautenhahn and Chrystopher Nehaniv, eds. The MIT Press, 2002. “Action recognition in the premotor cortex” Gallese V, Fadiga L, Fogassi L, Rizzolatti G BRAIN , 119: 593-609 Part 2 APR 1996 “Learning by watching - extracting reusable task knowledge from visual observation of human-performance” Kuniyoshi Y, Inaba M, Inoue H IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, 10 (6): 799-822 DEC 1994 “Sensory-Motor Primitives as a Basis for Learning by Imitation: Linking Perception to Action and Biology to Robotics.” Maja J Mataric, in "Imitation in Animals and Artifacts", Kerstin Dautenhahn and Chrystopher Nehaniv, eds., MIT Press, 2002, 392-422

References: 

References “From mirror neurons to imitation: facts and speculations”, Rizzolatti G, Fadiga L, Fogassi L and Gallese V, in: Meltzoff AN and Prinz W (Eds.) "The imitative mind: development, evolution, and brain bases", New York: Cambridge University Press, 2002 “Computational approaches to motor learning by imitation” Schaal S, Ijspeert A, Billard A PHILOSOPHICAL TRANSACTIONS OF THE ROYAL SOCIETY OF LONDON SERIES B-BIOLOGICAL SCIENCES, 358 (1431): 537-547 MAR 29 2003 “Movement planning and imitation by shaping nonlinear attractors” Schaal S, PROCEEDINGS OF THE 12TH YALE WORKSHOP ON ADAPTIVE AND LEARNING SYSTEMS 2003 “Robots that imitate humans” Scassellati B. Breazeal C. Trends in Cognitive Science, 6(11):481 487, November 2002. “Real-time visual system for interaction with a humanoid robot”, Ude A., Shibata T. and Atkeson C. G., Robotics and Autonomous Systems, 37:115 125, 2001. Stefan Weber, Odest C. Jenkins, and Maja J. Mataric´. "Imitation Using Perceptual and Motor Primitives". In International Conference on Autonomous Agents, pages 136-137, Barcelona, Spain, Jun 2000 "Event-Based Analysis of Video “, Zelnik-Manor L. and  Irani M., IEEE CONFERENCE ON COMPUTER VISION AND  PATTERN RECOGNITION, December 2001 (CVPR'01).

Slide58: 

“The great end of life is not knowledge but action.” (Thomas H. Huxley) Perception? Action?