logging in or signing up TaoYu Herminia Download Post to : URL : Related Presentations : Share Add to Flag Embed Email Send to Blogs and Networks Add to Channel Uploaded from authorPOINTLite Insert YouTube videos in PowerPont slides with aS Desktop Copy embed code: Embed: Flash iPad Dynamic Copy Does not support media & animations Automatically changes to Flash or non-Flash embed WordPress Embed Customize Embed URL: Copy Thumbnail: Copy The presentation is successfully added In Your Favorites. Views: 467 Category: Education License: All Rights Reserved Like it (0) Dislike it (0) Added: February 27, 2008 This Presentation is Public Favorites: 0 Presentation Description No description available. Comments Posting comment... Premium member Presentation Transcript Motion Capture (Mocap) and Motion Data Related Technologies : Motion Capture (Mocap) and Motion Data Related Technologies Tao Yu Comp 790-058 – Robot Motion Planning Fall 2007 September 24, 2007OUTLINE: OUTLINE Mocap Overview Representation of Motion Motion Signal Processing Motion Synthesis from Examples What is motion capture: What is motion capture “Recording of motion for immediate or detailed analysis and playback” David J Sturman, “A Brief History of Motion Capture for Computer Character Animation”, Character Motion Systems, SIGGRAPH 94, Course 9 “The creation of a 3d representation of a live performance” Alberto Menache, Understanding Motion Capture for Computer Animation and Video Games Mocap OverviewRoot of motion capture: Root of motion capture Eadweard Muybridge (1830-1904) Etienne-Jules Marey (1830-1904) Harold Edgerton (1903-1990) Mocap OverviewEadweard Muybridge (1830-1904): Eadweard Muybridge (1830-1904) “Father of the motion picture” several cameras – successive pictures photographs of human and animal motion zoopraxiscope (zoogyroscope, zoetrope) – a device for playing still images in sequence http://www.cotianet.com.br/photo/great/Muybridge.htm http://en.wikipedia.org/wiki/Image:The_Horse_in_Motion.jpg Mocap OverviewEtienne-Jules Marey (1830-1904): Etienne-Jules Marey (1830-1904) birds photographic gun http://www.nrw-forum.de/img_ausst/img_press/Marey.jpg http://www.inrp.fr/Tecne/Acexosp/Actimage/Images/Marey2.jpg http://www.rickwisedp.com/St%20Marys/COMM%20158/images/Marey%20Photo%20Gun.jpg Mocap OverviewHarold Edgerton (1903-1990): Harold Edgerton (1903-1990) high speed and stop motion photography exposures as small as a millionth of a second electronic flash stroboscope http://www.personal.psu.edu/users/a/r/ark176/Assignment%204.htm http://www.personal.psu.edu/users/a/r/ark176/Assignment%204.htm Mocap OverviewRotoscoping: Rotoscoping Allowed animators to trace cartoon characters over photographed frames of live performances. Invented in 1915 by Max Fleischer Koko the Clown Snow White Mocap OverviewRotoscoping: Rotoscoping “rotoscoping can be thought of as a primitive form or precursoro to motion capture, where the motion is ‘captured’ painstakingly by hand.” - Sturman Mocap Overview “modern” era of mocap, 1970’s-present: “modern” era of mocap, 1970’s-present more players commercial players multiple uses 70’s: development of magnetic systems 80’s: development of optical systems 90’s: mocap is hot, 00’s: mocap is used more frequently for feature films Mocap OverviewOverview: Motion capture systems: Overview: Motion capture systems Types of Mocap systems: Outside-In sources (e.g., reflective markers) on body external sensors (e.g., cameras) optical systems Inside-Out sensors on body external sources magnetic systems Inside-In sources and sensor on body mechanical systems Mocap Overviewimplementation of a motion capture system: implementation of a motion capture system prosthetic acoustic magnetic optical Liverman, The Animator’s Motion Capture Guide Mocap Overviewmechanical/prosthetic capture: mechanical/prosthetic capture Inside-In external structure attached to performer structure detects changes optic mechanical Mocap Overviewmechanical/prosthetic capture: mechanical/prosthetic capture advantages computes rotations directly portable, unlimited range less expensive can capture multiple performances simultaneously no built in positional reference disadvantages external structure unwieldy cannot change the configuration, i.e., a hand capture can’t be used for an arm Mocap Overviewacoustic capture: acoustic capture Inside-Out still developing Active Markers – transmitters are attached to the performer and sequentially emit audio signal, a “click” receivers measure the time to receive the signal, triangulate and compute the position of the transmitter advantages no occlusion disadvantages cables unwieldy rate of transmission not high enough to support enough transmitters size of the capture area is limited sound reflections can reduce accuracy Mocap Overviewoptical capture (passive): optical capture (passive) markers are attached to a performer passive reflective markers more on active markers later a system of cameras record the position of the markers 2-many cameras http://cgm.cs.mcgill.ca/~athens/cs644/Projects/2004/MichelLanglois-PhilippeKunzle/MoCap1.jpg Mocap Overviewoptical capture (passive): optical capture (passive) advantages freedom of movement high quality capture high throughput fast sampling (200 fps at a high resolution) can capture fast motions can have a large capture space can capture many markers disadvantages occlusion, markers are can be hidden from the camera additional performers will increase occlusion may be able to add redundant cameras marker crossover, which marker are you looking at? cost $$$ extensive post processing (the marker’s have to be located and identified) Mocap Overviewmajor optical players: major optical players Motion Analysis http://www.motionanalysis.com/ Films Lord of the Rings King Kong Matrix Final Fantasy Games NBA Live 2004 Grand Theft Auto III Mortal Kombat 4 (Midway) ViconPeak http://www.viconpeak.com/ Films Polar Express Harry Potter and the Prisoner of Azkaban The Hulk Spider Man Games All-Star Baseball 2002 Buffy the Vampire Slayer Everquest II NHL 2K3 (Mocap by Red Eye Studio) Mocap Overviewmagnetic capture: magnetic capture Outside/In receivers are attached to the performer’s body compute the position and orientation from receiver to central magnetic transmitter AC and DC systems originally developed for targeting in tactical military aircraft advantages no occlusion disadvantages cabling can interfere with movement (improvements?) capture area can be limited by transmitter metal in vicinity can interfere with system capture volume can be limited Mocap OverviewMocap pipeline: Mocap pipeline Planning for capture Setup and calibrate system Capture performer, obtain marker positions Retarget to a character Edit/cleanup Mocap Overviewplanning: planning motion capture requires serious planning anticipate how the data will be used garbage in garbage out shot list games motions need to be able to blend into one an another capture base motions and transitions which motions transition into which other transitions cycles/loops Mocap Overviewmovement flowchart for games: movement flowchart for games Planning and Directing Motion Capture For Games By Melianthe Kines Gamasutra January 19, 2000 URL: http://www.gamasutra.com/features/20000119/kines_01.htm Mocap Overviewprocessing passive markers: processing passive markers each camera records capture session extraction: markers need to be identified in the image determines 2d position problem: occlusion, marker is not seen use more cameras markers need to be labeled which marker is which? problem: crossover, markers exchange labels may require user intervention compute 3d position: if a marker is seen by at least 2 cameras then its position in 3d space can be determined http://www.xbox.com/NR/rdonlyres/3164D1BE-C1C4-46A1-90F0-26507CF2C9BD/0/ilmnflfever2003lightscam001.jpg Mocap Overviewretargeting: the character: retargeting: the character the character is controlled by skeleton to control the skeleton, need to specify joint rotations muscles? Mocap Overviewretargeting: retargeting capture motion on performer positions of markers are recorded retarget motion on a virtual character motion is usually applied to a skeleton a skeleton is hierarchical linked joints need rotation data! need to convert positions to rotations Mocap Overviewperformer→ actor → character: performer→ actor → character the actor is used to convert marker positions to rotational data markers are handles on the actor actor should have similar proportions as the performer joint rotations of the actor are applied to the character there are still issues with proportions Alias Motionbuilder: actor and markers Mocap Overviewretargeting problems: retargeting problems Mocap Overviewretargeting problems: retargeting problems Mocap OverviewApplications: Applications Entertainment Medicine Arts / Education Science / Engineering Mocap OverviewEntertainment: Live Action Films: Entertainment: Live Action Films Computer generated characters in live action films (e.g. Battle Droids and many others in Star Wars Prequels, Gullum in The Lord of the Rings, King Kong in King Kong , Davy Jones in Pirates of Caribbean) Mocap OverviewEntertainment: 3D computer animations: Entertainment: 3D computer animations Characters in computer animated files (e.g. Polar Express, Monster House) Mocap OverviewEntertainment: Video Games: Entertainment: Video Games Video games by Electronic Arts, Gremlin, id, RARE, Square, Konami, Namco, and others, (e.g. Enemy Territory, Devil May Cry) Mocap OverviewMedicine: Medicine Medicine (e.g., gait analysis, rehabilitation) Sports medicine (e.g. injury prevention, performance analyses, performance enhancement) Gait Analysis Service Mocap OverviewArts / Education: Arts / Education Dance and theatrical performances Archiving (e.g., Marcel Marceau) OSU/ACCAD Mocap OverviewScience / Engineering: Science / Engineering Computer Science (e.g., human motion database, indexing, recognitions) Engineering (e.g., Biped robot developments) Ergonomic product design Military (e.g., field exercises, virtual instructors, and role-playing games) Mocap OverviewAspects of the Problem: Aspects of the Problem “Gross” Body movement NOT: Appearance Models Facial animation Cloth, clothing, secondary movement Hands How to describe moving target – “Character” How to describe movement mathematically Motion RepresentationHuman Model: Human Model Humans are complex! Motion Representation Human motion can be understood at a very fine level of detail!Abstractions: Abstractions Representation of complex human structure with varying degrees of simplification 206 bones, muscles, fat, organs, clothing, … Motion Representation 206 bones, complex joints 53 bones Kinematic jointsStandard simplified modelsof humans: Standard simplified models of humans Small numbers of degrees of freedom for gross motion Articulated figures Rigid pieces Sometimes stretching allowed Kinematic joints Rotations between pieces Motion RepresentationAn example articulated figure : An example articulated figure The skeleton: 17 joints, each with 3 degrees of freedom (DOFs) Root position (3 DOFs) and orientation (3 DOFs) Totally 57 DOFs Motion RepresentationPose representation: Pose representation Initial configurations of articulated figure (Binding pose) Usually T-Pose Rigid pieces undergo rigid transformation Hierarchical representation of the configuration Motion RepresentationComputing positions: Computing positions Each joint i provides a local rotation Ri and a local translation Ti. Concatenating Ri and Ti gives local Mi, the local transform at each joint. The transformation of the point x on joint j is found by concatenating all previous transforms in the hierarchy as follows: Motion RepresentationParameterizing Rotations: Parameterizing Rotations Goal: encode rotations in a vector Rn - > “set of rotations” Give “names” to members of the set of possible rotations Many ways to do this, all flawed No perfect method Use the best one for the job Motion RepresentationGoals for Parameterization: Goals for Parameterization Compact (as few variables as possible) Complete Every rotation can be represented 1-to-1 Every rotation has one value Every value has one rotation Singularity free “close” rotations are “close” in value Motion RepresentationParameterizations ofRotations: Parameterizations of Rotations Rotation Matrices Euler Angles Axis Angle formulation Unit Quaternions Motion RepresentationComparison of body representation: Comparison of body representation Motion RepresentationMotion definition: Motion definition Motion is a function of time Given time, provide a pose m(t): R -> Rn Often represented as samples Sparse samples + interpolation Dense samples (at frames) Motion RepresentationPreliminaries: Preliminaries A motion maps times to configurations m(t): R -> Rn Vector-valued, time-varying signal Motion Signal ProcessingChallenge and solution: Challenge and solution Get a specific motion From capture, keyframe, … Specific character, action, mood, … Want something else But need to preserve original But we don’t know what to preserve Can’t characterize motion well enough Apply signal processing techniques to designing, modifying, and adapting animated motion [Bruderlin & Williams 95][Witkin & Popovic 95] Motion Signal ProcessingManipulating motion: Manipulating motion Manipulate time m(t) = m0( f(t) ) F : R -> R “time warp” Manipulate value m(t) = f(m0(t)) F : Rn-> Rn F : (R, Rn)-> Rn Motion Signal ProcessingMotion blending: Motion blending “Add” two motions together Really interpolate m(t) = a m0(t) + (1-a) m1(t) Note: this is a per-frame operation Interpolation between poses Interpolating root position – Linear interpolation Interpolating joint rotation – SLERP (Spherical Linear intERPolation) Motion Signal ProcessingHow to use blending: How to use blending Interpolate similar motions Be sure to make time correspondence Transition between motions Time varying blend (a=0 -> 1) Over a short period of time A bad pose isn’t such a big deal Avoids discontinuities m(t) = a(t) m0(t) + (1-a(t)) m1(t) Motion Signal ProcessingBlending for transition motion: Blending for transition motion Okan et al. Siggraph ’02 Motion Signal ProcessingTime warping: Time warping Dynamic Time Warping (DTW): Non-linear signal matching procedure originating in field of speech recognition Finding the optimal sample correspondences between the two signals by computing the global minimum value of warping cost function Motion Signal ProcessingTime warping + interpolation: Time warping + interpolation Motion Signal ProcessingDisplacement map: Displacement map A.K.A Motion Warping Motion Signal ProcessingDisplacement map: Displacement map A.K.A Motion Warping Changing the shape of a signal locally through a displacement map while maintaining continuity and preserving the global shape of the signal. Motion Signal ProcessingDisplacement map (cntd.): Displacement map (cntd.) Mapping procedure Motion Signal ProcessingThe Challenge: The Challenge High Quality, Expressive Motion Need motion capture (examples) Flexible, long-running, controllable Need synthesis Synthesis from Examples! Motion Synthesis from ExamplesIdea:Put Clips Together: Idea: Put Clips Together New motions from pieces of old ones! Good news: Keeps the qualities of the original (with care) Can create long and novel “streams” (keep putting clips together) Challenges: How to connect clips? How to decide what clips to connect? Motion Synthesis from ExamplesConnecting clips: transition: Connecting clips: transition Transitions between motions can be hard Simple method: Blends between aligned motions Cleanup footskate artifacts When motions are similar Believe that blending will “work” Heuristic based on geometry Not perfect method Measure and use threshold Apply threshold conservatively Motion Synthesis from ExamplesSimilarity Metric: Similarity Metric Factor out invariances and measure Motion Synthesis from ExamplesMotion Graphs (Kovar et al. ’02): Motion Graphs (Kovar et al. ’02) Start with a database of motions, each with type and constraint information. Goal: add transitions at opportune points. Other Motion Graph-like projects elsewhere Differ in details. Motion Synthesis from ExamplesMotion Graphs: Motion Graphs Idea: automatically add transitions within a motion database Quality: restrict transitions Control: build walks that meet constraints Motion Synthesis from ExamplesUsing motion graph: Using motion graph Any walk on the graph is a valid motion Generate walks to meet goals Random walks (screen savers) Search to meet constraints An example: Motion Synthesis from Examples Given a path Find a motion that minimizes distance Combinatorial optimizationWhy is this good?: Why is this good? Search the graphs for motions Look ahead avoids getting stuck Cleanup motions as generated Plan “around” missing transitions Optimization gets close as possible Motion Synthesis from Examples Not OK for Interactive Apps! Need different tradeoffs You do not have the permission to view this presentation. In order to view it, please contact the author of the presentation.