Computer Vision and Media Group:Selected Previous Work: Computer Vision and Media Group: Selected Previous Work David Gibson, Neill Campbell Colin Dalton Department of Computer Science University of Bristol


Duck: The AutomaticGeneration of 3D Models: Duck: The Automatic Generation of 3D Models Generating 3D computer models is difficult Put object on turntable Take 8 pictures of it from different angles Crank the handle… No skilled user or expensive equipment Make avatars by spinning person on chair


Cog and Stepper: Cog and Stepper Automatically inject ‘life’ into computer animations 3D swathe through 4D space time Where space is 3D computer model Or just to make things look strange!


Casablanca: Motion Ripper: Casablanca: Motion Ripper Computer animation driven by film Animator labels a small number of points System then tracks these points over all frames Motions are extracted and used to drive animation


Laughing ManMotion Ripper Part 2: Laughing Man Motion Ripper Part 2 Automatic video creation Points are marked and tracked System learns the motions System generates new motions which are different but ‘correct’ Forever!


Slide11: AutoArch: The Automatic Archiving of Wildlife Film Footage David Gibson, Neill Campbell David Tweed, Sarah Porter Department of Computer Science University of Bristol


Motivation: Motivation BBC Natural History Unit Manual archiving/meta data generation Reuse problematic Inefficient/time consuming Expensive Limited access Obvious need to automate


Objectives: Objectives Generate efficient visual representations Video segmentation Visual browsing/summarisation Visual searching Generate as much meta data automatically Camera motions/effects Scene structure Scene content


System Overview: System Overview


Video Segmentation: Video Segmentation


Visual Summarisation: Visual Summarisation Key frame extraction


Visual Summarisation Tree: Visual Summarisation Tree Level of detail


Visual Searching: Visual Searching Layered 2D representation of high D clip space


Motion Analysis using point tracking: Motion Analysis using point tracking Camera Motion Estimation Event/Area of Interest Detection Gait Analysis Foreground/Background Separation Combine with Colour and Texture for Classification See cheetah track avi


Camera Pan: Camera Pan BCD0111.09_0085.eps lines = 47, curls = 98, shorts = 5 long lines = 47, mode = 95.00, mean = 95.21, std = 4.15 zoom centre = (603.01, 63.65), val = -0.2356 zoom residual per line = 22.92 zoom residual #2 per line = 28.92 Average line vector: 109.94 -8.27 pan/tilt angle: 94.30, vector: (109.94 -8.27) pan/tilt residual per line = 21.67 pan/tilt residual #2 per line = 33.38 percentage of lines within 5% of mode: 89.36


Camera Zoom: Camera Zoom BCD0113.15_0067.eps lines = 142, curls = 1, shorts = 7 long lines = 134, mode = 340.00, mean = 227.24, std = 128.76 zoom centre = (182.97, 55.52), val = 0.2063 zoom residual per line = 4.86 zoom residual #2 per line = 6.90 Average line vector: -3.81 17.28 pan/tilt angle: 347.57, vector: (-3.81 17.28) pan/tilt residual per line = 13.85 pan/tilt residual #2 per line = 16.13 percentage of lines within 5% of mode: 17.16


Tracking Failure: Tracking Failure This could be an interesting event in its self: flocking, herding, close up of lots of activity, shot grouping, etc.


Event/Area of InterestDetection: Event/Area of Interest Detection


Frequency Analysis:Gait Detection: Frequency Analysis: Gait Detection FFT After trajectory segmentation


Foreground/BackgroundExtraction: Foreground/Background Extraction Which pixels are foreground?


Animal Identification: Animal Identification Give models a name: = cheetah = elephant = zebra = lion


Some Problems: Some Problems Noise in images Noise in measurements Camouflage Occlusion Answer: Need higher level models See next few slides


Model Based Tracking: Model Based Tracking


Lion Tracking: Lion Tracking Synchronise horse model with lion points Move and deform horse model to lion points See avi To do: Improve spatial deformation, especially for legs, using colour and texture


Multiple Object Tracking: Multiple Object Tracking


Conclusions: Conclusions Visualisation is very powerful Combined with text is even better! Assists searching and communication Lots of meta data can be auto generated Assists archiving Help to prioritise manual archiving Can be applied to any visual media