arv1 introduction

CIS 849:Autonomous Robot Vision: 

CIS 849: Autonomous Robot Vision Instructor: Christopher Rasmussen Course web page: www.cis.udel.edu/~cer/arv September 5, 2002

Purpose of this Course: 

Purpose of this Course To provide an introduction to the uses of visual sensing for mobile robotic tasks, and a survey of the mathematical and algorithmic problems that recur in its application

What are “Autonomous Robots”?: 

What are “Autonomous Robots”? Mobile machines with power, sensing, and computing on-board Environments Land (on and under) Water (ditto) Air Space ???

What Can/Will Robots Do?: 

What Can/Will Robots Do? Near-term: What People Want Tool analogy Never too far from human intervention, whether physically or via tele-operation Narrow tasks, limited skills “3-D”: Dirty, Dangerous, and Dull jobs

What Can/Will Robots Do? Task Areas: 

What Can/Will Robots Do? Task Areas Industry Transportation & Surveillance Search & Science Service

What Might Robots Do?: 

What Might Robots Do? Long-term: What They Want “Mechanical animal” analogy may become appropriate Science fiction paradigm On their own Self-directed generalists

Industry: 

Industry Ground coverage Harvesting, lawn-mowing (CMU) Snow removal Mine detection Inspection of other topologies MAKRO (Fraunhofer): Sewer pipes CIMP (CMU): Aircraft skin MAKRO CIMP

CMU Demeter: 

CMU Demeter

Transportation & Surveillance: Ground: 

Transportation & Surveillance: Ground Indoors Clodbusters (Penn) Many others Highways, city streets VaMoRs/VaMP (UBM) NAVLAB/RALPH (CMU) StereoDrive (Berkeley) Off-road Ranger (CMU) Demo III (NIST, et al.) VaMoRs Ranger

Penn Clodbuster: 

Penn Clodbuster Obstacle avoidance with omnidirectional camera

UBM VaMoRs: 

UBM VaMoRs Detecting a ditch with stereo, then stopping

Transportation & Surveillance: Air: 

Transportation & Surveillance: Air Fixed wing (UBM, Florida) Helicopters (CMU, Berkeley, USC, Linkoping) Blimp (IST, Penn) Florida MAV UBM autonomous landing aircraft

USC Avatar: 

USC Avatar Landing on target (mostly)

Search & Science: 

Search & Science Urban Search & Rescue Debris, stairs Combination of autonomy & tele-operation Hazardous data collection Dante II (CMU) Sojourner (NASA) Narval (IST) Dante II Sojourner Narval

USF at the WTC: 

USF at the WTC Urbot & Packbot reconnoiter surrounding structures courtesy of CRASAR

Service: 

Service Grace (CMU, Swarthmore, et al.): “Attended” AI conference Register, interact with other participants Navigate halls, ride elevator Guides Polly (MIT): AI lab Minerva (CMU): Museum Personal assistants Nursebot (CMU): Eldercare Robotic wheelchairs Grace

CMU Minerva: 

CMU Minerva In the Smithsonian

What Skills Do Robots Need?: 

What Skills Do Robots Need? Identification: What/who is that? Object detection, recognition Movement: How do I move safely? Obstacle avoidance, homing Manipulation: How do I change that? Interacting with objects/environment Navigation: Where am I? Mapping, localization

Why Vision?: 

Why Vision? Pluses Rich stream of complex information about the environment Primary human sense Good cameras are fairly cheap Passive  stealthy Minuses Line of sight only Passive  Dependent on ambient illumination

Aren’t There Other Important Senses?: 

Aren’t There Other Important Senses? Yes— The rest of the human “big five” (hearing, touch, taste, smell) Temperature, acceleration, GPS, etc. Active sensing: Sonar, ladar, radar But… Mathematically, many other sensing problems have close visual correlates

The Vision Problem: 

The Vision Problem

Computer Vision Outline: 

Computer Vision Outline Image formation Image processing Motion & Estimation Classification

Outline: Image Formation: 

Outline: Image Formation 3-D geometry Physics of light Camera properties Focal length Distortion Sampling issues Spatial Temporal

Outline: Image Processing: 

Outline: Image Processing Filtering Edge Color Shape Texture Feature detection Pattern comparison

Outline: Motion & Estimation: 

Outline: Motion & Estimation Computing temporal image change Magnitude Direction Fitting parameters to data Static Dynamic (e.g., tracking) Applications Motion Compensation Structure from Motion

Outline: Classification: 

Outline: Classification Categorization Assignment to known groups Clustering Inference of group existence from data Special case: Segmentation

Visual Skills: Identification: 

Visual Skills: Identification Recognizing face/body/structure: Who/what do I see? Use shape, color, pattern, other static attributes to distinguish from background, other hypotheses Gesture/activity: What is it doing? From low-level motion detection & tracking to categorizing high-level temporal patterns Feedback between static and dynamic

Minerva Face Detection: 

Minerva Face Detection Finding people to interact with

Penn MARS project: 

Penn MARS project Airborne, color-based tracking Blimp, Clodbusters

Visual Skills: Movement: 

Visual Skills: Movement Steering, foot placement or landing spot for entire vehicle MAKRO sewer shape pattern Demeter region boundary detection

Florida Micro Air Vehicle (MAV): 

Florida Micro Air Vehicle (MAV) Horizon detection for self-stabilization

UBM Lane & vehicle tracking (with radar): 

UBM Lane & vehicle tracking (with radar)

Visual Skills: Manipulation: 

Visual Skills: Manipulation Moving other things Grasping: Door opener (KTH) Pushing, digging, cranes Clodbusters push a box cooperatively KTH robot & typical handle

Visual Skills: Navigation: 

Visual Skills: Navigation Building a map [show “3D.avi”] Localization/place recognition Where are you in the map? Minerva’s ceiling map Laser-based wall map (CMU)

Course Prerequisites: 

Course Prerequisites Strong background in/comfort with: Linear algebra Multi-variable calculus Statistics, probability Ability to program in: C/C++, Matlab, or equivalent

Course Details: 

Course Details First 1/3 of classes: Computer vision review by professor Last 2/3 of classes: Paper presentations, discussions led by students One major programming project Grading 10%: Two small programming assignments 30%: Two oral paper presentations + write-ups 10%: Class participation 50%: Project

Readings: 

Readings All readings will be available online as PDF files Textbook: Selected chapters from pre-publication draft of Computer Vision: A Modern Approach, by D. Forsyth and J. Ponce Web page has other online vision resources Papers: Recent conference and journal articles spanning a range of robot types, tasks, and visual algorithms

Presentations: 

Presentations Each student will submit short written analyses of two papers, get feedback, then present them orally Non-presenting students should read papers ahead of time and have some questions prepared. I will have questions, too :)

Project: 

Project Opportunity to implement, test, or extend a robot-related visual algorithm Project proposal due in October; discuss with me beforehand Data I will provide “canned” data, or gather your own We will have a small wheeled robot to use for algorithms requiring live feedback Due Wednesday, November 27 (just before Thanksgiving break)

More questions?: 

More questions? Everything should be on the web page: www.cis.udel.edu/~cer/arv or e-mail me at cer@cis.udel.edu