Drury CHI04 wk shoppres

Improving Human-Robot InteractionJill Drury, The MITRE Corporation : 

Improving Human-Robot Interaction Jill Drury, The MITRE Corporation Collaborators: Holly Yanco, UMass Lowell Jean Scholtz, NIST Mike Baker, Bob Casey, Dan Hestand, Brenden Keyes, Phil Thoren (UML)

Methodology for Evaluating HRI: 

Methodology for Evaluating HRI Two approaches: field and laboratory Field work: so far, robotics competitions See many different user interfaces but have no control over what operator does Difficult to collect data Can see what they did – but there isn’t time to determine why Best used to get an idea of the difficulties in the real world Can identify “critical events” but don’t know for certain whether operator was aware of them

Methodology for Evaluating HRI: 

Methodology for Evaluating HRI Laboratory studies Take what we learned in the real world and isolate factors to determine effects Repeatability is still difficult to achieve due to fragile nature of robots

Analysis Frameworks: 

Analysis Frameworks Taxonomy to define human/robot system Detailed definition of human-robot interaction awareness Coding scheme/metrics for analyzing data Scholtz’ evaluation guidelines

Taxonomy: 

Taxonomy Autonomy Amount of intervention Human-robot ratio Level of shared interaction Composition of robot teams Available sensors Sensor Fusion Criticality Time Space

Awareness as a concept from CSCW: 

Awareness as a concept from CSCW CSCW software: “...makes the user aware that he is part of a group, while most other software seeks to hide and protect users from each other” [Lynch et al. 1990] HRI software: Makes humans aware of robots’ status and activities via the interface

What is “awareness”?: 

What is “awareness”? No standard definition in the CSCW field We’ve seen at least 16 different definitions! Are many different types of awareness, e.g. Concept awareness Conversational awareness Group-structural awareness Informal awareness Peripheral awareness Common thread: understanding that participants have of each other in a shared environment Situation awareness Social awareness Task awareness Workspace awareness

But CSCW systems are different from robotic systems…: 

But CSCW systems are different from robotic systems… CSCW: Multiple humans interacting via a CSCW system Robotics: Single or multiple humans interacting with a single or multiple robots Non-symmetrical relationships between humans and robots; e.g., differences in Free will Cognition

Tailoring an awareness definition for HRI: a base case: 

Tailoring an awareness definition for HRI: a base case Given one human and one robot... ... HRI awareness is the understanding that the human has of the location, activities, status, and surroundings of the robot; and the knowledge that the robot has of the human’s commands necessary to direct its activities and the constraints under which it must operate

An awareness framework: General case: 

An awareness framework: General case Given n humans and m robots working together on a synchronous task, HRI awareness consists of five components: Human-robot awareness Human-human awareness Robot-human awareness Robot-robot awareness Humans’ overall mission awareness

General case: a detailed look: 

General case: a detailed look Given n humans and m robots working together on a synchronous task, HRI awareness consists of five components: Human-robot: the understanding that the humans have of the locations, identities, activities, status and surroundings of the robots. Further, the understanding of the certainty with which humans know this information. Human-human: the understanding that the humans have of the locations, identities and activities of their fellow human collaborators

General case, concluded: 

General case, concluded Robot-human: the robots’ knowledge of the humans’ commands needed to direct activities and any human-delineated constraints that may require command noncompliance or a modified course of action Robot-robot: the knowledge that the robots have of the commands given to them, if any, by other robots, the tactical plans of the other robots, and the robot-to-robot coordination necessary to dynamically reallocate tasks among robots if necessary. Humans’ overall mission awareness: the humans’ understanding of the overall goals of the joint human-robot activities and the measurement of the moment-by-moment progress obtained against the goals.

Coding Scheme: Problems Relating to Critical Incidents: 

Coding Scheme: Problems Relating to Critical Incidents Critical incident: Robot has, or could, cause harm or damage Types of problems: Local navigation Global navigation Obstacle encounter Vehicle state Victim identification (specific to search and rescue)

Some Metrics for HRI: 

Some Metrics for HRI Time spent navigating, on UI overhead and avoiding obstacles Amount of space covered Number of victims found Critical incidents Positive outcomes Negative outcomes Operator interventions Amount of time robot needs help Time to acquire situation awareness Reason for intervention

Scholtz’ Guidelines (tailored): 

Scholtz’ Guidelines (tailored) Is sufficient status and robot location information available so that the operator knows the robot is operating correctly and avoiding obstacles? Is the information coming from the robots presented in a manner that minimizes operator memory load, including the amount of information fusion that needs to be performed in the operators’ heads? Are the means of interaction provided by the interface efficient and effective for the human and the robot (e.g., are shortcuts provided for the human)? Does the interface support the operator directing the actions of more than one robot simultaneously? Will the interface design allow for adding more sensors and more autonomy?

Design Guidelines: 

Design Guidelines Enhance awareness Provide a map of where the robot has been Provide more spatial information about the robot in the environment to make the operators more award of their robot’s immediate surroundings Lower cognitive load Provide fused sensor information to avoid making the user fuse data mentally Display important information near or fused with the video image

Design Guidelines, concluded: 

Design Guidelines, concluded Increase efficiency Provide user interfaces that support multiple robots in a single window, if possible In general, minimize the use of multiple windows and maximize use of the primary viewing area Provide help in choosing robot modality Give the operator assistance in determining the most appropriate level of robot autonomy at any given time

Fusing Information: 

Fusing Information Victims can be missed in video images

Fusing Infrared and Color Video: 

Fusing Infrared and Color Video

Fusing Infrared and Color Video: 

Fusing Infrared and Color Video

Fusing Infrared and Color Video: 

Fusing Infrared and Color Video

Other Sensor Modalities for USAR: 

Other Sensor Modalities for USAR CO2 detection Audio

Overlay of four sensor modalities: 

Overlay of four sensor modalities

Overlay of four sensor modalities: 

Overlay of four sensor modalities