McFarlane 063003

Engaging Innate Human Cognitive Capabilities to Coordinate Human Interruption in Human-Computer Interaction: The HAIL System: 

Engaging Innate Human Cognitive Capabilities to Coordinate Human Interruption in Human-Computer Interaction: The HAIL System June 30, 2003 Dr. Daniel C. McFarlane Lockheed Martin Advanced Technology Laboratories mcfarlane@acm.org 856-792-9708

Table of Contents: 

Table of Contents HAIL Overview Cognitive System Research Cognitive System Development R&D Process Summary

HAIL Overview: 

HAIL Overview What is HAIL? Technology Overview

What is HAIL?: 

What is HAIL? Human Alerting and Interruption Logistics alert mediation technology HAIL has two parts: The HAIL Engine: A cognitive system that delivers cognition-based resources for human alerting activities based on proven empirical human-subjects research The HAIL System: An open architecture decision-support system (DSS) component built around the HAIL Engine that delivers the capabilities of the HAIL Engine for human decision-makers within their work environments The first transition of the HAIL technology is into the US Navy’s Aegis Weapon System The HAIL technology has two parts: (1) A cognitive system as a core HAIL Engine; and (2) A decision-support system (DSS) that realizes the capabilities of the HAIL Engine for human users in real world work environments.

What is HAIL? (cont.): 

What is HAIL? (cont.) Support for increased human capability: (1) Reduce human interruption rate by decoupling the alert rate from human interruption rate The HAIL solution is generic and does not address the platform-specific issue of reducing alert rate overall (2) Increase human control of information environment with negotiation-based coordination of interruption events (3) Increase operator access into past information with context recovery aids HAIL is a platform-independent software component that actualizes the accepted NRL NCARAI alerting research findings for improving operator performance in the presence of alerts.

Technology Overview (cont.): 

Technology Overview (cont.) The HAIL Engine resides at the Business Tier in an N-Tier Architecture. It encapsulates proven business logic on how humans succeed at making effective decisions in alerting-dense work environments.

Technical Overview (cont.)Rational Unified Process (RUP)-guided transformation of proven research results into a mature component-based cognitive system component: 

The HAIL software development approach is using the same tools and process as Aegis Open Architecture to minimize transition effort. Technical Overview (cont.) Rational Unified Process (RUP)-guided transformation of proven research results into a mature component-based cognitive system component Founded on Vision of future human workload for interrupt-driven responsibilities Interdisciplinary NRL research Documented requirements from multiple domains Process RUP-guided (Rational Unified Process) Iterative spiral software development User-centric UI prototyping and usability testing HAIL Alert Engine – A Reusable Component-Based Open System Approach OO, Open Architecture, User-Centric, Modular, Extensible, Reusable, Well Documented, Robust, Scalable

Cognitive Systems Research: 

Cognitive Systems Research Vision Problem Description Approaches Examples Research Foundation Human-Subjects Experiment

Vision of Future Human Workload for Interrupt-Driven Responsibilities : 

Vision of Future Human Workload for Interrupt-Driven Responsibilities “Future human-computer interface will be rooted in delegation, not the vernacular of direct manipulation.” (Nicholas Negroponte, “Being Digital,” 1995, p. 101) Increased alerting requirement identified through experience in applied research on intelligent command and control systems at NRL ISART (Interactive Situation Assessment and Rollup Tool)

Problem Description : 

Problem Description Success of tactical operations depends on high quality decision-making. However, this is not possible without better alerting technology The problem has four parts: (1) People have cognitive limitations (2) No decision-support for management of context switching in current alerting mechanisms (3) Alerting mechanism delivers a large quantity of critical information but its current human interruption rate outpaces operators’ ability to keep up (4) Information overload continues to worsen There is a threshold for people’s ability to handle interruptions without support, and when this threshold is exceeded, situational awareness fails and decision quality drops People have limited ability to handle interruptions without support.

Approaches: 

Approaches Remove Person Training Incentives Selection of Personnel Construction of a Cognitive System for a DSS

Example Domain – Commercial Air: 

Example Domain – Commercial Air National Transportation Safety Board, 1988 Northwest Airlines Detroit Metropolitan Airport McDonnell-Douglas DC-9-61 Preflight checklist interrupted by Air Traffic Control Change in taxi-way instructions and weather alert for possible windshear Never resumed checklist task; Flaps in wrong position for take-off Task resumption aids could have prevented this accident Too many interruptions No operator control No trend information No recovery support Many platforms could be improved with Interruption Support technology.

Example Domain – Aegis Alerting: 

Example Domain – Aegis Alerting

Example Domain – Aegis Alerting: 

Example Domain – Aegis Alerting Too many interruptions No operator control of alert queue No trend information No support for context recovery after interruption Aegis needs the HAIL technology to remain “Fully Mission Capable.”

Research FoundationAll HAIL publications are available on the HAIL-SS web sitehttp://www.atl.lmco.com/projects/hail/: 

Product Development NRL HAIL NASA Other ONR Manning Affordability TADMUS University MSR Research SSC-SD The results of McFarlane’s NRL HAIL Research were published as a two-article stand-alone issue of the journal “Human-Computer Interaction,” 2002, 17(3). HAIL is founded on proven empirical research. Research Foundation All HAIL publications are available on the HAIL-SS web site http://www.atl.lmco.com/projects/hail/ Results Interruption negatively impacts performance Negotiation support for interruptions gives biggest overall win except when an alert task is time-critical People perform better when primed for interruption People switch contexts more easily when given context recovery support

HAIL Basic Research: 

Goal: to invent a cognitive system to support a DSS to support increased human workload capability for alert-based activities Theoretical products A General Definition of Human Interruption A Taxonomy of Human Interruption Empirical findings Negotiation is the best overall interruption coordination solution except for cases where small differences in timelines of handling alerts are critical HAIL Basic Research

Human-Subjects Experimentation: 

Human-Subjects Experimentation Single-factor within-subjects Latin squares design Thirty-six subjects (18 male, 18 female), average age 24.7 (min. 18, max. 47) Six treatments — four experimental + two control Twenty-four trials; 4.5 minute each One hour of practice followed by one hour of experiment Pilot testing to set difficulty (59, 80)

Negotiated is Best for Primary Task: 

Negotiated is Best for Primary Task

Immediate is Worst for Interruption Task: 

Immediate is Worst for Interruption Task

Small Timeliness Problem with Negotiated and Schedule: 

Small Timeliness Problem with Negotiated and Schedule

Negotiated is Efficient for Interruption Task: 

Negotiated is Efficient for Interruption Task

Negotiated is Best for Keying Effectiveness: 

Negotiated is Best for Keying Effectiveness

Negotiated is Best for Keying Efficiency: 

Negotiated is Best for Keying Efficiency

Slide24: 

Cognitive System Development Product Overview Scope of HAIL Technology HAIL Technology Capabilities Technical Approach HAIL Engine Capabilities Work Process Software Development Operational Impact Programmatic Overview HAIL Team Overview

Product OverviewDomain-independent decision-support engine for alert management: 

Before HAIL After HAIL HAIL-SS is producing a reusable general component-based software product for alert management. Product Overview Domain-independent decision-support engine for alert management HAIL-SS is producing two products (1) A platform-independent software engine that delivers a intelligent alert management critical future naval capability for improved C2 decision-support in dense information environments

Product Overview (cont.)Aegis is the first transition target: 

HAIL-SS is producing a validated integration of the HAIL Engine within the Aegis platform. Product Overview (cont.) Aegis is the first transition target HAIL-SS is producing two Products (cont.) (2) A HAIL Engine integrated, tested, and validated within an Aegis Weapon System

Scope of HAIL-SS TechnologyThree Development Activities for a C2 Alerting Mechanism: 

User Interface: Human Factors Eng. to dynamically balance multimedia cueing to match operators’ cognitive needs (flashing, alarm sounds, color, spatial layout) Dependent on domain Research complete: no Application Cognitive System Component [HAIL Engine]: DSS engineering to manage alerts (negotiation-based task management; interactive visualization; automatic trend inference; context recovery support) Independent of domain Research complete: yes Alert Generation: Systems engineering to generate alerts to support tactical action (hard real-time computing; operations; tactical role modeling) Dependent on domain Research complete: no Scope of HAIL-SS Technology Three Development Activities for a C2 Alerting Mechanism The HAIL Engine will enhance both current and future UI and Alert Generation technologies.

HAIL Technology Capabilities: 

HAIL Technology Capabilities Detect significant trends Minimize human weaknesses for monitoring change over time Trend inference automation Switch contexts without making errors Minimize human weaknesses for context switching Context recovery support Receive informational alert content without interruption Leverage human ability to assimilate information Visualization support for SA Optimum prioritization and scheduling of interruptions Leverage human ability to dynamically prioritize and schedule Negotiation-based support for coordination Fewer interruptions Decouple alert rate from interruption rate Alert management Better alert SA Break design gridlock Broker alert management Warfighter Benefits Purpose Cognitive System Capabilities

Technical Approach: 

Technical Approach Integrated technology definition/design/transition process Leverage existing research Reusable HAIL alerting engine Open architecture, components-based design and development Multiple domain applications driving requirements Use Navy domain application to Define requirements Test and evaluate in a realistic environment Support transition Link with AOA schedule requirement Phase 1: Test and Evaluation of First Prototype Phase 2: Evaluation in Operational Setting using Baseline 7P1 A 2-cycle development approach ensures that the HAIL product is user-centered.

HAIL Alerting Engine Capabilities: 

HAIL Technology HAIL-Based Alerting Application User Interface HAIL Alerting Engine Capabilities Action alert management Supports multiple interruption strategies Presentation management Supports presentation of information-only alerts in multiple modalities Context recovery support Supports domain-dependent recovery mechanisms Automated assessment Supports inference of higher-level alerts and trends HAIL technology provides future naval capability for action alert management, presentation management, context recovery support, and automated assessment.

Work Process: 

Evaluation Design Evaluation Prototype Platform-Specific Application Design Application Requirements Definition Human Cognitive Requirements Prototype Platform-Specific Application Integrate and Test Core HAIL Technology Detailed design Architecture HAIL Research Risk Definition Work Process Human cognitive requirements and application requirements drive the development process.

RUP-Guided S/W Development ProcessHAIL process and toolset is consistent with Aegis Open Architecture: 

RUP-Guided S/W Development Process HAIL process and toolset is consistent with Aegis Open Architecture UML Documentation Use Cases Activity Diagrams Visio and Altia Design Mockups for supporting spiral iterations of user Interface (UI) design and usability testing. HAIL is using industry standard development process and documentation artifacts.

Operational Impact (Estimated)Successfully Handle Higher Information Volume: 

Operational Impact (Estimated) Successfully Handle Higher Information Volume Better decision-making and situational awareness of naval warfighters through support for human alerting Product will have general utility potential for several platforms; examples are for Aegis Frequency of interruptions reduced 50-80% Button actions required reduced 50-80% Time and effort required for critical alerts improved 15-50% Correlate alerts with tracks on the TACSIT improved 40-60% Efficiency in resuming primary task after interruption improved 10-30% Perceived workload reduced 10-30% Quality of situational awareness improved Perceived stressfulness reduced 20-30% The HAIL cognitive system will support greatly improved human capabilities for dealing with alerts in complex multitasking work contexts.

Programmatic OverviewFunded by ONR and founded on the results of interdisciplinary theory-based research from NRL: 

Programmatic Overview Funded by ONR and founded on the results of interdisciplinary theory-based research from NRL Technology Maturity NRL/ONR HAIL ‘95-‘00 Other Naval Systems Measured in TRL’s ONR KSA FNC HAIL-SS ‘02-‘04 PEOTSC ‘05-‘? (existing TTA) Product Development Research Integration Other; … The HAIL-SS Web Site: http://www.atl.lmco.com/projects/hail/ HAIL-SS is: An ONR KSA FNC project to develop and test a generic alerting software component that actualizes the accepted NRL NCARAI alerting research findings for improving operator performance in the presence of alerts HAIL-SS is on contract with the ONR KSA FNC Program (N00014-02-C-0014, begun 4/02), LM ATL lead The HAIL technology starts with NRL-proven solution to a recognized future Navy problem and is paying down the risk for acquisition.

HAIL-SS Team OverviewFull-coverage team for creation of a successful HAIL product: 

HAIL-SS Team Overview Full-coverage team for creation of a successful HAIL product This integrated, diverse team provides insight and expertise covering all areas relating to Aegis.

R&D Process: 

R&D Process Process

Process: 

TRL 5 TRL 3 TRL 2 TRL 1 TRL 4 TRL 7 TRL 8 TRL 9 TRL 6 “The theory-based empirical experiments” Lab Publication in Public Domain Future Mission Requirements Mature COTS Technology “Fully mission capable” Fielded Surface Combatant Process ISART HAIL (NRL ‘95-‘00) (ATL ‘00) Better Human Alerting Mechanisms (Lessons Learned ‘91-’92) HAIL-SS (ONR KSA FNC ’02-’04) LM NE&SS Aegis HAIL-SS is an ONR-funded product maturation effort.

Summary: 

Summary Summary

Summary: 

Summary HAIL is producing a platform-independent technology composed of two things: (1) The HAIL Engine: A cognitive system that delivers cognition-based resources for human alerting activities (2) The HAIL System: An open architecture decision-support system (DSS) component built around the HAIL Engine HAIL supports increased human capability: (1) Decouples the alerting rate from the human interruption rate (2) Increases user control for coordination of interruption events and recovering context after interruption