2 models for CE PDBO blok1

2. Models for cognitive ergonomics: 

2. Models for cognitive ergonomics 2.1. the concept of models 2.2. models in cognitive psychology 2.3. cognitive ergonomics models

2.1. the concept of models: 

2.1. the concept of models “M is a model of A if M can be used to answer questions about A” (Ross, 1983) A model is a representation of relevant characteristics of an object (Rohr & Tauber, 1984) A model is a description that can be communicated, of a certain aspect of part of the real world, viewed at a certain level of abstraction (Oberquelle, 1984)

representation of relevant characteristics of an object: 

representation of relevant characteristics of an object user system User inter-face Mental model of system

aspect of part of the real world, viewed at a certain level of abstraction: 

aspect of part of the real world, viewed at a certain level of abstraction system User interface Aspect 1, graphics Aspect 2, dialogue

the concept of modelsinternal models & external models: 

the concept of models internal models & external models External models are for communication, should be represented explicitly Internal models are for “execution”: there is an “agent” who uses the model to make decisions based on the behavior of the model, and to make predictions on the behavior of the modeled reality if the agent is human: mental model if the agent is a machine: program, database

2.2. models in psychology: 

2.2. models in psychology 2.2.1. models of human information processing modern theories, e.g., Barsalou mental models in Cognitive Psychology 2.2.2. mental models in Cognitive Ergonomics - Norman

2.2.1. Model of human information processing: 

2.2.1. Model of human information processing long term MEMORY THOUGHT working MEMORY input PERCEPTION output MOTOR/BEHAVIORAL TASK/ STIMULI

Slide8: 

ACTIVITITES OF THE COGNITIVE SYSTEM PERCEPTION: complex sensorial processes primary images containing all information about the concrete features of objects and phenomena that act direct action upon the sensory systems (visual, acoustic, kinesthetic, olfactory, gustatory). MEMORY: ability to remember, recognize and recall information is encoded, stored and retrieved. active: structuring, constructive and creative psychic mechanism. THOUGHT: the process of information processing in working memory.

Slide9: 

VISUAL PERCEPTION - A COMPUTATIONAL THEORY (pattern recognition) PRIMARY PROCESSING SECONDARY PROCESSING - RECOGNITION Visual Stimuli Descendant processing Primary sketch 2.5 D sketch 3D Representation texture movement color distance position gestalt geons depth form principles segmentation Processing modules

Slide10: 

VISUAL PERCEPTION - A COMPUTATIONAL THEORY (pattern recognition) Data-driven (bottom-up ) processing 1. mechanisms of edge detection, processing of texture, movement, color, distance and depth, position and form from processing of shadow of stimulus (2.5-D sketch,) - automated/ modular/preattentional/ unconsciousness processing 2. edge organization - gestalt principles of perception (applied to 2.5-D sketch) : proximity, similarity, good continuation, closure example: IwOuLdLiKeToDrInKaBeErInArOmAnIaNbAr. 3. geons generation (geometrical ions) segmentation of the 2.5-D sketch (intermediary image) in zones of maximal concavity 4. Recognition geons activate from memory the objects made-up by the respective geons, matching the 2.5-D sketch with the representation stored in memory

Slide11: 

A. Proximity principle B. Similarity principle C. Good continuation principle C D D. Closure principle A B GESTALT PRINCIPLES OF PERCEPTION

Slide12: 

VISUAL PERCEPTION - A COMPUTATIONAL THEORY (pattern recognition) Conceptually driven (top-down) processing concepts and higher-level processes influence pattern recognition applied to 2.5-D sketch recognition in the case of: - verbal stimuli (word superiority effect; sentence superiority effect) T E C T THE WORK MUST GET DONE. WORK

Slide13: 

VISUAL PERCEPTION - A COMPUTATIONAL THEORY (pattern recognition) I’m zhizkizg tz enzoy zhiz wezk-ezd az thz sezsize. - objects (object superiority effect) - human faces OBS.: Importance of implicit/tacit knowledge about the (statistical) regularities of the world in pattern recognition (physical support of things, reciprocal occlusion, occurrence probability, relative size, position and rigidity of objects etc.). Violation of tacit assumptions causes visual illusions. This assumptions are not specific (they are applied automatically to any object).

Slide14: 

The importance of context Illusions - at level of modular processing

Slide15: 

AUTOMATIC PROCESSING Learned automatic processing Frequent association of a stimulus with a response produces a production having a relatively autonomous status. Then, the perception of the stimulus produces the response with no intention to do so (ex. classical conditioning). in perception: orienting learned reflexes (shifting the attention to important stimuli in the environment - ex.: a speaker who utters your name) (used by advertising industry) in memory: upon encoding a particular stimulus, people may activate automatically information associated with it in memory

Slide16: 

STRATEGIC PROCESSING EXECUTIVE PRODUCTIONS cognitive mechanism that establish and execute acquired goals “if-then” productions current state of the environment and the cognitive system determine which executive production is fired in order to pursue the high-level goals of the cognitive system number of executive productions that can be fired at once is extremely limited (one or possibly few) because of centralized strategic resource

Slide17: 

MECHANISMS OF STRATEGIC PROCESSING executive productions operates on goals, scripts and reminded episodes to select and coordinate information processing subsystems during goal-directed behavior repeating particular patterns of strategic processing produces new productions that automate repeated parts of relevant scripts, it freeing executive productions to work on more subtle aspects of task or to perform multiple tasks simultaneously skills develops as increasing amounts of the processing (that executive production perform) become automated

Slide18: 

PERFORMANCE CHARACTERISTICS OF INFORMATION PROCESSING 1. Limited capacity information processing resource that limits strategic processing was theorized as: - limited processing energy (analogous with an electrical source) (if one strategic task require all of the available processing energy, no other task can be accomplish simultaneously) - a single executive processor that applies and manages executive productions - it can only execute one strategic task at a time but can switch quickly from one strategic to another. Limits on strategic processing arise from the limited ability of the executive processor to switch between the tasks. Elementary operations are: compare / choose / repeat / compute / transform

Slide19: 

PERFORMANCE CHARACTERISTICS OF INFORMATION PROCESSING 2. Selectivity to achieve a goal a cognitive system must be able to select and coordinate information processing subsystems (ex. perceptual, motor, memory), locations in perceptual fields and categories in memory corresponds to what theorists often mean by attention selection can be specific (ex.particular ear or eye, particular information in memory)

Slide20: 

DISCUSSIONS Presence of a single executive does not imply that it controls the entire cognitive system: The executive may primarily schedule and monitor (it may also direct processing when goals are new, difficult and dangerous). Many basic processes in perception and movement, many acquired productions which control many automated skills (ex. driving, typing), lie beyond its scope.

Slide21: 

SENSORY MEMORY consist in persistence (prolongation) of the sensorial representation of the stimulus after the stimulus is no longer acting on the receptors specific to a certain type of sensation format: neuro-physiological codes capacity: unlimited but the cognitive system will process further only the relevant stimuli duration: - visual memory 100 ms - auditory memory 200 ms / 2 s automatic/ pre-attentional retention of the precategorical information (information is in an unprocessed form)

Slide22: 

LONG TERM MEMORY (LTM) all the knowledge the cognitive system owns unlimited capacity duration: whole life of neural system memory systems in LTM: explicit vs. implicit semantic vs. episodic (memory of general knowledge about our environment vs. memory of personal events) format (encoding) of the information: verbal / analog (images) / semantic (propositional) retrieval (activation) of knowledge: parallel search process

Slide23: 

WORKING MEMORY (WM) activated part of long-term memory encoding: verbal, analogical (image), semantic (propositional) capacity: depending on the level of expertise (by chunking); maximum capacity of attention: 7 +/- 2 chunks limited capacity for a certain type of information (auditory, visual, motor) limits for certain types of information are independent one from another (ex. if maximum of visual information is in WM this does not decrease the maximum of auditory information that can be in WM).

Relation between LTM, WM and attention: 

when there is a specific goal to be reached LTM WM Attention when there is no specific goal to be reached LTM WM = Attention Relation between LTM, WM and attention

Slide25: 

ORGANIZATION OF KNOWLEDGE IN LONG TERM MEMORY EXPLICIT MEMORY: content is accessible to consciousness and can be tested by recall and recognition tests low level structures propositional network (semantic memory) semantic network (semantic memory) high level structures (complex units of knowledge) schema/script/frame/plan (semantic and episodic memory) mental models (semantic and episodic memory) IMPLICIT MEMORY: content is hardly accessible to consciousness and cannot be tested by recall and recognition tests. production systems (cognitive and motor skills, priming, conditioned reflexes)

Slide26: 

animal breath have skin fish swim shark bird fly have feather eagle chicken predator symbol of power dangerous not eatable SEMANTIC NETWORK - represents semantic contents from well-structured knowledge domains - knowledge are represented by a network of nodes and relation between nodes - nodes represents the concepts and the relations between nodes are labeled - meaning of a concept (or node) is given by the pattern of its relations among which it participates. type type property type type property

Slide27: 

SCHEMAS (Rumelhart, 1980) represents generic concepts stored in memory underlying objects, situations, events, sequence of events, actions and sequence of actions they are used for a class of stereotypical situation they vary the very simple to the very complex are organized in a hierarchical fashion: variables which have fixe value (the kernel) slots with optional values (pheripheral) which can be filled in with particular instances of the concepts. If the instances are not specified then the slots will have default values (prototypes) can be embedded one in other, e.g., Human body (Head, Trunk, Limbs) Head (Face, Ears, Hair) relation between the elements are in spatial-temporal contiguity (ex. bread - butter) active processing devices (top-down processing) which produce an interpretation of the world - they adapt reality to knowledge they are assumed to be shared across individuals (in a culture?)

Slide28: 

SPECIAL TYPES OF SCHEMAS SCRIPTS (Shank and Abelson,1977) schemas for frequently occurring sequences of events in a particular context 2 categories of variables: roles (filled by persons) and props (filled by objects) includes: - entries condition - scenes - results scripts are the result of social learning maintanance of the scripts is guaranted by a set of social contingencies FRAMES (Minsky, 1975) - static representation - schemas that do not possess active processors - between the elements of the frame are enabling or causal relations - framework that is adapted to fit reality generic frames (class): ex.: car ( color, brand, driver, engine, transmission, wheels) specific frames (instantiated) - in a particular context ex.: my friend’s car (black, Mercedes, John, 4 wheel drive)

Slide29: 

Driver Fuel Engine Transmission Wheels gasoline diesel type flows rotates rotates FRAME FOR “CAR” buys operates John Liz Fixed values (kernel) Specific instances (default values) type type operates type four-cylinder type steel alloy six-cylinder type standard

Slide30: 

Task: Arrange a project meeting Plan (meeting (project)) Consult (information source, information token, project meeting) Identify (information source, information token, project meeting) Search (information source, information token) Retrieve (information token, information source) Store (information token, project meeting, working memory) Select (media message) Identify (long-term memory,constrain, project meeting) Choose (media, constrains ) Send message (meeting (project)) Consult (information token, information source) Identify (information source, information token, letter) Search (information source, information token) Retrieve (information token, information source) Store (information token, information source) Represent (information token, message) Write (information token, message, media) Compare (message, information token) Edit (information token, message) Store (message copy, message file, media) Execute (transaction requirements, message) . frame-based representation: how to create and send message to arrange a project meeting (Keane and Johnson, 1987)

Slide31: 

Script: eating at a restaurant Entry condition hungry, had money, restaurant open Roles diner, waiter, cashier Props tables, money, chairs, menu, cutlery, food Entry scene Diner enters restaurant. Waiter seats diner at table. Waiter places menu on table. Diner begins to read menu. Ordering scene Diner selects food from menu. Diner signals to waiter. Waiter approaches the table. Diner orders food. Waiter leaves. Eating scene Waiter brings food to the table. Waiter leaves. Diner eats food with cutlery. Diner finishes eating food. Leaving scene Diner signals to waiter. Waiter approaches table. Diner asks waiter for bill. Diner checks bill. Diner approaches cashier. Diner gives cashier bill and money. Cashier checks money. Diner leaves restaurant.

Slide32: 

MENTAL MODELS - dynamic representation frames in which the relations between (and attributes of) the elements are analogous to a physical/organizational/procedural structure in the world (component parts and relations between these) reflecting the actual state of affairs in the world. parts of it become instantiated being triggered by an input (stimulus, problem, event) can be run producing quasi-continuos simulation of the events (over space and time) and can explain how events occurred (comprises explanatory principles)

Slide33: 

IMPLICIT MEMORY - TYPES OF KNOWLEDGE Cognitive and motor skills they develop from a script-like representation of knowledge Conditioned reflexes it develops by association of stimulus with response Priming it develops by frequent exposure to a stimulus, modifying the judgement value of the stimulus

Slide34: 

IMPLICIT MEMORY - KNOWLEDGE REPRESENTATION representation are in the form of production rules; a production is a condition-action (if-then) pair IF (condition for triggering) THEN (do these actions) production rules are organized in production systems production systems can be general or specific (defining expertise in a certain domain) they are triggered automatically by categorization of relevant stimuli by matching current state of problem-solving (as a pattern in working memory) or of a stimulus against the conditions of the productions rules they are hardly accessible to consciousness when needed, the script (on the basis on which production system has been developed) can be reconstructed

A variant of the model, Card, Moran, Newell: 

A variant of the model, Card, Moran, Newell

Slide36: 

Some principles perceptual processor cycle time varies inversely to stimulus intensity cognitive processor cycle time: shorter with more task load shorter with more information shorter with practice in task domain

A variant of the model, Card, Moran, Newell: 

A variant of the model, Card, Moran, Newell Some laws Fitts’ law: time T to move hand to target of size S at distance D: Tpos = 100[70~120] msec/bit log2 (D/S + .5) Power law of practice: time Tn needed to complete a task at trial n: Tn = T1 nª , where a = .4[.2~.6]

A variant of the model, Card, Moran, Newell: 

A variant of the model, Card, Moran, Newell Example: simple reaction time

A variant of the model, Card, Moran, Newell: 

A variant of the model, Card, Moran, Newell Example: symbol detection

Calculation symbol detection: 

Calculation symbol detection RT = Tperceptual processor + 2 Tcentral processor + T motor processor = 100[50~200] + 2*(70[25~170]) + 70[30~100] = 310[130~640] msec

2.2.2. Mental models from the point of view of Cognitive Ergonomics: 

2.2.2. Mental models from the point of view of Cognitive Ergonomics The functions of mental models in using complex systems What type of mental models are needed for using complex systems How to “measure” mental models

The functions of mental models in using complex systems: 

The functions of mental models in using complex systems Planning execution of task delegation evaluation interpretation

What type of mental models are needed for using complex systems: 

What type of mental models are needed for using complex systems

Characteristics of mental models: 

Characteristics of mental models Incomplete (users are generaly aware of this) parsimoneus, just sufficient basic knowledge can only partly be "run" unstable, change over time, using different systems new experiences (even experts) vague boudaries (e.g., application / operating system) superstition

How to “measure” mental models: 

How to “measure” mental models Reconstructing semantic networks / frames: Pathfinder Algorithm - allows insight in “group knowledge” and comparison between groups Hermeneutic interpretation of individual instantiations Instantiated frames: Teach-back task “what is …?” Instantiated scripts: Teach-back task “how to …?”

DIGITAL INK PEN: 

DIGITAL INK PEN

Scenario 1: 

Scenario 1 I discover a small garden yesterday, it was warm and dry and there was almost nobody around. And it was really the most relaxing place I had been to in months…

Slide56: 

So I sat down, I did some writing and drawing and thinking that it was months that I communicated with my friends’ back home… I decided that I should send them some of my garden sketches by e-mail. record

Slide57: 

So, in order to send the e-mails: I pressed the mode button on the side of the DI. record

Slide58: 

That changes the mode from record my drawing and hand writing... to command mode which allows me to tell DI what I wanted to do next. command record

Slide59: 

While in command mode the elliptical screen slowly scrolls through the command words reminding me of what option I want to use. send

Slide60: 

In this case I wanted to send e-mails to my friends so I simply write: the word send followed by their e-mail addresses. The “->” terminates the command and sends the message Once the send command has been written and read, the screen displays the progress of the command Now my sketches are on their computers. Send to luisa@vu.nl-> Send to pepe@vu.nl-> sending

Scenario 2: 

Scenario 2 I was wandering around the city and I bumped into Chris. We have only met once or twice before but we ended up talking for about one hour about my new job and his new business in Piltsburg…

Slide62: 

We had such a good time and I really wanted to get together again with him soon to talk again. So I took his phone number and his e-mail address and stored them in my pen just by writing on the back of an old receipt. record Chris@vu.nl 020- 4567890

Scenario 3: 

Scenario 3 I am getting ready to go on vacation and I have all this staff that I have to plan: client meetings,… tons of work to do and what better place to keep track of important clients than on a napkin of a cafeteria.

Slide64: 

The great think of the DI is that you can write on any surface that you want. I clipped my pen cap on the upper left hand corner of the napkin, then I mark two opposite corners. The clip combined with the two points lets the pen know the boundaries of the writing surface.

Slide65: 

So I wrote down a bunch of meetings, whom I am going to meet, and when and what I will be talking about and I was done. record 10:00 GUI meeting 12:30 lunch with Chris 14:00 meeting with the design team

Slide66: 

I left my list for the waiter to take away and… all I have to do is take my DI home tonight, put it into the “DI well” and then I will download my list of things to do into my computer and I will be ready to go. download DI well

Scenario 4: 

Scenario 4 With DI I can also check my e-mail. Incoming e-mail is indicated by a sound from DI (“ding”). Ding! e-mail

Slide68: 

When I want to read e-mails I press the mode button and write or speak the word get message to see the e-mail or e-mails I received get message

Slide69: 

then the senders of my e-mails appear numbered serially. To move through the list I can simply write or speak the words up or down 1.luisa 2.chris

Slide70: 

So if I want to read the message of Chris I write or speak the word show message And the message of Chris will appear on the screen. Then I could write back to him using the DI e-mail or fax command. 2. chris

Slide71: 

So those are things that DI can do. It’s a new innovative product that works off patterns that we are already familiar with. In fact, when DI is off I can use it as a regular pen.

DIGITAL INK PEN: 

DIGITAL INK PEN

Slide73: 

We would like to know how you imagine the “Digital Ink Pen” after seeing the scenarios. Therefore, explain to your friend “Lucas” what the “Digital Ink Pen” is. You can use text, drawings, etc.

Slide74: 

Your friend “Lucas” wants to send a fax to the administration of the faculty. Explain to him how to do this using the Digital Ink Pen.

Slide75: 

“Lucas” has five messages in his “Digital Ink Pen” and he is not interested in the third one. Try to explain to him how to delete this e-mail from the list of e-mails he has.

Slide76: 

How could Lucas dictate (speaking) a letter to Digital Ink and download it later to his computer?

Slide77: 

Teach-back questions: “what is”: We would like to know how you imagine the “Digital Ink Pen” after seeing the scenarios. Therefore, explain to your friend “Lucas” what the “Digital Ink Pen” is. You can use text, drawings, etc. “how to” 1: Your friend “Lucas” wants to send a fax to the administration of the faculty. Explain to him how to do this using the Digital Ink Pen. “how to” 2: Y“Lucas” has five messages in his “Digital Ink Pen” and he is not interested in the third one. Try to explain to him how to delete this e-mail from the list of e-mails he has. “how to” 3: Y How could Lucas dictate (speaking) a letter to Digital Ink and download it later to his computer?

Some examples: 

Some examples of protocols and their interpretation

Slide79: 

Well understood, but rather conservative regarding the dialogue possibilities

Slide80: 

Extrapolation based on intuition and consistency

Slide81: 

Reliability in interpreting visual protocols ? requires training

Slide82: 

User created verbal command mode

Slide83: 

User supposed restrictions in dialogue - why ??

Slide84: 

A wish list of functionality and with context specifically referred to

Slide85: 

The pen “does understand”

2.3. models in cognitive ergonomics: 

2.3. models in cognitive ergonomics 2.3.1. model of changeability 2.3.2. conceptual models of systems and of interaction 2.3.3. task models

2.3.1. model of changeabilityof cognitive functions: 

2.3.1. model of changeability of cognitive functions Stable, mainly influenced resistant to change by environment personality cognitive strategies: individual factors: styles: knowledge: -intelligence -field(in)dependence -impulsivity -schemas (spatial ability) -verbalisers / imagers -serialists / holists -production rules adapt to the user the user may adapt

Individual knowledge: 

Individual knowledge Long term memory explicit implicit instruction / exploration meta-communication by system (user interface): implicit (dialogue, lexicon) explicit (error messages, help system, yellow stickers) content domain task knowledge system knowledge

strategies: 

strategies Domain dependent may be learned adapting to strategies is often useful: impulsivity - “undo” an item of an impulsivity test: “find the picture identical to the target figure on the top”

Cognitive styles: 

Cognitive styles Domain independent / generic changeable through “education” imagers - symbolisers / field (in)dependence two fragments from a field(in)dependency test: “read the codes of the figures in the matrix as fast as possible”

Personality factors: 

Personality factors Stable, needs adaptation to e.g. spatial ability (intelligence factor) low s.a. is a handicap in working with complex system objects that cannot be presented as a whole adaptation can overcome handicaps: navigation aids abstract overview

2.3.2. conceptual models of systems and of interactionNorman: 

2.3.2. conceptual models of systems and of interaction Norman Norman (1983): target system mental model conceptual model Conceptual model: perceptual interface & conceptual interface the user’s virtual machine the model for design & for meta-communication (help)

conceptual models of systems and of interactionCLG: 

conceptual models of systems and of interaction CLG Moran (1981) Command Language Grammar (CLG) views on conceptual model: linguistic view - modeling and analyzing the interaction the psychological view - modeling and analyzing user knowledge and mental model the design view - modeling and analyzing the interface (the user’s virtual machine)

conceptual models of systems and of interactionCLG: 

conceptual models of systems and of interaction CLG Levels of analysis conceptual component task level semantic level communication component syntax level interaction level (key stroke level) material component spatial layout level apparatus level

conceptual component: 

conceptual component task level tasks for the user / tasks delegated to the system task procedures, objects in task domain semantic level tasks to delegate: system objects with attributes system operations user specifications, system states and state changes

communication component: 

communication component syntax level dialogue style lexicographic structure user actions and system acts temporal structure interaction level (key stroke level) user key-strokes, mouse handling, voice input, eye gaze system signs (beep, cursor indication, cursor movements, tactile feedback)

material component: 

material component The domain of classical ergonomics spatial layout level screen design, window layout spatial relation of input and output devices to work environment (light conditions, sound conditions) apparatus level characteristics of hardware (shape of buttons, key press characteristics)

2.3.3. task models: 

2.3.3. task models Users of complex systems have a task model related to the use of technology (internal model) designers of complex systems need a task model to make design decisions on how to support the user (external model) for details, see unit on (task) analysis