Andre Emotions

Views:
 
     
 

Presentation Description

No description available.

Comments

Presentation Transcript

State of the Art in Emotion and Personality: 

State of the Art in Emotion and Personality Elisabeth André University of Augsburg, Augsburg, Germany andamp; DFKI GmbH, Saarbrücken, Germany

Why should we care about emotional agents?: 

Why should we care about emotional agents? To increase believability: Joe Bates: The role of emotions in Believable Agents, CACM 1994: „... emotion is one of the primary means to achieve this believability, this illusion of life, because it helps us know that characters really care about what happens in the world, that they truly have desires.' To engage the user: David Freeman: Four Ways to Use Symbols to Add Emotional Depth to Games, Game Developer 2002: „... shallow emotional experience in games featuring stories and characters will increasingly stand out negatively in consumers‘ mind

Why should we care about emotional agents? : 

Why should we care about emotional agents? Central to human experience Essential for movies, drama, games Social glue (affection, guilt) Motivators (fear) Readily manipulated (by storytellers, salesmen) Central to many theories of intelligence Emotion, cognition, behavior tightly coupled Focus of Attention (Matthews and Wells) Belief formation (Frijda, Manstead, and Bem) Decision Making (Damasio, Murnighan) Learning, Tutoring, Training (Lepper, Frasson, Lester)

Why should we care about emotional agents?: 

Why should we care about emotional agents? Key to Pedagogy: Example: CARMEN Subject matter concerns highly emotionally charged situation Emotions and how people cope critical to subject matter Help users develop positive emotional attitude To influence the user‘s behavior by showing empathy Example: Computer Petz

Issues and Directions: 

Issues and Directions Sources of emotions Task-oriented emotions Social and organizational factors Consequences of Emotions Expressive behaviors Impact on speech generation and synthesis Impact on decision-making Impact on Human Participant Impact on believability, engagement Impact on learning

Schema for comparing different approaches: 

Schema for comparing different approaches Underlying theory of emotion and personality Purpose of emotions and personality Emotion and personality input Emotion and personality processing Emotion and personality output Implementation aspects

Models of Personality and Emotions: 

Models of Personality and Emotions Personality: Duration: long-term Focus: factors that determine personality are diffuse and indirect Five Factor Model: Extraversion, Agreeableness, Openness, Conscientiousness, Neurotism Emotions: Duration: Short-lived Focus: influenced by particular events, agents or objects OCC Model: Emotions as valenced reactions to events, agents or objects'

OCEAN: 

OCEAN Open: curious, broad interests, creative, original, imaginative, untraditional Conscientious: organized, reliable, hard-working, self-disciplined, honest, clean Extravert: sociable, active, talkative, optimistic, fun-loving affectionate Agreeable: good-natured, trusting, helpful, forgiving, gullible, straightforward Neurotic: worries, nervous, emotional, insecure, inadequate, hypochondriac

The OCC-Model: 

The OCC-Model Emotions as valenced reactions to: The desirability of events with respect to goals The praiseworthiness of other agents (users) with respect to standards And the appealingness of objects with respect to attitudes

Global Structure of Emotion Types: 

Global Structure of Emotion Types hope fear CONFIRMED DISCONFIRMED satisfaction fears-confirmed relief disappointment PROSPECT-BASED SELF AGENT OTHER AGENT DESIRABLE FOR OTHER UNDESIRABLE FOR OTHER CONSEQUENCES FOR SELF CONSEQUENCES FOR OTHER PROSPECTS RELEVANT PROSPECTS IRRELEVANT FOCUSING ON ASPECTS OF OBJECTS CONSEQUENCES OF EVENTS VALENCED REACTION TO pleased, displeased etc. FOCUSING ON ACTIONS OF AGENTS approving, disapproving etc. liking, disliking etc.

Classification of Operationalized Models: 

Classification of Operationalized Models Communication-driven Models Selection of communicative actions and emotions based on an effect to be achieved on the user Appraisal-driven Models Simulation of dependencies between emotions and cognitive processes

Examples of Communication-driven Models: 

Examples of Communication-driven Models André et al., Lester et al.: Speech acts drive the selection and sequencing of emotive behaviors. Ball/Breese: Convey empathy by expression styles that reflect the emotional state of the user Poggi/Pelachaud: Use of facial expressions to express affect

Examples of Appraisal-Driven Models: 

Examples of Appraisal-Driven Models Implementations of the OCC-Modell: Elliott: The Affective Reasoner Reilly: Believable Social and Emotional Agents Gratch: EMiLE - Plan-based model of emotion appraisal Cognitive Architectures: Sloman et al. Simulations: Blumberg et al.: Learning and Emotions El-Nasr et al.: FLAME – Fuzzy Logic Adaptive Model of Emotions

Emile: Plan-based Model of Emotion Appraisal: 

Emile: Plan-based Model of Emotion Appraisal Intensity Variables: Probability with which a goal can be achieved. Importance of a goal Examples: IntensityHope(goal) = Import(goal) P(goal) IntensityJoy(goal) = Import(goal) IntensityFear(goal) = Import(goal)[1-P(goal)] IntensityDistress(goal) = Import(goal) IntensityAnger(goal) = Import(goal)P(threat) Literatur: J. Gratch: Emile: Marshalling passions in training and education. Proc. of the 4th International Conference on Autonomous Agents, Barcelona, Spain, 2000.

Dialogue between two Emile-Agents: 

Dialogue between two Emile-Agents

Emotional Agents for Training: 

Emotional Agents for Training Model communicative function of emotion Convey information about agent’s internal state Goal priorities Situation assessment Action tendencies Create drama and tension Demands motivational and Behavioral consistency

Cognitive Appraisal Theory (Lazarus, OCC, Frijda): 

Cognitive Appraisal Theory (Lazarus, OCC, Frijda) Environment Goals, Beliefs Emotion Coping Behavior Give agent consistent internal state Assess state vis-à-vis the world Convey the assessment

Slide18: 

Treat Child Troops_helping Troops_helping Goal Child Healthy hope fear Anger Pr: 0.1 U: 60 Pr: 0.1 U: 40 Coping Behavior Émile, Gratch2000 Sources of Emotion: Plan-based appraisal TROOPS LEAVE Gratch andamp; Marsella: Tears and Fears, Autonomous Agents 2001

Relationship between Personality and Emotions: 

Relationship between Personality and Emotions Problem: Unified model that could be directly implemented seems to be missing? Basic Approaches: Map emotions to behaviors in a personality-specific ways (e.g. see Bates 1994) Treat personality as a variable that determines the intensity of a certain emotion (e.g. see Allen 2000)

Relationship between Personality and Emotions I: 

Relationship between Personality and Emotions I + positive influence, - negative influence Source: Allen, DFKI, 2000

Relationship between Personality and Emotions II: 

Relationship between Personality and Emotions II + positive influence, - negative influence Source: Allen, DFKI, 2000

Presence: 

Presence

Affective Processing Examples: 

Affective Processing Examples (Un)Desirable Situation: User continuously clicks the agent‘s body. This leads to the emotional state distressed, because the event is undesirable for the agent. intensity = (value of undesirability of andlt;eventandgt;) x degree of neuroticism Praiseworthy Situation: The agent gives additional information to an request. The user praises this behavior. The approving of the agent‘s own action, leads to the emotional state pride. intensity = (value of praiseworthiness) x degree of extraversion

Slide24: 

Emotional Behavior Facial Expression Body Language Action Variants Boston Dynamics Inc. Haptek Inc.

How to Convey Personality: An Example: 

How to Convey Personality: An Example Extrovert Characters more expansive gestures speak louder more powerful phrases tend to take initiative in a dialogue talkative tend to look at conversational partner Introvert Characters tight gestures speak at low voice indirect, hesitant speech waits for others to take the initiative says only the minimum tend to avoid looking at conversational partner

Expression of Personality: 

Expression of Personality Talking Posture 1 cautious, hesitant appeal for compliance avoids body-gestures Talking Posture 2 active, self-confident uses body-gestures

Expression of Emotion: 

Expression of Emotion I’m embarrassed. I’m angry. I’m pleased.

Carmen’s Bright IDEAS (Marsella, Johnson, LaBore): 

Carmen’s Bright IDEAS (Marsella, Johnson, LaBore)

Physical Focus Model Marsella2000: 

Physical Focus Model Marsella2000 Analysis of Emotion-predicting motion Analysis of movement in clinical settings (Freedman) Coping strategies (Lazarus) Organize behavior around distinct modes Bodyfocus – inward directed Communicative – outward directed Consistent interpretation of agent Internal state andamp; behavior Emotions andamp; relation to environment Coordinate different modalities

Physical Focus Modes: 

Physical Focus Modes Selected based on emotional state High distress  Bodyfocus Inner-directed attention Increased self-touching Gaze aversion (limited perception) High anger or joy  Communicative Outward directed attention Increased use of communicative gestures Directed gaze ('unlimited' perception)

Affective Speech: 

Affective Speech Source: Cahn, MIT Media Lab.

Expression of Emotions via Speech: 

Expression of Emotions via Speech Anger Disgust Fear Gladness Sadness Surprise

How to Represent the Input to the Surface Realization Components?: 

How to Represent the Input to the Surface Realization Components? Virtual Human Markup Language (VHML) combines: EML: Emotion Markup Language GML: Gesture Markup Language SML: Speech Markup Language (based on SSML) FAML: Facial Animation Markup Language BAML: Body Animation Markup Language XHTML: eXtensible Hypertext Markup Language DMML: Dialogue Manager Markup Language

Example of a Sentence Annotated with APML: 

Example of a Sentence Annotated with APML andlt;APMLandgt; andlt;turn-allocation type='take turn'andgt; andlt;performative type='inform' affect='sorry-for' certainty='certain'andgt; I'm sorry to tell you that you have been diagnosed as suffering from what we call angina pectoris, andlt;/performativeandgt; andlt;belief-relation type='elaboration-object-attribute'andgt; which andlt;performative type='inform' certainty='certain'andgt; appears to be andlt;adjectival type='small'andgt;mild andlt;/adjectivalandgt; andlt;/performativeandgt;andlt;/belief-relationandgt; andlt;/turn-allocationandgt; andlt;/APMLandgt;

Examples of Communikative Functions in Greta: 

Examples of Communikative Functions in Greta

Hamlet: Display Or Not Display: 

Hamlet: Display Or Not Display Context1: 3-year-old girl would like to make a present to her mother und cuts a dress. The mother comes in and sees the disaster. Oh, a present for me: you are a very nice girl! But the ribbon is made from my suit that I need for my work. If you cut it, I can’t wear it anymore. Regel: IF (Feel Ag Anger) AND (Adoptive Ag I) AND NOT (Comprehension-I) THEN NOT(Display Ag Anger) Context2: The child has already cut a dress. Oh, Oh, a present for me: you are a very nice girl! But the ribbon is made from my suit! I already told you a hundred times not to touch my things. Regel: IF (Feel Ag Anger) AND (Adoptive Ag I) THEN (Display Ag Anger) Quelle: De Carolis et al., Univ. of Bari, 2001.

D-Plan: 

D-Plan n1 (Discuss (Preparation I present)) n2 (Thank Ag I present) n3 (Inform Ag I (Seen Ag present)) n4 (Inform Ag I (Appreciates Ag present)) n5 (Reproach Ag I (Cut I suit)) n9 (Reproach Ag I Consequences(Cut I suit)) n6 (Explain Ag I Negative-Event (Cut I suit)) n7 (Inform Ag I Use(Ag suit)) n8 (Inform Ag I Effect(Cut I suit)) Contrast Justification Justification Elab

E-Plan: 

E-Plan n1 (Discuss (Preparation I present)) n2 (Thank Ag I present) (Display Ag Joy) n3 (Inform Ag I (Seen Ag present)) (Display Ag Joy) n4 (Inform Ag I (Appreciates Ag present)) (Display Ag Joy) n5 (Reproach Ag I (Cut I suit)) n9 (Reproach Ag I Consequences(Cut I suit)) n6 (Explain Ag I Negative-Event (Cut I suit)) n7 (Inform Ag I Use(Ag suit)) n8 (Inform Ag I Effect(Cut I suit)) Contrast Justification Justification Elab

Resulting APML-Structure: 

Resulting APML-Structure Suppression of Anger: andlt;?xml version='1.0'?andgt; andlt;APMLandgt; andlt;affective type='joy'andgt;Oh, a andlt;deictic obj='present' coord='0 30 0'andgt; present for me: andlt;/deicticandgt;you are a very nice girl! andlt;/affectiveandgt; andlt;belief-relation type='contrast'andgt; But andlt;/belief-relationandgt;the ribbon is made andlt;deictic obj='suit' coord='10 -25 0'andgt; from my suit andlt;/deicticandgt; that I need for my work. If you andlt;topic-comment type='comment'andgt;cut it andlt;/topic-commentandgt;, I can't wear it anymore. andlt;/APMLandgt;

Resulting APML-Structure: 

Resulting APML-Structure Expression of Anger: andlt;?xml version='1.0'?andgt; andlt;APMLandgt; andlt;affective type='joy'andgt;Oh, a andlt;deictic obj='present' coord='0 30 0'andgt; present andlt;/deicticandgt; for me: you are a very nice girl! andlt;/affectiveandgt; andlt;belief-relation type='contrast'andgt;Butandlt;/belief-relationandgt; andlt;affective type='anger'andgt; the ribbon is made from my andlt;deictic obj='suit' coord='15 0 0'andgt; suit. andlt;/deicticandgt; I already told you a hundred times not to touch to my things!andlt;/affectiveandgt; andlt;/APMLandgt;

Bayesian Networks: 

Bayesian Networks Generation of Behaviors: Set the values of the variables for personality and emotions on the state the agent should convey. The Bayesian network will then predict a probability distribution for possible behaviors. Recognition of Personality and Emotions (e.g. that of the user) Values for variables result from observations The Bayesian Network provides an estimation of the personality and emotions and of an agent.

Conditional Probabilities for SpeechVol and WdsActive: 

Conditional Probabilities for SpeechVol and WdsActive

Bidirectional Use: 

Bidirectional Use Quelle: Breese andamp; Ball, Microsoft Research, 2000.

Bayes’sches Netzwerk zur Modellierung von Persönlichkeit und Emotionen: 

Bayes’sches Netzwerk zur Modellierung von Persönlichkeit und Emotionen Quelle: Breese andamp; Ball, Microsoft Research, 2000.

Bayesian Network which associates different expression styles with paraphrases of the concept „Greet“ : 

Bayesian Network which associates different expression styles with paraphrases of the concept „Greet' Table represents the probability with which different paraphrases will be perceived as a positive greeting. Simulation: Each style node (e.g.. wdsPositive) has a value distribution which depends on the agent’s personality and emotions. A negative emotion increases the probability that the agent says 'Oh, you again'. Recognition: For a friendly agent, the utterance 'Oh, you again' is associated with a far more negative emotion than for an unfriendly agent. Quelle: Breese andamp; Ball, Microsoft Research, 2000.

Use of Bayesian Networks to Resolve Conflicts in Greta: 

Use of Bayesian Networks to Resolve Conflicts in Greta A communicative act is usually conveyed via different channels of the face, such as eye brows, mouth shape, gaze, head direction and head movements For instance, „sorry-for' is conveyed via a special movement of the head as well as a special form of the eye brows, while „certain' is expressed by a single signal, namely a special form of the eye brows.

Conflict Resolution: 

Conflict Resolution A special difficulty arises due to the fact that several tags may encompass one and the same piece of text. For instance, the sentence: „I'm sorry to tell you that you have been diagnosed as suffering from what we call angina pectoris' is tagged for times. Each of these tags corresponds to a given facial expression. Some of these expressions may refer to the same channel. It may even happen that different values are assigned to the same channel. „Sorry-for' is expressed by oblique eye brows, while „certain' is conveyed by a small frown.

Use of a Bayesian Network for Conflict Resolution: 

Use of a Bayesian Network for Conflict Resolution

Use of Bayesian Networks to Model Emotions: 

Use of Bayesian Networks to Model Emotions Advantages: Handling uncertainty Connections between nodes are intuitively comprehensible since they represent the relationship between cause and effect easy to extend suitable both for recognition as well as simulation Problems: Determination of the initial probabilities only suitable for modeling independent matters of fact

Affective Jewelry: 

Affective Jewelry

Expressive Animations: What is already available and what is missing?: 

Expressive Animations: What is already available and what is missing? Expression of affect, but personality hardly addressed (especially for the face) Body mostly neglected and not synchronized with the face Commercial animation tools are available, but: SDK/API often badly documented (e.g. Pulse 3D) Usually rely on pre-authored audio

Expressive Animations: What is already available and what is missing?: 

Expressive Animations: What is already available and what is missing? A lot of manual work is still required from the human designer who have to control a large number of low-level parameters. Promising line of research: EMOTE computational model of effort and shape (Badler et al. 2001) Control through the usage of high-level parameters that represent qualitative aspects of movements

Expressive Speech: What is missing?: 

Expressive Speech: What is missing? No interaction between speech-related parameters, such as FO contour, and other linguistic information like sentence type Trade-off between flexibility of acoustic modeling and perceived naturalness In order to express a large number of emotional states with a natural-sounding voice, either rule-based techniques need to become more natural-sounding or selection-based techniques more flexible. Need of more appropriate evaluation techniques We need to move away from forced-choice tests that use abstract emotion words towards tests measuring naturalness of an utterance given an emotion-defining context. M. Schröder, Emotional Speech Synthesis – A Review, Proc. Eurosppeech 2001.

Cognitive Models: What is missing?: 

Cognitive Models: What is missing? No uniform model of personality, emotion and social context Models usually start from a given set of input parameters, but do not say where they come from Dynamic aspects of emotions are hardly addressed

Conclusion: 

Conclusion Much work is needed to bring the single pieces of the jigsaw together. To use affect effectively, it must be used both at an appropriate level for the application domain, and as an all-encompassing component of the system - from graphic design to system architecture to application.

Supporting Affective Interactions for Real-time Applications (SAFIRA): 

Supporting Affective Interactions for Real-time Applications (SAFIRA) Overall objective: Bring to the software community an enabling technology to support affective interactions, in particular: To create a framework to enrich interactions and applications with an affective dimension; To implement a toolkit for affective computing combining a set of components addressing affective knowledge acquisition, representation, reasoning, planning, communication and expression; To verify under which conditions the hypothesis that emotion, as well as other affective phenomena, contributes to improve rationality and general intelligent behavior of the synthetic characters.

authorStream Live Help