Project PRAIA Pedagogical Rational

1
Project PRAIAPedagogical Rational Affective
Agents

• Patrícia Jaques, Edilson Pontarolo, Magda Bercht,
  
• Rosa Vicari, Sylvie Pesty

Projeto Capes-Cofecub 2007-2010
2
Team

• Brazilian Team
• Rosa Vicari (Head 2007-2008) - UFRGS
• Patrícia A. Jaques (Head 2009-2010) - UNISINOS
• Aline Villavicencio - UFRGS
• Magda Bercht - UFRGS
• French Team
• Sylvie Pesty (Head) - LIG
• Jean-Paul Sansonnet - LIMSI
• Veronique Auberge - LIG
• Jean-Claude Martin - LIMSI

3
Research Subjects
Brazilian Researchers
Inference of emotions by - Appraisal model - Face
Intelligent Learning Environments
Handle affect in natural language
Application of affective pedagogical tactics by
animated pedagogical agents
French Researchers
Affect in Speech
Embodied Conversational Agents
Multimodal expression of emotions by Animated
Conversational agents
4
Affective Computing

• Definition
• computing that relates to, arises from or
  deliberately influence emotions PICARD, 97.

• An affective computational system must present
  one of the following capabilities

• Recognise
• Express

• Recognize
• Express

• Possess emotions

5
(1) Infer Users Affective States
Observable Behaviour
Skin Conductivity
Cardiac Rithm
Temperature
Electromyogram
Ocular
Gesture/Facial Expressions
Language (text, dialogue)
Acoustic
Respiration
6
The Cognitive Approach of Emotion

• Emotions are elicited and differentiated on the
  basis of a persons subjective evaluation
  (appraisal) of the personal significance of a
  situation, event or object (Scherer).
• OCC model
• model of classification of 22 emotion which
  describes the cognitive process of evaluation
  that elicits an emotion.

7
OCC Model
VALENCED REACTION TO
VALENCED REACTION TO
CONSEQUENCES OF EVENTS
ACTIONS OF AGENTS
ASPECTS OF OBJECTS
social, moral and behavioral standards
CONSEQUENCES OF EVENTS
ACTIONS OF AGENTS
ASPECTS OF OBJECTS
liking disliking etc.
liking disliking etc.
pleased displeased etc.
Approving disapproving etc.
pleased displeased etc.
Approving disapproving etc.
FOCUSING ON
FOCUSING ON
FOCUSING ON
FOCUSING ON
CONSEQUENCES FOR OTHER
CONSEQUENCES FOR SELF
SELF AGENT
OTHER AGENT
CONSEQUENCES FOR OTHER
CONSEQUENCES FOR SELF
SELF AGENT
OTHER AGENT
PROSPECTS IRRELEVANT
PROSPECTS RELEVANT
PROSPECTS IRRELEVANT
PROSPECTS RELEVANT
DESIRABLE FOR OTHER
UNDESIRABLE FOR OTHER
DESIRABLE FOR OTHER
UNDESIRABLE FOR OTHER
Happy for Resentment
Gloating Pity
Pride Shame
Admiration Reproach
Happy for Resentment
Gloating Pity
Pride Shame
Admiration Reproach
Joy Distress
Joy Distress
FORTUNES-OF-OTHERS
WELL-BEING
ATRIBUTION
FORTUNES-OF-OTHERS
WELL-BEING
ATTRIBUTION
Love Hate
Love Hate
Hope Fear
Hope Fear
ATRACTION
ATRACTION
DISCONFIRMED
CONFIRMED
DISCONFIRMED
CONFIRMED
Gratification Remorse
Gratitude Anger
Gratification Remorse
Gratitude Anger
Satisfaction Fear-confirmed
Disappointment Releaf
Satisfaction Fear-confirmed
Disappointment Releaf
WELL-BEING / ATRIBUTION COMPOUNDS
WELL-BEING / ATTRIBUTION COMPOUNDS
PROSPECT-BASED
PROSPECT-BASED
8
The Cognitive Approach of Emotion

• Emotions are elicited and differentiated on the
  basis of a persons subjective evaluation
  (appraisal) of the personal significance of a
  situation, event or object (Scherer).
• OCC model
• model of classification of 22 emotion wich
  describes the cognitive process of evaluation
  that elicits an emotion.

Events of the world
9
Animated Pedagogical Agents

• The pedagogical agents who use the multimedia
  resources to provide to the user an animated
  character with characteristics similar to ones of
  alive intelligent creatures.
• Communication has a more anthropomorphic and
  social nature.
• They must be believable (Loyal and Bates)
• They must have empathy (Hayes-Roth) (Cooper)

10
Previous Work

• PhD Thesis of Patricia Jaques
• Supervisor Rosa Vicari
• Sandwich with Sylvie Pesty

11
Pat (Pedagogical and Affective Tutor)

• Animated Pedagogical Agent
• Infer emotions
• Satisfaction/Disappointment
• Joy/Distress
• Gratitude/Anger
• Shame
• Apply affective pedagogical tactics
• Increase students self-ability
• Motivate student
• Promote intrinsic motivation

From Students actions in the system interface
11
12
Satisfaction and Disappointment

• According to OCC Model
• satisfaction and disappointment are elicited when
  events of the world are appraised (evaluated)
  according to their desirability with respect to
  the users goals.
• Satisfaction
• when one is pleased about the confirmation of the
  prospect of a desirable event.
• Disappointment
• when one is displeased about the disconfirmation
  of the prospect of a desirable event.
• Example student is frustrated because he
  obtained a bad grade in a test.

13
Satisfaction and Disappointmentin our work

• It is necessary to define

• events that can happen

14
Next Step Users Goals
15
What goals does the student have?

• According to Ames (1990), students can have
• Performance Goals (extrinsic)
• they believe that performance is important and
  they want to demonstrate that they have
  abilities. They feel successful when they please
  the teacher or do better than other students,
  rather than when they understand something new.
• Mastery/Learning Goals (intrinsic)
• oriented towards developing new skills and
  abilities, trying to understand their work,
  improving their level of competence, and learning
  new things.
• How to determine students goals?
• Motivated Strategies for Learning Questionnaire
  (MSLQ).
• It is applyed the first time the student access
  the system.

16
Next Step Events Desirability
17
Pat (Pedagogical and Affective Tutor)
Students emotion
Choice of the Affective Tactic
Chosen Tactics
Student motivational orientation
Apply Affective tactics
Event

• Evaluation in the web-based learning environment
  JADE
• Content earth time zones
• Statistical test (t-test) enables us to deduce
  that students who interacted with Pat had better
  performance than students who interacted with
  JADE version without agent.

17
18
Current Work

• PhD Thesis of Edilson Pontarolo
• Supervisor Rosa Vicari
• Co-supervisor Patricia Jaques
• Sandwich with Sylvie Pesty

19
Proposed Work

• A model to infer emotions a student feels towards
  other students during synchronous interaction in
  the context of a collaborative learning game.
• The emotions inference is psychologically based
  on cognitive appraisal theory.
• Pride/Shame
• Admiration/Reproach
• According to OCC model
• Big-Five model of personality traits

20
Collaborative Game
j1_d2
j1_d1
Collaboration
Collaboration
Synchronous competition
Shared problem
Shared problem
j2_d2
j2_d1
21
Collaborative game
Feedback
22
User Affective Model
McCrae Sutin (2007) Roberts Robins
(2000) Basic tendencies (traits) ? Characteristic
adaptation ? Behavior tendencies Ortony,
Clore Collins (1988) Interaction
Appraisal (behavioral standards) ? Attribution
emotions
23
User Affective Model
McCrae Sutin (2007) Roberts Robins
(2000) Basic tendencies (traits) ? Characteristic
adaptation ? Behavior tendencies Ortony,
Clore Collins (1988) Interaction
Appraisal (behavioral standards) ? Attribution
emotions
24
Variables and dependencies employed to infer
students goals and standards
25
Future work

• Improve the inference of emotions by integrating
  a mechanism for recognition of emotions by face
• The information about users emotions is used by
  an Embodied Conversational Agent (ECA), which
  will be integrated to this platform.
• ECAs are intelligent agents with a humanlike
  representation that are able to engage in a
  conversation with humans Cassel and Sullivan
  2000.
• In a learning environment, recognizing the
  students emotions can increase the believability
  of an ECA by making possible to maintain a more
  credible dialog with the students Lester and
  Stone 1997.
• As a consequence, the ECA will have a more
  effective base to support collaboration.

26
Thanks for your attention!
Profa. Patrícia A. Jaques - pjaques_at_unisinos.br ht
tp//www.inf.unisinos.br/pjaques/
27
Big-Five Model
extroverted
introvert
Extroversion
pleasant
Agreeableness
unpleasant
pleasant
Conscientiousness
pleasant
conscientious
Emotional Stability
negligent
Openness to Experience
28
Reconhecimento de Emoções por Face
2
1
3
Identificação da Face na Imagem
Identificação dos Pontos de Características
Faciais
Identificação de Características da Face
4
5
Classificação de Emoções
Identificação de AUs
29
Reconhecimento de Emoções por Face

• Taxas de Sucesso

30
Students peer-related emotions
31
Results and Discussion
Personality traits correlation CT x,y /
x ? T, y ? T, T1..4, x?y µT 
?  r x,y   / 6  0,182
Pearson's product-moment coefficient, r (-1 r
1)
32
Results and Discussion
Goals correlation CO i,j / i ? O, j ? O,
O1..5, i?j µO  ?  r i,j    / 10  0,234 r
Beat_adversaries , Beat_partner 0,485
Standards correlation CN    n,m / n ? N, m ? N,
N1..5, n?m µN  ?  r n,m  / 10  0,266 r
Beat_user, Motivate_user 0,473
33
Results and Discussion
Correlations Traits x Goals CTO t,o / t ? T,
o ? O, T1..4, O1..5 µTO  ?  r t,o  / 20  
0,107 r Stability , Have_Fun 0,340
Correlations Traits x Standards CTN t,n /
t ? T, n ? N, T1..4, N1..5 µTN  ?  r t,n  
/ 20  0,142 r Extroversion ,
Standard_Motivate_User 0,320
34
Results and Discussion
Fishers Exact Test (FET)
pvalue (0 pvalue 1 ) , given - Fixed
marginal totals - Null hypothesis (A and B
conditionally independent)
35
Results and Discussion
Two-tailed FET results Traits x Goals
36
Results and Discussion
Two-tailed FET results Traits x Standards
37
Results and Discussion
Quantitative Refinement traits x goals, traits x
standards
40 incomplete cases yesnonull 40
completed cases yesno
t,o / t ? T, k ? O t,n / t ? T, n ? N
Estimation-Maximization (EM) Algorithm
Lauritzen (1995)
Conditional Probability Tables P( Goals Traits
) P( Standards Traits )
38
Results and Discussion
Attribution emotions users actions
39
Results and Discussion
Quantitative Refinement emotions x users
actions
351 incomplete cases yesnonull 351
completed cases yesno
k,s / k ? (PA U N) , sProud ? sShame
EM Algorithm
Conditional Probability Tables P( Proud
Standards ? users actions ) P( Shame
Standards ? users actions )
40
Results and Discussion
Attribution emotions partners actions
41
Results and Discussion
Quantitative Refinement emotions x partners
actions
351 incomplete cases yesnonull 351
completed cases yesno
k,s / k ? (PP U N) , sAdmiration ?
sReproach
EM Algorithm
Conditional Probability Tables P( Reproach
Standards ? partners actions ) P( Admiration
Standards ? partners actions )
42
Pedagogical Agents

• Nowadays, many educational systems are being
  implemented using an agent paradigm.
• These intelligent agents that have an educational
  or pedagogical role to facilitate or improve
  learning are called Pedagogical Agents Gurer
  1998.

• ITS modelled using an multi-agent approach

• Animated Pedagogical Agents

Animated Pedagogical Agents
43
Collaborative Game
Collaboration Competition Protocol
Socket TCP/IP
Internet
client
server
44
Affective Computing