Situation Models in Comprehension, Memory, and Augmented Cognition

1
Situation Models in Comprehension, Memory,and
Augmented Cognition

• G.A. Radvansky David Copeland
• University of Notre Dame
• Joe Magliano
• Northern Illinois University

2
Situation Models and Augmented Cognition

• Majority of situation model research focuses on
  narrative discourse.
• How can we apply this to augmented cognition?

3
Situation Models

• Mental representations of events
• Situations in the world are made up of a complex
  of multiple components.
• Situation models should reflect this complexity.

4
Situation Models

• Situation model components
• Spatial temporal framework
• Spatial location
• Time period
• Entities
• Entity properties
• Physical characteristics
• Goals
• Emotional states
• Identity
• Et cetera

5
Situation Models

• Situation model components (contd)
• Structural relations
• Social relations
• Spatial relations
• Ownership relations
• et cetera
• Linking relations
• Temporal relations
• Causal relations

6
Areas of Study (2002-2003)

1. Open-ended coach comments
2. Experimentally controlled coach comments
3. Social experiences

7
Open-ended Comments

• On-going and interactive presentation.
• (Radvansky and Copeland)
• Person is an active participant in the situation.
• People played video games along with a coach.
• How are the coachs comments related to player
  success in the context of the situation?

8
Open-ended Comments

• Procedure
• Player sat in one room.
• Coach in adjacent room with only a monitor.
• Coach and player communicated via a
  microphone-headset system.
• Game play and conversations recorded to DVD.

9
Open-ended Comments

• Master of the Skies The Red Ace

10
Open-ended Comments

• Mission goals
• Single ground target
• Secondary goal destroy enemy fighters
• Multiple ground targets
• Secondary goal destroy enemy fighters
• Destroy enemy fighters (no ground targets)
• Secondary goal none
• DV probability of mission goal success

11
Open-ended Comments

• Quake II

12
Open-ended Comments

• Mission goals
• Primary goals
• Move through an area to the exit
• Kill enemy soldiers
• Secondary goals
• Pick up health, weapons and ammunition
• DV probability of mission goal success

13
Open-ended Comments

• Risk 2

14
Open-ended Comments

• Mission goals
• Primary goals
• Defend own capital country
• Conquer enemy capital countries
• Secondary goals
• Conquer other enemy countries
• Defend other own countries
• DV probability of mission goal success

15
Open-ended Comments
Coachs comment types

• Episodic
• Misinterpretation
• Knowledge Gap
• Spatial Attention
• Action Suggestion

• Status Attention
• Emotion Control
• Positive Reinforcement
• Negative Comment
• Other Comments

16
Open-ended Comments

• Spatial Attention
• Directing players attention to a region in
  space.
• to your left, on the ground, or past the
  bridge
• Action Suggestion
• Suggesting that the player take an action.
• fire!, pull up, or use your bombs
• Status Attention
• Directing the players attention to their
  physical status, or status in the goal plan.
• youre hit, missed him, or new fighters
  present

17
Open-ended Comments

• Proportions of comments
• Red
  Ace Quake Risk
• Episodic .03 .05 lt.01
• Misinterpretation .02 .03 .02
• Knowledge Gap .08 .15 .07
• Spatial Attention .11 .14 .09
• Action Suggestion .20 .28 .36
• Status Attention .29 .12 .24
• Emotion Control .01 lt.01 lt.01
• Positive Reinforcement .11 .05 .03
• Negative Comment lt.01 lt.01 lt.01
• Other Comments .15 .19 .07

18
Open-ended Comments

• Comments per minute.
• Failure Success
• Red Ace 3.36 4.02
• Quake 1.74 2.64
• Risk 5.64 6.12

19
Open-ended CommentsRed Ace (aerial combat)

• Enemy Fighters
• 1 Target gt 1 Target No Targets
  Targets
• Episodic --- --- --- ---
• Misinterpretation --- --- ---
  ---
• Knowledge Gap --- --- --- ---
• Spatial Attention .04 .04 .06
  ---
• Action Suggestion --- .02 .08
  ---
• Status Attention --- .06 .02
  .007
• Emotion Control --- --- .09 ---
• Positive Reinforcement .001 .002 lt.001
  .04
• Negative Comment --- --- --- ---
• Other Comments --- --- .04 .04
• p values

20
Open-ended CommentsQuake (ground combat)

• Hits Spatial Enemies
  Items
• on self shifts killed
  picked-up
• Episodic lt.001 .007 lt.001 .001
• Misinterpretation --- --- .003
  lt.001
• Knowledge Gap lt.001 .04 lt.001 .001
• Spatial Attention .01 .08 .007 .04
• Action Suggestion .001 --- lt.001
  .002
• Status Attention --- .08 .04 .04
• Emotion Control --- --- .06 .06
• Positive Reinforcement --- --- ---
  ---
• Negative Comment --- --- --- ---
• Other Comments .04 --- .001 .06
• p values

21
Open-ended CommentsRisk (combat strategy)

• Change Change Enemy Battles
• in threat in power Attacks won
• Episodic --- --- --- ---
• Misinterpretation --- --- --- ---
• Knowledge Gap --- --- --- ---
• Spatial Attention --- .02 --- .09
• Action Suggestion --- .04 --- .08
• Status Attention --- .08 --- ---
• Emotion Control --- --- --- ---
• Positive Reinforcement --- --- ---
  ---
• Negative Comment --- --- --- ---
• Other Comments --- --- --- ---
• p values

22
Conclusions

• Comprehension of an interactive situation
  involves many situational factors.
• The relative importance of the situational
  components is a function of
• The complexity of the situation.
• The dynamics of the situation.
• The type of goals of the person.

23
Controlled Comments

• Coaches were experimenters.
• Comments were delivered using a pre-arranged
  scheme.
• Comments given in blocked format
• Spatial Attention
• Status Attention
• Episodic Reminders
• Action Suggestion
• Control (no comments)
• 6-7 minute blocks
• Random order

24
Controlled Comments

• Experiment 1
• Comments given every 5 seconds
• Situation status not considered
• n 29

25
Controlled Comments

• Results
• (per block)

Control Episodic Spatial Status Suggestion
Wins 1.07 0.72 0.69 0.76 1.14
Crashes 4.07 3.93 4.00 4.03 3.69
significant marginally significant
26
Controlled Comments

• Results
• (per block)

Control Episodic Spatial Status Suggestion
Enemy planes 10.07 10.86 10.10 7.66 9.03
Targets destroyed 0.29 0.22 0.19 0.30 0.26
Hits taken 29.31 31.00 30.52 28.17 26.66
significant marginally significant
27
Conclusions

• Giving comments every 5 s does not help
  performance.
• Performance was somewhat worse in the comment
  conditions relative to the control.
• Dumb augmented systems can hurt performance.

28
Controlled Comments

• Experiment 2 (in progress)
• Comments given under 2 conditions
• Helpful comments
• Non-foregrounded situational information
• Primary goal oriented
• n 17
• Non-helpful comments
• Foregrounded situation information
• Secondary goal oriented
• Players were NOT deliberately mislead
• n 14

29
Controlled Comments

• Results
• (per block)

Condition Control Episodic Spatial Status Suggestion
Good Wins 0.12 0.29 0.71 0.59 0.29
Crashes 5.35 4.71 4.53 4.65 4.53

Bad Wins 0.57 0.36 0.64 0.43 0.43
Crashes 4.29 4.64 4.29 4.00 4.50
significant marginally significant
30
Controlled Comments

• Results
• (per block)

Condition Control Episodic Spatial Status Suggestion
Good Enemy planes 4.24 5.59 4.65 3.88 5.00
Targets dest. 0.11 0.12 0.24 0.19 0.11
Hits taken 33.71 33.53 32.41 27.59 29.65

Bad Enemy planes 6.07 7.71 5.07 5.57 5.71
Targets dest. 0.17 0.22 0.14 0.18 0.15
Hits taken 31.29 38.50 30.71 30.86 31.14
significant marginally significant
31
Conclusions

• Non-helpful comments do not affect performance
  (compared to control condition).
• Situationally-oriented helpful comments
  positively affect performance.

32
Controlled Comments

• Surprise memory tests given after main task was
  completed.
• Tested memory for various parts of the
  experienced situations.

33
Controlled Comments

• Memory for self

Which plane was Airco Bristol Spad Sopwith
fastest 2 6 10 11
most maneuverable 3 9 7 10
heaviest 9 4 8 8
34
Controlled Comments

• Entities and Entity Properties

How many Mean Actual
tanks in France (8)? 4.54 5
bombs to destroy bridge? 4.46 4-5
bombs to destroy hanger? 1.14 1
AA guns in France (7)? 7.17 8
Allies in the African Desert (14)? 2.66 1
Allies in France (7)? 1.52 1
Enemy planes in France (4)? 8.86 4
Enemy planes around the POW camp (18)? 4.07 4
Did the AA guns ? Fixed Swivel
15 14
35
Controlled Comments

• Spatial relations

Starting Missions In Air Take-off
Desert Bridge 10 19
France Bridge 21 8
France AA Guns 19 10
POW Camp 17 12
36
Controlled Comments

• Spatial relations

Question Left Right Front Behind
Where are the tanks (8)? 8 15 6 0
Where are the oil wells (17)? 14 11 2 2
Where are the hangers (2)? 4 15 10 0
Where is the ocean (13)? 9 4 12 4
Where is the viaduct (22)? 5 2 22 0
River Bridge Town Lake
What is the landmark coming out of the mountains (7)? 9 2 12 6
37
Social Experiences

• Inferring the goals of strangers in unsolicited
  social interactions
• (Magliano)
• To what extent do people use situational
  components to infer anothers goals?

38
Social Experiences

• Types of stranger goals
• Social facilitation
• Become friend
• Romantic encounter
• Solicitation of goods and services
• Want money
• Car battery jump
• Harming
• Rob
• Kill (that means you!)

39
Social Experiences

• Situation components available
• Space
• Bar, classroom, or dark alley
• Time
• Early morning, afternoon, or late evening
• Characteristics of self
• Casually dressed, calm, or scared
• Characteristics of the stranger
• Casually dressed, angry, or gender
• Actions of the stranger
• Introduces self, hands a pamphlet, or yells at you

40
Social Experiences
Task
CUE X GOAL MATRIX

• Experiment 1 Eliciting Cues Associated with
  Stranger Goals
• Consider a goal and
• and answer questionnaire
• that elicits situational cues associated with
  it.
• E.g., Goal Rob you
• Where are you?
• What time is it?
• What does the stranger
• look like?
• What is the stranger
• doing?

G1 G2 G3 Gn
C1
C2
C3

Cn
This matrix contains data regarding how strongly
and uniquely a cue is associated with a goal
41
Social Experiences
Task and Results
CUE X GOAL MATRIX

• Experiment 2 Assessing extent to which
  individual cues are predictive of goals
• Participants were presented with Cue - Goal pairs
  
• Participants judged how predictive a cue was of a
  goal

G1 G2 G3 Gn
C1
C2
C3

Cn
This matrix contains data regarding how strongly
and uniquely a cue is associated with a goal
42
Social Experiences

• Percent of variance account for in likelihood
  judgments per situation component.
• Space 20
• Time 12
• Self characteristics 7
• Stranger characteristics 34
• Stranger actions 39

43
Conclusions

• People are sensitive to cues when inferring the
  goals of strangers.
• Like text research, there is a hierarchy of
  component importance.
• However, the nature of the hierarchy is different.

44
Take Home Message

• Situation model theory can be extended beyond
  discourse comprehension.
• There are similarities to how situation models
  are constructed, used, and remembered across many
  domains of thought and experience.
• However, there are important differences that
  need to be further explored.