context

1
2nd Balkan Vision Science Meeting
September 2008, Slovenia
Aspects of face perception gaze direction and
emotional expression
Dejan Todorovic Laboratory of experimental
psychology Department of psychology University of
Belgrade, Serbia
2
Information carried by faces

• general features
• gender
• race
• age
• specific, particular features
• identity
• current, changeable features
• facial expression
• gaze direction

3
Current features

• Usually depend on the configuration profile of
  facial and head muscles

facial expression
gaze direction
4
Current features

• may also depend on
• placement of inner facial features (eyes, nose,
  mouth) within face outline
• head orientation and inclination
• examples
• simple schematic faces
• more realistic faces

5
schematic faces
gaze direction
straight ahead
note what happens when all inner facial features
(eyes, nose, mouth) are shifted horizontally
leftwards/rightwards
6
schematic faces
gaze direction
perceived gaze direction
straight ahead
leftwards
rightwards
7
schematic faces
'Necker faces' bi-stable gaze direction
red guy looks leftwards, green guy looks at you
looks leftwards
looks at you
8
EXPERIMENTS
STIMULI
iris eccentricity
face eccentricity
9
Experiment 1 design

• stimuli
• 21 portraits ? 8 repetitions 168 trials (random
  order)
• preceded by 21 practice trials
• subject task
• 'does the portrait look at your face or not?'
• response measures
• percent response type ('at my face', 'not at my
  face')
• reaction times
• 15 subjects
• extension of research by Maruyama Endo (1984)

10
Experiment 1 results
Effects of face eccentricity and iris
eccentricity on percentages of looks at me
answers
is compensated by iris eccentricity shift
in opposite direction
face eccentricity shift in one direction
11
Experiment 2 design

• subject task
• 'what is the gaze direction of the portrait, in
  degrees?'
• response measure
• judged angle of gaze direction
• help protractor scheme below the computer screen
• stimuli
• 21 portraits ? 4 repetitions 84 trials
• 15 subjects

• obtained data converted from 0 to 180 range
• to -90 to 90 range
• negative leftwards
• positive rightwards
• zero looks at me

12
Experiment 2 results
Effects of face eccentricity and iris
eccentricity on judged angle of gaze direction

• judged angle of gaze direction increases with
  iris eccentricity
• judged angle increases/decreases with
  positive/negative face eccentricity
• average effect 12 degrees

13
Experiment 2 multiple regression

• independent variables iris eccentricity (IE),
  face eccentricity (FE)
• dependent variable judged angle of gaze
  direction (GD)
• regression equation on means
• GD 1.35 IE 0.63 FE 2.29, R2 0.985
• general model for perceived gaze direction
• GD iwIE fwFE (in concrete case iw1.35,
  fw0.63)
• special case portrait looks at me
• GD iwIE fwFE 0
• this is true in two cases
• (1) IE FE 0
• meaning centered irises in symmetrical face
• (2) IE/FE -fw/iw const
• in concrete case -0.63/1.35 -0.47
• meaning particular counterbalance
  of iris and face
  eccentricity

14
REALISTIC FACES

• Do these findings apply for realistic faces?
• many authors claim that only IE affects perceived
  gaze

Emerys (2000) rule for perception of gaze
direction
not quite true ... none of depicted portraits
are looking in the indicated directions!
15
REALISTIC FACES
irises rightwards
gaze rightwards
gaze at observer
head frontal
head leftwards
16
REALISTIC FACES
irises centered
gaze at observer
gaze rightwards
head frontal
head rightwards
17
REALISTIC FACES
irises leftwards
gaze leftwards
gaze at observer
head rightwards
head leftwards
orientation of head affects perceived gaze
direction
18
REALISTIC FACES
'Necker faces' bi-stable gaze direction
19
Experiment synthetic faces
iris eccentricity
head orientation
task does portrait look at me or not?
20
Results
Effects of head orientation and iris
eccentricity on percentages of looks at me
answers
results are similar as for schematic faces
21
Conclusions

• perceived gaze direction is based not only on
  iris position but also on head orientation cues
• head orientation cues e.g. face eccentricity
• indicates head orientation relative to observer
• iris position indicates gaze direction with
  respect to head
• 'gazing at me' percept based on a counter-balance
  of head and iris cues

22
schematic faces
facial expression
neutral mood
note what happens when all inner facial features
(eyes, nose, mouth) are shifted vertically
bottomwards/topwards
23
schematic faces
facial expression
perceived facial expression
neutral mood
poor mood
good mood
24
Experiment 1 schematic faces
three factors
1. features placement
2. mouth curvature
3. iris location
PRESENTATION 18 faces on sheet of
paper, randomized placement TASK rating
scale rate expression on 7-step bipolar
sad-happy scale SUBJECTS N83
25
Experiment 1 Results
Effects of feature placement and mouth curvature
on facial expression rating

• both feature placement and mouth curvature affect
  rated mood
• no effect of gaze direction

26
Experiment 2 more realistic faces
two factors
1. feature placement
2. mouth curvature
PRESENTATION 18 faces on sheet of
paper, randomized placement (mixed with Exp. 3
figures) TASK rating scale rate expression on
7-step bipolar sad-happy scale SUBJECTS N71
27
Experiment 2 Results
Effects of feature placement and mouth curvature
on facial expression rating

• both feature placement and mouth curvature affect
  rated mood

28
Hypothesis

• possible explanation of effect of vertical
  features shift on perceived facial expression
• feature location is associated with head
  inclination
• head is low / high projection of features is
  low/high
• head inclination is associated with mood
• head is low / high mood is poor / good
• thus feature location is associated with mood

head low
head high
features high
features low
poor mood
good mood
29
Experiment 3 head inclination
two factors
1. head inclination
2. mouth curvature
PRESENTATION, TASK, SUBJECTS as in Experiment 2
CONFOUNDING EFFECT projected mouth
curvature depends on head inclination
30
Experiment 3 Results
Effects of head inclination and mouth curvature
on facial expression rating

• head inclination affects rated mood for sad
  faces
• no clear effect for neutral and happy faces
• result structure replicated in two additional
  experiments

31
Conclusions

• perceived gaze direction and facial expression do
  not depend only on registration of action of
  facial muscles
• they also depend on facial feature placement and
  head orientation / inclination
• horizontal feature placement is a cue of head
  turn
• vertical feature placement may be a cue of head
  inclination, which is associated with mood
• this explanation only applies for sad faces
• no effect for neutral and happy faces
• for such faces, there may exist a facial
  constancy effect (independence from head
  inclination)

32
THANK YOU!
33
Experiment 1 results (reaction times)
Effects of face eccentricity and iris
eccentricity on reaction times to looks at me
answers

• more frequent answers are faster
• reactions to eccentric faces are slower

34
Experiment 1 fitting percentages
fit model normal curve
a peak b spread
-27 -18 -9 0
9 18 27
-27 -18 -9 0
9 18 27
35
Experiment 2 fitting judged gaze angles
fit model straight lines

• almost identical slopes for the three face
  eccentricities
• /- 20 face eccentricity adds/subtracts 12
  degrees

36
Comments

• criticism
• what is the relevance of studies with schematic
  faces for the perception of gaze direction in
  real faces?
• answers
• visual system does not have special mechanisms
  for detection of gaze direction in schematic
  faces
• schematic faces probably activate same face
  perception mechanisms as real faces
• shown in EEG and fMRI studies
• schematic faces are much simpler and more easily
  controlled as stimuli than real faces
• shift of facial features is probably a cue of
  head turn
• nevertheless
• it is of interest to study real(istic) faces

37
REALISTIC FACES
Wollaston (1824)
38
Experiment

• stimuli synthetic faces
• software Poser 5
• model default male head

two factors
1. iris eccentricity 6 levels
2. head turn 4 levels
39
Experiment

• stimulus presentation
• 24 portraits ? 7 repetitions 168 trials (random
  order)
• preceded by 24 practice trials
• subjects three groups of 11
• subject tasks
• group 1 head orientation task
• 'is the head oriented leftwards or rightwards'
• group 2 iris location task
• 'are the irises located leftwards or rightwards
  from center?'
• group 3 gaze direction task
• 'does the portrait look at your face or not?'
• response measure
• percentage of responses

40
Experiment results 1. head task

• 'head rightwards' responses

threshold for head orientation 5 head
turn equivalent to 1.8 degrees head turn
angle agrees with Wilson et al. (2000)
41
Experiment results 2. iris task

• 'irises rightwards' responses

threshold for iris eccentricity 5.3 iris
shift equivalent to 2 minutes of arc within upper
range of previous data
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
Experiment conclusions
possible algorithms of perception of gaze 1.
analytic separate encoding of head and iris
cues combining cues to compute gaze direction 2.
holistic effective stimulus is the whole facial
pattern match pattern to stored template
associated with gaze direction note both
algorithms configurational do not depend on local
information only
46
Experiment results 3. gaze task

• 'gazes at me' responses

47
schematic faces
'Necker face'
48
Experiment 4

• PRESENTATION
• on computer screen
• pairs of faces from Exp. 3
• TASK
• which of the two faces looks happier?
• STIMULI
• all possible pairings of 9 faces
• 36 pairs x 4 replications 144 stim.
• randomized, balanced for side
• SUBJECTS n18
• DEPENDENT VARIABLE
• happiness index for each face
• percentage of times face chosen
• as the happier one in the pair

• RESULTS
• Head inclination affects rated mood for sad
  faces
• No clear effect for neutral and happy faces
• replicated in another experiment