Psychophysics 3

1
Psychophysics 3

• Research Methods
• Fall 2009
• Tamás Bohm

2
Signal detection theory

• Aka. sensory decision theory (SDT)
• A model a data analysis method for decision
  problems with uncertainty (noise)
• Originates from World War II aircraft detection
  on radar signals
• Today widely used in psychophysics, medicine,
  radiology and machine learning

3
Signal detection theory

• Experiment setup
• In some trials a stimulus (signal) is presented,
  in others there is no stimulus
• Observer reports if she/he saw a signal or not
• Calculate how many times the observer detected a
  signal when she/he was presented one (hit rate)
• Is the hit rate all we want to know?
• Two observers achieved the same hit rate. Are
  they certainly behaving the same way?
• NO, we also need to know how many times the
  observer said I see when there was no signal
  (false alarm rate)

4
Signal detection theory

• Confusion matrix contains all the information
  about the observers performance

5
Signal detection theory

• Confusion matrix contains all the information
  about the observers performance
• As columns add up to 100, it is enough to know
  one item from each column

40 trials
20
20
100
100
18
6
90
30
2
14
10
70
6
Signal detection theory

• Perfect detection

100
0
100
0
7
Signal detection theory

• No detection at all (1st example) always
  reporting Seen

100
100
0
0
8
Signal detection theory

• No detection (2nd example) always reporting Not
  seen

0
0
100
100
9
Signal detection theory

• No detection (3rd example) flipping a coin

50
50
50
50
10
Signal detection theory

• No detection (4th example) reporting Seen in
  30 of the trials (no matter what is presented)

Rows equal ? no detection
30
30

70
70

11
Signal detection theory

• Receiver operating characteristic (ROC)

100
hit rate
100
false alarm rate
12
Signal detection theory

• Receiver operating characteristic (ROC)

100
hit rate
90
30
10
70
100
false alarm rate
13
Signal detection theory

• Receiver operating characteristic (ROC)

100
Perfect detection
hit rate
100
0
0
100
100
false alarm rate
14
Signal detection theory

• Receiver operating characteristic (ROC)

100
No detection always yes
hit rate
100
100
0
0
100
false alarm rate
15
Signal detection theory

• Receiver operating characteristic (ROC)

100
No detection always no
hit rate
0
0
100
100
100
false alarm rate
16
Signal detection theory

• Receiver operating characteristic (ROC)

No detection reporting yes in 50 of the
trials (flipping a coin)
100
hit rate
50
50
50
50
100
false alarm rate
17
Signal detection theory

• Receiver operating characteristic (ROC)

No detection reporting yes in 40 of the trials
100
hit rate
40
40
60
60
100
false alarm rate
18
Signal detection theory

• Receiver operating characteristic (ROC)

No detection reporting yes in 30 of the trials
100
hit rate
30
30
70
70
100
false alarm rate
19
Signal detection theory

• Receiver operating characteristic (ROC)

No detection reporting yes in 60 of the trials
100
hit rate
60
60
40
40
100
false alarm rate
20
Signal detection theory

• Receiver operating characteristic (ROC)

100
Diagonal no detection
hit rate
100
false alarm rate
21
Signal detection theory

• SDT model
• No way to remove the noise
• But sensation can be separated from decision by
  using ROCs

SL ß
YES
Sensation level (SL)
Signal present/absent
Sensation
Decision
NO
SL lt ß
Noise
Criterion (ß)
22
Signal detection theory
SL ß
YES
Sensation level (SL)
Signal present/absent
Sensation
Decision
NO
SL lt ß
(Noise)
Criterion (ß)
Without noise perfect detection is possible
signal absent
criterion
signal present
probability
sensation level
23
Signal detection theory
SL ß
YES
Sensation level (SL)
Signal present/absent
Sensation
Decision
NO
SL lt ß
(Noise)
Criterion (ß)
signal absent
criterion
signal present
100
0
probability
0
100
sensation level
24
Signal detection theory
SL ß
YES
Sensation level (SL)
Signal present/absent
Sensation
Decision
NO
SL lt ß
Noise
Criterion (ß)
signal absent(noise only)
signal present(signalnoise)
criterion
Noise smears the distributions? perfect
detection is impossible (if the two
distributions overlap)
probability
sensation level
online demo
25
Signal detection theory
Sensation level
Sensation level
http//www-psych.stanford.edu/lera/psych115s/note
s/signal/
26
Signal detection theory
hit rate
Sensation level
false alarm rate
Sensation level
27
Signal detection theory
ß 6
ß 6
ROC curve
ß 8
ß 8
hit rate
ß 10
ß 10
false alarm rate
28
Signal detection theory

• Criterion (ß) specifies where we are on the ROC
  curve
• The ROC curve is specified by sensory capacities
  only(discriminability)

ß
hit rate
false alarm rate
probability
sensation level
29
Signal detection theory

• Discriminability how well the observer can
  separate the presence of signal from its
  absence overlap between the two
  distributions bowing out of the ROC curve
• Measured by d (discriminability index,also
  called sensitivity)

http//www-psych.stanford.edu/lera/psych115s/note
s/signal/
30
Signal detection theory

• d selects the ROC curve
• ß specifies a point on the selected ROC curve
• same information as hit rate false alarm rate,
  but
• hit rate, false alarm rateboth reflect
  sensation decision characteristicscannot
  separate the two
• d depends only on sensation
• ß depends only on decision

ß
The two processes are separated
http//psych.hanover.edu/JavaTest/Media/Chapter2/M
edFig.ROC.html
31
Signal detection theory

• Fechners methods
• Is a stimulus detectable? Yes or no?
• Clear-cut threshold value (with some variability)
  that can be measured
• Stimulus intensity gt threshold ? detectable
• Stimulus intensity lt threshold ? not detectable
• Dichotic outcome, categorical model

• Signal detection theory
• How well is it detectable? How sensitive the
  observer is to the stimulus?
• Measured by d
• The higher d is, the more the stimulus is
  detectable
• d 0 ? not detectable at all
• Scalar outcome, dimensional model

32
Signal detection theory

• Problem with Fechners methods criterion

Sensation level (SL)
Sensation
Stimulus
(Noise)
33
Signal detection theory

• Psychophysical measurements with SDT
• Create a stimulus set with a range of intensities
  (like in the method of constant stimuli)
• Test each stimulus many times with each observer
• On each trial, either present a randomly selected
  stimulus or do not present anything
• Ask the observer if he/she detected the stimulus
• Calculate the hit rate and false alarm rate for
  each observer, for each stimulus intensity
• Use the formula/table to calculate d for each
  case
• Examine how d changes with intensity the higher
  d is for a stimulus intensity, the greater the
  observers ability to detect this intensity

http//psych.hanover.edu/JavaTest/Media/Chapter2/M
edFig.SignalDetection.html
34
Signal detection theory

• Main results changes in d values

CaudekRubin Vision Res. 2001
35
Signal detection theory

• There is also a ß value for each d value
• It can be informative about the decision
  behavior
• Balanced false alarm and miss rates are equal
• Liberal the observer says yes whenever there
  may be a signal
• Conservative decision is yes only when it is
  almost certain that there is a signal

balanced
conservative
liberal
probability
sensation level