Memory across Eye-Movements: 1/f Dynamic in Visual Search

1
Memory across Eye-Movements 1/f Dynamic in
Visual Search
Deborah J. Aks UW-Whitewater Department of
Psychology

• April 1, 2003 Chaos Complex Systems Seminar
• August 2001. Society for Chaos Theory in
  Psychology the Life Sciences.
• April 2002 Math 991- A. Assadi-- UW Vision
  Course)

2
Collaborators
Gregory Zelinsky SUNY- Stonybrook
Julien C. Sprott UW-Madison
Aks, D. J. Zelinsky G. Sprott J. C. (2002).
Memory Across Eye-Movements 1/f Dynamic in
Visual Search. Nonlinear Dynamics, Psychology and
Life Sciences, 6 (1), 1-15.
3
What guides eye-movements during complicated
visual search?
4

• Memory?
• Are there correlations across sequence of
  fixations?

• Deterministic rules?
• Simple set of neuronal interaction rules (e.g.,
  SOC) ?

5
Find
6
(No Transcript)
7
(No Transcript)
8
Do we keep track of where we look?
Is there memory in search?
9
Horowitz, T.S. Wolfe, J. M. (1998). Visual
Search has no memory. Nature, 357, 575-577.
Finding Random repositioning of stimuli
does not affect search RTs
No memory?
10
Overview

• QUESTIONS.
• What guides complicated eye movements?
• Random or non-random process?
• Is there memory across fixations?
• METHOD OF TESTING.
• Challenging visual search task
• KEY ANALYSES
• Power law relation?
• Coloring of noise --gt Memory across eye-movements
• Fourier analysis
• Iterated Functions Systems (IFS) Test

11

• RESULTS
• Raw fixations --gt short term memory (1/f2 brown
  noise)
• Fixation differences --gt long term memory (1/f
  pink noise)
• MODEL. Self-organized criticality (SOC)
  (Bak, Tang, Wiesenfeld, 1987)
• CONCLUSION
• There is memory across eye-movements!
• SOC model predicts relative eye movements.

12
Horowitz, T.S. Wolfe, J. M. (1998). Visual
Search has no memory. Nature, 357, 575-577.
Finding Random repositioning of stimuli
does not affect search RTs
Key Press RTs vs. Eye Movements
13
What does visual search teach us?

• Cognitive processes!
• Speed Accuracy
• Mechanisms
• Automatic or Attention
• Search strategy
• Parallel, Serial, random or?

14
Features...

• Find the odd item
• Discriminate by..
• Color xxxxxxx
• Size xxxxxxx
• Orientation ------l---
• Depth
• Movement xxxx--gt

x

15
Look for the red L
L
L
L
L
L
16
L
L
L
L
L
L
L
L
L
17
Feature search is easy!

• Fast (300ms)
• Parallel (0-10ms/item)
• No attention needed

500
400
300
0 ms/item
5
10
15
of items
18
Conjunction Search

• Find...combination of features
• 2 orientations (particular arrangement)
• Find L among Ts

T
T
L
T
T
19
T
T
T
T
T
T
T
T
T
L
T
T
T
20

21
T
T
T
T
L
T
T
T
T
T
T
T
T
22
Conjunction search is hard!

• Slower
• Sequential
• Focused attention needed

Conjunction
40 ms/item
700
500
Feature
0 ms/item
300
5
10
15
of items
23

• Feature search is easy
• Fast (300ms)
• Parallel (lt10ms/item)
• Focused attention not needed (i.e.,Can use a
  distributed form of attention )
• Conjunction search is difficult
• Slow (gt500ms)
• Serial (gt10ms/item)
• Focused attention needed

24
(No Transcript)
25
(No Transcript)
26
What guides search?

• Environmental information.

27
(No Transcript)
28
(No Transcript)
29
What guides search?

• Environmental information.
• Internal cognitive process
• Attention.
• Memory?
• Deterministic Process Self-Organized
  Criticality (SOC)?

30
Memory in visual search?
We are able to keep track of where we
look!Inhibition of return (Klein, 1982)
Failure to replicate inhibition of return (Wolfe
Pokorny, 1990)

• Random repositioning of stimuli does not affect
  search RTs
• (Horowitz Wolfe, 1998)
• Inattentional amnesia in search (Wolfe, 1999)

• Memory for locations in search (Kristjansso,2000)
  
• Identity of objects accumulates over time
  (Treisman Gelade, 1980)

31
Non-systematic eye-movements
Engle, 1977 Ellis Stark, 1988 Scinto
Pillalamarri, 1986 Krendel Wodinsky, 1960
Groner Groner, 1982
32
Visual Search Task
Find the upright
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
33
Method.

• Each trial contained 81 Ts.
• 400 trials lasting 2.5 hours.
• 1 second central fixation
• Eight 20-minute sessions separated by 5-minute
  rest
• Generation V dual purkinje-image (DPI) tracker

34

• Map trajectory of eyes
• Duration x,y coordinates for each
  fixation.----------------------------------------
  ------------------
• Differences between fixations
• xn xn1 yn yn1
• Distance (x2 y2)1/2
• Direction Arctan (y/x).

35
Analyses

• Descriptive Correlational Statistics

• Power spectra (FFT)

• Iterated Function Systems (IFS) test

36
Results

• 24 fixations per trial (on average)

• 7.6 seconds (SD 6.9 sec) per trial (316
  ms/item).

• Mean fixation duration 212 ms (SD 89 ms)

• 10,215 fixations across complete search
  experiment.

37
Series of Fixation Differences (yn1- yn)
38
Scatter plot of 10,215 eye fixations for the
entire visual search experiment.
Eye Fixations
39
Delay Plot of Fixations yn -vs- y n1
40
Across 8 sessions we see scaling properties

• Fixation frequency decreased from 1888 to 657

• Fixation duration increased from 206 to 217 ms.

• Fixation differences
• xn xn1 decreased
• yn yn1 increased

41
Spectral analysis Fast-Fourier Transform (FFT)
Power vs. Frequency Regression slope power
exponent f a
f -2 1/ f 2 Brown noise
42
Power law indicates

• Adaptive fractal properties
• Scale invariance
• Flexible system

• Strength of memory
• Steepness of the slope (on a log-log scale)
  reflects..
• correlation across data points
• duration of memory

43
1/f 0 noise -- flat spectrum no correlation
across data points
White Noise
Pink Noise
1/f noise --shallow slope extremely long term
correlation
Brown Noise
1/f 2 noise-- steep slope short-term
correlation.
44
Power Spectra on raw fixations
???????
45
???????
46
Power Spectra of first differences across
fixations
? -.6
47
Distance across eye fixations (x2 y2) 1/2
? -.47
? -0.3
? -1.8
48
Iterated Function Systems --IFS Test--(Peak
Frame, 1994 Stewart, 1989).
Fixation Series
49
Start
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55
(No Transcript)
56
(No Transcript)
57
(No Transcript)
58
(No Transcript)
59
(No Transcript)
60
(No Transcript)
61
(No Transcript)
62
(No Transcript)
63
(No Transcript)
64
(No Transcript)
65
(No Transcript)
66
(No Transcript)
67
(No Transcript)
68
(No Transcript)
69
(No Transcript)
70
(No Transcript)
71
(No Transcript)
72
(No Transcript)
73
(No Transcript)
74
White Noise
1/f ?
PinkNoise
1/f ?
Brown Noise
1/f ?
75
Raw Fixations
Clustering along diagonals reveals short-term,
highly correlated consecutive data points
Brown(ish) noise
76
Fixation differences
Triangular microstructure associated w/
long-term, loosely correlated consecutive data
points
Pink(ish) noise
77
IFS Test Fixation direction
Anti-correlated
Clustering indicates correlated fixations.
Direction of fixations show anti-correlated
movements a indicated by absence of main
diagonals.
78
IFS tests yields patterns consistent w/ FFT
results
79
Summary of results

• Sequence of
• Absolute eye positions --gt 1/f brown noise
• Short-term memory.
• Differences-between-fixations --gt 1/f pink
  noise
• Longer-term memory.

80
Model

• Hebb, 1969 Rummelhardt McClelland, 1985
• Neuronal interactions ---gt

implicit guidance
Could eye movements be described by a simple set
of neuronal interaction rules (e.g., SOC) that
produce 1/f behavior?
81
SOC Network(Adapted from Bak, Tang,
Wiesenfeld, 1987)
4
0
Increasing Neural Activation ---gt
82

• Stimulate 1 neuron

83
Z(x,y) initially stimulated site
Threshold rule For Z(x,y) gt Zcr 3
As individual neurons are activated beyond a
threshold (of 3), activity (4) is dispersed to
surrounding cells.
84
Z(x,y) -gt Z(x,y) - 4
Activity in the original site is depleted to
zero.
85
Z(x,y)-gt Z(x,y) 1
Surrounding activity increases by 1
86
(No Transcript)
87
(No Transcript)
88
4
0
Neural SOC
89
Neural SOC w/ eye movement trails
90
Eye movements are pulled to the site(s) of
greatest activation
91
(No Transcript)
92
(No Transcript)
93
(No Transcript)
94
(No Transcript)
95
(No Transcript)
96
(No Transcript)
97
Simple set of SOC rules..

• For Z(x,y) gt Zcr
• Z(x,y) -gt Z(x,y) - 4
• Z(x 1,y)-gt Z(x 1,y) 1
• Z(x,y 1) -gt Z(x,y 1) 1

can produce

• Complex effective search

98
Mainzer, K. (1997). Thinking in complexity The
complex dynamics of matter, mind mankind.
Berlin Springer. Pg. 128
99
Posner Raichel, 1994). Images of mind. New
York Scientific American Series. W/ citation
in Palmer, S. (1999). Vision Science Photons
to phenomenology. Boston MIT.
100
CONCLUSIONS

• There is memory across eye-movements!

• Neural SOC model --gt 1/f relative eye-movements.

• Simple self-organizing system--gt effective search

101
http//psychology.uww.edu/Aks/papers/AZS01.ppt

• Aks, D. J. Zelinsky G. Sprott J. C. (2002).
  Memory Across Eye-Movements 1/f Dynamic in
  Visual Search. Nonlinear Dynamics, Psychology and
  Life Sciences, 6 (1).

102
Bluebird contributed by www.Sierra foothill.org
103
And thanks to Bob for, among a of things,
getting me to reduce the of slides in this
talk.
Phew, Debs down to 93 slides
104
Refs
References Bak, P. (1996). How nature works the
science of self-organized criticality. New York
Springer-Verlag. Bak, P., Tang, C. (1989).
Earthquakes as a self-organized critical
phenomenon. Journal of Geophysics Research -
Solar. Earth Planet, 94, 15635-15637. Bak, P.,
Tang, C., Wiesenfeld, K. (1987).
Self-organized criticality An explanation of
1/f noise. Physical Review Letters, 59,
381-384. Bak, P., Tang, C., Wiesenfeld, K.
(1988). Self-organized criticality. Physical
Review A, 38, 364-374. Ellis, S. R., Stark, L.
(1986). Statistical dependency in visual
scanning. Human Factors, 28(4),
421-438. Gilden, D. L. (1996). Fluctuations in
the time required for elementary decisions.
Psychological Science, 8 (4), 296302.
105
Gilden, D. L., Thornton, T., Mallon, M. (1995).
1/f noise in human cognition. Science, 267,
1837-1839. Horowitz, T.S. Wolfe, J. M. (1998).
Visual Search has no memory. Nature, 357, 575-577.
Irwin, D. E. (1993). Memory for spatial position
across saccadic eye movements. In G. dYdewalle
J.V. Van Rensbergen (Eds.), Perception and
Cognition. Irwin D. E., (1996). Integration
and accumulation of information across saccadic
eye movements. In Inui, T., McLelland, J. L.
(Eds., Attention and performance XVI
Information integration in perception and
communication(pp. 125-155). London MIT
Press. Jeffrey, H. J. (1992). Chaos game
visualization of sequences. Computers
Graphics, 16, 25-33. Jonides, J., Irwin, D. E.,
Yantis, S. (1981). Integrating visual
information from successive fixations. Science,
215, 192-194.
106
Kelso, S. (1992). Dynamic Patterns. Cambridge
MIT Press. Klein R. (1988). Inhibitory tagging
system facilitates visual search. Nature, 334,
430-431. Klein R. (1980). Does oculomotor
readiness mediate cognitive control of visual
attention? In R.S. Nickerson (Ed.), Attention
and Performance VIII, (pp.259-276). Hillsdale,
N.J. Erlbaum. Klein R. MacInnes, W.J. (1999).
Inhibition of return is a foraging facilitator in
visual search. Psychological Science, 10,
346-352. Rensink, R., ORegan,J.K., Clark,J.J.
(1997). To see or not to see The need for
attention to perceive changes in scenes.
Psychological Science, 8, 368-373. Krendel, E.
S., Wodinsky, J. (1960). Search in an
unstructured visual field. Journal of the
Optical Society of America, 50, 562-568.
Kristjansson, A. (2000). In search of
Remembrance Evidence for memory in visual
search. Psychological Science, 11(4), 328-332.
107
Mainzer, K. (1997). Thinking in complexity The
complex dynamics of matter, mind mankind.
Berlin Springer. Mandelbrot, B. B. (1982). The
fractal geometry of nature. SF Freeman. Matin,
L. (1974). Saccadic supression A review and
analysis. Psychological Bulletin, 81,
899-917. Maylor, E. (1985). Facilitatory and
inhibitory components of orienting in visual
space. In M.I. Posner B.B. Marin (Eds.),
Attention and performance XI (pp. 189-204).
Hillsdale NJ Erlbaum. Maylor, E. A., Hockey,
R. (1985). Inhibitory component of externally
controlled covert orienting in visual space.
Journal of Experimental Psychology Human
Perception Performance, 11, 777-787. Megaw.
E.D. Richardson, J. (1979). Target uncertainty
and visual scanning strategies. Human Factors,
21, 303-315. Miller, S. L., Miller, W. M.,
McWhorter, P. J. (1993). Extremal dynamics A
unifying physical explanation of fractals, 1/f
noise and activated processes. Journal of
Applied Physics, 73, 2617-2628.
108
Peak, D., Frame, M. (1994). Chaos Under
Control The Art and Science of Complexity. New
York W. H. Freeman and Co.. Scinto, L.,
Pillalamarri, R., Karsh, R. (1986). Cognitive
strategies for visual search. Acta Psychologica,
62, 263-292. Sprott, J. C., Rowlands, G.
(1995). Chaos Data Analyzer. Raleigh, NC Physics
Academic Sofware (American Institute of
Physics). Stassinopoulos, D., Bak, P. (1995).
Democratic Reinforcement. A principle for Brain
function. Physical Review E, 51,
5033. Treisman, A., Gelade, G. (1988). A
feature integration theory of attention.
Cognitive Psychology, 12, 97-136. Watson, A. B.
(1987). Efficiency of an image code based on
human vision. Journal of the Optical Society of
America, A 4 (12), 2401-2417. Wolfe, J. M.
(1994). Guided search 2.0 A revised model of
visual search. Psychonomic Bulletin Review 1,
202-238.
109
(No Transcript)
110
(No Transcript)
111
(No Transcript)
112
(No Transcript)
113
(No Transcript)
114
SOC neuronal activity w/o eye movement trails
115
(No Transcript)
116
(No Transcript)
117
(No Transcript)
118
(No Transcript)
119
(No Transcript)
120
(No Transcript)
121
(No Transcript)
122
(No Transcript)
123
(No Transcript)
124
(No Transcript)
125
(No Transcript)