Motion: Psychophysics 1

1
CNS186 (S. Shimojo)
Motion Psychophysics (1) A Dynamic Overview
2
1. Motion as a fundamental visual
dimension -Which comes first? Engineer's view
static snapshots --dynamic image
processing Psychologist's view dynamic
perception -- static perception -Motion as a
visual module Paradox in the waterfall
illusion Motion-selective neurons (Barlow,
1963) Motion blindness (Zihl, von Cramon,
Mai, 1983) Psychophysics (Nakayama Tyler,
1981) Modelling (Reichardt, 1961) Motion
sensation is not a secondary product of space
time, and it is a separate and independent
visual process.
3
2. Functional benefits of image motion
processing What is the biological function of
motion perception? "To identify velocity of
moving object" is a very insufficient answer, as
listed below (Nakayama, 1984). (1)Encoding of
the third dimension -Surface in depth (Rogers
Graham, 1979) and the issue of cues -Motion
parallax (Kitazaki Shimojo, 1996) -Kinetic
depth effect (Wallach O'Connell, 1953) (2)Time
to collision(TTC) -Accurate judgment on time of
collision, without information (Lee, 1976)
4
(3)Image segmentation -Common Fate (Koffka,
1926) -Anti-camoflage (Julesz, 1971) (4)Motion
as a proprioceptive sense (Gibson,
1954) -Development (Lee Aronson, 1974) -Optical
Flow (Nakayama Loomis, 1974 van Dooren
Koenderink, 1986) -Heading perception (Gibson,
1966 Banks, et al., 1996) (5)Motion as a
stimulus to drive eye movements -Vergence/pursuit
as independent mechanisms (Hering's
observation Rashbass, 1961)
5
Vection (Visually-induced sensation of self
motion)

1) ????????????(????????? Lee Aronson, 1974)
14-16 mos. olds
The surrounding visual environment suddenly
moves.
Floor stationary.
6
Vection (Visually-induced sensation of self
motion)

1) ????????????(????????? Lee Aronson, 1974)
14-16 mos. olds
The surrounding visual environment suddenly
moves.
Floor stationary.

Immediately after the onset of spontaneous
walking, vision starts controlling postural
reflexes.
7

• ?????? ----- ??(??)????????????????

Vision is proprioception (J. J. Gibson)? Even
early in development.
National Science Museum, Tokyo
8
(6)Motion as required for pattern
vision -Stablized image (Riggs, Ratliff,
Cornsweet Cornsweet, 1953) -Tradeoff with
spatial frequency (Kelly, 1979) (7)Perceiving
real moving objects -We do not perceive retinal
motion as object motion (pursuit eye movement of
retinal afterimageYasui Young,
1975) -Retinal motion vs. conscious perception of
motion -Illusory differential motion in RDS
(Tyler, 1974) Note that to perceive movement of
object is just a small fraction of the
biological functions.
9
3. Taxonomy -Real / apparent -Short range / long
range (Anstis, 1980 Cavanagh, 1991) -1st order /
2nd order /3rd order (Werkhoben, Sperling,
Chubb, 1993) 1st based on luminance-contrast,
Fourier energy 2nd based on features/saliency,
analogous to 1st 3rd based on attentive
tracking cf. selective motion aftereffect
(Nishida Sato, 1993) -Fourier /
non-fourier -Active / passive (Cavanagh, 1992)
10
4. Integration and segregation -Random-dot
kinematogram(RDK B. Julesz) -Dmax(15 arc min.)
(Braddick, 1974) -Dmin(5 arc sec) (Nakayama
Tyler, 19841) -Temporal integration (Nakayama
Silverman, 1984) -Spatial integration Relative
motion and cortical magnification (Murakami
Shimojo, 1993, 1995, 1996) -Motion in depth
(Regan Bevery, 1973) -Attenuation of motion in
equiluminant condition (Ramachandran
Gregory, 1978) -Motion capture (Ramachandran
Cavanagh, 1987)
11
5. Ambiguity and higher-order -Aperture problem
(Ullman Hildreth, 1983) Sneak effect
(Hildreth, 1983 Nakayama Silverman, 1983)
-Slit view (Parks, 1965 Shimojo Richards,
1985) -Relation to attention (Chaundri, 1990
Hikosaka, Miyauchi Shimojo, 1993a,
b) -Structure from motion (Ullman, 1979) Motion
defines contours and structures, not just vice
versa. -Biomotion (Johansson, 1973)
Development of biomotion perception (Berthenthal,
et al., 1984)
12
(No Transcript)
13
(No Transcript)
14
http//www.biomotionlab.ca/walking.php
15
References Banks, M., Ehrlich, S., Backus, B.,
Crowell, J. (1996) Estimating heading during
real and simulated eye movements. Vision
Research, 36, 431- 443. Bertenthal, B.,
Profitt, D. R., Cutting, J. E. (1984) Infant
sensitivity to figural coherence in
biomechanical motions. Journal of Experimental
Child Psychology, 37, 213-230. Chaudhuri, A.
(1990) Modulation of the motion aftereffect by
selective attention. Nature, 344,
60-62. Cavanagh, P. (1991) Short-range vs
long-range motion Not a valid distinction.
Spatial Vision, 5, 303-309. Cavanagh, P. (1992)
Attention-based motion perception. Science, 257,
1563-1565. Hikosaka, O., Miyauchi, S. and
Shimojo, S. (1993) Voluntary and stimulus-
induced attention detected as motion sensation.
Perception, 22, 517- 526, 1993. Hikosaka, O.,
Miyauchi, S. and Shimojo, S. Focal visual
attention produces illusory temporal order and
motion sensation. Vision Research, 33,
1219-1240, 1993.
16
Murakami, I., Shimojo, S. (1993) Motion capture
changes to induced motion at higher luminance
contrasts, smaller eccentricities, and larger
inducer sizes. Vision Research, 33,
2091-2107. Murakami, I., Shimojo, S. (1995)
Modulation of motion aftereffect by surround
motion and its dependence on stimulus size and
eccentricity. Vision Research, 35,
1835-1844. Murakami, I., Shimojo, S. (1996)
Assimilation-type and contrast-type bias of
motion induced by the surround in a random-dot
display evidence for center-surround
antagonism. Vision Research, 36,
3629-3639. Nakayama, K. (1985) Biological image
motion processing A review. Vision Research,
25, 625-660. Nishida, S. Sato, T. (1993) Two
kinds of motion aftereffect reveal different
types of motion processing. Invest. Ophthalmol.
Vis. Sci., 34, 1363. Lee, D. N., and Aronson, E.
(1974) Visual proprioceptive control of standing
in human infants. Perception Psychophysics,
15, 529-532. Johansson, G. (1973) Visual
perception of biomotion and a model for its
analysis. Perception Psychophysics, 1973, 14,
202-211. Werkhoben, P., Sperling, G. Chubb, C.
(1993) Vision Research, 33, 463-485. (Note
Most of references older than 1984 are included
in Nakayama(1984).)
17
CNS186 (S. Shimojo)
Motion Psychophysics (2) Motion, Changes
and Positions
18

• 1. Real motion vs. apparent motion.
• What is real motion? From the point of vision it
  may not be easy
• to define real motion, but it is certainly one
  property of the world
• which has contributed significantly to the
  evolution of brains in
• all animals.
• Reference frames. All motion is relative.
  Inertial frame A frame
• in which you can do no experiment, physical or
  psychophysical,
• to determine how (or whether) the frame is
  moving.
• Experiments in support of why apparent motion
  (such as seen in
• movies) may be considered similar to real
  motion (Barlow, 1979
• Burr, 1979).
• - Experiments supporting why it may be considered
  different.

19

• 2. Motion and snapshots.
• - Two sides to real motion Flashes and
  Translation. From the point of view of vision,
  is the flash of the bar a true "part" of the
  trajectory of the moving bar?
• - Three different types of experiments show that
  the above part/whole
• distinction is not justified i.e. Motion
  process is special.
• 1) A very brief (1 sec) flash of light persists
  in vision for about 120 msec
• (Di Lollo, 1977). A moving object does not
  appear spatially smeared such that
• the extent of the smear is equal to the product
  of object velocity and 120 msec
• (Burr, 1980).
• 2) Parts of the receptive field that produce the
  maximum response in a cell are
• different for moving objects and flashed
  objects the wave of cell activity
• travels ahead of the moving edge (Berry et al,
  1997).
• 3) A flashed item presented in alignment with a
  smoothly moving item produces
• the "flash-lag effect" in which the flashed
  item appears to lag the moving item
• (The flash lag effect Nijhawan, 1997).

20

• - Discrete vs. Continuous sensory environments
  The often mentioned
• analogy between a camera and the visual system
  fails in most cases
• (unless the sensory environment consists
  entirely of flashes).
• 3. Flash lag effect.
• - Motion extrapolation and color decomposition
  (Nijhawan, 1997).
• Is object motion necessary for producing the
  flash-lag effect (or for
• motion extrapolation to work) or retinal image
  motion (e.g. caused by
• smooth pursuit) sufficient?
• Why does the world appear stationary when the
  retinal image moves
• due to eye-movements? Autokinetic effect.
  Double-flash experiment
• (Nijhawan Shimojo, in preparation).
• - Discrete vs. Continuous sensory environments
  The often mentioned
• analogy between a camera and the visual system
  fails in most cases
• (unless the sensory environment consists
  entirely of flashes).

21
Flash Lag (Motion Extrapolation)
(1) Complete
(Nijhawan, 1994) demo?
Perceived as synchronized
Flash
Possible accounts (a) Spatial extrapolation
(Nijhawan, 1994) (b) Temporal delay (Whitney
Murakami, 1998) (Both of them implicitly
assume exclusive role of pre-trajectory.) (c)
Postdiction (Eagleman Sejnowski, 2000)
22
Flash lag effect - demo.
23
(2) Flash Initiated (Khurana Nijhawan,
1995)
Full Flash Lag!
(3) Flash Terminated
No Flash Lag!
No theory based on pre-trajectory can not account
for these, thus supporting the postdiction
idea (Eagleman Sejnowski, 2000).
24
Flash Lag Effect in Color
Physical
Stimulus
FP
Color smoothly changing
Flash
t
t
Perceptual
Duration of each frame 73ms Total sequence 2.2
sec.
Delay of flash relative to the color-changing
object, leading to apparent mismatch of colors.
25
Color Flash Lag Effect Results
100
G-R
trials in which flash disk is perceived to be
greener
2 x FLE
R-G
50
Flash Lag Effect (ms)
0
-876 0 876 (ms)
Color flash lag 400ms
Color of flashed disk
26
Color
400 (ms) Luminance 40 Spatial
Frequency 80 Entropy 95
Flash Lag in Entropy
Orderly
Disorderly
Flash
t
100 50 0
2 X FLE
of trials in which the flashed patch is
perceived to be more disorderly
-438 0
438 (ms)
27

• Flash Lag and Generalized Flash Lag
• The direction of the delay effect, the same i.e.
  flash?lag,
• continuously changing feature?ahead (although
  amount of
• the delay varies wildly).
• (2) Flash initiation condition -- The same
  effect (Flash
• termination condition -- No effect) ?
  Postdiction?
• (3) Shared neural mechanisms (?)
• - Evidence against adaptation, persistence,
  and attention.
• - Priming? Backward masking?

• Postdiction, or some mechanism to re-construct
  the near-past as
• present.
• - Not just limited to motion perception.

28
To generalize further??
Perceiving-the-Present (Changizi,
Nijihawan Shimojo, Cognitive Science, in press)

• 1) Flash-lag and Generalized Flash-lag can be
  seen as a compensation mechanism in the brain,
  that compensates its own neural delay (can not
  compensate in the case of flash).
• 2) So far, having a dynamic and smooth change in
  the stimulus is the necessary condition, but
• 3) What if, even with a snapshot the brain tends
  to perceive a near future?
• 4) Such a prediction turned out to be consistent
  with a variety of visual illusions (both dynamic
  and static).

29
Hering illusion
Radiational lines everywhere in the real world
1)Artificial (roads, buildings, sky, etc.)
2)Observer locomotion --optical flow
smearing. In most cases, the locomotion direction
overlaps with the vantage point. Prediction by
the Perceiving-the-present(under this
assumption) We perceive the projected image in
the next moment (to compensate The delay). eg.
when passing through the door entrance.
30
????????????????(?????????????? ???)??????????????
?????????????
Hering, Ponzo Orbison Illusions
Ponzo
Orbison
Hering
31

• Backward Perceptual Phenomena
• - Backward masking
• - Color Phi smooth color change in apparent
  motion (Kolers von Gruenau,76)
• - Cutaneous rabbit (Geldard Sherrick, 72)
• 1) Flash lag effect (particularly the flash
  initiation cases).
• 2) Generalized FLE.
• 3) Perception of Present.
• 4) Auditory-induced visual rabbit.
• 5) Backward motion capture.
• 6) Large time window of optimal TMS delay for
  scotoma.
• 7) Backward filling-in by TMS.
• 8) Replay effect by dual TMS, and flash lag.
• cf. Backward referral (Libet, et al., 79)

32
Backward Motion Capture (Kamitani, 1997)
Inertia
x
90 deg. shift
(unambiguous)
Capture? - YES.
180 deg. shift
(ambiguous)
t
Prior unambiguous motion captures posterior
ambiguous motion.
Backward Inertia?
x
180 deg. shift
Capture Backward?
(ambiguous)
90 deg. shift
(unambiguous)
t
Posterior unambiguous motion captures prior
ambiguous motion?
33
x
180 deg. shift
Task Where did the marked stripe go?
90 deg. shift
t
-4
Duration of 2nd frame
100
Duration of 2nd frame Capture! 100ms Ambiguous
(balanced).
HM
50
Response
30 m
s
0
60 m
s
90 m
s
120 m
s
150 m
s
180 m
s
4
3
-1 1
-2
-3
-4
2
34

• - Main accounts of the flash lag effect
• 1) Spatial extrapolation (Nijhawan, 1993)
• 2) Temporal lag (Whitney Murakami, 1999)
• 3) Postdiction (Eagleman Sejnowski, 1999)
• 4. Generalized flash lag effect.
• The flash lag is generalized to visual attributes
  other than motion,
• such as color, luminance, and "entropy." (Sheth
  Shimojo, 2000)
• - A masking/priming based account?
• - Asynchronous binding (Chi, 2000)
• The flash initiated case yields the typical flash
  lag, whereas the
• flash terminated case does not (in favor for
  the postdiction idea?).
• 5. Perceiving-the-present
• - Perhaps the entire purpose of the visual
  processing is not just to
• recover information about the distal stimulus,
  but rather (or also)
• to recover the proximal stimulus at present
  moment with the neural
• delay compensated by extrapolation (Changizi,
  Nijhawan Shimojo).

35
References Barlow, H. (1979) Reconstructing
the visual image in space and time. Nature
279, 189-190. Berry et al (1997) Anticipation of
moving stimuli by the retina. Society for
Neuroscience (Abstracts) 23, 763.5. Burr, D.
(1980) Motion smear. Nature 284, 164-165. Burr,
D. (1979) Acuity for apparent vernier offset.
Vision Research 19, 835-837. Di Lollo, V. (1977)
Temporal characteristics of iconic memory.
Nature 267, 241-243. Nijhawan, R. (1997)
Visual decomposition of colour through motion
extrapolation. Nature 386, 66-69. Sheth, B.,
Nijihawan, R. and Shimojo, S. (2000) Changing
objects lead briefly flashed ones. Nature
Neuroscience, 3, 489-495, 2000.
36
(No Transcript)