Chapter 6: Time The Thinking Eye, The Seeing Brain by J. T. Enns

1
Chapter 6 TimeThe Thinking Eye, The Seeing
Brain by J. T. Enns
2
Sensitivity to change

• Differences between objects can be tracked as
  fast as 30 ms.
• At California highway speed, you only travel
  about 30 feet before you can tell something has
  changed in your view (the car in front brakes, a
  hippo appears in the road, etc.) and begin to
  move your foot to brake.
• About twice the length of an ordinary room.
• Question How are the absolute limits of the
  perceptual machinery related to what we see?

3
Absolute time ? speed of sight

• temporal resolution means how small a period of
  time you can detect a change
• Speed of sight means how long from the time
  your eye (retina) detects an event to when you
  are aware of the it
• absolute time means the time between a physical
  event and when you are aware of it

4
Temporal resolution

• resolution is as little as 30ms, about 30 times a
  minute
• But, temporal resolution is identical to
  absolute (or physical) time nor is it
  directly correlated
• see flash lag experiment

5
Flash Lag

• In this experiment (below),
• (a) if perception was an instantaneous picture of
  the absolute time of the display the parts of the
  bar would be seen in their real locations
• (b) if perception lags behind the physical event
  in a directly correlated way, the parts of the
  bar would still be seen together
• but, of course, they would directly correlate to
  the real (physical) position of the bars cont

6
Flash Lag

• Neither (a) nor (b) happens (below).
• What we see is the lines offset!
• the offset is 10 degrees
• Therefore, the Speed of sight seems to have
  added 50 ms to the temporal resolution

7
Temporal Resolution

• Temporal resolution is the reliable detection of
  differences in
• luminance over time
• Edge orientation over time
• Motion over time
• see experiment next

8
Figure 6.1. The flash-lag effect
       This gives you a sense of the limits of
temporal resolution        Perception of a brief
flash lags behind perception of a continuously
moving object.
9
The Speed of Sight

• Whats the point?
• We believe, against all rational hope, that
  vision is instantaneous.
• How do we know the correlation between the small
  time our retina is stimulated by light to the
  time we become conscious of an object or act upon
  the information?
• In other words, how is the speed of the machinery
  related to the real speed of sight (as opposed to
  the commonsense instantaneous speed of sight)?

10
The Flash-Lag Effect and the Speed of Sight

• In figure 6.1 Half a bar visibly pivots smoothly
  around its end while the other half is invisible.
  
• At some point, they both flash.
• If perception captured the absolute reality, both
  halves would appear as one complete bar flashing.
• The perception might be delayed by 30ms but that
  doesnt matter
• What we see is not instantaneous or consistently
  lagging what is happening (by 30ms).
• The Speed of Sight shows the bar extension off by
  20 degrees or double the time it takes for the
  machinery (60 ms).

11
The Illusion of Speed

• While you perceive yourself as quick, for an
  android, youre moving in slow motion
• The Speed of Sight Subjective Speed
• Subjective speed can be unpredictable.
• Did you ever rush to get out of the house only to
  find that it took twice as long as usual?
• Speed of the Machinery Absolute Speed

12
Seeing takes time

• You are driving at night, in the fog, visibility
  is 20 feet. Something looms in the fog. In the
  temporal hierarchy, V1, MT (mid temporal) and FEF
  (frontal eye fields) are activated in 40ms and V4
  (color) in 80ms, 120ms to activate the frontal,
  60ms to activate the motor, 20ms-40ms to move the
  muscles. (cont)

13
Are we dead yet?

• 300ms has passed, less than the time it takes you
  to read cat and say cat!.
• Your foot now begins to hit the brake!
• Look! The fog has concealed a stopped
  tractor-trailer blocking the road!
• At 65 mph, your head caves in, your lungs fill
  with blood, your foot never depresses the brake,
  your already dead.

14
Near Past Present

• Your life exists in the near past and your
  living your life is actually like an instant
  replay of the real thing
• actual (physical) time is unknowable, it is
  insensible.
• By the time you notice the truck, your actual
  life is over, the instant replay stops.

15
But officer, I didnt see it stopping!

• Whats the point?
• Instead of 0 feet, by Speed of Sight ,

you travel 20 feet before the speed of sight
registers the change!
Question If its dark, do you think speed of
sight stays constant?
16
What happened when?
The flash lag effect presents a smoothly rotating
bar that occasionally includes a brief flash in
which the two extensions of the rotating bar
appeared. What so odd about that? Well,
bizarrely, instead of seeing these brief flashes
as extensions of the rotating bar occurring at
precisely the same time as the rotating bar, we
see the flashed extension as lagging behind the
rotating bar even when they are happening at the
same time! Its Ripleys Believe it Or Not. Why?
17
The Motion Extrapolation Theory
One theory of the flash lag claimed that when
processing a predictable event, the brain neurons
anticipate the flash and prepare the neurons
for representing the continuously moving object
but not the flashed extension. This theory failed
two tests. (1) Even if the objects motion is
completely random, the flash lag effect occurs,
(2) experiments show that what happens to the
moving object after the flash has more of an
impact on the delay. Bizarre as it seems, the
speed after the flash not before the flash
determines the extent of the lag.
18
The Differential Latency Theory
Another theory states that a moving object can be
seen in less time than a flashed object. The idea
is that since visual motion is transmitted
through the magno ganglion neurons to the dorsal
stream that is specialized for motion, we see the
moving object faster. To bad for the theory,
then, that the flash lag effect happens even when
there is no movement before the flash at all.
Both the bar and the flash happen at once but the
bar continues.
19
More Strange Results
Later, we will see an experiment in which a
colored disc on one side of fixation changes
gradually from green to red in small steps of
color, with the display changing every 75 ms. At
some point during this transition, the changing
color disc is a particular shade of yellow. At
precisely the same time, the identical color is
flashed for a 75-ms period in another location an
equal distance from where the eye was positioned.
The participants' task is to indicate which disc
appears as more "red." The reports are always
shifted, as far as the changing colored disc is
concerned, in the direction in which the colors
are changing. That means, in this example, that
the changing disc is seen as more "red" or
"orange" than the simultaneously flashed and
identically colored yellow comparison disc. In
order to be seen as equal in color, the flashed
disc has to appear over 300 ms before its
same-color partner in the gradually changing
disc. Clearly, object motion is not a critical
feature of the flash-lag effect. The fact that
the same effect is found for a stationary object
changing only in color means that the dorsal
visual stream, thought to be largely color blind,
is not even a player in this effect.
20
Time
The flash-lag effect now appears to many vision
scientists to be a consequence of the fact that
object perception takes time (this chapter) and
that only one object can be attended to at a
time. It does not have to be thought of as a
consequence of seeing objects in motion or of
seeing unpredictable brief flashes of objects.
Considered in this way, the flash-lag effect
occurs because the participants' task requires
them to process certain visual features only
after processing another feature. We can call
the first feature the defining attribute of the
display and the second feature the report
attribute. In the experiments involving rotating
bars, the defining feature is the visual position
of the flashed bar. It takes some time for the
neural activity from that event to result in a
bar visible at a given location. Only once that
information has been established, in neural
terms, can the report feature be evaluated, which
in this case is the current position of the
smoothly rotating bar. By the time the position
of the flashed bar has been determined, the
moving bar has moved along to a new position on
the screen.
21
Time Again
The same analysis applies to the color-change
experiment. The color of the color constant
flashed disc is the defining feature in this
task. It takes time for participants to evaluate
its color once they have completed this task,
their report attribute (color of the changing
disc) has changed considerably. Naturally,
changing the order of these processes does not
alter the outcome. If the defining attribute is
the color of the changing disc at the time the
other disc is flashed, by the time the changing
disc color has been determined it will be
different from the color of the color-constant
flashed disc. Although we fail to really get
the absolute temporal order, the seen events
maintain an accurate analogical order in the
?-dimension (tau dimension).
22
The ?-dimension
The brains ability to maintain the
quasi-temporal ordering is so strong that a
smoothly rotating bar could stop on screen at
random (before, after or during the flash bar)
and participants would, nevertheless, agree on
the analogical temporal order. The ?-dimensional
ordering was accurate down to 30ms. It takes the
brain time to integrate the changes into the
neural network (the synchronous neural assembly)
that governs perception. But, even it if takes
100ms, the brain maintains the analogical
ordering. While it is folding the perception in,
the display changes, of course.
23
Seeing Takes Time

• Even at their best, neuronal steps happen only
  once every few ms (we saw in chapter 2)
• Even the earliest cortical activation, areas V1,
  MT (middle temporal area), and FEF (the frontal
  eye fields), at least 40 ms elapses from retinal
  stimulation to cortical activation.
• In addition, the detection of location, color,
  orientation involve another two or more classes
  of neurons and take more time.
• Ground-figure detection, as we saw, takes another
  100 ms.
• Color analysis and location take another 100 ms.
• Now you ask yourself, should I stop or swerve,
  is it dog or rat (you dont break for rats),
  another 100 ms.

24
The Specious Present

• Whats the point?
• Although we live in the Subjective Moment, the
  moment isnt the NOW.
• We live in the Near Past from the absolute
  temporal point of view
• Absolute Space and Absolute time are theoretical
  scientific constructs
• Subjective Space and Subjective Time are
  commonsense theoretical constructs (223) that
  involve an analogical extension of temporal
  concepts (before, now, soon, etc) that
  appear to work in an analogical temporal
  dimension.

25
Rules of construction

• Two rules of thumb are built into our perceptual
  system
• Object Stability
• According to our folk psychology, figures and
  grounds are stable, reidentification criteria are
  easy because the world is quiet, predictable,
  uniform
• Coherence
• Interpretation how we make sense of things
  presumes a high correlation between features,
  locations and movement patterns, the correlations
  are built in. For example,
• Trees, tall and thin, dont float, people dont
  morph into dogs, people tend to be taller than
  ants all sorts of programmed-in background

26
Temporal coherence

• The result of the processing through the temporal
  hierarchy (above) that becomes a chaos of
  temporal brain events is the temporal coherence
  of the cat in the physical world.
• How does the brain put the visual features of the
  cat back together so that we can see the cat?

27
seamlessness

• Two factors contribute to temporal coherence
• the brain mechanism that unifies the activity of
  widely separated regions of the brain
• the visual system assumes temporal coherence in
  the physical world

28
Insensibles

• Time, unlike spatial magnitude, has no properties
• since it has no properties, there are no
  receptors, or pick-ups for time, there are no
  time cells that are specialized like those for
  luminance gradients, color differences, the
  detection of shape, nothing like rods and cones
  for tracking time.

29
Time

• Time is not sensible so temporality must be an
  emergent characteristic of the way that neurons
  behave
• neurons respond to the onset of an event for
  which their receptive field is specialized, when
  the event disappears, the neuron may not
  respond to the end
• neurons dont keep records of their own on-off
  history (226)

30
Neural Synchrony

• Even if rate and duration cant help account for
  time
• when a given neuron fires, a neural pattern
  occurs in neighboring neurons
• this neural synchrony has a structure that can
  be thought of as an oscillation or wave.
• the overall structure of all neurons involved
  binds the neurons to the task, it is called
  neural binding, a tune, as it were.

31
Absolute time

• An event of neural binding in response to a face
  (227) is not time-locked to the triggering
  event.
• the brain activity is generated internally to the
  brain without anchor to the absolute event
• brain time isnt absolute time
• one tune might give rise to another tune as in
  the face recognition button pushing experiment
  (see 228)

32
Tricking Mother Nature

• If neural synchrony and oscillation occur in the
  temporal coherence of vision and
• these assume stability and coherence
• then,
• we can experimentally trick the visually system
  by tinkering with these assumptions to see how it
  works

33
Figure 6.2. The Construction of Subjective Time

• Figure 6.2. A brief flash and its associated
  neural activity over time
•         Perception is delayed.
•         Perception lingers beyond the duration
  of the flash.
• A 10 ms absolute event becomes a 50 ms neural
  event.
• The Neural event is not suddenly on/off, it
  changes degree it is an intensive magnitude with
  depth, so to speak, not an extensive magnitude
• events have to be propagated throughout the
  network

34
Question

• What happens during the period when neural
  activity extends beyond the temporal existence of
  the brief flash?
• vision scientists have efficient methods of
  studying the consequences of persisting neural
  activity by creating illusions in time

35
Neural Persistence and Subjective Temporal
Integration

• Figure 6.3. Two brief flashes
•         (a) separated in time,
•         (b) of two different spatial patterns of
  dots,
•         (c) that form a spatial matrix with a
  missing dot when combined.
• When the flashed patterns in (b) are separated
  by t(c)) and we see the two patterns as
  simultaneously present.
• The temporal integration is subjective, not
  objective (find this on the web for extra credit,
  Marshall Di Lollo illusion)

36
Subjective Discontinuity

• Figure 6.4. Two brief flashes of spatial patterns
  that overlap in time do not necessarily result in
  the perception of simultaneous patterns items
  can be left out.
•         Perception of events in time is not only
  delayed, it is constructed with more or less.
• When the different patterns overlap by less than
  50 ms, we see them as discontinuous anyway.
• We see patterns and structures that arent in
  absolute space, they exist in subjective space

37
Questions

• How could the visual system fail to see objects
  that are presented simultaneously?
• Why would vision construct an illusion of
  discontinuity when presented with objects that
  are, in fact, simultaneous?

38
Answers

• In the first case, when the flash was brief, by
  the time the synchrony took place, it was based
  on activity of both patterns of 12 dots (232).
  Does this happen in music?
• In the second case, the synchrony and
  oscillations from the first pattern get in the
  way of those for the second so observers are
  unsure

39
Objects

• The Point of examples in temporal integration?
• Seeing takes time (233)
• Absolute temporal sequences in neural events is
  not the same as the speed of sight. The speed of
  sight involves a virtual temporal dimension in
  which past, present, and future are
  analogical extensions of the terms used in
  Newtonian mechanics. As yet, no research is out
  that deals with this issue.

40
The Flash-lag effect

• Earlier we saw
• The flash-lag effect was a consequence of
• Object perception takes time
• Only one object can be attended to at the same
  time
• It isnt a consequence of motion or
  unpredictability (of flashes) (239)
• It takes time for the neural activity to result
  in a spatially fixed visible bar, evaluating the
  features means the bar has moved.
• The same holds in the color-change experiments
  for the flashed discs.

41
The Magical Number One, Again

• Flash-lag begs the question Why only one? (239)
• If humans divide attention among details of 2
  objects, only one can be seen during the same
  slice of time
• Subjectively we are convinced we see everything
• Absolutely, we see one thing
• Only one synchronous neural assembly can govern
  control of visual consciousness at any one point
  in time (240).
• I.e., the brain only hears one tune at a time.
• Why? Unknown.

42
Balance

• The perceptual system evolved to strike a balance
  between attentional capture and task-set bias.
• In other words, although we have to be capable of
  guarding against a sneak attack from our right
  flank, we cant stop calculating how we are going
  to hit Alley Oop over the head if he attacks.

43
Reduction

• attention is not some mysterious
  folk-psychological mumbo-jumbo.
• Attention refers to a dynamic visual processing
  during which some information is selected at the
  expense of others information or by neglecting
  other information
• neglect of information can be caused by
  attentional blink, a reduced period of
  sensitivity when viewing a sequence of images
  following a detection of a first instance of a
  target class that push attention
• Neglect can be caused by visual discontinuities
  will pull vision and nearby events before and
  after are blocked. (243)

44
Temporal Correlation and Perceptual Causality

• In addition to time, we have the closely related
  perception of cause and effect.
• Vision is designed to skip or jump to
  conclusions sacrificing speed for accuracy.
• 247

45
Correlations

• We can examine the tendency towards finding a
  cause (even if none exists) by using sequences
  that are fake but mimic moving objects in the
  real world (247).

46
A display sequence universally identified as one
ball bumping and launching another ball
Figure 6.12. A display sequence universally
identified as one ball bumping and launching
another ball         Perception of causality is
universal.         Perception of causality is
sensitive to small changes in timing.
47
Isolating Visual Properties

• Many people find it impossible to believed that
  concluding to causality is the work of the
  visual system rather than some high-level
  cognitive system.
• To silence critics, it was necessary to show that
  perceptions of causality were governed by
  temporal and geometric aspects of displays rather
  than thought processes.
• the launching and passing examples prove
  this, see next.

48
Causality

• Figure 6.12. A display sequence universally
  identified as one ball bumping and launching
  another ball
•         Perception of causality is universal.
•         Perception of causality is sensitive to
  small changes in timing.
• To show that higher-level reasoning is not
  required for causality Scholl et al. made the
  following

49
Figure 6.14. Perception of causality is context
sensitive
        (a) Passing sequence seen in
isolation.         (b) Passing sequence seen
in context of launching sequence now also looks
like launching.
50
Figure 6.15. A frequency distribution of response
times has smaller periodic peaks overlaid on it

•   30 ms a special time unit for the brain?
  (smallest time to make discriminations in
  temporal ordering?)
• Other constants
• 100 ms time for object formation to complete?
• 500 ms time needed to switch tasks? (253)

51
The Now

• One final unit of time.
• Consider the question
• What is the longest visual event that can be
  accurately reproduced in motor actions?
• The answer to this question is the same as the
  answer to
• How long is the present?
• The near past is your present and it is 30ms,
  the blink of an eye is 300ms.

52
Time after Time

• How is it that although subjective time doesnt
  match absolute time, we survive?
• Immediate experience recent past
• Our brains and eyes construct a subjective time
• We live in our construction
• We spent millions of years furnishing our
  construct
• There are no movies in absolute time, no grace,
  no music, no speech

53
The End
54
Figure 6.5. Two brief successive flashes of
shapes favors perception of the second shape
        Masking by camouflage at interval 0
ms         Masking by interruption at intervals
from 50 ms 150 ms.         Masking shows the
vulnerability of the object formation process
during the first 100 ms after an object is
presented.
55
Figure 6.6. Standing wave masking
        (a) No masking when displays cycle at
fast rates, so that visible persistence occurs
for all display shapes.         (b) Strong
masking occurs when the displays cycle at a rate
that leaves the object formation processes of the
center bar vulnerable to interruption or
competition from neighboring bars.         (c)
No masking occurs when the neighboring bars are
too far away.
56
Figure 6.7. Standing wave masking with feature
migrations.
        (a) The length of an unseen central bar
is seen in the flanking masking bars.        
(b) The color of an unseen central bar is seen
in the flanking masking bars.
57
Figure 6.8. Perception of change over time
        (a) Perception of a flashed color disk
is compared with perception of a disk smoothly
changing in color.         (b) Although the
flash may be identical in color to the changing
disk at the time of the flash, it appears to be
the color of the changing disk at an earlier
point in time.
58
Figure 6.9. Two modes of attention pull and push
        (a) Pull Attention is drawn rapidly to
a suddenly changing event, such as the appearance
of the outline box in the temporal stream of
letters.         (b) Push Attention shifts more
slowly when it is switched voluntarily in
response to a particular cue (the digit 7) that
must first be identified.
59
Figure 6.10. Find the gray rectangle.
        This search task is slowed by the
presence of a salient feature (black color) in
one item that is irrelevant to the task of
finding a particular shape (rectangle).        
Singleton mode look for conspicuous
discontinuities.         Feature mode look
for a specific rectangle shape.         The
black color does not influence the search task
when it is performed in feature mode.
60
Figure 6.11. A task in which participants are
looking for words identifying a job that people
perform for pay
        Participants are not distracted by many
foil words, showing they can stay on
task.         Participants are distracted by
foil words that identify human occupations,
showing that their task set cannot be made as
specific as the experiment demands.
61
Figure 6.13. Display sequences universally
identified as (a) pulling, (b) scattering,
and (c) bursting
62
Figure 6.16. The relationship between perceived
and reproduced time
        3000 ms the duration of the subjective
now or psychological moment?