Chapter 8: Stimulus Control

1
Chapter 8Stimulus Control
2
Stimulus Control

• When social or nonsocial events precede operant
  behavior and affect its occurrence, they are
  called controlling stimuli.
• A controlling stimulus (S) is said to alter the
  probability of an operant, in the sense that the
  response is more (or less) likely to occur when
  the stimulus is present.
• Discriminative stimulus (SD) - a controlling
  stimulus that sets the occasion for reinforcement
  of an operant.
• S-delta (S?)or extinction stimulus- a stimulus
  that sets the occasion for nonreinforcement or
  extinction of an operant.

3
What do these have in common?

• Visiting a restaurant when neon signs are
  illuminated?

• Running only where the footprints are shallow (or
  numerous)

4

• Cat meows only at the window with lights on

• Drive in the lane with least traffic

• Dressing up for a date

5
Answer All involve stimulus control!

• Stimulus control occurs when 1) a response occurs
  in the presence of a stimulus and 2) does not
  occur in its absence
•   organism is able to discriminate two or more
  different situations
• proof is in the behavior

6
Emitted versus occasions
Operants can and do occur in the absence of any
eliciting stimulus, they are said to be freely
emitted. However, when an SD comes to control
occurrences of an operant, to alter its
probability of occurring, then it is said that
the SD occasions the operant. The term occasion
as a verb dictates that the operant is under the
stimulus control of an antecedent.
7
Differential Reinforcement and Discrimination

• When an organism makes a response in one
  situation but not in another, we say that the
  animal shows a discrimination between the
  situations. The simplest way to train a
  differential response or discrimination is to
  reinforce an operant in one situation and
  withhold reinforcement in the other.
• Stimulus control refers to a change in behavior
  that occurs when either an SD or S? is presented.
  When an SD is presented, the probability of
  response increases when an S? is presented, the
  probability of response decreases.

8
How does stimulus control come about?

• Through differential reinforcement
• 1. Reinforce in the presence of one stimulus
• SD response? SR
• Red light press bar? food

Press lever
Food
9
Do not reinforce in the presence of another
stimulus

• S? response? Ext
• Purple light press bar? no food

Press lever
Extinction

• if stimulus control is demonstrated, the rat
  should press bar only when the red light is on,
  and never press when the purple light is on

10

• OR you may present an aversive stimulus as the
  consequence
• purple light press bar? shock

Shock

• purple light called S- or S? because it signals
  punisher

11
Stimulus Control and Multiple Schedules

• Behavior analysts often use multiple schedules of
  reinforcement to study stimulus control in the
  laboratory.
• On a multiple schedule, two or more simple
  schedules are presented one after the other and
  each schedule is accompanied by a distinctive
  stimulus.

12
Stimulus Control

• One way to measure the stimulus control exerted
  by the SD and at any moment is to use a
  discrimination index (ID), This index compares
  the rate of response in the SD component to the
  sum of the rates in both SD and S? phases.
• ID(SDrate)/(SDrate S? rate )

13
Behavioral Contrast

• Occurs within subjects
• Operant responding is the dependent variable
• Measured on multiple schedules of reinforcement
• Two or more simple schedules that alternate
• Subject does not get to choose on which
  schedule to respond

14
Behavioral Contrast

• Positive contrast- when rate of response in an
  unchanged setting increases with a decline in
  behavior in another situation.
• Negative contrast- when rate of response declines
  in an unaltered situation with increases in
  behavior in another setting.

15
Behavioral Contrast (cont.)

• Negative Behavioral Contrast
• A decrease in response rate in one component of a
  multiple schedule of reinforcement because the
  conditions of reinforcement in another component
  have gotten better
• Positive Behavioral Contrast
• An increase in response rate in one component of
  a multiple schedule because the conditions of
  reinforcement in another component have gotten
  worse

16
Behavioral Contrast (cont.)

• Both positive and negative contrast is commonly
  reported
• Produced by the change in reinforcement in the
  other component

17
Behavioral Contrast

• Positive contrast
• Increase in unchanged component
• Negative contrast
• Decrease in unchanged component

In this example, subjects were reinforced on the
same schedule of reinforcement in phase 1 in
phase 2, responding during the light was on the
same schedule of reinforcement but during the
noise, responding was no longer reinforced and
positive contrast occurred in responding during
the light.
18
Why contrast?

• Positive contrast increase in unchanged
  component
• Results from slower responding in changed
  responding
• Additivity theory classical plus operant
  conditioning
• Relative rate of reinforcement

19
The Moral Behind Contrast Effects

• Contrast effects are instrumental in
  demonstrating that reinforcement in other
  situations can alter present behavior
• Even when the other reinforcement is currently
  available

20
Contrast Effects
It is almost a cliché to say that when a child is
acting up at school, i.e., being disruptive,
hyperactive, etc.,there must be problemsat
home. Maybe but not so fast Maybe mom and dad
have implemented some strict requirements on
doing homework at home, reducing their sons
reinforcements at home. Positive contrast
theory predicts that the childs behavior will
increase in some form at school.
21
Generalization

• An organism that responds in one situation but
  not in another is said to discriminate between
  the settings.
• An organism that behaves similarly in different
  situations is said to generalize across
  circumstances.

22
Stimulus Generalization

• Stimulus generalization occurs when an operant
  that has been reinforced in the presence of a
  specific discriminative stimulus also is emitted
  in the presence of other stimuli. The process is
  called stimulus generalization because the
  operant is emitted to new stimuli that presumably
  share common properties with the discriminative
  stimulus.
• Generalization and discrimination refer to
  differences in the precision of stimulus control.
• Discrimination refers to the precise control of
  an operant by a stimulus, and generalization
  involves less precise regulation of operant
  behavior.

23
Stimulus Generalization

• Stimulus Generalization The transfer of a
  learned response from one stimulus to another,
  similar stimulus.

24
Basic Procedure

• Generalization training Responses to a stimulus
  (S) are reinforced.
• Discrimination training Responses to one
  stimulus are reinforced, and responses to a
  second stimulus (S-) are extinguished, put on
  extinction.

25
Basic Procedure

• Generalization test A variety of stimuli from
  the same stimulus continuum are shown, one at a
  time, under extinction conditions.

26
Basic Procedure
Positive Stimulus
Generalization Test
27
Basic Procedure
Stimulus Color
28
Generalization Gradient

• A generalization gradient shows the relationship
  between the probability of response and stimulus
  value.

29
Gradients and Learning

• The accuracy of responding improves as the
  individual becomes more experienced with the
  stimulus dimension.

30
Experience Changes the Gradient
Novice
Expert
S
S
31
The Guttman and Kalish procedure in an animation
Instructions The following slides are animated
automatically. You will only have to click your
mouse to advance from slide 1, (next slide
labeled Acquisition phase) to slide 2 (the
subsequent slide labeled Test phase). The slides
will advance themselves once you get to slide 2.
You will see a figure of a pigeon approach a
response key, a colored circle, and the pigeons
pecking responses are indicated by the star
shapes. The delivery of the food reinforcer is
illustrated by a group of small seed-like shapes
appearing in a food-hopper at the bottom of the
slide. They disappear when consumed by the
subject.
32
Instructions The following slides are animated
automatically. You will only have to click your
mouse to advance from slide 1, (next slide
labeled Acquisition phase) to slide 2 (the
subsequent slide labeled Test phase). The slides
will advance themselves once you get to slide 2.
You will see a figure of a pigeon approach a
response key, a colored circle, and the pigeons
pecking responses are indicated by the star
shapes. The delivery of the food reinforcer is
illustrated by a group of small seed-like shapes
appearing in a food-hopper at the bottom of the
slide. They disappear when consumed by the
subject.
33

• Slide 1 Acquisition Phase S key lit at
  580nm
• S - unlit key

34

• Slide 2 Test Phase presentation of 11 stimuli
  (530nm through 630nm) in random order in 12
  different blocks responses are not
  reinforced.

35
Test Phase presentation of 11 stimuli (530nm
through 630nm) in random order in 12
different blocks responses are not
reinforced.
36

• Test Phase presentation of 11 stimuli (530nm
  through 630nm) in random order in 12
  different blocks responses are not
  reinforced.

37
Test Phase presentation of 11 stimuli (530nm
through 630nm) in random order in 12
different blocks responses are not
reinforced.
38

• Test Phase presentation of 11 stimuli (530nm
  through 630nm) in random order in 12
  different blocks responses are not
  reinforced.

39

• Test Phase presentation of 11 stimuli (530nm
  through 630nm) in random order in 12
  different blocks responses are not
  reinforced.

40

• Test Phase presentation of 11 stimuli (530nm
  through 630nm) in random order in 12
  different blocks responses are not
  reinforced.

41
Stimulus Generalization as a Measure of Stimulus
Control
Training S
Pigeons were trained to peck in the presence of a
colored light of 580 nm wavelength and then
tested in the presence of other colors.
After Guttman Kalish, 1956
42
Generalization Gradient
As the previous slide showed, stimuli that have
similar dimensions to the S (or SD) occasions
more responses than stimuli that are more
divergent. Or, increasing physical similarity to
the training stimulus greater probability of
response
43
Absolute and Relative Stimulus Control

• Peak shift is an unusual effect from the point of
  view of control solely by a stimulus.
• Absolute stimulus control means that the
  probability of response is highest in the
  presence of the stimulus value used in training.
• In fact, this occurs when reinforcement is the
  only procedure used to establish stimulus control
  (and not extinction).
• The shift in the peak of the generalization
  gradient may reflect relative, rather than
  absolute, stimulus control. Relative stimulus
  control means that an organism responds to
  differences among the values of two or more
  stimuli.

44
Peak Shift

• Peak Shift refers to the change in the peak of a
  generalization gradient away from the stimulus
  that signals extinction.

45
Peak Shift
S
S-
46
Peak Shift

• Body distortions in anorexia nervosa Anorexics
  think that a larger than normal body size must be
  avoided, is too big! and think that a thinner
  than normal body size is the ideal

S
S-
47
Absolute and Relative Stimulus Control

• There are other ways of showing relational
  control by stimuli. To study generalization
  gradients and peak shift, the researcher usually
  arranges the presentation of SD or S? so that one
  follows the other. This is called successive
  discrimination.
• An alternative procedure is labeled simultaneous
  discrimination -the SD and the S? are presented
  at the same time and the organism responds to one
  or the other.

48
The method by which you train affects the degree
of stimulus control

• SD only
• 2. SD and S-delta or SD and S-

49
SD only (non-differential)

• A response trained to one discriminative stimulus
  may occur in the presence to others that are
  physically similar (generalization

One relation is trained
50
Responses to similar stimuli are learned
51
Another example

• What is trained
• Upon seeing a furry four-legged creature Say
  dog?praise
• What is learned
• Sees Cat dog?
• Sees Horse dog?

52
Bottom line with SD only

• Generalization to all physically similar stimuli
  is likely when SD only is trained
• How does one learn NOT to generalize (i.e.,
  discriminate)?

53
SD and S-delta training orSD and S- training
(presence-absence training)

• When a stimulus correlating with extinction or
  punishment is trained with the SD relationship,
  finer discriminations develop

Two relations trained
54
What is learned

• A finer discrimination between 500 and 600 units
  of redness
• Note slope is steeper with discrim. training

55
Errorless Discrimination and Fading

• When the SD and the S? are alternately presented
  as in successive discrimination, the organism
  initially makes many errors. That is, the animal
  or person continues to respond in the presence
  of the S-delta on the basis of generalization.
• As extinction and reinforcement progress, a
  differential response occurs to the SD and S?.

56
Errorless Discrimination and Fading

• A pigeon is taught to peck a green key for food.
  Once this behavior is well established, the color
  on the key is changed to blue and pecking is not
  reinforced. The blue and green colors are
  alternately presented and the corresponding
  schedules of extinction or reinforcement are in
  effect. During the early sessions, the onset of
  extinction will generate emotional behavior that
  interferes with ongoing operant behavior.

57
Fading and Errorless Discrimination

• Extinction is an aversive procedure. Pigeons
  flap their wings in an aggressive manner and will
  work for an opportunity to attack another bird
  during the presentation to the S? on a multiple
  schedule. Birds will peck a different key if
  pecking turns off the extinction stimulus,
  implying that the stimulus is aversive.

58
Fading and Errorless Discrimination

• There are other problems with successive
  discrimination procedures. Because emotional
  behavior is generated, discriminative responding
  takes a long time to develop. In addition,
  spontaneous recovery of S-delta responding from
  session to session interferes with the
  acquisition of a discrimination. Finally, even
  after extensive training, birds and other
  organisms continue to make errors by responding
  in the presence of the signal for extinction.

59
Errorless Discrimination

• Errorless discrimination is successful because
  the trainer or teacher does not allow the
  organism to make mistakes by responding to the
  extinction stimulus.
• Errorless discrimination involves gradually
  introducing the S? initially at a very weak
  intensity such that responding to it is very low
  probability. Over repeated trials, the intensity
  of the S? is gradually increased. Eventually the
  S? can be presented in its full intensity and the
  subject will not respond to it. A discrimination
  between the SD and the S? was acquired without
  the errors of responding to the S?.

60
Errorless discrimination

• Training a discrimination by gradually
  introducing the s-delta such that the organism
  makes few (if any) responses to it
• Red key peck disk? reinforcement
• Green key peck disk? no reinforcement
• First introduced as a dark disk and was
  systematically faded into illumination

(Off)
SR
EXT
61
SD stays same
S-delta faded in
By trial 50
By trial 100
By trial 150
62
Result very few errors were made

• Few responses (errors) to the green key
• Under traditional procedure, responses to green
• key must occur for learning to take place
• Most (all) responses to the red key

63
Errorless Discrimination Learning

• Terrace (1966)
• S? introduced early and at low intensity
• Fading procedure used
• S? not aversive
• No contrast

64
Complex stimulus control

• Relational control when a relative property is
  reinforced, e.g., the larger (smaller, greener,
  etc.), that is what is learned

Darker is trained, darker is learned
SR
SR
SR
65
A diagram of a typical matching to sample
procedure with pigeons Correct responses of
matching the comparison stimulus to the
sample stimulus are reinforced. Responses to the
incorrect comparison Stimulus are followed by a
blackout of the chamber lights and extinction.
66
A matching to sample procedure as done with seals
or sea lions. Go here for video
clip http//pbs-saf.onstreammedia.com/cgi-bin/vis
earch?userpbs-saftemplatetemplate.htmlqueryAn
imalEinsteinscategory0viKeywordAnimalEinstei
ns
67
The matching to sample set up to teach verbal
interactions between two pigeons in Skinner
Epsteins Columban simulations.
68
Delayed Matching to Sample and Remembering

• This variation of the matching to sample task has
  been used to investigate behavior said to reflect
  cognition and memory
• Time b/w the offset of the stimulus and the onset
  of the comparison is known as the retention
  interval
• Theoretically, the organism is covertly doing
  something that helps to retain the information
  about the sample

69
Directed Forgetting

• In a variation of the DMTS procedure, during the
  delay a specific stimulus was presented
  indicating the matching comparison stimulus would
  be presented after the delay. If another stimulus
  was presented during the delay, the matching
  comparison stimulus would not be presented and no
  correct response was possible.
• Douglas Grant found that the probability of a
  correct response to the comparison stimulus could
  be reduced by presenting the stimulus for no
  correct comparison stimulus and in fact the
  correct comparison stimulus was then presented.
  This effect was termed directed forgetting and
  its effect was varied depending upon when the
  stimulus was presented in the delay interval.
  Attributing directed forgetting to stimulus
  control is a more straightforward explanation
  than to inferred disrupted rehearsal.

70
Here is a matching version of the task (DMTS)
used with pigeons.
Here the sample stimulus is presented, a delay
period is introduced, followed by the
presentation of the comparison stimuli. The
duration of the sample stimulus and the delay
duration are manipulated as variables.
71

• Discrimination learning
• Small number of simple stimuli with well-defined
  and unambiguous differences between S and S-
• Concept Formation
• Use of more stimuli, with more abstract defining
  features
• Positive elements, defining the stimulus as
  belonging to the concept. A fish is defined by
  specific stimulus features if the features are
  present in a stimulus, the stimulus is a fish.
• Negative elements, defining the stimulus as not
  belonging to the concept, a starfish is clearly
  not a fish. Neither is a whale but a whale is
  easily confused as being a fish

72
Non-Human Studies on Natural Concept Formation

• Herrnstein - Pigeons
• Pigeons can learn to form concepts of a person as
  discriminated from non-persons, i.e., statues,
  mannequins, etc., a specific person from others,
  etc. The abstract stimulus class of person
  readily learned.
• Pigeons can learn to form natural concepts of
  persons, trees, fish, etc., with a high degree of
  accuracy. Artificial concepts such as buildings
  also learned.
• Pigeons appear to respond to two-dimensions
  objects (photographs) as representations of three
  dimensional objects

73
Conditional Discrimination

• An SD or S? can be defined as a function of
  additional contextual stimuli. A pigeon may have
  to learn that if the houselight of an operant
  chamber is illuminated, pecking the green
  response key delivers food. If no houselight,
  pecking the red key delivers food, Thus, if
  houselight present, green key SD, red key S?.
  If no houselight, green key S?, red key SD.
• To a sexual sadist and their masochist partner,
  no means no outside of their sex lives. But
  in the context of sexual activities, the
  masochists cries of No, no, no means Yes,
  yes, yes to the sadist. In the same context,
  pink or red (so-called safe words) will mean
  no or stop.

74
Stimulus Control andLife

• Studying
• Sleeping (Insomnia)
• Writing

75
Discriminating internal sensations can be trained
by discrimination procedures

• What is trained
• Drug press drug lever? food
• (reinforcement of saying Yes, Ive had drug
  when there is drug in system)
• Saline press drug lever? no food
• (extinction of saying Yes, Ive had drug when
  there is NO drug in system)
• AND
• Drug press no drug lever? no food
• (extinction of saying No, I dont have drug
  when there IS drug in system)
• Saline press no drug lever? food
• (extinction of saying No, I dont have drug
  when there IS drug in system)

76
What is trained.
S-delta
SD
Amphetamine (mg/kg)
77
What is learned

• Test different levels of drug (low to high) the
  more similar the drug feels to the trained
  drug, the more the animal generalizes
• What shape do you think the curve is?