Markov Models

1
Markov Models
2
Agenda

• Homework
• Markov models
• Overview
• Some analytic predictions
• Probability matching
• Stochastic vs. Deterministic Models
• Gray, 2002

3
Choice Example

• A person is given a choice between ice cream and
  chocolate.
• The person can be
• Undecided.
• Choose ice cream.
• Choose chocolate.
• There is some probability of going from being
  undecided to
• Staying undecided and giving no decision.
• Choosing ice cream.
• Choosing chocolate.

4
Markov Processes

• States
• The discrete states of a process at any time.
• Transition probabilities
• The probability of moving from one state to
  another.
• The Markov property
• How a process gets to a state in unimportant. All
  information about the past is embodied in the
  current state.

5
State Space
SIce Cream
SUndecided
SChocolate
6
Transition Probabilities
P(SUSU)1-(? ?)
SIce Cream
P(SI_CSI_C)1
P(SI_CSU)?
SUndecided
P(SCSU)?
P(SCSC)1
SChocolate
Note The transition probabilities out of a node
sum to 1. How can this model be made equivalent
to Luce Choice?
7
Transition Probabilities
P(SUSU)1-(? ?)
SIce Cream
P(SI_CSI_C)1
P(SI_CSU)?
SUndecided
P(SCSU)?
P(SCSC)1
SChocolate
P(SI_CSU) v(I_C)/(v(I_C)v(C)) P(SCSU)
v(C)/v(I_C)v(C))
8
Transparent Responses
P(SUSU)1-(? ?)
SIce Cream
P(SI_CSI_C)1
P(SI_CSU)?
SUndecided
P(RI_CSI_C)1
P(SCSU)?
P(SCSC)1
SChocolate
P(RnoneSU)1
P(RI_CSC)0
9
Transparent Responses
P(SUSU)1-(? ?)
SIce Cream
P(SI_CSI_C)1
P(SI_CSU)?
SUndecided
P(RI_CSI_C).8
P(SCSU)?
P(SCSC)1
SChocolate
P(RnoneSU)1
P(RI_CSC).2
10
State Sequence
Time
SU
SU
SU
SC
SC
Hidden

None
None
None
Choc.
Choc.
Observed


t1
t3
t4
t5
t2
11
Matrix Form of Transition Probabilities
To
SU SI_C SC
SU 1-(??) ? ?
SI_C 0 1 0
SC 0 0 1
From
12
Some Analytic Solutions
Where ?P(SI_CSU) and ?P(SCSU)
13
Some Analytic Solutions

• What happens if
• ??1?
• t1?

14
?.25, ?.4
15
More Analytic Solutions
16
?.25, ?.4
17
Problem?

• Why dont the choices sum to 1?

18
More Results

• The matrix form is very convenient for
  calculations.
• It is easy to calculate all moments.
• More to come with random walks

19
Pair Clustering

• Batchelder Riefer, 1980
• Free recall of clusterable pairs.
• Implements a Markov model for the probability
  that a pair is clustered on a particular trial.
• Are MPTs Markov models?

20
Probability Matching

• Paradigm
• Warning light
• Prediction P(R1), P(R2)
• Feedback P(E1)?, P(E2)1-?
• Typical result
• P(R1)? ?

21
Probability Matching

• Can be implemented via a Markov model.
• Assume win-stay/lose-shift paradigm
• If correct, make same prediction
• If incorrect, shift response with probability
  ?.
• Associate an element with most recent event,
  but not perfectly.

22
Next Trial
R1E1 R2E1 R1E2 R2E2
R1E1 ? 0 1-? 0
R2E1 ? ? (1- ?) ? ? (1-?) (1-?) (1-?)
R1E2 (1-?) ? ? ? (1-?)(1-?) ?(1-?)
R2E2 0 ? 0 1-?
Current Trial

• RiEj Response i and then Feedback j.
• ? Probability of Feedback 1.
• ? Probability of switching after error.

23
(No Transcript)
24
Markov Property
25
Light2
Light1
26
Markov Property
P(Honk) 0
P(Honk) 0
L1/Go
L2/Go
.7
.3
.3
L1/Stop
L2/Stop
.7
P(Honk) .3
P(Honk) .4
27
Markov Property
L1/Go
L2/Go
P(Honk) .3
P(Honk) .4
L1/Stop
L2/Stop
P(Honk) .8
P(Honk) .7
L1/Stop Repeat
L2/Stop Repeat
28
Stochastic vs. Deterministic

• Stochastic model The processes are
  probabilistic.
• Deterministic The processes are completely
  determined.

29
Stochastic Models Imply

• Psychological events are uncertain
• Even if we had all the knowledge we needed we
  could still not figure out what a person is going
  to do next.
• Or

30
Stochastic Models Imply

• The model does not capture all aspects of the
  behavior in question
• Allows the model to focus on certain parts of
  behavior and ignore others.
• You may believe behavior is deterministic, but
  still rely on a stochastic model.
• Allows the modeler to finesse some ignorance.
• OR

31
Stochastic Models Imply

• Some parts of the task are truly random
• E.g., feedback schedule from the experimenter in
  a probability matching task.

32
Limitation of Stochastic Models

• You need to test them on populations of behavior,
  not individual behaviors.
• E.g., I gave Participant X a single choice and
  she chose ice cream.
• Can we test the model against this datum?