Psychology: Classical and Operant Conditioning

1
Reward Systems

• Psychology Classical and Operant Conditioning
• Game Systems (Chapters8 and 13 Oxland)

2
Psychology of Behavior

• Pavlov Classical conditioning
• UCS (food) -gt salivation (UCR)
• CS (Bell) paired with UCS (food)
• UCS -gt salivation (CR)
• Examples in Games ?

3
Psychology of Behavior

• Thorndike (1898) Instrumental conditioning
• positive consequences strengthen behaviors to
  make them more likely in similar situations
• Instrumental Learning or trial and error
• Example cat learning to push a lever to get out
  of the box where she finds food.

4
Psychology of Behavior

• Skinner (1938)
• Operant Conditioning the process of learning
  behaviors from the environment through
  consequences.
• Positive reinforcement
• Negative reinforcement
• Punishment
• Chaining and shaping
• Reward schedule systems
• reinforcement is the chances of increasing
  probability of behavior occurrence.

5
Operant Conditioning

• Primary reinforcers e.g., food, has power to
  reinforce behavior
• Conditioned reinforcer e.g., money, increase
  probability of behavior by association with
  primary reinforcer
• Increase Behavior
• Action -gt positive Reinforcer
• ive Reinforcement
• Action -gt no negative Reinforcer
• ive reinforcement

6
Operant Conditioning

• Primary reinforcers e.g., food, has power to
  reinforce behavior
• Conditioned reinforcer e.g., money, increase
  probability of behavior by association with
  primary reinforcer
• Decrease Behavior
• Action -gt negative reinforcer
• ive punishment
• Action -gt no positive reinforcer
• ive punishment

7
Scheduling Reinforcers

• After behavior is learned
• Occurrence of reward after each behavior satiates
  the reward
• Different types of schedules work
• Fixed Ratio reinforce after x times of action
• Fixed Interval reinforce after time interval
• Variable Ratio reinforce after x times where x
  is variable between some interval
• Variable Interval reinforce after time interval
  x where x is variable between some interval

8
Shaping behavior

• Actionx -gt positive outcome
• Action1 -gt CS (positive) until conditioned
• Extinction of CS positive
• Action2 -gt CS (positive) until conditioned
• Extinction of CS positive
• Actionx -gt CS (positive)

9
Chaining and Shaping

• Action1 -gt Action2 -gtAction3 -gt positive outcome
• Action3 -gt CS (positive) until conditioned
• Action2 -gt Action3 -gt CS (positive) until
  conditioned
• Action1 -gt Action2 -gtAction3 -gt CS (positive)

10
Why?

• Reward systems in games
• Tutorials
• Teaching the player about game system
• Raph Koster connection between games and
  learning
• Addiction system
• Any more?

11
Application in Games

• What are the rewards?
• How to schedule them?
• Challenges
• Rewards are all conditioned, so people like
  different things
• How to integrate schedule of different rewards?

12
Rewards and Punishment

• Resources
• Chapter 8 and 13 Oxland

13
Types of Rewards/Reinforcers

• Gameplay Rewards
• help move player forward in the game, e.g.
• New weapon
• A Key
• New power
• Hidden Rewards
• cannot count on player finding them
• Visual rewards
• Others?

14
Types of Rewards/Reinforcers

• Clearing a level and advancing to the next
• Solving puzzles
• Achieving goals
• Winning
• Gaining points
• Gaining powerups
• Acquisition of items
• Acquisition of skills, improving skills or stats
• Exploration
• Achieving a top score
• Status

15
Scheduling of Reinforcers
16
Reinforcer scheduling

• Ratio strain the player looses interest due to
  effort required and the reward.
• Variable ratio schedule is used for level
  advancement
• these schedules have the least succeptibility to
  ratio strain.
• a schedule that is the shows the most resistance
  to extinction, produces no post-reinforcement
  pause, and which is the most effective at
  producing high levels of response over extended
  periods.

17
Game Feedback

• Resources
• Chapter 6 Oxland

18
Types of Feedback

• Visual feedback
• Color
• Lighting
• Animation/action
• Your character animations (Devil May Cry)
• Sound
• Effects
• Music
• Goal achievement
• Character power, health, states
• Inventory

19
When is Feedback triggered?

• Action-based (e.g., footsteps sound with walking)
• Tension-based (e.g., Tetris)
• Event-based (e.g., monster spots you)

20
Critique

• Take one segment of the game play within your
  game. Describe
• Reinforcers used positive, negative
• Punishment
• Reinforcement schedule
• Reinforcement chaining
• How does that impact learning game system?
• Does that make the game fun?
• Take a segment of the game
• Outline all types of feedback you see and hear
• Outline when they are triggered