Opponent interactions between serotonin and dopamine

1
Opponent interactions between serotonin and
dopamine
We will talk

• About Dopamine and Serotonin
• About TD learning
• How these neurotransmitters interact in TD
  learning

• Review of D.Daw, S.Kakade and P. Dayan
  paperDavidov Dmitry

2
Dopamine
One of most researched neurotransmitters

• As shown
• Strong evidence for being related to reward in TD
  learning system

3
Serotonin (5HT)

• Most mysterious of the main vertebrate
  neuromodulators

4
Dopamine Serotonin controversyEvidences

• Pharmacological
• (Dopamine involved in behaviors that serotonin
  inhibits)
• Microdialisis
• Electrophysiological

5
Appetitive Vs Aversive system

• A lot of evidences for existence of two separate
  systems

Aversive
Appetitive
Stopping delivery of a long sequence of rewards
can also be aversive
As well as stopping delivery of punishments can
be appetitive
X
X
6
How to predict future?... - TD learning
modelFor each trial

• Functions
• V(t) Value
• r(t) reward

What is needed is to approximate V(t) - Actually
we need to update V(t) in light of reward r(t)
Phasic component of error signal
7
Problematic point

• How to represent negative errors?
• Pauses in neuronal firing?
• Baseline is extremely weak
• No evidence that downstream areas can detect
  such pauses

8
Another point

• Outside of laboratory trials are not episodic,
  but all life is like one continuous trial

Solution
Use DIFFERENTIAL values V(t) Summed
difference between observed reward and some
average reward r
9
How it should look like?
X
Reward
Extinction
?Net Responce?
Phasic Dopamin
Tonic Serotonin
10
Symmetry wins?

• Strange fact
• Elevated dopamine in response to aversive
  stimuli

Suggestion
Dopamine and serotonin are oposite in both
actions!
11
Work to DO and OVER-simplifications

• How do the serotonin and dopamine systems
• actually divide their function?
• Are there any significant serotonin phasic
  responses?
• Is learning rate actually fixed?
• Adaptive learning rate?
• Are there only 2 time scales?
• Hierarchy of time scales?

12
Conclusions
V
V

• Dopamine and Serotonin activity are likely to be
  involved in a kind of TD learning system
• Phasic and Tonic signals play distinct roles
• The functions of Dopamine and Serotonin may
  mirror each other