New paper: A prospective code for value in the serotonin system

I’m thrilled to share that the final chapter of my PhD thesis is now published in Nature! This project is part of a long-term effort to come up with a simple(r) explanation of what serotonin does in the brain. It’s been an absolute pleasure to work with Richard Naud and Jean-Claude Béïque at the University of Ottawa and to have the opportunity to collaborate with Jeremiah Cohen and Cooper Grossman, without whom this work wouldn’t have been possible. Big thanks to the whole team!

Skimmed the paper and didn’t quite get it? No problem. In this post, I’ll share the inuititve ideas behind our main modelling results using a series of interactive widgets.

Adaptation

Neurons communicate by “firing” impulses of electrical activity called action potentials, and the rate of these impulses — the firing rate — is thought to be the main currency of information in the brain. So, if we want to know what serotonin neurons do, then surely figuring out what kinds of stimuli (e.g., rewards or punishments) cause serotonin neuron firing to go up or down will point us in the right direction?

Maybe not. Faced with a constant stimulus, most neurons respond by slowly decreasing their firing rate. This property is called adaptation, and serotonin neurons have it in spades.

Try playing with the widget below to build an intuition for how adaptation affects the response of a simulated neuron to a step-shaped stimulus. Hover over the sliders for more information about each parameter.

While the positive and negative deflections in the widget above are equally large, in real neural data these features are often asymmetric. This asymmetry can arise for a variety of reasons, from intrinsic properties of the neurons themselves (such as the simple fact that firing rates can’t be negative) to artifacts of common measurement techniques (such as the greater sensitivity of calcium dyes to increases vs. decreases in activity). For the sake of simplicity, I won’t deal with these asymmetries here, but they’re worth keeping in mind.

Prospective coding

Why do serotonin neurons have such strong adaptation? It’s hard to say for sure, but one possible reason is to compensate for sluggishness of downstream brain areas that receive serotonergic input. This idea is called prospective coding — “prospective” because it adjusts for downstream distortions ahead of time.

To get an idea of how adaptation could compensate for downstream sluggishness, try adjusting the adaptation strength and decoding timescale in the widget below. To keep things simple and intuitive, I’ve set the adaptation timescale to be very short so that full adaptation kicks in right away. With the right tweaks, you should be able to get any decoder, no matter how slow, to perfectly recover the step stimulus.

Why go through all the trouble of applying adaptation and then reversing it with a sluggish decoder? Energy efficiency is one possibility. Action potentials cost a lot of energy, whereas allowing the downstream effects of serotonin neuron activity to slowly build up is basically free. This means that strong adaptation in serotonin neurons coupled with a slow decoder allows the same stimulus to be broadcast throughout the brain more cheaply than with a “low adaptation, fast decoder” setup.

Value

So far we’ve seen that the activity of serotonin neurons is heavily coloured by adaptation, and we’ve built some intuition for how this could allow serotonin neurons to broadcast something throughout the brain in an energetically-frugal manner. But broadcast what?

Previous work has shown that serotonin neurons exhibit elevated activity shortly before and during rewards. This is reminiscent of a quantity called state value in reinforcement learning theory. While state value is technically defined as a cumulative weighted sum of future rewards, with more proximal rewards being weighted more heavily, this cumulative sum looks more like a series of isolated ramps when rewards are far apart.

To build an intuition for how state value evolves over time, try adjusting the parameters in the widget below. The graph shows state value in a trace conditioning experiment consisting of a series of trials in which an animal is presented with a sensory stimulus that predicts a reward.

A prospective code for value

Our titular model is basically the result of applying prospective coding to a state value signal.

Try adjusting the sliders in the widget below to build an intuition for how neural/animal and experimental parameters interact to produce the variety of ramps, bumps, plateaus, and dips exhibited by serotonin neurons.

To see how this model captures specific results from the serotonin literature, try the presets in this older version of the widget above. (Or check back here later. 😅)