My research focuses on the neuroscience of reinforcement learning (RL). My methods are computational modeling, behavioral psychophysics, pupillometry, EEG and fMRI and my goal is to use these methods to pick apart the algorithms underlying behavior. My work splits into three separate projects each probing a different aspect of RL: the learning rate, the state space and the explore-exploit dilemma.

How fast should we learn?

This work looks at the rate at which learning occurs. Traditionally this `learning rate' is assumed to be constant over time, but in many cases a constant learning rate is suboptimal, suggesting that performance could be improved if the learning rate can change. This project seeks to test whether the learning rate does indeed vary over time and investigate the neural and computational factors that determine its setting.

Learning and representing the state space

At the heart of all RL models is the concept of a state space -- an abstract representation of a task that describes its underlying structure. In most models of behavior, it is assumed that animals know the state space before learning begins, as if all tasks come with an `instruction set'. In the real world this is clearly not true and the state space must be learned. This project asks: How the is state space is learned? and Where are the states represented?

The explore-exploit dilemma

When you go to your favorite restaurant do you order the same thing as usual, a dish you are sure to find tasty but that yields no information, or do you try one of the specials and learn something new? This simple conundrum, deciding between going with what you know or trying something different, is at the heart of the exploration-exploitation dilemma and whether it's a cow looking for greener grass or a human looking for love, this problem is ubiquitous and important to solve. This work probes the neural mechanisms underlying the explore-exploit tradeoff in humans.