Dopamine, Uncertainty and TD Learning

Yael Niv, Michael O. Duff and Peter Dayan

Substantial evidence suggests that the phasic activities of dopaminergic neurons in the primate midbrain represent a temporal difference error in predictions of future reward. Experimental observations in a task involving uncertain rewards have been interpreted as contradicting this interpretation, suggesting that dopamine activity represents uncertainty directly. We reinterpret these data in terms commensurate with prediction error, and study the effects of different forms of stochasticity in temporal representations.