THE HEBREW UNIVERSITY OF JERUSALEM
Introduction to Learning and Behavior: Conditioning and the Brain
Course 76919, Spring 2006



Yael Niv
Email: yaelniv at alice nc huji ac il
Tel: 054-6556464


Format and content:

The course will comprise of 10 three-hour lectures (Thursdays 14-17).

Each lecture will concentrate on one topic in the area of animal learning, with the aim of starting from the behavioral phenomena, and continuing to either (or all) of: theoretical models of the learning and decision making process, computational models of the behavior, and brain areas/systems known to be involved.

Course requirements: Active class participation, reading of assigned literature, 5 home assignments (including programming in Matlab), final exam.

Syllabus topics:

  1. Introduction to the empirical psychology of learning - Aim, structure, requirements of the course. Framework of experimental psychology and what is it good for, classes of models, how can we look at learning in the brain PPT presentation
  2. Classical conditioning I - Pavlov, basic procedure, phenomena, blocking and overshadowing, Resorla-Wagner model, second order conditioning, temporal difference model, dopamine (Schultz) PPT presentation
  3. Classical conditioning II - Appetitive vs. aversive and inhibitory vs. excitatory conditioning, opponent process theory, fMRI of aversive conditioning (Seymour), conditioned inhibition and dopamine, competition and the Kalman filter (Dayan), applications of classical conditioning to humans PPT presentation
  4. Instrumental conditioning I - Thorndike, procedure types, partial reinforcement, Skinner, free operant reinforcement schedules, Actor-Critic models and neural substrates (fMRI - O'Doherty, cyclic voltammetry - Wightman) PPT presentation
  5. Motivation and free operant action selection - Behavior, RL model of free operant responding (Niv), effects of motivation on behavior (energizing vs. directing), the role of DA (depletion studies, Salamone) PPT presentation
  6. Instrumental conditioning II: Actions and Habits - S-R vs. R-O, devaluation manipulations and behavior, incentive learning, neural substrates of goal directed and habitual responding (Dickinson, Balleine, Killcross), Uncertainty based arbitration (Daw) PPT presentation
  7. Classical and Instrumental - different learning mechanisms? - One vs. two process theories, omission, Pavlovian-instrumental transfer, neural substrates
    + Extinction - "Be a theoretician" PPT presentation
  8. Learning as a model of mental disorders: Latent inhibition and schizophrenia - Animal models of neural disorders, behavior, neural substrates, acquisition vs. expression deficit, models of LI PPT presentation
  9. Attention and Associability - Behavioral phenomena, Pearce and Hall, Mackintosh, Wagner's SOP, attention and associability (Kakade, Dayan, Montague), Amygdala and attention (Holland and Gallagher), Ach and inference (Yu and Dayan) PPT presentation
  10. Generalization and Discrimination - Pearce (Configural) vs. R-W (elemental), behavioral phenomena, learning latent causes and Bayesian inference (Courville and Daw), course summary PPT presentation


Reading:

  • For 16.3: PR Montague, P Dayan, and TJ Sejnowski (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16: 1936-1947. PDF
  • For 16.3 (optional): RS Sutton and AG Barto (1990) Time-derivative models of Pavlovian reinforcement. gzipped postscript
  • For 23.3: B Seymour et al. (2004) Temporal difference models describe higher order learning in humans. Nature 429,664-667. PDF
  • For 23.3: PN Tobler, A Dickinson and W Schultz (2003) Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J Neurosci 23(32), 10402-10410. PDF
  • P Dayan, S Kakade and PR Montague (2000) Learning and selective attention. Nat Neurosci supp. Vol 3 1218-1223. PDF
  • For 30.3: J O'Doherty et al. (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452-454 PDF
  • For 6.4: Y Niv, ND Daw and P Dayan (2005) How fast to work: Response vigor, motivation and tonic dopamine. NIPS PDF
  • For 27.4: A Dickinson (1985) Actions and habits: the development of behavioural autonomy. Phil Trans of Royal Soc of Lon B 308(1135), 67-78.
  • For 27.4: S Killcross and E Coutureau (2003) Coordination of actions and habits in the medial prefrontal cortex of rats. Cerebral Cortex 13, 400-408. PDF
  • For Assignment 4: EC Tolman (1948) Cognitive maps in rats and men. Psychological Review 55(4), 189-208. Full text
  • For Assignment 5: MA Gluck and CE Myers (1993) Hippocampal mediation of stimulus representation: A computational theory. Hippocampus 3(4), 491-516.


Exercises and Assignments:

Note: Your 4 highest graded assignments will form 20% of your final grade. Assignments handed in late will lose 10 points for each day late.


Bibliography: (Some books I recommend, and used to prepare mainly the psychology part of the course)

  • Domjan, M. - The Principles of Learning and Behavior (2003 - 5th Edition) - A contemporary textbook about animal learning.
  • Mackintosh, N.J. - The Psychology of Animal Learning (1974) - Great book, although hard to find (out of print).
  • Mackintosh, N.J. - Conditioning and Associative Learning (1983) - Shorter than the above and covers mostly theoretical aspects, not for complete beginners.
  • Dickinson, A. - Contemporary Animal Learning Theory (1980) - Although this was contemporary a long time ago, it is still very good and easy to read.
  • Sutton, R. and Barto, A. - Reinforcement Learning (1998) - A very good and accessible book explaining the computational field of reinforcement learning.

Extra Material:


Interesting (and useful) Links:

Last modified: Yael Niv - 5.5.2006