Carlos Diuk-Wasser, Ph.D.

Post-Doctoral Research Fellow
Department of Psychology and Princeton Neuroscience Institute
Princeton University


I moved! I am currently a Data Scientist at Facebook
I study the neural bases of hierarchical behavior. I am interested in how humans learn and plan action sequences in unknown, complex environments. I conduct behavioral and fMRI experiments and build computational models using the theoretical framework of reinforcement learning.

I am currently a Postdoctoral Research Fellow in the Niv and Botvinick labs.

Download my full CV.

Postdoctoral Research Fellow, Princeton University (2009-present) Advisors: Yael Niv, Matt Botvinick
Ph.D. in Computer Science, Rutgers University (2009). Advisor: Michael Littman
Licenciatura in Computer Science, University of Buenos Aires (2003).


Also check my Google Scholar profile.

Pre-prints submitted or under review:

2012. “Discovering hierarchical task structure”, Carlos Diuk, Natalia Córdova, Yael Niv and Matthew M. Botvinick. Draft available.

Peer-reviewed Publications:

In press. “Divide and conquer: hierarchical reinforcement learning and task decomposition in humans.”, Carlos Diuk, Anna Schapiro, Natalia Córdova, Jose Ribas-Fernandes, Yael Niv and Matthew M. Botvinick. In Computational and Robotic Models of the Hierarchical Organization of Behavior. Edited by Baldassare G and Mirolli M. Springer Verlag.

2013. “Hierarchical Learning Induces Two Simultaneous, But Separable, Prediction Errors in Human Basal Ganglia”, Carlos Diuk, Karin Tsai, Jonathan Wallis, Matthew M. Botvinick and Yael Niv. The Journal of Neuroscience. link

2012. “A quantitative philology of introspection”, Carlos Diuk, Diego F. Slezak, Iván Raskovsky, Mariano Sigman and Guillermo Cecchi. Frontiers in Integrative Neuroscience. link

2011. “A Neural Signature of Hierarchical Reinforcement Learning”, Jos?J.F. Ribas-Fernandes, Alec Solway, Carlos Diuk, Joseph T. McGuire, Andrew G. Barto, Yael Niv and Matthew M. Botvinick. Neuron, Volume 71, Issue 2, 370-379. abstract

2010. “Generalizing Apprenticeship Learning across Hypothesis Classes”, Thomas J. Walsh, Kaushik Subramanian, Michael L. Littman and Carlos Diuk. ICML 2010. pdf

2009. “The Adaptive k-Meteorologists Problem and Its Application to Structure Learning and Feature Selection in Reinforcement Learning”, Carlos Diuk, Lihong Li and Bethany Leffler. ICML 2009. pdf / videolecture

2009. “Exploring Compact Reinforcement-Learning Representations with Linear Regression”, Thomas J. Walsh, Istv? Szita, Carlos Diuk, and Michael L. Littman. UAI 2009. pdf

2008. “An Object-Oriented Representation for Efficient Reinforcement Learning”, Carlos Diuk, Andre Cohen and Michael L. Littman. ICML 2008. pdf / videolecture

2008. “Hierarchical Reinforcement Learning”, Carlos Diuk and Michael Littman. Encyclopedia of Artificial Intelligence, IGI Global, July 2008.

2007. “Efficient Structure Learning in Factored-state MDPs”, Alexander L. Strehl, Carlos Diuk and Michael L. Littman. AAAI 2007.pdf

2007. “An adaptive anomaly detector for worm detection ”, John Mark Agosta, Carlos Diuk, Jaideep Chandrashekar and Carl Livadas. Second Workshop on Tackling Computer Systems Problems with Machine Learning Techniques (sysML-07). pdf

2006. “A Hierarchical Approach to Efficient Reinforcement Learning in Deterministic Domains”, Carlos Diuk, Alexander L. Strehl and Michael L. Littman. AAMAS’06. pdf

2003. “Una herramienta computacional para la reconstrucci? de genealog?s hist?icas.”, Carlos Diuk. Licenciatura Dissertation. Dept. of Computer Science, Universidad de Buenos Aires. pdf

2002. “Computer tools for reconstructing a genealogy”, Carlos Diuk and Enrique Tándeter. International Journal of History and Computing. Edinburgh University Press. pdf

PhD Thesis: "An object-oriented representation for efficient reinforcement learning". pdf

Other publications and talks:

2013. “Compositional policy priors”, Wingate, David; Diuk, Carlos; O'Donnell, Timothy; Tenenbaum, Joshua; Gershman, Samuel. Technical Report 2013-007. MIT CSAIL. link

2010. “Hierarchical Reinforcement Learning: An fMRI Study of learning in a two-level gambling task ”, Carlos Diuk, Matthew Botvinick, Andrew Barto and Yael Niv. Society for Neuroscience Meeting 2010 (SfN 2010). pdf

2010. “The emergence of the modern concept of introspection: a quantitative linguistic analysis”, Ivan Raskovsky, Diego Fernández Slezak, Carlos Diuk and Guillermo Cecchi. NAACL Young Investigators Workshop 2010. pdf

2006. Invited Speaker at AAMAS Hierarchical Autonomous Agents and Multi-Agent Systems: “A Hierarchical Approach to Efficient Reinforcement Learning”.

2006. “Using Classifiers to Transfer Knowledge ”, Thomas J. Walsh, Carlos Diuk and Michael Littman. Presented at the New York Academy of Science Machine Learning Symposium.

2006. “Efficient exploration and learning of structure in factored-state MDPs ”, Carlos Diuk, Michael L. Littman, Alexander L. Strehl. Presented at NIPS Workshop “Towards a New Reinforcement Learning?”.

2005. “A Hierarchical Approach to Efficient Reinforcement Learning in Factored State Spaces”, Carlos Diuk, Michael L. Littman, and Alexander L. Strehl. Presented at the the 22nd International Conference on Machine Learning (ICML 2005), Workshop on Rich Representations for Reinforcement Learning, Bonn, Germany, 2005.

Spring 2013 Assistant Instructor PSY/NEU259 Introduction to Cognitive Neuroscience. Princeton University.
Fall 2012 Instructor at Champalimaud Reinforcement Learning and Neuroscience Course. Champalimaud Center for the Unknown, Lisbon, Portugal.
Summer 2012 Invited Professor at University of Buenos Aires.
Summer 2010 Reinforcement Learning at Escuela de Ciencias Informáticas, University of Buenos Aires.
Fall 2008 CS500 - Bayesian Reinforcement Learning at Rutgers University.
Spring 2004 CS344 - Design and Analysis of Algorithms at Rutgers University
Fall 2003 CS344 - Design and Analysis of Algorithms at Rutgers University