Monthly 208 pp. per issue 8 1/2 x 11, illustrated Founded: 1989 ISSN 0898-929X E-ISSN 1530-8898
|
February 2015, Vol. 27, No. 2, Pages 319-333
Posted Online August 29, 2014.
(doi:10.1162/jocn_a_00709)
© 2014 Massachusetts Institute of Technology
Cognitive Control Predicts Use of Model-based Reinforcement Learning A. Ross Otto1*, Anya Skatova2*, Seth Madlon-Kay3, and Nathaniel D. Daw11New York University 2University of Nottingham 3Duke University *The first two authors contributed equally to this article, and ordering was determined arbitrarily.
Accounts of decision-making and its neural substrates have long posited the operation of separate, competing valuation systems in the control of choice behavior. Recent theoretical and experimental work suggest that this classic distinction between behaviorally and neurally dissociable systems for habitual and goal-directed (or more generally, automatic and controlled) choice may arise from two computational strategies for reinforcement learning (RL), called model-free and model-based RL, but the cognitive or computational processes by which one system may dominate over the other in the control of behavior is a matter of ongoing investigation. To elucidate this question, we leverage the theoretical framework of cognitive control, demonstrating that individual differences in utilization of goal-related contextual information—in the service of overcoming habitual, stimulus-driven responses—in established cognitive control paradigms predict model-based behavior in a separate, sequential choice task. The behavioral correspondence between cognitive control and model-based RL compellingly suggests that a common set of processes may underpin the two behaviors. In particular, computational mechanisms originally proposed to underlie controlled behavior may be applicable to understanding the interactions between model-based and model-free choice behavior. Cited byVishnu P. Murty, Finnegan Calabro, Beatriz Luna. (2016) The role of experience in adolescent cognitive development: Integration of executive, memory, and mesolimbic systems. Neuroscience & Biobehavioral Reviews 7046-58. Online publication date: 1-Nov-2016. CrossRef Adam J. Culbreth, Andrew Westbrook, Ziye Xu, Deanna M. Barch, James A. Waltz. (2016) Intact Ventral Striatal Prediction Error Signaling in Medicated Schizophrenia Patients. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging 1:5474-483. Online publication date: 1-Sep-2016. CrossRef Arkady Konovalov, Ian Krajbich. (2016) Gaze data reveal distinct choice processes underlying model-based and model-free reinforcement learning. Nature Communications 712438. Online publication date: 11-Aug-2016. CrossRef Miriam Sebold, Daniel J. Schad, Stephan Nebe, Maria Garbusow, Elisabeth Jünger, Nils B. Kroemer, Norbert Kathmann, Ulrich S. Zimmermann, Michael N. Smolka, Michael A. Rapp, Andreas Heinz, Quentin J. M. Huys. (2016) Don't Think, Just Feel the Music: Individuals with Strong Pavlovian-to-Instrumental Transfer Effects Rely Less on Model-based Reinforcement Learning. Journal of Cognitive Neuroscience 28:7985-995. Online publication date: 31-May-2016. Abstract | Full Text | PDF (560 KB) | PDF Plus (400 KB)  Candace M. Raio, Elizabeth V. Goldfarb, Karolina M. Lempert, Peter Sokol-Hessner. (2016) Classifying emotion regulation strategies. Nature Reviews Neuroscience 17:8532-532. Online publication date: 9-Jun-2016. CrossRef Andrea M. F. Reiter, Lorenz Deserno, Tilmann Wilbertz, Hans-Jochen Heinze, Florian Schlagenhauf. (2016) Risk Factors for Addiction and Their Association with Model-Based Behavioral Control. Frontiers in Behavioral Neuroscience 10. Online publication date: 17-Mar-2016. CrossRef Hilary J. Don, Micah B. Goldwater, A. Ross Otto, Evan J. Livesey. (2016) Rule abstraction, model-based choice, and cognitive reflection. Psychonomic Bulletin & Review. Online publication date: 23-Feb-2016. CrossRef Madeleine E. Sharp, Karin Foerde, Nathaniel D. Daw, Daphna Shohamy. (2016) Dopamine selectively remediates ‘model-based’ reward learning: a computational approach. Brain 139:2355-364. Online publication date: 17-Dec-2015. CrossRef Wendy Wood, Dennis Rünger. (2016) Psychology of Habit. Annual Review of Psychology 67:1289-314. Online publication date: 4-Jan-2016. CrossRef Nicholas Shea, Chris D. Frith. (2016) Dual-process theories and consciousness: the case for ‘Type Zero’ cognition: Table 1.. Neuroscience of Consciousness 2016:1niw005. Online publication date: 8-May-2016. CrossRef Fiery Cushman, Adam Morris. (2015) Habitual control of goal selection in humans. Proceedings of the National Academy of Sciences 112:4513817-13822. Online publication date: 12-Oct-2015. CrossRef Y-Lan Boureau, Peter Sokol-Hessner, Nathaniel D. Daw. (2015) Deciding How To Decide: Self-Control and Meta-Decision Making. Trends in Cognitive Sciences 19:11700-710. Online publication date: 1-Nov-2015. CrossRef L Deserno, T Wilbertz, A Reiter, A Horstmann, J Neumann, A Villringer, H-J Heinze, F Schlagenhauf. (2015) Lateral prefrontal model-based signatures are reduced in healthy individuals with high trait impulsivity. Translational Psychiatry 5:10e659. Online publication date: 13-Oct-2015. CrossRef Claire M. Gillan, A. Ross Otto, Elizabeth A. Phelps, Nathaniel D. Daw. (2015) Model-based learning protects against forming habits. Cognitive, Affective, & Behavioral Neuroscience 15:3523-536. Online publication date: 24-Mar-2015. CrossRef Roshan Cools. (2015) The cost of dopamine for dynamic cognitive control. Current Opinion in Behavioral Sciences 4152-159. Online publication date: 1-Aug-2015. CrossRef Joshua W. Buckholtz. (2015) Social norms, self-control, and the value of antisocial behavior. Current Opinion in Behavioral Sciences 3122-129. Online publication date: 1-Jun-2015. CrossRef Andrew Westbrook, Todd S. Braver. (2015) Cognitive effort: A neuroeconomic approach. Cognitive, Affective, & Behavioral Neuroscience 15:2395-415. Online publication date: 12-Feb-2015. CrossRef Danielle S Bassett, Muzhi Yang, Nicholas F Wymbs, Scott T Grafton. (2015) Learning-induced autonomy of sensorimotor systems. Nature Neuroscience 18:5744-751. Online publication date: 6-Apr-2015. CrossRef
|