Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

From reinforcement learning models to psychiatric and neurological disorders

Nature Neuroscience volume 14, pages 154–162 (2011)Cite this article

Subjects

Abstract

Over the last decade and a half, reinforcement learning models have fostered an increasingly sophisticated understanding of the functions of dopamine and cortico-basal ganglia-thalamo-cortical (CBGTC) circuits. More recently, these models, and the insights that they afford, have started to be used to understand important aspects of several psychiatric and neurological disorders that involve disturbances of the dopaminergic system and CBGTC circuits. We review this approach and its existing and potential applications to Parkinson's disease, Tourette's syndrome, attention-deficit/hyperactivity disorder, addiction, schizophrenia and preclinical animal models used to screen new antipsychotic drugs. The approach's proven explanatory and predictive power bodes well for the continued growth of computational psychiatry and computational neurology.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Principles of computational psychiatry and computational neurology.
Figure 2: Anatomy and modeling of CBGTC loops.
Figure 3: The probabilistic selection task58.

Similar content being viewed by others

References

  1. Charney, D.S. et al. Neuroscience research agenda to guide development of a pathophysiologically based classification system. in A Research Agenda for DSM-V (eds. Kupfer, D.J., First, M.B. & Regier, D.A.) 31–83 (American Psychiatric Association, Washington, D.C., 2002).

  2. Hyman, S.E. Can neuroscience be integrated into the DSM-V? Nat. Rev. Neurosci. 8, 725–732 (2007).

    Article  CAS  PubMed  Google Scholar 

  3. Cools, R. Dopaminergic modulation of cognitive function-implications for L-DOPA treatment in Parkinson's disease. Neurosci. Biobehav. Rev. 30, 1–23 (2006).

    Article  CAS  PubMed  Google Scholar 

  4. Frank, M.J., Samanta, J., Moustafa, A.A. & Sherman, S.J. Hold your horses: impulsivity, deep brain stimulation, and medication in parkinsonism. Science 318, 1309–1312 (2007).

    Article  CAS  PubMed  Google Scholar 

  5. Maia, T.V. Reinforcement learning, conditioning, and the brain: successes and challenges. Cogn. Affect. Behav. Neurosci. 9, 343–364 (2009).

    Article  PubMed  Google Scholar 

  6. Dayan, P., Niv, Y., Seymour, B. & Daw, N.D. The misbehavior of value and the discipline of the will. Neural Netw. 19, 1153–1160 (2006).

    Article  PubMed  Google Scholar 

  7. McClure, S.M., Daw, N.D. & Montague, P.R. A computational substrate for incentive salience. Trends Neurosci. 26, 423–428 (2003).

    Article  CAS  PubMed  Google Scholar 

  8. Dayan, P. & Balleine, B.W. Reward, motivation, and reinforcement learning. Neuron 36, 285–298 (2002).

    Article  CAS  PubMed  Google Scholar 

  9. Niv, Y., Daw, N.D., Joel, D. & Dayan, P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl.) 191, 507–520 (2007).

    Article  CAS  Google Scholar 

  10. Wiecki, T.V. & Frank, M.J. Neurocomputational models of motor and cognitive deficits in Parkinson's disease. Prog. Brain Res. 183, 275–297 (2010).

    Article  CAS  PubMed  Google Scholar 

  11. Frank, M.J., Santamaria, A., O'Reilly, R.C. & Willcutt, E. Testing computational models of dopamine and noradrenaline dysfunction in attention deficit/hyperactivity disorder. Neuropsychopharmacology 32, 1583–1599 (2007).

    Article  CAS  PubMed  Google Scholar 

  12. Frank, M.J., Scheres, A. & Sherman, S.J. Understanding decision-making deficits in neurological conditions: insights from models of natural action selection. Phil. Trans. R. Soc. Lond. B 362, 1641–1654 (2007).

    Article  Google Scholar 

  13. Waltz, J.A., Frank, M.J., Robinson, B.M. & Gold, J.M. Selective reinforcement learning deficits in schizophrenia support predictions from computational models of striatal-cortical dysfunction. Biol. Psychiatry 62, 756–764 (2007).

    Article  PubMed  PubMed Central  Google Scholar 

  14. Redish, A.D. Addiction as a computational process gone awry. Science 306, 1944–1947 (2004).

    Article  CAS  PubMed  Google Scholar 

  15. Sutton, R.S. & Barto, A.G. Reinforcement Learning: An Introduction (MIT Press, Cambridge, Massachusetts, 1998).

  16. Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Bayer, H.M., Lau, B. & Glimcher, P.W. Statistics of midbrain dopamine neuron spike trains in the awake primate. J. Neurophysiol. 98, 1428–1439 (2007).

    Article  PubMed  Google Scholar 

  18. Daw, N.D., Kakade, S. & Dayan, P. Opponent interactions between serotonin and dopamine. Neural Netw. 15, 603–616 (2002).

    Article  PubMed  Google Scholar 

  19. Schultz, W. Dopamine signals for reward value and risk: basic and recent data. Behav. Brain Funct. 6, 24 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Morris, G., Schmidt, R. & Bergman, H. Striatal action-learning based on dopamine concentration. Exp. Brain Res. 200, 307–317 (2010).

    Article  PubMed  Google Scholar 

  21. Tsai, H.C. et al. Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science 324, 1080–1084 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Maia, T.V. Two-factor theory, the actor-critic model, and conditioned avoidance. Learn. Behav. 38, 50–67 (2010).

    Article  PubMed  Google Scholar 

  23. O'Doherty, J. et al. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454 (2004).

    CAS  PubMed  Google Scholar 

  24. Alexander, G.E., DeLong, M.R. & Strick, P.L. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9, 357–381 (1986).

    Article  CAS  PubMed  Google Scholar 

  25. Yin, H.H. & Knowlton, B.J. The role of the basal ganglia in habit formation. Nat. Rev. Neurosci. 7, 464–476 (2006).

    Article  CAS  PubMed  Google Scholar 

  26. Mink, J.W. The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 50, 381–425 (1996).

    Article  CAS  PubMed  Google Scholar 

  27. Prescott, T.J., Gurney, K. & Redgrave, P. Basal ganglia. in The Handbook of Brain Theory and Neural Networks (ed. Arbib, M.A.) 147–151 (MIT Press, Cambridge, Massachusetts, 2003).

  28. Frank, M.J. Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and non-medicated Parkinsonism. J. Cogn. Neurosci. 17, 51–72 (2005).

    Article  PubMed  Google Scholar 

  29. Samejima, K., Ueda, Y., Doya, K. & Kimura, M. Representation of action-specific reward values in the striatum. Science 310, 1337–1340 (2005).

    Article  CAS  PubMed  Google Scholar 

  30. Albin, R.L., Young, A.B. & Penney, J.B. The functional anatomy of basal ganglia disorders. Trends Neurosci. 12, 366–375 (1989).

    Article  CAS  PubMed  Google Scholar 

  31. DeLong, M.R. Primate models of movement disorders of basal ganglia origin. Trends Neurosci. 13, 281–285 (1990).

    Article  CAS  PubMed  Google Scholar 

  32. Nambu, A., Tokuno, H. & Takada, M. Functional significance of the cortico-subthalamo-pallidal 'hyperdirect' pathway. Neurosci. Res. 43, 111–117 (2002).

    Article  PubMed  Google Scholar 

  33. Bogacz, R. & Gurney, K. The basal ganglia and cortex implement optimal decision making between alternative actions. Neural Comput. 19, 442–477 (2007).

    Article  PubMed  Google Scholar 

  34. Brown, J.W., Bullock, D. & Grossberg, S. How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades. Neural Netw. 17, 471–510 (2004).

    Article  PubMed  Google Scholar 

  35. Frank, M.J. Hold your horses: a dynamic computational role for the subthalamic nucleus in decision making. Neural Netw. 19, 1120–1136 (2006).

    Article  PubMed  Google Scholar 

  36. Gurney, K., Prescott, T.J. & Redgrave, P. A computational model of action selection in the basal ganglia. II. Analysis and simulation of behaviour. Biol. Cybern. 84, 411–423 (2001).

    Article  CAS  PubMed  Google Scholar 

  37. Frank, M.J. & Fossella, J.A. Neurogenetics and pharmacology of learning, motivation, and cognition. Neuropsychopharmacology 36, 133–152 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  38. Hikida, T., Kimura, K., Wada, N., Funabiki, K. & Nakanishi, S. Distinct roles of synaptic transmission in direct and indirect striatal pathways to reward and aversive behavior. Neuron 66, 896–907 (2010).

    Article  CAS  PubMed  Google Scholar 

  39. Kravitz, A.V. et al. Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature 466, 622–626 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Gerfen, C.R. Molecular effects of dopamine on striatal-projection pathways. Trends Neurosci. 23, S64–S70 (2000).

    Article  CAS  PubMed  Google Scholar 

  41. Shen, W., Flajolet, M., Greengard, P. & Surmeier, D.J. Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321, 848–851 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Creese, I., Sibley, D.R., Hamblin, M.W. & Leff, S.E. The classification of dopamine receptors: relationship to radioligand binding. Annu. Rev. Neurosci. 6, 43–71 (1983).

    Article  CAS  PubMed  Google Scholar 

  43. Waltz, J.A., Frank, M.J., Wiecki, T.V. & Gold, J.M. Altered probabilistic learning and response biases in schizophrenia: behavioral evidence and neurocomputational modeling. Neuropsychology published online, doi:10.1037/a0020882 (22 November 2010).

  44. Haber, S.N. The primate basal ganglia: parallel and integrative networks. J. Chem. Neuroanat. 26, 317–330 (2003).

    Article  PubMed  Google Scholar 

  45. Postuma, R.B. & Dagher, A. Basal ganglia functional connectivity based on a meta-analysis of 126 positron emission tomography and functional magnetic resonance imaging publications. Cereb. Cortex 16, 1508–1521 (2006).

    Article  PubMed  Google Scholar 

  46. Frank, M.J., Loughry, B. & O'Reilly, R.C. Interactions between frontal cortex and basal ganglia in working memory: a computational model. Cogn. Affect. Behav. Neurosci. 1, 137–160 (2001).

    Article  CAS  PubMed  Google Scholar 

  47. O'Reilly, R.C. & Frank, M.J. Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput. 18, 283–328 (2006).

    Article  PubMed  Google Scholar 

  48. Baier, B. et al. Keeping memory clear and stable: the contribution of human basal ganglia and prefrontal cortex to working memory. J. Neurosci. 30, 9788–9792 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Tzschentke, T.M. Measuring reward with the conditioned place preference paradigm: a comprehensive review of drug effects, recent progress and new issues. Prog. Neurobiol. 56, 613–672 (1998).

    Article  CAS  PubMed  Google Scholar 

  50. Lobo, M.K. et al. Cell type-specific loss of BDNF signaling mimics optogenetic control of cocaine reward. Science 330, 385–390 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. Frank, M.J. & Claus, E.D. Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. Psychol. Rev. 113, 300–326 (2006).

    Article  PubMed  Google Scholar 

  52. Kalanithi, P.S. et al. Altered parvalbumin-positive neuron distribution in basal ganglia of individuals with Tourette syndrome. Proc. Natl. Acad. Sci. USA 102, 13307–13312 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  53. Biederman, J. & Spencer, T. Attention-deficit/hyperactivity disorder (ADHD) as a noradrenergic disorder. Biol. Psychiatry 46, 1234–1242 (1999).

    Article  CAS  PubMed  Google Scholar 

  54. Guillin, O., Abi-Dargham, A. & Laruelle, M. Neurobiology of dopamine in schizophrenia. Int. Rev. Neurobiol. 78, 1–39 (2007).

    Article  CAS  PubMed  Google Scholar 

  55. Nikolaus, S., Antke, C. & Muller, H.W. In vivo imaging of synaptic function in the central nervous system. II. Mental and affective disorders. Behav. Brain Res. 204, 32–66 (2009).

    Article  PubMed  Google Scholar 

  56. Stahl, S.M. Beyond the dopamine hypothesis to the NMDA glutamate receptor hypofunction hypothesis of schizophrenia. CNS Spectr. 12, 265–268 (2007).

    Article  PubMed  Google Scholar 

  57. Rolls, E.T., Loh, M., Deco, G. & Winterer, G. Computational models of schizophrenia and dopamine modulation in the prefrontal cortex. Nat. Rev. Neurosci. 9, 696–709 (2008).

    Article  CAS  PubMed  Google Scholar 

  58. Frank, M.J., Seeberger, L.C. & O'Reilly, R.C. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306, 1940–1943 (2004).

    Article  CAS  PubMed  Google Scholar 

  59. Moustafa, A.A., Sherman, S.J. & Frank, M.J. A dopaminergic basis for working memory, learning and attentional shifting in Parkinsonism. Neuropsychologia 46, 3144–3156 (2008).

    Article  PubMed  Google Scholar 

  60. Cools, R., Miyakawa, A., Sheridan, M. & D'Esposito, M. Enhanced frontal function in Parkinson's disease. Brain 133, 225–233 (2010).

    Article  CAS  PubMed  Google Scholar 

  61. Mink, J.W. Basal ganglia dysfunction in Tourette's syndrome: a new hypothesis. Pediatr. Neurol. 25, 190–198 (2001).

    Article  CAS  PubMed  Google Scholar 

  62. Delfs, J.M. & Kelley, A.E. The role of D1 and D2 dopamine receptors in oral stereotypy induced by dopaminergic stimulation of the ventrolateral striatum. Neuroscience 39, 59–67 (1990).

    Article  CAS  PubMed  Google Scholar 

  63. Walters, J.R., Bergstrom, D.A., Carlson, J.H., Chase, T.N. & Braun, A.R. D1 dopamine receptor activation required for postsynaptic expression of D2 agonist effects. Science 236, 719–722 (1987).

    Article  CAS  PubMed  Google Scholar 

  64. Steeves, T.D. & Fox, S.H. Neurobiological basis of serotonin-dopamine antagonists in the treatment of Gilles de la Tourette syndrome. Prog. Brain Res. 172, 495–513 (2008).

    Article  CAS  PubMed  Google Scholar 

  65. Palminteri, S. et al. Pharmacological modulation of subliminal learning in Parkinson's and Tourette's syndromes. Proc. Natl. Acad. Sci. USA 106, 19179–19184 (2009).

    Article  PubMed  PubMed Central  Google Scholar 

  66. Peterson, B.S. et al. Neuroanatomical circuitry. in Tourette's Syndrome: Tics, Obsessions, Compulsions. Developmental Psychopathology and Clinical Care (eds. Leckman, J.F. & Cohen, D.J.) 230–259 (John Wiley & Sons, New York, 1999).

  67. Bohlhalter, S. et al. Neural correlates of tic generation in Tourette syndrome: an event-related functional MRI study. Brain 129, 2029–2037 (2006).

    Article  CAS  PubMed  Google Scholar 

  68. Barkley, R.A. Behavioral inhibition, sustained attention, and executive functions: constructing a unifying theory of ADHD. Psychol. Bull. 121, 65–94 (1997).

    Article  PubMed  Google Scholar 

  69. Sagvolden, T. & Sergeant, J.A. Attention deficit/hyperactivity disorder—from brain dysfunctions to behaviour. Behav. Brain Res. 94, 1–10 (1998).

    Article  CAS  PubMed  Google Scholar 

  70. Sonuga-Barke, E.J. Causal models of attention-deficit/hyperactivity disorder: from common simple deficits to multiple developmental pathways. Biol. Psychiatry 57, 1231–1238 (2005).

    Article  PubMed  Google Scholar 

  71. Sagvolden, T., Johansen, E.B., Aase, H. & Russell, V.A. A dynamic developmental theory of attention-deficit/hyperactivity disorder (ADHD) predominantly hyperactive/impulsive and combined subtypes. Behav. Brain Sci. 28, 397–419 (2005).

    Article  PubMed  Google Scholar 

  72. Smith, A.J., Becker, S. & Kapur, S. A computational model of the functional role of the ventral-striatal D2 receptor in the expression of previously acquired behaviors. Neural Comput. 17, 361–395 (2005).

    Article  PubMed  Google Scholar 

  73. Pattij, T. & Vanderschuren, L.J. The neuropharmacology of impulsive behaviour. Trends Pharmacol. Sci. 29, 192–199 (2008).

    Article  CAS  PubMed  Google Scholar 

  74. Winstanley, C.A., Theobald, D.E., Dalley, J.W. & Robbins, T.W. Interactions between serotonin and dopamine in the control of impulsive choice in rats: therapeutic implications for impulse control disorders. Neuropsychopharmacology 30, 669–682 (2005).

    Article  CAS  PubMed  Google Scholar 

  75. Kheramin, S. et al. Effects of orbital prefrontal cortex dopamine depletion on inter-temporal choice: a quantitative analysis. Psychopharmacology (Berl.) 175, 206–214 (2004).

    Article  CAS  Google Scholar 

  76. Durstewitz, D., Seamans, J.K. & Sejnowski, T.J. Dopamine-mediated stabilization of delay-period activity in a network model of prefrontal cortex. J. Neurophysiol. 83, 1733–1750 (2000).

    Article  CAS  PubMed  Google Scholar 

  77. Maia, T.V. & Cleeremans, A. Consciousness: converging insights from connectionist modeling and neuroscience. Trends Cogn. Sci. 9, 397–404 (2005).

    Article  PubMed  Google Scholar 

  78. Figner, B. et al. Lateral prefrontal cortex and self-control in intertemporal choice. Nat. Neurosci. 13, 538–539 (2010).

    Article  CAS  PubMed  Google Scholar 

  79. Volkow, N.D., Fowler, J.S., Wang, G.J., Swanson, J.M. & Telang, F. Dopamine in drug abuse and addiction: results of imaging studies and treatment implications. Arch. Neurol. 64, 1575–1579 (2007).

    Article  PubMed  Google Scholar 

  80. Panlilio, L.V., Thorndike, E.B. & Schindler, C.W. Blocking of conditioning to a cocaine-paired stimulus: testing the hypothesis that cocaine perpetually produces a signal of larger-than-expected reward. Pharmacol. Biochem. Behav. 86, 774–777 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  81. Dezfouli, A. et al. A neurocomputational model for cocaine addiction. Neural Comput. 21, 2869–2893 (2009).

    Article  PubMed  Google Scholar 

  82. Redish, A.D., Jensen, S. & Johnson, A. A unified framework for addiction: vulnerabilities in the decision process. Behav. Brain Sci. 31, 415–437, discussion 437–487 (2008).

    PubMed  PubMed Central  Google Scholar 

  83. Schoenbaum, G., Roesch, M.R. & Stalnaker, T.A. Orbitofrontal cortex, decision-making and drug addiction. Trends Neurosci. 29, 116–124 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  84. Berridge, K.C. & Robinson, T.E. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res. Brain Res. Rev. 28, 309–369 (1998).

    Article  CAS  PubMed  Google Scholar 

  85. Kapur, S. Psychosis as a state of aberrant salience: a framework linking biology, phenomenology, and pharmacology in schizophrenia. Am. J. Psychiatry 160, 13–23 (2003).

    Article  PubMed  Google Scholar 

  86. Corlett, P.R., Honey, G.D. & Fletcher, P.C. From prediction error to psychosis: ketamine as a pharmacological model of delusions. J. Psychopharmacol. 21, 238–252 (2007).

    Article  CAS  PubMed  Google Scholar 

  87. Corlett, P.R. et al. Disrupted prediction-error signal in psychosis: evidence for an associative account of delusions. Brain 130, 2387–2400 (2007).

    Article  CAS  PubMed  Google Scholar 

  88. Murray, G.K. et al. Substantia nigra/ventral tegmental reward prediction error disruption in psychosis. Mol. Psychiatry 13, 267–276 (2008).

    Article  CAS  Google Scholar 

  89. Waltz, J.A. et al. Patients with schizophrenia have a reduced neural response to both unpredictable and predictable primary reinforcers. Neuropsychopharmacology 34, 1567–1577 (2009).

    Article  CAS  PubMed  Google Scholar 

  90. Strauss, G.P. et al. Deficits in positive reinforcement learning and uncertainty-driven exploration are associated with distinct aspects of negative symptoms in schizophrenia. Biol. Psychiatry (in the press).

  91. Wadenberg, M.L. & Hicks, P.B. The conditioned avoidance response test re-evaluated: is it a sensitive test for the detection of potentially atypical antipsychotics? Neurosci. Biobehav. Rev. 23, 851–862 (1999).

    Article  CAS  PubMed  Google Scholar 

  92. Fibiger, H.C., Phillips, A.G. & Zis, A.P. Deficits in instrumental responding after 6-hydroxydopamine lesions of the nigro-neostriatal dopaminergic projection. Pharmacol. Biochem. Behav. 2, 87–96 (1974).

    Article  CAS  PubMed  Google Scholar 

  93. Fantin, G. & Bottecchia, D. Effect of nucleus accumbens destruction in rat. Experientia 40, 573–575 (1984).

    Article  CAS  PubMed  Google Scholar 

  94. Wadenberg, M.L., Ericson, E., Magnusson, O. & Ahlenius, S. Suppression of conditioned avoidance behavior by the local application of (–)sulpiride into the ventral, but not the dorsal, striatum of the rat. Biol. Psychiatry 28, 297–307 (1990).

    Article  CAS  PubMed  Google Scholar 

  95. Salamone, J.D. & Correa, M. Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behav. Brain Res. 137, 3–25 (2002).

    Article  CAS  PubMed  Google Scholar 

  96. Yin, H.H., Ostlund, S.B. & Balleine, B.W. Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico-basal ganglia networks. Eur. J. Neurosci. 28, 1437–1448 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  97. Maia, T.V., Cooney, R.E. & Peterson, B.S. The neural bases of obsessive-compulsive disorder in children and adults. Dev. Psychopathol. 20, 1251–1283 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  98. Dayan, P., Kakade, S. & Montague, P.R. Learning and selective attention. Nat. Neurosci. 3 Suppl: 1218–1223 (2000).

    Article  CAS  PubMed  Google Scholar 

  99. Gershman, S.J., Blei, D.M. & Niv, Y. Context, learning, and extinction. Psychol. Rev. 117, 197–209 (2010).

    Article  PubMed  Google Scholar 

  100. Maia, T.V. Fear conditioning and social groups: statistics, not genetics. Cogn. Sci. 33, 1232–1251 (2009).

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Preparation of this article was funded by a Research Associate Award from the New York State Psychiatric Institute and the Research Foundation for Mental Hygiene, by National Institute of Mental Health grant R01 MH080066 and by a grant from the Michael J. Fox Foundation for Parkinson's Research.

Author information

Authors and Affiliations

  1. Department of Psychiatry, Columbia University, New York, New York, USA

    Tiago V Maia

  2. New York State Psychiatric Institute, New York, New York, USA

    Tiago V Maia

  3. Departments of Cognitive, Linguistic and Psychological Sciences, and Psychiatry and Human Behavior, Brown University, Providence, Rhode Island, USA

    Michael J Frank

  4. Brown Institute for Brain Science, Brown University, Providence, Rhode Island, USA

    Michael J Frank

Authors
  1. Tiago V Maia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael J Frank
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Tiago V Maia or Michael J Frank.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Maia, T., Frank, M. From reinforcement learning models to psychiatric and neurological disorders. Nat Neurosci 14, 154–162 (2011). https://doi.org/10.1038/nn.2723

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.2723

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing