Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

Generalization in vision and motor control

Abstract

Learning is more than memory. It is not simply the building of a look-up table of labelled images, or a phone-directory-like list of motor acts and the corresponding sequences of muscle activation. Central to learning and intelligence is the ability to predict, that is, to generalize to new situations, beyond the memory of specific examples. The key to generalization, in turn, is the architecture of the system, more than the rules of synaptic plasticity. We propose a specific architecture for generalization for both the motor and the visual systems, and argue for a canonical microcircuit underlying visual and motor learning.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Tuned units in inferotemporal cortex.
Figure 2: A model of visual learning.
Figure 3: The generalization architectures of the visual and motor systems.
Figure 4: Spinal force fields combine linearly.

Similar content being viewed by others

References

  1. Vapnik, V. N. Statistical Learning Theory (Wiley, New York, 1998).

    MATH  Google Scholar 

  2. Bülthoff, H. & Edelman, S. Psychophysical support for a two-dimensional view interpolation theory of object recognition. Proc. Natl Acad. Sci. USA 89, 60–64 (1992).

    Article  ADS  Google Scholar 

  3. Riesenhuber, M. & Poggio, T. Hierarchical models of object recognition in cortex. Nature Neurosci. 2, 1019–1025 (1999).

    Article  CAS  Google Scholar 

  4. Riesenhuber, M. & Poggio, T. The Visual Neurosciences Vol. 2 (eds Chalupa, L. M. & Werner, J. S.) 1640–1653 (MIT Press, Cambridge, MA, 2003).

    Google Scholar 

  5. Palmeri, T. & Gauthier, I. Visual object understanding. Nature Rev. Neurosci. 5, 291–303 (2004).

    Article  CAS  Google Scholar 

  6. Hubel, D. H. & Wiesel, T. N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. 160, 106–154 (1962).

    Article  CAS  Google Scholar 

  7. Hubel, D. & Wiesel, T. Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat. J. Neurophysiol. 28, 229–289 (1965).

    Article  CAS  Google Scholar 

  8. Mountcastle, V. B. Modality and topographic properties of single neurons of cat's somatic sensory cortex. J. Neurophysiol. 20, 408–434 (1957).

    Article  CAS  Google Scholar 

  9. Gross, C. G. Handbook of Sensory Physiology Vol. VII/3B (eds Autrum, H., Jung, R., Lowenstein, W., Mckay, D. & Teuber, H.-L.) (Springer, Berlin, 1973).

    Google Scholar 

  10. Bruce, C., Desimone, R. & Gross, C. Visual properties of neurons in a polysensory area in the superior temporal sulcus of the macaque. J. Neurophysiol. 46, 369–384 (1981).

    Article  CAS  Google Scholar 

  11. Perrett, D. et al. Viewer-centred and object-centred coding of heads in the macaque temporal cortex. Exp. Brain Res. 86, 159–173 (1991).

    Article  CAS  Google Scholar 

  12. Perrett, D. & Oram, M. Neurophysiology of shape processing. Img. Vis. Comput. 11, 317–333 (1993).

    Article  Google Scholar 

  13. Logothetis, N., Pauls, J. & Poggio, T. Shape representation in the inferior temporal cortex of monkeys. Curr. Biol. 5, 552–563 (1995).

    Article  CAS  Google Scholar 

  14. Logothetis, N. & Sheinberg, D. Visual object recognition. Annu. Rev. Neurosci. 19, 577–621 (1996).

    Article  CAS  Google Scholar 

  15. Kobatake, E. & Tanaka, K. Neuronal selectivities to complex object features in the ventral visual pathway of the macaque cerebral cortex. J. Neurophysiol. 71, 856–857 (1994).

    Article  CAS  Google Scholar 

  16. Kobatake, E., Wang, G. & Tanaka, K. Effects of shape-discrimination training on the selectivity of inferotemporal cells in adult monkeys. J. Neurophysiol. 80, 324–330 (1998).

    Article  CAS  Google Scholar 

  17. DiCarlo, J. & Maunsell, J. Form representation in monkey inferotemporal cortex is virtually unaltered by free viewing. Nature Neurosci. 3, 814–821 (2000).

    Article  CAS  Google Scholar 

  18. Booth, M. & Rolls, E. View-invariant representations of familiar objects by neurons in the inferior temporal visual cortex. Cereb. Cortex 8, 510–523 (1998).

    Article  CAS  Google Scholar 

  19. Tanaka, K. Neuronal mechanisms of object recognition. Science 262, 685–688 (1993).

    Article  ADS  CAS  Google Scholar 

  20. Sato, T. Interactions of visual stimuli in the receptive fields of inferior temporal neurons in awake monkeys. Exp. Brain Res. 77, 23–30 (1989).

    Article  CAS  Google Scholar 

  21. Hietanen, J., Perrett, D., Benson, P. & Dittrich, W. The effects of lighting conditions on responses of cells selective for face views in the macaque temporal cortex. Exp. Brain Res. 89, 157–171 (1992).

    Article  CAS  Google Scholar 

  22. Missal, M., Vogels, R. & Orban, G. Responses of macaque inferior temporal neurons to overlapping shapes. Cereb. Cortex 7, 758–767 (1997).

    Article  CAS  Google Scholar 

  23. Poggio, T. A. Theory of how the brain might work. Cold Spring Harb. Symp. Quant. Biol. 4, 899–910 (1990).

    Article  Google Scholar 

  24. Olshausen, B. A. & Field, D. J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996).

    Article  ADS  CAS  Google Scholar 

  25. Pouget, A., Dayan, P. & Zemel, R. S. Computation and inference with population codes. Annu. Rev. Neurosci. 26, 381–410 (2003).

    Article  CAS  Google Scholar 

  26. Pouget, A. & Sejnowski, T. J. Spatial transformations in the parietal cortex using basis functions. J. Cogn. Neurosci. 9, 222–237 (1997).

    Article  CAS  Google Scholar 

  27. Georgopoulos, A. P., Kalaska, J. F., Caminiti, R. & Massey, J. T. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527–1537 (1982).

    Article  CAS  Google Scholar 

  28. Amirikian, B. & Georgopoulos, A. P. Modular organization of directionally tuned cells in the motor cortex: is there a short-range order? Proc. Natl Acad. Sci. USA 100, 12474–12479 (2003).

    Article  ADS  CAS  Google Scholar 

  29. Merzenich, M. M. & Brugge, J. F. Representation of the cochlear partition of the superior temporal plane of the macaque monkey. Brain Res. 50, 275–296 (1973).

    Article  CAS  Google Scholar 

  30. Wise, S. P., Moody, S. L., Blomstrom, K. J. & Mitz, A. R. Changes in motor cortical activity during visuomotor adaptation. Exp. Brain Res. 121, 285–299 (1998).

    Article  CAS  Google Scholar 

  31. Paz, R., Boraud, T., Natan, C., Bergman, H. & Vaadia, E. Preparatory activity in motor cortex reflects learning of local visuomotor skills. Nature Neurosci. 6, 882–890 (2003).

    Article  CAS  Google Scholar 

  32. Gribble, P. L. & Scott, S. H. Overlap of internal models in motor cortex for mechanical loads during reaching. Nature 417, 938–941 (2002).

    Article  ADS  CAS  Google Scholar 

  33. Gandolfo, F., Li, C. R., Benda, B., Padoa-Schioppa, C. & Bizzi, E. Cortical correlates of motor learning in monkeys adapting to a new dynamic environment. Proc. Natl Acad. Sci. USA 97, 2259–2263 (2000).

    Article  ADS  CAS  Google Scholar 

  34. Li, C. R., Padoa-Schioppa, C. & Bizzi, E. Neuronal correlates of motor performance and motor learning in the primary motor cortex of monkeys adapting to an external force field. Neuron 30, 593–607 (2001).

    Article  CAS  Google Scholar 

  35. Grillner, S. & Wallen, P. Central pattern generators for locomotion, with special reference to vertebrates. Annu. Rev. Neurosci. 8, 233–261 (1985).

    Article  CAS  Google Scholar 

  36. Stein, P. S., Victor, J. C., Field, E. C. & Currie, S. N. Bilateral control of hindlimb scratching in the spinal turtle: contralateral spinal circuitry contributes to the normal ipsilateral motor pattern of fictive rostral scratching. J. Neurosci. 15, 4343–4355 (1995).

    Article  CAS  Google Scholar 

  37. Loeb, G. E. Motoneurone task groups: coping with kinematic heterogeneity. J. Exp. Biol. 115, 137–146 (1985).

    CAS  PubMed  Google Scholar 

  38. Bizzi, E., Giszter, S. & Mussa-Ivaldi, F. A. Computations underlying the execution of movement: a novel biological perspective. Science 253, 287–291 (1991).

    Article  ADS  CAS  Google Scholar 

  39. Mussa-Ivaldi, F. A., Giszter, S. F. and Bizzi, E. Linear combinations of primitives in vertebrate motor control. Proc. Natl Acad. Sci. USA 91, 7534–7538 (1994).

    Article  ADS  CAS  Google Scholar 

  40. Jing, J., Cropper, E. C., Hurwitz, I. & Weiss, K. R. The construction of movement with behavior-specific and behavior-independent modules. J. Neurosci. 24, 6315–6325 (2004).

    Article  CAS  Google Scholar 

  41. Saltiel, P., Tresch, M. C. & Bizzi, E. Spinal cord modular organization and rhythm generation: an NMDA iontophoretic study in the frog. J. Neurophysiol. 80, 2323–2339 (1998).

    Article  CAS  Google Scholar 

  42. Grillner, S. Handbook of Physiology — The Nervous System edn 4 (eds Brookhart, J. M. & Mountcastle, V. B.) 1179–1236 (American Physiological Society, Bethesda, MD, 1981).

    Google Scholar 

  43. d'Avella, A., Saltiel, P. & Bizzi, E. Combinations of muscle synergies in the construction of a natural motor behaviour. Nature Neurosci. 6, 300–308 (2003).

    Article  CAS  Google Scholar 

  44. Lemay, M. A., Galagan, J. E., Hogan, N. & Bizzi, E. Modulation and vectorial summation of the spinalized frog's hindlimb end-point force produced by intraspinal electrical stimulation of the cord. IEEE Trans. Neural. Syst. Rehabil. Eng. 9, 12–23 (2001).

    Article  CAS  Google Scholar 

  45. Aoyagi, Y., Stein, R. B., Mushahwar, V. K. & Prochazka, A. The role of neuromuscular properties in determining the end-point of a movement. IEEE Trans. Neural. Syst. Rehabil. Eng. 12, 12–23 (2004).

    Article  CAS  Google Scholar 

  46. Tresch, M. C. & Bizzi, E. Responses from the spinal microstimulation in the chronically spinalized rats and their relationship to spinal systems activated by low threshold cutaneous stimulation. Exp. Brain Res. 129, 401–416 (1999).

    Article  CAS  Google Scholar 

  47. Kargo, W. J. & Giszter, S. F. Rapid correction of aimed movements by summation of force field primitives. J. Neurosci. 20, 409–426 (2000).

    Article  CAS  Google Scholar 

  48. Mussa-Ivaldi, F. A. in Proc. 1997 IEEE Int. Symp. Computational Intelligence in Robotics and Automation 84–90 (IEEE Computer Society, Los Alamitos, California, 1997).

    Google Scholar 

  49. Shadmehr, R. & Mussa-Ivaldi, F. A. Adaptive representation of dynamics during learning of a motor task. J. Neurosci. 14, 3208–3224 (1994).

    Article  CAS  Google Scholar 

  50. Chance, F., Nelson, S. & Abbott, L. Complex cells as cortically amplified simple cells. Nature Neurosci. 2, 277–282 (1999).

    Article  CAS  Google Scholar 

  51. Thoroughman, K. & Shadmer, R. Learning of action through adaptive combination of motor primitives. Nature 407, 740–746 (2000).

    Article  ADS  Google Scholar 

  52. Douglas, R. & Martin, K. A functional microcircuit for cat visual cortex. J. Physiol. (Lond.) 440, 735–769 (1991).

    Article  CAS  Google Scholar 

  53. Borg-Graham, L. J., Monier, C. & Frégnac Y. Visual input evokes transient and strong shunting inhibition in visual cortical neurons. Nature 393, 369–373 (1998).

    Article  ADS  CAS  Google Scholar 

  54. Mussa-Ivaldi, F. A. & Bizzi, E. Motor learning through the combination of primitives. Phil. Trans. R. Soc. Lond. B 355, 1755–1769 (2000).

    Article  CAS  Google Scholar 

  55. Giese, M. & Poggio, T. Neural mechanisms for the recognition of biological movements. Nature Rev. Neurosci. 4, 179–192 (2003).

    Article  CAS  Google Scholar 

  56. Hastie, T., Tibshirani, R. & Friedman, J. The Elements of Statistical Learning (Springer, Basel, 2001).

    Book  Google Scholar 

  57. Rodman, H. R., Scalaidhe, S. P. & Gross, C. G. Response properties of neurons in temporal cortical visual areas of infant monkeys. J. Neurophysiol. 70, 1115–1136 (1993).

    Article  CAS  Google Scholar 

  58. Belkin, M., Niyogi, P. & Sindhwani, V. Technical Report TR–2004–05 (University of Chicago, Chicago, 2004).

  59. Douglas, R. & Martin, K. Neural circuits of the neocortex. Annu. Rev. Neurosci. 27, 419–451 (2004).

    Article  CAS  Google Scholar 

  60. Salinas, E. & Abbott, L. F. Transfer of coded information from sensory to motor networks. J. Neurosci. 15, 6461–6474 (1995).

    Article  CAS  Google Scholar 

  61. Hogan, N. An organizing principle for a class of voluntary movements. J. Neusosci. 4, 2745–2754 (1984).

    Article  CAS  Google Scholar 

  62. Poggio, T. & Smale, S. The mathematics of learning: dealing with data. Notices Am. Math. Soc. 50, 537–544 (2003).

    MathSciNet  MATH  Google Scholar 

  63. Maruyama, M., Girosi, F. & T. Poggio, T. A. Connection Between GRBF and MLP. AI Memo 1291 (Massachusetts Institute of Technology, Cambridge, Massachusetts 1992).

    Google Scholar 

  64. Carandini, M. & Heeger, D. J. Summation and division by neurons in visual cortex. Science 264, 1333–1336 (1994).

    Article  ADS  CAS  Google Scholar 

  65. Carandini, M., Heeger, D. J. & Movshon, J. A. Linearity and normalization in simple cells of the macaque primary visual cortex. J. Neurosci. 17, 8621–8644 (1997).

    Article  CAS  Google Scholar 

  66. Reichardt, W., Poggio, T. & Hausen, K. Figure-ground discrimination by relative movement in the visual system of the fly II: towards the neural circuitry. Biol. Cybern. 46, 1–30 (1983).

    Article  Google Scholar 

  67. Yu, A. J., Giese, M. A. & Poggio, T. Biophysiologically plausible implementations of the maximum operation. Neural Comput. 14, 2857–2881 (2002).

    Article  Google Scholar 

Download references

Acknowledgements

We thank S. Mussa-Ivaldi, R. Shadmeher, G. Kreiman, and M. Riesenhuber for insightful and helpful comments. This research was sponsored by grants from NIH, Office of Naval Research, DARPA and National Science Foundation. Additional support was provided by Eastman Kodak Company, Daimler Chrysler, Honda Research Institute, NEC Fund, Siemens Corporate Research, Toyota, Sony and the McDermott chair (T.P.).

Author information

Authors and Affiliations

Authors

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Poggio, T., Bizzi, E. Generalization in vision and motor control. Nature 431, 768–774 (2004). https://doi.org/10.1038/nature03014

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nature03014

This article is cited by

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing