Self-images in the video monitor coded by monkey intraparietal neurons
Introduction
When we use a virtual reality apparatus (such as teleoperator system) or when we play a video game, we feel our self-image projected onto the image of the hand or visual ques (i.e. a certain figure or an abstract point, such as cursors, which moves in accordance with the movement of our own hand) appearing in the video monitor, as a part of or an extension of our own hands (Harman et al., 1999). Essential elements of such an image of our own body should be comprised of neural representations about the dimension, posture and movement of the corresponding body parts in relation to the environmental space. Thus, its production requires integration of somatosensory (intrinsic) and visual (extrinsic) information of our own body in space. Human patients with a lesion in the parietal cortex describe introspective experiences of distortion, neglect, or extinction of their body images (Holmes, 1918, Triggs et al., 1994, Berlucchi and Aglioti, 1997), and the monkey intraparietal cortex possesses multimodal representations of the body and the peripersonal space which are used for planning purposeful movements (Andersen et al., 1997). This intraparietal area of the cerebral cortex receives projections from the somatosensory cortex conveying intrinsic information about the current posture (Seltzer and Pandya, 1980, Iwamura, 1998) and is simultaneously fed (via a dorsal stream of visual information processing) with extrinsic information about spatial locations and movements of the objects and our own body parts (Ungerleider and Mishkin, 1982, Colby et al., 1988). Thus, it seems reasonable to assume that the body-image is created and stored as response properties of a group of neurons in this brain area.
In our earlier report, we trained macaque monkeys to use rake-shaped tools to extend their reaching distance, and found in the intraparietal cortex a group of bimodal (somatosensory and visual) neurons which seemed to represent the image of the hand into which the tool was incorporated as its extension (Iriki et al., 1996). That is, around the somatosensory receptive field resided in the hand/forearm area was formed the visual receptive field defined as a territory in the space where a neuron responded to the moving visual stimuli. Tool-use induced an expansion of the visual receptive field only when monkeys intended to use tools to retrieve distant objects, but the modification was never induced when just holding it as an external object. Thus, this modification was not related to mere physical appearance of the tool simply held by the hand, but rather to psychological experience that the tool was assimilated to the hand. This mental process was recently confirmed to occur also in human patients’ brain (Berti and Frassinetti, 2000, Farne and Ladavas, 2000, Marvita et al., 2001). We also demonstrated (Obayashi et al., 2000) that visual responses of these intraparietal bimodal neurons also represent mental (or introspective) processes to create and sustain ‘subjective’ body-image in the brain, not only when it is visible but also when it is invisible and creating such an image only in the brain. Thus, among other brain areas which contain neurons representing similar bimodal properties for the body parts in the direct environments (Leinonen et al., 1979, Leinonen and Nyman, 1979, Colby and Duhamel, 1991, Colby et al., 1993, Graziano and Gross, 1993, Graziano and Gross, 1994, Fogassi et al., 1996, Graziano et al., 1997, Rizzolatti et al., 1997, Graziano, 1999), response property of bimodal neurons in intraparietal area is most likely to represent self-image in the monitor, which should necessitate complex mental processes for the self-images to be projected.
In the present experiment, we trained monkeys to recognize their image in a video monitor. Although it has been generally believed earlier that monkeys are not capable of doing so (Thompson and Boatright-Horowitz, 1994, Tomasello and Call, 1997), this experimental paradigm appears feasible because monkeys are known able to use the mirror to guide their movements (Itakura, 1987a, Itakura, 1987b). Using above paradigm, we attempted to prove above hypothesis by demonstrating that the visual receptive field (RF) of these bimodal neurons will be newly formed around the image of the hand projected onto the video screen so as to code the video-image of the hand as an extension of the self, and if represented images are subject to introspective manipulation independent of the state of the existing body parts in the actual space of reality. Preliminary results have been published in an abstract form (Iriki et al., 1997).
Section snippets
Subjects and training
Four Japanese monkeys (Macaca fuscata; male, 4–7 kg) were used. After the monkeys were familiarized with the laboratory environment, the first surgery was performed (under Nembutal anesthesia; 30 mg/kg, intra venous (i.v.)) to install five screws on the skull to fix their head to the monkey chair. Then, the monkeys were trained to sit quietly on a primate chair with their head fixed in an upright position. Food pellets were placed on a table (75×85 cm) at the monkeys’ waist height. Initially,
Utilization of self-image in the video monitor as a reference of hand movement
Monkeys sat in front of a table, and an opaque panel was installed under the eyes to mask their view (Fig. 1A) with a little window, which allowed direct viewing when it was open. Monkeys were trained to retrieve food (either with or without using a rake) relying entirely on a real-time video image shown on the monitor. For acquisition of this skill, monkey's hand-movement had to be displayed on the video monitor without any time delay. Thus, the coincidence of the movement of the real hand and
Recognizing the self in the video monitor
It has been generally believed that macaque monkeys do not recognize their ‘self image’ in a mirror or video monitor (Thompson and Boatright-Horowitz, 1994, Tomasello and Call, 1997). Only primates of evolutionary higher level than the chimpanzee were thought to recognize the self in the mirror (Gallup, 1970, Gallup, 1977, Gallup, 1982). Although, macaque monkeys have been reported to be able to use the mirror to guide their movements which cannot be observed through direct vision (Itakura,
Acknowledgements
We thank Professor Melvyn A. Goodale for valuable comments in the earlier version of the manuscript. Supported by JSPS (Japan Society for the Promotion of Science) Research for the Future program.
References (44)
- et al.
The body in the brain: neural bases of corporeal awareness
Trends Neurosci.
(1997) - et al.
Heterogeneity of extrastriate visual areas and multiple parietal areas in the macaque monkey
Neuropsychologia
(1991) - et al.
Spatial representations for action in parietal cortex
Cogn. Brain Res.
(1996) - et al.
Spatial maps for the control of movement
Curr. Opin. Neurobiol.
(1998) - et al.
Active manual control of object views facilitates visual recognition
Curr. Biol.
(1999) Hierarchical somatosensory processing
Curr. Opin. Neurobiol.
(1998)Facial expressions in nonhuman primates
- et al.
Parietal cortex: from sight to action
Curr. Opin. Neurobiol.
(1997) - et al.
Converging visual and somatic sensory cortical input to the intraparietal sulcus of the rhesus monkey
Brain Res.
(1980) - et al.
Multimodal representation of space in the posterior parietal cortex and its use in planning movements
Annu. Rev. Neurosci.
(1997)
When far becomes near: remapping of space by tool use
J. Cogn. Neurosci.
Studies in Cognitive Growth
Topographic organization of cortical afferents to extrastriate visual area PO in the macaque: a dual tracer study
J. Comp. Neurol.
Ventral intraparietal area of the macaque: anatomic location and visual response properties
J. Neurophysiol.
Dynamic size-change of hand peripersonal space following tool use
Neuroreport
Coding of peripersonal space in inferior premotor cortex (Area F4)
J. Neurophysiol.
Chimpanzees: self-recognition
Science
Self-recognition in primates: a comparative approach to the bidirectional properties of consciousness
Am. Psychologist
Self-awareness and the emergence of mind in primates
Am. J. Primatol.
Where is my arm? The relative role of vision and proprioception in the neuronal representation of limb position
Proc. Natl. Acad. Sci. USA
A bimodal map of space: somatosensory receptive fields in the macaque putamen with corresponding visual receptive fields
Exp. Brain Res.
The representation of extrapersonal space: a possible role for bimodal visual-tactile neurons
Cited by (160)
Body-tool integration: past, present, and future
2023, Cognitive Archaeology, Body Cognition, and the Evolution of Visuospatial PerceptionInterplay of tactile and motor information in constructing spatial self-perception
2022, Current BiologyCitation Excerpt :Our results would suggest that causal inference also links spatial dimensions of movement and touch and that percepts of motor action are modulated, as well as percepts of tactile consequences. This provides a novel addition to existing causal inference scenarios, which involved arbitrary spatial transformation from action to sensory input.37 Our results suggest that the voluntary motor command may promote integration between the multiple sensory and motor signals that are present during self-touch.
4.13 - The Evolution of Parietal Areas Involved in Hand Use in Primates
2020, The Senses: A Comprehensive Reference: Volume 1-7, Second Edition4.18 - Dorsal and Ventral Streams in the Sense of Touch
2020, The Senses: A Comprehensive Reference: Volume 1-7, Second EditionThe parietal lobe and tool use
2018, Handbook of Clinical NeurologyBodily self-consciousness and its disorders
2018, Handbook of Clinical Neurology