Thoughts, behaviour, and brain dynamics during navigation in the real world

Abstract

How does the human brain allow us to interact with and navigate through a constantly changing world? Whilst controlled experiments using functional brain imaging can give insightful snapshots of neuronal responses to relatively simplified stimuli, they cannot hope to mirror the challenges faced by the brain in the real world. However, trying to study the brain mechanisms supporting daily living represents a huge challenge. By combining functional neuroimaging, an accurate interactive virtual simulation of a bustling central London (UK), and a novel means of ‘reading’ participants' thoughts whilst they moved around the city, we ascertained the online neural correlates underpinning navigation in this real-world context. A complex choreography of neural dynamics was revealed comprising focal and distributed, transient and sustained brain activity. Our results provide new insights into the specific roles of individual brain areas, in particular the hippocampus, retrosplenial, and frontal cortices, as well as offering clues about how functional specialisations operate within dynamic brain systems.

Introduction

How the human brain processes the continuous and highly complex inputs from the external world to produce a seamless reality and an integrated sense of who and where we are remains a fundamental mystery in neuroscience. Whilst controlled experiments can give snapshots of neuronal responses to simplified stimuli, they cannot hope to mirror the challenges faced by the brain in the real world (Burgess et al., 2002, Hasson et al., 2004). Thus, several recent functional magnetic resonance imaging (fMRI) studies have examined brain activity elicited by viewing more naturalistic stimuli, such as Hollywood movies (Bartels and Zeki, 2004, Hasson et al., 2004, Zacks et al., 2001). Analyses using subsequent ratings of various attribute intensities (Bartels and Zeki, 2004), subsequent classification of event boundaries (Zacks et al., 2001), or events detected from the BOLD signal itself (Hasson et al., 2004) enabled the segregation of neural activity linked to specific events within the overall continuous stream of complex naturalistic stimulation. Whilst studies such as these are starting to provide clues about how functional specialisations might operate within dynamic brain systems, a fundamental limitation remains. Passively watching a movie is substantially different from most of our everyday activities that involve engaging interactively with the world around us. However, trying to ascertain the ‘online’ neural correlates of functioning in the real world represents a huge challenge.

One field where attempts have been made is in relation to arguably the most ubiquitous of our everyday activities, navigation. Previous functional neuroimaging studies using interactive virtual reality (VR) environments have identified a distributed network of brain regions engaged during active navigation including the hippocampus, parahippocampus, caudate nucleus, parietal and retrosplenial cortices, and regions within prefrontal cortex (PFC) (Aguirre et al., 1996, Ekstrom et al., 2003, Gron et al., 2000, Hartley et al., 2003, Maguire et al., 1998, Shelton and Gabrieli, 2002, Voermans et al., 2004). The functional role of some of these structures has been probed further by correlating their activity with measures such as navigational accuracy, typically averaged over epochs of 30–60 s (Hartley et al., 2003, Maguire et al., 1998). However, even in these experiments with more naturalistic interactive settings, unitary temporally gross measures fail to capture the multi-faceted and highly dynamic operation of the human navigation system.

We set out to extend the previous passive, and the temporally insensitive studies by integrating three factors (see below) in a novel way in the context of fMRI. We chose to focus on navigation, a prime example of the human brain operating in a real-world context. Our approach permitted a remarkably detailed characterisation of the unfolding navigation process that has eluded previous studies. Accompanying this, we were able to pinpoint the transient and highly specific engagement and disengagement of particular brain regions, such as the hippocampus, revealing the choreography of neural dynamics underlying this complex behaviour, whilst also providing new insights into the function of individual brain areas.

The first key factor required for this approach is a truly realistic context in which behaviour can occur. Although VR is now in common use in cognitive neuroscience, even the best environments have a somewhat deserted, stark, and simplistic quality (Ekstrom et al., 2003, Gron et al., 2000, Hartley et al., 2003, Iaria et al., 2003, Maguire et al., 1998, Shelton and Gabrieli, 2002, Voermans et al., 2004, Wolbers and Buchel, 2005). By contrast, we identified a commercially available video game that overcame these constraints. ‘The Getaway’ (© Sony Computer Entertainment Europe) is set in London (UK). Over 110 km (70 miles) of drivable roads have been accurately recreated from Ordinance Survey map data, covering 50 km² (20 square miles) of the city centre. The one-way systems, working traffic lights, the busy London traffic, and an abundance of Londoners going about their business are all included (see Fig. 1 and Movie 1, Appendix C). Conveniently, one can simply navigate freely around the city using the game console, with a normal ground-level first person perspective, in a car of one's choice.

The second vital ingredient in our experiment was the choice of subjects. Navigation ability is variable (Maguire et al., 2003), particularly in a large city. Thus, in order to ensure a consistent and accurate level of performance as a platform for our analyses, licensed London taxi drivers participated. In London (UK), taxi drivers engage in extensive training and have to pass examinations set by the Public Carriage Office in order to obtain a license to operate.

The third, and most crucial element of the study was the means by which we ‘read the thoughts’ of subjects as they navigated in London. We achieved this in the following way. During the fMRI scan, subjects responded to customers' requests (heard via headphones) by delivering them to their required destinations within virtual London, whilst driving a London taxi. They also heard customers make navigationally irrelevant statements, and request new destinations en route (see Materials and methods and Fig. 2). Subjects' navigation performance during scanning was recorded onto video tape. Immediately postscan, and without prior warning, subjects watched the video replay of their performance and were interviewed using a verbal report protocol (Ericsson and Simon, 1980). Simply put, this involved getting subjects to review their performance and report on what they had been thinking whilst they had been doing the task in the scanner. If carried out in a stringent manner and with a pre-established protocol (ours was established from piloting with taxi drivers), verbal reports can give remarkable access to cognition (Ericsson and Simon, 1980, Jack and Roepstorff, 2003). Our plan was to analyse the transcribed thoughts of the subjects and use this to model every second of the fMRI time series (see Appendix A). During scanning, subjects' eye movements were also recorded. The veridicality of the verbal reports was further tested using these independent eye-tracking data.