Every human cognitive function, such as visual object recognition, is realized in a complex spatio-temporal activity pattern in the brain. Current brain imaging techniques in isolation cannot resolve the brain's spatio-temporal dynamics, because they provide either high spatial or temporal resolution but not both. To overcome this limitation, we developed an integration approach that uses representational similarities to combine measurements of magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) to yield a spatially and temporally integrated characterization of neuronal activation. Applying this approach to 2 independent MEG-fMRI data sets, we observed that neural activity first emerged in the occipital pole at 50-80 ms, before spreading rapidly and progressively in the anterior direction along the ventral and dorsal visual streams. Further region-of-interest analyses established that dorsal and ventral regions showed MEG-fMRI correspondence in representations later than early visual cortex.
Together, these results provide a novel and comprehensive, spatio-temporally resolved view of the rapid neural dynamics during the first few hundred milliseconds of object vision. They further demonstrate the feasibility of spatially unbiased representational similarity-based fusion of MEG and fMRI, promising new insights into how the brain computes complex cognitive functions.
Spatio-temporal dynamics of information flow in ventral visual cortex
We recorded MEG and fMRI data while participants viewed a set of 92 different objects (image set previously used by Kiani et al., 2007 & Kriegeskorte et al., 2008).
We applied the MEG-fMRI fusion technique, and display the results in volume. You see three views on the brain (from left to right:) from the side, from the top and from the front). Note that the fMRI volume acquired was limited to a slab covering occipital and temporal cortex. This revealed the spatio-temporal dynamics of information flow in ventral visual cortex.
Spatio-temporal dynamics of information related to specific categories
Finally, we determined the flow of information related to specific categories. For this, we masked the MEG-fMRI fusion results with spatial masks indicating the locus where category-related activity was present. We conducted this analysis for several categorical distinctions present in the 92 image data set: (1) animacy, (2) faces vs. bodies (3) human vs. animal faces, (4) human vs. animal bodies, (5) naturalness. The analysis revealed the spatio-temporal information flow for the first four categorical distinctions.
This work was funded by National Eye Institute grant EY020484 and a Google Research Faculty Award (to A.O.), the McGovern Institute Neurotechnology Program (to D.P. and A.O.), a Feodor Lynen Scholarship of the Humboldt Foundation (to R.M.C) and an Emmy Noether grant of the DFG (CI 241/1-1), and was conducted at the Athinoula A. Martinos Imaging Center at the McGovern Institute for Brain Research, Massachusetts Institute of Technology. We thank Chen Yi and Carsten Allefeld for helpful comments on methodological issues, and Seyed-Mahdi Khaligh-Razavi and Santani Teng for helpful comments on the manuscript.