Currently watching resting state data collected on our scanner
(
)
Stephen Mazurchuk
Currently watching resting state data collected on our scanner
(
)
Central Question:
What are the neural substrates that support concept representation?
Why?
Why should something like word meaning be localizable in the cortex?
not so
Plato
Aristotle
Universals (forms)
* universals boded well with Christian theology
John Locke
David Hume
Essay Concerning Human Understanding
(17th Century)
"Tabula rasa"
"label these residuals of stimulation with which the cortex is populated "memory images" and would emphasize the differentiation of such from the sensory stimulation itself"
Wernicke. (1874). Der Aphasiche Symptomencomplex.
Carl Wernicke
"label these residuals of stimulation with which the cortex is populated "memory images" and would emphasize the differentiation of such from the sensory stimulation itself"
Wernicke. (1874). Der Aphasiche Symptomencomplex.
"The concept of the word "bell," for example, is formed by the associated memory images of visual, tactual and auditory perceptions. These memory images represent the essential characteristic features of the object, bell."
Carl Wernicke
Animal
Vehicle
Tool
Plant
Animal
Vehicle
Tool
Plant
What information can fMRI tell us about concept representation?
(for today)
--> Susceptibility correction, Nuisance regressors
--> project volumetric data to cortical surface
--> warp data to common space
Surface normalization
Pial Surface Segmentation
Given that we have:
How do we compare the model to the data??
We compare if the similarity between pairs of neural activation patterns correlates with the model-predicted pairwise similarity
Banana
Suppose we have two models (or representations of the data). How do we compare them?
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Voxel 1 | Voxel 2 | Voxel 3 | Voxel 4 | Voxel 5 | |
---|---|---|---|---|---|
Car | .23 | .58 | .49 | .78 | .86 |
Airplane | .98 | .28 | .34 | .18 | .52 |
Chicken | .62 | .82 | .91 | .36 | .17 |
Model 2
Model 1
How we compare models?
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Car | Airplane | Chicken | |
---|---|---|---|
Car | |||
Airplane | |||
Chicken |
How we compare models?
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Car | Airplane | Chicken | |
---|---|---|---|
Car | 1 | ||
Airplane | |||
Chicken |
How we compare models?
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Car | Airplane | Chicken | |
---|---|---|---|
Car | 1 | .62 | |
Airplane | |||
Chicken |
How we compare models?
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Car | Airplane | Chicken | |
---|---|---|---|
Car | 1 | .62 | .1 |
Airplane | |||
Chicken |
How we compare models?
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Car | Airplane | Chicken | |
---|---|---|---|
Car | 1 | .62 | .1 |
Airplane | 1 | .12 | |
Chicken |
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Car | Airplane | Chicken | |
---|---|---|---|
Car | 1 | .62 | .1 |
Airplane | 1 | .12 | |
Chicken | 1 |
Voxel 1 | Voxel 2 | Voxel 3 | Voxel 4 | Voxel 5 | |
---|---|---|---|---|---|
Car | .23 | .58 | .49 | .78 | .86 |
Airplane | .98 | .28 | .34 | .18 | .52 |
Chicken | .62 | .82 | .91 | .36 | .17 |
Voxel 1 | Voxel 2 | Voxel 3 | Voxel 4 | Voxel 5 | |
---|---|---|---|---|---|
Car | .23 | .58 | .49 | .78 | .86 |
Airplane | .98 | .28 | .34 | .18 | .52 |
Chicken | .62 | .82 | .91 | .36 | .17 |
Car | Airplane | Chicken | |
---|---|---|---|
Car | 1 | .42 | .06 |
Airplane | 1 | .31 | |
Chicken | 1 |
Called a DSM
Advice
Banana
Celebration
Yeo, T. B. et al. The organization of the human cerebral cortex estimated by intrinsic functional connectivity. J Neurophysiol 106, 1125–1165 (2011).
Depends if subjects correlate with each other
Yes!
Our model does better than the between subject correlation
Yes!
Our model does better than the between subject correlation
But
Yes!
Our model does better than the between subject correlation
But
What's the best it could be?
Q:
What vector would have the smallest distance
(most similarity)
to all of the vectors?
The average vector!
The best that a model could be is given by:
The average correlation of one subject's similarity matrix with the average similarity matrix of all other subjects
Intuition
The model is about 1/2 as good as the best model to the data
One approach would be doing RSA within many different ROI's
... The computationally exhaustive approach would be to generate a ROI at every vertext
*FWE corrected, threshold at p=.05
Regions where CREA is at least 20% of the noise ceiling
&
Noise ceiling is > .01
"These three cases then suggest that animate objects may be recognized and revisualized by the left occipital lobe while the same functions for inanimate objects proceed through functional activity of the right lobe"
* 2nd revised edition, 1940
Cortex, 1966
"Impairment of auditory comprehension for body part names, but not for object names"
Supramarginal Gyrus
posterior Superior Temporal Sulcus
*Clusterwise corrected at p=.001
MSTP:
Joseph Barbieri, PhD
Calvin Williams, MD, PhD
Nita Salzman, MD, PhD
Gil White, MD
Sid Rao, MD, PhD
Ann Moll
Kim Peplinksi
Language Lab:
Jeffrey Binder, MD
Leonardo Fernandino, PhD
Songhee Kim, PhD
Lisa Conant, PhD
Alex Helfand, PhD
Jia-Qing Tong
RCC
Matt Flister, PhD
Biophysics
MSTP:
Joseph Barbieri, PhD
Calvin Williams, MD, PhD
Nita Salzman, MD, PhD
Gil White, MD
Sid Rao, MD, PhD
Ann Moll
Kim Peplinksi
Language Lab:
Jeffrey Binder, MD
Leonardo Fernandino, PhD
Songhee Kim, PhD
Lisa Conant, PhD
Alex Helfand, PhD
Jia-Qing Tong
RCC
Matt Flister, PhD
Biophysics
Noppeney, U., Price, C. J., Penny, W. D. & Friston, K. J. Two Distinct Neural Mechanisms for Category-selective Responses. Cereb Cortex 16, 437–445 (2006).
Fernandino, L. et al. Concept Representation Reflects Multimodal Abstraction: A Framework for Embodied Semantics. Cereb Cortex 26, 2018–2034 (2016).
Caspers, S., Zilles, K., Laird, A. R. & Eickhoff, S. B. ALE meta-analysis of action observation and imitation in the human brain. Neuroimage 50, 1148–1167 (2010).