Stephen Mazurchuk
10/28/21
Central Question:
What are the neural substrates that support concept representation?
Why?
Why should something like word meaning be localizable in the cortex?
Point to a corresponding picture among 10 cards belonging to the same category from either auditory or visual presentation of a name
Suzuki, K., Yamadori, A. & Fuji, T. Category-specific comprehension deficit restricted to body parts. Neurocase 3, 193–200 (1997)
"These three cases then suggest that animate objects may be recognized and revisualized by the left occipital lobe while the same functions for inanimate objects proceed through functional activity of the right lobe"
* 2nd revised edition, 1940
Key Point
Few fMRI studies have focused on determining the lexical-semantic neural correlates of body-part knowledge
(sometimes termed body-image)
Explain how focal Lesions to the cortex can result in deficits to a particular category of concepts
Goal:
Explain how focal Lesions to the cortex can result in deficits to a particular category of concepts
Goal:
Some areas of the cortex preferentially process certain sensory, motor, affective, and other experiential phenomena
Concepts are composed of sensory, motor, affective, and other experiential content
Conclusion:
1)
2)
"experiential features"
The spatial distribution and disruption of category knowledge is explained by "experiential" accounts of concept representation
Propositions:
(hypothesis)
both primary sensory cortices, but also adjacent association cortices
"Natural categories" of concepts arise from how concepts "cluster" on sensory, motor, and affective dimensions
3)
Explain how focal Lesions to the cortex can result in deficits to a particular category of concepts
Goal:
Some areas of the cortex preferentially process certain sensory, motor, affective, and other experiential phenomena
Concepts are composed of sensory, motor, affective, and other experiential content
Conclusion:
1)
2)
"experiential features"
The spatial distribution and disruption of category knowledge is explained by "experiential" accounts of concept representation
Propositions:
(hypothesis)
both primary sensory cortices, but also adjacent association cortices
"Natural categories" of concepts arise from how concepts "cluster" on sensory, motor, and affective dimensions
3)
Neural Representation of Concepts
Model Representation of Concepts
Neural Representation of Concepts
Model Representation of Concepts
Concept Representation as Experiential Attributes
Neural Representation of Concepts
Model Representation of Concepts
Surface normalization
Pial Surface Segmentation
Abdomen
Alligator
Ankle
Apathy
Apple
Armpit
Neural Representation of Concepts
Model Representation of Concepts
Suppose we have two models (or representations of the data). How do we compare them?
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Voxel 1 | Voxel 2 | Voxel 3 | Voxel 4 | Voxel 5 | |
---|---|---|---|---|---|
Car | .23 | .58 | .49 | .78 | .86 |
Airplane | .98 | .28 | .34 | .18 | .52 |
Chicken | .62 | .82 | .91 | .36 | .17 |
Model 2 (Neural Data)
Model 1 (experiential)
How we compare models?
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Car | Airplane | Chicken | |
---|---|---|---|
Car | |||
Airplane | |||
Chicken |
How we compare models?
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Car | Airplane | Chicken | |
---|---|---|---|
Car | 1 | ||
Airplane | |||
Chicken |
How we compare models?
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Car | Airplane | Chicken | |
---|---|---|---|
Car | 1 | .62 | |
Airplane | |||
Chicken |
How we compare models?
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Car | Airplane | Chicken | |
---|---|---|---|
Car | 1 | .62 | .1 |
Airplane | |||
Chicken |
How we compare models?
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Car | Airplane | Chicken | |
---|---|---|---|
Car | 1 | .62 | .1 |
Airplane | 1 | .12 | |
Chicken |
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Taste | Shape | Color | |
---|---|---|---|
Car | 0 | 4.2 | 4.9 |
Airplane | 0 | 5.1 | 4.3 |
Chicken | 5.5 | 2.2 | 2.2 |
Car | Airplane | Chicken | |
---|---|---|---|
Car | 1 | .62 | .1 |
Airplane | 1 | .12 | |
Chicken | 1 |
Voxel 1 | Voxel 2 | Voxel 3 | Voxel 4 | Voxel 5 | |
---|---|---|---|---|---|
Car | .23 | .58 | .49 | .78 | .86 |
Airplane | .98 | .28 | .34 | .18 | .52 |
Chicken | .62 | .82 | .91 | .36 | .17 |
Voxel 1 | Voxel 2 | Voxel 3 | Voxel 4 | Voxel 5 | |
---|---|---|---|---|---|
Car | .23 | .58 | .49 | .78 | .86 |
Airplane | .98 | .28 | .34 | .18 | .52 |
Chicken | .62 | .82 | .91 | .36 | .17 |
Car | Airplane | Chicken | |
---|---|---|---|
Car | 1 | .42 | .06 |
Airplane | 1 | .31 | |
Chicken | 1 |
Called an RDM
Advice
Banana
Celebration
(actual model has 65 features)
Yeo, T. B. et al. The organization of the human cerebral cortex estimated by intrinsic functional connectivity. J Neurophysiol 106, 1125–1165 (2011).
51,040 data points in each correlation
The best that a model could be is given by:
The average correlation of one subject's similarity matrix with the average similarity matrix of all other subjects
Intuition:
Lower Noise Ceiling
CREA Correlation
In general, our model explains about 1/2 of all explainable variance
* With a particular focus on the category of body-parts
Mean Ratings For Individual Categories
Clustering Analysis of Concepts based on experiential features
* With a particular focus on the category of body-parts
For a patch of cortex, train a linear Support Vector Machine (SVM) to classify a pattern as being a body part or not
Decoding
"If decoding succeeds on the test set, then the region must contain some information about the decoded variable"
Non-bodyparts
\(\psi_2\)
Body parts
\(\psi_1\)
toggle
FDR p=.005
* With a particular focus on the category of body-parts
** Important **
The encoding model predicts the distance between activation patterns (RDM) for body parts (test set) without having any body-part words in the training set
Encoding
(Predict activation patterns for body-parts using observed activation patterns for other words)
Decoding
Body parts
Animals
\(\psi_1\)
\(\psi_2\)
Model
Words
Features
Model
Words
Features
Encoding
For a given voxel
For word \(i\)
Car | Airplane | Chicken | |
---|---|---|---|
Car | 1 | .62 | .1 |
Airplane | 1 | .12 | |
Chicken | 1 |
Car | Airplane | Chicken | |
---|---|---|---|
Car | 1 | .42 | .06 |
Airplane | 1 | .31 | |
Chicken | 1 |
Predicted Correlational Structure
Observed Correlational Structure
predicted voxel response
feature \(1\) rating for word \(i\)
Shrinkage!
regular least squares
penalization
Need to choose the strength of the regularization parameter (\(\lambda \))
An important concept in shrinkage is the "effective" degrees of freedom associated with a set of parameters.
Plan: Do PCA to estimate intrinsic dimensionality. Then estimate shrinkage parameter
CREA encoding model
Body part ROI
Encoding Model ROI
FWEP p=.005
I bought chicken for dinner
bought chicken dinner
bought pork dinner
bought steak dinner
remove stop words
context words
target word
*No area survived multiple comparisons correction
W2V Encoding Performance
* With a particular focus on the category of body-parts
Is the classification accuracy the same for all categories in the body-part ROI?
Animal
Artifact
Body part
Plant\Food
* With a particular focus on the category of body-parts
Schwoebel, J. & Coslett, H. B. Evidence for Multiple, Distinct Representations of the Human Body. J Cognitive Neurosci 17, 543–553 (2005)
All three subjects with body image lesions had suffered temporal lesions; as shown in Figure 2, the lesion involved portions of Brodmann's area 37 as well as underlying white matter in 2 subjects
BA 37
Peak BP Area
In the 9 patients with body part anomia, oral naming of concrete entities was evaluated, and this revealed that 4 patients had disproportionately worse naming of body parts relative to other types of concrete entities
Kemmerer, D. & Tranel, D. Searching for the elusive neural substrates of body part terms: A neuropsychological study. Cognitive Neuropsych 25, 601–629 (2008)
lexical verses conceptual deficits
Of 12 tests, 4 required production of terms, and 7 required comprehension of terms
Goldstein, E. B. & Brockmole, J. Sensation and Perception. (Cengage Learning, n.d.)
* With a particular focus on the category of body-parts
Takeaway:
Decent evidence that the activation patterns within the brains "conceptual system" are somewhat explained by the sensory-motor (experiential) features being processed
MSTP:
Joseph Barbieri, PhD
Calvin Williams, MD, PhD
Nita Salzman, MD, PhD
Gil White, MD
Sid Rao, MD, PhD
Language Lab:
Jia-Qing Tong
Jeffrey Binder, MD
Leonardo Fernandino, PhD
Lisa Conant, PhD
Songhee Kim, PhD
Alex Helfand, PhD
Ann Moll
Kim Peplinksi
Joe Heffernan, MS
Jed Mathis
Samantha Drane
Belle Banke
W2V Shrinkage
Body-Parts | |
---|---|
instep | spine |
eyebrow | muscle |
pancreas | waist |
leg | cheek |
trachea | skeleton |
abdomen | liver |
ligament | cartilage |
clavicle | knuckle |
kidney | navel |
testicle | thumb |
forearm | eyelid |
nipple | elbow |
stomach | diaphragm |
finger | shoulder |
beard | tooth |
skull | wrist |
nostril | pelvis |
torso | intestines |
heel | armpit |
toenail | belly |
earlobe | bladder |
retina | ankle |
thigh | mustache |
fingernail | nose |
lip | forehead |
Called Autotopagnosia
Renzi, E. D. & Scotti, G. Autotopagnosia: Fiction or Reality?: Report of a Case. Arch Neurol-chicago 23, 221–227 (1970)
sometimes termed body-image
Tasks were: