EEG Project Progress/Status/Next
- Discussion on Literature
- Better fine-tuning strategy
- Current progress (On Notion)
- Plan for NeurIPS Datasets and benchmark?
How do achieve a better finetuning strategy?
Currently, given (B, T, C, D) we compress the signal via a sequence of layers
- Wav2vec2 + EEGNET
- Wav2vec2 + Shallownet
- Wav2vec2 + MLP + Shallownet
- Wav2vec2 + Attention + MLP
- Wav2vec2 + Attention + xVector + MLP
Are these the best approaches?
So far with end-2-end training Wav2vec2 + Shallownet achieves ~65 on single subject/single session but drops to ~45 with proper evaluation
How do achieve a better finetuning strategy?
Currently, given (B, T, C, D) we compress the signal via a sequence of layers
- How about a BERT-like approach (with a CLS token)?
- How about changing the objective function and removing convolutions.. i.e. only using a single transformer on raw input?
- With L2 loss objective?
- We need a simple fine-tuning pipeline without adding extra complications
A bit of literature
NEURO-GPT
- Uses Temple University Hospital (TUH) EEG corpus
- selected 22 channels only
- Processed 20,000 (19,000 train/ 1000 val) EEG recordings (~5656 hrs)
- Split EEG signal into N chunks, each chunk has dim C * T.
- Each chunk is treated independent sample
LaBraM
- Collected and pre-trained a 6M - 369M Transformer model on more than 2,500 hours of diverse EEG data
- Handle EEG signals with various channels and time lengths
- They evaluate four downstream tasks in BCI, they surpass all SOTA methods by a large margin
- Train a neural tokenizer to discretize EEG signals into discrete neural tokens
- During pre-training, part of EEG patches are masked while the objective is to predict masked tokens from visible patches.
- Eval on 2 datasets, not a big margin as they claimed... relatively compared to architecture complexity.
LARGE TRANSFORMERS ARE BETTER EEG LEARNERS
- Introduce AdaCT, Adapters to convert time series data into spatio-temporal 2D pseudo-images or text.
- Then use pretrained models for text/image
- No involvement with EEG pretraining directly.
- Looks like Fake work (I didn't even get how the convert it to text)
