Spurious alignment between large language models and brains can emerge from non-robust methods and overlooked confounds

TL;DR

This paper systematically analyzes methodological pitfalls and confounding variables in studies of LLM-brain alignment, revealing that some influential findings arise from non-robust methods.

Problem

Research using LLMs to predict neural signals is growing, but lacks large-scale robustness assessments. The impact of data splitting, activation extraction methods, and confounds like positional signals and word rate is often overlooked.

Approach

The authors analyze a wide range of models and methods across three neural datasets. They evaluate the effects of shuffled train-test splits, biases from different activation extraction techniques, and the competitive performance of confounding variables.

Results & Contribution

The study shows that shuffled splits lead to spurious results and that activation extraction can bias outcomes toward specific model classes. Confounds like positional signals and word rate perform competitively with trained LLMs and fully account for the predictivity of untrained models. This work provides a crucial re-evaluation of methodological robustness in LLM-brain alignment research.