Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines

TL;DR

Deep learning models for predicting gene perturbation effects were evaluated against simple linear baselines, and none outperformed the baselines.

Problem

Accurately predicting the effects of genetic perturbations (e.g., gene knockout) on the transcriptome is crucial for understanding disease mechanisms and developing therapies. Recent deep learning foundation models have shown potential, but systematic evaluation of whether they outperform simple linear models was lacking.

Approach

The researchers compared five foundation models and two other deep learning models against simple linear baselines (e.g., mean effect, linear regression). Performance in predicting transcriptome changes after single and double perturbations was evaluated across multiple datasets.

Results & Contribution

No deep learning model consistently outperformed the linear baselines. This indicates that complex deep learning models are not always better and underscores the necessity of thorough benchmarking in method development. The study also highlights limitations of current approaches and suggests future research directions.