Trade-offs in Medical LLM Adaptation: An Empirical Study in French QA

TL;DR

An empirical study comparing CPT, SFT, and their combination for medical domain adaptation of LLMs in French QA.

Problem

The optimal strategy for adapting LLMs to specialized domains and languages is unclear. This study disentangles the effects of CPT and SFT to provide practical guidance.

Approach

Compare CPT, SFT, and CPT+SFT across three model families, sizes, and initializations. Evaluate on MCQA and OEQA using automatic metrics and LLM-as-a-Judge. Include cross-lingual experiments.

Results & Contribution

For MCQA, SFT alone is cost-effective with minimal gain from adding CPT. For OEQA, CPT improves overlap metrics while SFT may degrade generation. Provides guidelines for selecting adaptation strategies under computational constraints.