Suraj Biswas, Saurabh Gupta, Pritam Mukherjee
A method to address the problem that embedding similarity from biomedical language models is high even for unrelated domains, by applying contrastive learning and knowledge-graph-based hard negative mining, and accelerating inference with OpenVINO.
Large Behavioural Models (LBMs) that infer causal relationships based on individual behavioral and health data use embedding similarity as evidence of causality. However, existing biomedical language models (BioBERT, PubMedBERT, etc.) output high cosine similarity (0.76–0.92) for pairs from unrelated domains (e.g., cortisol level and stock-market volatility), leading to false causal edges in the causal graph. This propagates errors throughout the LBM's inference pipeline.
1) Contrastive pass: Contrastive learning on 72,034 pairs improves PubMedBERT's BIOSSES correlation from 0.633 to 0.828 and within-vs-across-domain separation from 1.05x to 1.63x. 2) BODHI: Mines hard negatives from edges absent in a biomedical knowledge graph for additional training, lifting separation to 2.30x and the discrimination gap to +0.392 (at a 4.5% BIOSSES cost). 3) OpenVINO optimization: Using FP16 on Intel Xeon 6737P (AMX), single-query latency drops from 1367 ms to 10 ms (133x), reaching 555 sentences/sec. FP16 is found to be more efficient than INT8 on this silicon.