Detecting hallucinations in large language models using semantic entropy

TL;DR

A general method to detect hallucinations (confabulations) in LLMs by measuring semantic uncertainty based on entropy.

Problem

LLMs often generate confident but factually incorrect outputs (hallucinations), causing serious issues in high-stakes fields like law and medicine. Existing methods to encourage truthfulness are only partially successful, and a general method to detect hallucinations for new, unseen questions is needed.

Approach

Proposes semantic entropy, which computes uncertainty at the level of meaning rather than specific word sequences, considering multiple expressions of the same idea. This detects arbitrary and incorrect generations (confabulations) that the model produces without confidence. The method works across datasets and tasks without prior task knowledge or task-specific data.

Results & Contribution

The method consistently detects hallucinations across multiple datasets and tasks, and robustly generalizes to new, unseen tasks. It helps users identify when to be cautious with LLM outputs, mitigating reliability issues and expanding the potential use of LLMs.