Wei Wu
An eight-week longitudinal study of silent failures in an LLM agent runtime proposes a five-class mechanism-based taxonomy, highlighting the danger of the 'fail-plausible' pattern where the LLM transforms errors into plausible narratives.
In long-running autonomous LLM agent systems, errors can be hidden from users as 'silent failures'. Existing tests and governance fail to detect or prevent these failures, especially when the LLM disguises errors as natural text.
Over eight weeks, 22 incidents were collected from a personal assistant agent runtime in production since March 2026, with full root-cause analysis. Failures were classified into five mechanism-based classes, and detection paths, prevention potential, and latency were analyzed.
70% of silent failures were detected by user observation; post-hoc audits had 0% ex-ante prevention but 87% regression blocking. Failure latency was determined by failure mechanism, not code complexity, with the longest-lived failures occurring at component boundaries. A five-class taxonomy and defense framework are presented, with all postmortems and artifacts released.