Lee Sharks
Proposes the Provenance Erasure Rate (PER) metric to measure the proportion of claims in AI-composed outputs that lack explicit attribution to original sources.
When AI retrieval systems compose answers from multiple sources, attribution information is lost, making it difficult for users to assess information reliability. Existing metrics (ROUGE, BERTScore) only measure content preservation, while citation precision/recall only check citation existence without evaluating the connection between sources and claims.
PER performs a binary judgment on whether each claim in the output is explicitly attributed to its original source, calculating the proportion of claims with erased provenance. Claims can be weighted, and PER is distinguished from citation precision/recall and AIS support metrics. A case study of a Google AI Overview demonstrates PER=1.0, and a validation agenda is proposed.
PER is a new metric that quantifies source transparency, contributing to accountability, labor accounting, and retrieval transparency in AI search systems. It is orthogonal and complementary to existing metrics. However, it is at the proposal stage and requires empirical validation.