Adding provenance to AI outputs
Compliance asked why the system said that, and you cannot answer. What to record, when to record it, and what it costs to retrofit.
Written for
“Legal or an enterprise customer just asked 'why did it say that?' and we can't answer.”
Somebody has asked why the system produced a particular output. Possibly a customer, possibly compliance, possibly a regulator. You have the output and the input and nothing in between, and the honest answer is that you cannot reconstruct it.
Provenance is not logging. Logging tells you what happened. Provenance lets you prove why a specific record has the value it does, months later.
What to record
Per output record, not per request. The distinction matters: request logs get rotated away, and the record outlives them by years.
| Field | Why |
|---|---|
| Source identifier + content hash | Proves which version of the input produced this |
| Location within the source | Page, cell, character offset, timestamp, frame |
| Model name and exact version | "GPT-4" is not a version. Behaviour changes underneath you |
| Prompt version | Hash or semantic version of the template used |
| Retrieved context identifiers | For RAG: which passages, and what they scored |
| Confidence | Per field, not per document |
| Reviewer and timestamp | If a human touched it, who and when |
| Pipeline version | The code that produced it |
Store this alongside the value, in the same row or an adjacent table keyed to it. Not in a log aggregator with a 30-day retention policy.
The location field is the one that matters
Everything else is metadata. The location is what turns "the system says the total is £4,210" into "the total is £4,210, from page 3, the cell at these coordinates, in the document uploaded on this date, and here is the crop."
That distinction is the whole thing. It converts an assertion into evidence, and it is what a person actually needs when they are deciding whether to trust you.
For video and audio, the equivalent is a timestamp, and it should be precise enough to jump to. In one system we built, the design constraint was that every extracted claim had to trace to a named person, a named date, and a timestamp inside a specific video — and that constraint shaped every other decision in the pipeline.
Retrofitting, honestly
Adding provenance to a system that has none is not a small job, and anyone telling you otherwise has not done it. The work is roughly:
- Extend the schema. Adding columns is easy. Deciding what they mean is not.
- Thread the context through. Every stage now has to carry origin metadata forward. This is the actual work, and it touches everything.
- Backfill or accept a boundary. Existing records have no provenance. Either reprocess — expensive, and the model version has changed so results may differ — or accept a cutover date and be explicit that records before it cannot be explained. The second is usually correct and needs to be a stated decision, not a discovered one.
- Surface it. Provenance nobody can see is provenance nobody trusts. It needs to be in the UI, next to the value.
Expect two to four weeks for a moderate pipeline. It is far cheaper to design in from the start, which is why we build it in by default rather than as a compliance feature bolted on later.
What it buys you beyond compliance
- Debugging becomes possible. "Why is this wrong?" stops being archaeology.
- Enterprise sales stop stalling. This question appears in procurement, and "we log everything" is not an answer that passes.
- Trust degrades gracefully. When the system is wrong, a user who can see where the number came from corrects it. A user who cannot stops trusting the whole system.
That last point is the commercial argument. A system that is 94% accurate and explainable is more usable than one that is 97% accurate and opaque, because the first one tells you which 6% to check.
Related: our approach to extraction · the reliability audit