If you cannot replay it, you cannot debug it

An agent run goes wrong. Not crashes — goes wrong. It contacted the wrong account, or skipped a step, or produced something that was confidently incorrect. Somebody wants to know why.

The logs say the run started, four tools were called, and it completed successfully. Every line is true and none of it answers the question, because the question is about a decision and logs record events.

An agent failure is rarely a stack trace. It is a sequence of individually reasonable steps that added up to something wrong, and reconstructing it needs the inputs, not the outcomes.

Three questions it has to answer

The design test for a transcript is whether it can answer these without anyone guessing:

1. What did the model see? The exact messages, tools and context at each step. Not a summary — the bytes.
2. What did it decide, and what came back? Tool calls with resolved arguments, and the results returned.
3. Could this be reproduced? Model version, prompt version, temperature, seed if you have one, and the retrieved context if any.

Question three is the one usually missing, and it is the one that makes the difference between investigating and speculating. A run you cannot re-execute is a run you can only theorise about.

The shape

create table run_transcript (
  run_id      text not null,
  seq         int  not null,
  at          timestamptz not null default now(),

  kind        text not null,     -- message | tool_call | tool_result
                                 -- | decision | error
  -- The exact payload. Large, compressible, and the whole point.
  payload     jsonb not null,

  -- Reproduction context, recorded per step because it can change mid-run.
  model       text not null,
  prompt_version text not null,
  temperature numeric(3,2),

  -- Cost, so a transcript doubles as the usage ledger.
  input_tokens  int,
  output_tokens int,
  cost_usd      numeric(12,6),

  primary key (run_id, seq)
);

Recording prompt_version and model per step rather than per run matters more than it looks. A run that spans a deploy can use two prompt versions, and runs that span deploys are exactly the ones that behave strangely.

Record the resolved arguments

The single most common gap: logging that a tool was called, without logging what it was called with.

// Useless six weeks later.
log.info("tool call", { tool: "send_email" })

// Answers the question.
await transcript.append(runId, {
  kind: "tool_call",
  payload: {
    tool: "send_email",
    args: resolvedArgs,          // AFTER template expansion and defaults
    argsHash: hash(resolvedArgs),
    grantChecked: "send_email:domain=ourcompany.com",
    approvedBy: approval?.actor ?? null,
  },
})

Resolved, not templated. The bug is usually in what a value expanded to, and a transcript showing {to: "{{contact.email}}"} records the intention rather than the action.

Redact at write time, not at read time

Transcripts contain whatever the agent touched — customer records, message bodies, credentials passed as tool arguments. That is the point, and it is also a liability.

Redact on the way in:

const REDACT = [
  { path: "args.apiKey",     mode: "drop" },
  { path: "args.to",         mode: "hash" },   // correlatable, not readable
  { path: "payload.content", mode: "truncate", keep: 500 },
]

hash rather than drop for identifiers is the useful middle. You can still answer "did this run touch the same recipient twice?" without storing the address in a table that has a broader access list than the source system does.

Set a retention period and enforce it. Ninety days covers essentially every investigation; indefinite retention of full agent transcripts is a breach waiting for a reason.

Cost and volume

Transcripts are large — a 20-step run with retrieved context can be several megabytes. Three things keep it manageable:

Store payloads compressed. JSONB with TOAST compression in Postgres handles this reasonably; blob storage with a pointer row is better past a certain volume.

Truncate what you can reconstruct. If retrieved chunks are addressable by id, store the ids and not the text. A 40KB context becomes a list of ten strings, and the text is recoverable from the corpus.

Tier by outcome. Full transcripts for runs that failed, were halted, hit a budget ceiling, or had an action rejected at review. Summary-only for clean runs. In practice most runs are clean, so this is a large saving on the axis that does not matter.

Keep that policy honest, though: sampling clean runs at some low rate is worth the storage, because "it worked but produced something odd" is a real category and it does not trip any of the automatic triggers.

The property that makes it worth building

A transcript that captures inputs and versions lets you replay a run against a changed prompt without re-executing any side effects:

// Replay: feed the recorded inputs to a new prompt version and diff
// the decisions. No emails sent, no records written.
const original = await transcript.load(runId)
const replayed = await dryRun(original.inputs, { promptVersion: "v7" })
const diff = compareDecisions(original.decisions, replayed.decisions)

That turns "we think the new prompt fixes it" into a checkable claim across every historical failure you have. It is the agent equivalent of a regression suite, and the runs it tests are real ones rather than cases somebody imagined.

Which is the argument for the whole thing. The transcript is not an audit obligation you satisfy grudgingly — it is the corpus your evals run against, and you cannot build it retroactively.

More in the approval gate is a state machine, retry is not resume, and adding provenance to AI outputs.