Stop parsing JSON out of prose

The first version of every extraction pipeline asks for JSON in the prompt and pulls it back out of the reply:

const reply = await model.complete(`Extract the invoice. Return JSON only.\n\n${doc}`)
const json = reply.match(/\{[\s\S]*\}/)?.[0]
const invoice = JSON.parse(json!)

It works immediately, which is why it survives into production. Then it fails on about one document in fifty, and those are not random documents — they are disproportionately the difficult ones, where the model hedged, explained itself, or produced something that is valid JSON and the wrong shape.

Prompt-and-parse asks the model to please produce a shape. Native structured output constrains what it is able to produce. Those are different guarantees.

The four ways it breaks

A preamble. "I'll extract the invoice details for you:" followed by the object. The greedy regex above handles this one, which is why people conclude the regex is sufficient.

A trailing note. "Note: the tax line was unclear." — genuinely useful information, and it breaks a parser expecting the reply to end at the brace. It also correlates with difficult documents, so you lose the cases you most wanted to see.

Markdown fencing, inconsistently. Sometimes ```json, sometimes bare, sometimes fenced with a language tag that varies. Every parser accumulates strip rules and each new rule is written after an incident.

Valid JSON, wrong shape. The worst one. {"total": "1,240.50"} instead of a number, {"lineItems": {}} instead of an array, a field omitted entirely because the model judged it not applicable. JSON.parse succeeds and the bad value flows downstream looking exactly like a good one — the failure described at length in a schema that fails loudly.

The first three are annoying and visible. The fourth is silent, and no amount of regex hardening touches it.

Use the native mechanism

Every major provider now constrains generation to a schema. On the Claude API that is a tool with an input schema, and forcing the tool choice:

const INVOICE_TOOL = {
  name: "record_invoice",
  description: "Record the extracted invoice.",
  input_schema: {
    type: "object",
    properties: {
      supplier: { type: "string" },
      total:    { type: "number" },          // a number, not a string
      currency: { type: "string", enum: ["EUR", "GBP", "USD"] },
      lineItems: {
        type: "array",
        minItems: 1,
        items: {
          type: "object",
          properties: {
            description: { type: "string" },
            amount:      { type: "number" },
          },
          required: ["description", "amount"],
        },
      },
    },
    required: ["supplier", "total", "currency", "lineItems"],
  },
} as const

const res = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 2048,
  tools: [INVOICE_TOOL],
  // The model cannot reply with prose. It must call this tool.
  tool_choice: { type: "tool", name: "record_invoice" },
  messages: [{ role: "user", content: [image, { type: "text", text: PROMPT }] }],
})

const block = res.content.find((c) => c.type === "tool_use")
const invoice = block.input          // already an object, already shaped

There is no parsing step, so there is nothing for a preamble to break. The first three failure modes are gone structurally rather than handled.

It also costs fewer output tokens, since the model is not generating explanatory prose you discard — a small, permanent saving on every call.

The schema is where the accuracy is

The mechanism guarantees shape. It does not guarantee correctness, and the schema is your main lever on the second:

Enums over free strings. currency: { enum: [...] } makes "euros" unrepresentable. Every value you can enumerate, enumerate.

Bounds. minItems: 1 on line items forces the model to either find one or fail, rather than returning [] and moving on.

Describe the fields. Descriptions inside the schema are read by the model and are the highest-leverage place to put disambiguating instructions — much more effective than the same sentence buried in a long system prompt:

issuedOn: {
  type: "string",
  description:
    "Invoice issue date as YYYY-MM-DD. If the document uses an ambiguous " +
    "numeric format like 03/04/2026 and nothing in the document resolves " +
    "which component is the month, use the ambiguous_dates field instead.",
}

Make refusal representable. If the schema has no way to say "unreadable", the model will produce a value. Give it the option, per teaching the extractor to refuse.

Still validate on receipt

Provider-side constraint handles JSON Schema. It does not handle your business rules, and those are where the real errors live:

const parsed = InvoiceSchema.safeParse(block.input)   // zod, valibot, etc.
if (!parsed.success) return review(raw, parsed.error)

// Cross-field arithmetic. No schema expresses this, and it catches
// the largest share of numeric misreads for zero additional cost.
const summed = parsed.data.lineItems.reduce((n, li) => n + li.amount, 0)
if (!closeEnough(summed + parsed.data.tax, parsed.data.total)) {
  return review(parsed.data, "line items do not sum to total")
}

Two layers, and they catch different things. The schema catches shape; the validator catches meaning. A hallucinated number almost never happens to satisfy the arithmetic, so the constraint does the detection for you.

When you cannot use it

Some models and some self-hosted setups have no constrained decoding. If you are stuck with prompt-and-parse, three things narrow the gap:

Prefill the assistant turn with the opening brace so a preamble is impossible.

Use a streaming JSON parser that tolerates a truncated tail, so a response cut off by max_tokens yields the fields that did arrive instead of throwing away the whole document.

Validate against a real schema library on every response, and route failures to review rather than retrying blindly — a retry on a document the model cannot handle costs money and produces the same failure.

That gets you most of the way. It is still a mitigation of a problem the native mechanism does not have.

More in a schema that fails loudly, when extraction accuracy collapses, and how we build extraction pipelines.