OCR or an LLM for invoice extraction?
Traditional OCR is brittle on layout. LLMs invent numbers. They fail differently, and which failure you prefer decides the architecture.
Written for
“Our OCR breaks on new layouts and the LLM makes things up. Which problem do I want?”
The question is usually framed as accuracy: which one is more accurate on invoices? That framing does not survive contact with production, because the two approaches do not fail in comparable ways.
Traditional OCR fails loudly on layouts it has not seen. An LLM fails quietly by inventing a plausible number. Choose the failure you can detect.
How each actually fails
Template OCR — zone-based extraction, where you tell the system the total lives in a box at coordinates roughly here.
Fails when the layout changes. A supplier redesigns their invoice, the box moves, and the field comes back empty or captures the wrong number. The failure is usually obvious: empty fields, or a value that fails a type check.
LLM extraction — hand the document to a model with a schema and ask for the fields.
Handles layout variation remarkably well. Fails by producing a number that is
formatted correctly, in the right field, of the right magnitude, and wrong. It
read 1,240.50 as 1,240.05, or it took the subtotal when you wanted the total,
or the invoice had two totals and it picked confidently.
That second failure mode is much more expensive, because nothing downstream can tell it from a correct answer.
The comparison people actually want
| Template OCR | LLM extraction | |
|---|---|---|
| New layout, never seen | Breaks | Usually handles it |
| Handwriting | Poor | Better, still risky |
| Cost per page | Very low | Higher, controllable |
| Latency | Milliseconds | Seconds |
| Deterministic | Yes | No, without care |
| Failure visibility | Usually loud | Usually silent |
| Auditability | Coordinates | Needs deliberate design |
What we actually build, and why
Neither, exactly. The architecture that survives is layered:
-
A cheap deterministic pass first. Text-layer extraction and regex for things with rigid formats — invoice numbers, dates, VAT identifiers, IBANs. These have checksums and patterns. Never ask a model for something a regex can prove.
-
A model for structure and the messy fields. Line items, descriptions, anything positional or free text. This is where the model is genuinely better than anything else available.
-
Arithmetic as a check, not as extraction. Line items must sum to the subtotal. Subtotal plus tax must equal the total. This is the highest-value validation in the entire pipeline: it catches a large share of model misreads for free, because a hallucinated number rarely satisfies the arithmetic.
-
Confidence, and a threshold that gates. Below the line goes to a review queue, not into the ledger.
The arithmetic check is the thing most teams do not build, and it catches more errors than any model upgrade will.
Choosing, if you must choose one
Template OCR alone is defensible when you receive invoices from fewer than a dozen suppliers whose formats do not change, and volume is high enough that cost per page matters.
LLM extraction alone is defensible for a prototype, or when a human reviews every record anyway — in which case you are building an assistive tool, and the silent-failure objection largely goes away.
For anything that writes to a system of record unreviewed, layer them. The cost difference is small and the failure profile is completely different.
Related: build vs buy and our extraction work.