Where traditional OCR still wins

Vision models read documents better than classical OCR on nearly every axis people talk about. They handle novel layouts, they understand what a field means, they cope with rotation and poor contrast, and they do not need a template per supplier.

There are three things they do not give you, and each one is occasionally the thing the system is built around.

The question is not which is more accurate. It is whether you need coordinates, determinism, or a cost per page measured in hundredths of a cent.

One: coordinates

A vision model tells you the total is €1,240.50. Classical OCR tells you the string 1.240,50 sits at pixels (412, 883) to (498, 901) on page one.

That rectangle is worth more than it looks:

A reviewer needs it. The whole argument for a four-second review is showing a cropped image of the uncertain field. Without a bounding box, the reviewer gets the whole page and the review takes sixty seconds instead.

Verification needs it. "The model says the total is X, and here is the pixel region it came from" is a checkable claim. "The model says X" is not.

Redaction needs it. Removing a national insurance number from a stored document requires knowing where it is, not what it says.

Vision models can be asked for approximate coordinates, and the results are usable for pointing a reviewer at roughly the right area. They are not exact, and they should not be relied on for anything that must be precise — redaction being the obvious example, where "approximately here" is not a safety property.

Two: determinism

Run OCR on the same image twice and get the same string. Run a vision model twice and you may get two different readings of a smudged digit.

That is usually fine, and it is occasionally disqualifying:

Reproducibility requirements. A regulated process that has to produce the same output from the same input on audit.

Consistency across a corpus. The same supplier name extracted three different ways across a thousand invoices produces three ledger entries. Normalisation helps; determinism at the source helps more.

Debugging. A non-reproducible error is a much harder error to fix.

Temperature zero reduces variation substantially and does not eliminate it. If byte-identical output on repeat runs is a requirement rather than a preference, that is a real constraint on the architecture — and worth surfacing during scoping rather than discovering during an audit.

Three: cost per page

This is the one that decides high-volume architectures.

A page image on a standard-tier model is around 1,560 tokens. At $1/MTok that is about $0.0016 per page in input alone, before instructions or output.

Classical OCR is roughly one to two orders of magnitude cheaper per page, and open-source engines running on your own hardware are effectively free at the margin.

At 10,000 pages a month nobody cares. At 10 million pages a month the difference is the entire budget, and it becomes worth asking which pages genuinely need a model.

The hybrid we actually build

Not a choice — a split, with each half doing what it is good at:

async function extract(page: PageImage): Promise<Extraction> {
  // 1. OCR first. Cheap, fast, and it produces the coordinate map
  //    that everything downstream needs.
  const ocr = await ocrEngine.recognise(page)   // words + boxes + per-word conf

  // 2. Decide whether the model is needed at all.
  if (isKnownTemplate(page, ocr) && templateFieldsComplete(ocr)) {
    // Known supplier, known layout, all fields located by rule.
    // No model call. This is most of the volume in a mature pipeline.
    return fromTemplate(ocr)
  }

  // 3. Model reads the page for meaning — which value is the total,
  //    what the columns mean, how to handle a layout never seen before.
  const understood = await vision.extract(page, {
    // Give it the OCR text as a transcript alongside the image. Better
    // than either alone: exact characters plus visual layout.
    transcript: ocr.text,
    schema: INVOICE_SCHEMA,
  })

  // 4. Re-attach coordinates by matching extracted values back to
  //    OCR tokens. This is what gives the reviewer a crop.
  return withBoxes(understood, ocr)
}

Step four is the part worth building carefully. Matching 1240.50 back to the OCR token 1.240,50 requires normalising both sides, and a fuzzy match on the normalised forms. When it fails, the field simply has no box and the reviewer sees the page — degraded, not broken.

Step two is where the money is. In a pipeline with a concentrated supplier distribution — and most are, with a handful of suppliers making up the majority of volume — template matching on the OCR output handles the bulk with no model call at all, and the model is reserved for the tail.

Where OCR alone is still correct

Some documents do not need understanding:

Fixed-format forms. A government form where field 4b is always in the same place. Template plus OCR, no model, exact.

Full-text indexing. Making a scan archive searchable. You need text, not structure.

Pre-filtering. OCR a page cheaply to decide whether it is worth sending to a model at all. A page that OCRs to forty characters is probably blank or a separator sheet.

Where the model alone is correct

And some do not need coordinates:

Low volume, novel layouts. A few thousand pages a month of documents that are all different. Template matching has nothing to match; coordinates are not worth the pipeline complexity.

Semantic questions. "Is this invoice for services or goods?" is not a field on the page. OCR cannot answer it at all.

Photographs. Angle, curvature, shadow, partial occlusion. Classical OCR degrades sharply here and vision models degrade gracefully, which is a large practical difference on user-uploaded documents.

The general rule that falls out: OCR for where things are, the model for what they mean. Most production pipelines eventually want both, and the ones that picked one early usually rebuild.

More in OCR or an LLM for invoice extraction?, what a scanned page actually costs, and how we build extraction pipelines.