You probably do not need RAG

RAG became the default answer to "the model needs to know about our data", and for a large corpus of unstructured text it is the right one.

It also gets applied to three shapes of problem where it is strictly worse than the alternative — more infrastructure, more failure modes, worse answers. Two of those alternatives are a database query.

Retrieval is a solution to "the corpus does not fit". If your corpus fits, you have bought an approximation you did not need.

One: the corpus fits in the context window

This is the big one, and the arithmetic has moved under everybody.

Claude 4.6 and later ship a 1M-token context window at standard rates. A million tokens is roughly 750,000 words — about eight full-length books, or a company handbook, policy set and FAQ several times over.

If your entire corpus is 60,000 tokens, retrieval is solving a problem you do not have. Put all of it in the prompt, cache it, and the model sees everything with no chunking, no embedding model, no vector store, no recall ceiling, and no possibility of the answer being absent from the context.

The objection is cost, and it is worth actually computing. 60,000 tokens on Sonnet 5 at $2/MTok is $0.12 per query — genuinely expensive at volume. With prompt caching the same tokens read at $0.20/MTok:

60,000 × $0.20 / 1,000,000 = $0.012 per query

Just over a cent, against a RAG pipeline that costs a vector database, an embedding model, an ingestion job, a chunking strategy, and an ongoing eval burden — plus the retrieval failures that produce wrong answers.

The rough crossover, on stable corpora with reasonable traffic:

Note the second row's condition. A cache that never gets hit turns full-context into a 25% surcharge rather than a saving, so this depends on request frequency as much as on corpus size.

Two: the answer is in a database

A question with a precise answer stored in a structured field does not need semantic search over prose about that field.

"What is customer 4471's current plan?" is select plan from customers where id = 4471. Embedding a paragraph describing the customer, retrieving it by cosine similarity, and asking a model to read the plan out of it is an expensive, lossy, non-deterministic way to run that query — and it will occasionally be wrong, which the query never is.

The tell is that the answer has a canonical location. If you can name the table and column, you do not want retrieval:

// Not RAG. The model chooses and parameterises a query;
// the database produces the answer.
const tools = [
  {
    name: "get_customer_plan",
    input_schema: { type: "object", properties: { customerId: { type: "string" } } },
  },
]

Tool use over your own API keeps the model doing what it is good at — turning a question into a structured request — and keeps the facts coming from the system of record.

Plenty of real products need both: structured lookups for entity facts, retrieval for the prose. The mistake is forcing one shape onto both.

Three: the question is about the whole corpus

Retrieval returns the top k chunks. Some questions cannot be answered from any k:

• "How many of our contracts have auto-renewal clauses?"
• "What themes come up most in support tickets this quarter?"
• "Which suppliers raised prices this year?"

These are aggregations. The answer depends on every document, not on the most similar few. RAG will confidently produce a number derived from ten chunks, and that number will be wrong in a way nobody can detect from the output — the worst available failure mode.

The right architecture is to extract structure first and aggregate over it. Run extraction across the corpus once, write typed records, then answer with SQL:

-- Correct because it saw every contract, not the ten most similar ones.
select count(*) from contracts where has_auto_renewal;

That is an extraction pipeline rather than a retrieval system, and the distinction is worth naming early because the two have almost nothing in common architecturally.

What RAG is genuinely for

Not an argument against retrieval — an argument for using it where it fits, which is:

• A corpus too large for context, or growing.
• Questions answered by a passage, not by a field or an aggregate.
• Content that changes often enough that re-caching everything is wasteful.
• A need to cite the specific source of each claim.

That is a real and common shape: documentation, policies, knowledge bases, research archives, ticket histories. If your problem looks like that, build the retrieval system and evaluate it properly.

If it does not, the honest answer is that the fastest, cheapest and most accurate version of your feature has fewer moving parts than the architecture diagram everyone expects to see.

More in RAG or fine-tuning?, why your RAG returns wrong answers, and retrieval systems that are actually evaluated.