Retrieval without a k is a bill you cannot write down

PageIndex surfaced in X's trending list this morning as a vectorless RAG alternative, and the pitch is the one that has been circulating for a year: replace the vector index with a tree, and let a model reason its way down instead of measuring cosine distance.

What made me stop and read properly is that this project publishes its own numbers. A per-page indexing cost, an index-time benchmark, a measurement of what indexing costs against pasting the raw PDF into context, and an explicit split between the cheap model that builds the tree and the good model that walks it. That is more than most retrieval launches offer, and it is enough to price the thing properly.

The change that matters for a product already in production is not the cost. It is that retrieval stops having a k.

Vector RAG has one number that decides your cost, your latency and your eval metric at the same time. Take it away and all three stop being writable down — two of them improve, one of them you lose control of.

The knob

If you run vector RAG, your pipeline has a line in it like top_k = 10. That one integer is doing three jobs at once. It sets how many chunks the model reads, which is your cost per query. It sets how much text has to be ranked, which is most of your latency budget. And it appears in the name of your metric, recall@k, which is how you find out whether the system works.

It is a wonderful number. It is free to change — a deploy of one integer, no re-embedding, no re-indexing — and you can change it per request if you want to. It is the dial you turn when someone tells you the answer was in the document and the system missed it.

Tree search deletes it. In PageIndex's own framing, the tree replaces the vector index and retrieval becomes an agent walking that tree, classifying nodes as relevant or irrelevant. Nothing decides in advance how many nodes get opened. That is the design, not a bug in it. But it means the number your cost model is built on is now a property of the model's stopping behaviour on a given question.

Two of the three jobs get better. The third one is now a random variable.

The index price is the scary number, and it is not the cost

PageIndex's PyPI page puts local indexing at "about $0.001 per page", with a "$0.0011-per-page reference line" on the chart, and says a 1,000-page textbook "costs a little over a dollar and a few minutes."

Read that next to what embeddings cost and it looks like a disaster. It is 100× the embedding bill.

index, 100,000 pages     100,000 × $0.001      = $100.00   (their figure)
embed, same corpus       100,000 × 500 tok      = 50 MTok
                        50 MTok × $0.02/MTok    =   $1.00   (text-embedding-3-small)
                                                        ratio = 100×

One hundred times more, and the absolute number is still $100 — once. At their own stated throughput (1,098 pages in 4.5 minutes, so 244 pages a minute), a 100,000-page corpus indexes in about 6.8 hours:

100,000 pages ÷ 244 pages/min = 409.8 min = 6.8 hours

Compare that to the thing I have written about repeatedly: embedding a million documents is forty dollars, and the reason anyone cares is the rebuilds, not the first pass. Here the rebuild is the interesting part, and it is the next section's problem. As a one-time line item, $100 and an overnight job for a hundred thousand pages is not the reason to say no.

What a query costs when nothing caps the steps

This is the number the page does not give you, and it is the one your invoice sees. I have to build it from a stated shape, so here is the shape.

A 300-page report produces a tree of roughly 250 nodes. At about 40 tokens a node — a heading and a short summary, which is what a table of contents optimised for an LLM is — the full tree is 10,000 tokens. Assume each navigation step appends about 300 tokens of the agent's own decisions, and emits about 150 tokens of output. Assume the tree itself is a stable prefix and therefore a cache read, while the accumulated decisions are not.

At Claude Sonnet 5 rates — $2/MTok input, $0.20/MTok cache read, $10/MTok output — the input for step i is 10,000 cached plus 300·(i−1) at base:

The accumulated column is 300 × n(n−1)/2, which is the same quadratic I worked through in context window abuse — with one difference that matters. There, the quadratic term was conversation history you could cache most of. Here it is the agent's own reasoning, which is new on every step and cannot be cached at all. At thirty steps it is $0.2610 of the $0.3660 query: 71% of the bill is the model thinking about where to look.

Now compare it to the incumbent. A conventional pipeline at k=10 with 900-token chunks reads 9,000 tokens of retrieved text plus a question and a system prompt, and writes 400 tokens:

input   9,500 × $2/MTok   = $0.0190
output    400 × $10/MTok  = $0.0040
                    total = $0.0230 per query — and it is the same every time

Five steps is break-even. Everything past that is a multiplier nobody chose: 2.7× at ten, 5.0× at fifteen, 15.9× at thirty. The spread between the best and worst row at 100,000 queries a month is $34,250, and the only variable is how long the agent keeps looking.

That is not an argument against the architecture. The tail is where the multi-hop questions live — the ones where your vector index returns the wrong section confidently and a reasoning walk follows the actual cross-reference. Paying more on exactly the queries the cheap path got wrong is the right way round.

It is an argument for making the step count a measured, budgeted quantity instead of an emergent one. A step budget is the k you get to keep, and a cost ceiling is what stops a single pathological question from spending a month of a user's allowance.

You moved the recall problem from query time to index time

Here is the operational change I would actually plan around, and it comes from the vendor's own guidance rather than from anything I inferred. For the index model: "a basic model is sufficient". For the chat model: "use the best model you can afford."

Both of those are sensible in isolation and together they do something specific. The tree is built by a cheap model. The tree decides which subtrees the good model will ever consider. A section whose node summary omits the topic the user asked about is a section the agent has no reason to open — and no amount of model quality at query time recovers it, because that text never enters the context.

In vector RAG, the equivalent failure is survivable. A badly embedded chunk ranks low, you raise k from 10 to 30, the chunk comes back, and you have fixed a production incident with a config change. There is no equivalent move here. The correct answer is in a branch nobody opened, and the fix is to rebuild the tree.

And the rebuild is not free in the way the other one was:

raise k from 10 to 30      one integer, no re-index, no index-time cost
rebuild 100,000-page tree  $100.00 + ~6.8 h at their published throughput

Your recall knob went from free-and-per-request to paid-and-per-corpus. That is the real trade, and it is worth saying plainly: you are not just changing retrievers, you are moving a quality decision from query time to build time. The tree is a build artefact now. Version it, hash it, and treat a change to the index model the way you would treat a change to your chunking strategy — as a migration with a before and after, not a deploy.

The comparison they published is not the one your migration needs

The published cost comparison is against native PDF input: "2.1x, 3.4x, 7.8x, and 16.6x more at 52, 85, 198, and 420 pages", with 805 pages exceeding the context window entirely. Those are real numbers and the direction is right — PDF-token pricing is punishing at length.

But that answers should I stop pasting the PDF in?, which is a question you settled the day you built your vector index. If you already run retrieval, your incumbent is not full-context input, it is the pipeline in the table above, and the multiple that matters to you is the one in the last column of my table — not the one in theirs.

The accuracy figure has the same shape of gap. Their PyPI listing reports "98.7% accuracy" on FinanceBench against "vector RAG 50%", and VentureBeat covered the same result in January 2026, describing the system as Mafin 2.5 built on PageIndex. Read that carefully: it is a headline comparing a finished system against a baseline described in five words, with no harness, no chunk size, no k, and no model listed for the vector side. That is a press number, and I do not think less of the project for it — 98.7% on FinanceBench is a real result.

The artefact I would actually trust is the one they put in the repo: an open-source benchmark of 62 lookup questions across 34 PDFs and 1,945 pages, where the answers are facts in running text so there is no LLM judge to disagree with. That design removes the noisiest part of most retrieval evals. The limitation is that it measures answer correctness, so when a score drops it cannot tell you whether the tree failed to route or the model failed to read what it was given — the same decomposition problem I ran into measuring recall against answer quality.

The boundary I would check first

Local mode is text-based PDFs only, with page-level citations and no OCR or image understanding; scanned and image-rich documents, block-level citations, metadata and the MCP server are cloud-only. That is a reasonable product split and it is also exactly the line where the text layer lies. If a meaningful fraction of your corpus is scans, the self-hosted path silently degrades on precisely the documents that were already hardest, and you will find out from a user rather than from a metric.

What I would do before routing production traffic

1. Instrument step count before you change anything else. Log steps, tokens and stop reason per query. You cannot budget a distribution you have not measured, and the ratio of p99 to median step count is the whole risk.
2. Set a step ceiling and treat it as your new k. Fifteen steps caps a query at $0.1155 in this shape and makes the number quotable.
3. Build the tree with the good model, once, and A/B the trees. It is the one cheap-looking decision with an expensive reversal, and unlike k you cannot fix it per request.
4. Measure it against your current pipeline, not against full-context input. Same corpus, same questions, same model, both retrievers, per-query cost and steps recorded.

This is the kind of call that shows up in retrieval work that has to be evaluated rather than vibe-checked, and the step budget and cost ceiling are the same review an AI reliability audit applies to an agent that is already running. More on the arithmetic in context window abuse, what embeddings actually cost at scale, and why recall@k is not the metric you think it is.

The indexing cost of "about $0.001 per page", the "$0.0011-per-page reference line", the 1,000-page textbook at "a little over a dollar and a few minutes", the 9-to-1,098-page range completing in "roughly 13 seconds to 4.5 minutes", the "2.1x, 3.4x, 7.8x, and 16.6x" multiples at 52, 85, 198 and 420 pages, the 805-page context-window limit, the local-versus-cloud capability split, the index and chat model guidance, the 62-question benchmark over 34 PDFs and 1,945 pages, and the "98.7% accuracy" against "vector RAG 50%" are quoted from the project's PyPI page (pageindex 0.2.20, released 28 September 2026) and its GitHub README, both read on 29 September 2026. The FinanceBench result and the Mafin 2.5 attribution are from VentureBeat's coverage, published 30 January 2026. The trending entry that surfaced this is dated 29 September 2026 and I could not open it — everything above comes from the project's own documentation, which is where the numbers are.

The per-query table is my arithmetic on published Claude Sonnet 5 rates checked at claude.com/pricing on 29 September 2026, and the 250-node tree, the 40-token node, the 300-token step, the 150-token output, the 500-token page and the 100,000-page corpus are a stated scenario, not measurements — PageIndex publishes no per-query cost or step count. Their defaults name different models; the shape is what transfers between them.