The RAG bug that shows tenant A tenant B's data

The code looks correct. It even reads as defensive:

// Broken. Read it twice — the bug is in the order.
const hits = await index.query({ vector, topK: 10 })
const mine = hits.filter((h) => h.metadata.tenantId === user.tenantId)
return mine

Search first, filter second. Every returned chunk belongs to the right tenant, so every test passes and no wrong data is ever shown.

Then a customer asks a question whose answer sits in a competitor's document, and gets told the system has no information — while a different question returns nothing at all, because all ten nearest chunks belonged to someone else and the filter removed every one.

Post-filtering is not a security bug on the day you ship it. It is a correctness bug that becomes a security bug the moment somebody removes the filter to fix the empty results.

Why post-filtering breaks

topK: 10 means "the ten nearest vectors in the whole index". Not the ten nearest belonging to this tenant — the index does not know about tenants.

With 200 tenants sharing an index, a given tenant owns roughly half a per cent of it. The ten nearest chunks to any query are drawn from everybody's data, so on average you keep zero or one. Recall collapses, and it collapses silently, because what you return is correct — there is just almost nothing in it.

Then the debugging goes wrong in a predictable way. Empty results get reported as "search is broken". Someone raises topK to 100, which helps a little and is slow. Someone else notices the filter is discarding almost everything and, under pressure, tries removing it to see whether search works at all.

That is the moment the correctness bug becomes a disclosure. It is one line, it is written while debugging, and it does not look dangerous.

Filter inside the query

Every serious vector store supports a filter applied during the search, so the topK is computed over the eligible set:

// Correct. The tenant constraint is part of the search, so topK: 10
// means the ten nearest chunks THIS TENANT CAN SEE.
const hits = await index.query({
  vector,
  topK: 10,
  filter: { tenantId: { $eq: user.tenantId } },
})

In pgvector, it is a where clause on the same statement:

select id, content, embedding <=> $1 as distance
from chunks
where tenant_id = $2                -- applied during, not after
order by embedding <=> $1
limit 10;

One caveat worth knowing: with an ANN index, a highly selective filter can force the planner to scan more than you expect, and some engines degrade to sequential scan. Partial indexes per large tenant, or partitioning by tenant, fix that. It is a performance conversation — have it, but never by moving the filter back out of the query.

Make it impossible to forget

The real fix is structural. A filter that any call site can omit will eventually be omitted by a call site written in a hurry.

Put the tenant in the connection, not the argument. With Postgres row-level security, the database enforces it and application code cannot opt out:

alter table chunks enable row level security;

create policy tenant_isolation on chunks
  using (tenant_id = current_setting('app.tenant_id')::uuid);

// Set once, per request, at the connection boundary. Every query on this
// connection is scoped whether or not the author remembered.
await db.query(`set local app.tenant_id = $1`, [user.tenantId])

Now a developer who forgets the filter gets zero rows rather than someone else's. The failure mode is inverted, which is the whole objective.

Or give each tenant its own namespace. Most vector databases support namespaces or per-tenant indexes. Cross-tenant retrieval stops being a filter question and becomes impossible by addressing. Costs more at very high tenant counts; worth it wherever the data is genuinely sensitive.

Test for it properly

The test that catches this is not "does the filter work". It is a test where the correct answer exists in another tenant's data and must not be returned:

it("does not leak across tenants, even when the best match is elsewhere", async () => {
  // Tenant B holds a chunk that is a near-perfect match for the query.
  await index.upsert({ tenantId: "B", text: "The Q3 discount rate is 18%" })
  // Tenant A holds only weakly related content.
  await index.upsert({ tenantId: "A", text: "Our office is in Manchester" })

  const hits = await retrieve("what is the Q3 discount rate", { tenantId: "A" })

  expect(hits.every((h) => h.tenantId === "A")).toBe(true)
  expect(hits.map((h) => h.text).join()).not.toContain("18%")
})

The first assertion passes under both implementations. The second is the one that matters — and note that a post-filter implementation passes this test too. What post-filtering fails is the recall test:

it("returns tenant A's own answer despite a crowded index", async () => {
  // 500 chunks belonging to other tenants, all closer to the query
  // than tenant A's genuine answer.
  await seedNoise(500, { tenantId: "OTHER" })
  await index.upsert({ tenantId: "A", text: "Our refund window is 45 days" })

  const hits = await retrieve("refund window", { tenantId: "A" })
  expect(hits[0].text).toContain("45 days")   // post-filter returns nothing
})

Run both. The pair is what distinguishes the two implementations, and neither alone does.

The same bug in three other places

Once you recognise the shape — narrow the candidate set inside the query, not after it — it appears elsewhere:

• Document-level permissions. Retrieving then filtering by ACL has identical behaviour to the tenant case.
• Time windows. "Only current policy documents" applied post-hoc means the top-k fills with superseded versions.
• Language or region. Post-filtering a multilingual index returns nothing for users of the less common language.

All four are the same mistake, and all four are fixed by filtering first.

More in why your RAG returns wrong answers, filter first, then search, and retrieval systems that are actually evaluated.