Your RAG System Is Serving Stale Data Without Knowing It
Your document store and your vector index are two separate data structures. Every write diverges them. You never decided what to do about that.
What This Post Covers
Why the split architecture is a distributed system — and what that means for correctness.
The four consistency anomalies your pipeline produces right now.
Why the retriever can’t see the problem — and why your monitoring can’t either.
The consistency spectrum: from eventual to strong, and what each costs.
What a declared consistency model actually looks like in practice.
The tombstone problem: why deletion is the hardest case.
Here is a failure that looks like everything is working.
A policy document is updated at 2 PM. The editor saves the change, gets a confirmation, moves on.
The vector index is on a nightly re-index schedule. For the next ten hours, every query about that policy retrieves the old version — with full confidence, normal latency, healthy similarity scores, and a green dashboard.
The model answers authoritatively from a document that no longer exists.
No errors. No alerts. No anomalies in any metric you’re watching.
This is not an edge case. It is the default behavior of every RAG pipeline that has not explicitly decided otherwise. And the reason almost none of them have decided otherwise is that the decision was never framed correctly. It was never framed as a consistency problem.
It should have been. Because what you have built — a document store feeding a vector index through an ingestion pipeline — is a distributed system with two separate data structures that must be kept in sync. Every write to one diverges it from the other.
The ingestion pipeline is the synchronization mechanism.
lag between source truth and retrievable truth is the inconsistency window. And you, in all likelihood, have never defined how wide that window is allowed to be.
That is the absence of a consistency model. And it is the root cause of an entire class of RAG failures that no retriever tuning and no prompt engineering can fix.
The Split Architecture Is a Distributed System
Zoom out to the shape of the thing.
A standard RAG pipeline has at least two data stores: the authoritative document source (a database, a CMS, a file store, a wiki) and the vector index.
The ingestion pipeline sits between them, reading from one and writing to the other.
This is a classic distributed data pattern, and distributed data patterns have a well-understood failure mode: the two stores diverge.
The authoritative source changes. The ingestion pipeline runs.
The vector index updates. Between the source change and the index update, a reader who queries the index gets an answer that contradicts the source of truth.
That window — between source commit and index searchability — is the inconsistency window.
In a standard batch ingestion setup, it is measured in hours.
In a streaming CDC setup, it can be measured in seconds.
In a split-system architecture with asynchronous metadata and vector updates, one benchmark measured it at 3.54 milliseconds on average — but critically, under high write load, this window grows.
The window is always there.
The only question is whether you’ve declared how wide it’s allowed to be.
Most teams haven’t.
They know they re-index nightly, so the window is “up to 24 hours,” but they haven’t written that down as a consistency guarantee or an SLA.
They haven’t decided whether 24 hours is acceptable for their domain.
They haven’t built monitoring to detect when the actual window exceeds the intended one. And they have no mechanism to make the inconsistency visible to the user or the agent reading from the index.
The absence of a declaration is not the absence of a model. It is an implicit model: your system is eventually consistent, in a way nobody designed, with a staleness bound nobody measured.
The question is whether that’s acceptable for your use case, and it’s a question you haven’t been asked.
The Four Consistency Anomalies
Distributed systems have taxonomies for the ways consistency breaks.
RAG systems produce all of them, and naming them makes the problem tractable.
Stale reads: The most common.
A source document was updated, but the index hasn’t caught up.
A reader gets an answer from the old version.
The failure is silent because the retriever doesn’t know the version it returned is outdated.
Semantic similarity has no temporal dimension — a stale embedding scores just as high as a fresh one.
The system doesn’t know and can’t tell.
Consider the example from a 2026 research paper on temporal validity: an agent has accumulated both “the service runs on port 8000” and “the service runs on port 8080” — the second written after a migration.
A query about the port retrieves both. They are near-identical in embedding space, so both score high. RAG provides no basis for deciding which is current.
There is no mechanism in standard retrieval to prefer the more recent fact over the earlier one, and the model must choose between two maximally similar results with no temporal signal.
Phantom reads: A document that was deleted from the source is still retrievable from the index.
Deleted content can remain retrievable and appear as legitimate evidence. The model cites a document that no longer exists, or was retracted, or was superseded for reasons that make its retrieval actively harmful.
The deletion happened in the source. It did not propagate to the index. The phantom lives on.
Dirty reads: A document is in the middle of an update when a query arrives. The source has the new version. The index has the old.
A different query seconds later gets the new version.
Two simultaneous users asking the same question get different answers, grounded in different versions of the same document, with no indication that anything is inconsistent. This is especially acute in split architectures where metadata and vector updates are written separately — the metadata may reflect the new version while the embedding still encodes the old text.
Non-repeatable reads: An agent queries the index on step one of a multi-step reasoning loop and retrieves document A at version N.
On step seven, the same document has been updated to version N+1.
The agent’s context now contains contradictory information from the same source at two different points in time — and has no way to know it.
Every one of these anomalies is a named, well-understood consistency failure from distributed systems theory.
None of them are detectable at the vector search layer.
All of them look healthy on a latency dashboard.
Why Your Monitoring Can’t See It
This is the part that makes consistency failures particularly treacherous in RAG systems: they are invisible to every instrument you already run.
A stale retrieval returns a result with normal latency.
Similarity scores look healthy.
The model produces a fluent, confident answer.
No span throws an error.
No circuit breaker trips. No metric trends wrong.
The only way to know the answer was wrong is to already know the right answer — which means the failure is undetectable without a ground truth, and most systems have no ground truth attached to retrieval.
One benchmark stated it cleanly: a RAG answer can be wrong even when the model is healthy, the vector database is online, and the application latency dashboard is green.
The failure started earlier — a policy document changed, an entitlement record moved — and the retrieval layer kept serving older context.
This is the same problem that runs through this entire arc of posts.
The failure is green.
The dashboard is not lying — it is measuring the wrong things.
Latency and error rates are properties of the infrastructure layer.
Consistency is a property of the data layer. And the data layer has no instrumentation on most RAG systems, because the retrieval stack was built by the team that owned ML, not the team that owns data infrastructure, and the gap between them is where consistency assumptions go undeclared.
The Consistency Spectrum
Before you can fix it, you have to choose a position on the spectrum — because strong consistency is not free, and eventual consistency is not wrong. They are trade-offs, and the right choice depends on your domain.
Eventual consistency means updates propagate asynchronously. New and changed documents become retrievable after some lag — seconds to hours depending on the ingestion architecture.
For many workloads this is fine: a technical documentation site where the FAQ changes monthly, a product knowledge base where day-old retrieval causes no material harm.
The cost is low; the freshness guarantee is weak.
Bounded staleness means you commit to an explicit inconsistency window — say, documents are retrievable within five minutes of a source change. You enforce it by measuring the actual ingestion lag on every document class and alerting when the bound is violated. This is the first position that’s actually a model, because it makes the guarantee explicit and the violation detectable.
Read-your-writes consistency means that if a user or agent just wrote or triggered an update, their subsequent read should see that update.
In RAG terms: if an agent’s tool call updated a document, the agent’s next retrieval should return the updated version, not the old one. This requires routing the post-write read to a path that’s guaranteed fresh — typically bypassing the async ingestion path and reading directly from source, or using a synchronous index update.
Strong consistency means every read reflects the most recent committed write. It requires atomic writes across both the document store and the vector index, which typically means the document and embedding are updated in the same transaction.
The unified architecture that achieves a 0ms inconsistency window does exactly this — but it requires co-locating storage layers that most architectures separate, and it introduces write latency proportional to the cost of synchronous embedding generation.
The right point on this spectrum is a function of how fast your corpus changes and what the cost of staleness is.
A compliance-regulated enterprise where a stale policy answer creates legal exposure needs strong consistency or bounded staleness with a very short bound.
A general knowledge base where answers are informational needs only eventual consistency with reasonable bounds.
The declaration of which you have chosen is the consistency model.
The Tombstone Problem
Deletion deserves its own section because it is the hardest case and the one most teams get wrong.
When a document is deleted from the source, the expected behavior is that it stops being retrievable. This sounds obvious. It is not automatic.
The vector index holds embeddings by chunk ID. The source deletion removed the document.
Unless the ingestion pipeline explicitly propagates the deletion — writing what the database world calls a tombstone, an explicit removal marker — the chunk embeddings remain in the index. They are orphaned but alive. Retrievable. Authoritative-looking. Wrong.
Practical update workflow requires three invariants: stable IDs so updates replace the right chunks, versioning so you can purge old versions and audit what the model saw, and tombstones so deletions create explicit removal events and old embeddings don’t linger.
The tombstone problem compounds with multi-source ingestion.
If your index pulls from five sources and one of them has a deletion event that doesn’t translate into a format your ingestion pipeline recognizes, the document stays. Forever.
This is how retracted policies, outdated specs, and deprecated endpoints end up in production retrieval as active evidence.
Strong consistency of deletion requires that a document deletion in the source be reflected in the index before any subsequent query can retrieve it.
Eventual consistency of deletion means there’s a window — possibly hours — where the deleted document is still live. You need to know which you have and whether it’s acceptable.
What a Declared Consistency Model Looks Like
A consistency model is not a single switch. It is a set of decisions, one per document class and query type, with explicit bounds and explicit monitoring.
Classify your corpus by change rate and staleness cost: Static reference content can tolerate nightly re-index and days of staleness. Operational facts — access controls, pricing, policy, real-time status — need streaming ingestion and minute-level freshness bounds. Regulatory content may require strong consistency with synchronous updates. Treat them as separate pipelines with separate guarantees, not one pipeline with one schedule.
Define a freshness SLA per class and measure it: The inconsistency window is the lag between source commit and index searchability. Track it per document class, per ingestion path, alert when it exceeds the declared bound. This is not exotic monitoring — it requires a committed_at timestamp in the source and a indexed_at timestamp in the index, and a dashboard that shows the distribution of the gap. Most teams have neither timestamp.
Propagate deletions explicitly: Deletions must generate tombstones that the ingestion pipeline processes with at least the same priority as updates. A pipeline that re-indexes on a schedule but only processes additions is a pipeline where deleted content never leaves the index.
For agents: enforce read-your-writes: An agent that just triggered an update should read from a path that reflects it. The simplest implementation: after a write, bypass the async index and query the document store directly for a configurable freshness window before the index has caught up.
Monitor retrieval freshness, not just retrieval latency: Track the age distribution of retrieved chunks at query time. Alert when the median age of retrieved content for a query class exceeds the freshness SLA. Nearest-neighbor stability — comparing top-k results week-over-week — is the canary for silent index drift. Healthy systems maintain 85–95% overlap in top-10 results across time intervals. When overlap drops below 70%, active quality loss is underway, even if similarity scores look fine.
The Bottom Line
You have been treating your vector index as a replica of your document source. It is not a replica. It is an asynchronously updated projection, separated from the source by an ingestion pipeline, diverging from it the moment any write lands.
Distributed systems handle this with declared consistency models: explicit statements about how stale a read is allowed to be, what anomalies are tolerated, and what guarantees hold across writes.
RAG systems almost never have one — not because the problem is new, but because the people who built the retrieval stack weren’t thinking in distributed-systems terms.
The failure modes that result — stale reads, phantom deletes, dirty reads, non-repeatable reads — are not retrieval failures. They are consistency failures. And you cannot fix them with a better reranker, a tighter prompt, or a bigger embedding model.
You fix them by deciding, in writing, what your system promises about the relationship between a source update and a retrieval result. That decision — the consistency model — is the piece your RAG pipeline is missing.
Write it down. Then build the monitoring that proves you’re keeping the promise.