10 to the 23 AI logo
Stephen Lieberman
Through 1023AI
Contact

Practice Library

Governance patternstructural

Vetted sources only

Restrict what the model can retrieve to a vetted corpus — a bound on the contamination that open auto-retrieval would carry back in, not a scrub.

What it changes

dampenedContaminated records repeated as fresh output(retrieval restricted to a vetted corpus)

Who can pull it

Harness builderDeveloperDeploying organization

What it looks like institutionally

Retrieval layers propagate rather than sanitize their inputs: a retrieval-augmented system stays unfaithful even when the retrieved passage is correct, so open auto-retrieval rides whatever contamination the record store holds straight back into the model. The record-to-model pathway becomes a loop that re-teaches the system its own errors.

Restricting retrieval to a curated, vetted corpus bounds what can re-enter. Fact-checking against a curated, peer-reviewed corpus reaches high accuracy, so a vetted corpus cuts the contamination drawn back into the model relative to open retrieval — a meaningful reduction on the memory-contamination inflow the framework tracks.

It is a bound, not a guarantee: curated-corpus verification itself tops out short of certainty and degrades under knowledge conflict. A vetted corpus raises the floor on what the model retrieves; pair it with provenance labeling so the records inside the corpus still carry their verification status.

Addresses: Contamination drawn back into the model · Open retrieval propagating errors.

Deciding whether this lever fits your deployment?

Which patterns matter, and in what order, depends on your system's actual shape. Ranking your options on evidence, with what can backfire stated, is engagement work.

Work With 1023AIBrowse the Practice Library

Sources & Evidence

Claims made on this page and what supports them. The full registry lives in Evidence.

EmpiricalRetrieval layers propagate rather than sanitize their inputs: studies find retrieval-augmented systems remain …

Retrieval layers propagate rather than sanitize their inputs: studies find retrieval-augmented systems remain unfaithful even when the retrieved passage is correct, so faithfulness is bounded rather than assured.

faithfulragGroundingPreprintSave

FaithfulRAG (arXiv:2506.08938) — RAG systems struggle in knowledge-conflict scenarios even when relevant passages are retrieved (pessimistic end). https://arxiv.org/abs/2506.08938

https://arxiv.org/abs/2506.08938

Grounds: empirical cap: groundtruth_reliability (max)

faithfulragwithsparseautoencGroundingPreprintSave

Faithful RAG with Sparse Autoencoders (arXiv:2512.08892) — even with relevant passages retrieved, models contradict evidence / invent details; faithfulness is not guaranteed. https://arxiv.org/abs/2512.08892

https://arxiv.org/abs/2512.08892

Grounds: empirical cap: groundtruth_reliability (max)

ragevaluationsurveyGroundingPreprintSave

RAG evaluation survey (arXiv:2405.07437) — factuality evaluation is bounded by knowledge-base coverage and retrieval accuracy; what is checkable depends on what is documented. https://arxiv.org/abs/2405.07437

https://arxiv.org/abs/2405.07437

Grounds: empirical cap: frac_verifiable (max)

EmpiricalRestricting retrieval to a curated, vetted document set bounds what re-enters the model: retrieval-augmented s…

Restricting retrieval to a curated, vetted document set bounds what re-enters the model: retrieval-augmented systems fact-checking against a curated peer-reviewed corpus reach roughly 0.97+ accuracy and factuality evaluation is limited by knowledge-base coverage — what is checkable depends on what is documented — so a vetted corpus reduces contamination drawn back into the model relative to open retrieval, though faithfulness remains imperfect under knowledge conflict.

retrievalaugmentedcovidfactcGroundingPeer-reviewedSave

Retrieval-augmented COVID-19 fact-checking (PMC12079058) — CRAG/Self-RAG reach 0.972-0.978 accuracy against a curated 130k peer-reviewed corpus (optimistic ceiling). https://pmc.ncbi.nlm.nih.gov/articles/PMC12079058/

https://pmc.ncbi.nlm.nih.gov/articles/PMC12079058/

Grounds: empirical cap: groundtruth_reliability (max)

ragevaluationsurveyGroundingPreprintSave

RAG evaluation survey (arXiv:2405.07437) — factuality evaluation is bounded by knowledge-base coverage and retrieval accuracy; what is checkable depends on what is documented. https://arxiv.org/abs/2405.07437

https://arxiv.org/abs/2405.07437

Grounds: empirical cap: frac_verifiable (max)

faithfulragGroundingPreprintSave

FaithfulRAG (arXiv:2506.08938) — RAG systems struggle in knowledge-conflict scenarios even when relevant passages are retrieved (pessimistic end). https://arxiv.org/abs/2506.08938

https://arxiv.org/abs/2506.08938

Grounds: empirical cap: groundtruth_reliability (max)

EmpiricalThe verification channel itself is bounded — curated-corpus fact-checking tops out around 0.972–0.978 reliabil…

The verification channel itself is bounded — curated-corpus fact-checking tops out around 0.972–0.978 reliability and collapses under knowledge conflict.

retrievalaugmentedcovidfactcGroundingPeer-reviewedSave

Retrieval-augmented COVID-19 fact-checking (PMC12079058) — CRAG/Self-RAG reach 0.972-0.978 accuracy against a curated 130k peer-reviewed corpus (optimistic ceiling). https://pmc.ncbi.nlm.nih.gov/articles/PMC12079058/

https://pmc.ncbi.nlm.nih.gov/articles/PMC12079058/

Grounds: empirical cap: groundtruth_reliability (max)

faithfulragGroundingPreprintSave

FaithfulRAG (arXiv:2506.08938) — RAG systems struggle in knowledge-conflict scenarios even when relevant passages are retrieved (pessimistic end). https://arxiv.org/abs/2506.08938

https://arxiv.org/abs/2506.08938

Grounds: empirical cap: groundtruth_reliability (max)

faithfulragwithsparseautoencGroundingPreprintSave

Faithful RAG with Sparse Autoencoders (arXiv:2512.08892) — even with relevant passages retrieved, models contradict evidence / invent details; faithfulness is not guaranteed. https://arxiv.org/abs/2512.08892

https://arxiv.org/abs/2512.08892

Grounds: empirical cap: groundtruth_reliability (max)