Ten Citations, One Source: The Evidence Independence Problem in AI Decision Systems

Why AI systems must count evidence roots—not evidence appearances.

An AI-assisted procurement team is evaluating a new platform. The recommendation looks unusually strong: ten citations support the vendor’s claim that customers reduce operating costs by 15 percent. The sources include a trade publication, consulting briefs, product-comparison pages, a conference recap, and an analyst’s newsletter. The system presents their agreement as corroboration and assigns high confidence.

Then someone traces the citations and finds that every number leads to the same vendor-issued case study. The trade publication summarized it, the briefs repeated it, and the comparison sites recycled those summaries. The newsletter cited the recap, whose speaker quoted the vendor release. Ten visible sources were not ten independent observations. They were one assertion traveling through ten channels.

This is the evidence independence problem: repeated claims are treated as independent confirmation. AI makes an old problem faster and less visible. Retrieval systems gather many documents, generative models turn them into fluent consensus, and agent systems produce apparently separate recommendations. None guarantees that the underlying evidence is independent.

The governing principle is simple:

Count evidence roots, not evidence appearances.

Across research, procurement, credit, clinical triage, and inventory decisions, this determines whether a system measures corroboration or merely counts echoes.

Agreement Is Not the Same as Corroboration

Agreement is useful when independent measurements converge. Separate witnesses, datasets, methods, or authorities with non-overlapping access can strengthen a claim. Yet a hundred articles syndicated from one report add distribution, not a second observation; five dashboards built from one database field can display consistency while sharing one defect.

Xin Luna Dong, Laure Berti-Équille, and Divesh Srivastava showed why voting becomes unreliable when structured data sources copy one another. Their setting—structured items with known update histories—is narrower than today’s unstructured retrieval and generation, but the lesson transfers: dependent sources make raw source counts misleading. (VLDB, 2009)

AI decision systems need to distinguish two quantities:

• Nominal support: How many documents, outputs, citations, or systems appear to support the claim?
• Independent support: How many distinct observations, records, methods, datasets, or authorities actually support it?

The gap between those quantities is where confidence inflation occurs. Visible support increases, but the evidentiary foundation does not.

Repeated distribution is not meaningless. Ten syndicated appearances can signal market awareness, regulatory attention, reputational exposure, or narrative velocity. They simply should not be counted as ten independent confirmations.

Why AI Makes the Problem Harder to See

Traditional research leaves clues about repetition. AI systems compress that trail. A retrieval system may search hundreds of passages and return one synthesis with a row of citations while hiding the dependency structure beneath them.

Citation alone does not solve it. In one ALCE benchmark setting, even the strongest evaluated systems lacked complete citation support roughly half the time. That dataset-specific result should not be universalized, but it demonstrates that displaying citations and establishing complete support are different capabilities. (Gao et al., EMNLP 2023)

Model incentives add pressure. Anthropic found that preference feedback can favor responses matching a user’s beliefs over more accurate ones. OpenAI has argued that conventional accuracy evaluations may reward guessing while penalizing uncertainty or abstention. A system pushed to answer, agree, and sound certain can turn thin evidence into an authoritative narrative. (Anthropic, 2023; OpenAI, 2025)

The risk changes when an answer can approve a supplier, alter a credit limit, prioritize a patient queue, suspend an account, or move inventory. As consequence rises, the evidence threshold and independence test must rise with it.

Provenance Is Necessary, but It Does Not Prove Truth

Organizations often collapse separate trust questions. They authenticate a source and assume accuracy, confirm a document’s integrity and assume truth, or identify an origin and assume independence. Each control answers something different.

C2PA states the boundary clearly: Content Credentials can describe origin and history, but provenance alone cannot determine whether content is true, accurate, or factual—the same distinction explored in When Everything Can Speak, What Should We Believe? An evidence-aware architecture therefore needs a claim-level record of origin, transformation, conflict, and whether each item contributes new information. (C2PA Explainer, version 2.4)

Where False Confidence Enters the Business

Evidence dependence appears even in controlled enterprise systems. A dashboard, management report, AI summary, and forecast may all show the same inventory shortage while drawing from one incorrectly mapped ERP field. The outputs agree, but they add no independent confirmation. The same chain occurs when case notes repeat one classification or market reports inherit one dataset.

Several agents can improve coverage when they use different data, tools, methods, and instructions. Agents that share the same retrieval index, prompt pack, or stop rule remain correlated regardless of their role labels.

Two 2026 preprints illustrate the evaluation gap. ProvenAI separates correctness, citation fidelity, and document influence. DRNoise calls the tendency to stop despite truthful evidence elsewhere verification inertia, observed after one plausible misleading document was introduced. These are not settled standards, but they support testing how evidence shaped an answer. (ProvenAI; DRNoise)

OWASP recommends RAG controls including provenance, allowlists, integrity checks, approvals, and validation. An independence layer must then stop downstream repetition from turning a compromised record into apparent corroboration. (OWASP RAG Security Cheat Sheet)

Build the Evidence Model Around Claims, Not Documents

Knowledge systems store documents, while decisions are made from claims. A report may mix facts, calculations, interpretations, predictions, and recommendations; treating it as one source hides those differences.

First, decompose the claim. “The supplier will reduce costs by 15 percent without increasing implementation risk” separates into savings, observed conditions, and future risk. Each needs evidence and a trace to the nearest measurement, record, experiment, interview, dataset, or original assertion. Tracing distinguishes observation from repetition without presuming primary sources are correct.

Next, cluster copies, shared datasets, syndicated reports, derivative calculations, common model outputs, and inherited assumptions. Two studies can differ in method but share data. Independence must be assessed across data, observation, method, authority, timing, incentives, and transformation path.

Keep contradictions visible; averaging can erase the signal requiring investigation. A 2026 agent-trace survey taxonomizes relational provenance connecting support, contradiction, and recovery. It is an emerging design direction, not a production standard. (Survey preprint)

Confidence should reflect independent roots, quality, relevance, currency, and consistency. Evidence must fit the claim: an experiment for causation, an audited ledger for a balance, or controlling authority for a legal determination.

The result feeds a decision gate: answer, recommend, act, abstain, or escalate. NIST’s Generative AI Profile emphasizes validation, provenance, documented limitations, monitoring, and fact-checking, particularly when sources are numerous or unknown. Those controls matter when tied to a defined action threshold. (NIST AI 600-1, 2024)

The full operating sequence is:

1. Decompose the decision into atomic claims.
2. Trace each claim to its closest available evidence origin.
3. Cluster copied, syndicated, shared-data, and derivative sources.
4. Test independence across data, observation, method, authority, timing, incentives, and transformation path.
5. Preserve conflicts and investigate why they exist.
6. Calculate confidence from independent roots, quality, relevance, and currency.
7. Apply a consequence-based gate: answer, recommend, act, abstain, or escalate.
8. Propagate corrections to every downstream output that relied on the affected claim.

That last step matters. Correcting the source without correcting the summaries, dashboards, embeddings, reports, and decisions derived from it leaves the organizational echo intact.

The Minimum Viable Independence Layer

Start with high-consequence workflows such as procurement claims, credit exceptions, clinical triage, or inventory actions. Prefer existing lineage—ERP field IDs, hashes, source URIs, dataset versions, model IDs, and transformation logs—before asking a model to infer relationships from text.

Near-duplicate and embedding similarity can flag dependence but cannot prove a common root. Production designs need independence unknown as a first-class state that blocks unsupported confidence gains and triggers the appropriate gate.

What the Procurement Team Should Have Seen

The procurement team needed the unanswered diagnostic questions. Was the 15 percent reduction audited or estimated? What was the baseline, and were implementation and transition costs included? How were customers selected? Has an independent customer or controlled pilot reproduced the result under comparable conditions?

If answers are unavailable, the system should request auditable calculations, customer evidence, a controlled pilot, or contractual measures. It should conclude: “The support traces to one root, so independent corroboration has not been established.” That prevents false certainty without turning uncertainty into accusation.

What the User Should See

The interface should expose the decision state, not bury it in a provenance log:

• Nominal support: 10 cited items
• Independent roots: 1 confirmed; 0 additional; 2 unknown
• Dependency view: one root with its derivative documents grouped beneath it
• Contradictions: none found, but absence of contradiction is not corroboration
• Decision gate: human review; no automated approval
• Next evidence request: auditable customer results or a controlled pilot

This gives executives the decision boundary and reviewers a path into the cluster map.

A Reusable Evidence Independence Audit

This prompt can support research, procurement, compliance, risk, or multi-agent review when full sources and structured traces are available.

You are conducting an evidence-independence audit for a consequential decision.

Decision: [describe the decision]
Consequence if wrong: [low/moderate/high/critical and why]
Materials: [provide or link the source set]

1. Decompose the decision into atomic factual, calculated, inferred,
   predicted, and normative claims.
2. For each claim, identify every cited supporting and contradicting item.
3. Trace each item to the closest available origin: direct observation,
   record, dataset, experiment, interview, or original assertion.
4. Group items that copy, syndicate, summarize, transform, or share the
   same underlying data, method, authority, or assumption.
5. Report the nominal source count, confirmed independent-root count, and
   number of items whose independence remains unknown.
6. Evaluate each root for relevance, quality, currency, incentives,
   limitations, and contradiction.
7. State what is observed, inferred, predicted, uncertain, or normative.
8. Recommend one decision gate: answer, recommend, act, abstain, or
   escalate to human review. Explain the threshold used.
9. List the smallest set of additional evidence that would most improve
   confidence.

Required output fields: claim, root or cluster, dependence type, independence
status, contradiction, confidence rationale, decision gate, decision authority,
and smallest next evidence request.

Do not treat repetition as corroboration. Do not resolve contradictions by
averaging or omission. If lineage cannot be established, set independence to
unknown rather than assuming either dependence or independence.

Prompts are not durable governance. Production systems need an auditable, machine-readable record. This is an illustrative interface stub, not a standard or scoring model; its categorical judgments require documented criteria.

{
  "claim_id": "CLM-1042",
  "claim_text": "The platform reduces annual operating cost by 15 percent.",
  "claim_type": "predicted",
  "decision_impact": "high",
  "evidence": [
    {
      "evidence_id": "E-01",
      "source_uri": "https://example.org/vendor-case-study",
      "origin_id": "ROOT-01",
      "source_type": "original_assertion",
      "observed_at": "2026-06-30",
      "method": "vendor-reported customer case study",
      "supports": true,
      "independence_cluster": "ROOT-01",
      "dependence_type": "original_root",
      "independence_status": "confirmed_root",
      "quality_assessment": "limited: vendor-controlled and unaudited",
      "relevance_assessment": "directly addresses claimed cost reduction",
      "contradiction_reason": null,
      "correction_refs": []
    }
  ],
  "nominal_source_count": 10,
  "dependent_item_count": 7,
  "independent_root_count": 1,
  "independence_unknown_count": 2,
  "confidence": {
    "level": "low",
    "reason": "All supporting items derive from one vendor-controlled origin."
  },
  "decision_gate": "human_review",
  "decision_authority": "procurement_review_board",
  "next_evidence_request": "Auditable customer results or a controlled pilot.",
  "reviewed_at": "2026-09-08T00:00:00Z",
  "correction_refs": [],
  "supersedes": []
}

The exact fields will vary, but claim identity, source origin, dependence type, independence status, contradictions, confidence rationale, decision authority, review time, and correction history should be explicit in high-impact use. Without them, an organization may reproduce an answer while remaining unable to explain why the evidence deserved trust.

Evaluate the System That Evaluates the Evidence

Evidence independence belongs in model and workflow testing. Accuracy alone misses systems that reach the right answer for the wrong reason, cite unused sources, or fail when a plausible misleading record enters the corpus.

Tests should introduce controlled perturbations: add a syndicated article, duplicate a passage under a new domain, insert a plausible false record, remove the primary source, or change its timestamp. Confidence should change only for defensible reasons.

Organizations should also compare predicted confidence with verified outcomes and record human overrides. NIST’s public-facing AI documentation draft, released July 30, 2026, calls attention to sources, uses, limitations, validation, and lifecycle context. Dependence belongs in that record because the same model can be reliable in one information environment and overconfident in another. (NIST AI Standards “Zero Draft,” initial public draft, 2026)

Multi-agent voting is not an evaluation shortcut. Separate roles can improve coverage, but correlated systems do not provide independent validation. TRACE-CTI cautions that observed corroboration does not establish statistical independence or a causal effect from model-family diversity. Plurality must be tested, not assumed. (2026 preprint)

The Leadership Standard

Leaders need three mandatory questions for every high-consequence AI recommendation:

1. How many independent evidence roots support each material claim?
2. What data, methods, authorities, assumptions, and models do the apparent sources share?
3. What decision gate applies when lineage is incomplete, contradictory, or unknown?

Vendors should show how their systems detect duplication, shared data, copied claims, and citation influence, then propagate corrections. Systems that cannot expose those controls should not receive autonomous authority merely because their answers are fluent or heavily cited.

Institutions, experts, peer review, and consensus remain valuable. The objective is to identify what support each contributes. A conclusion reached independently through several qualified methods differs from one repeated by respectable channels; they should not receive the same evidentiary weight.

The trust problem is no longer whether AI can answer. It is whether the system can distinguish discovery from repetition, corroboration from consensus theater, and confidence from volume.

A trustworthy decision system does more than show its sources. It knows when ten citations are one source—and refuses to turn an echo into authority.

Appendix: Applying the Audit to the 15 Percent Claim

Decision: Approve the platform partly on a claimed 15 percent cost reduction. Consequence if wrong: High; the claim affects selection, spending, staffing, and projected return.

The claim separates into the historical reduction, baseline, included costs, customer relevance, and predicted buyer result. Ten items are visible: one confirmed vendor-controlled root, seven derivatives, and two with unknown lineage. No contradiction has been found, but its absence does not create corroboration.

The gate is human review. The procurement board should request the calculation, verify cost inclusion, obtain independent customer evidence, and require a pilot or contractual measure if the claim is material. The result may be true, but this source set does not independently corroborate it.

Sources

• Truth Discovery and Copying Detection in a Dynamic World — VLDB, 2009
• Enabling Large Language Models to Generate Text with Citations — EMNLP, 2023
• Towards Understanding Sycophancy in Language Models — Anthropic, 2023
• Why Language Models Hallucinate — OpenAI, 2025
• C2PA Explainer, version 2.4
• Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile — NIST AI 600-1, 2024
• RAG Security Cheat Sheet — OWASP
• ProvenAI — 2026 preprint
• DRNoise — 2026 preprint
• From Agent Traces to Trust — 2026 survey preprint
• Guidance and Templates for Public-Facing AI Documentation: An AI Standards “Zero Draft” — NIST initial public draft, 2026
• TRACE-CTI — 2026 preprint