Why Your AI Has No Decision Lineage

When someone asks why a model denied a particular claim on a particular Tuesday, most teams open their observability tool. They find a trace: a span for retrieval, a span for the model call, latencies, token counts, perhaps a truncated prompt. What they cannot find is which version of the policy manual the retriever returned, whether those chunks have since been re-embedded or edited, which revision of the prompt template was live, which model snapshot sat behind the provider's alias that afternoon, and what the eligibility service returned when the agent called it. The trace describes what happened to the request. It does not describe what produced the decision.

That second thing is lineage, and almost no production AI stack has it. Lineage is a directed acyclic graph of the immutable inputs that produced a decision — data snapshots, retrieved chunks with content hashes, prompt template version, model version and sampling parameters, guardrail verdicts, human approvals — captured at decision time, sealed into a tamper-evident chain, and queryable in both directions. Without it, every explanation you give a regulator, a court or a customer is a reconstruction, and reconstructions carry caveats.

Logs and traces are not lineage#

Logs and traces are built for operators. They are sampled, retention-limited, and deliberately stripped of payloads to control cost and keep personal data out of the telemetry pipeline. Their data model is an event stream or a timing tree: this span called that span, and it took 840 milliseconds. None of that is wrong for debugging latency. It is the wrong shape for accountability.

Lineage has a different data model. Its nodes are artifacts identified by content hash, so a node cannot silently change after the fact. Its edges mean "was used to produce", not "was called by". It is recorded synchronously, as part of making the decision, rather than reconstructed from telemetry later. And it is retained on the schedule of the decision it explains, which in lending, clinical or safety contexts can mean years, not the retention window your log vendor applies by default.

If you have to reconstruct what the model saw, you do not know what the model saw.

Where typical stacks lose it#

Lineage is rarely lost in one place. It leaks out of five ordinary engineering choices.

Mutable vector stores. Ingestion pipelines upsert chunks in place. The chunk ID survives while its text changes, and re-embedding the corpus with a new embedding model changes which neighbours any query returns. Unless retrieval runs against a named snapshot, you cannot know afterwards what the retriever could have returned, let alone what it did.

Unversioned prompt templates. Templates live as string literals in application code, assembled at runtime from feature flags and configuration. The deployed commit is usually knowable; the exact rendered prompt for a given decision usually is not.

Provider model aliases. An alias such as latest, or an undated model name, can be repointed by the provider. The model you evaluated is not necessarily the model that decided. Most provider responses include the resolved model version; most applications discard it.

Sampling. With temperature above zero, the same inputs can produce different outputs. Lineage therefore stores the output and its hash, plus the full parameter set and seed where the provider supports one. It never relies on regenerating an answer to learn what the answer was.

Unpersisted tool results. An agent calls a credit bureau, an eligibility service or an internal pricing API. The result exists only inside the context window and the provider's request, and neither is yours to query later.

The decision record#

The unit of lineage is a decision record, written once when the decision is made. It references its inputs by content hash rather than embedding them, so the record stays small and sensitive payloads can live in a separately controlled store with their own access policy.

{
  "decision_id": "dec_01JQ7Z6K4Q2M8V",
  "stream": "claims.adjudication",
  "decided_at": "2026-04-02T14:07:31.482Z",
  "subject": { "type": "claim", "id": "CLM-448120" },
  "outcome": { "action": "deny", "reason_code": "EXCLUSION_4B", "confidence": 0.71 },
  "model": {
    "requested": "vendor-model-large",
    "resolved_version": "vendor-model-large-2026-02-15",
    "params": { "temperature": 0.0, "top_p": 1.0, "max_tokens": 800, "seed": 7 }
  },
  "prompt": { "template_id": "claims-adjudicate", "template_version": 14,
              "template_sha256": "9f2c…e1", "rendered_sha256": "41ab…7c" },
  "inputs": [
    { "kind": "record", "ref": "claims.claim@lsn:0/3A7F2B10", "sha256": "c03d…9a" },
    { "kind": "chunk", "ref": "policy-docs@snap_2026-03-30#c_1182", "sha256": "77e1…04" },
    { "kind": "tool_result", "ref": "eligibility.check#call_3", "sha256": "b5a0…3f" }
  ],
  "guardrails": [
    { "name": "pii_egress", "version": "3.2.0", "verdict": "pass" },
    { "name": "coverage_policy", "version": "2026.03", "verdict": "requires_review" }
  ],
  "approvals": [
    { "role": "senior_adjuster", "principal": "u_5521", "verdict": "approved", "at": "2026-04-02T15:41:09Z" }
  ],
  "output_sha256": "e8f4…22",
  "prev_hash": "sha256:5b19…d0",
  "record_hash": "sha256:a4c7…9e"
}

Note what is absent: no latency, no token counts, no span IDs. Those belong in telemetry. Everything present answers one question — what did this decision depend on?

Hash chaining

A record that can be edited quietly is not evidence. Each record carries the hash of its predecessor in the same stream, and its own hash is computed over its canonical JSON, including prev_hash. Canonicalisation per RFC 8785 matters: without a byte-stable serialisation, a verifier written in another language will compute different hashes from identical content.

import { createHash } from "node:crypto";
import canonicalize from "canonicalize"; // RFC 8785 JCS

type Unsealed = Record<string, unknown>;
type Sealed = Unsealed & { prev_hash: string; record_hash: string };

const digest = (value: unknown): string =>
  "sha256:" + createHash("sha256").update(canonicalize(value)!, "utf8").digest("hex");

export function seal(record: Unsealed, prevHash: string): Sealed {
  const body = { ...record, prev_hash: prevHash };
  return { ...body, record_hash: digest(body) };
}

/** Returns the index of the first broken record, or -1 if the chain verifies. */
export function verifyChain(records: Sealed[], genesis: string): number {
  let expectedPrev = genesis;
  for (let i = 0; i < records.length; i++) {
    const { record_hash, ...body } = records[i];
    if (body.prev_hash !== expectedPrev || digest(body) !== record_hash) return i;
    expectedPrev = record_hash;
  }
  return -1;
}

Two operational points. First, chain per stream — per tenant and decision type — rather than globally, because appending requires reading the current head under a lock, and a single global chain becomes a write bottleneck. Second, a hash chain is tamper-evident, not tamper-proof: someone with write access could rewrite the whole chain. Periodically anchor each stream's head hash somewhere the writer cannot modify, such as object storage with a retention lock or a separate system under different administrative control. The decision ledger pattern builds on exactly this structure.

Querying lineage in both directions#

Store the DAG explicitly as edges: a decision's inputs are its parents, a chunk's parent is the document version it was cut from, and a decision can itself be the parent of a later decision when one agent's output feeds another. Forward queries answer "why did this happen?" Reverse queries answer the question that actually arrives during an incident: a policy document was wrong, so which decisions relied on it?

-- lineage_edge(child_id, parent_id, parent_kind, parent_sha256)
WITH RECURSIVE downstream AS (
  SELECT e.child_id, 1 AS depth
  FROM lineage_edge e
  WHERE e.parent_id = 'doc:claims-policy-manual@v7'
  UNION
  SELECT e.child_id, d.depth + 1
  FROM lineage_edge e
  JOIN downstream d ON e.parent_id = d.child_id
  WHERE d.depth < 12
)
SELECT r.decision_id, r.stream, r.decided_at,
       r.outcome ->> 'action' AS action, min(d.depth) AS hops
FROM downstream d
JOIN decision_record r ON r.decision_id = d.child_id
GROUP BY r.decision_id            -- primary key, so the other columns are functionally dependent
ORDER BY r.decided_at;

The walk passes through chunks to the decisions that consumed them, and on through any downstream decisions built on those. The depth bound is a guard against bad data, not a feature of a well-formed DAG.

Exact replay and counterfactual replay#

Lineage exists so you can replay. There are two modes, and they answer different questions.

Exact replay proves the record is complete. Re-render the prompt from the stored template version and stored inputs and check that its hash matches rendered_sha256. Re-run the deterministic parts — rules, guardrails, schema validators — and check that the verdicts match. You do not re-call the model expecting identical text; you compare against the stored output, because sampling and provider-side changes make regeneration an unreliable witness. If exact replay fails, the lineage has a gap, and you want to find that in a quarterly test rather than during a dispute.

Counterfactual replay swaps one node and re-runs. Substitute the corrected policy document, the candidate model version or the revised template, hold everything else fixed, and measure how many outcomes change. This is how you size the impact of a retracted document before deciding which customers to contact, and how you validate a model upgrade against real historical traffic instead of a synthetic eval set. It only works if the snapshots referenced by the record still exist, so snapshot retention must match decision retention.

A readiness mapping, not a certificate#

Lineage does not make a system compliant with anything by itself. It does supply the evidence several controls ask for. The EU AI Act's record-keeping obligation for high-risk systems, Article 12, requires that systems technically allow automatic recording of events over their lifetime, to support traceability and post-market monitoring; a hash-chained decision record is a direct implementation of that capability. The human oversight requirements of Article 14 are easier to evidence when approvals are lineage nodes carrying who approved, in what role, and what they were shown.

In the NIST AI RMF, lineage primarily supports MEASURE, where you track system behaviour in deployment and need to attribute changes in outcomes to changes in inputs, and MANAGE, where you respond to incidents and must scope their impact. The reverse lineage query above is, in practice, an incident-scoping tool. We treat these as readiness mappings in our governance work: what a control requires and which artifact satisfies it, not a claim that the artifact alone discharges it.

The checklist#

• Pin model versions and record the resolved version string from every provider response.
• Version prompt templates as data, with an ID, a version number and a content hash; log the rendered-prompt hash per decision.
• Retrieve against named, immutable index snapshots, and record chunk IDs and text hashes for everything placed in context.
• Persist tool results, or their hashes plus a pointer to a controlled store, keyed by tool call ID.
• Write one decision record per decision, synchronously, canonicalised with RFC 8785 and chained per stream; anchor stream heads outside the writer's control.
• Store lineage as explicit edges, and rehearse the reverse query before you need it.
• Run exact replay on a sample every quarter; treat any failure as a lineage defect.