Idempotency Keys Are a Governance Primitive

Most engineering teams meet idempotency keys in a payments integration. The client mints a random identifier, sends it in a header, and the processor promises that a retried charge will not become two charges. The pattern gets filed under network reliability and forgotten. That filing stops being adequate the moment a model starts taking actions. When an agent can issue a refund, file a suspicious activity report, amend a care plan or dispatch a field crew, the question your auditor, your regulator and your own incident review will ask is not "did the request succeed?" It is "how many times did the system decide to do this, and on what basis?"

An idempotency key is the cheapest mechanism that answers that question. It gives every intended effect a stable identity before the effect happens, binds that identity to the exact inputs that justified it, and forces every retry, replay and duplicate to resolve to the same recorded outcome. That is not a transport detail. It is the primitive that makes AI decisions countable, replayable and auditable, and most AI stacks we review in regulated firms do not have one at the boundary where it matters.

Exactly-once effect, not exactly-once delivery#

Distributed systems cannot guarantee exactly-once delivery. Networks drop acknowledgements, workers crash after committing but before responding, queues redeliver. What you can build is at-least-once delivery combined with an idempotent receiver, which yields exactly-once effect: the state of the system of record changes once, no matter how many times the message arrives.

Agent loops make this harder than a conventional API client, because they stack retry layers that do not know about each other. The HTTP client retries on timeout. The agent framework retries the step when a tool returns malformed output. The orchestrator re-runs the workflow node after a worker restart. The model itself, reading an ambiguous tool result, may decide to call the tool again with slightly reworded arguments. A human reviewer clicks approve twice. Without a shared identity for the intended effect, each layer can independently create a duplicate, and none of them can tell that it has.

An action without a stable identity cannot be counted. An action that cannot be counted cannot be governed.

Deriving the key#

There are two ways to derive a key, and AI systems need both.

Client-generated keys are random, typically a UUIDv4 or ULID minted once by the caller at the moment of intent. They are correct when the caller has a durable notion of "this specific attempt": a user pressing submit, a workflow step with its own persisted ID. The IETF Idempotency-Key header draft and Stripe-style APIs use this model. Its weakness in agent systems is that the caller is often a model producing a fresh tool call each turn, and a model will cheerfully mint a new identifier for the same intent.

Content-derived keys are a hash of the canonical inputs that define the decision. Two requests that mean the same thing produce the same key regardless of who sends them or how often. For a model-driven action, the inputs that define the decision include more than the business payload:

• the tenant, the business subject and the action (case ID, action type, amount)
• the model identifier and its pinned version, never a floating alias
• the hash of the prompt template that produced the decision
• the retrieval snapshot ID, meaning the version of the corpus the model actually saw
• the policy version the guardrails enforced

Including model and prompt identity is deliberate. It means the same refund proposed by a different model version is a different decision with a different key. Whether that second decision may produce a second effect is a business rule you enforce separately, with a uniqueness constraint on the subject and action, rather than something you want silently collapsed into the first.

import { createHash } from "node:crypto";
import canonicalize from "canonicalize"; // RFC 8785 JSON Canonicalization Scheme

export type DecisionInputs = {
  tenantId: string;
  subject: { type: string; id: string };
  action: string;
  params: Record<string, unknown>;
  model: { id: string; version: string };
  promptTemplateSha256: string;
  retrievalSnapshotId: string;
  policyVersion: string;
};

export function deriveIdempotencyKey(inputs: DecisionInputs): string {
  const canonical = canonicalize(inputs);
  if (canonical === undefined) throw new Error("decision inputs are not JSON-serialisable");
  const digest = createHash("sha256").update(canonical, "utf8").digest("hex");
  return `dk_${inputs.tenantId}_${digest.slice(0, 40)}`;
}

Canonicalisation matters more than the choice of hash. Key order, number formatting and string escaping differ between languages and libraries; RFC 8785 gives you a byte-stable serialisation, so a Python evaluation harness and a TypeScript gateway derive identical keys from identical inputs. Exclude everything that varies per attempt: timestamps, trace IDs, retry counters and the model's free-text rationale.

In practice we recommend a hybrid. The orchestrator derives a content key for each proposed action, and the workflow step's client-generated ID is recorded alongside it. The content key catches semantic duplicates across retry layers; the step ID tells you which attempt won.

Storage: key, fingerprint, outcome#

The idempotency store holds three things per key: the key, a fingerprint of the full request, and the recorded outcome. The fingerprint is not redundant. With client-generated keys it is the only way to detect a caller reusing a key for a different request. With content-derived keys it covers fields you intentionally left out of the key, such as a free-text note attached to the action.

CREATE TABLE idempotency_record (
  tenant_id       text        NOT NULL,
  idem_key        text        NOT NULL,
  request_sha256  bytea       NOT NULL,
  status          text        NOT NULL CHECK (status IN ('in_flight', 'completed', 'failed')),
  decision_id     uuid,
  response_body   jsonb,
  locked_until    timestamptz NOT NULL,
  created_at      timestamptz NOT NULL DEFAULT now(),
  expires_at      timestamptz NOT NULL,
  PRIMARY KEY (tenant_id, idem_key)
);

-- Claim the key atomically. No row returned means another attempt already holds it.
INSERT INTO idempotency_record
  (tenant_id, idem_key, request_sha256, status, locked_until, expires_at)
VALUES
  ($1, $2, $3, 'in_flight', now() + interval '30 seconds', now() + interval '90 days')
ON CONFLICT (tenant_id, idem_key) DO NOTHING
RETURNING idem_key;

The middleware around every effectful tool is a small state machine:

export async function withIdempotency(req: ActionRequest, run: (key: string) => Promise<Outcome>) {
  const { tenantId } = req.inputs;
  const key = deriveIdempotencyKey(req.inputs);
  const fingerprint = sha256(canonicalize(req.body)!);

  if (!(await store.claim(tenantId, key, fingerprint))) {
    const existing = await store.get(tenantId, key);
    if (!existing.requestSha256.equals(fingerprint))
      return reply(422, "idempotency key reused with a different request");
    if (existing.status === "completed")
      return replay(existing.responseBody, existing.decisionId);
    if (existing.status === "in_flight" && existing.lockedUntil > new Date())
      return reply(409, "original request is still being processed");
    // Stale lock or failure before any effect: reclaim with compare-and-set on locked_until.
    if (!(await store.reclaim(tenantId, key, existing.lockedUntil)))
      return reply(409, "concurrent reclaim");
  }

  const outcome = await run(key); // the key is passed downstream to the system of record
  await store.complete(tenantId, key, outcome);
  return outcome;
}

Three details decide whether this survives an audit.

A fingerprint mismatch is an error, never an overwrite. The IETF draft recommends 422 for a key reused with a different payload and 409 while the original request is still in flight. Quietly returning the earlier response would hide a real defect, usually an agent that mutated its arguments between retries.

Commit the outcome with the effect. If the refund is written to the same Postgres database, mark the key completed in the same transaction as the write. If the effect is an external call, pass the key downstream, since most payment and case-management APIs accept one, so the external system enforces the same identity. Record completion only after the downstream acknowledges.

TTL is a retention decision. Stripe-style APIs can prune keys after about a day because their retry windows are short. Agent systems replay much later: a drained queue backlog, a re-run of last night's batch, a replay during an investigation. Size the TTL to the longest plausible replay window. When a key does expire, the ledger entry must remain. The idempotency store is a cache of identity; the ledger is the record.

How the ledger references the key#

Every effectful decision should produce exactly one ledger entry, and that entry should carry the idempotency key in a unique column. The key then becomes the join between three views of the same event: the request (what was asked), the decision record (what the model saw and concluded) and the effect (what changed in the system of record).

With that join in place, questions that are otherwise forensic projects become queries. How many refunds did the agent issue last quarter? Count distinct keys, not log lines. Did any case receive two SAR filings? A unique constraint on (subject_id, action, reporting_period) in the ledger rejects the second insert, and the key tells you which decision attempted it. Which decisions were made under prompt template revision 14? The template hash is part of the key's preimage and stored on the decision record. This is the backbone of the decision ledger pattern: the key is the entry's natural identity.

What goes wrong without them#

The retry storm deserves emphasis. When a model provider degrades, every agent loop in flight times out together and retries together, usually on the framework's default backoff. Without idempotency at the effect boundary, a thirty-second outage becomes days of reconciliation. With it, the storm is noise that the in-flight lock absorbs.

What to do on Monday#

• Inventory every tool an agent or model-driven workflow can call that writes to a system of record. Each one needs an idempotency boundary; read-only tools do not.
• Define the canonical decision inputs for each action, including the pinned model version, prompt template hash, retrieval snapshot ID and policy version. Serialise them with RFC 8785 before hashing.
• Add an idempotency table with a tenant-scoped primary key, a request fingerprint and explicit states. Return 422 on fingerprint mismatch and 409 while in flight.
• Commit key completion in the same transaction as the effect, or pass the key to the downstream API and record completion on acknowledgement.
• Put the key on the ledger entry as a unique column, and set retention on the ledger rather than the cache.
• Test it the hard way: kill the worker after the effect commits and before the response returns, and confirm the retry is a replay. If you want a second set of eyes on the design, talk to us.