Idempotent API Design: Preventing Duplicate Transactions in Distributed Systems
Learn how to architect deterministic, fault-tolerant POST/PATCH endpoints with Idempotency-Keys, distributed Redis locks, and database transaction boundaries.
In distributed architectures and unreliable mobile/web networks, network timeouts cause clients to retry requests. Without deterministic idempotency, retries cause double charges, duplicate orders, and state corruption.
The Root Problem: Network Retries and the Two Generals Paradox
When a client makes an HTTP POST request to create a payment or reservation, the server may process the record successfully, but the TCP connection may drop before the HTTP 200 response reaches the client. The client, unaware of the server state, automatically retries. Without an idempotency layer, this creates duplicate financial transactions and inconsistent records.
Architecture Pattern: The Idempotency-Key Header with Redis & PostgreSQL
We implement a 3-tier state machine using an `Idempotency-Key` HTTP header. 1) When a request arrives, we execute an atomic `SETNX` in Redis with a short TTL (e.g. 30 seconds) in state `PROCESSING`. 2) If the key already exists and is `PROCESSING`, we return HTTP 409 (Conflict) or wait. 3) Once the database transaction commits, we persist the cached response status and payload into Redis with a 24-hour TTL and release the lock.
export async function idempotencyGuard(req: Request, key: string, redis: Redis) {
const cacheKey = `idempotency:${key}`;
// Atomic check-and-set lock
const acquired = await redis.set(cacheKey, JSON.stringify({ status: "PROCESSING" }), "NX", "EX", 30);
if (!acquired) {
const cached = await redis.get(cacheKey);
const parsed = JSON.parse(cached || "{}");
if (parsed.status === "PROCESSING") {
throw new ConflictError("Transaction currently being processed. Please do not retry concurrently.");
}
// Return deterministic cached response for safe retry
return { isCached: true, statusCode: parsed.statusCode, body: parsed.body };
}
return { isCached: false };
}Always enforce unique database constraints on idempotency keys alongside distributed caching.
Database Layer: Transaction Boundaries and Conflict Handling
In PostgreSQL, store the idempotency key in a dedicated `idempotency_records` table within the same ACID transaction as the primary business entity. If an identical transaction key is detected, execute an `ON CONFLICT (idempotency_key) DO UPDATE` or query the previously committed record. This guarantees that even if Redis restarts, the database provides a mathematically sound barrier against duplicates.
Need hands-on help with backend development services? Connect with Vamix engineers to architect, implement, or review your system.