Validate Webhook Event Ordering and Duplicates

TL;DR

Send the same event ID more than once, deliver newer and older sequence numbers in reverse order, and race identical requests concurrently. The consumer passes only when each event has one business effect and the resource finishes at the newest accepted version.

Key Takeaways

• Treat duplicate and out-of-order webhook delivery as normal distributed-system behavior, not an exceptional edge case.
• Correlate deliveries with an immutable event ID and order domain changes with a per-resource sequence number.
• Claim an event ID atomically before applying its business effect so concurrent duplicates cannot both succeed.
• Reject or ignore stale events without returning a retryable error that creates a delivery storm.
• Assert final business state, side-effect counts, and audit records instead of checking HTTP status alone.
• Run permutations and concurrent duplicate tests in CI to expose timing-sensitive consumer defects.

To validate webhook event ordering and duplicates, deliver controlled events with immutable IDs and per-resource sequence numbers, then assert the final business state and the exact number of side effects. A 200 response is not enough. The receiver must prove that repeated events do not repeat work and late events do not roll state backward.

This tutorial builds that proof with a runnable Node.js harness. Read the Webhook API testing complete guide for 2026 if you want the full delivery lifecycle, then use this focused test to exercise ordering and idempotency.

You will model one subscription whose status changes through numbered events. The harness deliberately sends duplicates, reverses delivery order, and races requests. You can then replace the in-memory store with your database and replace the local sender with your real product trigger.

What You Will Build

You will build a small webhook consumer and automated test suite that can:

• accept events with an immutable id, resource ID, type, and sequence number;
• process the same event only once, even when two copies arrive together;
• prevent an older event from overwriting a newer resource state;
• record whether each delivery was applied, duplicated, or stale;
• keep external side effects, such as emails, at exactly one per accepted transition;
• test normal order, reverse order, sequential duplicates, and concurrent duplicates.

The example uses an in-memory store to keep the behavior visible. Its atomic claim and compare-and-set operations represent constraints or transactions that a production database must provide. A plain read followed by a write is intentionally not sufficient under concurrency.

Prerequisites

Use Node.js 24 LTS or a newer supported Node.js release. The example relies only on node:http, node:test, node:assert, and the built-in fetch API, so no third-party packages are required. Confirm the runtime:

node --version

Create a clean directory and a package file:

mkdir webhook-order-lab
cd webhook-order-lab
npm init -y
npm pkg set type=module
npm pkg set scripts.test="node --test"

You should also know the producer's ordering contract. Ask whether order is global or only guaranteed per resource, which field represents a version, whether sequence values can have gaps, and whether replays preserve the original event ID. If the producer offers only timestamps, clarify precision and tie-breaking rules before writing assertions.

Verify the setup: run npm test. Node should finish successfully with zero tests. Confirm that package.json contains "type": "module" before continuing.

Step 1: Define an Event Contract for Ordering and Identity

Create event.mjs. Validate the fields your correctness rules depend on before touching state:

const allowedTypes = new Set([
  'subscription.activated',
  'subscription.paused',
  'subscription.cancelled',
]);

export function parseEvent(value) {
  if (!value || typeof value !== 'object') throw new Error('invalid event');
  if (typeof value.id !== 'string' || value.id.length === 0) {
    throw new Error('invalid id');
  }
  if (typeof value.resourceId !== 'string' || value.resourceId.length === 0) {
    throw new Error('invalid resourceId');
  }
  if (!allowedTypes.has(value.type)) throw new Error('invalid type');
  if (!Number.isSafeInteger(value.sequence) || value.sequence < 1) {
    throw new Error('invalid sequence');
  }

  return Object.freeze({
    id: value.id,
    resourceId: value.resourceId,
    type: value.type,
    sequence: value.sequence,
    occurredAt: String(value.occurredAt),
  });
}

export function statusFor(type) {
  return type.slice(type.indexOf('.') + 1);
}

The event ID answers, "Have I handled this exact event?" The sequence answers, "Is this event newer than the state I have?" They solve different problems. Do not use the sequence as an idempotency key because two distinct events can concern one resource version, and do not use the event ID to infer order because IDs are often random.

Validate identity at the producer boundary too. If a replay creates a new event ID, it is not detectable as the same delivery unless the payload contains another stable idempotency key. Document that limitation instead of hiding it in the test.

Verify the step: run the following command. Expect an object ending in sequence: 2; changing 2 to 0 must throw invalid sequence.

node --input-type=module -e "import {parseEvent} from './event.mjs'; console.log(parseEvent({id:'evt-2',resourceId:'sub-7',type:'subscription.paused',sequence:2,occurredAt:'2026-07-15T10:00:00Z'}))"

Step 2: Build Atomic Idempotency and Version Storage

Create store.mjs. JavaScript executes each synchronous method without interleaving, so these methods provide atomic operations for this single-process lab:

export function createStore() {
  const claimedEventIds = new Set();
  const resources = new Map();
  const effects = [];
  const audit = [];

  return {
    claim(eventId) {
      if (claimedEventIds.has(eventId)) return false;
      claimedEventIds.add(eventId);
      return true;
    },

    applyIfNewer(event, status) {
      const current = resources.get(event.resourceId);
      if (current && current.sequence >= event.sequence) return false;
      resources.set(event.resourceId, { status, sequence: event.sequence });
      return true;
    },

    addEffect(effect) {
      effects.push(Object.freeze({ ...effect }));
    },

    addAudit(entry) {
      audit.push(Object.freeze({ ...entry }));
    },

    resource(id) { return resources.get(id); },
    effects() { return [...effects]; },
    audit() { return [...audit]; },
  };
}

In production, back claim with a unique constraint on the producer and event ID, for example (provider, event_id). Back applyIfNewer with a transaction or conditional update such as UPDATE ... WHERE last_sequence < incoming_sequence. If the idempotency record, state update, and outbox record are separate uncoordinated writes, a crash can still create partial processing.

The in-memory model keeps claimed IDs forever. A production retention period must exceed the provider's maximum automatic retry and manual replay windows. Expiring keys too early makes an old replay look new. Retaining every full payload forever may create privacy and storage problems, so keep only the minimum evidence your policy requires.

Verify the step: run this command and expect true false, proving the same ID can be claimed only once.

node --input-type=module -e "import {createStore} from './store.mjs'; const s=createStore(); console.log(s.claim('evt-1'),s.claim('evt-1'))"

Step 3: Implement the Webhook Consumer

Create consumer.mjs. The consumer classifies each valid delivery as applied, duplicate, or stale:

import { parseEvent, statusFor } from './event.mjs';

export async function consume(rawEvent, store) {
  const event = parseEvent(rawEvent);

  if (!store.claim(event.id)) {
    store.addAudit({ eventId: event.id, outcome: 'duplicate' });
    return { outcome: 'duplicate' };
  }

  const applied = store.applyIfNewer(event, statusFor(event.type));
  if (!applied) {
    store.addAudit({ eventId: event.id, outcome: 'stale' });
    return { outcome: 'stale' };
  }

  store.addEffect({
    kind: 'status-notification',
    eventId: event.id,
    resourceId: event.resourceId,
  });
  store.addAudit({ eventId: event.id, outcome: 'applied' });
  return { outcome: 'applied' };
}

A stale delivery returns a successful business outcome because retrying the same old event cannot make it newer. The HTTP adapter added next will translate all three accepted outcomes to 200. Invalid input remains a client error. The exact status contract can differ, but a permanent stale condition should not trigger endless retries.

Notice that a stale event is still claimed. If it arrives again, the result becomes duplicate, but neither delivery changes state. The audit record preserves both observations. In a production transaction, claim the event, conditionally change state, and enqueue the side effect together. Deliver the side effect through an outbox worker so a crash cannot lose it after committing state.

Verify the step: expect { outcome: 'applied' } followed by { outcome: 'duplicate' }, one effect, and one resource at sequence 1.

node --input-type=module -e "import {createStore} from './store.mjs'; import {consume} from './consumer.mjs'; const s=createStore(); const e={id:'evt-1',resourceId:'sub-7',type:'subscription.activated',sequence:1,occurredAt:'2026-07-15T09:00:00Z'}; console.log(await consume(e,s),await consume(e,s),s.effects().length,s.resource('sub-7'))"

Step 4: Expose a Real HTTP Receiver

Create server.mjs so the tests exercise JSON parsing, requests, and responses rather than calling only a function:

import http from 'node:http';
import { consume } from './consumer.mjs';

export function createWebhookServer(store) {
  const server = http.createServer(async (req, res) => {
    if (req.method !== 'POST' || req.url !== '/webhooks/subscriptions') {
      res.writeHead(404).end();
      return;
    }

    try {
      const chunks = [];
      for await (const chunk of req) chunks.push(chunk);
      const raw = JSON.parse(Buffer.concat(chunks).toString('utf8'));
      const result = await consume(raw, store);
      res.writeHead(200, { 'content-type': 'application/json' });
      res.end(JSON.stringify(result));
    } catch (error) {
      res.writeHead(400, { 'content-type': 'application/json' });
      res.end(JSON.stringify({ error: error.message }));
    }
  });

  return {
    async listen() {
      await new Promise(resolve => server.listen(0, '127.0.0.1', resolve));
      const address = server.address();
      if (!address || typeof address === 'string') throw new Error('no port');
      return `http://127.0.0.1:${address.port}/webhooks/subscriptions`;
    },
    async close() {
      if (!server.listening) return;
      await new Promise((resolve, reject) =>
        server.close(error => error ? reject(error) : resolve())
      );
    },
  };
}

Listening on port zero asks the operating system for a free port, which makes parallel CI jobs less likely to collide. The adapter reads the raw request completely before parsing it. In a signed webhook system, preserve those exact bytes for verification before JSON.parse; follow the step-by-step webhook signature verification tutorial for that layer.

Do not treat every thrown error as 400 in a production service. Validation errors are permanent client failures, while database outages are server failures that may deserve retry. This compact adapter is deliberately limited to the lab's expected errors.

Verify the step: start the server from a short script or continue to the automated test in Step 5. A POST containing a valid event must return 200 and an applied outcome. A request with sequence zero must return 400.

Step 5: Test Normal Order and Sequential Duplicates

Create webhook-order.test.mjs with shared helpers and the first two tests:

import test from 'node:test';
import assert from 'node:assert/strict';
import { createStore } from './store.mjs';
import { createWebhookServer } from './server.mjs';

const event = (id, sequence, type) => ({
  id, sequence, type, resourceId: 'sub-7',
  occurredAt: `2026-07-15T10:00:0${sequence}Z`,
});

async function fixture(t) {
  const store = createStore();
  const server = createWebhookServer(store);
  const url = await server.listen();
  t.after(() => server.close());
  const send = payload => fetch(url, {
    method: 'POST',
    headers: { 'content-type': 'application/json' },
    body: JSON.stringify(payload),
  }).then(async response => ({
    status: response.status,
    body: await response.json(),
  }));
  return { store, send };
}

test('applies events in ascending order', async t => {
  const { store, send } = await fixture(t);
  await send(event('evt-1', 1, 'subscription.activated'));
  await send(event('evt-2', 2, 'subscription.paused'));

  assert.deepEqual(store.resource('sub-7'), { status: 'paused', sequence: 2 });
  assert.equal(store.effects().length, 2);
});

test('applies a repeated event only once', async t => {
  const { store, send } = await fixture(t);
  const payload = event('evt-1', 1, 'subscription.activated');
  const first = await send(payload);
  const second = await send(payload);

  assert.equal(first.body.outcome, 'applied');
  assert.equal(second.body.outcome, 'duplicate');
  assert.equal(store.effects().length, 1);
  assert.deepEqual(store.resource('sub-7'), { status: 'activated', sequence: 1 });
});

These assertions inspect what matters to users: final state and side-effect count. A test that checks only two successful HTTP responses would pass even if two notifications were sent. Keep transport assertions, but never substitute them for business invariants.

Verify the step: run npm test. Both tests must pass. Temporarily remove the claim check from consume and rerun; the duplicate test must fail because the effects count becomes two. Restore the check before continuing.

Step 6: Validate Webhook Event Ordering and Duplicates in Reverse Order

Append this test to webhook-order.test.mjs:

test('does not let a late older event reverse state', async t => {
  const { store, send } = await fixture(t);
  const newer = await send(event('evt-2', 2, 'subscription.paused'));
  const older = await send(event('evt-1', 1, 'subscription.activated'));

  assert.equal(newer.body.outcome, 'applied');
  assert.equal(older.body.outcome, 'stale');
  assert.deepEqual(store.resource('sub-7'), { status: 'paused', sequence: 2 });
  assert.equal(store.effects().length, 1);
  assert.deepEqual(
    store.audit().map(entry => entry.outcome),
    ['applied', 'stale'],
  );
});

test('treats a different event at the same sequence as stale', async t => {
  const { store, send } = await fixture(t);
  await send(event('evt-a', 2, 'subscription.paused'));
  const collision = await send(event('evt-b', 2, 'subscription.cancelled'));

  assert.equal(collision.body.outcome, 'stale');
  assert.deepEqual(store.resource('sub-7'), { status: 'paused', sequence: 2 });
  assert.equal(store.effects().length, 1);
});

The first test simulates a common queue race: sequence 2 arrives before sequence 1. The consumer accepts the newest known state and acknowledges the late older event without applying it. The second test exposes an upstream contract violation or competing events at one version. It chooses first-writer-wins because current.sequence >= event.sequence rejects equality. Your domain might need a deterministic tie-breaker, but it must be specified before the test.

Do not sort received webhooks in the receiver and assume the problem is solved. Waiting for missing sequence values can block forever because providers may filter event types, lose deliveries after retention, or intentionally expose a non-contiguous version. Prefer a state-fetch API when exact reconstruction is required: treat the webhook as a notification, then fetch the resource's authoritative current representation.

Verify the step: run npm test. Four tests must pass. Change >= to > in applyIfNewer; the equal-sequence test must fail, demonstrating that the collision rule is enforced.

Step 7: Race Concurrent Duplicates and Run Permutations

Append two stronger tests:

test('claims concurrent duplicate deliveries atomically', async t => {
  const { store, send } = await fixture(t);
  const payload = event('evt-race', 1, 'subscription.activated');

  const results = await Promise.all(
    Array.from({ length: 20 }, () => send(payload)),
  );

  const outcomes = results.map(result => result.body.outcome);
  assert.equal(outcomes.filter(value => value === 'applied').length, 1);
  assert.equal(outcomes.filter(value => value === 'duplicate').length, 19);
  assert.equal(store.effects().length, 1);
});

test('every delivery permutation finishes at the newest state', async t => {
  const sequences = [
    [1, 2, 3], [1, 3, 2], [2, 1, 3],
    [2, 3, 1], [3, 1, 2], [3, 2, 1],
  ];

  for (const order of sequences) {
    const store = createStore();
    for (const number of order) {
      await import('./consumer.mjs').then(({ consume }) =>
        consume(event(`evt-${number}`, number, [
          '', 'subscription.activated',
          'subscription.paused', 'subscription.cancelled',
        ][number]), store)
      );
    }
    assert.deepEqual(store.resource('sub-7'), {
      status: 'cancelled', sequence: 3,
    });
  }
});

Promise.all starts requests close together, but it does not prove every possible machine-level interleaving. The value of this test is that it breaks implementations that perform asynchronous read-then-write idempotency checks without a unique constraint. Repeat it against the real database and use enough parallel workers to cross application instances.

The permutation test checks an invariant rather than one path: after all unique events are observed, the final resource must represent the greatest sequence. For larger histories, generate randomized permutations with a fixed seed and print the seed on failure. Keep a few explicit cases because they are easier to understand in code review.

If the producer retries after a timeout, combine these assertions with the webhook retry and backoff testing guide. A receiver can commit its transaction and lose the response, which makes a duplicate retry both legitimate and unavoidable.

Verify the step: run npm test repeatedly, or use node --test --test-reporter=spec. All six tests must pass, every run must report exactly one effect for the race case, and the permutation test must always finish at sequence 3.

Troubleshooting

Both concurrent requests apply -> the implementation probably reads the idempotency table before either request writes it. Add a database unique constraint and insert the claim inside the same transaction as the state change. Catch only the specific uniqueness conflict and classify it as a duplicate.

An older event overwrites a newer state -> the version check is happening in application memory or outside the update transaction. Use a conditional update guarded by the stored sequence, then check the affected-row count. Locking every resource can work, but conditional writes often express the invariant more directly.

Valid events are marked stale after a deployment -> verify that sequence scope is per resource and that the stored value belongs to the same producer and environment. Test data from staging, multiple tenants, or a reset sandbox can collide if the key omits provider, account, or environment.

The provider sends no sequence number -> do not invent order from arrival time. Use an authoritative resource version, a monotonic domain field, or fetch current state from the provider. Timestamps are acceptable only when their precision, clock source, and tie-break behavior are documented.

Duplicates return errors and keep retrying -> return the documented success response after proving the original event was committed. If the first attempt is still in progress, coordinate on the idempotency record or return a response consistent with the provider's retry contract.

Replay tests fail signature checks -> preserve the exact body bytes, headers, and supported timestamp window. Use the local webhook payload replay tutorial to capture reproducible fixtures without silently changing serialization.

Where To Go Next

Return to the complete Webhook API testing guide to connect ordering tests with security, schema validation, retry behavior, observability, and release coverage. Ordering correctness is strongest when the producer contract, consumer transaction, and operational evidence agree.

Continue with these focused tutorials:

• Test webhook retry and backoff behavior to prove why duplicates occur and when delivery attempts stop.
• Test webhook signature verification step by step to authenticate exact request bytes before claiming an event ID.
• Replay webhook payloads during local testing to reproduce production order and duplicate failures safely.
• Test API idempotency behavior to apply the same one-effect invariant to synchronous APIs.

Adapt the harness in layers. First replace the store with the real repository, then start multiple consumer instances, then trigger events through the producer. Keep deterministic tests in CI because they identify broken invariants quickly.

Interview Questions and Answers

Q: Why can webhooks arrive out of order?

Queues, partitions, retries, multiple workers, and different processing times can reorder delivery. Even when a provider preserves order on one queue, a failed event can be retried after a newer event succeeds. A consumer should depend only on an explicitly documented ordering scope.

Q: How do you prevent duplicate webhook processing?

Use the provider's immutable event ID as an idempotency key and claim it atomically with a database unique constraint. Commit the claim, business update, and outbox record in one transaction. Return success for an already committed duplicate without repeating the effect.

Q: What is the difference between duplicate detection and ordering control?

Duplicate detection stops the same event ID from running twice. Ordering control stops a distinct but older event from reversing newer state. A robust consumer usually needs both an event ID and a per-resource version or sequence.

Q: How would you test concurrent duplicate deliveries?

Send many identical requests together through multiple application instances, then assert exactly one committed business effect and one idempotency record. Repeat against the real database because a single-process mock cannot reproduce every transaction race. Preserve audit evidence for all rejected duplicates.

Q: Should an out-of-order stale webhook return an error?

Usually it should be acknowledged after validation because retrying cannot make it current. The exact response follows the provider contract, but permanent staleness should not create a retry storm. Record the stale outcome for diagnosis.

Q: Can a timestamp replace a sequence number?

Only when the timestamp's authority, precision, and tie-breaking rule are reliable for the ordering scope. Arrival time is not a valid substitute. Prefer a monotonic resource version or fetch the provider's current state.

Q: What should a webhook ordering test assert?

Assert the newest final resource version, one effect per accepted event, no effect for duplicates or stale events, and an auditable outcome for every delivery. Also exercise reverse order, equal-version collisions, concurrent duplicates, and retry-after-commit behavior.

Best Practices to Validate Webhook Event Ordering and Duplicates

• Scope sequence numbers by provider, tenant, and resource instead of assuming a global order.
• Enforce idempotency with a unique database constraint, not a cache-only check.
• Make the state update conditional on the incoming version being greater.
• Commit the event claim, state change, and outbox record atomically.
• Acknowledge committed duplicates and permanent stale events according to the provider contract.
• Preserve compact audit evidence: event ID, resource ID, sequence, outcome, and processing time.
• Test every small permutation explicitly and add seeded randomized histories for broader coverage.
• Run concurrency tests across real service instances and the production database engine.

Avoid assuming exactly-once delivery. Networks cannot tell a sender whether the receiver committed work immediately before a response disappeared. Design for at-least-once delivery, then make the consumer's business effect effectively once through idempotency and transactions.

Also avoid holding events forever while waiting for a missing sequence. Gaps may be legitimate, and blocked queues can delay unrelated resources. If a complete ordered history is mandatory, use the API contract testing guide to define a bounded reconciliation process or retrieve authoritative state.

Conclusion

To validate webhook event ordering and duplicates, test business invariants under sequential repetition, reverse order, equal-version collisions, and concurrent races. The event ID prevents repeated processing, while a per-resource sequence prevents stale state changes. Atomic storage makes both guarantees survive concurrency.

Run the included Node.js suite first, then move the same assertions onto your real database and producer trigger. Finish by replaying a commit-then-timeout scenario. If the newest state remains stable and each accepted event produces one effect, your consumer is prepared for the delivery behavior real webhook systems create.

Interview Questions and Answers

Frequently Asked Questions

Related Guides

• Playwright drag and drop: Examples and Best Practices
• Playwright mock date and time: Examples and Best Practices
• Playwright popup and new tab handling: Examples and Best Practices
• Playwright tag and grep filters: Examples and Best Practices
• Test Webhook Retry and Backoff Behavior
• Testing webhooks: A Practical Guide (2026)