Idempotency and deduplication, in depth

Money events must be counted exactly once. Networks retry postbacks, your app retries on timeouts, and the delivery worker retries failed dispatches. XTRACKER is built so none of that double-counts a deposit. Here's the mechanism.

The dedup key

Every incoming event computes a deterministic key:

dedup_key = sha256(space_id | uid | canonical | discriminator)

The row is inserted with ON CONFLICT DO NOTHING on that key. A second event with the same key returns "status": "duplicate" and changes nothing.

The discriminator is where you control behaviour:

• if you pass an eid, the discriminator is the eid;
• otherwise it falls back to occurred_at_ms.

eid — time-independent dedup

eid is your unique id for a conversion — typically the network's trade/transaction id. Because it's the discriminator, a postback that re-fires at a different time still dedups:

…&event=deposit&uid=abc&value=25&eid=tx-42   (10:00:00) → queued
…&event=deposit&uid=abc&value=25&eid=tx-42   (10:00:07) → duplicate ✓

This is the safe default for registration and deposit. Always send it when the source has a unique id.

occurred_at_ms — pinned-time dedup

Without an eid, two events dedup only if they share the same occurred_at_ms. That's fine for a client you control: pin the timestamp once per logical event and reuse it on retries.

ts = int(time.time() * 1000)        # compute ONCE
await send_event("deposit", uid=uid, value="25", occurred_at_ms=ts)
# ... on a retry, reuse the SAME ts → dedups

The supplied client does this for you across its internal retries. If you omit the time entirely on a postback, the server stamps receive-time — fine for a single clean fire, but eid is stronger.

End-to-end safety

Dedup at ingest is only half the story. Delivery is idempotent too:

1. The event is written to event_inbox and committed before anything is published — a crash can't lose an accepted event.
2. Fan-out creates one event_outbox row per destination.
3. The worker retries failed rows with backoff. Your webhook receives a stable event_id so you can dedup on your side too.

So a conversion survives a Redis blip, a worker restart, or a destination outage — and still lands once.

Checklist

• Send eid for every registration and deposit.
• If you can't, pin occurred_at_ms and reuse it on retries.
• Dedup webhooks on event_id in your handler.
• Treat "duplicate" as success — it means the event was already recorded.

Next

• Canonical events — what carries value and needs an eid.
• Webhook payload reference — the event_id you dedup on.