This deduplicates incoming events by ID before pushing them into a processing pipeline. The unit tests are green. The production logs are full of "Dropping duplicate" messages for events the upstream system swears it sent only once.

const seen = new Set();

function ingest(rawJson) {
  const event = JSON.parse(rawJson);
  if (seen.has(event.id)) {
    console.log(`Dropping duplicate ${event.id}`);
    return;
  }
  seen.add(event.id);
  pipeline.push(event);
}

// Unit tests — small IDs, all green:
ingest('{"id": 1, "msg": "hi"}');
ingest('{"id": 2, "msg": "yo"}');

// Production — upstream emits 64-bit Snowflake IDs:
ingest('{"id": 9007199254740993, "msg": "first"}');   // accepted
ingest('{"id": 9007199254740995, "msg": "second"}');  // accepted
ingest('{"id": 9007199254740997, "msg": "third"}');   // "Dropping duplicate 9007199254740996"?!

The Bug

The third event has ID ...997, but the log says ...996. That's not a typo — that's the bug confessing itself.

JavaScript has exactly one numeric type for JSON.parse: IEEE-754 double-precision float. The largest integer it can represent exactly is Number.MAX_SAFE_INTEGER = 2⁵³−1 = 9007199254740991. Past that, only even integers are representable; odd integers round to the nearest even neighbor.

Walk through what JSON.parse actually produces:

• 9007199254740993 → stored as 9007199254740992
• 9007199254740995 → stored as 9007199254740996
• 9007199254740997 → stored as 9007199254740996 ← collides with the previous one

The set sees a "duplicate" because two distinct upstream IDs landed on the same float. The log even prints the rounded value, not the original — the original is gone before your code ever touches it. Snowflake IDs, Twitter status IDs, Discord IDs, and most database BIGINT primary keys all live happily above 2⁵³, so this hits any system that ingests them.

Tests pass because nobody seeds the fixtures with 16-digit IDs. The truncation happens inside the JSON parser, so even typeof event.id === "number" and Number.isInteger(event.id) return true on the corrupted value. There is no error, no warning, no obvious smell — just silently merged events.

The Fix

You cannot recover precision after JSON.parse has touched the number — the original digits never made it into memory as a JS value. You have to intercept the raw text. Two real options:

Option 1: Send IDs as strings. This is what Twitter learned to do (the id_str field). If you control the producer, this is the cleanest fix:

// Upstream emits: {"id": "9007199254740997", "msg": "third"}
// Consumer treats event.id as an opaque string key. Set dedup still works.

Option 2: Parse to BigInt. If you don't control the producer, use a library that handles big integers, or write a reviver that catches them before Number swallows them:

import JSONbig from 'json-bigint';
const parse = JSONbig({ useNativeBigInt: true }).parse;

function ingest(rawJson) {
  const event = parse(rawJson);   // event.id is now a BigInt
  if (seen.has(event.id)) { ... } // Set works with BigInt keys
  ...
}

A naive JSON.parse(rawJson, reviver) won't save you — by the time the reviver runs, the value is already a lossy Number. The fix has to live in the tokenizer.