You wrote counter = counter + 1 and your tests pass. Then production runs it under load and the count is off by 12%. Welcome to the read-modify-write race, the bug that every junior engineer ships and every senior engineer has scars from.

The pattern is deceptively simple. Three steps that look atomic but aren't:

• Read the current value
• Modify it in your application code
• Write the new value back

Between the read and the write, another process can read the same old value, compute its own update, and write first. Your write then overwrites theirs. The update is silently lost.

Concrete example: A video platform tracks view counts. The handler does SELECT views FROM videos WHERE id=42, gets 1000, then UPDATE videos SET views=1001 WHERE id=42. Two requests arrive simultaneously. Both read 1000. Both write 1001. You served two views; you recorded one.

The rule of thumb: If N concurrent writers each do a read-modify-write on the same row at request rate R, you'll lose roughly N × R × t updates per second, where t is the duration between read and write. At 100 req/s with a 10ms gap and 5 workers, that's ~5 lost updates per second — 432,000 per day. The bug scales with success.

Four fixes, in order of preference:

• Atomic operation: UPDATE videos SET views = views + 1 WHERE id = 42. The database does the read-modify-write inside a single row lock. Free, fast, correct. Use this whenever the operation can be expressed as a delta.
• Optimistic locking: Add a version column. Read it, then UPDATE ... WHERE version = $old_version. If zero rows match, someone beat you — retry. Good when computation between read and write is complex but conflicts are rare.
• Pessimistic locking: SELECT ... FOR UPDATE locks the row until commit. Correct, but other writers block. Use when conflicts are frequent and retries would be wasteful.
• Compare-and-swap (CAS): For Redis, use WATCH/MULTI/EXEC or Lua scripts. For in-memory, use AtomicInteger.compareAndSet. Same idea as optimistic locking, lower-level primitive.

What to watch for in code review: any pattern of get followed by set on shared state. Inventory decrements, balance updates, "last login" timestamps, idempotency-key checks, rate-limit counters. The get+set idiom is the smell. If you see it without a lock, version check, or atomic operation, flag it.

The brutal part: this bug rarely shows up in testing because tests are single-threaded. It only surfaces under real concurrency, and even then only intermittently. That's why it ships.