Python's re.sub Replacement Backreference Trap: The Redactor That Prints the Secret It Was Hiding

2026-09-10

This function redacts US-format SSNs in a body of text, letting the caller pick the placeholder string. It's been running quietly in a compliance pipeline for months.

import re

def redact_ssns(text: str, placeholder: str) -> str:
    """Replace SSN patterns with the caller-supplied placeholder."""
    return re.sub(r"(\d{3})-(\d{2})-(\d{4})", placeholder, text)

# The unit tests all use a hard-coded literal placeholder:
assert redact_ssns("call 123-45-6789", "REDACTED") == "call REDACTED"

# In production, the placeholder comes from a per-tenant config file
# so ops can customise it ("[hidden]", "***-**-****", etc.)
placeholder = load_tenant_config()["ssn_placeholder"]

# Tenant A: {"ssn_placeholder": "***-**-****"}
redact_ssns("SSN 123-45-6789", placeholder)
# → "SSN ***-**-****"    ✓

# Tenant B, whose YAML got auto-escaped somewhere upstream:
# {"ssn_placeholder": "[HIDDEN \\1]"}
redact_ssns("SSN 123-45-6789", placeholder)
# → "SSN [HIDDEN 123]"   ← the first three digits leak into the log

The Bug

The second argument to re.sub is not a literal string. It's a mini-template language. Sequences like \1, \2, and \g<name> are backreferences that get replaced with the corresponding captured group. Any other unknown backslash escape raises re.error.

Your pattern happens to capture the three SSN segments. When a placeholder contains \1 — because a tenant typed it, because a JSON round-trip turned \1 into an escape, because you built the replacement by concatenating another regex match — re.sub dutifully substitutes the captured digits. The redactor prints the very data it was meant to hide.

Two things make this hard to catch:

The fix is to bypass the template language entirely. re.sub accepts a callable as the replacement; whatever the callable returns is used verbatim, with no backreference expansion:

def redact_ssns(text: str, placeholder: str) -> str:
    return re.sub(r"(\d{3})-(\d{2})-(\d{4})", lambda _m: placeholder, text)

If you truly need a string replacement, escape it first — but note that re.escape is for patterns, not replacements. For replacements you must double every backslash yourself: placeholder.replace("\\", "\\\\"). The lambda form is safer because there's nothing to remember.

The general lesson: any API that takes a "string" but interprets escape sequences (re.sub, str.format, shell commands, SQL, HTML) is a lit fuse when the string comes from outside your test suite. If the API offers a "raw callback" alternative — a lambda, a parameterised query, a DOM-builder — reach for it before you reach for manual escaping.

Key Takeaway: re.sub's replacement string is a template with backreferences, not a literal — pass a lambda when the replacement is user-controlled, or your redactor will happily expand \1 into the secret it just matched.

All newsletters