The Reconciliation Loop Anti-Pattern: When Continuous Convergence Fights Itself

2026-08-23

Reconciliation loops are the beating heart of GitOps and Kubernetes: read desired state, read actual state, compute the diff, apply corrections, repeat forever. It's a beautiful model — until you have two loops reconciling the same resource with different opinions. Then you get a fight that never ends, burns CPU, and makes your dashboards flicker.

The pattern shows up whenever ownership is ambiguous. Classic scenarios:

Real example: A team ran Argo CD syncing a Deployment manifest with replicas: 2, and HPA scaling between 2 and 20. During a traffic spike HPA scaled to 15. Argo's next sync (every 3 minutes) scaled it back to 2. Latency spiked, HPA scaled up again, Argo scaled down. The oscillation lasted 40 minutes before someone noticed pods being killed mid-request. Fix: remove replicas from the manifest entirely, or add ignoreDifferences for that field in the Argo Application spec. Ownership must be exclusive per field, not per resource.

Rule of thumb: for any writable field on any resource, exactly one controller owns it. Zero owners means drift accumulates; two or more owners means a fight. Kubernetes 1.22+ made this explicit with Server-Side Apply field ownership — each manager claims specific fields, and conflicting writes error out instead of silently thrashing. Use it.

How to detect the fight: if the resourceVersion of an object increments faster than any human is touching it — say, more than 10 changes per hour with no deployment — two controllers are almost certainly wrestling. Check metadata.managedFields to see who's writing what.

Design defenses:

See it in action: Check out Codependency - Narcissist True Colours by Tim Fletcher to see this theory applied.
Key Takeaway: Reconciliation loops only converge when exactly one controller owns each field — anything else is a fistfight scheduled to run forever.

All newsletters