Do Machines Struggle Where Humans Do? LLM and Human Comprehension of Obfuscated Code

2026-07-01

Authors: Jack Le, Anh H. N. Nguyen, Tien N. Nguyen

ArXiv: 2606.31725v1

PDF: Download PDF

Imagine you're handed a piece of code where every variable is named a1, b2, c3, the control flow has been scrambled into a knot, and dead branches have been sprinkled in to throw you off. That's code obfuscation: deliberately messing up code so it still runs correctly but becomes painful for a human to read. Malware authors use it to hide what their programs do. Companies use it to protect intellectual property. Reverse engineers hate it.

We already know a fair bit about how obfuscation breaks human understanding. But here's a fresh question: do large language models (LLMs) — the same AI systems that increasingly help us read, review, and reverse-engineer code — struggle in the same places humans do? Or do they fail in totally different ways?

This paper takes a recent human study on code comprehension and rerouts the same experiment through several LLMs. The researchers used a framework called the Block Model, which slices "understanding code" into four layers:

They ran LLMs through five escalating tiers of obfuscation and measured where comprehension broke down at each level.

The key finding: LLMs and humans don't fail in the same way. Humans tend to stumble first at the local, low-level details — once variable names and structure are scrambled, working memory gets overwhelmed and everything downstream collapses. LLMs, by contrast, are surprisingly resilient at the atom and block levels (they can chew through weird identifiers just fine) but tend to fall apart at the relational and macro layers — the "what is this program actually doing, end to end" reasoning that requires holding a mental model together across the whole file.

That mismatch matters for anyone leaning on AI to help with reverse engineering, malware analysis, or auditing minified/obfuscated code. It suggests LLMs are useful for the tedious surface-level cleanup work — renaming variables, un-tangling a function — but shouldn't be trusted alone for the big-picture "what does this malware do?" question. And it points to a real weakness: obfuscation techniques designed to defeat humans by exhausting local working memory may not fool an LLM, while techniques that shred cross-function relationships will.

Why it matters: As LLMs become standard tools for reverse engineering and malware analysis, knowing exactly where they fail — and that it's not where humans fail — reshapes both how we use them and how adversaries might design obfuscation to defeat them.

All newsletters