Design Docs Are All You Need: An AI-native Machine-Learning Performance Tool

2026-09-07

Authors: Samuel Kushnir, Kimia Noorbakhsh, Kavya Sreedhar, Liqun Cheng

ArXiv: 2609.05364v1

PDF: Download PDF

Imagine you're a hardware architect trying to answer questions like "if we add more memory bandwidth to this chip, how much faster will Llama run?" To answer that, you need a performance model — a symbolic simulator that estimates how long an ML workload will take on hypothetical hardware. These models are essential for designing next-gen accelerators, but they're notoriously painful to maintain. Every time a new model architecture drops (mixture-of-experts, new attention variants, novel parallelism schemes), the assumptions baked into your framework break, and engineers spend weeks refactoring.

The authors of SMART make a provocative argument: stop maintaining the code. Maintain the design doc, and let AI regenerate the code.

Their reasoning is that AI coding agents have crossed a threshold. Regenerating an entire performance-modeling library from a well-written specification is now cheaper than paying down the accumulated tech debt of incremental patches. Instead of engineers wrestling with a decade of layered abstractions, they update a natural-language design document describing what the tool should do, and an AI agent produces the implementation.

Key ideas in the paper:

Why is this interesting beyond ML performance modeling? It's a concrete case study in a broader shift: when generation becomes cheap and reliable, the economics of software maintenance invert. Codebases that used to accrue technical debt become disposable artifacts. The valuable, durable thing is the specification. If this pattern holds, we may see whole classes of internal tooling — simulators, DSLs, glue code — move to this "spec-first, regenerate on demand" model.

The obvious tensions: how do you version and test regenerated code? How do you trust that two regenerations produce equivalent behavior? The paper is a proof-of-concept in a domain where correctness is checkable against ground-truth hardware measurements, which makes it a good testbed for the idea.

Why it matters: This paper is an early flag that capable coding agents may invert traditional software economics — making design docs, not code, the maintained artifact for fast-churning internal tooling.

All newsletters