Imagine you're running a massive kitchen where hundreds of chefs need to hand ingredients to each other in a precise sequence to produce a complicated meal. The traditional way to organize this is to give every chef a copy of the recipe and hope they coordinate in real time — each one asking, "What do I do next? Where's my ingredient?" That's basically how modern GPUs work: thousands of threads independently fetch data and instructions, with a lot of overhead spent figuring out what to do next.

Microsoft's new Maia 200 chip takes a different approach. Instead of thread-centric computing, it uses what the authors call a Software Defined Locally Accessed Dataflow Architecture (SDLA). Think of it as pre-choreographing the entire kitchen: the compiler works out ahead of time exactly which data moves where, when, and through which specialized memory. The hardware then just executes that dance without wasting cycles asking questions.

The raw numbers are impressive:

• 10,145 teraflops at FP4 precision (the low-precision format increasingly used to train and serve large models)
• 5,072 teraflops at FP8 precision
• 750 watts total power draw
• 7 TB/s of high-bandwidth memory throughput

For context, that FP8 number is competitive with NVIDIA's flagship datacenter chips, and the FP4 figure is very high. But the more interesting story isn't the peak numbers — it's the architectural bet. Maia 200 argues that as AI models get bigger, the bottleneck stops being "how fast can we do multiplications?" and becomes "how fast can we move data to the right place at the right time?" By making data movement a first-class thing the software controls explicitly — rather than an emergent property of thousands of threads competing for memory — you get better efficiency per watt and better scalability.

The tradeoff is that this puts enormous pressure on the compiler and software stack. If the hardware isn't figuring things out dynamically, the toolchain has to get the choreography right in advance. That's hard, but it's a bet that pays off if your workload is predictable — and large language model training and inference are, in fact, very predictable. You know the shape of every matrix multiplication in advance.

This is part of a broader trend of hyperscalers (Google's TPU, AWS's Trainium, Microsoft's Maia) building custom silicon tuned specifically to LLM workloads rather than paying NVIDIA margins for general-purpose GPUs.