Tal Hoffman's blog has a track record of producing clear, visual explanations of graphics programming and GPU architecture concepts. This post tackles one of the most fundamental — yet frequently misunderstood — distinctions in modern computing: what actually belongs on the CPU versus the GPU, and how work flows between them.

This matters more now than it has in years. The explosion of GPU computing driven by machine learning has pulled an enormous number of developers into GPU programming who didn't come up through the traditional graphics pipeline. Many of them carry CPU-centric mental models that lead to real performance pitfalls:

• Unnecessary round-trips — shuttling data back to the CPU for work that could stay on the GPU, stalling both processors while they wait on each other.
• Misplaced branching logic — writing divergent control flow that thrashes GPU warp/wavefront execution, when the work should have been restructured or offloaded to the CPU.
• Ignoring latency hiding — failing to overlap CPU preparation of the next frame or batch with GPU execution of the current one, leaving one processor idle while the other works.

A good treatment of CPU vs. GPU work decomposition covers the asymmetry between the two processors: CPUs optimized for low-latency sequential tasks with deep caches and branch prediction, GPUs optimized for high-throughput parallel tasks with massive register files and SIMD execution. The practical upshot is that choosing where work runs isn't just about "is this parallelizable?" — it's about data locality, synchronization costs, and pipeline bubbles.

For graphics programmers, this is bread and butter: culling on the CPU, draw call submission, command buffer recording, versus vertex shading, rasterization, and fragment work on the GPU. But the same principles apply directly to ML inference pipelines, compute shaders, and even database query engines that offload to GPUs. Understanding the boundary helps you design systems where both processors stay saturated rather than blocking on each other.

Hoffman's writing tends to be concise and diagram-heavy, which makes these architectural concepts accessible without dumbing them down. For anyone writing renderers, ML inference code, or heterogeneous compute pipelines, having a clean mental model of this CPU/GPU division pays dividends in every optimization decision you make.