FlashAttention is one of those algorithms that quietly rewrote what's possible in modern LLM training. It made attention IO-aware — recognizing that the bottleneck in transformer inference isn't FLOPs but memory bandwidth between HBM and on-chip SRAM. Every serious ML engineer has heard of it. Very few can actually explain the math behind why the online softmax trick works, or how the block-wise recomputation preserves numerical equivalence with the naive attention formulation.

This tutorial promises to fill exactly that gap. The URL — "learning-flashattention-the-hard-way-part-1" — signals the right pedagogical instinct. Not a hand-wavy overview. Not a code walkthrough that skips the derivations. The algebraic foundations. That means:

• The log-sum-exp trick and its numerical stability properties
• How the online softmax maintains a running normalizer that can be corrected after seeing each new block, without a second pass over the data
• The associativity properties that make tiled matrix multiplication mathematically equivalent to the full computation
• Why the backward pass can recompute activations instead of storing them — the memory-compute tradeoff that unlocks longer contexts

Content like this is genuinely rare. Tri Dao's original papers are dense, and most secondary explanations either stay too surface-level ("it tiles the attention matrix!") or dive straight into CUDA kernels without motivating why the algorithm is correct. A tutorial that patiently builds the algebra from first principles is exactly what practitioners moving from "user of FlashAttention" to "implementer of custom attention variants" need.

The value multiplies when you consider the current landscape: ring attention, sliding-window attention, PagedAttention, FlashAttention-3 — all descendants of the same core algebraic insight. Understanding the foundations means understanding the whole family tree. It's the difference between memorizing a formula and being able to derive the next variant yourself.

A Part 1 suggests a series, which is even better. This is the kind of long-form technical writing that ages well — the algebra won't change even as the kernels do. If riftstack.ai keeps this quality up, it's worth bookmarking regardless of whether individual posts hit the front page.