Conventional DDR memory sits on a PCB, connected to the CPU by 64 or 128 wires clocked at 3–6 GHz. That geometry hits a wall: PCB traces are lossy, pin counts on the package are limited, and every extra GT/s per pin costs power quadratically. HBM sidesteps the whole problem by putting the DRAM on the same silicon substrate as the processor and connecting them over a massively wide, slow bus.

The stack. An HBM module is 4, 8, 12, or 16 DRAM dies stacked vertically, connected by Through-Silicon Vias (TSVs) — copper cylinders drilled straight through the thinned silicon (~50 µm thick). At the bottom of the stack sits a base logic die that handles the interface, refresh, and repair. The stack talks to the host over an interposer: a passive silicon slab underneath both the HBM stack and the compute die, patterned with fine-pitch metal wires (~1 µm pitch) that carry the interface bus.

The interface. One HBM3 stack exposes 1024 data pins organized as 16 independent channels of 64 bits each, running at ~6.4 Gb/s per pin. Total bandwidth per stack: 819 GB/s. Compare to DDR5-6400 at 51 GB/s per channel — HBM delivers ~16× the bandwidth per stack, and modern GPUs use 4–8 stacks.

Why it works. The interposer wires are short (millimeters, not centimeters), have low capacitance, and run at low voltage swings. So each pin runs slowly, but there are 1024 of them. Slow-and-wide beats fast-and-narrow on energy: HBM3 costs about 3.5 pJ/bit vs. ~15 pJ/bit for GDDR6. On a GPU pushing 5 TB/s of memory traffic, that's the difference between 17W and 75W just for memory I/O.

Real-world example. An NVIDIA H100 SXM has 5 HBM3 stacks (one dummy for mechanical balance, 4 active) delivering 3.35 TB/s across ~5120 data pins. The stacks sit millimeters from the GPU die on a CoWoS (Chip-on-Wafer-on-Substrate) interposer from TSMC. This is why H100 modules cost $30K+ — the interposer is bigger than the GPU die, and 2.5D packaging yields are ugly.

Rule of thumb. Bandwidth per stack ≈ channels × width × data rate / 8. HBM3: 16 × 64 × 6.4 / 8 = 819 GB/s. Double stacks, double bandwidth, but interposer area and TSV yield scale nonlinearly — 6 stacks is roughly the practical ceiling before packaging cost dominates.

The catch. HBM can't be socketed or upgraded — it's bonded to the interposer at manufacturing. A single dead TSV in the stack scraps a $500 memory module and often the entire package with it.