x86 long mode setup issue

2026-08-16

Stack Overflow: View Question

Tags: assembly, x86, x86-64, bootloader

Score: 2 | Views: 150

The asker is writing a second-stage bootloader in NASM that must transition the CPU from 32-bit protected mode into 64-bit long mode. The specific failure point: the block that enables paging — from mov eax, cr0 through mov cr0, eax — triple-faults QEMU and reboots the machine.

This is one of the most notoriously fiddly transitions in all of x86 osdev, because long mode has a strict activation sequence and any single mis-ordered step produces the same silent triple fault.

Why it's hard: Long mode activation is a three-way handshake between CR0, CR4, and the EFER MSR, and it must be performed in the correct order:

  1. Disable paging in CR0 (if it was on).
  2. Build valid 4-level page tables (PML4 → PDPT → PD → PT), identity-mapping at least the code that will run immediately after the switch.
  3. Load cr3 with the physical address of the PML4.
  4. Set CR4.PAE (bit 5) — mandatory, long mode uses PAE-style entries.
  5. Set EFER.LME (bit 8) via wrmsr to MSR 0xC0000080.
  6. Set CR0.PG (bit 31) — this is the atomic switch that activates long mode.
  7. Far-jump through a GDT entry with the L bit (bit 53) set to reach 64-bit code.

Sketch of the diagnosis: The crash "on the paging block" almost always means one of:

Debugging approach I'd recommend to the asker: Run QEMU with -d int,cpu_reset -no-reboot -no-shutdown and attach GDB via -s -S. Single-step across the mov cr0, eax. The register dump on the triple fault reveals which invariant failed — a bad CR3, PAE off, or a fetch fault at RIP. Also verify the page tables by hand in the monitor with info mem after loading CR3 but before enabling PG.

Gotcha worth calling out: The far jump must immediately follow enabling PG — no data accesses in between — because your CS is still a 32-bit selector, and the CPU is now in a "compatibility" limbo where anything but that jump is undefined.

The challenge: Long mode entry is an all-or-nothing ritual where PAE, LME, PG, page tables, and a 64-bit GDT descriptor must all be correct simultaneously — one wrong bit gives you the same silent triple fault as any other.

All newsletters