Unable to Debug kernel with KGDB in vmware

2026-07-07

Stack Overflow: View Question

Tags: debugging, vmware, kgdb

Score: 0 | Views: 56

The asker has compiled a Linux kernel with CONFIG_KGDB enabled, flashed it onto a VMware VM, and wired up a virtual serial port as a socket pipe at /tmp/kgdb-socket. When they attach GDB from the host and issue target remote, they get "Ignoring packet error, continuing..." plus a warning about an unrecognized "timeout" item. Classic KGDB-over-serial pain.

This question is interesting because the failure mode is generic — "packet error" — but the root cause is almost always one of several very specific mismatches between three moving parts: the kernel's KGDB serial driver, the VMware virtual serial device, and GDB's remote protocol expectations. Each layer has its own baud, framing, and buffering assumptions.

Likely culprits, in order of probability:

Suggested approach: Start with the simplest topology — TCP serial in VMware pointing at telnet://localhost:8888, boot with console=ttyS0,115200 kgdboc=ttyS0,115200 kgdbwait, and connect via target remote localhost:8888. If that handshakes, layer in Unix sockets afterward. Also verify the kernel was built with CONFIG_KGDB_SERIAL_CONSOLE=y, not just CONFIG_KGDB=y.

Gotcha: VMware Workstation and ESXi handle serial pipes differently; the "yield CPU on poll" checkbox matters, and without it GDB packets get dropped under load. Also, if secure boot is on, kgdboc silently refuses to attach.

The challenge: KGDB failures manifest as a single generic error, but debugging them requires reasoning across four independent layers (kernel config, kernel cmdline, VMware serial plumbing, GDB protocol) that must all agree.

All newsletters