2026-07-20
Stack Overflow: View Question
Tags: assembly, x86-64, masm, calling-convention, abi
Score: 6 | Views: 169
The asker is porting a function originally written using GCC inline assembly into an MSVC x64 project. Because there's no intrinsic for ltr (Load Task Register), they've placed the routine in a standalone .asm file and are assembling with ml64.exe. They suspect MASM is generating code that violates the Windows x64 calling convention — placing arguments in the wrong registers.
Why this is interesting: The Windows x64 ABI is quite specific — the first four integer/pointer arguments go in RCX, RDX, R8, R9, with 32 bytes of "shadow space" reserved on the stack by the caller. Any asymmetry between what the C compiler emits at the call site and what the assembler emits inside the callee will silently corrupt state. The question probes whether ml64.exe's PROC directive with typed parameters actually honors this ABI, or whether it does something unexpected (like using fastcall-style register mapping that mirrors 32-bit conventions).
Approach:
PROC's parameter list entirely. MASM's high-level parameter syntax is a legacy holdover with limited x64 semantics. Write the function as a bare label and reference RCX/RDX/R8/R9 directly. This eliminates any translation layer MASM might apply.dumpbin /disasm asm.obj to verify exactly what bytes ml64 emitted, rather than trusting the source listing.cl /FAs. If the caller places the argument in RCX but MASM's PROC generates a prolog that reads from [RSP+8], that's the bug.ltr specifically, its operand is a 16-bit segment selector — declare it accordingly and load it via LTR CX after the caller passed it in RCX.Gotchas:
ltr is CPL=0 only. If this is user-mode code, it'll #GP regardless of ABI correctness — the asker should confirm they're in a kernel/hypervisor context.PROC with FRAME handling may add its own prolog that shifts RSP, changing offsets..pushreg/.setframe annotations will crash on any exception unwind, which can look like "wrong ABI" symptoms.C vs STDCALL in old code may still cause surprises.PROC abstraction actually respects the Windows x64 ABI, or whether hand-rolled register access is the only reliable path.
