From 9ea9019d29af694fd2737b40895bdfb4e3029b38 Mon Sep 17 00:00:00 2001 From: Anatol Belski Date: Wed, 29 Jul 2026 12:45:25 +0200 Subject: [PATCH] docs: Expand KDNET guide with NAT routing and SAC setup Add a NAT variant for debuggers on a different subnet or reached only through the host uplink. Source NAT suffices because KDNET is target initiated, so replies return along the conntrack entry with no inbound port forward. Note that KDNET needs a routable tap address from DHCP or a static SAC assignment, add SAC steps to enable it on a headless guest, and move busparams pinning into an optional section. Assisted-by: Claude:Opus-4.8 Signed-off-by: Anatol Belski --- docs/windows-kdnet-debugging.md | 108 +++++++++++++++++++++++++++----- 1 file changed, 94 insertions(+), 14 deletions(-) diff --git a/docs/windows-kdnet-debugging.md b/docs/windows-kdnet-debugging.md index 6a7873b34..9a3e75511 100644 --- a/docs/windows-kdnet-debugging.md +++ b/docs/windows-kdnet-debugging.md @@ -43,10 +43,90 @@ routing/firewalling must allow the chosen debug port -- in particular, open the UDP debug port inbound on the **debugger** host's firewall, otherwise the target's connection packets are dropped before WinDbg sees them. +### Routing the debug NIC with NAT for remote or cross subnet debuggers + +Bridging assumes the debug NIC can appear directly on a network the debugger +shares. That is not always possible or desirable. The debugger may live on a +different subnet reached only through the default route of the host, or the +host uplink may drop frames carrying the debuggee MAC. Many cloud and virtual +switch uplinks reject foreign MACs unless MAC address spoofing is explicitly +enabled. + +Because KDNET connections are always **target initiated**, source NAT on the +Cloud Hypervisor host is sufficient. The outbound UDP from the debuggee creates +a conntrack entry and the replies from the debugger return along it, so no +inbound port forward is needed on the Cloud Hypervisor host. Put the tap on its +own private subnet and masquerade it out the uplink. + +```bash +sudo ip addr add 192.168.250.1/24 dev chdbg0 +sudo ip link set chdbg0 up + +sudo sysctl -w net.ipv4.ip_forward=1 +sudo iptables -t nat -A POSTROUTING -s 192.168.250.0/24 -o eth0 -j MASQUERADE +sudo iptables -A FORWARD -i chdbg0 -o eth0 -j ACCEPT +sudo iptables -A FORWARD -i eth0 -o chdbg0 -m state --state RELATED,ESTABLISHED -j ACCEPT +sudo iptables -t mangle -A POSTROUTING -o chdbg0 -p udp -j CHECKSUM --checksum-fill + +sudo dnsmasq --port=0 --interface=chdbg0 --bind-interfaces \ + --dhcp-range=192.168.250.50,192.168.250.150,255.255.255.0,1h \ + --dhcp-option=option:router,192.168.250.1 --dhcp-authoritative +``` + +KDNET needs a routable address on the tap subnet, provided in one of two ways. +Either serve DHCP on the tap with the dnsmasq invocation above, or assign a +static address on the debug subnet from SAC as shown in the guest +configuration. A link local `169.254.x` address is not enough. The Linux +kernel refuses to forward it per RFC 3927, so those packets die on the tap +before reaching NAT. When using a static address, the dnsmasq line is not +needed. Set `hostip` to the debugger address, and the firewall on the debugger +must still allow the debug port inbound. + ## Guest configuration (Windows debuggee) -Identify the virtio-net adapter's PCI bus/device/function (KDNET selects the NIC -by `busparams`). The location of each adapter can be read with PowerShell: +Cloud Hypervisor exposes no VGA adapter, so with `--serial tty --console off` +the only built-in console is **SAC**, a minimal text menu over the serial line. +SAC is not a full shell. It can assign a NIC address with its `i` command and +open a plain `cmd` channel, which is enough to run `bcdedit`. The channel +prompts for the guest credentials, and once logged in it is an ordinary +Windows command prompt from which `powershell` can be started. PowerShell is +needed only for `busparams` discovery, so no RDP or SSH is required. See +[Windows Support](windows.md). + +### Enable KDNET from SAC + +Give the debug NIC a static address on the debug subnet, where `10` is the +device index printed by bare `i`, then open a command channel. This is the +alternative to serving DHCP on the tap. Either one gives KDNET a routable +address. + +``` +SAC>i +SAC>i 10 192.168.250.2 255.255.255.0 192.168.250.1 +SAC>cmd +SAC>ch -si 1 +``` + +In that channel, enable KDNET. `busparams` is omitted here so KDNET +autoselects the single virtio-net adapter. + +```bat +bcdedit /debug on +bcdedit /dbgsettings net hostip: port:<50000-50039> key: +``` + +- `hostip` is the WinDbg host address. +- `port` is a UDP port in the 49152-65535 range (50000-50039 is conventional). +- `key` is the debug encryption key (four dot-separated groups). Use a fixed + key, or omit it to let Windows generate one and print it. + +Reboot the debuggee after applying the settings. + +### Optionally pinning a specific NIC with busparams + +When the guest has more than one supported adapter, select the debug NIC +explicitly by its PCI bus/device/function. Read the location from a +PowerShell started in the logged in SAC `cmd` channel, no RDP or SSH needed. ```powershell Get-NetAdapter | ForEach-Object { @@ -56,22 +136,19 @@ Get-NetAdapter | ForEach-Object { } ``` -Then, from an elevated prompt on the debuggee: +Then, from an elevated prompt, pin it and reboot. ```bat -bcdedit /debug on -bcdedit /dbgsettings net hostip: port:<50000-50039> key: bcdedit /set "{dbgsettings}" busparams .. ``` -- `hostip` is the WinDbg host address. -- `port` is a UDP port in the 49152-65535 range (50000-50039 is conventional). -- `key` is the debug encryption key (four dot-separated groups). Use a fixed - key, or omit it to let Windows generate one and print it. -- `busparams` selects the virtio-net NIC. Omit it to let KDNET auto-select a - supported adapter. - -Reboot the debuggee after applying the settings. +- `busparams` selects the virtio-net NIC. Omit it entirely to let KDNET + autoselect a supported adapter. It is only a disambiguator, not a + requirement. When omitted, KDNET scans the PCI bus at boot. It picks the + first adapter that has a KDNET extensibility module and skips unsupported + ones. A single supported NIC therefore works without it. With several + supported NICs, leave only the debug NIC attached. Otherwise confirm which + one KDNET chose by MAC with `.kdtargetmac` in WinDbg. ## Debugger host (WinDbg) @@ -102,7 +179,10 @@ or configure an equivalent network kernel-debug connection in the WinDbg UI. debug port. Allow the port (and/or the `windbg.exe` program) inbound. KDNET connections are always initiated by the *target*, so the debugger must be listening before (or while) the target polls; start it first, or reboot - the debuggee with the debugger already running. + the debuggee with the debugger already running. When routing with NAT, KDNET + must have a routable address on the tap, from DHCP or a static SAC + assignment. Without one it uses a link local `169.254.x` source that the + kernel will not forward. - **No packets reach the debugger host.** Check tap bridging and host routing. When bridging guests through the host, note that host-originated replies can carry offloaded (incomplete) UDP checksums; if a guest ignores them, add an