Fix x64 stub ABI and marshal-path sizing; dedupe memory readers

x64 call stub was ABI-broken: fixed 0x20 frame left rsp misaligned at the
inner call (callee entry rsp ≡ 0, ABI requires ≡ 8) and, for 5+ args, wrote
stack args over the return address. Compute frame K ≡ 8 (mod 16), K ≥
0x20 + 8*stackArgs, so the callee sees a 16-aligned stack and stack args land
above the shadow window. Load register args as full 64-bit imm64 (was imm32,
which truncated pointers > 4 GiB). BuildCallStub now takes nuint[]; x86 range-
checks each arg against uint.MaxValue instead of silently truncating.

MarshalCache conflated managed and unmanaged width in one Size field: the
blittable path needs Unsafe.SizeOf<T> (bool = 1) while the marshal path needs
Marshal.SizeOf<T> (inline ByValTStr/ByValArray expand past the managed
pointer). Add MarshalSize; MemoryBase picks per TypeRequiresMarshal at all four
IO sites. Prevents PtrToStructure/StructureToPtr from over-reading/overwriting
the pinned scratch buffer (heap corruption on write).

Extract shared RPM/WPM into RpmHelper: honor partial reads (dead Array.Resize
removed), consistent write-return semantics; InProcessReader now guards
MainModule like ExternalReader.

Tests: x64 frame-alignment property + inline-marshal round-trip added (both
fail against the pre-fix code); existing x64 byte-expectation tests updated to
the new frame. Build clean, 100/100 pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
kbe
2026-07-21 22:30:10 +02:00
co-authored by Claude Opus 4.8
parent 184dec86ca
commit 12b9b6c03e
7 changed files with 343 additions and 101 deletions
+51
View File
@@ -196,6 +196,43 @@ public class MemoryBaseTests
reader.Dispose();
}
// ── Marshal-path round-trip ───────────────────────────────────────────────
//
// The marshal path was previously sized using MarshalCache.Size (managed
// layout width). When a struct carries an inline marshal-expanded field
// (ByValTStr, ByValArray, etc.) this width is smaller than the actual
// read/write width — writing overflows the pinned buffer and reading
// under-fetches the remote bytes, producing silent heap corruption.
//
// The marshal path must use MarshalCache.MarshalSize (= Marshal.SizeOf<T>)
// so the pinned buffer is large enough for PtrToStructure / StructureToPtr.
[Fact]
public void Read_struct_via_marshal_path_round_trips_inline_string()
{
using var reader = OpenSelf();
// A marshal-path struct carries a reference, so it cannot be pinned; the
// target must be an unmanaged buffer of the FULL marshal width. Pre-patch,
// Write sized its scratch buffer with MarshalCache.Size (managed pointer
// width, 8) and StructureToPtr overran it, while Read under-fetched the
// remote bytes — the string came back wrong. Post-patch both use
// MarshalSize (Marshal.SizeOf<InlineStr>).
int size = Marshal.SizeOf<InlineStr>();
IntPtr addr = Marshal.AllocHGlobal(size);
try
{
InlineStr original = new InlineStr { Name = "Hello, World!" };
Assert.True(reader.Write(addr, original));
InlineStr read = reader.Read<InlineStr>(addr);
Assert.Equal("Hello, World!", read.Name);
}
finally
{
Marshal.FreeHGlobal(addr);
}
}
// ── Graceful failure on invalid addresses ───────────────────────────────
[Fact]
@@ -243,3 +280,17 @@ public struct TestStruct : IEquatable<TestStruct>
public override int GetHashCode() => HashCode.Combine(X, Y);
public override string ToString() => $"({X}, {Y})";
}
/// <summary>
/// A struct whose managed layout is just a reference pointer (8 bytes) but whose
/// unmanaged marshal layout carries an inline character buffer. Exercised by
/// <see cref="MemoryBaseTests.Read_struct_via_marshal_path_round_trips_inline_string"/>
/// to catch regressions where the marshal path uses the managed width instead
/// of the marshal width.
/// </summary>
[StructLayout(LayoutKind.Sequential)]
public struct InlineStr
{
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 16)]
public string Name;
}