whitemagic/docs/memory-library-comparison.md

Process-Introspection Library Comparison — BlackMagic-old, MemorySharp, GreyMagic, BlackMagic

Date: 2026-07-21 Purpose: Compare four C# process-introspection libraries present in this repo and derive the design of a modern successor ("WhiteMagic"). The OpenSpec change whitemagic-foundation formalizes the design; this document is the supporting study.

The four subjects

Library	Location	Era / Platform	Role in this study
BlackMagic-old	reference/Blackmagic-old/	.NET FW 4.0, x86	The FASM "before" — initial commit, FASM as public API
MemorySharp	reference/MemorySharp/ (≡ lib/MemorySharp/, byte-identical)	.NET FW, x86	Contemporaneous peer that kept FASM behind a polished API
GreyMagic	reference/GreyMagic/	.NET FW, ~2016, x86	Introduces the in-process execution model + detour/patch/marshal-cache
BlackMagic (current)	reference/Blackmagic/	.NET 8, x86 + x64	The FASM "after" — modernized, FASM removed

reference/MemorySharp/ and lib/MemorySharp/ are identical copies (diff -rq clean).

Where FASM lives (the through-line)

FASM (the Flat Assembler, via the ManagedFasm C++/CLI wrapper in reference/fasm/) exists in these libraries for exactly one job: turning assembly text into machine code at runtime, so a small call-stub / injection-stub can be synthesized when the target address, arguments, and calling convention are only known at call time.

• BlackMagic-old — FASM is a public, load-bearing dependency: public ManagedFasm Asm { get; set; } sits directly on the facade (BMMain.cs:80), and SInject.cs builds its DLL-redirect injection stub as mnemonic text at runtime.
• MemorySharp — same dependency, wrapped: the assembler is internal (Fasm32Assembler, created unconditionally by AssemblyFactory), driven by calling-convention formatters behind Execute<T>.
• GreyMagic — FASM only on the external path (ExternalProcessReader.Asm); the in-process path needs no assembler because it calls functions as delegates directly.
• BlackMagic (current) — FASM removed entirely. Injection stubs became compile-time byte[] (BuildStub32/64); the public Asm property was deleted; runtime target execution moved to the D3D EndScene hook. See FASM-MIGRATION.md.

Conclusion: the assembly subsystem is not inherent to process introspection — it is a consequence of choosing CreateRemoteThread + "support arbitrary calling conventions" as the execution contract. Change the execution primitive (as current BM did) and the need for a runtime assembler evaporates. A managed assembler (Iced) or hand-emitted stubs cover the residual need with no native dependency and full x64 support.

Feature matrix

Axis	BM-old	MemorySharp	GreyMagic	BM (current)
Platform	FW 4.0, x86	FW, x86	FW, x86	.NET 8, x86 + x64
Handles	raw IntPtr	SafeMemoryHandle	SafeMemoryHandle	SafeMemoryHandle, nullable
Addressing	uint	IntPtr	IntPtr	IntPtr (64-bit-safe)
Process model	external	external	external + in-process	external (+ frame-hook)
Typed read/write	ReadInt etc.	Read<T> + marshal	Read<T> + MarshalCache	Read<T> where unmanaged
Pattern scanning	✅	❌ ("coming soon")	❌ (has PE parser)	✅ + cache
FASM / assembler	public Asm	internal, behind Execute<T>	Asm (external only)	none
Remote fn call	raw Asm stubs	Execute<T>(conv, args…) + async	in-proc delegates	D3D frame-hook stub
Function hooking	—	—	Detour mgr (reversible, CallOriginal)	Frame-hook only
Byte patching	—	❌ ("coming soon")	Patch mgr (named, reversible)	ad hoc
DLL injection	CreateThread + hijack	LoadLibrary via CreateThread	(via Asm)	CreateThread + hijack, x86/x64
Named allocation	—	RemoteAllocation	AllocatedMemory (by name)	AllocateMemory
PE parsing	—	—	PeHeaderParser	—
PEB / TEB	—	✅ ManagedPeb/ManagedTeb	—	—
Window mutation	—	✅ (move/resize/title/flash)	—	—
Input simulation	—	✅ (keyboard/mouse, no focus)	—	—
High-level ergonomics	simple facade	sharp[addr], module["fn"], Enum reads	CreateFunction<T>, vtable helpers	facade
Helpers	—	ApplicationFinder, HandleManipulator, Randomizer, Serialization, Singleton	MarshalCache, Utilities	—

What each does best (the "take from each" summary)

• BlackMagic-old → the clean minimal Open / Read / Write / FindPattern facade; it is also the historical proof that FASM was once load-bearing and can be retired.
• MemorySharp → high-level ergonomics: calling-convention Execute<T> + parameter marshalling + async, RemotePointer indexer, PEB/TEB, window + keyboard/mouse simulation, helper utilities.
• GreyMagic → the engine: dual in/out-of-process MemoryBase, MarshalCache<T> fast typed IO, reversible DetourManager + PatchManager, CreateFunction<T>/vtable helpers, named AllocatedMemory, PeHeaderParser.
• BlackMagic (current) → the modern platform: .NET 8, SafeMemoryHandle, nullable, Span<byte>, x64, pattern scanning + cache, rich DLL injection (CreateThread + thread-hijack, x86/x64 stubs), hand-assembled stubs (no FASM), per-frame hook (crash-safe execution), test coverage.

The crash-safety principle (why CreateRemoteThread needs care)

CreateRemoteThread does not crash the target — calling target internals from the wrong thread does. The target's main thread has exclusive affinity for its scripting VM, the render device, and the object model. A thread you spawn runs concurrently with it; the moment a payload touches that state (scripting-engine entry points, object traversal) it races the main thread → memory corruption → crash. This matches the "3 crashes in one session" recorded in FASM-MIGRATION.md.

The rule the successor must encode — split execution by payload safety:

1. CreateRemoteThread is safe for self-contained, thread-agnostic payloads: LoadLibrary (DLL injection), pure WinAPI, code touching only memory you own.
2. State-sensitive calls must run on the target's own thread, reached by hooking a per-frame function (D3D EndScene, or any frame function via a detour) and draining a work queue there each frame.
3. In-process (once a managed DLL is injected), call target functions directly as delegates — no thread crossing at all.

WhiteMagic — synthesis

A modern successor unifying the four. Full design in openspec/changes/whitemagic-foundation/.

WhiteMagic (facade — BM-old ergonomics)
├─ Core:        SafeHandle, native P/Invoke, x64             [BM current]
├─ MemoryBase (abstract Read/Write + MarshalCache)           [GreyMagic]
│   ├─ ExternalReader (RPM/WPM)
│   └─ InProcessReader (RPM/WPM on self-handle, injected)
├─ Discovery:   PatternScanner(+cache), PeHeaderParser       [BM current + GreyMagic]
├─ Allocation:  AllocatedMemory (named chunks)               [GreyMagic]
├─ Assembler:   IAssembler → { HandStubs | Iced }            [BM current; Iced replaces FASM]
├─ Execution (three tiers):
│   ├─ RemoteThreadExecutor  (CreateRemoteThread — safe payloads)   [MemorySharp Execute<T>, no FASM]
│   ├─ MainThreadPump        (frame-hook work queue)               [BM frame-hook + GreyMagic detour]  ← crash-safe
│   └─ InProcessInvoker      (CreateFunction<T> delegates)          [GreyMagic]
├─ Hooking:     DetourManager + PatchManager                 [GreyMagic]
├─ Injection:   CreateThread + ThreadHijack (x86/x64)        [BM current]
├─ HighLevel:   RemotePointer, Module["fn"], PEB/TEB,
│               input sim, window, async, helpers            [MemorySharp]
└─ Safety:      disassemble-before-splice (Iced), auto-restore
                all patches/detours on Dispose               [new]

Net result: BM's modern, FASM-free, x64 core + GreyMagic's dual-mode / detour / patch / marshal-cache engine + MemorySharp's high-level ergonomics — with a three-tier execution model whose default for state-sensitive calls is the crash-safe main-thread pump, while CreateRemoteThread stays available for the payloads it is genuinely safe for.

Deviations discovered during implementation

The design held, but building it surfaced corrections worth recording (each is detailed against its task in openspec/changes/whitemagic-foundation/tasks.md):

• InProcessReader reads via RPM/WPM on a self-handle, not unsafe direct deref — .NET cannot catch AccessViolationException, so a bad direct deref kills the host with no soft-failure path. The in-process speed win moves to the delegate-call and detour paths, not the reader (design decision D1, revised mid-Phase 2).
• MarshalCache<T> splits Size (managed, blittable) from MarshalSize (Marshal.SizeOf, marshal path) — a single size mis-sized structs whose unmanaged width differs (a bool field is managed-1 / unmanaged-4; inline ByValTStr/ByValArray under-sized the marshal buffer and corrupted the heap on write). MemoryBase picks per TypeRequiresMarshal at every IO site.
• x64 call stub is fully MS-x64-ABI compliant — 32-byte shadow space, 16-byte alignment at the inner call, full imm64 register loads (no >4 GiB pointer truncation), stack args above the shadow window. Proven at runtime by a live SSE callee whose aligned movaps faults on any misalignment (task 3.8), not just by byte-level encoding tests.
• RemoteModule/RemoteFunction follow PE export forwarders — kernel32!HeapAlloc → NTDLL.RtlAllocateHeap and similar resolve into the real target module; ordinal and API-set forwarders throw NotSupportedException rather than returning a wrong address (task 7.2).
• Detour prologue safety is tiered — the default StubAssembler length-decoder covers only the common x86/x64 prologue shapes and refuses any opcode outside that set (zero dependency); the optional IcedAssembler.GetPrologueLength decodes arbitrary prologues and is plugged in via DetourManager.PrologueLengthResolver when full validation is wanted (tasks 4.6, 8.3).
• Iced has no text parser — the design assumed arbitrary text assembly could be delegated to Iced, but Iced ships only a fluent code assembler and a decoder. IcedAssembler.Assemble bridges Intel-syntax text onto the fluent API by reflection (registers, immediates, labels; memory operands unsupported), rather than depending on a parser that does not exist (task 8.2).
• Injection bitness corrections — the thread-hijack injector enforces matching host/target bitness, so the 32-bit path always runs from a 32-bit caller and uses native GetThreadContext/SetThreadContext; the WOW64 context APIs (for 64-bit callers inspecting WOW64 targets) never apply here and were removed. ExternalReader validates QueryInformation/QueryLimitedInformation access and surfaces IsWow64Process failures instead of silently assuming host bitness.
• Bounds and protection hardening — AllocatedMemory range-checks typed IO against region size; Patch mirrors the detour's VirtualProtectEx dance; MainThreadPump guards the completion race on an already-completed TaskCompletionSource.