Files
whitemagic/docs/memory-library-comparison.md
T
2026-07-21 22:30:10 +02:00

8.3 KiB

Process-Introspection Library Comparison — BlackMagic-old, MemorySharp, GreyMagic, BlackMagic

Date: 2026-07-21 Purpose: Compare four C# process-introspection libraries present in this repo and derive the design of a modern successor ("WhiteMagic"). The OpenSpec change whitemagic-foundation formalizes the design; this document is the supporting study.

The four subjects

Library Location Era / Platform Role in this study
BlackMagic-old reference/Blackmagic-old/ .NET FW 4.0, x86 The FASM "before" — initial commit, FASM as public API
MemorySharp reference/MemorySharp/ (≡ lib/MemorySharp/, byte-identical) .NET FW, x86 Contemporaneous peer that kept FASM behind a polished API
GreyMagic reference/GreyMagic/ .NET FW, ~2016, x86 Introduces the in-process execution model + detour/patch/marshal-cache
BlackMagic (current) reference/Blackmagic/ .NET 8, x86 + x64 The FASM "after" — modernized, FASM removed

reference/MemorySharp/ and lib/MemorySharp/ are identical copies (diff -rq clean).

Where FASM lives (the through-line)

FASM (the Flat Assembler, via the ManagedFasm C++/CLI wrapper in reference/fasm/) exists in these libraries for exactly one job: turning assembly text into machine code at runtime, so a small call-stub / injection-stub can be synthesized when the target address, arguments, and calling convention are only known at call time.

  • BlackMagic-old — FASM is a public, load-bearing dependency: public ManagedFasm Asm { get; set; } sits directly on the facade (BMMain.cs:80), and SInject.cs builds its DLL-redirect injection stub as mnemonic text at runtime.
  • MemorySharp — same dependency, wrapped: the assembler is internal (Fasm32Assembler, created unconditionally by AssemblyFactory), driven by calling-convention formatters behind Execute<T>.
  • GreyMagic — FASM only on the external path (ExternalProcessReader.Asm); the in-process path needs no assembler because it calls functions as delegates directly.
  • BlackMagic (current) — FASM removed entirely. Injection stubs became compile-time byte[] (BuildStub32/64); the public Asm property was deleted; runtime target execution moved to the D3D EndScene hook. See FASM-MIGRATION.md.

Conclusion: the assembly subsystem is not inherent to process introspection — it is a consequence of choosing CreateRemoteThread + "support arbitrary calling conventions" as the execution contract. Change the execution primitive (as current BM did) and the need for a runtime assembler evaporates. A managed assembler (Iced) or hand-emitted stubs cover the residual need with no native dependency and full x64 support.

Feature matrix

Axis BM-old MemorySharp GreyMagic BM (current)
Platform FW 4.0, x86 FW, x86 FW, x86 .NET 8, x86 + x64
Handles raw IntPtr SafeMemoryHandle SafeMemoryHandle SafeMemoryHandle, nullable
Addressing uint IntPtr IntPtr IntPtr (64-bit-safe)
Process model external external external + in-process external (+ frame-hook)
Typed read/write ReadInt etc. Read<T> + marshal Read<T> + MarshalCache Read<T> where unmanaged
Pattern scanning ("coming soon") (has PE parser) + cache
FASM / assembler public Asm internal, behind Execute<T> Asm (external only) none
Remote fn call raw Asm stubs Execute<T>(conv, args…) + async in-proc delegates D3D frame-hook stub
Function hooking Detour mgr (reversible, CallOriginal) Frame-hook only
Byte patching ("coming soon") Patch mgr (named, reversible) ad hoc
DLL injection CreateThread + hijack LoadLibrary via CreateThread (via Asm) CreateThread + hijack, x86/x64
Named allocation RemoteAllocation AllocatedMemory (by name) AllocateMemory
PE parsing PeHeaderParser
PEB / TEB ManagedPeb/ManagedTeb
Window mutation (move/resize/title/flash)
Input simulation (keyboard/mouse, no focus)
High-level ergonomics simple facade sharp[addr], module["fn"], Enum reads CreateFunction<T>, vtable helpers facade
Helpers ApplicationFinder, HandleManipulator, Randomizer, Serialization, Singleton MarshalCache, Utilities

What each does best (the "take from each" summary)

  • BlackMagic-old → the clean minimal Open / Read / Write / FindPattern facade; it is also the historical proof that FASM was once load-bearing and can be retired.
  • MemorySharp → high-level ergonomics: calling-convention Execute<T> + parameter marshalling + async, RemotePointer indexer, PEB/TEB, window + keyboard/mouse simulation, helper utilities.
  • GreyMagic → the engine: dual in/out-of-process MemoryBase, MarshalCache<T> fast typed IO, reversible DetourManager + PatchManager, CreateFunction<T>/vtable helpers, named AllocatedMemory, PeHeaderParser.
  • BlackMagic (current) → the modern platform: .NET 8, SafeMemoryHandle, nullable, Span<byte>, x64, pattern scanning + cache, rich DLL injection (CreateThread + thread-hijack, x86/x64 stubs), hand-assembled stubs (no FASM), per-frame hook (crash-safe execution), test coverage.

The crash-safety principle (why CreateRemoteThread needs care)

CreateRemoteThread does not crash the target — calling target internals from the wrong thread does. The target's main thread has exclusive affinity for its scripting VM, the render device, and the object model. A thread you spawn runs concurrently with it; the moment a payload touches that state (scripting-engine entry points, object traversal) it races the main thread → memory corruption → crash. This matches the "3 crashes in one session" recorded in FASM-MIGRATION.md.

The rule the successor must encode — split execution by payload safety:

  1. CreateRemoteThread is safe for self-contained, thread-agnostic payloads: LoadLibrary (DLL injection), pure WinAPI, code touching only memory you own.
  2. State-sensitive calls must run on the target's own thread, reached by hooking a per-frame function (D3D EndScene, or any frame function via a detour) and draining a work queue there each frame.
  3. In-process (once a managed DLL is injected), call target functions directly as delegates — no thread crossing at all.

WhiteMagic — synthesis

A modern successor unifying the four. Full design in openspec/changes/whitemagic-foundation/.

WhiteMagic (facade — BM-old ergonomics)
├─ Core:        SafeHandle, native P/Invoke, x64             [BM current]
├─ MemoryBase (abstract Read/Write + MarshalCache)           [GreyMagic]
│   ├─ ExternalReader (RPM/WPM)
│   └─ InProcessReader (direct deref, injected)
├─ Discovery:   PatternScanner(+cache), PeHeaderParser       [BM current + GreyMagic]
├─ Allocation:  AllocatedMemory (named chunks)               [GreyMagic]
├─ Assembler:   IAssembler → { HandStubs | Iced }            [BM current; Iced replaces FASM]
├─ Execution (three tiers):
│   ├─ RemoteThreadExecutor  (CreateRemoteThread — safe payloads)   [MemorySharp Execute<T>, no FASM]
│   ├─ MainThreadPump        (frame-hook work queue)               [BM frame-hook + GreyMagic detour]  ← crash-safe
│   └─ InProcessInvoker      (CreateFunction<T> delegates)          [GreyMagic]
├─ Hooking:     DetourManager + PatchManager                 [GreyMagic]
├─ Injection:   CreateThread + ThreadHijack (x86/x64)        [BM current]
├─ HighLevel:   RemotePointer, Module["fn"], PEB/TEB,
│               input sim, window, async, helpers            [MemorySharp]
└─ Safety:      disassemble-before-splice (Iced), auto-restore
                all patches/detours on Dispose               [new]

Net result: BM's modern, FASM-free, x64 core + GreyMagic's dual-mode / detour / patch / marshal-cache engine + MemorySharp's high-level ergonomics — with a three-tier execution model whose default for state-sensitive calls is the crash-safe main-thread pump, while CreateRemoteThread stays available for the payloads it is genuinely safe for.