whitemagic/openspec/changes/whitemagic-foundation/design.md

Context

This repo contains four C# process-manipulation libraries studied in docs/memory-library-comparison.md:

• BlackMagic-old (reference/Blackmagic-old/) — .NET FW 4.0, x86, FASM as a public Asm property.
• MemorySharp (reference/MemorySharp/) — .NET FW, x86, FASM behind Execute<T>; rich high-level API (PEB/TEB, window, input, calling conventions) but unmaintained since 2016.
• GreyMagic (reference/GreyMagic/) — .NET FW, ~2016, x86; dual in/out-of-process MemoryBase, MarshalCache, DetourManager, PatchManager, CreateFunction<T>, PeHeaderParser.
• BlackMagic (reference/Blackmagic/, current) — .NET 8, x86 + x64, FASM removed, pattern scanning + cache, DLL injection (CreateThread + hijack), hand-assembled x86 stubs (SInject.cs EmitU8/EmitU32 byte emitter), CreateRemoteThread-based Execute (BMThread.cs). No frame hook exists.

The consuming use case is a WoW 3.3.5a bot. The dominant failure mode observed (FASM-MIGRATION.md) is game crashes when protected/game-state functions are invoked on a thread created by CreateRemoteThread, because WoW's main thread has exclusive affinity for the Lua VM, D3D9 device, and object manager.

WhiteMagic is a new, additive .NET 8 library that unifies the four. It reuses their ideas, not their assemblies. No project already depends on WhiteMagic, so there is no backward-compatibility constraint.

Goals / Non-Goals

Goals:

• Single modern (.NET 8, nullable, Span<byte>, SafeHandle) library that is bitness-agnostic (x86 + x64).
• A three-tier execution model whose default path for game-state calls is crash-safe (runs on the target's own thread), while CreateRemoteThread remains available for thread-agnostic payloads.
• Dual memory access: out-of-process (RPM/WPM) and in-process (direct deref) behind one abstract MemoryBase, with MarshalCache<T> for allocation-free typed IO.
• Reversible function hooking (DetourManager) and byte patching (PatchManager) with auto-restore on dispose.
• Replace FASM with an IAssembler seam: hand-emitted convention stubs by default, optional Iced backend for arbitrary assembly. Zero native dependency in the default configuration.
• Port DLL injection (CreateThread + thread-hijack, x86/x64) and pattern scanning + cache from current BlackMagic.
• Provide MemorySharp-grade ergonomics: RemotePointer indexer, Module["fn"].Execute(...), PEB/TEB, window mutation, input simulation, async wrappers, helpers.
• Test-first for all pure logic (assembler encodings, marshal cache, pattern matching, stub building, pump queue semantics).

Non-Goals:

• Modifying or replacing BlackMagic/MemorySharp/GreyMagic — WhiteMagic is additive.
• Shipping a full x86/x64 assembler ourselves — arbitrary assembly is delegated to Iced; only the fixed convention-stub shapes are hand-emitted.
• Managed-DLL injection bootstrapper (the CLR host that loads InProcessReader into the target). WhiteMagic exposes the in-process API surface; wiring an actual managed loader is a follow-up change.
• WoW-specific offsets, Lua unlock, or bot logic — those live in the consumer, not the library.
• Anti-cheat evasion.

Decisions

D1: Layered architecture with an abstract MemoryBase (from GreyMagic)

MemoryBase defines abstract ReadBytes/WriteBytes/Read<T>/Write<T>, relative/absolute addressing, and hosts the PatchManager. Two concrete readers:

• ExternalReader : MemoryBase — ReadProcessMemory/WriteProcessMemory over a SafeMemoryHandle. Owns allocation, injection, and the remote-thread + main-thread executors.
• InProcessReader : MemoryBase — reads the current process via ReadProcessMemory/WriteProcessMemory on a self-handle; owns the DetourManager and InProcessInvoker. (Revised from unsafe direct deref during Phase 2: .NET cannot catch AccessViolationException, so a bad deref kills the host with no soft-failure path; RPM-on-self fails soft. The in-process speed win moves to the delegate-call/detour paths, not the reader. See specs/memory-access.)

Why: GreyMagic proved this abstraction lets the same higher-level code (pattern scan, patch, high-level API) run in either mode. External is the primary path for a bot host; in-process is the fast/crash-free path once injected.

Alternatives considered: single external-only class (current BlackMagic) — rejected: forecloses the in-process delegate path, which is the cleanest crash-free execution. MemorySharp's factory-per-concern model (Assembly, Threads, Windows factories) — adopted selectively for the high-level surface, but the read/write core stays on MemoryBase for GreyMagic-style polymorphism.

D2: Three-tier execution, crash-safe by default (the headline)

Execution is split by payload safety, not by convenience:

Tier	Type	Use when	Mechanism
Remote thread	RemoteThreadExecutor	payload is thread-agnostic (LoadLibrary, pure WinAPI, self-contained shellcode)	CreateRemoteThread + convention stub, wait, exit code
Main-thread pump	MainThreadPump	payload touches game state (default)	queue delegate → drained on target thread via per-frame hook
In-process	InProcessInvoker	injected in-process	Marshal.GetDelegateForFunctionPointer, direct call

MainThreadPump installs a detour on a caller-supplied per-frame function address (an EndScene resolver ships as a convenience helper) via DetourManager. Each frame the hook drains a thread-safe queue and runs pending work items synchronously in the game's context, returning results/exceptions to the requesting thread through a completion handle. This is net-new — no frame hook exists in current BlackMagic to port. It is built on GreyMagic-style detours (D5) as its one underlying primitive; the pump is the first consumer of DetourManager.

Why: CreateRemoteThread does not itself crash the game — calling single-thread-affinity game internals from a foreign thread does. Making the pump the default for game calls encodes that rule so callers cannot trip the crash by accident, while power users retain raw CreateRemoteThread for the payloads it is safe for.

Alternatives considered: (a) always CreateRemoteThread (MemorySharp/old-BM) — rejected: the documented crash source. (b) always in-process (GreyMagic) — rejected: requires a managed loader in the target and is not always available; external must work standalone. (c) thread-hijack for every call — rejected: high risk, one-shot, poor for repeated calls; kept only for injection.

D3: IAssembler seam replacing FASM

IAssembler { byte[] Assemble(string asm, ulong origin = 0); }

Two backends:

• StubAssembler (default) — hand-emits the fixed calling-convention trampolines (cdecl/stdcall/thiscall/fastcall: push args, call, cleanup, ret) and the injection stubs, using an EmitU8/EmitU32/EmitU64 byte emitter (the pattern already in SInject.cs). No parsing, deterministic, x64-capable, zero dependency.
• IcedAssembler (optional) — wraps Iced for arbitrary user-supplied mnemonics when a caller genuinely needs runtime text assembly.

Why: The comparison established that the only thing FASM did was runtime text→bytecode, needed solely for stubs and (rarely) arbitrary asm. Stubs are a small fixed set best hand-emitted; arbitrary asm is better served by a modern, managed, x64, MIT-licensed assembler (Iced) than by a native C++/CLI FASM DLL. Retiring FASM removes the x86-only mixed-mode constraint.

Alternatives considered: keep FASM/Fasm.NET — rejected: native dependency, x86-only, unmaintained. Hand-emit everything including arbitrary asm — rejected: reimplementing an assembler is out of scope; Iced already exists.

D4: MarshalCache<T> for typed IO (from GreyMagic)

A static class MarshalCache<T> computes and caches Marshal.SizeOf, TypeCode, TypeRequiresMarshal, and IsIntPtr once per type. Read<T>/Write<T> branch on the cached flags: blittable types round-trip through Span/MemoryMarshal; marshal-required types use Marshal.PtrToStructure.

Why: GreyMagic's signature perf win — avoids per-call reflection. Current BlackMagic's where unmanaged constraint is faster still for blittable types but cannot express marshalled structs; the cache gives both.

D5: Reversible hooking and patching with auto-restore (from GreyMagic)

DetourManager/Detour (inline E9 jmp over a prologue, CallOriginal, Apply/Remove) and PatchManager/Patch (named byte patch, Apply/Remove/IsApplied). Both register into a manager that restores all live modifications on MemoryBase.Dispose. Detours are in-process only (they require executing the hook delegate in the target); patches work in both modes.

Safety addition: before splicing a detour, StubAssembler/IcedAssembler disassembles the target prologue to confirm the overwrite lands on instruction boundaries, avoiding the mid-instruction-splice crash class. When only StubAssembler is present, a minimal length-disassembler covers the common prologue shapes; full validation requires the Iced backend.

Why: MemorySharp promised Hook/Patch "coming soon" and never shipped them; GreyMagic shipped both and they are the cleanest available. Auto-restore prevents leaving the target corrupted after a crash of the host.

D6: High-level surface as opt-in factories (from MemorySharp)

RemotePointer (indexer sharp[addr]), RemoteModule/RemoteFunction (sharp["user32"]["MessageBoxA"].Execute(...)), ManagedPeb/ManagedTeb, WindowFactory (move/resize/title/flash/activate), Keyboard/Mouse (PostMessage + SendInput), and async wrappers over the executors. These sit above MemoryBase and the execution tiers; none is required for core memory work.

Why: This is the ergonomic layer that made MemorySharp pleasant. It is pure P/Invoke over the core — no assembler, no FASM.

D7: New projects, additive, test-first

WhiteMagic/ (library, net8.0-windows, AllowUnsafeBlocks) and WhiteMagicTest/ (xUnit). Task 1.3 creates WhiteMagic.slnx (SDK 10's default solution format). The reference libraries live under reference/ (git-ignored) and are not part of this build. Pure-logic components (assembler encodings, marshal cache, pattern matcher, stub builders, pump queue) get tests before implementation, mirroring the existing inject-and-assemble change's TDD discipline. Live-process behavior (RPM/WPM, injection, detours) is validated in integration tests gated on an available target.

Risks / Trade-offs

• Scope is large → Deliver in vertical slices (see Migration Plan). The minimum crash-safe slice is MemoryBase + ExternalReader + DetourManager + MainThreadPump; everything else layers on without reworking it.
• In-process tier needs a managed loader not in scope → Ship InProcessReader's API and delegate-call path now; mark the actual CLR-host injection as a follow-up. External + pump deliver crash-safety without it.
• Detour prologue splicing can crash if misaligned → Validate instruction boundaries before writing (D5); require Iced backend for full validation; keep Remove/auto-restore so a bad detour is recoverable.
• Iced adds a NuGet dependency → Isolated behind IAssembler; default StubAssembler keeps the library dependency-free. Only callers needing arbitrary asm opt in.
• MainThreadPump adds per-frame overhead and a hook on a hot function → Keep the drained-per-frame work bounded; the hook is a thin queue check when idle. Document that a wedged work item stalls the frame.
• x64 detours need larger/absolute jumps (14-byte push/ret or RIP-relative) → StubAssembler emits the correct form per bitness; covered by encoding tests.
• Blittable vs marshalled ambiguity in MarshalCache → Explicit flags and tests per type category; document that [StructLayout] is required for non-blittable remote structs.

Migration Plan

WhiteMagic is additive; there is nothing to migrate off. Delivery is phased so each slice is independently useful and testable:

1. Core — MemoryBase, ExternalReader, SafeMemoryHandle, MarshalCache, typed/string/bytes IO. (Replaces nothing; standalone.)
2. Crash-safe execution slice — StubAssembler, DetourManager, MainThreadPump, RemoteThreadExecutor. Proves game-state calls without crashes end-to-end. This is the headline deliverable.
3. Injection & discovery — DLL injection (CreateThread + hijack, x86/x64), pattern scanning + cache, PeHeaderParser, named AllocatedMemory.
4. In-process tier — InProcessReader, InProcessInvoker, CreateFunction<T>, vtable helpers (API surface; managed loader deferred).
5. High-level ergonomics — RemotePointer, RemoteModule/RemoteFunction, PEB/TEB, window, input, async, PatchManager polish, helpers.
6. Optional Iced backend — IcedAssembler, arbitrary-asm inject, full prologue validation.

Rollback: WhiteMagic is a separate assembly; removing its project reference reverts consumers with no effect on existing libraries.

Open Questions

• Frame-hook target: default to D3D9 EndScene, or accept a caller-supplied per-frame function address? Resolved: MainThreadPump takes a caller-supplied frame-function address (WoW-agnostic, library stays offset-free per Non-Goals); an EndScene resolver ships as a convenience helper only. Slice 2 depends on this — settled before slice 2 starts.
• Managed in-process loader: which host mechanism (custom CLR host vs. a native shim that calls CorBindToRuntimeEx/ICLRRuntimeHost)? Deferred to a follow-up change but affects the InProcessReader seam shape.
• Iced as default vs optional: keep hand-stubs default (zero dep) — confirmed — but should the library ship a WhiteMagic.Iced companion package rather than an optional reference? Package boundary TBD.
• Async model: Task-based wrappers (MemorySharp) vs. exposing the pump's completion handles directly. Likely both: pump returns a handle, async wrappers adapt it to Task<T>.