diff --git a/docs/PROJECT-STRUCTURE.md b/docs/PROJECT-STRUCTURE.md index fedab33..248ee30 100644 --- a/docs/PROJECT-STRUCTURE.md +++ b/docs/PROJECT-STRUCTURE.md @@ -75,11 +75,16 @@ S:\Game Hacking\Eushully\Himegari\ ← workspace root (three siblings) │ │ └── strings.jsonl every string, tagged by source opcode │ ├── data/ parsed data tables (*INIT → JSON) │ ├── scripts-json/ machine-readable full dumps (on demand via --json) + │ ├── textures/ AGF→BMP stills (convert_agf.py) — feeds the Godot render + │ ├── engine-dump/ UNPACKED engine dump (frida/dump_engine.py): range_.bin + manifest.json + │ ├── asset-index.json, asset-sections.json asset resolver data (parse_sys4ini / resolve_asset) │ ├── global-var-map.{json,md} partial global-variable name map │ ├── opcodes.json GENERATED from opcodes.toml (machine view for the C# VM) │ └── manifest.json, opcode-coverage.md (opcode-coverage.md GENERATED from opcodes.toml) │ - └── godot/ DELIVERABLE — the Godot/C# engine project (built in Phase A2+) + ├── engine/ DELIVERABLE — the .NET VM core (AgeEngine.sln: Age.Engine / Age.Cli / tests) + ├── tools/frida/ runtime-capture + engine-dump scripts (see tools/frida/README.md) + └── godot/ DELIVERABLE — the Godot/C# ADV front-end (references Age.Engine) ``` ## Conventions diff --git a/docs/phase-a-slice-plan.md b/docs/phase-a-slice-plan.md index 8bba758..640a87a 100644 --- a/docs/phase-a-slice-plan.md +++ b/docs/phase-a-slice-plan.md @@ -268,16 +268,28 @@ measured `0×0`, and the anchor-preserve math (`base' = center − (w_new/2, h_n corruption. Fix: seed `_slotDims[0] = (800,600)` (and record `create-texture(w,h)` dims) so the first CG's anchor stays an identity. This is the faithful stand-in for the skipped boot-time primary-surface creation. -**DEFERRED (next chunk) — the sprite/background anchor-record subsystem.** Everything blits through slot 0 -as an immediate-mode canvas; the anchor-preserve base globals **accumulate drift** across textures of -*different* sizes (same-size 800×600 CGs stay put; the first `BG*` 800×500 / `AE*` 800×800 / sprite starts -a drift that accumulates — `BG030A→(300,500)`, next→`(450,100)`, →`(800,350)`…, marching bottom-right). -The real engine doesn't drift because it stores each element's geometry in a **per-object record** via -`0x217/0x218/0x21a` (currently no-op) and restores it (the `0x12683` if-branch reading the `0x3239` record -table). Implementing that store/restore (+ the record layout, likely Frida-confirmed) is the fix for -backgrounds **and** sprites together. Fades/alpha (`AE*`, `0x202/0x203`) and green chromakey remain -deferred as before (the compositor is built to accept alpha later). Also out: true multi-surface (dest -handle is collapsed onto the screen). The full-screen opening path is unaffected by any of these. +**Post-opening bg/sprite drift — RESOLVED as a STATE-DIVERGENCE artifact, NOT a missing native op +(2026-07-06/07).** Symptom: everything blits through slot 0 as an immediate-mode canvas; the anchor-preserve +base globals **accumulate drift** across differently-sized textures (`BG030A→(300,500)`, next→`(450,100)`, +→`(800,350)`… marching bottom-right). We reverse-engineered the whole chain (systematic-debugging): +1. Root cause traced to **`0x215` = native graphics-object query** (opcodes.toml `query-gfx-object?`), which we + stub → `label_12649` takes the wrong branch → all draws collapse onto slot 0 → anchor-preserve reads foreign + textures → drift. +2. **Engine now statically analyzable (major, general unlock):** `SYS4AB.BIN` = `XOR-0xFF(AGE.EXE)` (dead end), + but `AGE.EXE` unpacks **in-place at 0x400000** in the live process → `tools/frida/dump_engine.py` → + `build/engine-dump/` (validated via AGF-decoder landmark; interpreter confirmed to run from the module, so + handlers are hookable). Handler ABI + object-record layout (`[esi+0x53d64]`, 120B/rec, cmd-type at rec+0x24) + decoded. See `docs/vm-mapping-plan.md` appendix + `tools/frida/README.md`. +3. **Live capture verdict (the resolution):** `tools/frida/capture_gfx_objects.py` polled the object-record + array through the **real** opening — it held only **3 persistent UI objects, ZERO CG objects**. So the real + game does **not** draw the opening CGs via the `0x212–0x21a` positioned-object path our headless VM uses; + with proper state it takes a different (direct) branch that we already render correctly. **⇒ the drift is + downstream of our unseeded headless VM taking `label_12649`'s else-branch (compute-from-drifting-base) where + the real game hits the if-branch (stored/record geometry). The fix is the Phase B state/choices-flow work, + not a separate native-op subsystem.** Seeding real per-scene/object state makes `label_12649` branch right. +Fades/alpha (`AE*`, `0x202/0x203`) + green chromakey + true multi-surface remain deferred; the compositor is +built to accept alpha later. The full-screen opening path is correct and unaffected. **Native gfx-op modeling +is only needed for scenes that genuinely use runtime-positioned sprites — revisit later with the dump in hand.** --- diff --git a/docs/tools-reference.md b/docs/tools-reference.md index 9c339f0..5724b18 100644 --- a/docs/tools-reference.md +++ b/docs/tools-reference.md @@ -78,10 +78,15 @@ All opcode knowledge (ABI, semantics, provenance, `depends_on`) is hand-edited * | `tools/frida/capture_resid_args.py` | Phase-2 probe: dumps the decoder's args / context / caller frame (established the loader carries only offsets, not names). | `py -3.11 -u -X utf8 tools/frida/capture_resid_args.py [pid]` · `--analyze` | running game → `build/frida-resid-args.jsonl` | | `tools/frida/find_globals_base.py` | Runtime-global RE (SHELVED — see `docs/global-memory-re.md`): flat-int32 signature scan for the VM global array. Finds nothing → layout isn't flat. | `--build-sig` · `py -3.11 -u -X utf8 tools/frida/find_globals_base.py [pid]` | `*INIT` → `build/globals-signature.json`; scans running game | | `tools/frida/find_global_by_sequence.py` | Runtime-global RE (SHELVED): differential resId value-scan + stability filter. Finds stack proxies; proved `G[0x62424]` is a transient arg-register. | `py -3.11 -u -X utf8 tools/frida/find_global_by_sequence.py [pid]` | running game + index → stdout | +| `tools/frida/dump_engine.py` | ★ **Dump the UNPACKED engine code** from the live process for offline static RE (native handlers). `AGE.EXE` unpacks in-place at `0x400000`; Kelebek VAs map `VA−0x400000` = file-off. Validated via the AGF-decoder landmark `+0x74f1f`. | `py -3.11 -u -X utf8 tools/frida/dump_engine.py [pid]` | running game → `build/engine-dump/{manifest.json,range_.bin}` | +| `tools/frida/probe_handlers.py` | Probe which region the interpreter executes from (module vs heap). Confirmed: **operand-fetch `+0x1b940` fires ~8500/s ⇒ interpreter runs from the module `0x400000`** (handlers hookable by dump address). | `py -3.11 -u -X utf8 tools/frida/probe_handlers.py [pid]` | running game → stdout (per-hook fire counts) | +| `tools/frida/capture_gfx_objects.py` | Capture the native gfx object-manager state: grab engine ctx (`esi` via operand-fetch `ecx`), poll the object-record array `[esi+0x53d64]` (20×120B; `field[0]=0xffffffff`=free, cmd-type at rec+0x24). **Finding: the real opening uses ZERO CG records ⇒ the bg/sprite drift is a state-divergence artifact, not a missing op.** | `py -3.11 -u -X utf8 tools/frida/capture_gfx_objects.py [pid] [secs]` | running game → `build/gfx-objects.jsonl` | + +*(Static disassembly of `build/engine-dump/range_00400000.bin` uses **capstone** — `py -3.11 -m pip install capstone`; VA `X` → file offset `X−0x400000`.)* ## Historical / one-off | Tool | Purpose | |---|---| | `probe_*.py` (`probe_header`, `probe_leads`, `probe_refs`, `probe_tables`, `probe_tags`, `probe_types`, `probe_xref`) | Container/opcode format-RE probes used to reverse the format originally. Kept for reproducibility; not part of the normal workflow. | -| `pack_check.py` | Checks whether `AGE.EXE` is packed (it is). No longer a blocker — we run our own VM. | +| `pack_check.py` | Checks whether `AGE.EXE` is packed (it is: entropy-8 code sections, zeroed IAT). `SYS4AB.BIN` is NOT a separate image — it's `XOR-0xFF(AGE.EXE)` byte-for-byte (0x2c header + XOR payload). The unpacked engine exists only in memory → dump it with `frida/dump_engine.py`. |