feat(a2b): asset-RE tooling + findings; render blocked on asset resolution

- tools/convert_agf.py (AGF->BMP stills), tools/frida/ (capture harness + README; Frida 17.15.3)
- docs/asset-resolution-re.md: foundational RE steering doc (resId->file; graphics+audio; not
  machine-verifiable -> Frida ground truth + human oracle). RE plan: parse SYS4INI, crack resId->name,
  backend render (A2b plan Tasks 3-5), audio, movies.
- Finding: SC0000 bg = slot-0 slideshow (res 0x23 first); resolution opaque (CGINIT not a name map,
  SYS4INI S4IC needs RE, Frida file-I/O noisy/memory-mapped, opening mixes MPEG movies).
- slice plan updated; texture ops already engine-driven (Task 1, prior commit).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
gamer147
2026-07-06 16:36:17 -04:00
parent d2d798bbac
commit b92e815850
5 changed files with 240 additions and 0 deletions

View File

@@ -0,0 +1,73 @@
# Asset Resolution — foundational RE (graphics + audio)
**The problem.** The bytecode loads assets by a small numeric **resource id** (`set-texture 0x23`,
`play-voice N`, …). To render/play the *real* asset — driven by the bytecode, not hardcoded — the
engine must resolve `resId → asset file`. This is **foundational** (nearly all visuals + all audio
depend on it) and **not machine-verifiable** (no pixel/audio oracle), which makes it the largest,
highest-risk area of the port. This doc is the steering state; it feeds the A2b render/audio slices.
## What's already landed
- **Graphics ops wired** (A2b-background, engine-driven): `create-texture 0x1f8` `(slot,w,h)`,
`set-texture 0x1f9` `(resId,slot)`, `draw-texture 0x1fb` `(slot,x,y,w,h)` promoted from VM stubs to
typed `IHost` methods; `CaptureHost` no-ops them (A1 trace-diff/A2a selftest stay green). The VM
now *drives* graphics; only resolution + backend rendering remain.
- **Audio ops named, not yet wired:** `play-voice 0xc4`, `play-bgm`.
- **Tools:** `tools/convert_agf.py` (AGF→BMP for *stills* via `AGF2BMP2AGF.exe`); `tools/frida/`
(runtime capture harness — see its README); Frida core installed (17.15.3).
## Findings (2026-07-06)
- **SC0000 background = a slot-0 full-screen slideshow.** The intro loads ~30 distinct full-screen
images into slot 0 in order (`set-texture 0x23→0`, `0x25→0`, `0x27→0`, …), each drawn 800×600.
Res `0x23` is the first. (There is also a persistent full-screen **slot 3** set *cross-context*,
not in SC0000 — inherited from the parent/system scene.)
- **The opening mixes movies + stills.** `AGF2BMP2AGF` reports `OP.AGF`/`MVB*.AGF` as
"unsupported type (possibly MPEG)" → DATA5 `MVB*` (210) and `OP`/`ED` are **movies**, not stills.
The opening's visible background did **not** match any `EV001*` still (confirmed by eye), so res
`0x23`'s file is not obvious from the name space alone — resolution is required.
- **Asset name spaces:** DATA2 = `EV*`/`EVM*` stills (985). DATA5 = `MVB*`/`OP`/`ED` movies (210).
DATA3 = `.OGG` audio (`BGM*`, `ANA*` voice).
- **The resolution chain is opaque statically.** `CGINIT` (`build/data/CGINIT.json`) is a
925-column *numeric* record table (row-major, sparse) — **not** an id→filename map.
**`SYS4INI.BIN` (magic `S4IC422`) is the authoritative asset index** the game + `BinExtractALF`
use (name ↔ archive ↔ offset ↔ size), but filenames are **not stored as plain ASCII** (an
`EV001AA` search misses), so it needs S4IC-format RE to parse.
- **Frida file-I/O is noisy.** `ReadFile` hooks on `DATA2.ALF` capture reads during the opening, but
the offsets/spans don't line up with extracted AGF sizes → the game likely **memory-maps** the
archives (so `ReadFile` offsets are OS paging, not clean per-asset loads) and/or uses async reads.
The robust hook is the game's **internal load-by-id function**, not file I/O.
## The RE plan (ordered)
1. **Parse `SYS4INI` (S4IC422) → an asset index** `{name, archive, offset, size}`. *Reusable and
bounded* — it names every asset in every DATA*.ALF, gives archive-offset→name (to rescue Frida
offsets), and is the **answer key** for step 2. Deliverable: `tools/parse_sys4ini.py` +
`build/asset-index.json`. (Format reference: asmodean's `exs4alf`, which `BinExtractALF` is based on.)
2. **Crack `resId → filename`.** With SYS4INI as the answer key, either (a) **order-correlate**: run
SC0000 in our engine to get the `set-texture(resId)` sequence, capture the real game's asset-load
order via a *reliable* Frida hook, and align them; or (b) **hook the internal load-by-id
function** directly (find via the opcode dispatch for `0x1f9`) to read `resId → name` at the
source. Likely underlying rule: `resId → CGINIT/table → name`. Deliverable: the mechanism +
`vm-map/resources.json` (or a generated map) seeding at least SC0000's slideshow.
3. **Wire the backend** (already designed — A2b-background plan Tasks 35): `ResourceMap` resolver +
Godot `TextureRect` compositing; render only resolved full-screen slots. Mechanical once (1)+(2) land.
4. **Audio** (parallel, same shape): resolve `play-voice`/`play-bgm` `id → OGG` via SYS4INI + a
Frida audio capture (hook `DATA3.ALF` reads or the audio-play fn); play via Godot. Reuses the
`tools/frida/` framework.
5. **Movies** (`OP`/`MVB`, MPEG) — a separate video-playback path; deferred.
## Validation reality (why this is the big haul)
Unlike the VM/dialogue work (byte-exact trace oracle), graphics + audio have **no machine oracle**.
Validation is: **Frida ground truth** (what the real game loads/plays for a scene) as the correctness
anchor, plus **human eyeball/ear**. Treat every mapping as provisional until Frida-confirmed; the
`resId→file` map is *data we curate against ground truth*, and the engine stays honest by only ever
rendering what the executed bytecode + the map produce (never a hardcoded image).
## Status
A2b-background: **machinery landed** (texture ops engine-driven, tools, findings). The **render is
blocked on asset resolution** (steps 12), which is promoted to its own foundational effort. Next:
either start step 1 (`SYS4INI` parser) or bank momentum with the Frida-free **choices** sub-slice
(static-RE opcode hunt) while resolution waits its scheduled turn.

View File

@@ -181,6 +181,17 @@ Toolchain: `godot --headless --path godot --import` → `dotnet build godot/Hime
**Next = A2b:** background via `AGF2BMP2AGF.exe`, `play-voice`/`play-bgm`, choices → VM globals,
just-enough `call-script`/state (unlocks richer scenes).
### A2b-Background — machinery landed; render blocked on asset resolution (2026-07-06)
Engine-driven texture ops shipped: `create/set/draw-texture` (0x1f8/0x1f9/0x1fb) promoted from VM stubs
to typed `IHost` methods (CaptureHost no-ops → trace parity kept; engine tests 8/8). Tools: `convert_agf.py`
(AGF→BMP stills) + `tools/frida/` capture harness (Frida 17.15.3 installed). **Blocked:** rendering the bg
needs `resId → asset file` resolution, which proved opaque — CGINIT isn't a filename map, SYS4INI (S4IC422)
needs format RE, Frida file-I/O offsets are noisy (memory-mapping), and the opening mixes movies (MVB/OP =
MPEG) with stills so eyeball-curation stalled too. **Asset resolution promoted to a dedicated foundational
RE effort** (graphics + audio; not machine-verifiable — Frida ground truth + human eye/ear are the oracle):
see `docs/asset-resolution-re.md`. The backend render (ResourceMap + Godot compositing) stays designed in
`docs/superpowers/plans/2026-07-06-a2b-background.md` Tasks 35, mechanical once resolution lands.
---
## Risks / open questions for A0

33
tools/convert_agf.py Normal file
View File

@@ -0,0 +1,33 @@
#!/usr/bin/env python3
"""Convert named AGF files (in extracted/DATA2 or DATA5) to BMP via AGF2BMP2AGF.exe,
into build/textures/. Run: py -3.11 -X utf8 tools/convert_agf.py EV001AA.AGF ..."""
import os, sys, shutil, subprocess
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import paths
EXE = paths.EXTRACTED / "DATA1" / "AGF2BMP2AGF.exe" # tool lives in extracted/DATA1
SRC_DIRS = [paths.EXTRACTED / "DATA2", paths.EXTRACTED / "DATA5"]
OUT = paths.BUILD / "textures"
def find(name):
for d in SRC_DIRS:
p = d / name
if p.exists():
return p
return None
def main(argv):
OUT.mkdir(parents=True, exist_ok=True)
for name in argv:
src = find(name)
if not src:
print(f"NOT FOUND: {name}"); continue
tmp = OUT / name
shutil.copy(src, tmp)
subprocess.run([str(EXE), name], cwd=str(OUT), check=True)
tmp.unlink(missing_ok=True) # drop the copied .AGF, keep the .BMP
print(f"converted {name} -> {OUT / (os.path.splitext(name)[0] + '.BMP')}")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))

35
tools/frida/README.md Normal file
View File

@@ -0,0 +1,35 @@
# Frida runtime-observation tools
Runtime instrumentation of the **live game** (`AGE.EXE`) to capture ground truth for the
subsystems that have **no static/machine oracle** — asset resolution, graphics, audio, and the
opaque effectful opcodes. Frida attaches to the *running* (unpacked-in-memory) process, so the
game's packer is bypassed for hooking.
Prereq: `py -3.11 -m pip install frida` (core only — `frida-tools` CLI is not needed and its
`pygments` shim fails to install in this env). Installed: frida 17.15.3.
## Standard workflow
1. Launch the game (via `AGE Patch.exe` to avoid the periodic system-check messagebox) to a safe
point (e.g. the title). **Attach after launch**, not `frida -f` — spawning the patcher wouldn't
hook its child `AGE.EXE`, and direct-spawn risks the messagebox.
2. Run the capture script (`py -3.11 -u -X utf8 tools/frida/<tool>.py`) — it attaches by process
name and installs the hooks; run **unbuffered** (`-u`) so status prints appear immediately.
3. Drive the game through the target scene (SC0000's opening auto-plays on new game).
4. Stop the script; analyse the log in `build/` and **order-correlate** with the engine's op
sequence (e.g. `vm0`/the C# recording host's `set-texture`/`play-voice` order for that scene).
## Tools
- `capture_graphics.py` — hooks `ReadFile` on the graphics archives (`DATA2/DATA5*.ALF`), resolving
each handle→path via `GetFinalPathNameByHandleW` (cached) + the read offset. Log →
`build/frida-reads.log` (`path<TAB>offset<TAB>size`).
## Known limitations (see docs/asset-resolution-re.md)
- **File-I/O offsets are noisy** — spans don't match extracted AGF sizes; the game likely
**memory-maps** the archives (so `ReadFile` offsets are OS paging noise, not clean per-asset
loads) and/or uses async/`OVERLAPPED` reads. The robust hook is the game's **internal
load-by-id function** (find via the opcode dispatch), not file I/O — a future tool.
- Correlation still needs the **`SYS4INI` (S4IC422) asset index** parsed to turn an archive offset
into a filename. That parser is the first foundational RE step.

View File

@@ -0,0 +1,88 @@
#!/usr/bin/env python3
"""Attach Frida to the running game and log READS from the graphics archives (DATA2/DATA5*.ALF)
with offset+size, in order — so we can correlate them with the engine's set-texture sequence.
The game opens the .ALF archives at startup (before we can attach) and reads CGs from the
persistent handles, so CreateFile never fires during play; we hook ReadFile and resolve each
handle -> path via GetFinalPathNameByHandleW (cached), capturing the read offset.
Flow: game running -> run this (attach) -> replay SC0000's opening -> stop this. Log lands in
build/frida-reads.log as `path<TAB>offset<TAB>size`.
Usage: py -3.11 -u -X utf8 tools/frida_capture_graphics.py [process_name] (default AGE.EXE)
"""
import frida, sys, time
from pathlib import Path
proc = sys.argv[1] if len(sys.argv) > 1 else "AGE.EXE"
OUT = Path(__file__).resolve().parents[2] / "build" / "frida-reads.log" # age-reimpl/build/
OUT.parent.mkdir(parents=True, exist_ok=True)
log = open(OUT, "w", encoding="utf-8")
JS = r"""
const k32 = Process.getModuleByName('kernel32.dll');
const GetFinalPathNameByHandleW = new NativeFunction(
k32.findExportByName('GetFinalPathNameByHandleW'), 'uint32', ['pointer','pointer','uint32','uint32']);
const SetFilePointer = new NativeFunction(
k32.findExportByName('SetFilePointer'), 'uint32', ['pointer','int32','pointer','uint32']);
const NUL = ptr(0);
const cache = {};
function pathOf(h) {
const key = h.toString();
let v = cache[key];
if (v !== undefined) return v;
let p = null;
try {
const buf = Memory.alloc(1040);
const n = GetFinalPathNameByHandleW(h, buf, 519, 0);
if (n > 0 && n < 519) p = buf.readUtf16String();
} catch (e) {}
cache[key] = p;
return p;
}
const rf = k32.findExportByName('ReadFile');
Interceptor.attach(rf, {
onEnter(args) {
const p = pathOf(args[0]);
if (!p || !/(data2|data5)\.alf$|\.agf$/i.test(p)) return; // graphics archives only
const size = args[2].toInt32();
const ov = args[4];
let off = -1;
try { off = ov.isNull() ? SetFilePointer(args[0], 0, NUL, 1) : ov.add(8).readU32(); } catch (e) {}
send({path: p, offset: off, size: size});
}
});
send({ready: true});
"""
def on_message(msg, data):
if msg.get("type") != "send":
if msg.get("type") == "error":
print("[frida-error]", msg.get("description"))
return
pl = msg["payload"]
if pl.get("ready"):
print("[frida] ReadFile hook live — replay the opening now.")
return
line = f"{pl['path']}\t{pl['offset']}\t{pl['size']}"
print("READ", line)
log.write(line + "\n"); log.flush()
try:
session = frida.attach(proc)
except frida.ProcessNotFoundError:
print(f"process '{proc}' not found. AGE-like:",
[(p.pid, p.name) for p in frida.enumerate_processes() if "age" in p.name.lower()])
sys.exit(2)
script = session.create_script(JS)
script.on("message", on_message)
script.load()
print(f"[frida] attached to {proc}; logging archive reads to {OUT}")
try:
while True:
time.sleep(0.5)
except KeyboardInterrupt:
pass