feat(a2b): asset-RE tooling + findings; render blocked on asset resolution
- tools/convert_agf.py (AGF->BMP stills), tools/frida/ (capture harness + README; Frida 17.15.3) - docs/asset-resolution-re.md: foundational RE steering doc (resId->file; graphics+audio; not machine-verifiable -> Frida ground truth + human oracle). RE plan: parse SYS4INI, crack resId->name, backend render (A2b plan Tasks 3-5), audio, movies. - Finding: SC0000 bg = slot-0 slideshow (res 0x23 first); resolution opaque (CGINIT not a name map, SYS4INI S4IC needs RE, Frida file-I/O noisy/memory-mapped, opening mixes MPEG movies). - slice plan updated; texture ops already engine-driven (Task 1, prior commit). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
73
docs/asset-resolution-re.md
Normal file
73
docs/asset-resolution-re.md
Normal file
@@ -0,0 +1,73 @@
|
|||||||
|
# Asset Resolution — foundational RE (graphics + audio)
|
||||||
|
|
||||||
|
**The problem.** The bytecode loads assets by a small numeric **resource id** (`set-texture 0x23`,
|
||||||
|
`play-voice N`, …). To render/play the *real* asset — driven by the bytecode, not hardcoded — the
|
||||||
|
engine must resolve `resId → asset file`. This is **foundational** (nearly all visuals + all audio
|
||||||
|
depend on it) and **not machine-verifiable** (no pixel/audio oracle), which makes it the largest,
|
||||||
|
highest-risk area of the port. This doc is the steering state; it feeds the A2b render/audio slices.
|
||||||
|
|
||||||
|
## What's already landed
|
||||||
|
|
||||||
|
- **Graphics ops wired** (A2b-background, engine-driven): `create-texture 0x1f8` `(slot,w,h)`,
|
||||||
|
`set-texture 0x1f9` `(resId,slot)`, `draw-texture 0x1fb` `(slot,x,y,w,h)` promoted from VM stubs to
|
||||||
|
typed `IHost` methods; `CaptureHost` no-ops them (A1 trace-diff/A2a selftest stay green). The VM
|
||||||
|
now *drives* graphics; only resolution + backend rendering remain.
|
||||||
|
- **Audio ops named, not yet wired:** `play-voice 0xc4`, `play-bgm`.
|
||||||
|
- **Tools:** `tools/convert_agf.py` (AGF→BMP for *stills* via `AGF2BMP2AGF.exe`); `tools/frida/`
|
||||||
|
(runtime capture harness — see its README); Frida core installed (17.15.3).
|
||||||
|
|
||||||
|
## Findings (2026-07-06)
|
||||||
|
|
||||||
|
- **SC0000 background = a slot-0 full-screen slideshow.** The intro loads ~30 distinct full-screen
|
||||||
|
images into slot 0 in order (`set-texture 0x23→0`, `0x25→0`, `0x27→0`, …), each drawn 800×600.
|
||||||
|
Res `0x23` is the first. (There is also a persistent full-screen **slot 3** set *cross-context*,
|
||||||
|
not in SC0000 — inherited from the parent/system scene.)
|
||||||
|
- **The opening mixes movies + stills.** `AGF2BMP2AGF` reports `OP.AGF`/`MVB*.AGF` as
|
||||||
|
"unsupported type (possibly MPEG)" → DATA5 `MVB*` (210) and `OP`/`ED` are **movies**, not stills.
|
||||||
|
The opening's visible background did **not** match any `EV001*` still (confirmed by eye), so res
|
||||||
|
`0x23`'s file is not obvious from the name space alone — resolution is required.
|
||||||
|
- **Asset name spaces:** DATA2 = `EV*`/`EVM*` stills (985). DATA5 = `MVB*`/`OP`/`ED` movies (210).
|
||||||
|
DATA3 = `.OGG` audio (`BGM*`, `ANA*` voice).
|
||||||
|
- **The resolution chain is opaque statically.** `CGINIT` (`build/data/CGINIT.json`) is a
|
||||||
|
925-column *numeric* record table (row-major, sparse) — **not** an id→filename map.
|
||||||
|
**`SYS4INI.BIN` (magic `S4IC422`) is the authoritative asset index** the game + `BinExtractALF`
|
||||||
|
use (name ↔ archive ↔ offset ↔ size), but filenames are **not stored as plain ASCII** (an
|
||||||
|
`EV001AA` search misses), so it needs S4IC-format RE to parse.
|
||||||
|
- **Frida file-I/O is noisy.** `ReadFile` hooks on `DATA2.ALF` capture reads during the opening, but
|
||||||
|
the offsets/spans don't line up with extracted AGF sizes → the game likely **memory-maps** the
|
||||||
|
archives (so `ReadFile` offsets are OS paging, not clean per-asset loads) and/or uses async reads.
|
||||||
|
The robust hook is the game's **internal load-by-id function**, not file I/O.
|
||||||
|
|
||||||
|
## The RE plan (ordered)
|
||||||
|
|
||||||
|
1. **Parse `SYS4INI` (S4IC422) → an asset index** `{name, archive, offset, size}`. *Reusable and
|
||||||
|
bounded* — it names every asset in every DATA*.ALF, gives archive-offset→name (to rescue Frida
|
||||||
|
offsets), and is the **answer key** for step 2. Deliverable: `tools/parse_sys4ini.py` +
|
||||||
|
`build/asset-index.json`. (Format reference: asmodean's `exs4alf`, which `BinExtractALF` is based on.)
|
||||||
|
2. **Crack `resId → filename`.** With SYS4INI as the answer key, either (a) **order-correlate**: run
|
||||||
|
SC0000 in our engine to get the `set-texture(resId)` sequence, capture the real game's asset-load
|
||||||
|
order via a *reliable* Frida hook, and align them; or (b) **hook the internal load-by-id
|
||||||
|
function** directly (find via the opcode dispatch for `0x1f9`) to read `resId → name` at the
|
||||||
|
source. Likely underlying rule: `resId → CGINIT/table → name`. Deliverable: the mechanism +
|
||||||
|
`vm-map/resources.json` (or a generated map) seeding at least SC0000's slideshow.
|
||||||
|
3. **Wire the backend** (already designed — A2b-background plan Tasks 3–5): `ResourceMap` resolver +
|
||||||
|
Godot `TextureRect` compositing; render only resolved full-screen slots. Mechanical once (1)+(2) land.
|
||||||
|
4. **Audio** (parallel, same shape): resolve `play-voice`/`play-bgm` `id → OGG` via SYS4INI + a
|
||||||
|
Frida audio capture (hook `DATA3.ALF` reads or the audio-play fn); play via Godot. Reuses the
|
||||||
|
`tools/frida/` framework.
|
||||||
|
5. **Movies** (`OP`/`MVB`, MPEG) — a separate video-playback path; deferred.
|
||||||
|
|
||||||
|
## Validation reality (why this is the big haul)
|
||||||
|
|
||||||
|
Unlike the VM/dialogue work (byte-exact trace oracle), graphics + audio have **no machine oracle**.
|
||||||
|
Validation is: **Frida ground truth** (what the real game loads/plays for a scene) as the correctness
|
||||||
|
anchor, plus **human eyeball/ear**. Treat every mapping as provisional until Frida-confirmed; the
|
||||||
|
`resId→file` map is *data we curate against ground truth*, and the engine stays honest by only ever
|
||||||
|
rendering what the executed bytecode + the map produce (never a hardcoded image).
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
A2b-background: **machinery landed** (texture ops engine-driven, tools, findings). The **render is
|
||||||
|
blocked on asset resolution** (steps 1–2), which is promoted to its own foundational effort. Next:
|
||||||
|
either start step 1 (`SYS4INI` parser) or bank momentum with the Frida-free **choices** sub-slice
|
||||||
|
(static-RE opcode hunt) while resolution waits its scheduled turn.
|
||||||
@@ -181,6 +181,17 @@ Toolchain: `godot --headless --path godot --import` → `dotnet build godot/Hime
|
|||||||
**Next = A2b:** background via `AGF2BMP2AGF.exe`, `play-voice`/`play-bgm`, choices → VM globals,
|
**Next = A2b:** background via `AGF2BMP2AGF.exe`, `play-voice`/`play-bgm`, choices → VM globals,
|
||||||
just-enough `call-script`/state (unlocks richer scenes).
|
just-enough `call-script`/state (unlocks richer scenes).
|
||||||
|
|
||||||
|
### A2b-Background — machinery landed; render blocked on asset resolution (2026-07-06)
|
||||||
|
Engine-driven texture ops shipped: `create/set/draw-texture` (0x1f8/0x1f9/0x1fb) promoted from VM stubs
|
||||||
|
to typed `IHost` methods (CaptureHost no-ops → trace parity kept; engine tests 8/8). Tools: `convert_agf.py`
|
||||||
|
(AGF→BMP stills) + `tools/frida/` capture harness (Frida 17.15.3 installed). **Blocked:** rendering the bg
|
||||||
|
needs `resId → asset file` resolution, which proved opaque — CGINIT isn't a filename map, SYS4INI (S4IC422)
|
||||||
|
needs format RE, Frida file-I/O offsets are noisy (memory-mapping), and the opening mixes movies (MVB/OP =
|
||||||
|
MPEG) with stills so eyeball-curation stalled too. **Asset resolution promoted to a dedicated foundational
|
||||||
|
RE effort** (graphics + audio; not machine-verifiable — Frida ground truth + human eye/ear are the oracle):
|
||||||
|
see `docs/asset-resolution-re.md`. The backend render (ResourceMap + Godot compositing) stays designed in
|
||||||
|
`docs/superpowers/plans/2026-07-06-a2b-background.md` Tasks 3–5, mechanical once resolution lands.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Risks / open questions for A0
|
## Risks / open questions for A0
|
||||||
|
|||||||
33
tools/convert_agf.py
Normal file
33
tools/convert_agf.py
Normal file
@@ -0,0 +1,33 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Convert named AGF files (in extracted/DATA2 or DATA5) to BMP via AGF2BMP2AGF.exe,
|
||||||
|
into build/textures/. Run: py -3.11 -X utf8 tools/convert_agf.py EV001AA.AGF ..."""
|
||||||
|
import os, sys, shutil, subprocess
|
||||||
|
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
import paths
|
||||||
|
|
||||||
|
EXE = paths.EXTRACTED / "DATA1" / "AGF2BMP2AGF.exe" # tool lives in extracted/DATA1
|
||||||
|
SRC_DIRS = [paths.EXTRACTED / "DATA2", paths.EXTRACTED / "DATA5"]
|
||||||
|
OUT = paths.BUILD / "textures"
|
||||||
|
|
||||||
|
def find(name):
|
||||||
|
for d in SRC_DIRS:
|
||||||
|
p = d / name
|
||||||
|
if p.exists():
|
||||||
|
return p
|
||||||
|
return None
|
||||||
|
|
||||||
|
def main(argv):
|
||||||
|
OUT.mkdir(parents=True, exist_ok=True)
|
||||||
|
for name in argv:
|
||||||
|
src = find(name)
|
||||||
|
if not src:
|
||||||
|
print(f"NOT FOUND: {name}"); continue
|
||||||
|
tmp = OUT / name
|
||||||
|
shutil.copy(src, tmp)
|
||||||
|
subprocess.run([str(EXE), name], cwd=str(OUT), check=True)
|
||||||
|
tmp.unlink(missing_ok=True) # drop the copied .AGF, keep the .BMP
|
||||||
|
print(f"converted {name} -> {OUT / (os.path.splitext(name)[0] + '.BMP')}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main(sys.argv[1:]))
|
||||||
35
tools/frida/README.md
Normal file
35
tools/frida/README.md
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
# Frida runtime-observation tools
|
||||||
|
|
||||||
|
Runtime instrumentation of the **live game** (`AGE.EXE`) to capture ground truth for the
|
||||||
|
subsystems that have **no static/machine oracle** — asset resolution, graphics, audio, and the
|
||||||
|
opaque effectful opcodes. Frida attaches to the *running* (unpacked-in-memory) process, so the
|
||||||
|
game's packer is bypassed for hooking.
|
||||||
|
|
||||||
|
Prereq: `py -3.11 -m pip install frida` (core only — `frida-tools` CLI is not needed and its
|
||||||
|
`pygments` shim fails to install in this env). Installed: frida 17.15.3.
|
||||||
|
|
||||||
|
## Standard workflow
|
||||||
|
|
||||||
|
1. Launch the game (via `AGE Patch.exe` to avoid the periodic system-check messagebox) to a safe
|
||||||
|
point (e.g. the title). **Attach after launch**, not `frida -f` — spawning the patcher wouldn't
|
||||||
|
hook its child `AGE.EXE`, and direct-spawn risks the messagebox.
|
||||||
|
2. Run the capture script (`py -3.11 -u -X utf8 tools/frida/<tool>.py`) — it attaches by process
|
||||||
|
name and installs the hooks; run **unbuffered** (`-u`) so status prints appear immediately.
|
||||||
|
3. Drive the game through the target scene (SC0000's opening auto-plays on new game).
|
||||||
|
4. Stop the script; analyse the log in `build/` and **order-correlate** with the engine's op
|
||||||
|
sequence (e.g. `vm0`/the C# recording host's `set-texture`/`play-voice` order for that scene).
|
||||||
|
|
||||||
|
## Tools
|
||||||
|
|
||||||
|
- `capture_graphics.py` — hooks `ReadFile` on the graphics archives (`DATA2/DATA5*.ALF`), resolving
|
||||||
|
each handle→path via `GetFinalPathNameByHandleW` (cached) + the read offset. Log →
|
||||||
|
`build/frida-reads.log` (`path<TAB>offset<TAB>size`).
|
||||||
|
|
||||||
|
## Known limitations (see docs/asset-resolution-re.md)
|
||||||
|
|
||||||
|
- **File-I/O offsets are noisy** — spans don't match extracted AGF sizes; the game likely
|
||||||
|
**memory-maps** the archives (so `ReadFile` offsets are OS paging noise, not clean per-asset
|
||||||
|
loads) and/or uses async/`OVERLAPPED` reads. The robust hook is the game's **internal
|
||||||
|
load-by-id function** (find via the opcode dispatch), not file I/O — a future tool.
|
||||||
|
- Correlation still needs the **`SYS4INI` (S4IC422) asset index** parsed to turn an archive offset
|
||||||
|
into a filename. That parser is the first foundational RE step.
|
||||||
88
tools/frida/capture_graphics.py
Normal file
88
tools/frida/capture_graphics.py
Normal file
@@ -0,0 +1,88 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Attach Frida to the running game and log READS from the graphics archives (DATA2/DATA5*.ALF)
|
||||||
|
with offset+size, in order — so we can correlate them with the engine's set-texture sequence.
|
||||||
|
|
||||||
|
The game opens the .ALF archives at startup (before we can attach) and reads CGs from the
|
||||||
|
persistent handles, so CreateFile never fires during play; we hook ReadFile and resolve each
|
||||||
|
handle -> path via GetFinalPathNameByHandleW (cached), capturing the read offset.
|
||||||
|
|
||||||
|
Flow: game running -> run this (attach) -> replay SC0000's opening -> stop this. Log lands in
|
||||||
|
build/frida-reads.log as `path<TAB>offset<TAB>size`.
|
||||||
|
|
||||||
|
Usage: py -3.11 -u -X utf8 tools/frida_capture_graphics.py [process_name] (default AGE.EXE)
|
||||||
|
"""
|
||||||
|
import frida, sys, time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
proc = sys.argv[1] if len(sys.argv) > 1 else "AGE.EXE"
|
||||||
|
OUT = Path(__file__).resolve().parents[2] / "build" / "frida-reads.log" # age-reimpl/build/
|
||||||
|
OUT.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
log = open(OUT, "w", encoding="utf-8")
|
||||||
|
|
||||||
|
JS = r"""
|
||||||
|
const k32 = Process.getModuleByName('kernel32.dll');
|
||||||
|
const GetFinalPathNameByHandleW = new NativeFunction(
|
||||||
|
k32.findExportByName('GetFinalPathNameByHandleW'), 'uint32', ['pointer','pointer','uint32','uint32']);
|
||||||
|
const SetFilePointer = new NativeFunction(
|
||||||
|
k32.findExportByName('SetFilePointer'), 'uint32', ['pointer','int32','pointer','uint32']);
|
||||||
|
const NUL = ptr(0);
|
||||||
|
const cache = {};
|
||||||
|
|
||||||
|
function pathOf(h) {
|
||||||
|
const key = h.toString();
|
||||||
|
let v = cache[key];
|
||||||
|
if (v !== undefined) return v;
|
||||||
|
let p = null;
|
||||||
|
try {
|
||||||
|
const buf = Memory.alloc(1040);
|
||||||
|
const n = GetFinalPathNameByHandleW(h, buf, 519, 0);
|
||||||
|
if (n > 0 && n < 519) p = buf.readUtf16String();
|
||||||
|
} catch (e) {}
|
||||||
|
cache[key] = p;
|
||||||
|
return p;
|
||||||
|
}
|
||||||
|
|
||||||
|
const rf = k32.findExportByName('ReadFile');
|
||||||
|
Interceptor.attach(rf, {
|
||||||
|
onEnter(args) {
|
||||||
|
const p = pathOf(args[0]);
|
||||||
|
if (!p || !/(data2|data5)\.alf$|\.agf$/i.test(p)) return; // graphics archives only
|
||||||
|
const size = args[2].toInt32();
|
||||||
|
const ov = args[4];
|
||||||
|
let off = -1;
|
||||||
|
try { off = ov.isNull() ? SetFilePointer(args[0], 0, NUL, 1) : ov.add(8).readU32(); } catch (e) {}
|
||||||
|
send({path: p, offset: off, size: size});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
send({ready: true});
|
||||||
|
"""
|
||||||
|
|
||||||
|
def on_message(msg, data):
|
||||||
|
if msg.get("type") != "send":
|
||||||
|
if msg.get("type") == "error":
|
||||||
|
print("[frida-error]", msg.get("description"))
|
||||||
|
return
|
||||||
|
pl = msg["payload"]
|
||||||
|
if pl.get("ready"):
|
||||||
|
print("[frida] ReadFile hook live — replay the opening now.")
|
||||||
|
return
|
||||||
|
line = f"{pl['path']}\t{pl['offset']}\t{pl['size']}"
|
||||||
|
print("READ", line)
|
||||||
|
log.write(line + "\n"); log.flush()
|
||||||
|
|
||||||
|
try:
|
||||||
|
session = frida.attach(proc)
|
||||||
|
except frida.ProcessNotFoundError:
|
||||||
|
print(f"process '{proc}' not found. AGE-like:",
|
||||||
|
[(p.pid, p.name) for p in frida.enumerate_processes() if "age" in p.name.lower()])
|
||||||
|
sys.exit(2)
|
||||||
|
|
||||||
|
script = session.create_script(JS)
|
||||||
|
script.on("message", on_message)
|
||||||
|
script.load()
|
||||||
|
print(f"[frida] attached to {proc}; logging archive reads to {OUT}")
|
||||||
|
try:
|
||||||
|
while True:
|
||||||
|
time.sleep(0.5)
|
||||||
|
except KeyboardInterrupt:
|
||||||
|
pass
|
||||||
Reference in New Issue
Block a user