OpenMaidEngine/docs/asset-resolution-re.md

# Asset Resolution — foundational RE (graphics + audio)

**The problem.** The bytecode loads assets by a small numeric **resource id** (`set-texture 0x23`,
`play-voice N`, …). To render/play the *real* asset — driven by the bytecode, not hardcoded — the
engine must resolve `resId → asset file`. This is **foundational** (nearly all visuals + all audio
depend on it) and **not machine-verifiable** (no pixel/audio oracle), which makes it the largest,
highest-risk area of the port. This doc is the steering state; it feeds the A2b render/audio slices.

## What's already landed

- **Graphics ops wired** (A2b-background, engine-driven): `create-texture 0x1f8` `(slot,w,h)`,
  `set-texture 0x1f9` `(resId,slot)`, `draw-texture 0x1fb` `(slot,x,y,w,h)` promoted from VM stubs to
  typed `IHost` methods; `CaptureHost` no-ops them (A1 trace-diff/A2a selftest stay green). The VM
  now *drives* graphics; only resolution + backend rendering remain.
- **Audio ops WIRED (2026-07-06):** `play-bgm 0xbf` / `play-voice 0xc4` → `IHost.PlayBgm/PlayVoice` →
  same resolver → OGG via Godot `AudioStreamPlayer`. See step 4 below.
- **Tools:** `tools/convert_agf.py` (AGF→BMP for *stills* via `AGF2BMP2AGF.exe`); `tools/frida/`
  (runtime capture harness — see its README); Frida core installed (17.15.3).

## Findings (2026-07-06)

- **SC0000 background = a slot-0 full-screen slideshow.** The intro loads ~30 distinct full-screen
  images into slot 0 in order (`set-texture 0x23→0`, `0x25→0`, `0x27→0`, …), each drawn 800×600.
  Res `0x23` is the first. (There is also a persistent full-screen **slot 3** set *cross-context*,
  not in SC0000 — inherited from the parent/system scene.)
- **The opening mixes movies + stills.** `AGF2BMP2AGF` reports `OP.AGF`/`MVB*.AGF` as
  "unsupported type (possibly MPEG)" → DATA5 `MVB*` (210) and `OP`/`ED` are **movies**, not stills.
  The opening's visible background did **not** match any `EV001*` still (confirmed by eye), so res
  `0x23`'s file is not obvious from the name space alone — resolution is required.
- **Asset name spaces:** DATA2 = `EV*`/`EVM*` stills (985). DATA5 = `MVB*`/`OP`/`ED` movies (210).
  DATA3 = `.OGG` audio (`BGM*`, `ANA*` voice).
- **The resolution chain is opaque statically.** `CGINIT` (`build/data/CGINIT.json`) is a
  925-column *numeric* record table (row-major, sparse) — **not** an id→filename map.
  **`SYS4INI.BIN` (magic `S4IC422`) is the authoritative asset index** the game + `BinExtractALF`
  use (name ↔ archive ↔ offset ↔ size). Filenames aren't plain ASCII because the whole directory
  is **LZSS-compressed** (not encrypted). **DONE (2026-07-06):** `tools/parse_sys4ini.py` parses it
  → `build/asset-index.json` (13206 entries). See step 1 below.
- **Frida file-I/O is noisy.** `ReadFile` hooks on `DATA2.ALF` capture reads during the opening, but
  the offsets/spans don't line up with extracted AGF sizes → the game likely **memory-maps** the
  archives (so `ReadFile` offsets are OS paging, not clean per-asset loads) and/or uses async reads.
  The robust hook is the game's **internal load-by-id function**, not file I/O.

## The RE plan (ordered)

1. **Parse `SYS4INI` (S4IC422) → an asset index** `{name, archive, offset, size}`. **✅ DONE
   (2026-07-06).** `tools/parse_sys4ini.py` → `build/asset-index.json`: 5 archives (DATA1–5),
   13206 real entries (2 `@` placeholders skipped). **Format:** `uint32 packed_size @0x134`, then an
   LZSS stream at `0x138` running to EOF (GARbro-style: 0x1000 zero-filled ring buffer, init pos
   0xFEE, control bits LSB→MSB, 1=literal / 0=two-byte backref `off=(hi&0xf0)<<4|lo`, `len=3+(hi&0xf)`).
   Decompresses to `uint32 arc_count`, `arc_count × char[256]` archive names, `uint32 file_count`,
   then `file_count ×` 80-byte records `{char name[64]; u32 arc_id, file_number, offset, size}`.
   **Validated:** decompressed length (1058783) equals the stored size dword at `0x12c`; per-archive
   counts match the `extracted/` ground truth exactly (DATA2=985, DATA3=39, DATA4=9733, DATA5=210);
   all 13206 `offset+size` fit inside their real `.ALF`; 837 name-matched files → 0 size mismatches.
   `files[]` preserves directory order (feeds step 2's order-correlation). Re-run:
   `py -3.11 -X utf8 tools/parse_sys4ini.py --check`. (Ref: asmodean's `exs4alf` / GARbro Eushully `ArcALF.cs`.)
2. **Resolve `resId → asset file`.** **✅ SOLVED for scene-manifest references (2026-07-06);
   system/global raw ids are a separate path identified 2026-07-10.**

   **The rule:** SYS4INI's file list is organized into **SECTIONS, one per scene** — each is a
   `SCxxxx.BIN` script entry followed by that scene's **asset MANIFEST**: every asset it references,
   across *all* archives and types (EV/BG/CS/AE graphics **and** OGG/WAV audio), interleaved in usage
   order. `file_number` is the **0-based index within the section**. So:

   > **`resId → files[ section_base(scene) + resId ]`**, where `section_base` = the start of the SYS4INI
   > section containing the scene's `SCxxxx.BIN`.

   Manifest rule holds for `set-texture(resId)` and `play-voice(id)`. **⚠ `play-bgm` is the EXCEPTION —
   it does NOT use the manifest; it uses direct literal names `BGM{id:03d}.OGG` (see step 4, by-ear
   corrected 2026-07-06).** **Tool:** `tools/resolve_asset.py --build` → `build/asset-sections.json`
   (359 sections, 136 scenes); `resolve_asset.py <SCENE> [resId]` resolves. **Validated:** `file_number ==
   position − section_base` for 12848/13206 files (97%); SC0000 resolves 17/17 across archives vs the Frida
   capture (`0x25→EV052CA`, `0x36→BG030A` background, `0x6c→EM* effect`); 586/595 distinct captured loads
   (all sections) satisfy `files[base+fn]==name`. This is the derivable rule that generalizes to any
   AGE game with the same container — **the "scope" was just which SYS4INI section the scene lives in.**
   (The old `play-bgm 5→BGM006` validation point was a mis-attribution — the real game plays BGM005.)

   **System/global-id exception (identified 2026-07-10; not yet implemented).** Some SYSTEM4 loads use
   the SYS4INI record's universal `raw_index` directly, including the two `@` placeholder records, rather
   than a scene-local manifest index. `SYSTEM4.BIN` writes `G[0x69b]=0x337e`, then
   `set-texture(G[0x69b], slot=0x11)`. SYS4INI `raw_index 0x337e` is `DATA1/SO001.AGF`, the shared
   800×300 RGBA system-chrome sheet. The filtered `files[]` list omits placeholders, so treating `0x337e`
   as a `files[]` position currently mis-resolves it to `SETROUTE.BIN`. This path needs a distinct
   `raw_index → entry` lookup; the scene-manifest rule above remains correct for SC texture/voice ids.

   *How we got here (condensed):* first confirmed `resId == file_number` via Frida load-order correlation
   for SC0000's opening, but `file_number` is not globally unique so a per-scene "scope" was needed. A long
   hunt for the selector (thought it was native scene state; even tried reading `G[0x62424]` live — the
   VM global memory is structured/packed, see `docs/global-memory-re.md`) missed the real structure until a
   **full multi-archive capture** (user domain tip: DATA1 holds BG/CS/CB/CA/CP graphics by name prefix, not
   just DATA2 EV CGs) revealed `file_number == SYS4INI position` inside per-scene sections. Superseded tools:
   `tools/correlate_scope.py`, `vm0.py --settex` (VM set-texture trace; still useful, but vm0 diverges on
   branchy non-opening scenes — use the C# VM to trace those). Runtime note for future work: the game is
   **packed** (main VM logic in a per-run heap `r-x` region) and streams archives through a heap block-cache
   via `ReadFile` (not mmap); the stable AGF decoder is `AGE.EXE+0x74f1f`.
3. **Wire the backend.** **✅ FIRST-PASS RENDER LANDED (2026-07-06).** `Age.Engine/Sys4/ResourceMap.cs`
   (Resolve + BMP path) + `GodotAdvHost` texture ops → `TextureRect` compositing behind the dialogue;
   `IHost.DrawTexture` extended with dst x/y; 800×600 window; `convert_agf.py --scene` pre-converts a
   scene's manifest AGFs → BMP. The full-screen **event-CG layer renders end-to-end** from the executed
   bytecode. **Historical limitations at first landing (subsequently resolved in the Phase-A graphics
   slices):** sprites + `BG*` (routed through the CG-load subroutine) had garbage geometry because native
   graphics ops were stubbed (`0x208` get-texture-size + the sprite position/animation chain); fades
   (`AE*`) drew opaque (no alpha); the slot
   model approximated the game's immediate-mode blit-onto-slot-0 canvas. See `docs/phase-a-slice-plan.md`
   (A2b section) for the implementation history and current retained-object model.

   **Current system-chrome shortcut/gap (confirmed 2026-07-10).** `Main --boot` executes only
   `INITCONFIG/INIT2/INIT` through `CaptureHost` and copies their globals into the scene VM; it does not
   replay SYSTEM4's graphics side effects through `GodotAdvHost`. `convert_agf.py --scene SC0000` also
   converts only SC0000's manifest, so `build/textures/SO001.BMP` is absent. `CALLBACK_WINDOW.BIN` expects
   slot `0x11` to already contain SO001, draws the 800×227 textbox from `(0,0)`, and crops the lower-right
   buttons from the same sheet. In the port slot 17 is unpopulated, so retained handle `0xd2f0` resolves as
   a colored surfaceless object and the compositor draws the observed opaque black fill. A temporary decode
   verified SO001 is 800×300, 32-bpp, with substantial per-pixel alpha; the current rasterizer already
   consumes source alpha. The missing prerequisites are system-asset resolution/conversion and retained
   slot initialization, not new textbox drawing or button interaction.
4. **Audio.** **✅ WIRED (2026-07-06) — no Frida needed.** Same rule as textures:
   `play-bgm(id)`/`play-voice(id)` → `files[section_base(scene)+id]` → OGG. `IHost.PlayBgm/PlayVoice` +
   VM dispatch (`play-bgm` 0xbf / `play-voice` 0xc4, both argc 1); `ResourceMap.AudioPath` → loose
   `extracted/DATA{n}/{name}.OGG`; `GodotAdvHost` → `Main`'s two `AudioStreamPlayer` nodes
   (`AudioStreamOggVorbis.LoadFromBuffer`; BGM loops, voice interrupt-on-new). Non-Godot hosts no-op it
   → `--selftest`/8-8 byte-identical. SC0000 fires 18 BGM + 198 voice. **By-ear VALIDATED
   (2026-07-06):** voices play on their lines (`play-voice` med→HIGH). **BUT the two audio ops use DIFFERENT
   addressing — the earlier "unified graphics+audio manifest" claim was WRONG for BGM:**
   - **Voice** (`play-voice`) → per-scene manifest, `files[base+id]`, **offset 0** (same as textures). Proven:
     the manifest interleaves graphics/voice (`files[35]=EV049AA`, `[36]=MAN999`, `[37]=EV052CA`, `[38]=SYL0001`),
     so `id-1` would land voices on `.AGF` (silent) — they play, so offset is exactly 0.
   - **BGM** (`play-bgm`) → **DIRECT LITERAL NAME**, `id → BGM{id:03d}.OGG` (DATA3), NOT the manifest.
     Confirmed by ear (`play-bgm 5→BGM005`, `8→BGM008`; the manifest gave BGM006/009 = off-by-one) and proven
     by `play-bgm 0x23→BGM035.OGG` — a real standalone track (BGM set skips 030-034) the manifest mis-resolved
     to a graphics entry. Implemented as `ResourceMap.BgmPathById(id)`; `GodotAdvHost.PlayBgm` uses it.
     The prior "Frida-confirmed play-bgm 5→BGM006" record was a mis-attribution.

   Lily silent = correct (form-gated on `G[0xa57/0xa58/0xa59]`, unseeded). `play-sound-effect` (0xb4, argc 2)
   left stubbed — arg roles unconfirmed. See `docs/phase-a-slice-plan.md` (A2b-Audio). Diagnostic: `Age.Cli
   audio <SCENE>`.
5. **Movies** (`OP`/`MVB`, MPEG) — a separate video-playback path; deferred.

## Validation reality (why this is the big haul)

Unlike the VM/dialogue work (byte-exact trace oracle), graphics + audio have **no machine oracle**.
Validation is: **Frida ground truth** (what the real game loads/plays for a scene) as the correctness
anchor, plus **human eyeball/ear**. Treat every mapping as provisional until Frida-confirmed; the
`resId→file` map is *data we curate against ground truth*, and the engine stays honest by only ever
rendering what the executed bytecode + the map produce (never a hardcoded image).

## Status

A2b-background: **steps 1–3 landed.** Step 1 = `build/asset-index.json`. Step 2 = **`resId →
files[section_base(scene) + resId]`** via SYS4INI per-scene sections (`tools/resolve_asset.py` +
`build/asset-sections.json`) — no runtime capture, all archives/types + audio. Step 3 = **first-pass
render** (ResourceMap + GodotAdvHost texture ops → TextureRect compositing): the full-screen event-CG
layer renders end-to-end from the bytecode. Remaining (next chunk): the **graphics geometry/blend
subsystem** — native geometry ops (`0x208` + sprite position/animation) so sprites/`BG*` position, plus
alpha/blend for fades + chromakey. See `docs/phase-a-slice-plan.md` (A2b). Audio (step 4): **`play-voice`
uses the manifest** (`files[base+id]`); **`play-bgm` uses direct names** (`BGM{id:03d}.OGG`) — NOT unified.

## Native SFX resource proof (2026-07-11)

SFX uses the same scene-local rule as graphics and voice: `files[section_base(scene)+resource_id]`.
The matching native trace at SC0000 `0xc29` captures resource `0x28`, channel 0; static resolution yields
`DATA1/E0808.WAV`, and the port trace resolves the same file. The following `0xc31` preload uses the same
resource on native secondary channel 4. `play-bgm` remains the separate direct-name exception.

The current Phase-A backend deliberately continues through the extracted-file bootstrap: `ResourceMap.AudioPath`
accepts both OGG and WAV and Godot loads the WAV bytes into its fixed SC0000 channel pool. This does not change
the scoped VFS plan below: ALF/AAI mounting and in-process asset reads remain a separate foundation track.

## Runtime asset-VFS track (VFS-A complete 2026-07-11)

The pre-extracted tree and `build/textures/*.BMP` pipeline were a Phase-A bootstrap, not the desired final
runtime. The native-compatible target is a read-only virtual filesystem that preserves AGE's translation/mod
behavior:

> resolve the resource record → try a loose file with that record's name in the game/mod root → otherwise
> read exactly `offset..offset+size` from the record's ALF → decode the contained format in process.

Native evidence already proves this ordering for scripts: `resource_open_by_raw_id@0x44f390` indexes the
80-byte SYS4 record and calls `CreateFileA(record.name)` before opening `record.archive`, seeking to
`record.offset`, and reading `record.size`. The same service is the correct common seam for scripts,
graphics, voice/SFX, and later movie bytes. Resolution and opening must remain separate: scene-local ids and
universal `raw_index` ids select a record differently, but both records flow through the same loose-first
store.

### Proposed layers

1. **Catalog + read-only ALF store (VFS-A DONE).** `Sys4AssetCatalog` parses SYS4INI at runtime while preserving all 13208 raw records
   (including the two `@` placeholders), archive names, scene sections, and the existing three lookup modes:
   universal raw id, scene-local manifest id, and direct name where the opcode family genuinely uses one.
   An ALF is a payload container at this layer: open the named archive and return a bounded stream/byte range
   at the indexed offset/size. Before that fallback, probe the configured loose override roots by the record's
   exact basename. `Sys4AssetStore` opens a separate read-only file handle per request and constrains archive
   seek/read operations to the record range. `Sys4ScriptProvider` now loads both root scenes and nested
   `call-script` targets through this seam; `ResourceMap` uses the same live catalog. `build/asset-index.json`,
   `build/asset-sections.json`, `build/callscript-names.json`, and `extracted/` are validation/temporary
   graphics-audio artifacts, not script-runtime dependencies.
2. **AAI append mount.** Parse the installed `APPEND01.AAI` (`S4AC422`) and its paired `APPEND01.ALF` with
   the same catalog abstractions. First prove whether Himegari joins append records by a separate pack/tag,
   by name replacement, or by another table selected by the native high-byte-id path; do not invent mount
   precedence. Validate every parsed append entry against `BinExtractALF.exe` output before exposing it to
   the runtime.
3. **AGF decoder.** Decode an opened AGF stream directly to width/height + RGBA8. The MIT-licensed GARbro
   `ArcFormats/Eushully/ImageAGF.cs` provides a compact reference: `ACGF` (or zero) signature, type 1/2,
   LZSS-or-raw header section, 4/8/truecolor source pixels, LZSS-or-raw pixel section, bottom-up row/stride
   conversion, and optional `ACIF` LZSS alpha plane. Port only the algorithm and attribution into
   platform-neutral .NET code; do not carry GARbro's WPF/GameRes dependencies. Kelebek's extractor and the
   on-disk `BinExtractALF.exe` are validation references; the Kelebek repository exposes no clear license,
   so its code should not be copied without clarification.
4. **Runtime consumers.** Script loading is complete. Next make texture surfaces own
   decoded RGBA pixels rather than BMP paths, and load OGG/WAV from store bytes. Migrate one consumer at a
   time; retain extraction/conversion tools as diagnostics until parity is established.

### Acceptance gates

- Catalog: 13208 raw slots / 13206 real base entries; every ALF range is in bounds; scene-local mappings
  remain identical to the current resolver and `raw_index 0x337e` resolves to `SO001.AGF`.
- Store: representative base reads are byte-identical to `extracted/`; a temporary loose file with the same
  record name wins, and removing it deterministically reveals the archive bytes. Root path traversal is
  rejected and archive reads are bounded/thread-safe.
- AAI: entry names/counts/ranges and representative bytes match a disposable `BinExtractALF APPEND01.AAI`
  extraction; the native mount/selection rule is documented before integration.
- AGF: a sample matrix covers compressed/uncompressed sections, 4/8/24-bit source pixels, type 1/2, and
  alpha/no-alpha. Decoded dimensions and RGBA hashes/pixels match GARbro or `AGF2BMP2AGF`; `SO001.AGF`
  specifically decodes as 800×300 with its alpha plane intact.
- End to end: SC0000 can run without consulting `extracted/` or `build/textures`; slot 17 receives SO001,
  the translucent textbox/button chrome appears, root `.BIN` overrides still win, and the standard VM/Godot
  validation matrix remains green.

VFS-A passes these bounded gates in `Sys4AssetStoreTests`: every catalog field matches the generated
diagnostic index, all 136 scene views match the generated section oracle without cross-section spill, all
13206 archive ranges fit, `raw_index 0x337e` is `SO001.AGF`, representative payloads from every base archive
are byte-identical to `extracted/`, and synthetic removal of a loose override reveals the bounded ALF bytes.
Traversal, past-range seek/read, and concurrent reads are covered. Installed override enumeration corrected
an older inventory error: this tree contains 51 loose root BINs, comprising **49 archive-backed v1.03 script
overrides** (all byte-proven to win and differ from DATA1) plus root-only `SYS4INI.BIN` and `SYS4AB.BIN`.
There are not 52 archive copies available to shadow. APPEND01/AAI, AGF decode, audio consumers, and movie
`0x236` remain unimplemented by design.

### Deliberate non-goals

- Writing/repacking ALF or AAI; loose overrides already provide the native mod/translation workflow.
- AGF encoding, movie/video decoding for MPEG-like `OP/MVB*.AGF`, or SFX channel semantics.
- A generalized multi-mod dependency manager. Start with native game-root loose overrides; configurable
  ordered mod roots can be layered onto the same store later.
- Removing the extraction/conversion tools immediately. They remain independent parity oracles until the
  runtime readers have broad corpus coverage.

Primary implementation reference: [GARbro's Eushully AGF reader](https://github.com/morkt/GARbro/blob/master/ArcFormats/Eushully/ImageAGF.cs)
and [ALF reader](https://github.com/morkt/GARbro/blob/master/ArcFormats/Eushully/ArcALF.cs), MIT licensed.
Secondary validation reference: [Kelebek's extractor](https://github.com/Kelebek1/Eushully-Decompiler/blob/master/extract_alf.py).