Files
OpenMaidEngine/docs/asset-resolution-re.md
2026-08-15 09:49:04 -04:00

40 KiB
Raw Permalink Blame History

Asset Resolution — foundational RE (graphics + audio)

The problem. The bytecode loads assets by a small numeric resource id (set-texture 0x23, play-voice N, …). To render/play the real asset — driven by the bytecode, not hardcoded — the engine must resolve resId → asset file. This is foundational (nearly all visuals + all audio depend on it) and not machine-verifiable (no pixel/audio oracle), which makes it the largest, highest-risk area of the port. This doc is the steering state; it feeds the A2b render/audio slices.

What's already landed

  • Graphics ops wired (A2b-background, engine-driven): create-texture 0x1f8 (slot,w,h), set-texture 0x1f9 (resId,slot), draw-texture 0x1fb (slot,x,y,w,h) promoted from VM stubs to typed IHost methods; CaptureHost no-ops them (A1 trace-diff/A2a selftest stay green). The VM now drives graphics; only resolution + backend rendering remain.
  • Audio ops WIRED (2026-07-06): play-bgm 0xbf / play-voice 0xc4IHost.PlayBgm/PlayVoice → same resolver → OGG via Godot AudioStreamPlayer. See step 4 below.
  • Tools: tools/convert_agf.py (AGF→BMP for stills via AGF2BMP2AGF.exe); tools/frida/ (runtime capture harness — see its README); Frida core installed (17.15.3).

Findings (2026-07-06)

  • SC0000 background = a slot-0 full-screen slideshow. The intro loads ~30 distinct full-screen images into slot 0 in order (set-texture 0x23→0, 0x25→0, 0x27→0, …), each drawn 800×600. Res 0x23 is the first. (There is also a persistent full-screen slot 3 set cross-context, not in SC0000 — inherited from the parent/system scene.)
  • The opening mixes movies + stills. AGF2BMP2AGF reports OP.AGF/MVB*.AGF as "unsupported type (possibly MPEG)" → DATA5 MVB* (210) and OP/ED are movies, not stills. The opening's visible background did not match any EV001* still (confirmed by eye), so res 0x23's file is not obvious from the name space alone — resolution is required.
  • Asset name spaces: DATA2 = EV*/EVM* stills (985). DATA5 = MVB*/OP/ED movies (210). DATA3 = .OGG audio (BGM*, ANA* voice).
  • The resolution chain is opaque statically. CGINIT (build/data/CGINIT.json) is a 925-column numeric record table (row-major, sparse) — not an id→filename map. SYS4INI.BIN (magic S4IC422) is the authoritative asset index the game + BinExtractALF use (name ↔ archive ↔ offset ↔ size). Filenames aren't plain ASCII because the whole directory is LZSS-compressed (not encrypted). DONE (2026-07-06): tools/parse_sys4ini.py parses it → build/asset-index.json (13206 entries). See step 1 below.
  • Frida file-I/O is noisy. ReadFile hooks on DATA2.ALF capture reads during the opening, but the offsets/spans don't line up with extracted AGF sizes → the game likely memory-maps the archives (so ReadFile offsets are OS paging, not clean per-asset loads) and/or uses async reads. The robust hook is the game's internal load-by-id function, not file I/O.

The RE plan (ordered)

  1. Parse SYS4INI (S4IC422) → an asset index {name, archive, offset, size}. DONE (2026-07-06). tools/parse_sys4ini.pybuild/asset-index.json: 5 archives (DATA15), 13206 real entries (2 @ placeholders skipped). Format: uint32 packed_size @0x134, then an LZSS stream at 0x138 running to EOF (GARbro-style: 0x1000 zero-filled ring buffer, init pos 0xFEE, control bits LSB→MSB, 1=literal / 0=two-byte backref off=(hi&0xf0)<<4|lo, len=3+(hi&0xf)). Decompresses to uint32 arc_count, arc_count × char[256] archive names, uint32 file_count, then file_count × 80-byte records {char name[64]; u32 arc_id, file_number, offset, size}. Validated: decompressed length (1058783) equals the stored size dword at 0x12c; per-archive counts match the extracted/ ground truth exactly (DATA2=985, DATA3=39, DATA4=9733, DATA5=210); all 13206 offset+size fit inside their real .ALF; 837 name-matched files → 0 size mismatches. files[] preserves directory order (feeds step 2's order-correlation). Re-run: py -3.11 -X utf8 tools/parse_sys4ini.py --check. (Ref: asmodean's exs4alf / GARbro Eushully ArcALF.cs.)

  2. Resolve resource_id → asset file. NATIVE RULE CONFIRMED IN GHIDRA AND IMPLEMENTED (2026-07-21): resource operands are universal packed SYS4INI/AAI ids. There is no scene-relative path or fallback.

    asset_catalog_parse_base_tables@0x44e7e0 creates one flat base entry array in serialized SYS4INI order. asset_open_indexed_entry@0x44f390 receives the operand unchanged. If its high byte is zero, native bounds-checks and directly indexes that base array. Otherwise the signed high byte selects a mounted AAI catalog and the low 24 bits index only that catalog. The selected record then follows exact loose-basename-first, bounded-ALF-second opening. The opener receives no executing-script identity, section base, or active manifest and contains no fallback branch.

    This one contract is shared by script loading, set-texture (0x1f9), mode-1 texture load (0x249), voice, SFX, cursor, and both movie paths. Type-specific facades should filter the selected packed record after lookup, not reinterpret the numeric id. play-bgm remains the separate direct-name exception: BGM{id:03d}.OGG.

    SC0010 provides the decisive static check that SC0000 could not because SC0000 begins at catalog zero. SC0010 bytecode executes set-texture 0x21; raw entry 0x21 is SO013A.AGF, while adding SC0010's catalog position 0x11e lands on unrelated COL0023.OGG. Its first voices are 0x120, 0x121, and 0x122, which directly name raw LILA1414.OGG, LILB0053.OGG, and LILC0054.OGG. Their file_number values 2/3/4 describe grouping inside the SC0010 run, but native bytecode already contains the absolute indexes 0x11e + file_number.

    The former files[section_base(scene)+resId] model inverted that relationship. The 586/595 Frida correlation and the strong file_number == position - group_start pattern remain useful evidence about catalog construction/order, but they do not describe runtime resolution. SC0000 starting at zero hid the mistake, while later large ids often fell outside the invented scene range and happened to reach the port's former raw fallback. Low raw ids used from later scripts could instead be silently misresolved. tools/resolve_asset.py and build/asset-sections.json are therefore correlation/manifest-inventory diagnostics only; they must not drive runtime lookup.

    0x1f9 and 0x249 also do not represent scene-local versus raw addressing. Both pass the same packed operand unchanged to the same opener and use the same colorkey/load path. Their native distinction is surface mode: 0x1f9 creates ordinary mode 0, while 0x249 creates the tiled large-image mode 1 wrapper. FIELD's 0x32da..0x32dd map sheets and SYSTEM4's 0x337e SO001 are ordinary examples of the universal base indexes used throughout the corpus, not special fallback cases.

  3. Wire the backend. FIRST-PASS RENDER LANDED (2026-07-06). Age.Engine/Sys4/ResourceMap.cs (Resolve + BMP path) + GodotAdvHost texture ops → TextureRect compositing behind the dialogue; IHost.DrawTexture extended with dst x/y; 800×600 window; convert_agf.py --scene pre-converts a scene's manifest AGFs → BMP. The full-screen event-CG layer renders end-to-end from the executed bytecode. Historical limitations at first landing (subsequently resolved in the Phase-A graphics slices): sprites + BG* (routed through the CG-load subroutine) had garbage geometry because native graphics ops were stubbed (0x208 get-texture-size + the sprite position/animation chain); fades (AE*) drew opaque (no alpha); the slot model approximated the game's immediate-mode blit-onto-slot-0 canvas. See docs/phase-a-slice-plan.md (A2b section) for the implementation history and current retained-object model.

    System chrome ownership (resolved 2026-07-20). The normal Godot entry now runs SYSTEM4 as the live root, so its SO000/SO001 loads, ADV-layout definitions, and retained slot initialization execute through GodotAdvHost before TITLE and child scenes. Asset lookup decodes VFS-owned AGF bytes directly and does not depend on a scene-limited converted BMP. CALLBACK_WINDOW.BIN consequently inherits slot 0x11 with SO001 and can crop its textbox/buttons normally. The explicit --scene SC0000 --boot diagnostic still injects the same known inherited layout and surface state because it intentionally bypasses SYSTEM4; that shortcut is no longer the shipped/default route.

    Resource lookup is independent of the active VM frame. Every texture surface retains the packed catalog id supplied by bytecode, and nested helpers/sibling scripts use the same global base/append catalogs. Script-context bracketing remains useful for diagnostics/page location, but it must not alter texture, voice, cursor, SFX, or movie resolution.

  4. Audio. BGM/voice/SFX wired; VFS bytes complete (2026-07-11); packed-raw SFX corrected 2026-07-20. IHost.PlayBgm/PlayVoice + VM dispatch (play-bgm 0xbf / play-voice 0xc4, both argc 1); ResourceMap.ReadAudio opens the resolved catalog entry through IAssetStore; GodotAdvHost passes the bytes to Main's players (AudioStreamOggVorbis.LoadFromBuffer; BGM loops, voice interrupt-on-new). Non-Godot hosts no-op it → --selftest/8-8 byte-identical. SC0000 fires 18 BGM + 198 voice. By-ear VALIDATED (2026-07-06): voices play on their lines (play-voice med→HIGH). BUT the two audio ops use DIFFERENT addressing:

    • Voice (play-voice) → the universal packed SYS4INI/AAI resource contract shared with textures and movies. Native passes the bytecode operand unchanged to asset_open_indexed_entry.
    • BGM (play-bgm) → DIRECT LITERAL NAME, id → BGM{id:03d}.OGG (DATA3), not the packed resource table. Confirmed by ear (play-bgm 5→BGM005, 8→BGM008) and proven by play-bgm 0x23→BGM035.OGG — a real standalone track (BGM set skips 030-034) the old scene model mis-resolved to a graphics entry. Implemented as ResourceMap.ResolveBgm(id); GodotAdvHost.PlayBgm uses it. The prior "Frida-confirmed play-bgm 5→BGM006" record was a mis-attribution.

    Lily silent = correct (form-gated on G[0xa57/0xa58/0xa59], unseeded). SFX WAV entries use universal packed catalog ids through the same byte store while retaining the existing channel lifecycle. See docs/phase-a-slice-plan.md. Diagnostic: Age.Cli audio <SCENE>.

  5. Movies (OP/MVB, MPEG) — 0x236 and 0x20f receive universal packed resource ids and read the selected payload through IAssetStore; see the movie sections below.

Validation reality (why this is the big haul)

Unlike the VM/dialogue work (byte-exact trace oracle), graphics + audio have no machine oracle. Validation is: Frida ground truth (what the real game loads/plays for a scene) as the correctness anchor, plus human eyeball/ear. Treat every mapping as provisional until Frida-confirmed; the resId→file map is data we curate against ground truth, and the engine stays honest by only ever rendering what the executed bytecode + the map produce (never a hardcoded image).

Status

A2b-background: steps 13 landed; step 2's packed resolver was corrected 2026-07-21. Step 1 = build/asset-index.json. Step 2 originally normalized through inferred per-scene sections; native RE proved runtime operands are already universal packed ids, and every typed runtime facade now uses that contract. tools/resolve_asset.py and build/asset-sections.json remain grouping/correlation diagnostics, not runtime inputs. Step 3 = first-pass render (ResourceMap + GodotAdvHost texture ops → TextureRect compositing): the full-screen event-CG layer renders end-to-end from the bytecode. Remaining (next chunk): the graphics geometry/blend subsystem — native geometry ops (0x208 + sprite position/animation) so sprites/BG* position, plus alpha/blend for fades + chromakey. See docs/phase-a-slice-plan.md (A2b). Audio (step 4): voice uses the universal packed catalog; play-bgm uses direct names (BGM{id:03d}.OGG).

Native SFX resource proof (2026-07-11; addressing corrected 2026-07-20)

SFX uses the universal packed catalog rule shared by graphics/voice/movies. A zero high byte directly indexes SYS4INI; a nonzero high byte selects the matching AAI mount and uses the low 24-bit index. The matching native trace at SC0000 0xc29 captures raw id 0x28, channel 0, which is DATA1/E0808.WAV; SC0000 being the first section previously hid the distinction. TITLE makes it decisive: 0x2aea is raw SE020.WAV for hover, 0x3321 is raw SE015.WAV for activation, and neither fits TITLE's 14-entry inferred group. play-bgm remains the separate direct-name family.

The Phase-A backend now resolves the OGG/WAV catalog entry and opens it through IAssetStore; Godot decodes the returned bytes into its existing BGM, voice, and fixed SC0000 SFX channel players. The earlier ResourceMap.AudioPath extracted-file bootstrap is retired.

Godot's WAV loader adds one host-specific compatibility boundary. SC0000 0xc29 starts E0808.WAV with the EV052DA glow. Its PCM is valid, but its trailing RIFF LIST/INFO fields contain Japanese CP932 text (IPRD, IGNR, and ICMT). Godot assumes INFO text is UTF-8 and formerly emitted one Unicode warning per invalid CP932 byte each time the sound was loaded; the duplicated SC0000 burst came from loading raw id 0x28 on two channels. A complete extracted-corpus scan found 238 RIFF/WAVE files, 61 INFO chunks, and no other LIST type, explaining the same warnings around combat SFX. A later declared-extent audit corrected the earlier claim that every container ended exactly at its RIFF boundary: seven entries have trailing bytes. Six have small 74- or 234-byte tails; A1215.WAV is a 323,009-byte concatenation whose first RIFF ends at 157,940, followed by a multipart-upload header and a second complete RIFF.

The correction is deliberately confined to the Godot frontend. RiffWaveSanitizer removes only LIST chunks whose form type is INFO from the transient byte array passed to AudioStreamWav.LoadFromBuffer, updates the RIFF length, and preserves every functional chunk (fmt , data, smpl, cue , and unknown chunks) byte-for-byte, including padding. The shared engine, original loose/archive payloads, and ResourceMap.ReadAudio output remain untouched. The real E0808 Godot input shrinks from 688,570 to 688,336 bytes while retaining an identical PCM data chunk and loads headlessly without a Unicode warning. Synthetic chunk/padding tests and the installed E0808 regression cover the adapter.

A1215.WAV exposes a separate compatibility boundary. Native AGE's wav_decoder_open@0x488790 uses WinMM mmioDescend to enter the first RIFF/WAVE, finds its first data child, and streams exactly that child's declared cksize; appended bytes are never decoded. Godot's whole-buffer importer instead walks beyond the first RIFF and interprets the multipart header's Cont as a chunk id, causing repeated out-of-range seeks. RiffWaveSanitizer.PrepareForGodot now validates the first RIFF and all of its declared children, removes any INFO chunks, and returns a transient buffer ending exactly at that RIFF's declared extent. It returns the original array without rewriting when the RIFF is invalid and preserves the packed/original bytes in every case. The installed A1215 regression proves 323,009 source bytes become the exact 157,940-byte first RIFF with the 157,896-byte PCM data chunk unchanged; the Godot selftest loads that result through AudioStreamWav.LoadFromBuffer without seek errors.

Runtime asset-VFS track (VFS-A/B/C complete 2026-07-11)

The pre-extracted tree and build/textures/*.BMP pipeline were a Phase-A bootstrap, not the desired final runtime. The native-compatible target is a read-only virtual filesystem that preserves AGE's translation/mod behavior:

resolve the resource record → try a loose file with that record's name in the game/mod root → otherwise read exactly offset..offset+size from the record's ALF → decode the contained format in process.

Native evidence proves this ordering for scripts: asset_open_indexed_entry@0x44f390 indexes the 80-byte SYS4 record and calls CreateFileA(record.name) before opening record.archive, seeking to record.offset, and reading record.size. The same service is the correct common seam for scripts, graphics, voice/SFX, and movie bytes. Resolution and opening remain separate only by concern: the universal packed id selects one record, then the store applies loose-first/archive-second opening. There is no scene-local numeric addressing mode.

Native/port fallback fidelity recheck and correction (2026-08-15). A fresh /v2 decompile and disassembly comparison confirmed the normal installed-asset contract above, including base/append packed selection, exact loose basename, archive offset/size, and successful-open catalog tracking. Sys4AssetStore now also matches the two previously divergent open details: any ordinary loose-file open failure (IOException or UnauthorizedAccessException) continues through later override roots and then the archive, and both loose and archive handles request FileOptions.SequentialScan. A loose file that opens but fails later decoding remains authoritative in both implementations; decode failure never reveals the archive copy.

Native still throws for a base id outside the table, a missing signed append selector, an append id outside its table, or a selected entry that cannot be opened loose or archived. The typed port intentionally retains its safer null result for invalid, unmounted, placeholder, or wrong-type packed records. ResourceMap now emits a deduplicated diagnostic for each (resource kind, packed id, failure) through an optional host callback; Godot sends these to GD.PushWarning, and the CLI audio/gfx diagnostics send them to standard error. Thus malformed scripts or mods remain nonfatal without becoming silent.

The corresponding Ghidra helper is named asset_register_open_handle_range@0x44edf0; it stores the selected handle, current position, and readable length in the FileDB stream table.

Proposed layers

  1. Catalog + read-only ALF store (VFS-A DONE). Sys4AssetCatalog parses SYS4INI at runtime while preserving all 13208 raw records (including the two @ placeholders), archive names, diagnostic scene groupings, universal packed ids, and direct-name lookup where an opcode family genuinely uses it. An ALF is a payload container at this layer: open the named archive and return a bounded stream/byte range at the indexed offset/size. Before that fallback, probe the configured loose override roots by the record's exact basename. Sys4AssetStore opens a separate read-only file handle per request and constrains archive seek/read operations to the record range. Sys4ScriptProvider now loads both root scenes and nested call-script targets through this seam; ResourceMap uses the same live catalog. build/asset-index.json, build/asset-sections.json, build/callscript-names.json, and extracted/ are validation/temporary graphics-audio artifacts, not script-runtime dependencies.

  2. AAI append mount (VFS-B DONE). Sys4AssetCatalog also parses the installed APPEND01.AAI (S4AC422) and mounts its paired APPEND01.ALF. This is a separate selector-keyed catalog, not a filename overlay: AGE scans *.AAI, reads the selector at AAI header offset 0x108, and stores the successfully loaded catalog in mounted_aai[selector]. A resource id with a nonzero high byte selects that slot; its low 24 bits index only the selected append table. Installed APPEND01 is selector 1. Within that catalog the ordinary exact loose-record-name then ALF-range precedence still applies, so the literal $1$... names are preserved. A later successfully enumerated AAI with the same selector replaces the earlier pointer in the native loop; no base/append name replacement occurs. Native extracts the selector with arithmetic SAR 24; the port rejects sign-bit selectors rather than guessing behavior for ids that would index before AGE's mount table.

    Append bootstrap/data sequence (launch boundary verified 2026-07-24). Installed APPEND01 contains 81 records, including 39 SYS4 scripts. Record zero is $1$AUTORUN.BIN (0x01000000), a clean 43-instruction script. Its first 22 instructions call packed records 0x01000001..0x01000016 in exact order: append fragments for EBINIT, CNINIT, ITINIT, SKINIT, ILINIT, AFINIT, TRINIT, MAINIT, ALINIT, CDINIT2, MPINIT, LAINIT, OBINIT, STINIT2, RTINIT, CGINIT, SPINIT, CTINIT, CVINIT, CIINIT, VIINIT, and SCINIT. It then marks append state and installs $1$CCINIT.BIN at 0x01000017, $1$SCJUMP.BIN at 0x01000018, and the packed READY/CLOSE/ROUND and message handlers through 0x01000029 into existing dispatch globals. The called INIT fragments populate references to the append graphics and six append scenario scripts.

    These INIT fragments are additive bytecode, not same-name catalog overlays. In particular, $1$EBINIT.BIN (0x01000001) writes directly into the established EBINIT global arrays and adds sparse unit-definition rows 81 and 900..905; $1$CNINIT.BIN adds their display names and voice-family mappings. Mounting an AAI therefore only makes its packed records addressable. Executing its AUTORUN is the separate operation that materializes append data and registers append control flow.

    Tooling can inspect one packed INIT fragment without claiming that AUTORUN has executed: extract_init.py EBINIT --packed-id 0x01000001 resolves the selector/index through the AAI catalog, reads the bounded ALF payload in memory, and applies the base EBINIT geometry to its additive writes. The separate APPEND01-EBINIT result preserves packed provenance and can be queried by unit id or name with init_table_profile.py --record; it is not an ordered merged runtime image.

    The base/loose script corpus contains no explicit reference to packed id 0x01000000 because INIT2 invokes native opcode 0x143 once at 0x17f. Its handler scans mounted selectors 1..255 in ascending order and queues (selector << 24) | 0, selecting record zero without a filename lookup. The engine executes those queued scripts serially before resuming INIT2. This occurs after INIT2's 23 base initializer calls and registry setup, and immediately before TUNE, proving base-then-append patch order. The port now implements op 0x143 over the mounted-selector view exposed by Sys4ScriptProvider, snapshots and sorts the selectors, constructs each packed record-zero id, and executes the scripts serially through normal VM frames. Natural SYSTEM4 boot regression-proves BTANINIT2 -> $1$AUTORUN.BIN -> $1$EBINIT.BIN -> TUNE.BIN; VFS mounting and append patch application are therefore both active. Native queue/frame details are canonical in docs/engine-re.md.

  3. AGF decoder (VFS-C DONE). Age.Engine/Sys4/AgfDecoder.cs decodes an opened AGF payload directly to a tightly packed, top-down width/height + RGBA8 surface. The MIT-licensed GARbro ArcFormats/Eushully/ImageAGF.cs provides a compact reference: ACGF (or zero) signature, type 1/2, LZSS-or-raw header section, 4/8/truecolor source pixels, LZSS-or-raw pixel section, bottom-up row/stride conversion, and optional ACIF LZSS alpha plane. Port only the algorithm and attribution into platform-neutral .NET code; do not carry GARbro's WPF/GameRes dependencies. Kelebek's extractor and the separately obtained local BinExtractALF.exe is a validation reference; the Kelebek repository exposes no clear license, so its code should not be copied without clarification. The focused LzssDecoder is shared with Sys4AssetCatalog; raw and compressed information/pixel/ACIF sections use the same bounded primitive.

  4. Runtime consumers (packed-id correction complete 2026-07-21). Script loading, textures, voice, SFX, cursors, and modal/non-modal movies all select through ResolvePacked before type filtering. The VM retains bytecode operands unchanged instead of asking the host for scene normalization. Godot caches decoded RGBA and movie identities by the full packed id, preserving AAI selectors rather than colliding with equal low indexes in the base catalog. It supplies synchronous dimensions to opcode 0x208 and no longer reads build/textures/*.BMP. BGM remains direct-name. Focused regressions cover SC0010's low 0x21 texture and 0x120..0x122 voices plus append-pack identity. Extraction, grouping, and conversion tools remain diagnostics.

Acceptance gates

  • Catalog: 13208 raw slots / 13206 real base entries; every ALF range is in bounds; packed base/append selection matches native and raw_index 0x337e resolves to SO001.AGF.
  • Store: representative base reads are byte-identical to extracted/; a temporary loose file with the same record name wins, and removing it deterministically reveals the archive bytes. Root path traversal is rejected and archive reads are bounded/thread-safe.
  • AAI: entry names/counts/ranges and representative bytes match a disposable BinExtractALF APPEND01.AAI extraction; the native mount/selection rule is documented before integration.
  • AGF: a sample matrix covers compressed/uncompressed sections, 4/8/24-bit source pixels, type 1/2, and alpha/no-alpha. Decoded dimensions and RGBA hashes/pixels match GARbro or AGF2BMP2AGF; SO001.AGF specifically decodes as 800×300 with its alpha plane intact.
  • End to end: SC0000 can run without consulting extracted/ or build/textures; slot 17 receives SO001, the translucent textbox/button chrome appears, root .BIN overrides still win, and the standard VM/Godot validation matrix remains green.

VFS-A passes these bounded gates in Sys4AssetStoreTests: every catalog field matches the generated diagnostic index, all 136 scene views match the generated section oracle without cross-section spill, all 13206 archive ranges fit, raw_index 0x337e is SO001.AGF, representative payloads from every base archive are byte-identical to extracted/, and synthetic removal of a loose override reveals the bounded ALF bytes. Traversal, past-range seek/read, and concurrent reads are covered. Installed override enumeration corrected an older inventory error: this tree contains 51 loose root BINs, comprising 49 archive-backed v1.03 script overrides (all byte-proven to win and differ from DATA1) plus root-only SYS4INI.BIN and SYS4AB.BIN. There are not 52 archive copies available to shadow.

VFS-C passes its bounded gates in AgfDecoderTests: synthesized fixtures cover raw/compressed sections, 4/8-bit palettes, 24/32-bit truecolor expansion, padded bottom-up rows, type 1/2, and ACIF/no-ACIF alpha. Five installed assets spanning raw/compressed metadata and pixel/alpha combinations match the existing AGF2BMP2AGF BMP oracle pixel-for-pixel. SO001 resolves through universal raw id 0x337e, decodes to 800×300 with intermediate alpha values, and is inherited in surface slot 17 before SC0000. A windowed page-1 capture with build/textures/ moved aside showed the translucent textbox edge and bottom-right controls. Texture runtime no longer consults extracted/ or build/textures/.

Audio byte migration passes its bounded gate in Sys4AssetStoreTests: archive-only SC0000 reads identify BGM005.OGG and MAN999.OGG as Ogg streams and E0808.WAV as RIFF/WAVE without consulting extracted/. A windowed SC0000 run with extracted/ moved aside crossed both voice sites and the first SFX sequence, recording BGM005 plus E0808.WAV load/start/preload on channels 0/0/4 with no Godot OGG/WAV decode errors. Channel, loop, interruption, timing, fade, load/start, and release behavior is unchanged.

Windows cursor payloads (implemented 2026-07-21)

Cursor artwork is catalog-backed, not embedded in AGE.EXE. The eight ADV edge cursors at raw ids 0x3318..0x331f are 326-byte, uncompressed 1-bpp 32x32 Windows CUR payloads. FIELD's drag-pan callback instead loads raw id 0x32ce, catalog entry CURSOR09.CUR at DATA1.ALF:611394962+766. Its CUR directory and DIB header specify one 32x32 image, hotspot (16,14), 4 bits per pixel, no compression, and a default 16-entry BGRA palette. The 744-byte image consists of the 40-byte BITMAPINFOHEADER, 64-byte palette, 512-byte color/XOR bitmap, and 128-byte 1-bpp AND mask. The original runtime decoder accepted only 1-bpp XOR data and therefore rejected this valid color cursor. CurDecoder now accepts the installed uncompressed 1-bpp and 4-bpp variants, derives each palette size from the DIB metadata, calculates separate DWORD-aligned XOR and AND strides, expands packed high-nibble-first 4-bpp indices through the BGRA palette, and retains the separate 1-bpp AND transparency mask. The installed-asset regression checks CURSOR09's dimensions, hotspot, transparent background, grayscale, and a non-gray palette pixel.

SC0000 movie payload and presentation (2026-07-11)

The first implementation was validated at SC0000 0x236@0x13c8. Universal base-catalog id 0x33 resolves through the authoritative catalog to DATA1.ALF:CHAPTER.AGF (archive offset 3,908,816; size 8,194,052). Despite the .AGF name, its payload begins with MPEG program-stream pack start code 00 00 01 BA; the installed asset is MPEG-1 program stream video at 800x600, 29.97 fps, approximately 11.98 seconds, with video stream E0 and audio stream C0.

ResourceMap.ReadMovie accepts the resolved AssetEntry, reads it through the injected IAssetStore, and rejects a non-MPEG-pack payload. Godot has no built-in MPEG decoder suitable for this asset, so the Windows backend adapts those already-owned VFS bytes to the system DirectShow MPEG source using a private temporary file. DirectShow decodes RGB32 on an MTA thread; the sample-grabber callback converts bottom-up BGRA to top-down RGBA and publishes only the newest frame to the retained compositor surface. The temporary file is an implementation adapter, never an alternate resolver or dependency on extracted/. The audio pin is intentionally left unrendered for this bounded slice.

Archive-only tests verify the exact catalog entry, payload size/header, and an actual decoded 800x600 RGBA frame; both still pass with the entire extracted/ tree physically moved aside and restored afterward. SC0000's native capture verifies operands (0x33, 0, 2, 0) and immediate VM continuation at 0x13d1. In Godot the real site opens the same 8,194,052 VFS bytes, publishes changing 800x600 frames, and retains them across pre-yield static surface preparation. The later 0x21c presentation service remains parked until DirectShow EOF, then scene cleanup stops the movie. The initial invisible result was compositor-only: the static (assetId,colorKey) image cache froze the first movie sample, while an extra current-sample background copy was covered by the correctly positioned retained movie object. Dynamic movie surfaces now bypass that cache and publish only at their retained z-position. A windowed run reached first frame 101 and stop frame 190, and manual observation confirmed visible changing video. That milestone left movie audio unrendered; the synchronized FFmpeg/Godot audio path landed later as described below. The real-scene trace remains a separate extracted-present test because the current Paths.Scripts() test bootstrap still locates its root *.BIN fixtures there; migrating that test/bootstrap path is unrelated to movie asset loading and was not folded into this slice.

BTL combat-effect movie resolution and decoder boundary (diagnosed 2026-07-21)

BTL's movie call at BTL.BIN@0x2b21 is the same non-modal opcode 0x236 and, like every native resource consumer, its table supplies universal packed SYS4INI ids. The first accepted combat run reached five such ids:

Raw id Catalog asset MPEG size Sequence size
0x2bd8 MVB958.AGF 40,964 bytes 280x352
0x2af1 MVB001.AGF 90,116 bytes 280x352
0x2bde MVB955.AGF 22,532 bytes 280x352
0x2af5 MVB004.AGF 69,636 bytes 280x352
0x2bca MVB914.AGF 133,124 bytes 400x400

All five archive payloads begin with MPEG program-stream pack code 00 00 01 BA; none is a corrupt still AGF. Universal packed resolution was implemented across texture, voice, and both movie paths on 2026-07-21. The following manual combat run produced no movie unresolved warnings: the BTL ids reached ResourceMap.ResolveMovie, VFS reads, and the movie backend as native requires.

Resolution is not the only blocker. An archive-backed probe of the actual Windows backend found that the current DirectShow graph decodes MVB914 and exposes a positive stop time, but rejects all four 280x352 assets while connecting the MPEG video decoder with HRESULT 0x80040217. Wider probes establish the boundary in the current filter stack: 208/288/304/400/800-pixel widths decode, while 280/360/520/600-pixel widths fail; the latter are all 8 mod 16. This is not a rare content edge: 125 installed MVB*.AGF files are 280x352. Requesting RGB24 instead of RGB32 does not change negotiation. The evidence localizes the failure to DirectShow/filter compatibility with these non-16-aligned MPEG display widths; it does not prove which internal stock filter imposes the restriction.

The resolver-only acceptance run makes that split visible in one exchange. MVB961 (0x2be3, 112,644 bytes) and MVB238 (0x2b94, 143,364 bytes) both declare 280x352 and fail at IGraphBuilder.Connect(sourceOut, sampleGrabberIn) with 0x80040217. In the same run MVB908 (0x2bc2, 126,980 bytes) declares 400x400, starts with a 333 ms stop time, publishes RGBA frames, reaches completion, and is stopped by script cleanup. This is decisive backend evidence rather than a resolver, VFS, signature, or corrupt-asset problem.

The user nevertheless observed the combat presentation stall after that sequence. Static BTL tracing rules out 0x23f == -1 as an infinite-loop mechanism by itself. local 0x44, the total effect horizon, is first set to base_time + 1000 at 0x2230; each effect can only extend that maximum with effect_start + duration at 0x2b64..0x2b8b. A failed duration of -1 can understate that one extension but cannot make the callback count negative. After callbacks, 0x2492..0x2515 independently scans movie surfaces 7..10 through 0x23a, sleeping 16 ms only while one reports active. The successful movie stopped MVB908 line is emitted by the subsequent 0x2518 cleanup, proving that explicit movie wait exited in the logged run. The final stall therefore was not localized; it may be after movie cleanup.

The port now nevertheless makes decoder failure safe and deterministic. A valid 0x236 movie whose host backend cannot initialize is marked completed explicitly and receives stop time 0, meaning an immediate effect, rather than the former diagnostic -1. Empty movie slots retain native 0x23f == -1. Successfully started decoders also carry a completion watchdog: the larger of five seconds or reported stop time plus two seconds (capped at five minutes), with a 30-second default when timing is unavailable. Expiry forces the same completed state so a backend that starts but never signals EOF cannot hold 0x21c indefinitely. If the reported combat stall survives this guard, capture the VM/service coordinate after 0x2518 and treat it as a separate BTL timed-presentation bug.

The FFmpeg replacement and destination-surface work are recorded in docs/platform-portability.md and docs/phase-b-framework.md. After video/audio corpus validation and clean audible LOGO/OP/CHAPTER acceptance, the DirectShow/temp-file adapter was deleted; packed MPEG decode now has one runtime path. Movie identity also remains typed across the entire retained-surface lifecycle. The packed catalog is immutable while mounted, so once an id enters the movie path its .AGF-named MPEG payload cannot fall through to the ACGF still decoder before the first frame or after the last live binding is detached. The latter guard closes a real cross-thread cleanup window: BTL clears GfxState and releases the host surface in one VM instruction, while the compositor may hold a snapshot from immediately before that pair. Without remembered movie typing, the snapshot briefly attempted to decode MVB955/MVB004/MVB914 as still images after successful playback and printed false expected an ACGF image warnings. This does not retain a frame or playback instance; it only prevents an immutable movie resource from being reclassified.

Modal startup/ending movie resources (implemented 2026-07-20)

Opcode 0x20f uses the same universal packed resource contract as 0x236. Its complete corpus is LOGO.BIN (0x335f,42,4), OP.BIN (0x3364,42,4), and ED.BIN (0x3324,42,dynamic_flags). Base records 0x335f, 0x3364, and 0x3324 are respectively LOGO.AGF, OP.AGF, and ED.AGF; all begin with MPEG pack code 00 00 01 BA. Native 0x20f also arms modal run-state 0x2000; unlike 0x236, these six-instruction wrapper scripts depend on the movie service itself to park until EOF/input cancellation before they release surface 42.

ResourceMap.ResolveMovie supplies that typed universal lookup, while ReadMovie remains the MPEG signature gate. IHost.PlayModalMovieToSurface is distinct from the non-modal call for lifecycle only: Godot reuses the asynchronous FFmpeg decoder and retained compositor but parks the VM thread until EOF or mouse/Accept/Cancel input. The wrapper's following release then tears down the completed/cancelled movie. Native attaches the movie renderer to the already-created target without replacing that surface's resource identity or color key. The port now does the same: MovieSurfaceRegistry resolves the live instance pixels while the VM retains the mutable surface's resourceId=0, colorKey=-1. The former synthetic colorKey=0 binding made exact-black MPEG pixels transparent and was not part of the native asset-resolution contract. The FFmpeg backend now renders embedded audio through an engine-owned synchronized path rather than an unmanaged default-device side path. Timestamped stereo float PCM feeds a per-playback Godot AudioStreamGenerator; the sound-hardware position drives video presentation, and modal cancellation or surface cleanup releases both sides together.

VFS-B passes its bounded gates in Sys4AssetStoreTests: the installed AAI expands from the LZSS stream at 0x118 (expanded size at 0x110, packed size at 0x114) to one APPEND01.ALF archive and 81 80-byte records. All records carry selector 1 and literal $1$ names. The full directory has stable SHA-256 23F0C104A45C099CEFB7D333362716EDE6F20B9EC53E4C3705A8E3A87063708E over its ordered record fields. The private installed-data integration gate can run a machine-local bin/BinExtractALF.exe into a disposable directory, compares all 81 names, validates every range and size, and byte-compares all 81 payloads. Sys4ScriptProvider resolves a real append script through 0x01xxxxxx; direct base-name lookup deliberately does not see append records.

Deliberate non-goals

  • Writing/repacking ALF or AAI; loose overrides already provide the native mod/translation workflow.
  • AGF/MPEG encoding. Runtime MPEG decode and synchronized movie audio now live behind the portable FFmpeg boundary documented in docs/platform-portability.md; authoring or repacking those streams remains out of scope.
  • A generalized multi-mod dependency manager. Start with native game-root loose overrides; configurable ordered mod roots can be layered onto the same store later.
  • Removing the extraction/conversion tools immediately. They remain independent parity oracles until the runtime readers have broad corpus coverage.

Primary implementation reference: GARbro's Eushully AGF reader and ALF reader, MIT licensed. Secondary validation reference: Kelebek's extractor.