Implement base SYS4 asset store

docs/PROJECT-STRUCTURE.md

@@ -16,9 +16,9 @@ S:\Game Hacking\Eushully\Himegari\           ← workspace root (three siblings)
 │   ├── AGE.EXE, AGERC.DLL, *.dll             shipped engine (packed). Stays intact and
 │   │                                         runnable in place — Frida launches it if needed.
 │   ├── DATA1-5.ALF, APPEND01.ALF/.AAI        shipped archives (~2.3 GB).
-│   ├── *.BIN                                 52 loose patch-override scripts (v1.03) —
-│   │                                         AUTHORITATIVE over their DATA1 copies. Plus
-│   │                                         non-script indices (SYS4INI=S4IC, SYS4AB=S4AB).
+│   ├── *.BIN                                 49 loose patch-override scripts (v1.03) —
+│   │                                         AUTHORITATIVE over DATA1. Plus two root-only
+│   │                                         engine files (SYS4INI=S4IC, SYS4AB=S4AB).
 │   └── *.exe (uninstallers), SAS0099.OGG …   other shipped files.
 │
 ├── extracted/                               ← DERIVED (game-side) — extracted ALF contents,
@@ -96,6 +96,8 @@ S:\Game Hacking\Eushully\Himegari\           ← workspace root (three siblings)
     │   └── manifest.json, opcode-coverage.md   (opcode-coverage.md GENERATED from opcodes.toml)
     │
     ├── engine/                              DELIVERABLE — the .NET VM core (AgeEngine.sln: Age.Engine / Age.Cli / tests)
+    │   └── Age.Engine/Sys4/                 runtime catalog parser, loose-first bounded ALF asset store,
+    │                                       script provider, and temporary resource facade
     ├── tools/frida/                         runtime-capture + engine-dump scripts (see tools/frida/README.md)
     └── godot/                               DELIVERABLE — the Godot/C# ADV front-end (references Age.Engine)
 ```

docs/asset-resolution-re.md

@@ -166,7 +166,7 @@ The current Phase-A backend deliberately continues through the extracted-file bo
 accepts both OGG and WAV and Godot loads the WAV bytes into its fixed SC0000 channel pool. This does not change
 the scoped VFS plan below: ALF/AAI mounting and in-process asset reads remain a separate foundation track.
 
-## Candidate runtime asset-VFS track (scoped 2026-07-10; not started)
+## Runtime asset-VFS track (VFS-A complete 2026-07-11)
 
 The pre-extracted tree and `build/textures/*.BMP` pipeline were a Phase-A bootstrap, not the desired final
 runtime. The native-compatible target is a read-only virtual filesystem that preserves AGE's translation/mod
@@ -184,12 +184,16 @@ store.
 
 ### Proposed layers
 
-1. **Catalog + read-only ALF store.** Parse SYS4INI at runtime while preserving all 13208 raw records
+1. **Catalog + read-only ALF store (VFS-A DONE).** `Sys4AssetCatalog` parses SYS4INI at runtime while preserving all 13208 raw records
    (including the two `@` placeholders), archive names, scene sections, and the existing three lookup modes:
    universal raw id, scene-local manifest id, and direct name where the opcode family genuinely uses one.
    An ALF is a payload container at this layer: open the named archive and return a bounded stream/byte range
    at the indexed offset/size. Before that fallback, probe the configured loose override roots by the record's
-   exact basename. Keep `build/asset-index.json` as a diagnostic artifact, not a runtime dependency.
+   exact basename. `Sys4AssetStore` opens a separate read-only file handle per request and constrains archive
+   seek/read operations to the record range. `Sys4ScriptProvider` now loads both root scenes and nested
+   `call-script` targets through this seam; `ResourceMap` uses the same live catalog. `build/asset-index.json`,
+   `build/asset-sections.json`, `build/callscript-names.json`, and `extracted/` are validation/temporary
+   graphics-audio artifacts, not script-runtime dependencies.
 2. **AAI append mount.** Parse the installed `APPEND01.AAI` (`S4AC422`) and its paired `APPEND01.ALF` with
    the same catalog abstractions. First prove whether Himegari joins append records by a separate pack/tag,
    by name replacement, or by another table selected by the native high-byte-id path; do not invent mount
@@ -202,7 +206,7 @@ store.
    platform-neutral .NET code; do not carry GARbro's WPF/GameRes dependencies. Kelebek's extractor and the
    on-disk `BinExtractALF.exe` are validation references; the Kelebek repository exposes no clear license,
    so its code should not be copied without clarification.
-4. **Runtime consumers.** Make script loading accept streams/bytes from the store, make texture surfaces own
+4. **Runtime consumers.** Script loading is complete. Next make texture surfaces own
    decoded RGBA pixels rather than BMP paths, and load OGG/WAV from store bytes. Migrate one consumer at a
    time; retain extraction/conversion tools as diagnostics until parity is established.
 
@@ -222,6 +226,16 @@ store.
   the translucent textbox/button chrome appears, root `.BIN` overrides still win, and the standard VM/Godot
   validation matrix remains green.
 
+VFS-A passes these bounded gates in `Sys4AssetStoreTests`: every catalog field matches the generated
+diagnostic index, all 136 scene views match the generated section oracle without cross-section spill, all
+13206 archive ranges fit, `raw_index 0x337e` is `SO001.AGF`, representative payloads from every base archive
+are byte-identical to `extracted/`, and synthetic removal of a loose override reveals the bounded ALF bytes.
+Traversal, past-range seek/read, and concurrent reads are covered. Installed override enumeration corrected
+an older inventory error: this tree contains 51 loose root BINs, comprising **49 archive-backed v1.03 script
+overrides** (all byte-proven to win and differ from DATA1) plus root-only `SYS4INI.BIN` and `SYS4AB.BIN`.
+There are not 52 archive copies available to shadow. APPEND01/AAI, AGF decode, audio consumers, and movie
+`0x236` remain unimplemented by design.
+
 ### Deliberate non-goals
 
 - Writing/repacking ALF or AAI; loose overrides already provide the native mod/translation workflow.

docs/himegari-port-reference.md

@@ -93,10 +93,10 @@ save-format work remain deferred.
 ### Immediate (no tools needed beyond what's on disk)
 1. ~~**Relocate the `Output\` tree**~~ **DONE** — workspace now at `S:\Game Hacking\Eushully\Himegari\姫狩りダンジョンマイスター\`.
 2. **Convert remaining AGFs** in DATA2 (985 files) and DATA5 (210 files) with `AGF2BMP2AGF.exe`. *(Deferred — graphics not needed yet.)* The 3-file DATA1 gap is `CHAPTER.AGF`, `LOGO.AGF`, `TEST.AGF`.
-3. ~~**Inventory the script files**~~ **DONE** — see [script-inventory.md](script-inventory.md). Key findings: all 481 scripts share magic `SYS4422 `; 52 loose root-dir `.BIN` files are patch overrides that shadow DATA1 copies (use those as authoritative); heavy game logic (damage calc, dungeon loop, battle flow) lives in bytecode, favoring a VM re-implementation in Godot.
+3. ~~**Inventory the script files**~~ **DONE** — see [script-inventory.md](script-inventory.md). Key findings: all 481 scripts share magic `SYS4422 `; 49 loose root-dir script `.BIN` files shadow DATA1 copies (plus two root-only engine BINs; use overrides as authoritative); heavy game logic (damage calc, dungeon loop, battle flow) lives in bytecode, favoring a VM re-implementation in Godot.
 1. ~~**Relocate the `Output\` tree**~~ **DONE** — workspace now at `S:\Game Hacking\Eushully\Himegari\姫狩りダンジョンマイスター\`.
 2. **Convert remaining AGFs** in DATA2 (985 files) and DATA5 (210 files) with `AGF2BMP2AGF.exe`. *(Deferred — graphics not needed yet.)* The 3-file DATA1 gap is `CHAPTER.AGF`, `LOGO.AGF`, `TEST.AGF`.
-3. ~~**Inventory the script files**~~ **DONE** — see [script-inventory.md](script-inventory.md). Key findings: all 481 scripts share magic `SYS4422 `; 52 loose root-dir `.BIN` files are patch overrides that shadow DATA1 copies (use those as authoritative); heavy game logic (damage calc, dungeon loop, battle flow) lives in bytecode, favoring a VM re-implementation in Godot.
+3. ~~**Inventory the script files**~~ **DONE** — see [script-inventory.md](script-inventory.md). Key findings: all 481 scripts share magic `SYS4422 `; 49 loose root-dir script `.BIN` files shadow DATA1 copies (plus two root-only engine BINs; use overrides as authoritative); heavy game logic (damage calc, dungeon loop, battle flow) lives in bytecode, favoring a VM re-implementation in Godot.
 
 ### Header structure — DONE (hex-first, pre-Ghidra)
 See [sys4-format-notes.md](sys4-format-notes.md). Confirmed across all 481 files: 60-byte header (magic `SYS4422 ` + 13 u32 fields, all offsets in dwords), body split into CODE + 3 typed pointer tables (tags 0x71/0x03/0x8F, 1 dword each, 100% pure) + inline strings. Strings are XOR-0xFF cp932, referenced by a `0x02 <dword-offset>` tagged operand — verified by decoding real dialogue out of `SC0030.BIN`. Remaining unknowns (opcode dispatch, flag fields F0/F2/F3/F5) need the VM.

docs/name-resolution.md

@@ -24,10 +24,11 @@ mechanism in `engine-re.md` (“op 0x03 (call-script)…”). Tooling: `parse_sy
 `build/callscript-names.json` (id→name); `sys4load` renders `call-script 0x1ab =ADDITEM.BIN`; the
 `build/disasm/*.asm` call graph now reads by name. The one caveat: index the RAW SYS4INI records
 (*including* the 2 `@` placeholders) — `asset-index.json` carries each entry's `raw_index` (= the id)
-for exactly this. **Remaining (functional, not naming):** the C# VM still stubs `call-script`
-execution; implementing it (load `.BIN` by id, push frame, run, return) is the follow-up. The original
-analysis (kept below for provenance) had concluded this was engine-level and deferred — it was, and
-the Ghidra loop is what resolved it.
+for exactly this. **Runtime (VFS-A):** `Sys4AssetCatalog` now reads that raw table directly and
+`Sys4ScriptProvider` opens the selected record through loose-first/bounded-ALF storage; generated JSON is
+only the disassembler annotation and parity oracle. The VM executes the loaded target as a nested frame.
+The original analysis (kept below for provenance) had concluded this was engine-level and deferred — it
+was, and the Ghidra loop is what resolved it.
 
 **What it is (original framing).** `call-script N` (Kelebek opcode 0x03) carries a bare number —
 `0x329d`, `0x2ade` — the id of an engine entry point. To render `call RECOVER` instead of

docs/phase-a-slice-plan.md

@@ -1055,7 +1055,7 @@ separate slices. No implementation commit was made.
 This is an optional high-leverage detour before movie `0x236` or SFX. It replaces the Phase-A
 pre-extracted/pre-converted asset bootstrap with the native loose-override/archive-fallback model and removes
 the runtime dependency on `extracted/` plus `build/textures/`. Canonical format/architecture detail and source
-references live in `docs/asset-resolution-re.md` §“Candidate runtime asset-VFS track”; this section defines
+references live in `docs/asset-resolution-re.md` §“Runtime asset-VFS track”; this section defines
 slice boundaries only.
 
 Land it as three bounded slices, not one archive/codec rewrite:
@@ -1090,6 +1090,21 @@ fixtures and installed-game integration checks rather than committing proprietar
 modding contract and benefits scripts, UI chrome, SFX, and movies. It is not required to continue opcode
 coverage immediately, so choosing movie/SFX next remains valid.
 
+### VFS-A — base SYS4 catalog + ALF byte reads DONE (2026-07-11)
+
+`Sys4AssetCatalog` now runtime-parses `SYS4INI.BIN` into all 13208 raw slots (including two `@`
+placeholders), 13206 real/name records, bounded scene sections, and universal raw-id lookup.
+`Sys4AssetStore` applies native `loose exact-basename -> ALF offset/size` precedence with traversal rejection,
+per-open file handles, and a seek/read boundary. `Sys4ScriptProvider` and all CLI/Godot root-script paths use
+store bytes; `ResourceMap` consumes the live catalog instead of generated asset JSON.
+
+Validation: 116 engine tests; every field against `build/asset-index.json`; all 136 generated scene views;
+all 13206 archive ranges; representative bytes across DATA1..5 against `extracted/`; synthetic precedence,
+range, traversal, and concurrency tests; CLI SC0000 run; Godot build/selftest. The installed root has 49
+archive-backed script overrides and two root-only BIN engine files, not the historically reported 52
+shadowing scripts; all 49 were byte-proven to win and differ from their archive payload. VFS-B APPEND01/AAI,
+VFS-C AGF, audio migration, and movie `0x236` remain separate.
+
 ### Phase A — native SC0000 SFX family (`0xb4`/`0xb5`/`0xb6`/`0xc2`/`0xd9`) DONE (2026-07-11)
 
 Native RE and the matching trace resolve the bounded family. `0xb4(resource,channel)` synchronously loads

docs/remake-architecture-and-roadmap.md

@@ -23,8 +23,8 @@ foundation; the runtime + backends + mod system is the bulk of the remaining wor
    asset rules) selected by a manifest.
 3. **Modding is architecture, not an afterthought.** The data model, content loading, and script
    dispatch are designed so mods can override assets, edit data, patch scripts, and inject
-   host-language hooks. The engine already hints at this: 52 loose root `.BIN` files natively
-   shadow their archived copies — a built-in override mechanism we generalize.
+   host-language hooks. The engine already hints at this: 49 loose root script `.BIN` files natively
+   shadow their archived copies — a built-in override mechanism VFS-A generalizes.
 4. **The original owns the content; we own the engine.** Users provide their AGE install; the
    runtime imports/loads it. This keeps us on the right side of distribution and mirrors ScummVM.
 5. **De-risk with vertical slices.** Prove "run one scene end-to-end" before breadth. Nothing is
@@ -231,8 +231,8 @@ is shared across the family), disassembler/assembler, the whole extraction metho
 **Per-game (inherent content work):** the **global-var map** (globals are game-specific), the
 **data-table layouts** (each game's `*INIT` differs), assets, and any game-specific effectful behavior.
 The **call-script registry is no longer a per-game long pole** — it's a raw index into that game's
-SYS4INI file table, derived automatically by `parse_sys4ini.py` (`build/callscript-names.json`); the
-resolver is generic. So the remaining long pole is really just the **global-var map**. Process: point
+SYS4INI file table, parsed directly by the runtime catalog (and exported by `parse_sys4ini.py` as
+`build/callscript-names.json` for tooling); the resolver is generic. So the remaining long pole is really just the **global-var map**. Process: point
 the toolchain at the new game's archives, re-run extraction, rebuild its global map, author a profile.
 **This is the core payoff of the VM approach:** the *engine* cost amortizes across all AGE games; only
 content-mapping recurs — far less than re-coding each game's logic bespoke.

docs/script-inventory.md

@@ -7,11 +7,12 @@ during disassembler work).
 
 Source: `extracted\DATA1\` (extracted from `DATA1.ALF`).
 
-**Patch overrides:** 52 loose `.BIN` files sit in the game root directory and shadow
-their DATA1 counterparts at runtime (sizes differ slightly — e.g. `FIELD.BIN` root
-200,536 vs archive 200,224). These are the v1.03 / append-patch versions and should be
-treated as **authoritative** over the archive copies. Two engine files exist only in
-the root: `SYS4INI.BIN` (272 KB) and `SYS4AB.BIN` (1.08 MB).
+**Patch overrides (runtime re-counted 2026-07-11):** 49 loose `.BIN` scripts sit in the game root and
+shadow DATA1 counterparts at runtime (sizes differ slightly — e.g. `FIELD.BIN` root 200,536 vs archive
+200,224). These v1.03 / append-patch versions are **authoritative**. Two additional engine BINs exist only
+in the root: `SYS4INI.BIN` (272 KB) and `SYS4AB.BIN` (1.08 MB), for 51 root BINs total. The earlier count of
+52 shadowing scripts conflated this inventory and was not reproducible; VFS-A enumerates and byte-checks all
+49 real catalog/name intersections.
 
 ---
 
@@ -100,8 +101,8 @@ outside scenes lives.
 `call-script <id>` (opcode 0x03) loads another script by a **raw index into the SYS4INI file table**
 (id = the entry's `raw_index` = its global position in SYS4INI). This is the resolved call-graph
 registry — there is no separate id→code table; SYS4INI is it. Mechanism: `engine-re.md` (op 0x03
-section); id→name single source: `build/callscript-names.json` (from `parse_sys4ini.py`); `sys4load`
-and the regenerated `build/disasm/*.asm` corpus now render targets by name
+section); the runtime source is `Sys4AssetCatalog` over SYS4INI, while the mechanically generated
+`build/callscript-names.json` feeds `sys4load` diagnostics. The regenerated `build/disasm/*.asm` corpus renders targets by name
 (`call-script 0x1ab =ADDITEM.BIN`). **297 distinct scripts are called** across the corpus (3002 sites);
 the hottest are `HISTORY` (backlog), `MENU`, `HIDEWIN`, `BUNKI` (branch), `MES` (message), `ADDITEM`,
 `ADDEN`, `LOOK`, `RENDERMAP`. Scenes (`SCxxxx.BIN`) load through the *same* id-indexed loader.
@@ -126,4 +127,4 @@ derivable on demand from the corpus; materialize a doc only if a consumer needs
    (176 B) → `MENU.BIN` (3 KB) → `CALCDMG` → a mid-size `SC####`.
 3. **The `*INIT` giants are likely data tables**, decodable early even with a
    partial opcode map — instant win for extracting item/skill/enemy/stage databases.
-4. **Use root-directory overrides, not archive copies**, for the 52 patched scripts.
+4. **Use root-directory overrides, not archive copies**, for the 49 archive-backed patched scripts.

docs/sys4-format-notes.md

@@ -112,8 +112,8 @@ task. These tags give a head start on labeling the disassembly.*
 
 ## Patch-override caveat (re-confirmed)
 
-52 loose `.BIN` in the game root shadow their DATA1 copies at runtime and differ
-slightly in size. The disassembler should target the **root** copies where present.
+49 loose script `.BIN` files in the game root shadow DATA1 copies at runtime and differ slightly in size;
+two additional root-only engine BINs bring the root total to 51. The disassembler should target the **root** copies where present.
 The header format is identical (same magic/layout) so tooling is copy-agnostic.
 
 ## What's solid vs. what needs Ghidra

docs/tools-reference.md

@@ -72,12 +72,16 @@ The `engine/` .NET solution (`AgeEngine.sln`) is the runtime VM; `godot/` is the
 Python, but listed here as the things you *run*. Build: `dotnet build engine/AgeEngine.sln`; test:
 `dotnet test engine/AgeEngine.sln`. Run a CLI command: `dotnet run --project engine/Age.Cli -- <cmd>`.
 
-**call-script executes** on the product paths: they inject `Sys4ScriptProvider` (id→`.BIN`, via
-`build/callscript-names.json`), so `call-script <id>` loads & runs the target as a nested subroutine
+**call-script executes** on the product paths: they inject `Sys4ScriptProvider`, which runtime-parses
+`SYS4INI.BIN` and opens `.BIN` bytes through the native loose-first/bounded-ALF store, so `call-script <id>` loads & runs the target as a nested subroutine
 frame sharing globals. `trace`/`audio`/`gfx` stay **provider-less** (call-script stubbed) — base-ISA /
 subsystem oracles. Test scenes are **synthesized** via `Age.Engine/Sys4/ScriptAssembler` (see
 [[testing-synthesize-dont-disable]]: synthesize test data, never disable a feature to keep a golden green).
 
+The runtime SYS4 front-end is `Sys4AssetCatalog` (raw-id, scene-local, and name views), `IAssetStore` /
+`Sys4AssetStore` (exact-basename loose roots, then a bounded ALF range), and `Sys4ScriptProvider` (cached
+root/call-script parsing). Generated asset/callscript JSON remains a tooling and test oracle only.
+
 | Command | Purpose | Notes |
 |---|---|---|
 | `run <file.BIN>` | Execute a script; print steps, show-text count, **call-script dispatch count**, the first 30 lines (each tagged with its source script), and the distinct source scripts. | `CaptureHost` (headless); **executes call-script**. |
@@ -153,7 +157,7 @@ texture ops (no GPU context) — run windowed for real scenes. User args (after
 | Tool | Purpose | Run | Reads → Writes |
 |---|---|---|---|
 | `tools/frida/capture_native_transforms.py` | Capture native `0x21f`/`0x223`/`0x234` worker operands, corrected integer base/anchor coordinates, all one-shot/cyclic retained fields, the one-shot 4×4 matrix, and the final post-cyclic 4×4 matrix. Optional handle filter; read-only. | `py -3.11 -u -X utf8 tools/frida/capture_native_transforms.py [secs] [pid|AGE.EXE] [--handle 0xHANDLE]` | running game → `build/native-transform-trace.jsonl` |
-| `parse_sys4ini.py` | Parse `SYS4INI.BIN` (S4IC422, LZSS-compressed) into the authoritative asset index — name ↔ archive ↔ offset ↔ size for all DATA*.ALF (the `resId→file` answer key). Each entry carries `raw_index` (its 0-based position in the SYS4INI record table incl. `@` placeholders) = the engine's universal file id. Also emits the **`call-script <id> → name`** map (id = `raw_index`; see `engine-re.md`). | `parse_sys4ini.py [--check]` (`--check` validates vs `extracted/` + `.ALF` sizes) | `姫狩り…/SYS4INI.BIN` → `build/asset-index.json` + `build/callscript-names.json` |
+| `parse_sys4ini.py` | Parse `SYS4INI.BIN` (S4IC422, LZSS-compressed) into the diagnostic JSON asset-index mirror — name ↔ archive ↔ offset ↔ size for all DATA*.ALF. Each real entry carries universal `raw_index`; the runtime parses SYS4INI itself, while these generated files remain tooling/test oracles. Also emits the `call-script <id> → name` annotation map. | `parse_sys4ini.py [--check]` (`--check` validates vs `extracted/` + `.ALF` sizes) | `姫狩り…/SYS4INI.BIN` → `build/asset-index.json` + `build/callscript-names.json` |
 | `resolve_asset.py` | ★ **The static asset resolver.** SYS4INI is sectioned (one per scene: `SCxxxx.BIN` + its cross-archive manifest; `file_number` = index within section). Resolves `resId → files[section_base(scene) + resId]` for graphics AND audio, no capture. | `resolve_asset.py --build` · `resolve_asset.py <SCENE> [resId]` | `build/asset-index.json` → `build/asset-sections.json`; resolves any (scene, resId) |
 | `resolve_frida_reads.py` | Rescue noisy Frida archive-read offsets → asset names via the index (per-archive range search; drops 0x20000 paging reads); recovers the per-scene asset load order. | `resolve_frida_reads.py [reads.log] [-o out.json]` | `build/frida-reads.log` + `build/asset-index.json` → `build/frida-asset-loads.json` |
 | `convert_agf.py` | Convert AGF stills to BMP via `AGF2BMP2AGF.exe` (searches all `extracted/DATA*`). `--scene` batch-converts a scene's whole SYS4INI manifest — feeds the Godot render. | `convert_agf.py EV052CA.AGF …` · `convert_agf.py --scene SC0000` | `extracted/DATA*/*.AGF` → `build/textures/*.BMP` |

docs/vm-mapping-plan.md

@@ -28,7 +28,7 @@
 ## Global constraints
 
 - **Python:** `py -3.11 -X utf8 …` always (Shift-JIS output needs utf8 mode on Windows).
-- **Authoritative copies:** the 52 loose game-folder `.BIN` shadow their `extracted/DATA1/` copies at runtime — target the game-folder copy where both exist. `sys4load.load()` is copy-agnostic; `paths.scripts()` resolves the override.
+- **Authoritative copies:** 49 loose game-folder script `.BIN` files shadow `extracted/DATA1/` copies at runtime; two engine BINs are root-only. Target the game-folder copy where both exist. `sys4load.load()` is copy-agnostic; `paths.scripts()` resolves the override.
 - **Units:** all script offsets/counts are DWORDS (×4 bytes), relative to body start `0x3C`.
 - **Instruction rule:** `len_dwords = 1 + 2*argc`; args are `(type,value)`; **stop code decode at the first inline-string/array offset**, not blindly at `F8`.
 - Record confidence per finding (confirmed-by-bytes / confirmed-by-runtime / hypothesis).