Decode AGF textures through the asset store
This commit is contained in:
@@ -85,7 +85,7 @@ S:\Game Hacking\Eushully\Himegari\ ← workspace root (three siblings)
|
||||
│ │ └── strings.jsonl every string, tagged by source opcode
|
||||
│ ├── data/ parsed data tables (*INIT → JSON)
|
||||
│ ├── scripts-json/ machine-readable full dumps (on demand via --json)
|
||||
│ ├── textures/ AGF→BMP stills (convert_agf.py) — feeds the Godot render
|
||||
│ ├── textures/ AGF-to-BMP stills (convert_agf.py) - diagnostic pixel oracle only
|
||||
│ ├── engine-dump/ UNPACKED engine dump (frida/dump_engine.py): range_<base>.bin + manifest.json
|
||||
│ ├── asset-index.json, asset-sections.json asset resolver data (parse_sys4ini / resolve_asset)
|
||||
│ ├── global-var-map.{json,md} partial global-variable name map (auto shape inference; feeds globals.toml merge)
|
||||
@@ -97,7 +97,7 @@ S:\Game Hacking\Eushully\Himegari\ ← workspace root (three siblings)
|
||||
│
|
||||
├── engine/ DELIVERABLE — the .NET VM core (AgeEngine.sln: Age.Engine / Age.Cli / tests)
|
||||
│ └── Age.Engine/Sys4/ runtime catalog parser, loose-first bounded ALF asset store,
|
||||
│ script provider, and temporary resource facade
|
||||
│ script provider, AGF-to-RGBA8/LZSS decoder, and resource facade
|
||||
├── tools/frida/ runtime-capture + engine-dump scripts (see tools/frida/README.md)
|
||||
└── godot/ DELIVERABLE — the Godot/C# ADV front-end (references Age.Engine)
|
||||
```
|
||||
|
||||
@@ -166,7 +166,7 @@ The current Phase-A backend deliberately continues through the extracted-file bo
|
||||
accepts both OGG and WAV and Godot loads the WAV bytes into its fixed SC0000 channel pool. This does not change
|
||||
the scoped VFS plan below: ALF/AAI mounting and in-process asset reads remain a separate foundation track.
|
||||
|
||||
## Runtime asset-VFS track (VFS-A complete 2026-07-11)
|
||||
## Runtime asset-VFS track (VFS-A and VFS-C complete 2026-07-11)
|
||||
|
||||
The pre-extracted tree and `build/textures/*.BMP` pipeline were a Phase-A bootstrap, not the desired final
|
||||
runtime. The native-compatible target is a read-only virtual filesystem that preserves AGE's translation/mod
|
||||
@@ -199,16 +199,20 @@ store.
|
||||
by name replacement, or by another table selected by the native high-byte-id path; do not invent mount
|
||||
precedence. Validate every parsed append entry against `BinExtractALF.exe` output before exposing it to
|
||||
the runtime.
|
||||
3. **AGF decoder.** Decode an opened AGF stream directly to width/height + RGBA8. The MIT-licensed GARbro
|
||||
3. **AGF decoder (VFS-C DONE).** `Age.Engine/Sys4/AgfDecoder.cs` decodes an opened AGF payload directly
|
||||
to a tightly packed, top-down width/height + RGBA8 surface. The MIT-licensed GARbro
|
||||
`ArcFormats/Eushully/ImageAGF.cs` provides a compact reference: `ACGF` (or zero) signature, type 1/2,
|
||||
LZSS-or-raw header section, 4/8/truecolor source pixels, LZSS-or-raw pixel section, bottom-up row/stride
|
||||
conversion, and optional `ACIF` LZSS alpha plane. Port only the algorithm and attribution into
|
||||
platform-neutral .NET code; do not carry GARbro's WPF/GameRes dependencies. Kelebek's extractor and the
|
||||
on-disk `BinExtractALF.exe` are validation references; the Kelebek repository exposes no clear license,
|
||||
so its code should not be copied without clarification.
|
||||
4. **Runtime consumers.** Script loading is complete. Next make texture surfaces own
|
||||
decoded RGBA pixels rather than BMP paths, and load OGG/WAV from store bytes. Migrate one consumer at a
|
||||
time; retain extraction/conversion tools as diagnostics until parity is established.
|
||||
so its code should not be copied without clarification. The focused `LzssDecoder` is shared with
|
||||
`Sys4AssetCatalog`; raw and compressed information/pixel/ACIF sections use the same bounded primitive.
|
||||
4. **Runtime consumers.** Script and texture loading are complete. `ResourceMap.ResolveTexture` preserves
|
||||
scene-local resolution and falls back to universal raw ids for SYSTEM4 assets; `GodotAdvHost` caches
|
||||
decoded RGBA surfaces by catalog identity and supplies synchronous dimensions to opcode `0x208`.
|
||||
Godot no longer reads `build/textures/*.BMP`. OGG/WAV byte migration remains a separate follow-up;
|
||||
retain extraction/conversion tools as diagnostics until parity is established.
|
||||
|
||||
### Acceptance gates
|
||||
|
||||
@@ -233,8 +237,17 @@ are byte-identical to `extracted/`, and synthetic removal of a loose override re
|
||||
Traversal, past-range seek/read, and concurrent reads are covered. Installed override enumeration corrected
|
||||
an older inventory error: this tree contains 51 loose root BINs, comprising **49 archive-backed v1.03 script
|
||||
overrides** (all byte-proven to win and differ from DATA1) plus root-only `SYS4INI.BIN` and `SYS4AB.BIN`.
|
||||
There are not 52 archive copies available to shadow. APPEND01/AAI, AGF decode, audio consumers, and movie
|
||||
`0x236` remain unimplemented by design.
|
||||
There are not 52 archive copies available to shadow.
|
||||
|
||||
VFS-C passes its bounded gates in `AgfDecoderTests`: synthesized fixtures cover raw/compressed sections,
|
||||
4/8-bit palettes, 24/32-bit truecolor expansion, padded bottom-up rows, type 1/2, and ACIF/no-ACIF alpha.
|
||||
Five installed assets spanning raw/compressed metadata and pixel/alpha combinations match the existing
|
||||
`AGF2BMP2AGF` BMP oracle pixel-for-pixel. SO001 resolves through universal raw id `0x337e`, decodes to
|
||||
800×300 with intermediate alpha values, and is inherited in surface slot 17 before SC0000. A windowed
|
||||
page-1 capture with `build/textures/` moved aside showed the translucent textbox edge and bottom-right
|
||||
controls. Texture runtime no longer consults `extracted/` or `build/textures/`; current audio consumers
|
||||
still use extracted OGG/WAV paths by design. APPEND01/AAI, audio migration, and movie `0x236` remain
|
||||
unimplemented by this slice.
|
||||
|
||||
### Deliberate non-goals
|
||||
|
||||
|
||||
@@ -1050,7 +1050,7 @@ opcode/ctx lint, 481-script decode, RECOVER, and `git diff --check` clean. SC000
|
||||
**85/129 to 87/129 handled (67.4%)**, with 42 GAP ops / 188 GAP instructions. Movie `0x236` and SFX remain
|
||||
separate slices. No implementation commit was made.
|
||||
|
||||
### Candidate foundation track — native asset VFS + ALF/AAI/AGF readers (SCOPED 2026-07-10; NOT STARTED)
|
||||
### Foundation track — native asset VFS + ALF/AAI/AGF readers (VFS-A/C DONE; VFS-B PENDING)
|
||||
|
||||
This is an optional high-leverage detour before movie `0x236` or SFX. It replaces the Phase-A
|
||||
pre-extracted/pre-converted asset bootstrap with the native loose-override/archive-fallback model and removes
|
||||
@@ -1105,6 +1105,26 @@ archive-backed script overrides and two root-only BIN engine files, not the hist
|
||||
shadowing scripts; all 49 were byte-proven to win and differ from their archive payload. VFS-B APPEND01/AAI,
|
||||
VFS-C AGF, audio migration, and movie `0x236` remain separate.
|
||||
|
||||
### VFS-C — in-process AGF to RGBA8 + SC0000 system chrome DONE (2026-07-11)
|
||||
|
||||
`AgfDecoder` now consumes `IAssetStore` bytes and emits a platform-neutral, tightly packed top-down RGBA8
|
||||
surface. Its shared 4 KiB-ring `LzssDecoder` covers raw/compressed information, pixel, and optional ACIF
|
||||
alpha sections; expansion covers 4/8-bit palettes, 24/32-bit truecolor, DIB row padding, and bottom-up
|
||||
orientation. Synthetic fixtures exercise the whole format matrix. Installed `AE000A`, `AE001A`,
|
||||
`BG030A`, `EV052CA`, and `SO001` match the existing converter's pixels exactly; SO001 is 800×300
|
||||
with non-binary per-pixel alpha.
|
||||
|
||||
`ResourceMap` now resolves texture ids through the scene manifest with a universal raw-id fallback,
|
||||
then decodes through the loose-first VFS. Godot and the CLI no longer use `BmpHeader`, `TexturePath`, or
|
||||
pre-converted BMPs. SYSTEM4's inherited SO001 surface is seeded into slot `0x11` before SC0000, allowing
|
||||
the existing callback-window crops to render the textbox and control icons. Windowed page-1 validation
|
||||
passed with `build/textures/` temporarily unavailable; source alpha produced the translucent upper edge
|
||||
and the bottom-right control row was visible.
|
||||
|
||||
Validation: 124/124 engine tests, CLI/Godot builds, Godot threaded selftest, pixel parity for five installed
|
||||
assets, and the windowed no-BMP capture. VFS-B APPEND01/AAI, OGG/WAV migration, movie `0x236`, and unrelated
|
||||
opcode work remain separate.
|
||||
|
||||
### Phase A — native SC0000 SFX family (`0xb4`/`0xb5`/`0xb6`/`0xc2`/`0xd9`) DONE (2026-07-11)
|
||||
|
||||
Native RE and the matching trace resolve the bounded family. `0xb4(resource,channel)` synchronously loads
|
||||
|
||||
@@ -69,6 +69,10 @@ Three layers, cleanly separated:
|
||||
the offline analysis/authoring chain; they were the reference implementation and stay useful for
|
||||
modders. They share the *format spec* (documented), not code — acceptable for a small, stable
|
||||
container format.
|
||||
- **Native content foundation status (2026-07-11).** VFS-A runtime-parses the base SYS4 catalog and
|
||||
applies loose-first bounded ALF reads; VFS-C decodes AGF directly to platform-neutral RGBA8 and feeds
|
||||
Godot without pre-extracted or pre-converted texture files. APPEND01/AAI mounting and audio-byte consumers
|
||||
remain separate follow-ups.
|
||||
- **Profile = manifest.** Adding a game = a new profile + its maps. Adding an engine version = a new
|
||||
front-end plugin + profiles that reference it. See §5.
|
||||
|
||||
|
||||
@@ -160,7 +160,7 @@ texture ops (no GPU context) — run windowed for real scenes. User args (after
|
||||
| `parse_sys4ini.py` | Parse `SYS4INI.BIN` (S4IC422, LZSS-compressed) into the diagnostic JSON asset-index mirror — name ↔ archive ↔ offset ↔ size for all DATA*.ALF. Each real entry carries universal `raw_index`; the runtime parses SYS4INI itself, while these generated files remain tooling/test oracles. Also emits the `call-script <id> → name` annotation map. | `parse_sys4ini.py [--check]` (`--check` validates vs `extracted/` + `.ALF` sizes) | `姫狩り…/SYS4INI.BIN` → `build/asset-index.json` + `build/callscript-names.json` |
|
||||
| `resolve_asset.py` | ★ **The static asset resolver.** SYS4INI is sectioned (one per scene: `SCxxxx.BIN` + its cross-archive manifest; `file_number` = index within section). Resolves `resId → files[section_base(scene) + resId]` for graphics AND audio, no capture. | `resolve_asset.py --build` · `resolve_asset.py <SCENE> [resId]` | `build/asset-index.json` → `build/asset-sections.json`; resolves any (scene, resId) |
|
||||
| `resolve_frida_reads.py` | Rescue noisy Frida archive-read offsets → asset names via the index (per-archive range search; drops 0x20000 paging reads); recovers the per-scene asset load order. | `resolve_frida_reads.py [reads.log] [-o out.json]` | `build/frida-reads.log` + `build/asset-index.json` → `build/frida-asset-loads.json` |
|
||||
| `convert_agf.py` | Convert AGF stills to BMP via `AGF2BMP2AGF.exe` (searches all `extracted/DATA*`). `--scene` batch-converts a scene's whole SYS4INI manifest — feeds the Godot render. | `convert_agf.py EV052CA.AGF …` · `convert_agf.py --scene SC0000` | `extracted/DATA*/*.AGF` → `build/textures/*.BMP` |
|
||||
| `convert_agf.py` | Convert AGF stills to BMP via `AGF2BMP2AGF.exe` (searches all `extracted/DATA*`). `--scene` batch-converts a scene's whole SYS4INI manifest. Since VFS-C, output is a diagnostic pixel-parity oracle; the runtime decodes AGF bytes directly. | `convert_agf.py EV052CA.AGF …` · `convert_agf.py --scene SC0000` | `extracted/DATA*/*.AGF` → `build/textures/*.BMP` |
|
||||
|
||||
## Runtime capture (Frida)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user