Implement native DAT container codec

This commit is contained in:
gamer147
2026-07-24 14:00:33 -04:00
parent 15f23abf5c
commit 6ffd21cd3e
10 changed files with 856 additions and 5 deletions

View File

@@ -101,8 +101,9 @@ S:\Game Hacking\Eushully\Himegari\ ← workspace root (three siblings)
│ └── manifest.json, opcode-coverage.md (opcode-coverage.md GENERATED from opcodes.toml)
├── engine/ DELIVERABLE — the .NET VM core (AgeEngine.sln: Age.Engine / Age.Cli / tests)
── Age.Engine/Sys4/ runtime catalog parser, loose-first bounded ALF asset store,
── Age.Engine/Sys4/ runtime catalog parser, loose-first bounded ALF asset store,
│ script provider, AGF/LZSS and Windows CUR decoders, and resource facade
│ └── Age.Engine/Persistence/ native S3SD/S4SD container codec and profile/numbered DAT lifecycle seam
├── native/ authored native runtime boundaries
│ └── age_movie_ffmpeg/ project-owned FFmpeg C ABI, immutable Windows dependency manifest,
│ and bootstrap/build scripts (outputs stay under disposable build/)

View File

@@ -1722,6 +1722,16 @@ inventing a second save container.
a profile/save service so extended mode can later add JSON inspection/export, namespaced mod state, migrations,
or a friendlier editor without changing compatibility-mode opcode semantics or the native import/export path.
**Port correspondence (2026-07-24, codec foundation implemented):**
`Age.Engine.Persistence.NativeSaveContainerCodec` now reads and writes the common header, Shift-JIS game id,
SYSTEMTIME/playtime/version metadata, both CRC layers, version-2 compression wrapper, and exact reversible
DWORD transform. `Sys4.LzssEncoder` emits the same 4 KiB-ring token dialect already consumed by
`LzssDecoder`, falling back to native verbatim storage when compression does not shrink. The
`INativeDatStore` boundary and `DirectoryNativeDatStore` own shared `$$SAVE.DAT` → `SAVE.DAT` /
`SAVE.BAK` replacement/fallback and direct numbered `SAVE##.DAT` writes. Payload schemas, `RT.DAT`,
thumbnails, and opcode wiring remain deliberately above or after this layer; the existing `GameSession`
JSON snapshot is unchanged.
### Opcode `0xae` continues numbered-save stack restoration (2026-07-20)
Opcode `0xae` is the load-side rendezvous paired with serialized script-frame state. Its handler,

View File

@@ -3355,6 +3355,32 @@ No runtime persistence code changed in this reconnaissance slice. The canonical
references now describe the native ABI; the corresponding `/v2` handlers and codec helpers are named,
commented, and saved.
### Persistence implementation step 1 — native DAT codec and store boundary (2026-07-24)
The reusable compatibility floor is now implemented without prematurely inventing either shared-profile or
numbered-state payload objects. `NativeSaveContainerCodec` owns the exact `0x124` header and `0x14` codec
frame, Shift-JIS game identity, SYSTEMTIME/playtime/version fields, inner and outer CRC pairs, rolling
seed/odd-multiplier DWORD expansion, its exact-division inverse, and the SaveVersion2>=2 wrapper. The new
`LzssEncoder` shares the existing native 4096-byte-ring dialect and uses the native equal-length verbatim
fallback when compression is not beneficial.
`INativeDatStore` is the payload-agnostic runtime seam. Its directory implementation performs shared
`$$SAVE.DAT` → `SAVE.DAT` replacement with `SAVE.BAK` fallback and direct `SAVE##.DAT` reads/writes, while
validating generation, compatibility id, game id, and the native layout-version exception. It does not
serialize the VM's whole global bank, change `GameSession.ToJson`, wire persistence opcodes, or claim
`RT.DAT`/`.STH` support.
Seven focused tests cover independent standard CRC vectors, compressed and verbatim LZSS round trips,
S3SD/S4SD header/frame offsets, deterministic transform products, both codec-version branches, corruption
rejection after outer-CRC repair, layout-2 version compatibility, shared backup recovery, and direct numbered
files. The full engine suite passes 359 tests. The opcode table count regression now distinguishes 248
Himegari-observed opcodes from the mapped-but-unused shared ABI opcode `0x19f`.
**Next persistence step:** implement the typed shared `SAVE.DAT` logical payload and profile-owned selected
integer/string cell maps, then connect `0x1a2`/`0x1a3` and `0x1a9`/`0x1aa`. Add native `RT.DAT` and its
read-message lifecycle after that shared ownership is live; numbered active-frame layouts remain the later,
larger payload.
## Data-semantics sidebar: focused append EBINIT inspection (2026-07-24)
The static INIT surface now accepts a universal packed script id for focused append inspection.

View File

@@ -361,7 +361,8 @@ In scope:
Deferred to bounded follow-ups unless the happy path requires them:
- Full configuration UI and every setting.
- Load/save implementation and save-format reversal.
- Full load/save opcode and logical-payload implementation. Native format reversal and the common DAT
codec/store foundation landed on 2026-07-24.
- Extras, galleries, replay modes, and unrelated submenus.
- Menu visual polish that does not obstruct correct selection or state production.

View File

@@ -466,7 +466,8 @@ domains and lifecycle—shared `SAVE.DAT`/`SAVE.BAK`, `RT.DAT`/`RT.BAK`, numbere
paired BMP `.STH` thumbnails. Keep the codec behind a profile/save-service boundary. Human-readable
JSON inspection/export, migrations, and namespaced mod state are additive extended-mode work, not a
replacement for compatibility-mode import/export. The recovered native contract lives in
`docs/engine-re.md`.
`docs/engine-re.md`. The common container codec and payload-agnostic shared/numbered DAT store boundary
landed on 2026-07-24; logical payload services and opcode wiring remain Phase B work.
### Phase C — Externalize & modding foundation
- Add **editable named data overlays** mapped explicitly onto the VM's `*INIT`-produced state; external