Implement native layout-3 save restoration
This commit is contained in:
gamer147
@@ -104,7 +104,8 @@ S:\Game Hacking\Eushully\Himegari\ ← workspace root (three siblings)
|
||||
│ ├── Age.Engine/Sys4/ runtime catalog parser, loose-first bounded ALF asset store,
|
||||
│ script provider, AGF/LZSS and Windows CUR decoders, and resource facade
|
||||
│ └── Age.Engine/Persistence/ native S3SD/S4SD + S3RT codecs, shared payload/ReadTextDB,
|
||||
│ numbered DAT/STH pair + BMP codec, and profile-owned state
|
||||
│ layout-3 numbered state/history/gfx, DAT/STH pair + BMP codec,
|
||||
│ and profile-owned state
|
||||
├── native/ authored native runtime boundaries
|
||||
│ └── age_movie_ffmpeg/ project-owned FFmpeg C ABI, immutable Windows dependency manifest,
|
||||
│ and bootstrap/build scripts (outputs stay under disposable build/)
|
||||
|
||||
@@ -1801,8 +1801,44 @@ validates its distinct compatibility id, queries metadata without decoding paylo
|
||||
paired `.DAT`/`.STH` copy/delete status layering. `NumberedThumbnailCodec` reads and writes the native BMP
|
||||
dialect through host surface capture/replacement, and ops `0x1a0`, `0x1ab`–`0x1af` (including `0x1ad`'s
|
||||
frame marker) are wired. Godot injects the store at `user://SAVE`. Full layout-3 global/frame/history/
|
||||
resource/retained-gfx serialization and the `0x19e`/`0x1a1`/`0xae` save-resume path remain the next layer;
|
||||
the existing `GameSession` JSON snapshot is unchanged.
|
||||
`NativeNumberedSaveCodec`, `NativeTextHistoryCodec`, and `NativeGfxPersistenceCodec` now own the complete
|
||||
layout-3 numbered body and appended history tail. The VM wires `0x19e`, data-only `0x19f`, full load
|
||||
`0x1a1`, and the active branch of `0xae`; the existing `GameSession` JSON snapshot remains a separate
|
||||
diagnostic/extended-mode surface.
|
||||
|
||||
#### Layout-3 byte map and port correspondence (2026-07-24)
|
||||
|
||||
The decoded layout-3 body begins with `cutoff` at `+0x00`; its fixed length is
|
||||
`0x5718 + cutoff*0x414`. The fixed prefix contains two saved engine/context words, ten state DWORDs,
|
||||
100 three-DWORD resource reload records, 1,000 20-byte surface reload records, then frames beginning at
|
||||
`+0x5304`. Each `0x414`-byte frame contains parent context, packed script id, up to 256 saved local-return
|
||||
indices, and two terminal control indices. The three SYS4 footer tables now have explicit runtime roles:
|
||||
|
||||
- d259 indexes T1/F7-F8, the op-`0x71` read-message reset sites;
|
||||
- d260 indexes T2/F9-F10, resumable `call-script` sites (forced to `-1` on the cutoff frame);
|
||||
- each local return index addresses T3/F11-F12, whose stored target plus three is the live return offset.
|
||||
|
||||
After the fixed bytes are six bank counts followed by the ordinary (de-protected) integer bank, float
|
||||
storage, a DWORD-sized CP932 string blob, the pointer-family banks, and retained graphics. Graphics begin
|
||||
with record size `0x2d4` and object count; every object is `{handle,0x2d4 raw record}`. The range-transform
|
||||
first/count and another `0x2d4` record follow. AGE overallocates this region
|
||||
(`0x2e1 + objectCount*0x2d8` DWORDs) relative to the records it writes; the codec preserves the native
|
||||
logical sizing and zeroes the unused allocation slack.
|
||||
|
||||
The text-history tail is outside the S4SD container. Its 12-byte header is
|
||||
`{logicalBytes,logicalBytes,storedBytes}` followed by the same 4 KiB-ring LZSS/verbatim dialect. Logical
|
||||
history is entry count + 8-byte entries, record count + the first 11 DWORDs of each retained record, then
|
||||
a DWORD-counted CP932 string blob whose DWORDs are bitwise inverted. Installed `SAVE00.DAT` validates the
|
||||
whole boundary read-only: cutoff 1, six counts `[402459,1,789,1,1,1]`, 211 gfx objects, a 21-byte history
|
||||
tail, and exact consumption of the S4SD container before that tail.
|
||||
|
||||
The port tracks the native global banks separately at runtime, captures the marked frame chain, serializes
|
||||
live surfaces and retained objects, and reloads host textures from the 20-byte surface records. Full load
|
||||
first replaces banks/history/gfx, unwinds the obsolete managed call chain, runs `CALLBACK_LOAD.BIN` when
|
||||
the mounted script provider resolves it, starts the saved root at its `0xae` rendezvous, recursively
|
||||
reconstructs child frames, resumes parents after their saved T2 call sites,
|
||||
and finally resumes the terminal frame at its T1 boundary. Successful `0x19e` also flushes shared
|
||||
`SAVE.DAT`/`RT.DAT`, matching `context_state_serialize`.
|
||||
|
||||
### Opcode `0xae` continues numbered-save stack restoration (2026-07-20)
|
||||
|
||||
@@ -1815,14 +1851,13 @@ loads `CALLBACK_LOAD.BIN` or the saved entry script, and resets the current cont
|
||||
|
||||
On an active restore, the handler reads `set:SaveVersion1`/`set:SaveVersion2`, selects the matching saved
|
||||
frame layout, replaces the current frame PC with its saved resume or call target, and advances through the
|
||||
serialized contexts. At the saved terminal context it clears the restore flag and reinstates the saved
|
||||
context/return state. The corpus placement supports that control-flow role: 305 calls overwhelmingly follow
|
||||
serialized contexts. `script_frame_restore_saved_layout@0x40f2d0` loads each packed script and expands its
|
||||
T1/T2/T3 indices back into live PC/call/return offsets. At the saved terminal context the handler clears
|
||||
the restore flag and reinstates the saved context/return state. The corpus placement supports that control-flow role: 305 calls overwhelmingly follow
|
||||
coroutine-resume or call boundaries, including SC0000's main-loop resume sequence.
|
||||
|
||||
The opcode remains an effectful port gap. It is a no-op during every currently reachable port execution,
|
||||
but its actual branch cannot be implemented until numbered saves serialize and restore the active
|
||||
`ExecFrame` chain. Counting an unconditional no-op as coverage would conceal that dependency, so it stays
|
||||
grouped with `0x1ad` rather than receiving a placeholder VM case.
|
||||
The port now implements both branches: an ordinary one-instruction no-op outside restoration, and the
|
||||
T1/T2/T3-driven managed-frame reconstruction described above while a full numbered load is active.
|
||||
|
||||
### ADV read-message Skip and shared `RT.DAT` history (2026-07-18)
|
||||
|
||||
@@ -1947,10 +1982,9 @@ finds typed metadata; op `0x1d4` finds the voice pair. `HISTORY.BIN` dispatches
|
||||
|
||||
History is independent of `RT.DAT`, but native full save fidelity does serialize the live backlog.
|
||||
`text_history_serialize@0x451d00` writes the index and packed records/strings after context/numbered-save
|
||||
serialization, and `text_history_deserialize@0x456130` restores them on the matching load path. This is a
|
||||
future numbered-save integration seam, not a reason to choose a shared-profile backend now: an in-memory
|
||||
History button can be complete first, while save/load restoration stays deferred with the wider storage
|
||||
architecture decision.
|
||||
serialization, and `text_history_deserialize@0x456130` restores them on the matching load path. The port's
|
||||
`NativeTextHistoryCodec` now implements that appended tail directly against the same live
|
||||
`AdvTextHistory` model; it remains independent of shared profile state and `RT.DAT`.
|
||||
|
||||
History's display support consists of the ordinary presentation operations: primary/ruby font sizes
|
||||
(`0x75/0x197`), font weight (`0x2bd`), colors/effect mode/offset (`0x76/0x77/0x78/0x1a4`), layout origin
|
||||
|
||||
@@ -400,7 +400,9 @@ Implemented as a whole-stack root-reload boundary in the persistent VM. A reques
|
||||
### 0xae `continue-save-load-stack-restore` (u00415130, argc 0)
|
||||
- **summary:** () - during serialized save restoration, replace the current frame PC with its saved resume/call target and advance through the saved script-context stack; otherwise a no-op.
|
||||
- **grounding:** source=investigation, confidence=high
|
||||
- **evidence:** Ghidra /v2: op_0xae_continue_save_load_stack_restore@0x416790 first tests ctx+0x53d24 (set by save_data_deserialize_and_begin_restore@0x40fd10). When clear it returns. When set, it selects the serialized frame layout through set:SaveVersion1/2, restores the current PC from that layout's saved return/call target, advances through contexts with FUN_0040f2d0, and clears the restore flag on reaching the saved terminal context. Its 305 corpus sites overwhelmingly follow coroutine-resume/call boundaries, which provide the rendezvous points used while reconstructing the stack.
|
||||
- **evidence:** Ghidra /v2: op_0xae_continue_save_load_stack_restore@0x416790 first tests ctx+0x53d24 (set by save_data_deserialize_and_begin_restore@0x40fd10). When clear it returns. When set, it selects the serialized frame layout through set:SaveVersion1/2, restores the current PC from that layout's saved return/call target, advances through contexts with script_frame_restore_saved_layout@0x40f2d0, and clears the restore flag on reaching the saved terminal context. Its 305 corpus sites overwhelmingly follow coroutine-resume/call boundaries, which provide the rendezvous points used while reconstructing the stack.
|
||||
|
||||
Layout 3 frame d259 indexes SYS4 T1 read-message reset sites, d260 indexes T2 call-script sites, and the saved local return stack indexes T3 local-call sites. Port status (2026-07-24): the active path reconstructs the saved recursive frame chain and resumes the terminal frame at its T1 boundary.
|
||||
|
||||
### 0xc8 `sleep` (sleep, argc 1)
|
||||
- **summary:** Pause the current script for <duration> milliseconds while retained presentation continues.
|
||||
@@ -476,14 +478,14 @@ record-zero failure. Natural SYSTEM4 boot proves BTANINIT2 -> `$1$AUTORUN.BIN` -
|
||||
- **grounding:** source=investigation, confidence=high
|
||||
- **evidence:** Ghidra /v2: op_0x19e_save_numbered_slot@0x4278b0 formats SAVE%2.2d.DAT, checks an existing header and prompts before replacing an incompatible file, opens CREATE_ALWAYS, reads set:SaveVersion2 then set:SaveVersion1, and calls context_state_serialize@0x40d320. Corpus: two calls in SAVE.BIN and SELSTAGE.BIN.
|
||||
|
||||
Uses `set:SaveVersion1` and `set:SaveVersion2` to choose the numbered payload layout. Status is 0 on success and 1 on refusal, open/create failure, or serializer failure. Numbered `.DAT` files do not use the shared profile's temp/backup replacement scheme.
|
||||
Uses `set:SaveVersion1` and `set:SaveVersion2` to choose the numbered payload layout. Status is 0 on success and 1 on refusal, open/create failure, or serializer failure. Numbered `.DAT` files do not use the shared profile's temp/backup replacement scheme. Port status (2026-07-24): layout 3 writes fixed state, six native banks, T1/T2/T3 frame records, surface/resource reload state, retained gfx records, the appended history tail, and then flushes shared SAVE.DAT/RT.DAT.
|
||||
|
||||
### 0x19f `load-numbered-slot-data-only` (load-numbered-slot-data-only, argc 2)
|
||||
- **summary:** (status_out)(slot) - decode `SAVE%02d.DAT` without restoring the active script-frame chain or text history.
|
||||
- **grounding:** source=investigation, confidence=high
|
||||
- **evidence:** Ghidra /v2: op_0x19f_load_numbered_slot_data_only@0x427a40 opens SAVE%2.2d.DAT and calls save_data_deserialize_and_begin_restore@0x40fd10(handle, SaveVersion1, SaveVersion2, 0, 0), then decodes protected integer globals. Missing/open failure writes status 1; otherwise the decoder result is returned. Corpus count: 0.
|
||||
|
||||
This is the data-only companion to full-resume opcode 0x1a1. It selects the configured SaveVersion layout and restores serialized state with both runtime/history restore flags clear. Himegari's shipped script corpus does not call it, but it belongs to the shared SYS4 persistence ABI.
|
||||
This is the data-only companion to full-resume opcode 0x1a1. It selects the configured SaveVersion layout and restores serialized state with both runtime/history restore flags clear. Himegari's shipped script corpus does not call it, but it belongs to the shared SYS4 persistence ABI. Port status (2026-07-24): implemented for layout 3 through the same bank/resource/gfx decoder without activating history or frame restoration.
|
||||
|
||||
### 0x1a0 `query-numbered-save-metadata` (query-numbered-save-metadata, argc 9)
|
||||
- **summary:** (status_out)(slot)(year)(month)(day)(hour)(minute)(second)(playtime_seconds) - validate a numbered `.DAT` header and return its timestamp and accumulated playtime.
|
||||
@@ -497,7 +499,7 @@ Status 0 means valid metadata was written, 1 means the file could not be opened,
|
||||
- **grounding:** source=investigation, confidence=high
|
||||
- **evidence:** Ghidra /v2: op_0x1a1_load_numbered_slot_and_resume@0x427d30 calls save_data_deserialize_and_begin_restore@0x40fd10(handle, SaveVersion1, SaveVersion2, 1, 1), closes the file, and decodes protected integer globals. The loader activates the saved-frame state consumed by op_0xae. Corpus: one call in SAVE.BIN.
|
||||
|
||||
The caller pre-seeds status to zero. A missing/open failure writes 1; success starts the asynchronous native stack-restoration rendezvous and does not overwrite that zero. This is the ordinary load-game path, unlike data-only opcode 0x19f.
|
||||
The caller pre-seeds status to zero. A missing/open failure writes 1; success starts the asynchronous native stack-restoration rendezvous and does not overwrite that zero. This is the ordinary load-game path, unlike data-only opcode 0x19f. Port status (2026-07-24): layout 3 restores banks, history, surfaces/resources, and retained gfx, unwinds the obsolete managed call chain, then reconstructs saved frames through opcode 0xae.
|
||||
|
||||
### 0x1a2 `store-shared-profile-int` (store-shared-profile-int, argc 1)
|
||||
- **summary:** 0x1a2 (cell) — snapshot the selected global integer cell into AGE's shared SAVE.DAT profile table. Its native key is raw type byte `0x03` followed by eight lowercase ASCII hex digits for the lvalue's resolved global-bank index; the stored value is the cell's current raw 32-bit value. Insert-or-assign semantics replace an existing entry.
|
||||
|
||||
@@ -3468,6 +3468,37 @@ numbered-load restoration rendezvous `0xae` remains effectful and unimplemented
|
||||
the active branch of `0xae`; retain `0x19f` as the mapped but corpus-unused data-only load path. JSON
|
||||
inspection/export remains an additive extended-mode feature.
|
||||
|
||||
### Persistence implementation step 5 — complete layout-3 numbered save/load (2026-07-24)
|
||||
|
||||
The numbered body and resume path are now implemented as native binary compatibility mode.
|
||||
`NativeNumberedSaveCodec` reads/writes the `0x5718 + cutoff*0x414` fixed state, six separate native
|
||||
global banks, CP932 string blob, 100 resource records, 1,000 surface records, retained `0x2d4`-byte gfx
|
||||
objects, and range transform. `NativeTextHistoryCodec` owns the separate 12-byte-header LZSS history tail.
|
||||
The installed read-only `SAVE00.DAT` oracle decodes end to end: its container/body boundary, cutoff,
|
||||
`[402459,1,789,1,1,1]` banks, 211 retained objects, and 21-byte history tail all agree with the native
|
||||
serializer allocation and write order.
|
||||
|
||||
SYS4 parsing now retains all three footer tables. T1 is the saved message/reset resume table, T2 is the
|
||||
saved call-script table, and T3 maps intra-script returns. `ExecFrame.Pc` is kept current at opcode
|
||||
boundaries so `0x19e` captures frames root-through-`0x1ad` cutoff accurately. Full `0x1a1` load restores
|
||||
the global banks, history, reloadable surfaces, retained objects, and playtime, then deliberately unwinds
|
||||
the obsolete C# recursion, executes `CALLBACK_LOAD.BIN` when available, and enters the restore rendezvous.
|
||||
The active `0xae` path rebuilds the saved nested frames and resumes the terminal frame through T1;
|
||||
inactive `0xae` stays a no-op. Corpus-unused `0x19f` shares the state decoder without
|
||||
activating frame/history restoration. Successful `0x19e` appends history and flushes shared SAVE.DAT/RT.DAT.
|
||||
|
||||
The VM now carries integer, float, string, integer-pointer, and string-pointer banks separately across
|
||||
`GameSession`, correcting the prior flat-bank approximation while keeping compatibility fallbacks for
|
||||
older synthetic pointer tests. Focused coverage includes logical byte round-trips, CP932/LZSS history,
|
||||
container-tail preservation, native installed-save import, opcode save integration, retained gfx/surface
|
||||
restore, and a two-frame `0xae` reconstruction.
|
||||
|
||||
Validation: all 383 engine tests pass, including the installed read-only layout-3 oracle; SC0000 is now
|
||||
129/129 distinct opcodes handled. Opcode sources were regenerated after recording the T1/T2/T3 and
|
||||
implementation status. The remaining 1.0 persistence work is
|
||||
gameplay validation through Himegari's real SAVE.BIN UI and any corrections that reveals; JSON inspection,
|
||||
namespaced mod data, and migrations remain extended-mode work.
|
||||
|
||||
## Data-semantics sidebar: focused append EBINIT inspection (2026-07-24)
|
||||
|
||||
The static INIT surface now accepts a universal packed script id for focused append inspection.
|
||||
|
||||
@@ -39,12 +39,12 @@ Verified across all 481 files.
|
||||
```
|
||||
off field meaning evidence
|
||||
0x00 magic "SYS4422 " (0x53 59 53 34 34 32 32 20) 481/481 identical
|
||||
0x08 F0 scenario/script id or flags 98 distinct; 0x5 dominant (252×)
|
||||
0x0C F1 = 1 always 481/481 == 1
|
||||
0x10 F2 = 1 (457×) or 2 (20×); 4 outliers format/feature flag
|
||||
0x14 F3 0x06 (268×),0x04,0x01,0x05... 15 distinct — minor version?
|
||||
0x18 F4 = 1 always 481/481 == 1
|
||||
0x1C F5 = 2 (319×) or 1 (144×); rarely 3/5 flag
|
||||
0x08 F0 local integer count
|
||||
0x0C F1 local float count
|
||||
0x10 F2 local string count
|
||||
0x14 F3 local integer-pointer count
|
||||
0x18 F4 second pointer-family count (exact subtype unresolved)
|
||||
0x1C F5 local string-pointer count
|
||||
0x20 F6 = 0x1C always 481/481 == 0x1C (header-size marker)
|
||||
0x24 F7 table-1 entry count see below
|
||||
0x28 F8 table-1 offset == code-section length ordering F8<=F10<=F12<=EOF
|
||||
@@ -65,8 +65,8 @@ consistent with the SYS4/SYS5 family.
|
||||
```
|
||||
body[0 .. F8) CODE bytecode instruction stream
|
||||
body[F8 .. F10) TABLE-1 (F7 entries, 1 dword each) -> read-message boundaries (op 0x71)
|
||||
body[F10 .. F12) TABLE-2 (F9 entries, 1 dword each) -> targets of type 0x03
|
||||
body[F12 .. EOF) TABLE-3 (F11 entries, 1 dword each) -> targets of type 0x8F
|
||||
body[F10 .. F12) TABLE-2 (F9 entries, 1 dword each) -> call-script sites (op 0x03)
|
||||
body[F12 .. EOF) TABLE-3 (F11 entries, 1 dword each) -> local-call sites (op 0x8F)
|
||||
```
|
||||
|
||||
Every table entry is exactly **1 dword** — a pointer (dword index into body).
|
||||
@@ -81,16 +81,15 @@ tag identifying the pointed-to construct:
|
||||
| Table | count/off | Target dword tag | Hits | Meaning (inferred) |
|
||||
|---|---|---|---|---|
|
||||
| T1 | F7 / F8 | **0x71** | 26,445/26,445 | per-script read-message boundary index |
|
||||
| T2 | F9 / F10 | **0x03** | 3,018/3,018 | data/variable entries (operand at +2 large, e.g. addresses) |
|
||||
| T3 | F11 / F12| **0x8F** | 72,941/72,941 | instruction/line entries (largest table; operand at +2 huge) |
|
||||
| T2 | F9 / F10 | **0x03** | 3,018/3,018 | resumable call-script sites |
|
||||
| T3 | F11 / F12| **0x8F** | 72,941/72,941 | local-call sites used to reconstruct return stacks |
|
||||
|
||||
100% type purity — not a single target had a different tag. Native
|
||||
`read_text_db_find_message_index@0x468f50` searches T1 for the code DWORD coordinate most recently
|
||||
snapshotted by op `0x71`; its zero-based entry index and F7 count are the message index/count stored in
|
||||
shared `RT.DAT`. Thus T1 is not a generic label table even though every entry is a control-structure
|
||||
site. T3 is the big one
|
||||
(~73k entries corpus-wide), consistent with it being a per-instruction or
|
||||
per-source-line index (a debug/line table). T1 ≈ labels, T2 ≈ a smaller symbol set.
|
||||
site. Numbered layout-3 frame serialization proves the remaining roles: saved d259 indexes T1,
|
||||
d260 indexes T2, and each intra-script return maps through T3 (restored as `T3[index]+3`).
|
||||
|
||||
## Instruction stream — PARTIAL
|
||||
|
||||
@@ -161,7 +160,6 @@ The header format is identical (same magic/layout) so tooling is copy-agnostic.
|
||||
- Opcode dispatch — confirm tagged-operand model, enumerate opcodes
|
||||
- Meaning of F0/F2/F3/F5 flag fields
|
||||
- Exact operand grammar per instruction (how many dwords each opcode consumes)
|
||||
- Semantics of T1/T2/T3 beyond "label/data/line" guesses
|
||||
|
||||
## Loader — DONE
|
||||
|
||||
|
||||
Reference in New Issue
Block a user