From 001a5706f6c6d33051323328e928d4d8999a6ea0 Mon Sep 17 00:00:00 2001 From: gamer147 Date: Fri, 24 Jul 2026 16:23:01 -0400 Subject: [PATCH] Implement native layout-3 save restoration --- docs/PROJECT-STRUCTURE.md | 3 +- docs/engine-re.md | 58 ++- docs/opcode-reference.md | 10 +- docs/phase-a-slice-plan.md | 31 ++ docs/sys4-format-notes.md | 26 +- .../NativeNumberedSaveCodecTests.cs | 146 ++++++++ .../Age.Engine.Tests/NumberedSaveVmTests.cs | 226 ++++++++++++ engine/Age.Engine.Tests/TestSupport.cs | 8 +- engine/Age.Engine/Diagnostics/TraceEvent.cs | 2 +- engine/Age.Engine/Hosting/IScriptProvider.cs | 3 + engine/Age.Engine/Model/AdvTextHistory.cs | 33 +- engine/Age.Engine/Model/GfxState.cs | 106 ++++-- engine/Age.Engine/Model/Script.cs | 4 + .../Age.Engine/Persistence/NativeDatStore.cs | 38 ++ .../Persistence/NativeGfxPersistenceCodec.cs | 200 +++++++++++ .../Persistence/NativeNumberedSaveCodec.cs | 294 +++++++++++++++ .../Persistence/NativeTextHistoryCodec.cs | 179 ++++++++++ engine/Age.Engine/Sys4/Sys4Loader.cs | 13 + engine/Age.Engine/Vm/GameSession.cs | 15 +- engine/Age.Engine/Vm/VirtualMachine.cs | 336 +++++++++++++++++- vm-map/opcodes.toml | 9 +- 21 files changed, 1659 insertions(+), 81 deletions(-) create mode 100644 engine/Age.Engine.Tests/NativeNumberedSaveCodecTests.cs create mode 100644 engine/Age.Engine.Tests/NumberedSaveVmTests.cs create mode 100644 engine/Age.Engine/Persistence/NativeGfxPersistenceCodec.cs create mode 100644 engine/Age.Engine/Persistence/NativeNumberedSaveCodec.cs create mode 100644 engine/Age.Engine/Persistence/NativeTextHistoryCodec.cs diff --git a/docs/PROJECT-STRUCTURE.md b/docs/PROJECT-STRUCTURE.md index 2a64bcf..4470c83 100644 --- a/docs/PROJECT-STRUCTURE.md +++ b/docs/PROJECT-STRUCTURE.md @@ -104,7 +104,8 @@ S:\Game Hacking\Eushully\Himegari\ ← workspace root (three siblings) │ ├── Age.Engine/Sys4/ runtime catalog parser, loose-first bounded ALF asset store, │ script provider, AGF/LZSS and Windows CUR decoders, and resource facade │ └── Age.Engine/Persistence/ native S3SD/S4SD + S3RT codecs, shared payload/ReadTextDB, - │ numbered DAT/STH pair + BMP codec, and profile-owned state + │ layout-3 numbered state/history/gfx, DAT/STH pair + BMP codec, + │ and profile-owned state ├── native/ authored native runtime boundaries │ └── age_movie_ffmpeg/ project-owned FFmpeg C ABI, immutable Windows dependency manifest, │ and bootstrap/build scripts (outputs stay under disposable build/) diff --git a/docs/engine-re.md b/docs/engine-re.md index 5a359b1..ecb66fe 100644 --- a/docs/engine-re.md +++ b/docs/engine-re.md @@ -1801,8 +1801,44 @@ validates its distinct compatibility id, queries metadata without decoding paylo paired `.DAT`/`.STH` copy/delete status layering. `NumberedThumbnailCodec` reads and writes the native BMP dialect through host surface capture/replacement, and ops `0x1a0`, `0x1ab`–`0x1af` (including `0x1ad`'s frame marker) are wired. Godot injects the store at `user://SAVE`. Full layout-3 global/frame/history/ -resource/retained-gfx serialization and the `0x19e`/`0x1a1`/`0xae` save-resume path remain the next layer; -the existing `GameSession` JSON snapshot is unchanged. +`NativeNumberedSaveCodec`, `NativeTextHistoryCodec`, and `NativeGfxPersistenceCodec` now own the complete +layout-3 numbered body and appended history tail. The VM wires `0x19e`, data-only `0x19f`, full load +`0x1a1`, and the active branch of `0xae`; the existing `GameSession` JSON snapshot remains a separate +diagnostic/extended-mode surface. + +#### Layout-3 byte map and port correspondence (2026-07-24) + +The decoded layout-3 body begins with `cutoff` at `+0x00`; its fixed length is +`0x5718 + cutoff*0x414`. The fixed prefix contains two saved engine/context words, ten state DWORDs, +100 three-DWORD resource reload records, 1,000 20-byte surface reload records, then frames beginning at +`+0x5304`. Each `0x414`-byte frame contains parent context, packed script id, up to 256 saved local-return +indices, and two terminal control indices. The three SYS4 footer tables now have explicit runtime roles: + +- d259 indexes T1/F7-F8, the op-`0x71` read-message reset sites; +- d260 indexes T2/F9-F10, resumable `call-script` sites (forced to `-1` on the cutoff frame); +- each local return index addresses T3/F11-F12, whose stored target plus three is the live return offset. + +After the fixed bytes are six bank counts followed by the ordinary (de-protected) integer bank, float +storage, a DWORD-sized CP932 string blob, the pointer-family banks, and retained graphics. Graphics begin +with record size `0x2d4` and object count; every object is `{handle,0x2d4 raw record}`. The range-transform +first/count and another `0x2d4` record follow. AGE overallocates this region +(`0x2e1 + objectCount*0x2d8` DWORDs) relative to the records it writes; the codec preserves the native +logical sizing and zeroes the unused allocation slack. + +The text-history tail is outside the S4SD container. Its 12-byte header is +`{logicalBytes,logicalBytes,storedBytes}` followed by the same 4 KiB-ring LZSS/verbatim dialect. Logical +history is entry count + 8-byte entries, record count + the first 11 DWORDs of each retained record, then +a DWORD-counted CP932 string blob whose DWORDs are bitwise inverted. Installed `SAVE00.DAT` validates the +whole boundary read-only: cutoff 1, six counts `[402459,1,789,1,1,1]`, 211 gfx objects, a 21-byte history +tail, and exact consumption of the S4SD container before that tail. + +The port tracks the native global banks separately at runtime, captures the marked frame chain, serializes +live surfaces and retained objects, and reloads host textures from the 20-byte surface records. Full load +first replaces banks/history/gfx, unwinds the obsolete managed call chain, runs `CALLBACK_LOAD.BIN` when +the mounted script provider resolves it, starts the saved root at its `0xae` rendezvous, recursively +reconstructs child frames, resumes parents after their saved T2 call sites, +and finally resumes the terminal frame at its T1 boundary. Successful `0x19e` also flushes shared +`SAVE.DAT`/`RT.DAT`, matching `context_state_serialize`. ### Opcode `0xae` continues numbered-save stack restoration (2026-07-20) @@ -1815,14 +1851,13 @@ loads `CALLBACK_LOAD.BIN` or the saved entry script, and resets the current cont On an active restore, the handler reads `set:SaveVersion1`/`set:SaveVersion2`, selects the matching saved frame layout, replaces the current frame PC with its saved resume or call target, and advances through the -serialized contexts. At the saved terminal context it clears the restore flag and reinstates the saved -context/return state. The corpus placement supports that control-flow role: 305 calls overwhelmingly follow +serialized contexts. `script_frame_restore_saved_layout@0x40f2d0` loads each packed script and expands its +T1/T2/T3 indices back into live PC/call/return offsets. At the saved terminal context the handler clears +the restore flag and reinstates the saved context/return state. The corpus placement supports that control-flow role: 305 calls overwhelmingly follow coroutine-resume or call boundaries, including SC0000's main-loop resume sequence. -The opcode remains an effectful port gap. It is a no-op during every currently reachable port execution, -but its actual branch cannot be implemented until numbered saves serialize and restore the active -`ExecFrame` chain. Counting an unconditional no-op as coverage would conceal that dependency, so it stays -grouped with `0x1ad` rather than receiving a placeholder VM case. +The port now implements both branches: an ordinary one-instruction no-op outside restoration, and the +T1/T2/T3-driven managed-frame reconstruction described above while a full numbered load is active. ### ADV read-message Skip and shared `RT.DAT` history (2026-07-18) @@ -1947,10 +1982,9 @@ finds typed metadata; op `0x1d4` finds the voice pair. `HISTORY.BIN` dispatches History is independent of `RT.DAT`, but native full save fidelity does serialize the live backlog. `text_history_serialize@0x451d00` writes the index and packed records/strings after context/numbered-save -serialization, and `text_history_deserialize@0x456130` restores them on the matching load path. This is a -future numbered-save integration seam, not a reason to choose a shared-profile backend now: an in-memory -History button can be complete first, while save/load restoration stays deferred with the wider storage -architecture decision. +serialization, and `text_history_deserialize@0x456130` restores them on the matching load path. The port's +`NativeTextHistoryCodec` now implements that appended tail directly against the same live +`AdvTextHistory` model; it remains independent of shared profile state and `RT.DAT`. History's display support consists of the ordinary presentation operations: primary/ruby font sizes (`0x75/0x197`), font weight (`0x2bd`), colors/effect mode/offset (`0x76/0x77/0x78/0x1a4`), layout origin diff --git a/docs/opcode-reference.md b/docs/opcode-reference.md index 968e8e5..dbf0c08 100644 --- a/docs/opcode-reference.md +++ b/docs/opcode-reference.md @@ -400,7 +400,9 @@ Implemented as a whole-stack root-reload boundary in the persistent VM. A reques ### 0xae `continue-save-load-stack-restore` (u00415130, argc 0) - **summary:** () - during serialized save restoration, replace the current frame PC with its saved resume/call target and advance through the saved script-context stack; otherwise a no-op. - **grounding:** source=investigation, confidence=high -- **evidence:** Ghidra /v2: op_0xae_continue_save_load_stack_restore@0x416790 first tests ctx+0x53d24 (set by save_data_deserialize_and_begin_restore@0x40fd10). When clear it returns. When set, it selects the serialized frame layout through set:SaveVersion1/2, restores the current PC from that layout's saved return/call target, advances through contexts with FUN_0040f2d0, and clears the restore flag on reaching the saved terminal context. Its 305 corpus sites overwhelmingly follow coroutine-resume/call boundaries, which provide the rendezvous points used while reconstructing the stack. +- **evidence:** Ghidra /v2: op_0xae_continue_save_load_stack_restore@0x416790 first tests ctx+0x53d24 (set by save_data_deserialize_and_begin_restore@0x40fd10). When clear it returns. When set, it selects the serialized frame layout through set:SaveVersion1/2, restores the current PC from that layout's saved return/call target, advances through contexts with script_frame_restore_saved_layout@0x40f2d0, and clears the restore flag on reaching the saved terminal context. Its 305 corpus sites overwhelmingly follow coroutine-resume/call boundaries, which provide the rendezvous points used while reconstructing the stack. + +Layout 3 frame d259 indexes SYS4 T1 read-message reset sites, d260 indexes T2 call-script sites, and the saved local return stack indexes T3 local-call sites. Port status (2026-07-24): the active path reconstructs the saved recursive frame chain and resumes the terminal frame at its T1 boundary. ### 0xc8 `sleep` (sleep, argc 1) - **summary:** Pause the current script for milliseconds while retained presentation continues. @@ -476,14 +478,14 @@ record-zero failure. Natural SYSTEM4 boot proves BTANINIT2 -> `$1$AUTORUN.BIN` - - **grounding:** source=investigation, confidence=high - **evidence:** Ghidra /v2: op_0x19e_save_numbered_slot@0x4278b0 formats SAVE%2.2d.DAT, checks an existing header and prompts before replacing an incompatible file, opens CREATE_ALWAYS, reads set:SaveVersion2 then set:SaveVersion1, and calls context_state_serialize@0x40d320. Corpus: two calls in SAVE.BIN and SELSTAGE.BIN. -Uses `set:SaveVersion1` and `set:SaveVersion2` to choose the numbered payload layout. Status is 0 on success and 1 on refusal, open/create failure, or serializer failure. Numbered `.DAT` files do not use the shared profile's temp/backup replacement scheme. +Uses `set:SaveVersion1` and `set:SaveVersion2` to choose the numbered payload layout. Status is 0 on success and 1 on refusal, open/create failure, or serializer failure. Numbered `.DAT` files do not use the shared profile's temp/backup replacement scheme. Port status (2026-07-24): layout 3 writes fixed state, six native banks, T1/T2/T3 frame records, surface/resource reload state, retained gfx records, the appended history tail, and then flushes shared SAVE.DAT/RT.DAT. ### 0x19f `load-numbered-slot-data-only` (load-numbered-slot-data-only, argc 2) - **summary:** (status_out)(slot) - decode `SAVE%02d.DAT` without restoring the active script-frame chain or text history. - **grounding:** source=investigation, confidence=high - **evidence:** Ghidra /v2: op_0x19f_load_numbered_slot_data_only@0x427a40 opens SAVE%2.2d.DAT and calls save_data_deserialize_and_begin_restore@0x40fd10(handle, SaveVersion1, SaveVersion2, 0, 0), then decodes protected integer globals. Missing/open failure writes status 1; otherwise the decoder result is returned. Corpus count: 0. -This is the data-only companion to full-resume opcode 0x1a1. It selects the configured SaveVersion layout and restores serialized state with both runtime/history restore flags clear. Himegari's shipped script corpus does not call it, but it belongs to the shared SYS4 persistence ABI. +This is the data-only companion to full-resume opcode 0x1a1. It selects the configured SaveVersion layout and restores serialized state with both runtime/history restore flags clear. Himegari's shipped script corpus does not call it, but it belongs to the shared SYS4 persistence ABI. Port status (2026-07-24): implemented for layout 3 through the same bank/resource/gfx decoder without activating history or frame restoration. ### 0x1a0 `query-numbered-save-metadata` (query-numbered-save-metadata, argc 9) - **summary:** (status_out)(slot)(year)(month)(day)(hour)(minute)(second)(playtime_seconds) - validate a numbered `.DAT` header and return its timestamp and accumulated playtime. @@ -497,7 +499,7 @@ Status 0 means valid metadata was written, 1 means the file could not be opened, - **grounding:** source=investigation, confidence=high - **evidence:** Ghidra /v2: op_0x1a1_load_numbered_slot_and_resume@0x427d30 calls save_data_deserialize_and_begin_restore@0x40fd10(handle, SaveVersion1, SaveVersion2, 1, 1), closes the file, and decodes protected integer globals. The loader activates the saved-frame state consumed by op_0xae. Corpus: one call in SAVE.BIN. -The caller pre-seeds status to zero. A missing/open failure writes 1; success starts the asynchronous native stack-restoration rendezvous and does not overwrite that zero. This is the ordinary load-game path, unlike data-only opcode 0x19f. +The caller pre-seeds status to zero. A missing/open failure writes 1; success starts the asynchronous native stack-restoration rendezvous and does not overwrite that zero. This is the ordinary load-game path, unlike data-only opcode 0x19f. Port status (2026-07-24): layout 3 restores banks, history, surfaces/resources, and retained gfx, unwinds the obsolete managed call chain, then reconstructs saved frames through opcode 0xae. ### 0x1a2 `store-shared-profile-int` (store-shared-profile-int, argc 1) - **summary:** 0x1a2 (cell) — snapshot the selected global integer cell into AGE's shared SAVE.DAT profile table. Its native key is raw type byte `0x03` followed by eight lowercase ASCII hex digits for the lvalue's resolved global-bank index; the stored value is the cell's current raw 32-bit value. Insert-or-assign semantics replace an existing entry. diff --git a/docs/phase-a-slice-plan.md b/docs/phase-a-slice-plan.md index fc536a2..0784558 100644 --- a/docs/phase-a-slice-plan.md +++ b/docs/phase-a-slice-plan.md @@ -3468,6 +3468,37 @@ numbered-load restoration rendezvous `0xae` remains effectful and unimplemented the active branch of `0xae`; retain `0x19f` as the mapped but corpus-unused data-only load path. JSON inspection/export remains an additive extended-mode feature. +### Persistence implementation step 5 — complete layout-3 numbered save/load (2026-07-24) + +The numbered body and resume path are now implemented as native binary compatibility mode. +`NativeNumberedSaveCodec` reads/writes the `0x5718 + cutoff*0x414` fixed state, six separate native +global banks, CP932 string blob, 100 resource records, 1,000 surface records, retained `0x2d4`-byte gfx +objects, and range transform. `NativeTextHistoryCodec` owns the separate 12-byte-header LZSS history tail. +The installed read-only `SAVE00.DAT` oracle decodes end to end: its container/body boundary, cutoff, +`[402459,1,789,1,1,1]` banks, 211 retained objects, and 21-byte history tail all agree with the native +serializer allocation and write order. + +SYS4 parsing now retains all three footer tables. T1 is the saved message/reset resume table, T2 is the +saved call-script table, and T3 maps intra-script returns. `ExecFrame.Pc` is kept current at opcode +boundaries so `0x19e` captures frames root-through-`0x1ad` cutoff accurately. Full `0x1a1` load restores +the global banks, history, reloadable surfaces, retained objects, and playtime, then deliberately unwinds +the obsolete C# recursion, executes `CALLBACK_LOAD.BIN` when available, and enters the restore rendezvous. +The active `0xae` path rebuilds the saved nested frames and resumes the terminal frame through T1; +inactive `0xae` stays a no-op. Corpus-unused `0x19f` shares the state decoder without +activating frame/history restoration. Successful `0x19e` appends history and flushes shared SAVE.DAT/RT.DAT. + +The VM now carries integer, float, string, integer-pointer, and string-pointer banks separately across +`GameSession`, correcting the prior flat-bank approximation while keeping compatibility fallbacks for +older synthetic pointer tests. Focused coverage includes logical byte round-trips, CP932/LZSS history, +container-tail preservation, native installed-save import, opcode save integration, retained gfx/surface +restore, and a two-frame `0xae` reconstruction. + +Validation: all 383 engine tests pass, including the installed read-only layout-3 oracle; SC0000 is now +129/129 distinct opcodes handled. Opcode sources were regenerated after recording the T1/T2/T3 and +implementation status. The remaining 1.0 persistence work is +gameplay validation through Himegari's real SAVE.BIN UI and any corrections that reveals; JSON inspection, +namespaced mod data, and migrations remain extended-mode work. + ## Data-semantics sidebar: focused append EBINIT inspection (2026-07-24) The static INIT surface now accepts a universal packed script id for focused append inspection. diff --git a/docs/sys4-format-notes.md b/docs/sys4-format-notes.md index 1a33f3e..e8403ad 100644 --- a/docs/sys4-format-notes.md +++ b/docs/sys4-format-notes.md @@ -39,12 +39,12 @@ Verified across all 481 files. ``` off field meaning evidence 0x00 magic "SYS4422 " (0x53 59 53 34 34 32 32 20) 481/481 identical -0x08 F0 scenario/script id or flags 98 distinct; 0x5 dominant (252×) -0x0C F1 = 1 always 481/481 == 1 -0x10 F2 = 1 (457×) or 2 (20×); 4 outliers format/feature flag -0x14 F3 0x06 (268×),0x04,0x01,0x05... 15 distinct — minor version? -0x18 F4 = 1 always 481/481 == 1 -0x1C F5 = 2 (319×) or 1 (144×); rarely 3/5 flag +0x08 F0 local integer count +0x0C F1 local float count +0x10 F2 local string count +0x14 F3 local integer-pointer count +0x18 F4 second pointer-family count (exact subtype unresolved) +0x1C F5 local string-pointer count 0x20 F6 = 0x1C always 481/481 == 0x1C (header-size marker) 0x24 F7 table-1 entry count see below 0x28 F8 table-1 offset == code-section length ordering F8<=F10<=F12<=EOF @@ -65,8 +65,8 @@ consistent with the SYS4/SYS5 family. ``` body[0 .. F8) CODE bytecode instruction stream body[F8 .. F10) TABLE-1 (F7 entries, 1 dword each) -> read-message boundaries (op 0x71) -body[F10 .. F12) TABLE-2 (F9 entries, 1 dword each) -> targets of type 0x03 -body[F12 .. EOF) TABLE-3 (F11 entries, 1 dword each) -> targets of type 0x8F +body[F10 .. F12) TABLE-2 (F9 entries, 1 dword each) -> call-script sites (op 0x03) +body[F12 .. EOF) TABLE-3 (F11 entries, 1 dword each) -> local-call sites (op 0x8F) ``` Every table entry is exactly **1 dword** — a pointer (dword index into body). @@ -81,16 +81,15 @@ tag identifying the pointed-to construct: | Table | count/off | Target dword tag | Hits | Meaning (inferred) | |---|---|---|---|---| | T1 | F7 / F8 | **0x71** | 26,445/26,445 | per-script read-message boundary index | -| T2 | F9 / F10 | **0x03** | 3,018/3,018 | data/variable entries (operand at +2 large, e.g. addresses) | -| T3 | F11 / F12| **0x8F** | 72,941/72,941 | instruction/line entries (largest table; operand at +2 huge) | +| T2 | F9 / F10 | **0x03** | 3,018/3,018 | resumable call-script sites | +| T3 | F11 / F12| **0x8F** | 72,941/72,941 | local-call sites used to reconstruct return stacks | 100% type purity — not a single target had a different tag. Native `read_text_db_find_message_index@0x468f50` searches T1 for the code DWORD coordinate most recently snapshotted by op `0x71`; its zero-based entry index and F7 count are the message index/count stored in shared `RT.DAT`. Thus T1 is not a generic label table even though every entry is a control-structure -site. T3 is the big one -(~73k entries corpus-wide), consistent with it being a per-instruction or -per-source-line index (a debug/line table). T1 ≈ labels, T2 ≈ a smaller symbol set. +site. Numbered layout-3 frame serialization proves the remaining roles: saved d259 indexes T1, +d260 indexes T2, and each intra-script return maps through T3 (restored as `T3[index]+3`). ## Instruction stream — PARTIAL @@ -161,7 +160,6 @@ The header format is identical (same magic/layout) so tooling is copy-agnostic. - Opcode dispatch — confirm tagged-operand model, enumerate opcodes - Meaning of F0/F2/F3/F5 flag fields - Exact operand grammar per instruction (how many dwords each opcode consumes) -- Semantics of T1/T2/T3 beyond "label/data/line" guesses ## Loader — DONE diff --git a/engine/Age.Engine.Tests/NativeNumberedSaveCodecTests.cs b/engine/Age.Engine.Tests/NativeNumberedSaveCodecTests.cs new file mode 100644 index 0000000..cc1a4f2 --- /dev/null +++ b/engine/Age.Engine.Tests/NativeNumberedSaveCodecTests.cs @@ -0,0 +1,146 @@ +using Age.Engine.Model; +using Age.Engine.Persistence; + +public class NativeNumberedSaveCodecTests +{ + [Fact] + public void LayoutThreeRoundTripsNativeBanksFramesAndGfxRecords() + { + var frames = new[] + { + new NativeSavedScriptFrame(-1, 0, new[] { 2, 4 }, 8, 7), + new NativeSavedScriptFrame(0, 0x3389, Array.Empty(), 3, -1), + }; + NativeNumberedSaveState state = NativeNumberedSaveCodec.Empty(frames) with + { + SavedFrameOwner = 7, + EngineState = 9, + StateWords = Enumerable.Range(10, 10).ToArray(), + IntegerGlobals = new[] { 12, -3, 0x12345678 }, + FloatGlobals = new[] { BitConverter.SingleToInt32Bits(1.25f) }, + StringGlobals = new[] { "姫狩り", "", "save" }, + PointerGlobals = new[] { 2 }, + PointerStrings = new[] { 0 }, + LocalPointerScratch = new[] { -1 }, + GfxObjects = + [ + new NativeSavedGfxObject(0xcf08, + Enumerable.Range(0, NativeNumberedSaveState.GfxRecordSize) + .Select(i => unchecked((byte)i)).ToArray()), + ], + RangeTransformFirst = 100, + RangeTransformCount = 4, + RangeTransformRecord = Enumerable.Repeat((byte)0x5a, + NativeNumberedSaveState.GfxRecordSize).ToArray(), + }; + + byte[] encoded = NativeNumberedSaveCodec.Encode(state); + NativeNumberedSaveState decoded = NativeNumberedSaveCodec.Decode(encoded); + + Assert.Equal(state.SavedFrameOwner, decoded.SavedFrameOwner); + Assert.Equal(state.EngineState, decoded.EngineState); + Assert.Equal(state.StateWords, decoded.StateWords); + Assert.Equal(state.Frames[0].ParentContext, decoded.Frames[0].ParentContext); + Assert.Equal(state.Frames[0].ScriptId, decoded.Frames[0].ScriptId); + Assert.Equal(state.Frames[0].ReturnIndices, decoded.Frames[0].ReturnIndices); + Assert.Equal(state.Frames[0].ResumeIndex, decoded.Frames[0].ResumeIndex); + Assert.Equal(state.Frames[0].CallTargetIndex, decoded.Frames[0].CallTargetIndex); + Assert.Equal(-1, decoded.Frames[1].CallTargetIndex); + Assert.Equal(state.IntegerGlobals, decoded.IntegerGlobals); + Assert.Equal(state.FloatGlobals, decoded.FloatGlobals); + Assert.Equal(state.StringGlobals, decoded.StringGlobals); + Assert.Equal(state.PointerGlobals, decoded.PointerGlobals); + Assert.Equal(state.GfxObjects[0].Handle, decoded.GfxObjects[0].Handle); + Assert.Equal(state.GfxObjects[0].Record, decoded.GfxObjects[0].Record); + Assert.Equal(state.RangeTransformRecord, decoded.RangeTransformRecord); + } + + [Fact] + public void HistoryTailRoundTripsLogicalEntriesRecordsAndCp932Text() + { + var history = new AdvTextHistory(); + history.DefineLayout(1, 400, 120, 75, 340); + history.SetCursor(1, 8, 12); + history.AppendText(1, 0x123, "セーブ履歴", AdvTextStyle.Default with + { + PrimaryFontSize = 24, + TextColor = 0xff112233, + }); + history.AppendMetadata(42, 7, AdvTextStyle.Default); + + byte[] encoded = NativeTextHistoryCodec.Encode(history); + var restored = new AdvTextHistory(); + NativeTextHistoryCodec.DecodeInto(encoded, restored); + + Assert.Equal(history.Entries, restored.Entries); + Assert.Equal(2, restored.Records.Count); + Assert.Equal("セーブ履歴", restored.Records[0].Text); + Assert.Equal(42, restored.Records[1].Value); + Assert.Equal(7, restored.Records[1].AuxValue); + Assert.Equal(AdvTextHistoryRecordKind.Metadata, restored.Records[1].Kind); + } + + [Fact] + public void DirectoryStorePreservesHistoryTailAfterNativeContainer() + { + string root = Path.Combine(Path.GetTempPath(), "age-numbered-tail-" + Guid.NewGuid().ToString("N")); + try + { + var identity = new NativeSaveIdentity( + NativeSaveMagic.S4SD, 1, "test", 3, 10, NumberedCompatibilityId: 2); + var store = new DirectoryNativeDatStore(root, identity); + byte[] payload = NativeNumberedSaveCodec.Encode(NativeNumberedSaveCodec.Empty( + [new NativeSavedScriptFrame(-1, 0, Array.Empty(), -1, -1)])); + byte[] tail = [1, 2, 3, 4, 5]; + + store.SaveNumberedFile( + 3, payload, tail, + new NativeSystemTime(2026, 7, 5, 24, 12, 30, 0, 0), 99); + NativeNumberedSaveFile loaded = store.LoadNumberedFile(3)!; + + Assert.Equal(payload, loaded.Document.Payload); + Assert.Equal(tail, loaded.HistoryTail); + } + finally + { + if (Directory.Exists(root)) Directory.Delete(root, recursive: true); + } + } + + [Fact] + public void InstalledHimegariLayoutThreeAndHistoryTailDecodeWhenPresent() + { + string eushullyRoot = Path.Combine( + Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "Eushully"); + if (!Directory.Exists(eushullyRoot)) return; + string? path = Directory.EnumerateFiles( + eushullyRoot, "SAVE00.DAT", SearchOption.AllDirectories) + .FirstOrDefault(candidate => + { + try + { + NativeSaveMetadata metadata = NativeSaveContainerCodec.ReadMetadata( + File.ReadAllBytes(candidate)); + return metadata.CompatibilityId == 0x42323234 + && metadata.SaveVersion1 == 3 && metadata.SaveVersion2 == 10; + } + catch (Exception error) when ( + error is IOException or UnauthorizedAccessException or InvalidDataException) + { + return false; + } + }); + if (path == null) return; + + byte[] source = File.ReadAllBytes(path); + NativeSaveDocument document = NativeSaveContainerCodec.Decode(source); + NativeNumberedSaveState state = NativeNumberedSaveCodec.Decode(document.Payload); + var history = new AdvTextHistory(); + NativeTextHistoryCodec.DecodeInto(source.AsSpan(document.BytesConsumed), history); + + Assert.NotEmpty(state.Frames); + Assert.Equal(402459, state.IntegerGlobals.Count); + Assert.Equal(789, state.StringGlobals.Count); + Assert.Equal(NativeNumberedSaveState.GfxRecordSize, state.RangeTransformRecord.Length); + } +} diff --git a/engine/Age.Engine.Tests/NumberedSaveVmTests.cs b/engine/Age.Engine.Tests/NumberedSaveVmTests.cs new file mode 100644 index 0000000..4836b43 --- /dev/null +++ b/engine/Age.Engine.Tests/NumberedSaveVmTests.cs @@ -0,0 +1,226 @@ +using Age.Engine.Model; +using Age.Engine.Persistence; +using Age.Engine.Sys4; +using Age.Engine.Vm; +using System.Buffers.Binary; + +public class NumberedSaveVmTests +{ + private const int Immediate = 0; + private const int GlobalInt = 3; + private const int LocalInt = 9; + private static readonly OpcodeTable Table = OpcodeTableJson.Load(Paths.OpcodesJson); + private static readonly NativeSaveIdentity Identity = + new(NativeSaveMagic.S4SD, 0x4a343234, "numbered-vm-test", 3, 10, 0x42323234); + + [Fact] + public void SaveOpcodeWritesLayoutThreeStateHistoryAndRetainedGfx() + { + string root = NewTemporaryDirectory(); + try + { + var store = new DirectoryNativeDatStore(root, Identity); + Script script = WithPackedId(ScriptAssembler.Assemble(Table, "SAVE_TEST.BIN", + [ + (0x1ad, Array.Empty()), + (0x19e, [new Operand(GlobalInt, 0x20), new Operand(Immediate, 2)]), + (0x2, Array.Empty()), + ], []), 0x77); + var history = new AdvTextHistory(); + history.DefineLayout(1, 400, 120, 75, 340); + history.AppendText(1, 0x40, "保存", AdvTextStyle.Default); + var vm = new VirtualMachine( + script, Table, new RecordingHost(), textHistory: history, nativeDatStore: store); + vm.Globals[0x123] = 456; + vm.GlobalFloats[0] = BitConverter.SingleToInt32Bits(2.5f); + vm.GlobalStrings[4] = "姫狩り"; + vm.Gfx.SetSurface(3, 0x1234, 0xff00ff); + vm.Gfx.BindDraw(100, 3, 1, 2, 30, 40, 50, 60); + + vm.Run(); + + Assert.Equal(0, vm.Globals[0x20]); + NativeNumberedSaveFile file = store.LoadNumberedFile(2)!; + NativeNumberedSaveState state = NativeNumberedSaveCodec.Decode(file.Document.Payload); + Assert.Equal(0x6241b, state.IntegerGlobals.Count); + Assert.Equal(456, state.IntegerGlobals[0x123]); + Assert.Equal("姫狩り", state.StringGlobals[4]); + Assert.Equal(0x77u, state.Frames.Single().ScriptId); + Assert.Contains(state.GfxObjects, item => item.Handle == 100); + var restoredHistory = new AdvTextHistory(); + NativeTextHistoryCodec.DecodeInto(file.HistoryTail, restoredHistory); + Assert.Equal("保存", restoredHistory.Records.Single().Text); + Assert.NotNull(store.LoadShared()); + } + finally + { + Directory.Delete(root, recursive: true); + } + } + + [Fact] + public void FullLoadRestoresBanksThenResumesSavedScriptThroughOpcodeAe() + { + string root = NewTemporaryDirectory(); + try + { + var store = new DirectoryNativeDatStore(root, Identity); + Script resumed = WithTables(WithPackedId(ScriptAssembler.Assemble(Table, "RESUMED.BIN", + [ + (0xae, Array.Empty()), + (0x3, [new Operand(Immediate, 0x89)]), + (Table.ByLabel("mov")!.Value, + [new Operand(GlobalInt, 0x500), new Operand(GlobalInt, 0x123)]), + (0x2, Array.Empty()), + ], []), 0x88), scriptCallOffsets: [1]); + Script child = WithPackedId(ScriptAssembler.Assemble(Table, "CHILD.BIN", + [ + (0xae, Array.Empty()), + (Table.ByLabel("mov")!.Value, + [new Operand(GlobalInt, 0x501), new Operand(GlobalInt, 0x123)]), + (0x2, Array.Empty()), + ], []), 0x89); + Script loader = WithPackedId(ScriptAssembler.Assemble(Table, "LOADER.BIN", + [ + (0x1a1, [new Operand(LocalInt, 0), new Operand(Immediate, 1)]), + (0x2, Array.Empty()), + ], []), 0x99); + Script callback = WithPackedId(ScriptAssembler.Assemble(Table, "CALLBACK_LOAD.BIN", + [ + (Table.ByLabel("mov")!.Value, + [new Operand(GlobalInt, 0x502), new Operand(Immediate, 1)]), + (0x2, Array.Empty()), + ], []), 0x90); + NativeNumberedSaveState state = NativeNumberedSaveCodec.Empty( + [ + new NativeSavedScriptFrame(-1, 0x88, Array.Empty(), -1, 0), + new NativeSavedScriptFrame(0, 0x89, Array.Empty(), -1, -1), + ]) with + { + IntegerGlobals = DenseIntBank(0x124, (0x123, 456)), + FloatGlobals = [BitConverter.SingleToInt32Bits(3.5f)], + StringGlobals = ["復帰"], + PointerGlobals = [0x123], + PointerStrings = [0], + LocalPointerScratch = [0], + SurfaceRecords = NativeSurfaceRecords(3, 0x1234, 0xff00ff), + GfxObjects = [new NativeSavedGfxObject(100, NativeGfxRecord(3))], + }; + var history = new AdvTextHistory(); + history.DefineLayout(1, 100, 50, 0, 0); + history.AppendText(1, 0, "履歴復帰", AdvTextStyle.Default); + store.SaveNumberedFile( + 1, NativeNumberedSaveCodec.Encode(state), NativeTextHistoryCodec.Encode(history), + NativeSystemTime.FromLocalDateTime(DateTime.Now), 123); + var liveHistory = new AdvTextHistory(); + var vm = new VirtualMachine( + loader, Table, new RecordingHost(), provider: new MapProvider( + new() + { + [0x88] = resumed, + [0x89] = child, + }, + new(StringComparer.OrdinalIgnoreCase) + { + ["CALLBACK_LOAD.BIN"] = callback, + }), + textHistory: liveHistory, nativeDatStore: store); + vm.Globals[0x123] = 999; + + vm.Run(); + + Assert.Equal(456, vm.Globals[0x123]); + Assert.Equal(456, vm.Globals[0x500]); + Assert.Equal(456, vm.Globals[0x501]); + Assert.Equal(1, vm.Globals[0x502]); + Assert.Equal("復帰", vm.GlobalStrings[0]); + Assert.Equal(0x123, vm.GlobalPointers[0]); + Assert.Equal("履歴復帰", liveHistory.Records.Single().Text); + Assert.Equal(3, vm.Gfx.QuerySlot(100)); + RenderObject restoredObject = Assert.Single(vm.Gfx.SnapshotVisibleObjects()); + Assert.Equal(0x1234, restoredObject.SurfaceResId); + Assert.Equal((50, 60), (restoredObject.DstX, restoredObject.DstY)); + Assert.Equal("exit", vm.HaltReason); + } + finally + { + Directory.Delete(root, recursive: true); + } + } + + private static Script WithPackedId(Script source, uint packedId) + => new() + { + Name = source.Name, + PackedId = packedId, + Header = source.Header, + Instructions = source.Instructions, + IndexByOffset = source.IndexByOffset, + Strings = source.Strings, + BodyDwords = source.BodyDwords, + ReadMessageOffsets = source.ReadMessageOffsets, + ScriptCallOffsets = source.ScriptCallOffsets, + LocalCallOffsets = source.LocalCallOffsets, + }; + + private static Script WithTables( + Script source, + IReadOnlyList? scriptCallOffsets = null, + IReadOnlyList? localCallOffsets = null) + => new() + { + Name = source.Name, + PackedId = source.PackedId, + Header = source.Header, + Instructions = source.Instructions, + IndexByOffset = source.IndexByOffset, + Strings = source.Strings, + BodyDwords = source.BodyDwords, + ReadMessageOffsets = source.ReadMessageOffsets, + ScriptCallOffsets = scriptCallOffsets ?? source.ScriptCallOffsets, + LocalCallOffsets = localCallOffsets ?? source.LocalCallOffsets, + }; + + private static int[] DenseIntBank(int count, params (int Index, int Value)[] values) + { + var result = new int[count]; + foreach (var (index, value) in values) result[index] = value; + return result; + } + + private static byte[] NativeSurfaceRecords(int slot, int resourceId, int colorKey) + { + byte[] result = new byte[NativeNumberedSaveState.SurfaceRecordsSize]; + int at = slot * 20; + BinaryPrimitives.WriteInt32LittleEndian(result.AsSpan(at), resourceId); + BinaryPrimitives.WriteInt32LittleEndian(result.AsSpan(at + 4), unchecked((int)(0xff000000u | (uint)colorKey))); + BinaryPrimitives.WriteInt32LittleEndian(result.AsSpan(at + 8), 1); + return result; + } + + private static byte[] NativeGfxRecord(int sourceSlot) + { + byte[] result = new byte[NativeNumberedSaveState.GfxRecordSize]; + void Write(int offset, int value) + => BinaryPrimitives.WriteInt32LittleEndian(result.AsSpan(offset), value); + Write(0, 1); + Write(4, sourceSlot); + Write(8, 1); + Write(0x0c, 2); + Write(0x10, 31); + Write(0x14, 42); + Write(0x24, 50); + Write(0x28, 60); + Write(0x60, -1); + Write(0x238, 1); + Write(0x23c, 1); + return result; + } + + private static string NewTemporaryDirectory() + { + string path = Path.Combine(Path.GetTempPath(), "age-save-vm-" + Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(path); + return path; + } +} diff --git a/engine/Age.Engine.Tests/TestSupport.cs b/engine/Age.Engine.Tests/TestSupport.cs index 73b079f..ca5a97d 100644 --- a/engine/Age.Engine.Tests/TestSupport.cs +++ b/engine/Age.Engine.Tests/TestSupport.cs @@ -176,8 +176,14 @@ internal class RecordingHost : IHost internal sealed class MapProvider : IScriptProvider { private readonly Dictionary _m; - public MapProvider(Dictionary m) => _m = m; + private readonly Dictionary _byName; + public MapProvider(Dictionary m, Dictionary? byName = null) + { + _m = m; + _byName = byName ?? new(StringComparer.OrdinalIgnoreCase); + } public Script? GetById(long id) => _m.TryGetValue(id, out var s) ? s : null; + public Script? GetByName(string name) => _byName.TryGetValue(name, out var s) ? s : null; } /// A controlled test double: every call-script id resolves to the same script (typically a diff --git a/engine/Age.Engine/Diagnostics/TraceEvent.cs b/engine/Age.Engine/Diagnostics/TraceEvent.cs index 7965303..7832914 100644 --- a/engine/Age.Engine/Diagnostics/TraceEvent.cs +++ b/engine/Age.Engine/Diagnostics/TraceEvent.cs @@ -2,7 +2,7 @@ using Age.Engine.Model; namespace Age.Engine.Diagnostics; public enum TraceEventKind { Step, FrameEnter, FrameExit, CallScript, Stub, Halt } -public enum FrameCause { TopScene, CallScript, RootReload } +public enum FrameCause { TopScene, CallScript, RootReload, SaveRestore } /// An engine diagnostic fact. A readonly struct with a Kind discriminator and a shared /// field set — no per-event heap allocation. Only the fields relevant to a Kind are populated; the diff --git a/engine/Age.Engine/Hosting/IScriptProvider.cs b/engine/Age.Engine/Hosting/IScriptProvider.cs index 1edeaae..ffb58c2 100644 --- a/engine/Age.Engine/Hosting/IScriptProvider.cs +++ b/engine/Age.Engine/Hosting/IScriptProvider.cs @@ -8,6 +8,9 @@ public interface IScriptProvider /// The script for this id, or null if the id maps to no known script. Script? GetById(long id); + /// Resolve a fixed engine callback filename such as CALLBACK_LOAD.BIN. + Script? GetByName(string name) => null; + /// Selectors of currently mounted append catalogs. Opcode 0x143 scans these in /// ascending order and executes record zero from each catalog. IReadOnlyList MountedAppendSelectors => Array.Empty(); diff --git a/engine/Age.Engine/Model/AdvTextHistory.cs b/engine/Age.Engine/Model/AdvTextHistory.cs index 4d7430f..6621357 100644 --- a/engine/Age.Engine/Model/AdvTextHistory.cs +++ b/engine/Age.Engine/Model/AdvTextHistory.cs @@ -71,8 +71,8 @@ public sealed record AdvTextHistoryRenderBatch( AdvTextStyle Style); /// -/// Engine-owned retained ADV backlog. It deliberately has no persistence behavior: native numbered-save -/// restoration belongs to the future unified save architecture, while live HISTORY.BIN reads this model. +/// Engine-owned retained ADV backlog. Native numbered saves serialize this same model; live HISTORY.BIN +/// reads it directly, so restored history is immediately available to the script UI. /// public sealed class AdvTextHistory { @@ -185,6 +185,35 @@ public sealed class AdvTextHistory _navigationAnchorIndex = -1; } + /// Replace retained history from AGE's numbered-save history tail. + public void RestorePersistenceSnapshot( + IReadOnlyList entries, + IReadOnlyList records) + { + ArgumentNullException.ThrowIfNull(entries); + ArgumentNullException.ThrowIfNull(records); + _records.Clear(); + _records.AddRange(records); + _entries.Clear(); + _entries.AddRange(entries); + _pendingGroupStarts.Clear(); + _layouts.Clear(); + foreach (AdvTextHistoryRecord record in records) + { + var layout = GetOrCreateLayout(record.Layout.Slot); + layout.Width = record.Layout.Width; + layout.Height = record.Layout.Height; + layout.OriginX = record.Layout.OriginX; + layout.OriginY = record.Layout.OriginY; + layout.CursorX = record.Layout.CursorX; + layout.CursorY = record.Layout.CursorY; + layout.Right = record.Layout.Right; + layout.Bottom = record.Layout.Bottom; + } + CurrentLayoutSlot = entries.Count > 0 ? entries[^1].LayoutSlot : 0; + _navigationAnchorIndex = entries.Count - 1; + } + /// /// Resolve a logical entry relative to AGE's latest-boundary navigation anchor. Repeated calls do not /// mutate the anchor; HISTORY.BIN supplies cumulative deltas while counting and paging backward. diff --git a/engine/Age.Engine/Model/GfxState.cs b/engine/Age.Engine/Model/GfxState.cs index 3b8f169..ca2e8b2 100644 --- a/engine/Age.Engine/Model/GfxState.cs +++ b/engine/Age.Engine/Model/GfxState.cs @@ -56,6 +56,13 @@ public sealed record GfxDiagnosticSnapshot( BlockingGfxObjectDiagnostic? BlockingRangeTransform, IReadOnlyList BlockingObjects); +public sealed record GfxPersistenceSnapshot( + IReadOnlyList<(int Slot, long ResourceId, long ColorKey, bool Created)> Surfaces, + IReadOnlyList<(long Handle, GfxState.GfxObject Object)> Objects, + long RangeFirst, + long RangeCount, + GfxState.GfxObject RangeTransform); + /// A renderable view of one visible gfx object — the host composites these in ascending-handle order /// (= the engine's z-order) each frame. Built by ; the surface /// resId/colorkey are resolved from the object's live source slot at snapshot time (see docs/engine-re.md, @@ -293,31 +300,7 @@ public sealed class GfxState { if (!_objects.TryGetValue(sourceHandle, out var s)) return false; bool destinationIsNew = !_objects.ContainsKey(destinationHandle); - _objects[destinationHandle] = new GfxObject - { - V18 = s.V18, V24 = s.V24, V16c = s.V16c, - Field64 = s.Field64, Field68 = s.Field68, Field6c = s.Field6c, - Color = s.Color, HasColor = s.HasColor, StaticColorMode = s.StaticColorMode, - OneShotColorTarget = s.OneShotColorTarget, ColorDelayMs = s.ColorDelayMs, - ColorDurationMs = s.ColorDurationMs, OneShotColorEnabled = s.OneShotColorEnabled, - OneShotColorBlend = s.OneShotColorBlend, - SrcFrameCount = s.SrcFrameCount, SrcColumns = s.SrcColumns, SrcCell = s.SrcCell, - SrcPeriod = s.SrcPeriod, SrcStart = s.SrcStart, SrcAnim = s.SrcAnim, - ColorPeriod = s.ColorPeriod, ColorStart = s.ColorStart, ColorTarget = s.ColorTarget, - ColorAnim = s.ColorAnim, SourceSlot = s.SourceSlot, SrcRect = s.SrcRect, Visible = s.Visible, - ScaleCurrent = s.ScaleCurrent, ScaleTarget = s.ScaleTarget, - ScaleDelayMs = s.ScaleDelayMs, ScaleDurationMs = s.ScaleDurationMs, ScaleEnabled = s.ScaleEnabled, - TranslationCurrent = s.TranslationCurrent, TranslationTarget = s.TranslationTarget, - TranslationDelayMs = s.TranslationDelayMs, TranslationDurationMs = s.TranslationDurationMs, - TranslationEnabled = s.TranslationEnabled, - RotationCurrent = s.RotationCurrent, RotationTarget = s.RotationTarget, - RotationDelayMs = s.RotationDelayMs, RotationDurationMs = s.RotationDurationMs, - RotationChannelEnabled = s.RotationChannelEnabled, - OneShotAnimationControlFlags = s.OneShotAnimationControlFlags, - OneShotStartMs = s.OneShotStartMs, - RotationPeriodMs = s.RotationPeriodMs, RotationAxis = s.RotationAxis, - RotationEnabled = s.RotationEnabled, RotationStartMs = s.RotationStartMs, - }; + _objects[destinationHandle] = CloneState(s); if (destinationIsNew) InsertOrderedHandle(destinationHandle); CurrentObject = destinationHandle; MarkRetainedMutation(); @@ -418,6 +401,79 @@ public sealed class GfxState } } + public GfxPersistenceSnapshot CapturePersistenceSnapshot() + { + lock (_lock) + { + var surfaces = _surfaces + .Select(pair => ( + pair.Key, pair.Value.ResId, pair.Value.ColorKey, + _createdSurfaces.Contains(pair.Key))) + .OrderBy(item => item.Key) + .ToArray(); + var objects = _orderedObjectHandles + .Select(handle => (handle, CloneState(_objects[handle]))) + .ToArray(); + return new GfxPersistenceSnapshot( + surfaces, objects, _rangeTransformFirst, _rangeTransformCount, + CloneState(_rangeTransform)); + } + } + + public void RestorePersistenceSnapshot(GfxPersistenceSnapshot snapshot) + { + ArgumentNullException.ThrowIfNull(snapshot); + lock (_lock) + { + _surfaces.Clear(); + _createdSurfaces.Clear(); + foreach (var (slot, resourceId, colorKey, created) in snapshot.Surfaces) + { + _surfaces[slot] = (resourceId, colorKey); + if (created) _createdSurfaces.Add(slot); + } + _objects.Clear(); + _orderedObjectHandles.Clear(); + foreach (var (handle, state) in snapshot.Objects) + { + _objects[handle] = CloneState(state); + _orderedObjectHandles.Add(handle); + } + _orderedObjectHandles.Sort(); + _rangeTransformFirst = snapshot.RangeFirst; + _rangeTransformCount = snapshot.RangeCount; + _rangeTransform = CloneState(snapshot.RangeTransform); + MarkRetainedMutation(); + } + } + + private static GfxObject CloneState(GfxObject s) + => new() + { + V18 = s.V18, V24 = s.V24, V16c = s.V16c, + Field64 = s.Field64, Field68 = s.Field68, Field6c = s.Field6c, + Color = s.Color, HasColor = s.HasColor, StaticColorMode = s.StaticColorMode, + OneShotColorTarget = s.OneShotColorTarget, ColorDelayMs = s.ColorDelayMs, + ColorDurationMs = s.ColorDurationMs, OneShotColorEnabled = s.OneShotColorEnabled, + OneShotColorBlend = s.OneShotColorBlend, + SrcFrameCount = s.SrcFrameCount, SrcColumns = s.SrcColumns, SrcCell = s.SrcCell, + SrcPeriod = s.SrcPeriod, SrcStart = s.SrcStart, SrcAnim = s.SrcAnim, + ColorPeriod = s.ColorPeriod, ColorStart = s.ColorStart, ColorTarget = s.ColorTarget, + ColorAnim = s.ColorAnim, SourceSlot = s.SourceSlot, SrcRect = s.SrcRect, Visible = s.Visible, + ScaleCurrent = s.ScaleCurrent, ScaleTarget = s.ScaleTarget, + ScaleDelayMs = s.ScaleDelayMs, ScaleDurationMs = s.ScaleDurationMs, ScaleEnabled = s.ScaleEnabled, + TranslationCurrent = s.TranslationCurrent, TranslationTarget = s.TranslationTarget, + TranslationDelayMs = s.TranslationDelayMs, TranslationDurationMs = s.TranslationDurationMs, + TranslationEnabled = s.TranslationEnabled, + RotationCurrent = s.RotationCurrent, RotationTarget = s.RotationTarget, + RotationDelayMs = s.RotationDelayMs, RotationDurationMs = s.RotationDurationMs, + RotationChannelEnabled = s.RotationChannelEnabled, + OneShotAnimationControlFlags = s.OneShotAnimationControlFlags, + OneShotStartMs = s.OneShotStartMs, + RotationPeriodMs = s.RotationPeriodMs, RotationAxis = s.RotationAxis, + RotationEnabled = s.RotationEnabled, RotationStartMs = s.RotationStartMs, + }; + private readonly object _lock = new(); // ---- surfaces (image buffers per slot): ctx+0x52bd4[slot], from create/set-texture ---- diff --git a/engine/Age.Engine/Model/Script.cs b/engine/Age.Engine/Model/Script.cs index 13ecead..474a4c8 100644 --- a/engine/Age.Engine/Model/Script.cs +++ b/engine/Age.Engine/Model/Script.cs @@ -12,5 +12,9 @@ public sealed class Script public IReadOnlyList BodyDwords { get; init; } = Array.Empty(); /// T1 entries: code DWORD offsets of op-0x71 message boundaries. public IReadOnlyList ReadMessageOffsets { get; init; } = Array.Empty(); + /// T2 entries: code DWORD offsets of resumable call-script sites. + public IReadOnlyList ScriptCallOffsets { get; init; } = Array.Empty(); + /// T3 entries: code DWORD offsets used to reconstruct the intra-script call stack. + public IReadOnlyList LocalCallOffsets { get; init; } = Array.Empty(); public string GetString(int offset) => Strings.TryGetValue(offset, out var s) ? s : ""; } diff --git a/engine/Age.Engine/Persistence/NativeDatStore.cs b/engine/Age.Engine/Persistence/NativeDatStore.cs index 321b7ed..28e23fe 100644 --- a/engine/Age.Engine/Persistence/NativeDatStore.cs +++ b/engine/Age.Engine/Persistence/NativeDatStore.cs @@ -43,6 +43,8 @@ public sealed record NativeSaveIdentity( } } +public sealed record NativeNumberedSaveFile(NativeSaveDocument Document, byte[] HistoryTail); + public interface INativeDatStore { NativeSaveIdentity Identity { get; } @@ -53,6 +55,15 @@ public interface INativeDatStore NativeSaveMetadata? QueryNumberedMetadata(int slot); NativeSaveDocument? LoadNumbered(int slot); void SaveNumbered(int slot, ReadOnlySpan payload, NativeSystemTime timestamp, uint accumulatedPlaySeconds); + NativeNumberedSaveFile? LoadNumberedFile(int slot) + { + NativeSaveDocument? document = LoadNumbered(slot); + return document == null ? null : new NativeNumberedSaveFile(document, []); + } + void SaveNumberedFile( + int slot, ReadOnlySpan payload, ReadOnlySpan historyTail, + NativeSystemTime timestamp, uint accumulatedPlaySeconds) + => SaveNumbered(slot, payload, timestamp, accumulatedPlaySeconds); int DeleteNumberedPair(int slot); int CopyNumberedPair(int sourceSlot, int destinationSlot); byte[]? LoadNumberedThumbnail(int slot); @@ -160,6 +171,17 @@ public sealed class DirectoryNativeDatStore : INativeDatStore return File.Exists(path) ? LoadAndValidate(path, numbered: true) : null; } + public NativeNumberedSaveFile? LoadNumberedFile(int slot) + { + string path = Path.Combine(_root, NumberedFileName(slot)); + if (!File.Exists(path)) return null; + byte[] source = File.ReadAllBytes(path); + NativeSaveDocument document = NativeSaveContainerCodec.Decode(source); + _identity.Validate(document.Metadata, numbered: true); + return new NativeNumberedSaveFile( + document, source.AsSpan(document.BytesConsumed).ToArray()); + } + public NativeSaveMetadata? QueryNumberedMetadata(int slot) { string path = Path.Combine(_root, NumberedFileName(slot)); @@ -186,6 +208,22 @@ public sealed class DirectoryNativeDatStore : INativeDatStore WriteThrough(Path.Combine(_root, NumberedFileName(slot)), encoded); } + public void SaveNumberedFile( + int slot, + ReadOnlySpan payload, + ReadOnlySpan historyTail, + NativeSystemTime timestamp, + uint accumulatedPlaySeconds) + { + byte[] container = NativeSaveContainerCodec.Encode( + payload, _identity.CreateMetadata(timestamp, accumulatedPlaySeconds, numbered: true)); + byte[] encoded = new byte[checked(container.Length + historyTail.Length)]; + container.CopyTo(encoded, 0); + historyTail.CopyTo(encoded.AsSpan(container.Length)); + Directory.CreateDirectory(_root); + WriteThrough(Path.Combine(_root, NumberedFileName(slot)), encoded); + } + public int DeleteNumberedPair(int slot) { bool dataDeleted = TryDelete(Path.Combine(_root, NumberedFileName(slot))); diff --git a/engine/Age.Engine/Persistence/NativeGfxPersistenceCodec.cs b/engine/Age.Engine/Persistence/NativeGfxPersistenceCodec.cs new file mode 100644 index 0000000..0c3c6e8 --- /dev/null +++ b/engine/Age.Engine/Persistence/NativeGfxPersistenceCodec.cs @@ -0,0 +1,200 @@ +using System.Buffers.Binary; +using Age.Engine.Model; + +namespace Age.Engine.Persistence; + +internal static class NativeGfxPersistenceCodec +{ + private const int SurfaceCount = 1000; + private const int SurfaceRecordSize = 20; + + public static ( + byte[] SurfaceRecords, + IReadOnlyList Objects, + long RangeFirst, + int RangeCount, + byte[] RangeRecord) Capture(GfxState gfx) + { + GfxPersistenceSnapshot snapshot = gfx.CapturePersistenceSnapshot(); + byte[] surfaces = new byte[NativeNumberedSaveState.SurfaceRecordsSize]; + foreach (var (slot, resourceId, colorKey, created) in snapshot.Surfaces) + { + if ((uint)slot >= SurfaceCount || created || resourceId < 0) continue; + int at = slot * SurfaceRecordSize; + WriteInt(surfaces, at, unchecked((int)resourceId)); + WriteInt(surfaces, at + 4, PackNativeColorKey(colorKey)); + WriteInt(surfaces, at + 8, 1); + } + NativeSavedGfxObject[] objects = snapshot.Objects + .Select(item => new NativeSavedGfxObject(item.Handle, EncodeObject(item.Object))) + .ToArray(); + return ( + surfaces, objects, snapshot.RangeFirst, unchecked((int)snapshot.RangeCount), + EncodeObject(snapshot.RangeTransform)); + } + + public static GfxPersistenceSnapshot Decode(NativeNumberedSaveState state) + { + var surfaces = new List<(int Slot, long ResourceId, long ColorKey, bool Created)>(); + for (int slot = 0; slot < SurfaceCount; slot++) + { + int at = slot * SurfaceRecordSize; + int resourceId = ReadInt(state.SurfaceRecords, at); + int present = ReadInt(state.SurfaceRecords, at + 8); + if (present == 1 && resourceId >= 0) + surfaces.Add((slot, unchecked((uint)resourceId), + UnpackNativeColorKey(ReadInt(state.SurfaceRecords, at + 4)), false)); + } + var objects = state.GfxObjects + .Select(item => (item.Handle, DecodeObject(item.Record))) + .ToArray(); + return new GfxPersistenceSnapshot( + surfaces, objects, state.RangeTransformFirst, state.RangeTransformCount, + DecodeObject(state.RangeTransformRecord)); + } + + private static byte[] EncodeObject(GfxState.GfxObject value) + { + byte[] raw = new byte[NativeNumberedSaveState.GfxRecordSize]; + int flags = value.Visible ? 1 : 0; + if (value.RotationEnabled || value.SrcAnim || value.ColorAnim) flags |= 4; + WriteInt(raw, 0, flags); + WriteInt(raw, 4, value.SourceSlot); + WriteInt(raw, 8, value.SrcRect.X); + WriteInt(raw, 0x0c, value.SrcRect.Y); + WriteInt(raw, 0x10, checked(value.SrcRect.X + value.SrcRect.W)); + WriteInt(raw, 0x14, checked(value.SrcRect.Y + value.SrcRect.H)); + WriteVector(raw, 0x18, value.V18); + WriteVector(raw, 0x24, value.V24); + WriteInt(raw, 0x30, unchecked((int)value.StaticColorMode)); + WriteInt(raw, 0x34, unchecked((int)value.OneShotStartMs)); + WriteInt(raw, 0x38, unchecked((int)value.ColorDelayMs)); + WriteInt(raw, 0x3c, unchecked((int)value.ScaleDelayMs)); + WriteInt(raw, 0x40, unchecked((int)value.RotationDelayMs)); + WriteInt(raw, 0x44, unchecked((int)value.TranslationDelayMs)); + WriteInt(raw, 0x4c, unchecked((int)value.ColorDurationMs)); + WriteInt(raw, 0x50, unchecked((int)value.ScaleDurationMs)); + WriteInt(raw, 0x54, unchecked((int)value.RotationDurationMs)); + WriteInt(raw, 0x58, unchecked((int)value.TranslationDurationMs)); + WriteInt(raw, 0x60, unchecked((int)value.Color)); + WriteInt(raw, 0x64, unchecked((int)value.OneShotColorTarget)); + WriteScaleMatrix(raw, 0x6c, value.ScaleCurrent); + WriteScaleMatrix(raw, 0xac, value.ScaleTarget); + WriteFloat(raw, 0x16c, value.TranslationCurrent.X); + WriteFloat(raw, 0x170, value.TranslationCurrent.Y); + WriteFloat(raw, 0x174, value.TranslationCurrent.Z); + WriteFloat(raw, 0x1ac, value.TranslationTarget.X); + WriteFloat(raw, 0x1b0, value.TranslationTarget.Y); + WriteFloat(raw, 0x1b4, value.TranslationTarget.Z); + WriteInt(raw, 0x20c, unchecked((int)value.ColorStart)); + WriteInt(raw, 0x214, unchecked((int)value.RotationStartMs)); + WriteInt(raw, 0x220, unchecked((int)value.ColorPeriod)); + WriteInt(raw, 0x228, unchecked((int)value.RotationPeriodMs)); + WriteInt(raw, 0x230, unchecked((int)value.SrcPeriod)); + WriteInt(raw, 0x234, unchecked((int)value.SrcCell)); + WriteInt(raw, 0x238, unchecked((int)value.SrcFrameCount)); + WriteInt(raw, 0x23c, unchecked((int)value.SrcColumns)); + WriteInt(raw, 0x240, unchecked((int)value.ColorTarget)); + WriteVector(raw, 0x244, value.RotationAxis); + WriteInt(raw, 0x2d0, unchecked((int)value.OneShotAnimationControlFlags)); + return raw; + } + + private static GfxState.GfxObject DecodeObject(ReadOnlySpan raw) + { + if (raw.Length != NativeNumberedSaveState.GfxRecordSize) + throw new InvalidDataException("Native retained-gfx record has the wrong size."); + int left = ReadInt(raw, 8), top = ReadInt(raw, 0x0c); + int right = ReadInt(raw, 0x10), bottom = ReadInt(raw, 0x14); + int flags = ReadInt(raw, 0); + return new GfxState.GfxObject + { + Visible = (flags & 1) != 0, + SourceSlot = ReadInt(raw, 4), + SrcRect = (left, top, right - left, bottom - top), + V18 = ReadLongVector(raw, 0x18), + V24 = ReadLongVector(raw, 0x24), + StaticColorMode = ReadInt(raw, 0x30), + OneShotStartMs = ReadInt(raw, 0x34), + ColorDelayMs = ReadInt(raw, 0x38), + ScaleDelayMs = ReadInt(raw, 0x3c), + RotationDelayMs = ReadInt(raw, 0x40), + TranslationDelayMs = ReadInt(raw, 0x44), + ColorDurationMs = ReadInt(raw, 0x4c), + ScaleDurationMs = ReadInt(raw, 0x50), + RotationDurationMs = ReadInt(raw, 0x54), + TranslationDurationMs = ReadInt(raw, 0x58), + Color = unchecked((uint)ReadInt(raw, 0x60)), + HasColor = ReadInt(raw, 0x60) != -1, + OneShotColorTarget = unchecked((uint)ReadInt(raw, 0x64)), + ScaleCurrent = ReadScale(raw, 0x6c), + ScaleTarget = ReadScale(raw, 0xac), + TranslationCurrent = ReadDoubleVector(raw, 0x16c), + V16c = ReadLongFloatVector(raw, 0x16c), + TranslationTarget = ReadDoubleVector(raw, 0x1ac), + ColorStart = ReadInt(raw, 0x20c), + RotationStartMs = ReadInt(raw, 0x214), + ColorPeriod = ReadInt(raw, 0x220), + RotationPeriodMs = ReadInt(raw, 0x228), + SrcPeriod = ReadInt(raw, 0x230), + SrcCell = ReadInt(raw, 0x234), + SrcFrameCount = Math.Max(1, ReadInt(raw, 0x238)), + SrcColumns = Math.Max(1, ReadInt(raw, 0x23c)), + ColorTarget = unchecked((uint)ReadInt(raw, 0x240)), + RotationAxis = ReadLongVector(raw, 0x244), + OneShotAnimationControlFlags = unchecked((uint)ReadInt(raw, 0x2d0)), + RotationEnabled = (flags & 4) != 0 && ReadInt(raw, 0x228) > 0, + SrcAnim = (flags & 4) != 0 && ReadInt(raw, 0x238) > 1, + ColorAnim = (flags & 4) != 0 && ReadInt(raw, 0x220) > 0, + OneShotColorEnabled = ReadInt(raw, 0x4c) > 0, + ScaleEnabled = ReadInt(raw, 0x50) > 0, + RotationChannelEnabled = ReadInt(raw, 0x54) > 0, + TranslationEnabled = ReadInt(raw, 0x58) > 0, + }; + } + + private static int PackNativeColorKey(long colorKey) + => colorKey < 0 ? 0 : unchecked((int)(0xff000000 | (uint)colorKey & 0xffffff)); + + private static long UnpackNativeColorKey(int colorKey) + => colorKey == 0 ? -1 : unchecked((uint)colorKey) & 0xffffff; + + private static void WriteScaleMatrix(Span raw, int offset, (double X, double Y, double Z) scale) + { + WriteFloat(raw, offset, scale.X); + WriteFloat(raw, offset + 0x14, scale.Y); + WriteFloat(raw, offset + 0x28, scale.Z); + WriteFloat(raw, offset + 0x3c, 1); + } + + private static (double X, double Y, double Z) ReadScale(ReadOnlySpan raw, int offset) + => (ReadFloat(raw, offset), ReadFloat(raw, offset + 0x14), ReadFloat(raw, offset + 0x28)); + + private static void WriteVector(Span raw, int offset, (long X, long Y, long Z) vector) + { + WriteInt(raw, offset, unchecked((int)vector.X)); + WriteInt(raw, offset + 4, unchecked((int)vector.Y)); + WriteInt(raw, offset + 8, unchecked((int)vector.Z)); + } + + private static (long X, long Y, long Z) ReadLongVector(ReadOnlySpan raw, int offset) + => (ReadInt(raw, offset), ReadInt(raw, offset + 4), ReadInt(raw, offset + 8)); + + private static (long X, long Y, long Z) ReadLongFloatVector(ReadOnlySpan raw, int offset) + => ((long)ReadFloat(raw, offset), (long)ReadFloat(raw, offset + 4), (long)ReadFloat(raw, offset + 8)); + + private static (double X, double Y, double Z) ReadDoubleVector(ReadOnlySpan raw, int offset) + => (ReadFloat(raw, offset), ReadFloat(raw, offset + 4), ReadFloat(raw, offset + 8)); + + private static void WriteFloat(Span raw, int offset, double value) + => WriteInt(raw, offset, BitConverter.SingleToInt32Bits((float)value)); + + private static float ReadFloat(ReadOnlySpan raw, int offset) + => BitConverter.Int32BitsToSingle(ReadInt(raw, offset)); + + private static int ReadInt(ReadOnlySpan raw, int offset) + => BinaryPrimitives.ReadInt32LittleEndian(raw.Slice(offset, 4)); + + private static void WriteInt(Span raw, int offset, int value) + => BinaryPrimitives.WriteInt32LittleEndian(raw.Slice(offset, 4), value); +} diff --git a/engine/Age.Engine/Persistence/NativeNumberedSaveCodec.cs b/engine/Age.Engine/Persistence/NativeNumberedSaveCodec.cs new file mode 100644 index 0000000..4f0ea2a --- /dev/null +++ b/engine/Age.Engine/Persistence/NativeNumberedSaveCodec.cs @@ -0,0 +1,294 @@ +using System.Buffers.Binary; +using System.Text; + +namespace Age.Engine.Persistence; + +public sealed record NativeSavedScriptFrame( + int ParentContext, + uint ScriptId, + IReadOnlyList ReturnIndices, + int ResumeIndex, + int CallTargetIndex); + +public sealed record NativeSavedGfxObject(long Handle, byte[] Record); + +public sealed record NativeNumberedSaveState( + int SavedFrameOwner, + int EngineState, + IReadOnlyList StateWords, + byte[] ResourceRecords, + byte[] SurfaceRecords, + IReadOnlyList Frames, + IReadOnlyList IntegerGlobals, + IReadOnlyList FloatGlobals, + IReadOnlyList StringGlobals, + IReadOnlyList PointerGlobals, + IReadOnlyList PointerStrings, + IReadOnlyList LocalPointerScratch, + IReadOnlyList GfxObjects, + long RangeTransformFirst, + int RangeTransformCount, + byte[] RangeTransformRecord) +{ + public const int StateWordCount = 10; + public const int ResourceRecordsSize = 300 * 4; + public const int SurfaceRecordsSize = 20_000; + public const int GfxRecordSize = 0x2d4; +} + +/// AGE SaveVersion1=3 numbered-save logical payload. +public static class NativeNumberedSaveCodec +{ + private const int FixedPrefixSize = 0x5304; + private const int FrameSize = 0x414; + private const int FixedSuffixSize = 0x414; + private const int FrameReturnCapacity = 256; + private const int GfxAllocationDwords = 0x2d8; + private const int GfxConstantDwords = 0x2e1; + private static readonly Encoding NativeEncoding = CreateNativeEncoding(); + + public static byte[] Encode(NativeNumberedSaveState state) + { + ArgumentNullException.ThrowIfNull(state); + ValidateState(state); + + int cutoff = state.Frames.Count - 1; + byte[] strings = EncodeStrings(state.StringGlobals); + int stringDwords = strings.Length / 4; + int bankDwords = checked( + state.IntegerGlobals.Count + state.FloatGlobals.Count + state.StringGlobals.Count + + state.PointerGlobals.Count + state.PointerStrings.Count + state.LocalPointerScratch.Count); + int totalDwords = checked( + cutoff * 0x105 + 0x53ea + bankDwords + stringDwords + + GfxConstantDwords + state.GfxObjects.Count * GfxAllocationDwords); + byte[] payload = new byte[checked((totalDwords - 2) * 4)]; + + WriteInt(payload, 0, cutoff); + WriteInt(payload, 4, state.SavedFrameOwner); + WriteInt(payload, 8, state.EngineState); + WriteIntList(payload, 0x0c, state.StateWords); + state.ResourceRecords.CopyTo(payload, 0x34); + state.SurfaceRecords.CopyTo(payload, 0x4e4); + + for (int i = 0; i < state.Frames.Count; i++) + WriteFrame(payload, FixedPrefixSize + i * FrameSize, state.Frames[i], i == cutoff); + + int at = checked(0x5718 + cutoff * FrameSize); + int[] counts = + [ + state.IntegerGlobals.Count, state.FloatGlobals.Count, state.StringGlobals.Count, + state.PointerGlobals.Count, state.PointerStrings.Count, state.LocalPointerScratch.Count, + ]; + WriteIntList(payload, at, counts); + at += 24; + WriteIntList(payload, at, state.IntegerGlobals); + at += state.IntegerGlobals.Count * 4; + WriteIntList(payload, at, state.FloatGlobals); + at += state.FloatGlobals.Count * 4; + WriteInt(payload, at, stringDwords); + at += 4; + strings.CopyTo(payload, at); + at += strings.Length; + WriteIntList(payload, at, state.PointerGlobals); + at += state.PointerGlobals.Count * 4; + WriteIntList(payload, at, state.PointerStrings); + at += state.PointerStrings.Count * 4; + WriteIntList(payload, at, state.LocalPointerScratch); + at += state.LocalPointerScratch.Count * 4; + + WriteInt(payload, at, NativeNumberedSaveState.GfxRecordSize); + WriteInt(payload, at + 4, state.GfxObjects.Count); + at += 8; + foreach (NativeSavedGfxObject gfx in state.GfxObjects) + { + WriteInt(payload, at, unchecked((int)gfx.Handle)); + gfx.Record.CopyTo(payload, at + 4); + at += 4 + NativeNumberedSaveState.GfxRecordSize; + } + WriteInt(payload, at, unchecked((int)state.RangeTransformFirst)); + WriteInt(payload, at + 4, state.RangeTransformCount); + state.RangeTransformRecord.CopyTo(payload, at + 8); + return payload; + } + + public static NativeNumberedSaveState Decode(ReadOnlySpan payload) + { + if (payload.Length < 0x5718) + throw new InvalidDataException("Numbered-save layout 3 payload is truncated."); + int cutoff = ReadNonNegative(payload, 0, "saved frame cutoff"); + int fixedBytes = checked(0x5718 + cutoff * FrameSize); + Require(payload, 0, fixedBytes, "numbered-save fixed state"); + + int[] stateWords = ReadInts(payload, 0x0c, NativeNumberedSaveState.StateWordCount); + byte[] resources = payload.Slice(0x34, NativeNumberedSaveState.ResourceRecordsSize).ToArray(); + byte[] surfaces = payload.Slice(0x4e4, NativeNumberedSaveState.SurfaceRecordsSize).ToArray(); + var frames = new NativeSavedScriptFrame[cutoff + 1]; + for (int i = 0; i < frames.Length; i++) + frames[i] = ReadFrame(payload, FixedPrefixSize + i * FrameSize); + + int at = fixedBytes; + int[] counts = ReadInts(payload, at, 6); + if (counts.Any(count => count < 0)) + throw new InvalidDataException("Numbered-save layout 3 contains a negative bank count."); + at += 24; + int[] integers = ReadInts(payload, at, counts[0]); + at = checked(at + counts[0] * 4); + int[] floats = ReadInts(payload, at, counts[1]); + at = checked(at + counts[1] * 4); + int stringDwords = ReadNonNegative(payload, at, "string blob length"); + at += 4; + int stringBytes = checked(stringDwords * 4); + Require(payload, at, stringBytes, "numbered-save string blob"); + string[] strings = DecodeStrings(payload.Slice(at, stringBytes), counts[2]); + at += stringBytes; + int[] pointers = ReadInts(payload, at, counts[3]); + at = checked(at + counts[3] * 4); + int[] pointerStrings = ReadInts(payload, at, counts[4]); + at = checked(at + counts[4] * 4); + int[] localPointerScratch = ReadInts(payload, at, counts[5]); + at = checked(at + counts[5] * 4); + + int gfxRecordSize = ReadNonNegative(payload, at, "gfx record size"); + int gfxCount = ReadNonNegative(payload, at + 4, "gfx object count"); + if (gfxRecordSize != NativeNumberedSaveState.GfxRecordSize) + throw new InvalidDataException($"Unsupported native gfx record size 0x{gfxRecordSize:x}."); + at += 8; + var objects = new NativeSavedGfxObject[gfxCount]; + for (int i = 0; i < objects.Length; i++) + { + Require(payload, at, 4 + gfxRecordSize, "numbered-save gfx object"); + long handle = ReadInt(payload, at); + objects[i] = new NativeSavedGfxObject(handle, payload.Slice(at + 4, gfxRecordSize).ToArray()); + at += 4 + gfxRecordSize; + } + Require(payload, at, 8 + gfxRecordSize, "numbered-save range transform"); + long rangeFirst = ReadInt(payload, at); + int rangeCount = ReadInt(payload, at + 4); + byte[] rangeRecord = payload.Slice(at + 8, gfxRecordSize).ToArray(); + + return new NativeNumberedSaveState( + ReadInt(payload, 4), ReadInt(payload, 8), stateWords, resources, surfaces, frames, + integers, floats, strings, pointers, pointerStrings, localPointerScratch, objects, + rangeFirst, rangeCount, rangeRecord); + } + + public static NativeNumberedSaveState Empty(IReadOnlyList frames) + => new( + 0, 0, new int[NativeNumberedSaveState.StateWordCount], + new byte[NativeNumberedSaveState.ResourceRecordsSize], + new byte[NativeNumberedSaveState.SurfaceRecordsSize], + frames, Array.Empty(), Array.Empty(), Array.Empty(), + Array.Empty(), Array.Empty(), Array.Empty(), + Array.Empty(), 0, 0, + new byte[NativeNumberedSaveState.GfxRecordSize]); + + private static void WriteFrame(Span payload, int offset, NativeSavedScriptFrame frame, bool terminal) + { + if (frame.ReturnIndices.Count > FrameReturnCapacity) + throw new InvalidDataException("A native saved frame cannot hold more than 256 return entries."); + WriteInt(payload, offset, frame.ParentContext); + WriteInt(payload, offset + 4, unchecked((int)frame.ScriptId)); + WriteInt(payload, offset + 8, frame.ReturnIndices.Count); + for (int i = 0; i < frame.ReturnIndices.Count; i++) + WriteInt(payload, offset + 12 + i * 4, frame.ReturnIndices[i]); + WriteInt(payload, offset + 0x40c, frame.ResumeIndex); + WriteInt(payload, offset + 0x410, terminal ? -1 : frame.CallTargetIndex); + } + + private static NativeSavedScriptFrame ReadFrame(ReadOnlySpan payload, int offset) + { + int count = ReadNonNegative(payload, offset + 8, "frame return count"); + if (count > FrameReturnCapacity) + throw new InvalidDataException("Numbered-save frame return count exceeds 256."); + return new NativeSavedScriptFrame( + ReadInt(payload, offset), unchecked((uint)ReadInt(payload, offset + 4)), + ReadInts(payload, offset + 12, count), + ReadInt(payload, offset + 0x40c), ReadInt(payload, offset + 0x410)); + } + + private static byte[] EncodeStrings(IReadOnlyList strings) + { + using var stream = new MemoryStream(); + foreach (string text in strings) + { + byte[] encoded = NativeEncoding.GetBytes(text ?? ""); + stream.Write(encoded); + stream.WriteByte(0); + } + while ((stream.Length & 3) != 0) stream.WriteByte(0); + return stream.ToArray(); + } + + private static string[] DecodeStrings(ReadOnlySpan blob, int count) + { + var result = new string[count]; + int at = 0; + for (int i = 0; i < count; i++) + { + if (at >= blob.Length) + throw new InvalidDataException("Numbered-save string blob ended before its declared string count."); + int end = blob[at..].IndexOf((byte)0); + if (end < 0) + throw new InvalidDataException("Numbered-save string blob ended before its declared string count."); + result[i] = NativeEncoding.GetString(blob.Slice(at, end)); + at += end + 1; + } + return result; + } + + private static void ValidateState(NativeNumberedSaveState state) + { + if (state.Frames.Count == 0) throw new InvalidDataException("A numbered save requires at least one frame."); + if (state.StateWords.Count != NativeNumberedSaveState.StateWordCount) + throw new InvalidDataException("Numbered-save state word count must be 10."); + if (state.ResourceRecords.Length != NativeNumberedSaveState.ResourceRecordsSize) + throw new InvalidDataException("Numbered-save resource table must be 1,200 bytes."); + if (state.SurfaceRecords.Length != NativeNumberedSaveState.SurfaceRecordsSize) + throw new InvalidDataException("Numbered-save surface table must be 20,000 bytes."); + if (state.RangeTransformRecord.Length != NativeNumberedSaveState.GfxRecordSize + || state.GfxObjects.Any(item => item.Record.Length != NativeNumberedSaveState.GfxRecordSize)) + throw new InvalidDataException("Numbered-save gfx records must be 0x2d4 bytes."); + } + + private static int[] ReadInts(ReadOnlySpan source, int offset, int count) + { + int bytes = checked(count * 4); + Require(source, offset, bytes, "numbered-save integer array"); + var values = new int[count]; + for (int i = 0; i < count; i++) values[i] = ReadInt(source, offset + i * 4); + return values; + } + + private static void WriteIntList(Span destination, int offset, IReadOnlyList values) + { + for (int i = 0; i < values.Count; i++) WriteInt(destination, offset + i * 4, values[i]); + } + + private static int ReadNonNegative(ReadOnlySpan source, int offset, string name) + { + int value = ReadInt(source, offset); + if (value < 0) throw new InvalidDataException($"Numbered-save {name} is negative."); + return value; + } + + private static int ReadInt(ReadOnlySpan source, int offset) + { + Require(source, offset, 4, "numbered-save dword"); + return BinaryPrimitives.ReadInt32LittleEndian(source.Slice(offset, 4)); + } + + private static void WriteInt(Span destination, int offset, int value) + => BinaryPrimitives.WriteInt32LittleEndian(destination.Slice(offset, 4), value); + + private static void Require(ReadOnlySpan source, int offset, int count, string name) + { + if (offset < 0 || count < 0 || offset > source.Length - count) + throw new InvalidDataException($"{name} is truncated."); + } + + private static Encoding CreateNativeEncoding() + { + Encoding.RegisterProvider(CodePagesEncodingProvider.Instance); + return Encoding.GetEncoding(932); + } +} diff --git a/engine/Age.Engine/Persistence/NativeTextHistoryCodec.cs b/engine/Age.Engine/Persistence/NativeTextHistoryCodec.cs new file mode 100644 index 0000000..33c2547 --- /dev/null +++ b/engine/Age.Engine/Persistence/NativeTextHistoryCodec.cs @@ -0,0 +1,179 @@ +using System.Buffers.Binary; +using System.Text; +using Age.Engine.Model; +using Age.Engine.Sys4; + +namespace Age.Engine.Persistence; + +/// The non-container LZSS tail appended to a layout-3 numbered save. +public static class NativeTextHistoryCodec +{ + private const int HeaderSize = 12; + private const int RecordDwords = 11; + private static readonly Encoding NativeEncoding = CreateNativeEncoding(); + + public static byte[] Encode(AdvTextHistory history) + { + ArgumentNullException.ThrowIfNull(history); + byte[] logical = EncodeLogical(history); + byte[] stored = LzssEncoder.EncodeOrVerbatim(logical); + byte[] result = new byte[HeaderSize + stored.Length]; + WriteInt(result, 0, logical.Length); + WriteInt(result, 4, logical.Length); + WriteInt(result, 8, stored.Length); + stored.CopyTo(result, HeaderSize); + return result; + } + + public static void DecodeInto(ReadOnlySpan source, AdvTextHistory history) + { + ArgumentNullException.ThrowIfNull(history); + if (source.Length < HeaderSize) throw new InvalidDataException("Native text-history tail is truncated."); + int logicalLength = ReadNonNegative(source, 0, "logical length"); + int duplicateLength = ReadNonNegative(source, 4, "duplicate logical length"); + int storedLength = ReadNonNegative(source, 8, "stored length"); + if (duplicateLength != logicalLength) + throw new InvalidDataException("Native text-history length fields disagree."); + if (source.Length < HeaderSize + storedLength) + throw new InvalidDataException("Native text-history compressed stream is truncated."); + ReadOnlySpan stored = source.Slice(HeaderSize, storedLength); + byte[] logical = storedLength == logicalLength + ? stored.ToArray() + : LzssDecoder.Decode(stored, logicalLength, "native text history"); + DecodeLogical(logical, history); + } + + private static byte[] EncodeLogical(AdvTextHistory history) + { + using var stream = new MemoryStream(); + using var writer = new BinaryWriter(stream, Encoding.UTF8, leaveOpen: true); + writer.Write(history.Entries.Count); + foreach (AdvTextHistoryEntry entry in history.Entries) + { + writer.Write(entry.LayoutSlot); + writer.Write(entry.FirstRecordIndex); + } + writer.Write(history.Records.Count); + foreach (AdvTextHistoryRecord record in history.Records) + { + writer.Write(record.Layout.Slot); + writer.Write(record.Layout.OriginX); + writer.Write(record.Layout.OriginY); + writer.Write(record.Layout.Width); + writer.Write(record.Layout.Height); + writer.Write(unchecked((int)record.Value)); + writer.Write(unchecked((int)record.AuxValue)); + writer.Write(record.Style.PrimaryFontSize); + writer.Write(unchecked((int)record.Style.TextColor)); + writer.Write(record.Layout.CursorY); + writer.Write(unchecked((int)record.Flags)); + } + + using var strings = new MemoryStream(); + foreach (AdvTextHistoryRecord record in history.Records) + { + strings.Write(NativeEncoding.GetBytes(record.Text ?? "")); + strings.WriteByte(0); + } + while ((strings.Length & 3) != 0) strings.WriteByte(0); + byte[] blob = strings.ToArray(); + writer.Write(blob.Length / 4); + for (int i = 0; i < blob.Length; i += 4) + writer.Write(~BinaryPrimitives.ReadUInt32LittleEndian(blob.AsSpan(i, 4))); + return stream.ToArray(); + } + + private static void DecodeLogical(ReadOnlySpan source, AdvTextHistory history) + { + int at = 0; + int entryCount = ReadCount(source, ref at, "entry"); + var entries = new AdvTextHistoryEntry[entryCount]; + for (int i = 0; i < entryCount; i++) + entries[i] = new AdvTextHistoryEntry(ReadNext(source, ref at), ReadNext(source, ref at)); + + int recordCount = ReadCount(source, ref at, "record"); + var raw = new int[recordCount, RecordDwords]; + for (int i = 0; i < recordCount; i++) + for (int dword = 0; dword < RecordDwords; dword++) + raw[i, dword] = ReadNext(source, ref at); + + int stringDwords = ReadCount(source, ref at, "string blob dword"); + int stringBytes = checked(stringDwords * 4); + if (at > source.Length - stringBytes) + throw new InvalidDataException("Native text-history string blob is truncated."); + byte[] blob = source.Slice(at, stringBytes).ToArray(); + for (int i = 0; i < blob.Length; i += 4) + { + uint value = ~BinaryPrimitives.ReadUInt32LittleEndian(blob.AsSpan(i, 4)); + BinaryPrimitives.WriteUInt32LittleEndian(blob.AsSpan(i, 4), value); + } + string[] texts = DecodeStrings(blob, recordCount); + var records = new AdvTextHistoryRecord[recordCount]; + for (int i = 0; i < recordCount; i++) + { + var flags = unchecked((AdvTextHistoryRecordFlags)(uint)raw[i, 10]); + AdvTextHistoryRecordKind kind = + flags.HasFlag(AdvTextHistoryRecordFlags.VoicePair) ? AdvTextHistoryRecordKind.Voice : + flags.HasFlag(AdvTextHistoryRecordFlags.TypedMetadata) ? AdvTextHistoryRecordKind.Metadata : + AdvTextHistoryRecordKind.Text; + var layout = new AdvTextLayoutSnapshot( + raw[i, 0], raw[i, 3], raw[i, 4], raw[i, 1], raw[i, 2], + 0, raw[i, 9], raw[i, 3], raw[i, 4]); + var style = AdvTextStyle.Default with + { + PrimaryFontSize = raw[i, 7], + TextColor = unchecked((uint)raw[i, 8]), + }; + records[i] = new AdvTextHistoryRecord( + kind, flags, layout, style, texts[i], raw[i, 5], raw[i, 6], -1); + } + history.RestorePersistenceSnapshot(entries, records); + } + + private static string[] DecodeStrings(ReadOnlySpan blob, int count) + { + var result = new string[count]; + int at = 0; + for (int i = 0; i < count; i++) + { + if (at >= blob.Length) + throw new InvalidDataException("Native text-history string blob ended early."); + int end = blob[at..].IndexOf((byte)0); + if (end < 0) throw new InvalidDataException("Native text-history string blob ended early."); + result[i] = NativeEncoding.GetString(blob.Slice(at, end)); + at += end + 1; + } + return result; + } + + private static int ReadCount(ReadOnlySpan source, ref int at, string name) + { + int value = ReadNext(source, ref at); + if (value < 0) throw new InvalidDataException($"Native text-history {name} count is negative."); + return value; + } + + private static int ReadNext(ReadOnlySpan source, ref int at) + { + if (at > source.Length - 4) throw new InvalidDataException("Native text-history logical data is truncated."); + int value = BinaryPrimitives.ReadInt32LittleEndian(source.Slice(at, 4)); + at += 4; + return value; + } + + private static int ReadNonNegative(ReadOnlySpan source, int offset, string name) + { + int value = BinaryPrimitives.ReadInt32LittleEndian(source.Slice(offset, 4)); + if (value < 0) throw new InvalidDataException($"Native text-history {name} is negative."); + return value; + } + + private static void WriteInt(Span destination, int offset, int value) + => BinaryPrimitives.WriteInt32LittleEndian(destination.Slice(offset, 4), value); + + private static Encoding CreateNativeEncoding() + { + Encoding.RegisterProvider(CodePagesEncodingProvider.Instance); + return Encoding.GetEncoding(932); + } +} diff --git a/engine/Age.Engine/Sys4/Sys4Loader.cs b/engine/Age.Engine/Sys4/Sys4Loader.cs index 1c4962e..0c7cdab 100644 --- a/engine/Age.Engine/Sys4/Sys4Loader.cs +++ b/engine/Age.Engine/Sys4/Sys4Loader.cs @@ -29,6 +29,8 @@ public static class Sys4Loader throw new InvalidDataException($"{name}: invalid T1 read-message table"); int[] messageOffsets = dw.AsSpan(messageTableOffset, messageCount) .ToArray().Select(value => checked((int)value)).ToArray(); + int[] scriptCallOffsets = ReadOffsetTable(dw, fields[9], fields[10], nbody, name, "T2 script-call"); + int[] localCallOffsets = ReadOffsetTable(dw, fields[11], fields[12], nbody, name, "T3 local-call"); return new Script { Name = name, @@ -39,9 +41,20 @@ public static class Sys4Loader Strings = strings, BodyDwords = dw, ReadMessageOffsets = messageOffsets, + ScriptCallOffsets = scriptCallOffsets, + LocalCallOffsets = localCallOffsets, }; } + private static int[] ReadOffsetTable( + uint[] body, int count, int offset, int bodyLength, string name, string tableName) + { + if (count < 0 || offset < 0 || offset > bodyLength || count > bodyLength - offset) + throw new InvalidDataException($"{name}: invalid {tableName} table"); + return body.AsSpan(offset, count).ToArray() + .Select(value => checked((int)value)).ToArray(); + } + private static (List, Dictionary, Dictionary) DecodeCode(uint[] dw, int[] fields, int nbody, OpcodeTable table) { diff --git a/engine/Age.Engine/Vm/GameSession.cs b/engine/Age.Engine/Vm/GameSession.cs index 832fa6c..f2e53f9 100644 --- a/engine/Age.Engine/Vm/GameSession.cs +++ b/engine/Age.Engine/Vm/GameSession.cs @@ -7,8 +7,8 @@ using Age.Engine.Persistence; namespace Age.Engine.Vm; /// -/// A persistent global store carried across scenes. Every scene the game runs shares one flat global -/// bank (the engine's model); running scenes in isolation with empty state is why our headless VM +/// A persistent store carried across scenes. AGE keeps separate integer, float, string, and pointer +/// banks; running scenes in isolation with empty state is why our headless VM /// diverges from the real game (the bg/sprite geometry drift, the state-gated EMPTY scenes, Lily's /// form-gated voices are all state divergence — see docs/phase-a-slice-plan.md A2b-Geometry). /// @@ -20,7 +20,10 @@ namespace Age.Engine.Vm; public sealed class GameSession { public Dictionary Globals { get; } = new(); + public Dictionary GlobalFloats { get; } = new(); public Dictionary GlobalStrings { get; } = new(); + public Dictionary GlobalPointers { get; } = new(); + public Dictionary GlobalStringPointers { get; } = new(); /// AGE's selected profile-wide cells plus native shared SAVE.DAT/RT.DAT lifecycle. public SharedProfile SharedProfile { get; } /// Native shared/numbered save directory service used by persistence opcodes. @@ -45,13 +48,19 @@ public sealed class GameSession var vm = new VirtualMachine( script, table, host, options, provider, sink, TextHistory, SharedProfile, NativeDatStore); foreach (var kv in Globals) vm.Globals[kv.Key] = kv.Value; + foreach (var kv in GlobalFloats) vm.GlobalFloats[kv.Key] = kv.Value; foreach (var kv in GlobalStrings) vm.GlobalStrings[kv.Key] = kv.Value; + foreach (var kv in GlobalPointers) vm.GlobalPointers[kv.Key] = kv.Value; + foreach (var kv in GlobalStringPointers) vm.GlobalStringPointers[kv.Key] = kv.Value; vm.Run(); - // Globals are one flat space; last write wins — the engine's single global bank. + // Each native bank is shared by the session; last write wins within that bank. foreach (var kv in vm.Globals) Globals[kv.Key] = kv.Value; + foreach (var kv in vm.GlobalFloats) GlobalFloats[kv.Key] = kv.Value; foreach (var kv in vm.GlobalStrings) GlobalStrings[kv.Key] = kv.Value; + foreach (var kv in vm.GlobalPointers) GlobalPointers[kv.Key] = kv.Value; + foreach (var kv in vm.GlobalStringPointers) GlobalStringPointers[kv.Key] = kv.Value; return new SceneResult(vm.Emitted.ToList(), vm.HaltReason, vm.Steps); } diff --git a/engine/Age.Engine/Vm/VirtualMachine.cs b/engine/Age.Engine/Vm/VirtualMachine.cs index d53e0d7..54dc0d2 100644 --- a/engine/Age.Engine/Vm/VirtualMachine.cs +++ b/engine/Age.Engine/Vm/VirtualMachine.cs @@ -37,6 +37,11 @@ public sealed class VirtualMachine private readonly List _activeFrameNames = new(); private readonly List _activeExecutionFrames = new(); private ExecFrame? _saveResumeFrame; + private NativeNumberedSaveState? _loadedNumberedState; + private NativeNumberedSaveState? _retainedNativeNumberedState; + private int _restoreFrameIndex = -1; + private uint _accumulatedPlaySeconds; + private readonly long _sessionStartTimestamp; private ExecFrame? _debugActiveFrame; private long _debugActiveFrameId; private long _debugNextFrameId; @@ -68,9 +73,12 @@ public sealed class VirtualMachine public long CallScriptDispatches { get; private set; } public Dictionary Globals { get; } = new(); + public Dictionary GlobalFloats { get; } = new(); /// Native/profile-owned values read by scripts but maintained outside script-visible writes. public Dictionary ExternalGlobals { get; } = new(); public Dictionary GlobalStrings { get; } = new(); + public Dictionary GlobalPointers { get; } = new(); + public Dictionary GlobalStringPointers { get; } = new(); public GfxState Gfx { get; } = new(); public InputBindings InputBindings { get; } = new(); public List<(int Offset, string Text, string Script)> Emitted { get; } = new(); @@ -124,6 +132,7 @@ public sealed class VirtualMachine _sink = sink ?? NullTraceSink.Instance; TextHistory = textHistory ?? new AdvTextHistory(); _sharedProfile = sharedProfile ?? new SharedProfile(); _nativeDatStore = nativeDatStore; + _sessionStartTimestamp = System.Diagnostics.Stopwatch.GetTimestamp(); } /// Queue global writes and return only the identified active frame at its next opcode boundary. @@ -291,6 +300,10 @@ public sealed class VirtualMachine } private static long Gi(Dictionary d, int k) => d.TryGetValue(k, out var v) ? v : 0; + private int GlobalPointer(int index) + => GlobalPointers.TryGetValue(index, out int value) ? value : unchecked((int)Gi(Globals, index)); + private int GlobalStringPointer(int index) + => GlobalStringPointers.TryGetValue(index, out int value) ? value : unchecked((int)Gi(Globals, index)); private long ReadGlobal(int k) => ExternalGlobals.TryGetValue(k, out var v) ? v : Gi(Globals, k); private static string Gs(Dictionary d, int k) => d.TryGetValue(k, out var v) ? v : ""; private static long PyDiv(long a, long b) { if (b == 0) return 0; long q = a / b, r = a % b; if (r != 0 && (r < 0) != (b < 0)) q--; return q; } @@ -328,8 +341,9 @@ public sealed class VirtualMachine private long Read(Operand op) => op.Type switch { T_IMM => op.Value, - T_GINT or T_GFLOAT => ReadGlobal((int)op.Value), - T_GPTR => Gi(Globals, (int)Gi(Globals, (int)op.Value)), + T_GINT => ReadGlobal((int)op.Value), + T_GFLOAT => Gi(GlobalFloats, (int)op.Value), + T_GPTR => Gi(Globals, GlobalPointer((int)op.Value)), T_LINT => Gi(_cur.Locals.I, (int)op.Value), T_LFLOAT => Gi(_cur.Locals.F, (int)op.Value), T_LPTR => ReadIntCell(Ga(_cur.Locals.P, (int)op.Value)), @@ -340,8 +354,9 @@ public sealed class VirtualMachine { switch (op.Type) { - case T_GINT: case T_GFLOAT: Globals[(int)op.Value] = val; break; - case T_GPTR: Globals[(int)Gi(Globals, (int)op.Value)] = val; break; + case T_GINT: Globals[(int)op.Value] = val; break; + case T_GFLOAT: GlobalFloats[(int)op.Value] = val; break; + case T_GPTR: Globals[GlobalPointer((int)op.Value)] = val; break; case T_LINT: _cur.Locals.I[(int)op.Value] = val; break; case T_LFLOAT: _cur.Locals.F[(int)op.Value] = val; break; case T_LPTR: WriteIntCell(Ga(_cur.Locals.P, (int)op.Value), val); break; @@ -352,7 +367,7 @@ public sealed class VirtualMachine { T_STR => _cur.Script.GetString((int)op.Value), T_GSTR => Gs(GlobalStrings, (int)op.Value), - T_GSTRPTR => Gs(GlobalStrings, (int)Gi(Globals, (int)op.Value)), + T_GSTRPTR => Gs(GlobalStrings, GlobalStringPointer((int)op.Value)), T_LSTR => Gs(_cur.Locals.S, (int)op.Value), T_LSTRPTR => ReadStringCell(Ga(_cur.Locals.SP, (int)op.Value)), _ => "", @@ -363,7 +378,7 @@ public sealed class VirtualMachine switch (op.Type) { case T_GSTR: GlobalStrings[(int)op.Value] = val; break; - case T_GSTRPTR: GlobalStrings[(int)Gi(Globals, (int)op.Value)] = val; break; + case T_GSTRPTR: GlobalStrings[GlobalStringPointer((int)op.Value)] = val; break; case T_LSTR: _cur.Locals.S[(int)op.Value] = val; break; case T_LSTRPTR: WriteStringCell(Ga(_cur.Locals.SP, (int)op.Value), val); break; } @@ -465,7 +480,8 @@ public sealed class VirtualMachine T_LINT => VmAddress.LocalInteger((int)op.Value), T_LFLOAT => VmAddress.LocalFloat((int)op.Value), T_LSTR => VmAddress.LocalString((int)op.Value), - T_GPTR or T_GSTRPTR => VmAddress.Global((int)Gi(Globals, (int)op.Value)), + T_GPTR => VmAddress.Global(GlobalPointer((int)op.Value)), + T_GSTRPTR => VmAddress.Global(GlobalStringPointer((int)op.Value)), T_LPTR => Ga(_cur.Locals.P, (int)op.Value), T_LSTRPTR => Ga(_cur.Locals.SP, (int)op.Value), _ => VmAddress.Global((int)op.Value), @@ -492,7 +508,8 @@ public sealed class VirtualMachine { case T_LPTR: _cur.Locals.P[(int)destination.Value] = address; return true; case T_LSTRPTR: _cur.Locals.SP[(int)destination.Value] = address; return true; - case T_GPTR: case T_GSTRPTR: Globals[(int)destination.Value] = address.Address; return true; + case T_GPTR: GlobalPointers[(int)destination.Value] = address.Address; return true; + case T_GSTRPTR: GlobalStringPointers[(int)destination.Value] = address.Address; return true; default: return false; } } @@ -514,10 +531,13 @@ public sealed class VirtualMachine int address = checked((int)destination.Value + index); switch (destination.Type) { - case T_GINT: case T_GFLOAT: Globals[address] = value; break; + case T_GINT: Globals[address] = value; break; + case T_GFLOAT: GlobalFloats[address] = value; break; case T_LINT: _cur.Locals.I[address] = value; break; case T_LFLOAT: _cur.Locals.F[address] = value; break; - case T_GPTR: Globals[checked((int)Gi(Globals, (int)destination.Value) + index)] = value; break; + case T_GPTR: + Globals[checked(GlobalPointer((int)destination.Value) + index)] = value; + break; case T_LPTR: WriteIntCell(Ga(_cur.Locals.P, (int)destination.Value).Offset(index), value); break; } } @@ -528,9 +548,10 @@ public sealed class VirtualMachine { T_LINT => Gi(_cur.Locals.I, checked((int)operand.Value + offset)), T_LFLOAT => Gi(_cur.Locals.F, checked((int)operand.Value + offset)), - T_GINT or T_GFLOAT => ReadGlobal(checked((int)operand.Value + offset)), + T_GINT => ReadGlobal(checked((int)operand.Value + offset)), + T_GFLOAT => Gi(GlobalFloats, checked((int)operand.Value + offset)), T_LPTR => ReadIntCell(Ga(_cur.Locals.P, (int)operand.Value).Offset(offset)), - T_GPTR => Gi(Globals, checked((int)Gi(Globals, (int)operand.Value) + offset)), + T_GPTR => Gi(Globals, checked(GlobalPointer((int)operand.Value) + offset)), _ => Gi(Globals, checked((int)operand.Value + offset)), }; } @@ -541,7 +562,8 @@ public sealed class VirtualMachine T_LINT => (VmAddressSpace.LocalInteger, checked((int)operand.Value + offset)), T_LFLOAT => (VmAddressSpace.LocalFloat, checked((int)operand.Value + offset)), T_LPTR => PointerIdentity(Ga(_cur.Locals.P, (int)operand.Value).Offset(offset)), - T_GPTR => (VmAddressSpace.Global, checked((int)Gi(Globals, (int)operand.Value) + offset)), + T_GPTR => (VmAddressSpace.Global, + checked(GlobalPointer((int)operand.Value) + offset)), _ => (VmAddressSpace.Global, checked((int)operand.Value + offset)), }; @@ -554,6 +576,7 @@ public sealed class VirtualMachine : unchecked((int)Read(operand)).ToString(System.Globalization.CultureInfo.InvariantCulture); private sealed class RootReloadRequestedException : Exception { } + private sealed class NumberedRestoreRequestedException : Exception { } private sealed class ProcessExitRequestedException : Exception { } private sealed record DebugFrameReturnRequest(ExecFrame Frame, IReadOnlyDictionary GlobalWrites); private sealed record PreloadedScriptSlot(long ScriptId, ExecFrame Frame); @@ -571,7 +594,32 @@ public sealed class VirtualMachine FrameCause cause = FrameCause.TopScene; while (true) { - var outcome = RunFrame(new ExecFrame(root, rootEntry), cause); + FrameOutcome outcome; + try + { + outcome = RunFrame(new ExecFrame(root, rootEntry), cause); + } + catch (NumberedRestoreRequestedException) + { + if (_loadedNumberedState == null) throw; + Script? callback = _provider?.GetByName("CALLBACK_LOAD.BIN"); + if (callback != null) + { + int callbackEntry = callback.IndexByOffset.TryGetValue(0, out int ci) ? ci : 0; + FrameOutcome callbackOutcome = RunFrame( + new ExecFrame(callback, callbackEntry), FrameCause.SaveRestore, callback.PackedId); + if (callbackOutcome is FrameOutcome.Halted or FrameOutcome.ExitRequested) + { + outcome = callbackOutcome; + break; + } + } + root = ResolveSavedScript(_loadedNumberedState.Frames[0]); + rootEntry = FindRestoreRendezvous(root); + _restoreFrameIndex = 0; + cause = FrameCause.SaveRestore; + continue; + } if (outcome == FrameOutcome.RootReload) { // Native 0x9 performs the scene reset before attempting the resource-0 load. Keep @@ -666,6 +714,7 @@ public sealed class VirtualMachine { if (Steps >= _o.MaxSteps) { HaltReason ??= "STEP-LIMIT"; outcome = FrameOutcome.Halted; break; } Steps++; + frame.Pc = pc; if (_sink.TracingSteps) _sink.Emit(TraceEvent.Step(pc, frame.Script.Instructions[pc], _depth)); int next = Step(frame.Script.Instructions[pc], pc); _host.FrameYield(); @@ -727,6 +776,192 @@ public sealed class VirtualMachine } } + private NativeNumberedSaveState CaptureNumberedState() + { + const int himegariIntegerCount = 0x6241b; + const int himegariFloatCount = 1; + const int himegariStringCount = 0x315; + const int himegariPointerCount = 1; + + ExecFrame[] active; + int cutoff; + lock (_debugControlLock) + { + active = _activeExecutionFrames.ToArray(); + cutoff = _saveResumeFrame == null ? active.Length - 1 : Array.IndexOf(active, _saveResumeFrame); + } + if (cutoff < 0) + throw new InvalidDataException("No active script frame is available for a numbered save."); + + var frames = new NativeSavedScriptFrame[cutoff + 1]; + for (int i = 0; i <= cutoff; i++) + { + ExecFrame frame = active[i]; + int[] returns = frame.CallStack + .Where(returnPc => (uint)returnPc < (uint)frame.Script.Instructions.Count) + .Select(returnPc => + { + int returnOffset = frame.Script.Instructions[returnPc].Offset; + return FindTableIndex(frame.Script.LocalCallOffsets, returnOffset - 3); + }) + .Where(index => index >= 0) + .ToArray(); + int resumeIndex = CurrentReadMessageIndex(frame); + int callTargetIndex = i == cutoff || (uint)frame.Pc >= (uint)frame.Script.Instructions.Count + ? -1 + : FindTableIndex(frame.Script.ScriptCallOffsets, frame.Script.Instructions[frame.Pc].Offset); + frames[i] = new NativeSavedScriptFrame( + i - 1, frame.Script.PackedId, returns, resumeIndex, callTargetIndex); + } + + NativeNumberedSaveState basis = _retainedNativeNumberedState + ?? NativeNumberedSaveCodec.Empty(frames); + var gfx = NativeGfxPersistenceCodec.Capture(Gfx); + return basis with + { + Frames = frames, + IntegerGlobals = DenseValues(Globals, himegariIntegerCount), + FloatGlobals = DenseValues(GlobalFloats, himegariFloatCount), + StringGlobals = DenseStrings(GlobalStrings, himegariStringCount), + PointerGlobals = DensePointerValues(GlobalPointers, himegariPointerCount), + PointerStrings = DensePointerValues(GlobalStringPointers, himegariPointerCount), + LocalPointerScratch = new int[himegariPointerCount], + SurfaceRecords = gfx.SurfaceRecords, + GfxObjects = gfx.Objects, + RangeTransformFirst = gfx.RangeFirst, + RangeTransformCount = gfx.RangeCount, + RangeTransformRecord = gfx.RangeRecord, + }; + } + + private bool TryLoadNumberedState(int slot, bool restoreHistory) + { + if (_nativeDatStore == null) return false; + try + { + NativeNumberedSaveFile? file = _nativeDatStore.LoadNumberedFile(slot); + if (file == null) return false; + NativeNumberedSaveState state = NativeNumberedSaveCodec.Decode(file.Document.Payload); + ApplyNumberedState(state); + _retainedNativeNumberedState = state; + _accumulatedPlaySeconds = file.Document.Metadata.AccumulatedPlaySeconds; + if (restoreHistory) + { + if (file.HistoryTail.Length == 0) TextHistory.Clear(); + else NativeTextHistoryCodec.DecodeInto(file.HistoryTail, TextHistory); + _loadedNumberedState = state; + } + else + { + _loadedNumberedState = null; + _restoreFrameIndex = -1; + } + return true; + } + catch (Exception error) when ( + error is IOException or UnauthorizedAccessException or InvalidDataException + or ArgumentOutOfRangeException or OverflowException) + { + _loadedNumberedState = null; + _restoreFrameIndex = -1; + return false; + } + } + + private void ApplyNumberedState(NativeNumberedSaveState state) + { + Globals.Clear(); + for (int i = 0; i < state.IntegerGlobals.Count; i++) + if (state.IntegerGlobals[i] != 0) Globals[i] = state.IntegerGlobals[i]; + GlobalFloats.Clear(); + for (int i = 0; i < state.FloatGlobals.Count; i++) + if (state.FloatGlobals[i] != 0) GlobalFloats[i] = state.FloatGlobals[i]; + GlobalStrings.Clear(); + for (int i = 0; i < state.StringGlobals.Count; i++) + if (state.StringGlobals[i].Length != 0) GlobalStrings[i] = state.StringGlobals[i]; + GlobalPointers.Clear(); + for (int i = 0; i < state.PointerGlobals.Count; i++) + if (state.PointerGlobals[i] != 0) GlobalPointers[i] = state.PointerGlobals[i]; + GlobalStringPointers.Clear(); + for (int i = 0; i < state.PointerStrings.Count; i++) + if (state.PointerStrings[i] != 0) GlobalStringPointers[i] = state.PointerStrings[i]; + + GfxPersistenceSnapshot gfxSnapshot = NativeGfxPersistenceCodec.Decode(state); + for (int slot = 0; slot < 1000; slot++) _host.ReleaseSurface(slot); + Gfx.RestorePersistenceSnapshot(gfxSnapshot); + foreach (var (slot, resourceId, colorKey, created) in gfxSnapshot.Surfaces) + { + if (!created) _host.SetTexture(resourceId, slot, colorKey); + } + } + + private Script ResolveSavedScript(NativeSavedScriptFrame frame) + { + if (_s.PackedId == frame.ScriptId) return _s; + Script? script = _provider?.GetById(frame.ScriptId); + return script ?? throw new InvalidDataException( + $"Numbered save references unresolved script 0x{frame.ScriptId:x}."); + } + + private static int FindRestoreRendezvous(Script script) + { + for (int i = 0; i < script.Instructions.Count; i++) + if (script.Instructions[i].Opcode == 0xae) return i; + throw new InvalidDataException( + $"Saved script {script.Name} has no opcode 0xae restore rendezvous."); + } + + private static int ResolveTableOffset( + Script script, IReadOnlyList table, int index, int fallback) + { + if ((uint)index >= (uint)table.Count) return fallback; + return script.IndexByOffset.TryGetValue(table[index], out int pc) ? pc : fallback; + } + + private static int CurrentReadMessageIndex(ExecFrame frame) + { + for (int i = 0; i < frame.Script.ReadMessageOffsets.Count; i++) + if (frame.Script.ReadMessageOffsets[i] == frame.ReadMessageOffset) return i; + return -1; + } + + private static int FindTableIndex(IReadOnlyList table, int offset) + { + for (int i = 0; i < table.Count; i++) + if (table[i] == offset) return i; + return -1; + } + + private static int[] DenseValues(IReadOnlyDictionary source, int fixedCount) + { + var result = new int[fixedCount]; + foreach ((int index, long value) in source) + if ((uint)index < (uint)result.Length) result[index] = unchecked((int)value); + return result; + } + + private static int[] DensePointerValues(IReadOnlyDictionary source, int fixedCount) + { + var result = new int[fixedCount]; + foreach ((int index, int value) in source) + if ((uint)index < (uint)result.Length) result[index] = value; + return result; + } + + private static string[] DenseStrings(IReadOnlyDictionary source, int fixedCount) + { + var result = Enumerable.Repeat(string.Empty, fixedCount).ToArray(); + foreach ((int index, string value) in source) + if ((uint)index < (uint)result.Length) result[index] = value; + return result; + } + + private uint AccumulatedPlaySeconds() + { + double elapsedSeconds = System.Diagnostics.Stopwatch.GetElapsedTime(_sessionStartTimestamp).TotalSeconds; + return unchecked(_accumulatedPlaySeconds + (uint)Math.Min(uint.MaxValue, elapsedSeconds)); + } + private bool ServiceHotspotCallback() { int target; @@ -741,6 +976,7 @@ public sealed class VirtualMachine { if (Steps >= _o.MaxSteps) { HaltReason ??= "STEP-LIMIT"; break; } Steps++; + _cur.Pc = pc; if (_sink.TracingSteps) _sink.Emit(TraceEvent.Step(pc, _cur.Script.Instructions[pc], _depth)); int next = Step(_cur.Script.Instructions[pc], pc); _host.FrameYield(); @@ -819,6 +1055,76 @@ public sealed class VirtualMachine case "halve-strlen": // 0x1a6: strlen(native encoded bytes) >> 1 Write(a[0], NativeStringByteLength(ReadStr(a[1])) >> 1); return pc + 1; + case "save-numbered-slot": // 0x19e + { + if (_nativeDatStore == null) + { + Write(a[0], 1); + return pc + 1; + } + try + { + int slot = unchecked((int)Read(a[1])); + NativeNumberedSaveState state = CaptureNumberedState(); + byte[] payload = NativeNumberedSaveCodec.Encode(state); + byte[] history = NativeTextHistoryCodec.Encode(TextHistory); + NativeSystemTime timestamp = NativeSystemTime.FromLocalDateTime(DateTime.Now); + uint playSeconds = AccumulatedPlaySeconds(); + _nativeDatStore.SaveNumberedFile(slot, payload, history, timestamp, playSeconds); + _sharedProfile.Save(_nativeDatStore, timestamp, playSeconds); + Write(a[0], 0); + } + catch (Exception error) when ( + error is IOException or UnauthorizedAccessException or InvalidDataException + or ArgumentOutOfRangeException or OverflowException) + { + Write(a[0], 1); + } + return pc + 1; + } + case "load-numbered-slot-data-only": // 0x19f + { + if (!TryLoadNumberedState(unchecked((int)Read(a[1])), restoreHistory: false)) + Write(a[0], 1); + else + Write(a[0], 0); + return pc + 1; + } + case "load-numbered-slot-and-resume": // 0x1a1 + { + if (!TryLoadNumberedState(unchecked((int)Read(a[1])), restoreHistory: true)) + { + Write(a[0], 1); + return pc + 1; + } + throw new NumberedRestoreRequestedException(); + } + case "continue-save-load-stack-restore": // 0xae + { + if (_loadedNumberedState == null || _restoreFrameIndex < 0) + return pc + 1; + NativeSavedScriptFrame saved = _loadedNumberedState.Frames[_restoreFrameIndex]; + bool terminal = _restoreFrameIndex == _loadedNumberedState.Frames.Count - 1; + if (terminal) + { + _loadedNumberedState = null; + _restoreFrameIndex = -1; + return ResolveTableOffset(_cur.Script, _cur.Script.ReadMessageOffsets, saved.ResumeIndex, pc + 1); + } + + int parentIndex = _restoreFrameIndex; + NativeSavedScriptFrame childSaved = _loadedNumberedState.Frames[parentIndex + 1]; + Script child = ResolveSavedScript(childSaved); + _restoreFrameIndex = parentIndex + 1; + FrameOutcome childOutcome = RunFrame( + new ExecFrame(child, FindRestoreRendezvous(child)), FrameCause.SaveRestore, + childSaved.ScriptId); + _restoreFrameIndex = parentIndex; + if (childOutcome is FrameOutcome.Halted or FrameOutcome.ExitRequested) + return HALT; + return ResolveTableOffset( + _cur.Script, _cur.Script.ScriptCallOffsets, saved.CallTargetIndex, pc) + 1; + } case "query-numbered-save-metadata": // 0x1a0 { if (_nativeDatStore == null) @@ -1354,6 +1660,8 @@ public sealed class VirtualMachine // each invocation an independent diagnostic activation. loaded.Frame.CallStack.Clear(); loaded.Frame.EmitSeen.Clear(); + loaded.Frame.Pc = loaded.Frame.Script.IndexByOffset.TryGetValue(0, out int loadedEntry) + ? loadedEntry : 0; var outcome = RunFrame(loaded.Frame, FrameCause.CallScript, loaded.ScriptId); if (outcome == FrameOutcome.Halted) return HALT; if (outcome == FrameOutcome.RootReload) return ROOT_RELOAD; diff --git a/vm-map/opcodes.toml b/vm-map/opcodes.toml index 88abc09..10eca33 100644 --- a/vm-map/opcodes.toml +++ b/vm-map/opcodes.toml @@ -1724,11 +1724,12 @@ abi_source = "kelebek+decode-validated" name = "continue-save-load-stack-restore" category = "control" summary = "() - during serialized save restoration, replace the current frame PC with its saved resume/call target and advance through the saved script-context stack; otherwise a no-op." +details = "Layout 3 frame d259 indexes SYS4 T1 read-message reset sites, d260 indexes T2 call-script sites, and the saved local return stack indexes T3 local-call sites. Port status (2026-07-24): the active path reconstructs the saved recursive frame chain and resumes the terminal frame at its T1 boundary." noop_headless = false source = "investigation" confidence = "high" depends_on = [] -evidence = "Ghidra /v2: op_0xae_continue_save_load_stack_restore@0x416790 first tests ctx+0x53d24 (set by save_data_deserialize_and_begin_restore@0x40fd10). When clear it returns. When set, it selects the serialized frame layout through set:SaveVersion1/2, restores the current PC from that layout's saved return/call target, advances through contexts with FUN_0040f2d0, and clears the restore flag on reaching the saved terminal context. Its 305 corpus sites overwhelmingly follow coroutine-resume/call boundaries, which provide the rendezvous points used while reconstructing the stack." +evidence = "Ghidra /v2: op_0xae_continue_save_load_stack_restore@0x416790 first tests ctx+0x53d24 (set by save_data_deserialize_and_begin_restore@0x40fd10). When clear it returns. When set, it selects the serialized frame layout through set:SaveVersion1/2, restores the current PC from that layout's saved return/call target, advances through contexts with script_frame_restore_saved_layout@0x40f2d0, and clears the restore flag on reaching the saved terminal context. Its 305 corpus sites overwhelmingly follow coroutine-resume/call boundaries, which provide the rendezvous points used while reconstructing the stack." [[opcode]] op = 0xb4 @@ -3384,7 +3385,7 @@ abi_source = "kelebek+decode-validated" name = "save-numbered-slot" category = "control" summary = "(status_out)(slot) - write the active native VM/session state to `SAVE%02d.DAT`. The file is truncated/replaced in place after any compatibility-overwrite prompt; successful serialization also flushes shared `SAVE.DAT`/`RT.DAT`." -details = "Uses `set:SaveVersion1` and `set:SaveVersion2` to choose the numbered payload layout. Status is 0 on success and 1 on refusal, open/create failure, or serializer failure. Numbered `.DAT` files do not use the shared profile's temp/backup replacement scheme." +details = "Uses `set:SaveVersion1` and `set:SaveVersion2` to choose the numbered payload layout. Status is 0 on success and 1 on refusal, open/create failure, or serializer failure. Numbered `.DAT` files do not use the shared profile's temp/backup replacement scheme. Port status (2026-07-24): layout 3 writes fixed state, six native banks, T1/T2/T3 frame records, surface/resource reload state, retained gfx records, the appended history tail, and then flushes shared SAVE.DAT/RT.DAT." noop_headless = false source = "investigation" confidence = "high" @@ -3411,7 +3412,7 @@ abi_source = "kelebek" name = "load-numbered-slot-data-only" category = "control" summary = "(status_out)(slot) - decode `SAVE%02d.DAT` without restoring the active script-frame chain or text history." -details = "This is the data-only companion to full-resume opcode 0x1a1. It selects the configured SaveVersion layout and restores serialized state with both runtime/history restore flags clear. Himegari's shipped script corpus does not call it, but it belongs to the shared SYS4 persistence ABI." +details = "This is the data-only companion to full-resume opcode 0x1a1. It selects the configured SaveVersion layout and restores serialized state with both runtime/history restore flags clear. Himegari's shipped script corpus does not call it, but it belongs to the shared SYS4 persistence ABI. Port status (2026-07-24): implemented for layout 3 through the same bank/resource/gfx decoder without activating history or frame restoration." noop_headless = false source = "investigation" confidence = "high" @@ -3500,7 +3501,7 @@ abi_source = "kelebek+decode-validated" name = "load-numbered-slot-and-resume" category = "control" summary = "(status_out)(slot) - fully load `SAVE%02d.DAT`, including saved script frames and text history, then resume through `CALLBACK_LOAD.BIN` and opcode 0xae." -details = "The caller pre-seeds status to zero. A missing/open failure writes 1; success starts the asynchronous native stack-restoration rendezvous and does not overwrite that zero. This is the ordinary load-game path, unlike data-only opcode 0x19f." +details = "The caller pre-seeds status to zero. A missing/open failure writes 1; success starts the asynchronous native stack-restoration rendezvous and does not overwrite that zero. This is the ordinary load-game path, unlike data-only opcode 0x19f. Port status (2026-07-24): layout 3 restores banks, history, surfaces/resources, and retained gfx, unwinds the obsolete managed call chain, then reconstructs saved frames through opcode 0xae." noop_headless = false source = "investigation" confidence = "high"