diff --git a/docs/engine-re.md b/docs/engine-re.md index f5f7136..2e4bf23 100644 --- a/docs/engine-re.md +++ b/docs/engine-re.md @@ -1754,8 +1754,9 @@ live surfaces and retained objects, and applies the 20-byte surface records' exp load first replaces each serialized mutable bank prefix while preserving initialization-authored cells beyond its count, restores history/gfx and retained audio, unwinds the obsolete managed call chain, runs `CALLBACK_LOAD.BIN` when -the mounted script provider resolves it, starts the saved root at its `0xae` rendezvous, recursively -reconstructs child frames, resumes parents after their saved T2 call sites, +the mounted script provider resolves it, loads each saved script at its ordinary entry so its frame-local +prologue runs, lets that script reach its own `0xae` rendezvous, recursively reconstructs child frames, +resumes parents after their saved T2 call sites, and finally resumes the terminal frame at its T1 boundary. Successful `0x19e` also flushes shared `SAVE.DAT`/`RT.DAT`, matching `context_state_serialize`. @@ -1791,6 +1792,11 @@ separately gated by both `set:CreateObject` and `set:AutoFreeTex`; the registere path. Opcode `0x259` is also a real lifecycle operation: `op_0x259_script_entry_clear_surface_persistence_flags@0x417660` clears record `+0x08` and `+0x0c` across both native tables at every script entry. +Opcode `0x258` then declares the exceptions: +`op_0x258_set_surface_persistence_flags@0x4250a0` calls +`gfx_surface_set_persistence_flags@0x4159c0`, which replaces record `+0x08` from flags bit 0 and +record `+0x0c` from bit 1 in both tables. The port models bit 0 because layout-3 restoration consumes +that reload policy; the adjacent bit-1 consumer remains unnamed. Installed slot 15 records resource `0x3383` with reload flag zero. SYSTEM4 loads that atlas before the save UI, and BUNKI cuts its reusable choice-box corners, borders, and winged top ornament from it. The port formerly released all 1,000 host surfaces unconditionally, so post-load BUNKI retained its black @@ -1800,6 +1806,67 @@ native all-release branch. Port-authored saves emit the exact resource/reload/cr `-1` for unused records, and opcode `0x259` clears the modeled reload policy without releasing textures. The corrected native functions are renamed/commented in the saved `/v2` image. +#### Port-authored round trips: restored frame boundary and retained-object encoding (2026-07-24) + +An immediate `installed SAVE00 -> port SAVE03` round trip isolated two independent writer defects. +The source file contains two frames (`SYSTEM4.BIN -> FORT.BIN`), while the first port rewrite contained +four: the same gameplay pair plus `SAVE.BIN` and its nested helper. Native opcode `0x1ad` selects the +terminal gameplay frame before opening modal helpers. Full restore through `0xae` reinstates that saved +terminal context as the effective boundary. The managed restore cleared its temporary restore state but +did not restore `_saveResumeFrame`, so the next `0x19e` fell back to the currently deepest frame. The port +now marks the terminal restored `ExecFrame` before resuming at T1; a regression loads two frames, saves +from a nested helper, and proves the rewrite still contains only the original two. + +The first fresh rewrite (`SAVE04`) then exposed the other half of that managed/native representation gap. +It had two frames, but its restored SYSTEM4 ancestor changed from native `resume=8, call=8` to +`resume=-1, call=-1`. Native `op_0xae_continue_save_load_stack_restore@0x416790` restores the parent +context's real T1/T2/T3 coordinates before activating its child. The managed port instead leaves the +parent physically parked at the synthetic `0xae` rendezvous while `RunFrame` recursively runs FORT. +Recomputing a save record from that synthetic PC cannot find a T1 or T2 table entry. FORT itself loads +and runs, but when stage launch unwinds through SYSTEM4, the missing T2 continuation enters ordinary +Eushully boot. + +Each restored `ExecFrame` now shadows its original `NativeSavedScriptFrame` while a restored descendant +is active. An intervening `0x19e` serializes those original T1/T2/T3 indices; T3 also reconstructs the +live local-return stack, and the saved T1 coordinate seeds the frame's read-message state. The shadow is +discarded only when that child genuinely returns and the parent resumes at its restored T2 continuation. +Restored root-reload and process-exit outcomes also propagate through the synthetic recursion. A focused +load -> nested save -> reload regression proves the parent indices survive and the child is not invoked +again through ordinary `CallScript`. + +Manual slot 005 acceptance then proved that an immediate base rewrite could load and later launch a +stage. Slot 006, authored immediately after entering that stage, still restored a black dungeon with +visible but noninteractive UI. Its serialized FIELD range, 704 retained objects, map textures, and +native-looking 0.8 transform were intact. A software replay reproduced the black result because FIELD +had changed every restored map object's scale to `(0,0,1)`: global zoom selector `G[0x7682]` was 3, but +the frame-local lookup table that should yield 80 was still zero. + +Ghidra resolves the ordering precisely. `script_frame_restore_saved_layout@0x40f2d0` calls +`script_frame_load_resource@0x40e980`; that loader allocates and zeroes locals, loads the script body, +and initializes its PC to the script codebase, not to `0xae`. The restored script therefore executes its +ordinary entry prologue before reaching the rendezvous. FIELD's prologue constructs the inline zoom +table `[40, 50, 64, 80, 100, 124, 156]`; the port's direct jump to `0xae` skipped those copies and let +FIELD apply zero scale after the gfx record restore. + +Restored managed frames now likewise begin at script offset zero while retaining their saved T1/T2/T3 +coordinates for `0xae`. Synthetic root and child regressions prove both prologues execute. Replaying the +unchanged real slot 006 now selects zoom 80, leaves the map at 0.8 scale, produces a nonblack dungeon +raster, and reaches the gameplay poll. No slot rewrite is required for this correction. + +The same comparison exposed native retained-gfx pointer arithmetic. `context_state_serialize@0x40d320` +writes a handle and copies the meaningful `0xb5` DWORD / `0x2d4`-byte record, then advances its DWORD +cursor by `0x2d5`; `save_data_deserialize_and_begin_restore@0x40fd10` mirrors that stride. Early port +output packed entries contiguously. In addition, `gfx_object_init_default@0x472810` seeds six identity +matrices and three `0xffffffff` color defaults before the loader overwrites the complete record; the old +writer began with zero bytes and only filled modeled fields. It also wrote translation vectors over the +first row of the native 4x4 matrices instead of entries 12..14. + +`NativeNumberedSaveCodec` now emits the native `0xb54`-byte entry stride and recognizes the old tight +experimental layout for compatibility. `NativeGfxPersistenceCodec` begins new records from the exact +native defaults, writes row-vector translation matrices correctly, and retains the original raw record +behind the semantic object so still-unnamed fields survive native load/save cycles. The `/v2` serializer +and deserializer comments record the corrected stride and overwrite behavior. + ### Opcode `0xae` continues numbered-save stack restoration (2026-07-20) Opcode `0xae` is the load-side rendezvous paired with serialized script-frame state. Its handler, @@ -1816,6 +1883,12 @@ T1/T2/T3 indices back into live PC/call/return offsets. At the saved terminal co the restore flag and reinstates the saved context/return state. The corpus placement supports that control-flow role: 305 calls overwhelmingly follow coroutine-resume or call boundaries, including SC0000's main-loop resume sequence. +The load order is significant: `script_frame_restore_saved_layout` delegates script creation to +`script_frame_load_resource@0x40e980`, which initializes the new frame PC to the script codebase. +Consequently the script runs its normal frame-local initialization and reaches `0xae` itself. Starting +a restored frame directly at `0xae` is not native behavior; in FIELD it leaves the local zoom table +zeroed and collapses the restored dungeon to a black point. + The port now implements both branches: an ordinary one-instruction no-op outside restoration, and the T1/T2/T3-driven managed-frame reconstruction described above while a full numbered load is active. @@ -2534,7 +2607,24 @@ backbuffer zero instead and record current target `-1` at manager `+0xb530` (`En `0x20e` then calls `d3d_clear_render_target_black@0x471460`, which invokes `IDirect3DDevice9::Clear` with no rectangles, `D3DCLEAR_TARGET | D3DCLEAR_ZBUFFER`, color zero, depth 1.0, and stencil zero. The port tracks the selected target and forwards the clear to the host; the retained compositor reconstructs its main -backbuffer from black at publication boundaries, while offscreen clears also discard modeled text pixels. +backbuffer from black at publication boundaries, while offscreen clears discard both modeled pixels and +text draws. + +Opcode `0x222` publishes its retained handle range into whichever target `0x20d` selected; it is not +merely an onscreen repaint request. SAVE.BIN provides a compact end-to-end example. It creates slot 2 at +800x600, selects and clears it, then publishes handles `[0,0x130b0)` to capture the underlying gameplay +without the save-menu objects. It next creates/selects/clears slot 192 at 112x84, binds slot 2 through +handle `0x15f90`, scales that object to 14%, publishes the two-handle range, and passes slot 192 to +`0x1ae`. Dungeon scripts use the same selected-target publication to construct map surfaces. + +The port originally retained only an object-list snapshot for transitions and requested an ordinary +backbuffer repaint from `PresentObjectRange`; the selected surface's pixels remained its cleared black +allocation. That produced completely black port-authored `.STH` files and caused restored dungeon map +surfaces to turn black when their rebuild published offscreen. Godot now uses the platform-neutral +software affine compositor for selected-target `0x222`/`0x20c` publication, including source surfaces, +scaling, transforms, tint/opacity/blend, handle-range filtering, and real pixel clears. Backbuffer +publication remains on the existing renderer. `gfx_present_object_range@0x482230` records the refined +target ownership in the saved `/v2` image. This trace also corrects an important base-pointer assumption in earlier graphics notes. Offsets `+0x408` and `+0xb550` are relative to the retained-gfx manager at `EngineCtx+0x46614`, not to `EngineCtx` itself. diff --git a/docs/opcode-reference.md b/docs/opcode-reference.md index c33a9cd..a3d193c 100644 --- a/docs/opcode-reference.md +++ b/docs/opcode-reference.md @@ -405,9 +405,9 @@ Implemented as a whole-stack root-reload boundary in the persistent VM. A reques ### 0xae `continue-save-load-stack-restore` (continue-save-load-stack-restore, argc 0) - **summary:** () - during serialized save restoration, replace the current frame PC with its saved resume/call target and advance through the saved script-context stack; otherwise a no-op. - **grounding:** source=investigation, confidence=high -- **evidence:** Ghidra /v2: op_0xae_continue_save_load_stack_restore@0x416790 first tests ctx+0x53d24 (set by save_data_deserialize_and_begin_restore@0x40fd10). When clear it returns. When set, it selects the serialized frame layout through set:SaveVersion1/2, restores the current PC from that layout's saved return/call target, advances through contexts with script_frame_restore_saved_layout@0x40f2d0, and clears the restore flag on reaching the saved terminal context. Its 305 corpus sites overwhelmingly follow coroutine-resume/call boundaries, which provide the rendezvous points used while reconstructing the stack. +- **evidence:** Ghidra /v2: op_0xae_continue_save_load_stack_restore@0x416790 first tests ctx+0x53d24 (set by save_data_deserialize_and_begin_restore@0x40fd10). When clear it returns. When set, it selects the serialized frame layout through set:SaveVersion1/2, restores the current PC from that layout's saved return/call target, advances through contexts with script_frame_restore_saved_layout@0x40f2d0, and clears the restore flag on reaching the saved terminal context. The restore helper calls script_frame_load_resource@0x40e980, which initializes each new frame PC to its script codebase; the script therefore executes its ordinary prologue before reaching 0xae. Native parent contexts retain restored coordinates while the child runs. SAVE00 -> SAVE03 proved the port must retain the terminal 0x1ad boundary; SAVE00 -> SAVE04 proved a managed parent parked at synthetic 0xae must serialize original resume=8/call=8 rather than synthetic -1/-1; slot 006 proved direct entry at 0xae skips FIELD's zoom-table prologue and collapses the dungeon map. Its 305 corpus sites overwhelmingly follow coroutine-resume/call boundaries. -Layout 3 frame d259 indexes SYS4 T1 read-message reset sites, d260 indexes T2 call-script sites, and the saved local return stack indexes T3 local-call sites. Port status (2026-07-24): the active path reconstructs the saved recursive frame chain and resumes the terminal frame at its T1 boundary. The installed SAVE00 continuation gate proves SYSTEM4 -> FORT restoration reaches FORT's CHMENU gameplay poll; the synthetic gate asserts the child frame enters with SaveRestore rather than ordinary CallScript cause. +Layout 3 frame d259 indexes SYS4 T1 read-message reset sites, d260 indexes T2 call-script sites, and the saved local return stack indexes T3 local-call sites. Port status (2026-07-24): each saved script is loaded at its ordinary entry, runs its frame-local prologue, and reaches 0xae itself; the active path then reconstructs the saved recursive frame chain, preserves each synthetic ancestor's original T1/T2/T3 coordinates while its restored child is active, reinstates the terminal restored frame as opcode 0x1ad's save boundary, and resumes it at T1. The installed SAVE00 continuation gate proves SYSTEM4 -> FORT restoration reaches FORT's CHMENU gameplay poll; the unchanged port-authored dungeon slot 006 proves FIELD's pre-rendezvous zoom table is initialized; synthetic gates assert both root/child prologue execution, SaveRestore rather than ordinary CallScript entry, exclusion of nested SAVE helpers from a re-save, and reload of that rewrite without ordinary boot re-entry. ### 0xc8 `sleep` (sleep, argc 1) - **summary:** Pause the current script for milliseconds while retained presentation continues. @@ -790,9 +790,9 @@ Implemented through IHost.PlayModalMovieToSurface. Its operand uses the same nat - **evidence:** Ghidra 0x47ecc0 calls matrix builder 0x48afb1 for target obj+0x1ac. Consumer 0x472f00 uses delay obj+0x44, duration obj+0x58, current obj+0x16c, target obj+0x1ac, shared start obj+0x34, and retained-gfx frame time owner+0xb550 (EngineCtx+0x51b64). ### 0x222 `present-gfx-object-range` (present-gfx-object-range, argc 2) -- **summary:** (first_handle)(count) - flush/present retained graphics objects in the selected handle range and clear their pending update flags. +- **summary:** (first_handle)(count) - flush/present retained graphics objects in the selected handle range into the currently selected backbuffer or offscreen render target, then clear their pending update flags. - **grounding:** source=investigation, confidence=high -- **evidence:** Ghidra /v2: op_0x222_handler@0x4235e0 calls gfx_present_object_range@0x482230. The worker enters the graphics service, walks the retained-object map, processes flagged objects whose handles fall in [first,first+count), clears pending flags, and finalizes the render batch. HISTORY.BIN uses (0,60000) after rebuilding its retained presentation. +- **evidence:** Ghidra /v2: op_0x222_handler@0x4235e0 calls gfx_present_object_range@0x482230. The worker enters the graphics service, walks the retained-object map, processes flagged objects whose handles fall in [first,first+count), clears pending flags, and finalizes the render batch in the D3D target previously selected by op 0x20d. HISTORY.BIN uses (0,60000) for the backbuffer. SAVE.BIN instead renders [0,0x130b0) into 800x600 slot 2, then handle 0x15f90 at 14% scale into 112x84 slot 192; op 0x1ae writes slot 192 as the numbered .STH thumbnail. ### 0x223 `queue-surface-alpha-transition` (queue-surface-alpha-transition, argc 8) - **summary:** (command_key)(target_slot)(range_a_start)(range_a_count)(range_b_start)(range_b_count)(delay_ms)(duration_ms) — queue a type-0 timed alpha transition command in the separate ctx+0x414 command map. This is render-target/surface presentation state, not an object affine matrix. The render frame composites the two handle ranges into target_slot and ramps alpha 0->1 after delay over duration. @@ -924,6 +924,13 @@ The setter get-or-creates the object and writes the complete operand. During ret - **grounding:** source=investigation, confidence=high - **evidence:** Ghidra /v2: op_0x24e_handler@0x425070 writes operand 1 directly to EngineCtx.gfx_animation_service_flags at +0x51b80. The mapped field is also read by op 0x243: bit 1 suppresses its force-complete/clock-reset request. +### 0x258 `set-surface-persistence-flags` (set-surface-persistence-flags, argc 2) +- **summary:** (surface_slot)(flags) - replace the native surface record's numbered-save persistence flags; bit 0 controls asset reload on restore and bit 1 controls the adjacent still-unnamed field. +- **grounding:** source=investigation, confidence=high +- **evidence:** Ghidra /v2: op_0x258_set_surface_persistence_flags@0x4250a0 reads the slot and flags operands and calls gfx_surface_set_persistence_flags@0x4159c0. The worker writes flags&1 to record +0x08 and (flags>>1)&1 to +0x0c in both 1,000-record tables. This corrects the old upstream address/name association: 0x422fe0 is opcode 0x20f movie playback, not 0x258. Corpus declaration chains follow opcode 0x259 at script entry. + +Scripts place declaration chains immediately after opcode 0x259 clears both fields. The port models bit 0 because it is consumed by layout-3 restoration; bit 1 is retained as an identified native field but has no known runtime consumer yet. + ### 0x259 `script-entry` (script-entry, argc 0) - **summary:** zero-arg script/prologue entry; clears surface-record persistence fields +0x08 and +0x0c across both 1,000-record native tables before the declaration chain continues - **grounding:** source=investigation, confidence=high @@ -1150,10 +1157,6 @@ Port status (2026-07-24): implemented through the same profile-lifetime setting - **summary:** 1 imm; mov->0x21b->stmt-end; near save/load-messkip — likely line/stmt id, verify not msg-control - **grounding:** source=harness, confidence=med, noop_headless=True -### 0x258 `decl?` (u00422FE0, argc 2) -- **summary:** 2 imm; runs in a chain right after script-entry 0x259, enumerating ids — prologue declaration/registration? -- **grounding:** source=harness, confidence=low, noop_headless=True - ## unknown ### 0x2 `exit` (exit, argc 0) diff --git a/docs/phase-a-slice-plan.md b/docs/phase-a-slice-plan.md index ceb97eb..c5a0d51 100644 --- a/docs/phase-a-slice-plan.md +++ b/docs/phase-a-slice-plan.md @@ -3628,6 +3628,111 @@ mapped and implemented as the script-entry clear of record `+0x08`/`+0x0c` (the Regressions cover native-default preservation, opt-in all-release, flag-for-flag port-authored output, script-entry clearing, and the installed all-zero reload-bit oracle. +### Persistence implementation step 10 — publish selected offscreen render targets (2026-07-24) + +The first port-authored dungeon save exposed two symptoms with one graphics cause: its 112x84 `.STH` +was completely black, and loading it showed the dungeon map for one frame before the map surface became +black while the UI/minimap remained live. + +SAVE.BIN's bytecode makes the missing contract explicit. It renders the underlying gameplay handle range +into created 800x600 surface 2, binds that surface to handle `0x15f90`, scales it to 14% into created +112x84 surface 192, then asks opcode `0x1ae` to serialize surface 192. Both render passes use `0x20d` +target selection, `0x20e` clear, and `0x222` range publication. Dungeon map construction uses the same +offscreen path. Godot had modeled selection and black allocation, but `PresentObjectRange` only requested +an onscreen recomposition, so neither target ever received pixels. + +Selected-target `0x222` and `0x20c` now publish through a platform-neutral retained-surface rasterizer. +It filters the requested handle range and reproduces affine/range transforms, scaling, tint, opacity, +blend mode, color keys, created/source surfaces, and black clears before storing the target pixels. +Backbuffer publication is unchanged. A focused regression reproduces SAVE.BIN's scaled texture capture +and proves objects outside the range cannot leak into the target. The original black SAVE01 thumbnail is +a preserved diagnostic artifact; a newly overwritten save is required for visual acceptance of the +fixed `.STH`. The subsequent immediate-resave test showed that a separate numbered-state round-trip defect +still prevented the rewritten `.DAT` from resuming interactively. + +### Persistence implementation step 11 — repair port-authored numbered-state round trips (2026-07-24) + +The clean comparison target was `SAVE03`: load native slot 0 and save immediately, without entering a +dungeon. Loading that rewrite was completely black, proving the remaining failure was not dungeon-map +construction. The source slot has two saved frames; the rewrite had four and incorrectly included the +currently open save-menu/helper frames. Full `0xae` restoration had not re-established the terminal saved +frame as opcode `0x1ad`'s ongoing save cutoff, so the later `0x19e` fell back to the deepest active helper. +The terminal restore now retains its `ExecFrame` as `_saveResumeFrame`. A regression reconstructs a +two-frame save, invokes `0x19e` from a nested helper, and proves the new slot still contains only the two +gameplay frames. + +Binary comparison also corrected the retained-gfx writer. Native records are not tightly packed: +each handle plus meaningful `0x2d4`-byte record begins on a `0x2d5`-DWORD / `0xb54`-byte stride. The port +now emits that sparse layout and imports its earlier packed experimental files. New records start from +the exact `gfx_object_init_default` identity-matrix/color state; decoded native records retain an opaque +byte template so unnamed fields survive rewrites; and translation is encoded at row-vector matrix +entries 12..14 rather than over the matrix diagonal. Focused tests cover the native stride, all six +identity matrices, translation placement, opaque-byte preservation, and the restored save boundary. + +The `/v2` serializer/deserializer annotations and canonical format notes are updated. The next acceptance +action is to rebuild, load native slot 0, immediately overwrite a fresh slot, and load that fresh rewrite; +older slot 1/2/3 files remain useful compatibility inputs but retain the already-written bad frame chain. + +Validation: all 391 engine tests pass; opcode lint/tooling tests are clean; the Godot C# build has zero +warnings; the 211-object SAVE03 compatibility rewrite decodes with the native stride; and threaded +headless execution reports `SELFTEST OK`. + +### Persistence implementation step 12 — preserve restored ancestor continuations (2026-07-24) + +Fresh slot 004 passed the immediate load gate, but selecting a stage played the Eushully intro. Its frame +count was fixed at two, yet a decoded comparison found the decisive remaining difference: +native slot 000's SYSTEM4 parent stores `resume=8, call=8`, while slot 004 stored `-1,-1`. + +Native `0xae` restores each parent context to its real T1/T2/T3 coordinates before activating the next +saved child. The managed reconstruction instead recursively runs FORT while its SYSTEM4 `ExecFrame` is +physically parked at the synthetic `0xae` instruction. Re-save capture derived coordinates from that +synthetic PC, found neither a T1 nor T2 table entry, and wrote `-1`. FORT could initially run, but the +later stage-launch unwind had no SYSTEM4 T2 continuation and fell into ordinary boot. + +Restored frames now shadow their original serialized coordinates while a restored descendant remains +active. Re-saves use those T1/T2/T3 values; T3 rebuilds the live local-return stack; T1 initializes the +read-message coordinate; and the shadow clears only when the child genuinely returns to its parent. +Root-reload and process-exit results now propagate through this synthetic recursion as they do through +ordinary `call-script`. The existing two-frame load/re-save regression now asserts the ancestor's resume +and call indices and loads the rewrite again, rejecting any ordinary re-entry into the restored child. + +The `0xae` opcode source/reference, engine RE, roadmap, status memory, and `/v2` plate comment record the +refinement. Existing slot 004 retains its already-written `-1,-1`; acceptance requires another fresh +slot created from native slot 000, followed by load and stage launch. + +Validation: engine **392/392**, clean opcode lint/tooling, zero-warning Godot build, and threaded +`SELFTEST OK`. The remaining gate is interactive confirmation with a newly authored slot; the fix cannot +retroactively repair slot 004's serialized parent coordinates. + +### Persistence implementation step 13 — run restored script prologues before `0xae` (2026-07-24) + +Manual slot 005 acceptance passed both immediate base restoration and later stage launch. The next +dungeon-authored rewrite, slot 006, loaded with its UI present but a black, noninteractive map. Binary +and software-renderer inspection showed that the slot itself was healthy: it contained FIELD's saved +range, 704 retained objects, map textures, and a native-looking 0.8 transform. During restore, however, +FIELD overwrote the map objects with zero scale. Global zoom selector `G[0x7682]` was 3, while the +frame-local zoom table entry that should have returned 80 was zero. + +Native `script_frame_restore_saved_layout@0x40f2d0` calls +`script_frame_load_resource@0x40e980`, which creates each saved script frame at its ordinary codebase. +The script runs its entry prologue and reaches `0xae` itself. The managed restore instead created frames +directly at `0xae`, skipping FIELD's inline initialization of `[40, 50, 64, 80, 100, 124, 156]`. + +Restored frames now start at script offset zero while retaining the serialized T1/T2/T3 coordinates +consumed at `0xae`. Synthetic root and child tests require their pre-rendezvous prologues to execute. +The unchanged real slot 006 then replays with zoom 80, 0.8 object scale, a nonblack dungeon raster, and +the gameplay poll; the fix does not require rewriting that save. + +The adjacent surface declaration gap is also closed. Native opcode `0x258` sets surface-record `+0x08` +from flags bit 0 and `+0x0c` from bit 1 after `0x259` clears both tables. The port now applies bit 0 as +the numbered-load reload policy; bit 1 remains identified but has no known consumer. + +Validation: engine **393/393**, clean opcode lint/tooling, zero-warning Godot build, threaded +`SELFTEST OK`, and exact slot-006 software replay with zoom 80 and a nonblack dungeon raster. + +Acceptance: rebuild and interactively load existing slot 006. The expected result is the restored +dungeon map and controls without a re-save. + ## Data-semantics sidebar: focused append EBINIT inspection (2026-07-24) The static INIT surface now accepts a universal packed script id for focused append inspection. diff --git a/docs/remake-architecture-and-roadmap.md b/docs/remake-architecture-and-roadmap.md index 7bd21b8..57eb869 100644 --- a/docs/remake-architecture-and-roadmap.md +++ b/docs/remake-architecture-and-roadmap.md @@ -480,7 +480,19 @@ serialized mutable bank prefixes (preserving initialized unit/stage/string defin retained BGM/SFX state, preserves initialized flag-zero system surfaces while overlaying explicit saved reload records, reproduces the native configuration-gated all-surface release when explicitly enabled, and writes native per-slot reload/created metadata rather than treating every texture as reloadable. -Opcode `0x259` supplies its real script-entry reload-policy clear. JSON inspection/export, +Opcode `0x259` supplies its real script-entry reload-policy clear. Selected offscreen render targets now +receive actual retained-object pixels, closing the black-thumbnail half of the first port-authored-save +failure. The following immediate-resave oracle exposed and fixed two separate numbered-state defects: +`0xae` now re-establishes the saved `0x1ad` gameplay-frame boundary before a later save UI opens, and +retained objects use AGE's sparse stride plus native-initialized matrix/default bytes while preserving +unnamed fields. The next fresh rewrite also proved that reconstructed managed ancestor frames must retain +their original T1/T2/T3 coordinates while parked at synthetic `0xae`; preserving those coordinates fixes +the later SYSTEM4 unwind that otherwise re-entered the Eushully intro. Slot 005 subsequently passed the +base-load and stage-launch round trip. Dungeon-authored slot 006 exposed one further native-ordering +requirement: restored scripts must begin at their ordinary entry, run frame-local prologues, and reach +`0xae` themselves. Matching that order restores FIELD's 80% zoom table entry and produces the dungeon +map from the unchanged slot; interactive slot-006 confirmation is the remaining visual gate. +JSON inspection/export, namespaced mod data, and migrations remain additive extended-mode work rather than 1.0 compatibility requirements. diff --git a/docs/sys4-format-notes.md b/docs/sys4-format-notes.md index f22b5c1..9897cc9 100644 --- a/docs/sys4-format-notes.md +++ b/docs/sys4-format-notes.md @@ -245,13 +245,23 @@ Retained graphics then uses: ```text 0x00 gfx_record_size:u32 # 0x2d4 +0x04 gfx_object_count:u32 -+0x08 repeated { handle:u32, record[0x2d4] } ++0x08 repeated sparse entries: + handle:u32 + record[0x2d4] # 0xb5 meaningful DWORDs + padding[0x87c] + entry stride = 0x2d5 DWORDs = 0xb54 bytes ... range_first:u32 ... range_count:i32 ... range_transform_record[0x2d4] ... native allocation slack ``` +The sparse stride is not an allocator-only artifact. Both native writer and reader hold a DWORD pointer: +after the handle, they copy `0x2d4` **bytes** but advance by `0x2d4` **DWORDs**. Consecutive handles are +therefore `0x2d5` DWORDs apart. Early experimental port saves incorrectly packed +`{handle,record}` contiguously; compatibility import recognizes that layout, while all new output uses +the native sparse form. + The 1,000 surface records preserve the native 20-byte metadata cells verbatim: | Offset | Size | Meaning | @@ -270,9 +280,13 @@ surfaces expected to remain live across the load. The installed file is decisive are zero, while SYSTEM4's reusable choice-frame atlas remains recorded as resource `0x3383` in slot 15. A separate configuration-gated path can release all 1,000 surfaces before this loop, but Himegari's registered `CreateObject=1`, `AutoFreeTex=0` defaults leave it inactive. A retained `0x2d4` -record is structurally complete but not every internal graphics field is semantically named. The native -allocation is larger than the records actually written (`0x2e1 + object_count * 0x2d8` DWORDs in the -graphics sizing term), leaving zero/slack bytes after the meaningful range record. +record is structurally complete but not every internal graphics field is semantically named. Native +`gfx_object_init_default` initializes six embedded 4x4 matrices (current/target scale, rotation, and +translation) to identity, packed colors at `+0x60/+0x64/+0x240` to `0xffffffff`, and other fields to zero. +Deserialization initializes an object and then overwrites all `0x2d4` bytes, so a compatibility writer +must emit or preserve those defaults rather than zero-fill unnamed fields. The graphics sizing term remains +larger than the meaningful sparse entries and range record +(`0x2e1 + object_count * 0x2d8` DWORDs), leaving final allocation slack as well. The installed `SAVE00.DAT` validates the complete layout-3 decode: cutoff 1, global-bank counts `[402459,1,789,1,1,1]`, current BGM id `0x18`, retained SFX ids `0x3321` (channel 1) and diff --git a/engine/Age.Engine.Tests/NativeNumberedSaveCodecTests.cs b/engine/Age.Engine.Tests/NativeNumberedSaveCodecTests.cs index d8aa7ad..e001ab1 100644 --- a/engine/Age.Engine.Tests/NativeNumberedSaveCodecTests.cs +++ b/engine/Age.Engine.Tests/NativeNumberedSaveCodecTests.cs @@ -28,6 +28,8 @@ public class NativeNumberedSaveCodecTests new NativeSavedGfxObject(0xcf08, Enumerable.Range(0, NativeNumberedSaveState.GfxRecordSize) .Select(i => unchecked((byte)i)).ToArray()), + new NativeSavedGfxObject(0xcf09, + Enumerable.Repeat((byte)0xa5, NativeNumberedSaveState.GfxRecordSize).ToArray()), ], RangeTransformFirst = 100, RangeTransformCount = 4, @@ -53,7 +55,16 @@ public class NativeNumberedSaveCodecTests Assert.Equal(state.PointerGlobals, decoded.PointerGlobals); Assert.Equal(state.GfxObjects[0].Handle, decoded.GfxObjects[0].Handle); Assert.Equal(state.GfxObjects[0].Record, decoded.GfxObjects[0].Record); + Assert.Equal(state.GfxObjects[1].Handle, decoded.GfxObjects[1].Handle); + Assert.Equal(state.GfxObjects[1].Record, decoded.GfxObjects[1].Record); + Assert.False(decoded.LegacyTightGfxLayout); Assert.Equal(state.RangeTransformRecord, decoded.RangeTransformRecord); + + const int nativeEntryStride = (1 + NativeNumberedSaveState.GfxRecordSize) * 4; + int objectsAt = FindGfxObjects(encoded, 2, 0xcf08); + Assert.Equal(0xcf08, BinaryPrimitives.ReadInt32LittleEndian(encoded.AsSpan(objectsAt))); + Assert.Equal(0xcf09, BinaryPrimitives.ReadInt32LittleEndian( + encoded.AsSpan(objectsAt + nativeEntryStride))); } [Fact] @@ -153,4 +164,15 @@ public class NativeNumberedSaveCodecTests BinaryPrimitives.ReadInt32LittleEndian( state.SurfaceRecords.AsSpan(slot * 20 + 8)))); } + + private static int FindGfxObjects(byte[] payload, int count, int firstHandle) + { + for (int offset = 0; offset <= payload.Length - 12; offset += 4) + if (BinaryPrimitives.ReadInt32LittleEndian(payload.AsSpan(offset)) == + NativeNumberedSaveState.GfxRecordSize + && BinaryPrimitives.ReadInt32LittleEndian(payload.AsSpan(offset + 4)) == count + && BinaryPrimitives.ReadInt32LittleEndian(payload.AsSpan(offset + 8)) == firstHandle) + return offset + 8; + throw new Xunit.Sdk.XunitException("Encoded gfx table was not found."); + } } diff --git a/engine/Age.Engine.Tests/NumberedSaveVmTests.cs b/engine/Age.Engine.Tests/NumberedSaveVmTests.cs index a2458f8..21640b3 100644 --- a/engine/Age.Engine.Tests/NumberedSaveVmTests.cs +++ b/engine/Age.Engine.Tests/NumberedSaveVmTests.cs @@ -45,6 +45,8 @@ public class NumberedSaveVmTests vm.Gfx.SetSurfaceReloadOnRestore(7, true); vm.Gfx.ReleaseSurfaceRange(7, 1); vm.Gfx.BindDraw(100, 3, 1, 2, 30, 40, 50, 60); + vm.Gfx.SetCurrentTranslation(100, (7, 8, 9)); + vm.Gfx.SetRotationChannel(100, 0, 500, (0, 0, 1), 90); vm.Run(); @@ -58,7 +60,25 @@ public class NumberedSaveVmTests Assert.Equal(0x3321, state.SoundEffectResourceIds[1]); Assert.Equal("姫狩り", state.StringGlobals[4]); Assert.Equal(0x77u, state.Frames.Single().ScriptId); - Assert.Contains(state.GfxObjects, item => item.Handle == 100); + NativeSavedGfxObject savedObject = + Assert.Single(state.GfxObjects, item => item.Handle == 100); + Assert.Equal(0, ReadGfxField(savedObject.Record, 0x34)); + Assert.Equal(-1, ReadGfxField(savedObject.Record, 0x60)); + Assert.Equal(0, ReadGfxField(savedObject.Record, 0x238)); + Assert.Equal(-1, ReadGfxField(savedObject.Record, 0x240)); + foreach (int matrixOffset in new[] { 0x6c, 0xac, 0xec, 0x16c, 0x1ac }) + { + Assert.Equal(BitConverter.SingleToInt32Bits(1), ReadGfxField(savedObject.Record, matrixOffset)); + Assert.Equal(BitConverter.SingleToInt32Bits(1), ReadGfxField(savedObject.Record, matrixOffset + 0x14)); + Assert.Equal(BitConverter.SingleToInt32Bits(1), ReadGfxField(savedObject.Record, matrixOffset + 0x28)); + Assert.Equal(BitConverter.SingleToInt32Bits(1), ReadGfxField(savedObject.Record, matrixOffset + 0x3c)); + } + Assert.Equal(BitConverter.SingleToInt32Bits(1), ReadGfxField(savedObject.Record, 0x168)); + Assert.Equal(BitConverter.SingleToInt32Bits(7), ReadGfxField(savedObject.Record, 0x19c)); + Assert.Equal(BitConverter.SingleToInt32Bits(8), ReadGfxField(savedObject.Record, 0x1a0)); + Assert.Equal(BitConverter.SingleToInt32Bits(9), ReadGfxField(savedObject.Record, 0x1a4)); + Assert.Equal(BitConverter.SingleToInt32Bits(1), ReadGfxField(savedObject.Record, 0x130)); + Assert.Equal(BitConverter.SingleToInt32Bits(-1), ReadGfxField(savedObject.Record, 0x13c)); Assert.Equal(0x1234, ReadSurfaceField(state, 3, 0)); Assert.Equal(0, ReadSurfaceField(state, 3, 8)); Assert.Equal(-1, ReadSurfaceField(state, 4, 0)); @@ -88,14 +108,18 @@ public class NumberedSaveVmTests var store = new DirectoryNativeDatStore(root, Identity); Script resumed = WithTables(WithPackedId(ScriptAssembler.Assemble(Table, "RESUMED.BIN", [ + (Table.ByLabel("mov")!.Value, + [new Operand(GlobalInt, 0x510), new Operand(Immediate, 11)]), (0xae, Array.Empty()), (0x3, [new Operand(Immediate, 0x89)]), (Table.ByLabel("mov")!.Value, [new Operand(GlobalInt, 0x500), new Operand(GlobalInt, 0x123)]), (0x2, Array.Empty()), - ], []), 0x88), scriptCallOffsets: [1]); + ], []), 0x88), scriptCallOffsets: [6]); Script child = WithPackedId(ScriptAssembler.Assemble(Table, "CHILD.BIN", [ + (Table.ByLabel("mov")!.Value, + [new Operand(GlobalInt, 0x511), new Operand(Immediate, 22)]), (0xae, Array.Empty()), (Table.ByLabel("mov")!.Value, [new Operand(GlobalInt, 0x501), new Operand(GlobalInt, 0x123)]), @@ -168,6 +192,8 @@ public class NumberedSaveVmTests Assert.Equal(777, vm.Globals[0x124]); Assert.Equal(456, vm.Globals[0x500]); Assert.Equal(456, vm.Globals[0x501]); + Assert.Equal(11, vm.Globals[0x510]); + Assert.Equal(22, vm.Globals[0x511]); Assert.Equal(1, vm.Globals[0x502]); Assert.Equal("復帰", vm.GlobalStrings[0]); Assert.Equal(0x123, vm.GlobalPointers[0]); @@ -204,6 +230,163 @@ public class NumberedSaveVmTests } } + [Fact] + public void FullLoadReestablishesSaveBoundaryBeforeNestedSaveUiWritesAnotherSlot() + { + string root = NewTemporaryDirectory(); + try + { + var store = new DirectoryNativeDatStore(root, Identity); + int callScript = Table.ByLabel("call-script")!.Value; + Script resumed = WithTables(WithPackedId(ScriptAssembler.Assemble(Table, "RESUMED.BIN", + [ + (0xae, Array.Empty()), + (callScript, [new Operand(Immediate, 0x89)]), + (0x2, Array.Empty()), + ], []), 0x88), scriptCallOffsets: [1]); + Script child = WithPackedId(ScriptAssembler.Assemble(Table, "CHILD.BIN", + [ + (0xae, Array.Empty()), + (callScript, [new Operand(Immediate, 0x91)]), + (0x2, Array.Empty()), + ], []), 0x89); + Script saveUi = WithPackedId(ScriptAssembler.Assemble(Table, "SAVE_UI.BIN", + [ + (0x19e, [new Operand(LocalInt, 0), new Operand(Immediate, 2)]), + (0x2, Array.Empty()), + ], []), 0x91); + Script loader = WithPackedId(ScriptAssembler.Assemble(Table, "LOADER.BIN", + [ + (0x1a1, [new Operand(LocalInt, 0), new Operand(Immediate, 1)]), + (0x2, Array.Empty()), + ], []), 0x99); + byte[] opaqueRecord = NativeGfxRecord(3); + BinaryPrimitives.WriteInt32LittleEndian(opaqueRecord.AsSpan(0x68), 0x12345678); + NativeNumberedSaveState source = NativeNumberedSaveCodec.Empty( + [ + new NativeSavedScriptFrame(-1, 0x88, [], 8, 0), + new NativeSavedScriptFrame(0, 0x89, [], -1, -1), + ]) with + { + GfxObjects = [new NativeSavedGfxObject(100, opaqueRecord)], + }; + store.SaveNumberedFile( + 1, NativeNumberedSaveCodec.Encode(source), [], + NativeSystemTime.FromLocalDateTime(DateTime.Now), 0); + var vm = new VirtualMachine( + loader, Table, new RecordingHost(), + provider: new MapProvider(new() + { + [0x88] = resumed, + [0x89] = child, + [0x91] = saveUi, + }), + nativeDatStore: store); + + vm.Run(); + + NativeNumberedSaveFile? resaved = store.LoadNumberedFile(2); + Assert.NotNull(resaved); + NativeNumberedSaveState state = + NativeNumberedSaveCodec.Decode(resaved!.Document.Payload); + Assert.Equal([0x88u, 0x89u], state.Frames.Select(frame => frame.ScriptId)); + Assert.Equal(8, state.Frames[0].ResumeIndex); + Assert.Equal(0, state.Frames[0].CallTargetIndex); + Assert.Equal( + 0x12345678, + ReadGfxField(Assert.Single(state.GfxObjects).Record, 0x68)); + + Script reloader = WithPackedId(ScriptAssembler.Assemble(Table, "RELOADER.BIN", + [ + (0x1a1, [new Operand(LocalInt, 0), new Operand(Immediate, 2)]), + (0x2, Array.Empty()), + ], []), 0x9a); + var trace = new RecordingTraceSink(); + var roundTrippedVm = new VirtualMachine( + reloader, Table, new RecordingHost(), + provider: new MapProvider(new() + { + [0x88] = resumed, + [0x89] = child, + [0x91] = saveUi, + }), + sink: trace, + nativeDatStore: store); + + roundTrippedVm.Run(); + + Assert.DoesNotContain(trace.Events, item => + item.Kind == Age.Engine.Diagnostics.TraceEventKind.FrameEnter + && item.Name == "CHILD.BIN" + && item.Cause == Age.Engine.Diagnostics.FrameCause.CallScript); + Assert.Equal("exit", roundTrippedVm.HaltReason); + } + finally + { + Directory.Delete(root, recursive: true); + } + } + + [Fact] + public void RestoredChildRootReloadDiscardsSyntheticAncestorChain() + { + string root = NewTemporaryDirectory(); + try + { + var store = new DirectoryNativeDatStore(root, Identity); + int callScript = Table.ByLabel("call-script")!.Value; + int move = Table.ByLabel("mov")!.Value; + Script resumed = WithTables(WithPackedId(ScriptAssembler.Assemble(Table, "RESUMED.BIN", + [ + (0xae, Array.Empty()), + (callScript, [new Operand(Immediate, 0x89)]), + (move, [new Operand(GlobalInt, 0x600), new Operand(Immediate, 1)]), + (0x2, Array.Empty()), + ], []), 0x88), scriptCallOffsets: [1]); + Script child = WithPackedId(ScriptAssembler.Assemble(Table, "CHILD_RELOAD.BIN", + [ + (0xae, Array.Empty()), + (0x9, Array.Empty()), + ], []), 0x89); + Script reloaded = WithPackedId(ScriptAssembler.Assemble(Table, "RELOADED.BIN", + [ + (move, [new Operand(GlobalInt, 0x601), new Operand(Immediate, 1)]), + (0x2, Array.Empty()), + ], []), 0); + Script loader = WithPackedId(ScriptAssembler.Assemble(Table, "LOADER.BIN", + [ + (0x1a1, [new Operand(LocalInt, 0), new Operand(Immediate, 1)]), + (0x2, Array.Empty()), + ], []), 0x99); + NativeNumberedSaveState source = NativeNumberedSaveCodec.Empty( + [ + new NativeSavedScriptFrame(-1, 0x88, [], -1, 0), + new NativeSavedScriptFrame(0, 0x89, [], -1, -1), + ]); + store.SaveNumberedFile( + 1, NativeNumberedSaveCodec.Encode(source), [], + NativeSystemTime.FromLocalDateTime(DateTime.Now), 0); + var vm = new VirtualMachine( + loader, Table, new RecordingHost(), + provider: new MapProvider(new() + { + [0] = reloaded, + [0x88] = resumed, + [0x89] = child, + }), + nativeDatStore: store); + + vm.Run(); + + Assert.False(vm.Globals.ContainsKey(0x600)); + Assert.Equal(1, vm.Globals[0x601]); + } + finally + { + Directory.Delete(root, recursive: true); + } + } + [Fact] public void DataOnlyLoadReleasesAllSurfacesWhenBothNativeSettingsEnableIt() { @@ -272,6 +455,28 @@ public class NumberedSaveVmTests Assert.False(surface.ReloadOnRestore); } + [Fact] + public void SurfacePersistenceDeclarationSetsAndClearsReloadBit() + { + Script script = ScriptAssembler.Assemble(Table, "SURFACE_POLICY.BIN", + [ + (0x258, [new Operand(Immediate, 15), new Operand(Immediate, 1)]), + (0x258, [new Operand(Immediate, 16), new Operand(Immediate, 3)]), + (0x258, [new Operand(Immediate, 15), new Operand(Immediate, 0)]), + (0x2, Array.Empty()), + ], []); + var vm = new VirtualMachine(script, Table, new RecordingHost()); + vm.Gfx.SetSurface(15, 0x3383, 0); + vm.Gfx.SetSurface(16, 0x3384, 0); + + vm.Run(); + + GfxSurfacePersistenceState[] surfaces = + vm.Gfx.CapturePersistenceSnapshot().Surfaces.ToArray(); + Assert.False(Assert.Single(surfaces, item => item.Slot == 15).ReloadOnRestore); + Assert.True(Assert.Single(surfaces, item => item.Slot == 16).ReloadOnRestore); + } + private static Script WithPackedId(Script source, uint packedId) => new() { @@ -342,11 +547,21 @@ public class NumberedSaveVmTests Write(0x24, 50); Write(0x28, 60); Write(0x60, -1); - Write(0x238, 1); - Write(0x23c, 1); + Write(0x64, -1); + foreach (int matrixOffset in new[] { 0x6c, 0xac, 0xec, 0x12c, 0x16c, 0x1ac }) + { + Write(matrixOffset, BitConverter.SingleToInt32Bits(1)); + Write(matrixOffset + 0x14, BitConverter.SingleToInt32Bits(1)); + Write(matrixOffset + 0x28, BitConverter.SingleToInt32Bits(1)); + Write(matrixOffset + 0x3c, BitConverter.SingleToInt32Bits(1)); + } + Write(0x240, -1); return result; } + private static int ReadGfxField(byte[] record, int offset) + => BinaryPrimitives.ReadInt32LittleEndian(record.AsSpan(offset)); + private static string NewTemporaryDirectory() { string path = Path.Combine(Path.GetTempPath(), "age-save-vm-" + Guid.NewGuid().ToString("N")); diff --git a/engine/Age.Engine.Tests/RetainedSurfaceRasterizerTests.cs b/engine/Age.Engine.Tests/RetainedSurfaceRasterizerTests.cs new file mode 100644 index 0000000..48e508f --- /dev/null +++ b/engine/Age.Engine.Tests/RetainedSurfaceRasterizerTests.cs @@ -0,0 +1,41 @@ +using Age.Engine.Model; +using Age.Engine.Sys4; + +public class RetainedSurfaceRasterizerTests +{ + [Fact] + public void CompositeRangePublishesScaledRetainedTextureIntoOffscreenSurface() + { + var gfx = new GfxState(); + gfx.SetSurface(2, 0x1234, -1); + gfx.BindDraw(100, 2, 0, 0, 4, 4, 0, 0); + gfx.SetCurrentScale(100, (50, 50, 100)); + gfx.SetSurface(3, 0x5678, -1); + gfx.BindDraw(200, 3, 0, 0, 2, 2, 0, 0); + + byte[] sourcePixels = new byte[4 * 4 * 4]; + for (int index = 0; index < sourcePixels.Length; index += 4) + { + sourcePixels[index] = 0x44; + sourcePixels[index + 1] = 0x88; + sourcePixels[index + 2] = 0xcc; + sourcePixels[index + 3] = 0xff; + } + var source = new RgbaImage(4, 4, sourcePixels); + var excluded = new RgbaImage(2, 2, Enumerable.Repeat((byte)0xff, 16).ToArray()); + var destination = new RgbaImage(2, 2, new byte[16]); + + int rendered = RetainedSurfaceRasterizer.CompositeRange( + destination, gfx.SnapshotVisibleObjects(), 100, 1, + item => item.Handle == 100 ? source : excluded); + + Assert.Equal(1, rendered); + for (int index = 0; index < destination.Pixels.Length; index += 4) + { + Assert.Equal(0x44, destination.Pixels[index]); + Assert.Equal(0x88, destination.Pixels[index + 1]); + Assert.Equal(0xcc, destination.Pixels[index + 2]); + Assert.Equal(0xff, destination.Pixels[index + 3]); + } + } +} diff --git a/engine/Age.Engine/Model/GfxState.cs b/engine/Age.Engine/Model/GfxState.cs index b2a59a8..40d06be 100644 --- a/engine/Age.Engine/Model/GfxState.cs +++ b/engine/Age.Engine/Model/GfxState.cs @@ -101,6 +101,10 @@ public sealed class GfxState } public sealed class GfxObject { + // The native numbered-save record contains several full transform matrices and reserved fields + // beyond the port's semantic model. Preserve the decoded bytes so a native load/save round trip + // does not erase state that the port has not modeled yet; known fields are patched by the codec. + public byte[]? NativePersistenceRecord; public (long X, long Y, long Z) V18, V24, V16c; public long Field64, Field68, Field6c; public long Color = 0xffffffff; // native gfx_object_init_default obj+0x60: identity packed ARGB @@ -464,6 +468,7 @@ public sealed class GfxState private static GfxObject CloneState(GfxObject s) => new() { + NativePersistenceRecord = s.NativePersistenceRecord?.ToArray(), V18 = s.V18, V24 = s.V24, V16c = s.V16c, Field64 = s.Field64, Field68 = s.Field68, Field6c = s.Field6c, Color = s.Color, HasColor = s.HasColor, StaticColorMode = s.StaticColorMode, diff --git a/engine/Age.Engine/Persistence/NativeGfxPersistenceCodec.cs b/engine/Age.Engine/Persistence/NativeGfxPersistenceCodec.cs index bb8c1b8..82926ff 100644 --- a/engine/Age.Engine/Persistence/NativeGfxPersistenceCodec.cs +++ b/engine/Age.Engine/Persistence/NativeGfxPersistenceCodec.cs @@ -55,18 +55,27 @@ internal static class NativeGfxPersistenceCodec reload)); } var objects = state.GfxObjects - .Select(item => (item.Handle, DecodeObject(item.Record))) + .Select(item => (item.Handle, DecodeObject(item.Record, state.LegacyTightGfxLayout))) .ToArray(); return new GfxPersistenceSnapshot( surfaces, objects, state.RangeTransformFirst, state.RangeTransformCount, - DecodeObject(state.RangeTransformRecord)); + DecodeObject(state.RangeTransformRecord, state.LegacyTightGfxLayout)); } private static byte[] EncodeObject(GfxState.GfxObject value) { - byte[] raw = new byte[NativeNumberedSaveState.GfxRecordSize]; - int flags = value.Visible ? 1 : 0; - if (value.RotationEnabled || value.SrcAnim || value.ColorAnim) flags |= 4; + byte[] raw = value.NativePersistenceRecord is { Length: NativeNumberedSaveState.GfxRecordSize } + ? value.NativePersistenceRecord.ToArray() + : CreateDefaultObjectRecord(); + int flags = ReadInt(raw, 0); + flags = value.Visible ? flags | 1 : flags & ~1; + flags = value.OneShotColorEnabled || value.ScaleEnabled + || value.RotationChannelEnabled || value.TranslationEnabled + ? flags | 2 + : flags & ~2; + flags = value.RotationEnabled || value.SrcAnim || value.ColorAnim + ? flags | 4 + : flags & ~4; WriteInt(raw, 0, flags); WriteInt(raw, 4, value.SourceSlot); WriteInt(raw, 8, value.SrcRect.X); @@ -76,7 +85,7 @@ internal static class NativeGfxPersistenceCodec WriteVector(raw, 0x18, value.V18); WriteVector(raw, 0x24, value.V24); WriteInt(raw, 0x30, unchecked((int)value.StaticColorMode)); - WriteInt(raw, 0x34, unchecked((int)value.OneShotStartMs)); + WriteInt(raw, 0x34, EncodeNativeStart(value.OneShotStartMs)); WriteInt(raw, 0x38, unchecked((int)value.ColorDelayMs)); WriteInt(raw, 0x3c, unchecked((int)value.ScaleDelayMs)); WriteInt(raw, 0x40, unchecked((int)value.RotationDelayMs)); @@ -89,27 +98,34 @@ internal static class NativeGfxPersistenceCodec WriteInt(raw, 0x64, unchecked((int)value.OneShotColorTarget)); WriteScaleMatrix(raw, 0x6c, value.ScaleCurrent); WriteScaleMatrix(raw, 0xac, value.ScaleTarget); - WriteFloat(raw, 0x16c, value.TranslationCurrent.X); - WriteFloat(raw, 0x170, value.TranslationCurrent.Y); - WriteFloat(raw, 0x174, value.TranslationCurrent.Z); - WriteFloat(raw, 0x1ac, value.TranslationTarget.X); - WriteFloat(raw, 0x1b0, value.TranslationTarget.Y); - WriteFloat(raw, 0x1b4, value.TranslationTarget.Z); - WriteInt(raw, 0x20c, unchecked((int)value.ColorStart)); - WriteInt(raw, 0x214, unchecked((int)value.RotationStartMs)); + WriteRotationMatrix(raw, 0xec, value.RotationCurrent); + WriteRotationMatrix(raw, 0x12c, value.RotationTarget); + WriteTranslationMatrix(raw, 0x16c, value.TranslationCurrent); + WriteTranslationMatrix(raw, 0x1ac, value.TranslationTarget); + WriteFloat(raw, 0x1ec, value.RotationCurrent.X); + WriteFloat(raw, 0x1f0, value.RotationCurrent.Y); + WriteFloat(raw, 0x1f4, value.RotationCurrent.Z); + WriteFloat(raw, 0x1f8, value.RotationTarget.X); + WriteFloat(raw, 0x1fc, value.RotationTarget.Y); + WriteFloat(raw, 0x200, value.RotationTarget.Z); + WriteFloat(raw, 0x204, value.RotationCurrent.Angle); + WriteFloat(raw, 0x208, value.RotationTarget.Angle); + WriteInt(raw, 0x20c, EncodeNativeStart(value.ColorStart)); + WriteInt(raw, 0x214, EncodeNativeStart(value.RotationStartMs)); WriteInt(raw, 0x220, unchecked((int)value.ColorPeriod)); WriteInt(raw, 0x228, unchecked((int)value.RotationPeriodMs)); WriteInt(raw, 0x230, unchecked((int)value.SrcPeriod)); WriteInt(raw, 0x234, unchecked((int)value.SrcCell)); - WriteInt(raw, 0x238, unchecked((int)value.SrcFrameCount)); - WriteInt(raw, 0x23c, unchecked((int)value.SrcColumns)); - WriteInt(raw, 0x240, unchecked((int)value.ColorTarget)); + WriteInt(raw, 0x238, value.SrcAnim ? unchecked((int)value.SrcFrameCount) : 0); + WriteInt(raw, 0x23c, value.SrcAnim ? unchecked((int)value.SrcColumns) : 0); + if (value.ColorAnim) + WriteInt(raw, 0x240, unchecked((int)value.ColorTarget)); WriteVector(raw, 0x244, value.RotationAxis); WriteInt(raw, 0x2d0, unchecked((int)value.OneShotAnimationControlFlags)); return raw; } - private static GfxState.GfxObject DecodeObject(ReadOnlySpan raw) + private static GfxState.GfxObject DecodeObject(ReadOnlySpan raw, bool legacyTightLayout) { if (raw.Length != NativeNumberedSaveState.GfxRecordSize) throw new InvalidDataException("Native retained-gfx record has the wrong size."); @@ -118,13 +134,16 @@ internal static class NativeGfxPersistenceCodec int flags = ReadInt(raw, 0); return new GfxState.GfxObject { + NativePersistenceRecord = legacyTightLayout + ? CreateDefaultObjectRecord() + : raw.ToArray(), Visible = (flags & 1) != 0, SourceSlot = ReadInt(raw, 4), SrcRect = (left, top, right - left, bottom - top), V18 = ReadLongVector(raw, 0x18), V24 = ReadLongVector(raw, 0x24), StaticColorMode = ReadInt(raw, 0x30), - OneShotStartMs = ReadInt(raw, 0x34), + OneShotStartMs = DecodeNativeStart(ReadInt(raw, 0x34)), ColorDelayMs = ReadInt(raw, 0x38), ScaleDelayMs = ReadInt(raw, 0x3c), RotationDelayMs = ReadInt(raw, 0x40), @@ -138,11 +157,17 @@ internal static class NativeGfxPersistenceCodec OneShotColorTarget = unchecked((uint)ReadInt(raw, 0x64)), ScaleCurrent = ReadScale(raw, 0x6c), ScaleTarget = ReadScale(raw, 0xac), - TranslationCurrent = ReadDoubleVector(raw, 0x16c), - V16c = ReadLongFloatVector(raw, 0x16c), - TranslationTarget = ReadDoubleVector(raw, 0x1ac), - ColorStart = ReadInt(raw, 0x20c), - RotationStartMs = ReadInt(raw, 0x214), + TranslationCurrent = ReadMatrixTranslation(raw, 0x16c), + V16c = ReadLongMatrixTranslation(raw, 0x16c), + TranslationTarget = ReadMatrixTranslation(raw, 0x1ac), + RotationCurrent = ( + ReadFloat(raw, 0x1ec), ReadFloat(raw, 0x1f0), ReadFloat(raw, 0x1f4), + ReadFloat(raw, 0x204)), + RotationTarget = ( + ReadFloat(raw, 0x1f8), ReadFloat(raw, 0x1fc), ReadFloat(raw, 0x200), + ReadFloat(raw, 0x208)), + ColorStart = DecodeNativeStart(ReadInt(raw, 0x20c)), + RotationStartMs = DecodeNativeStart(ReadInt(raw, 0x214)), ColorPeriod = ReadInt(raw, 0x220), RotationPeriodMs = ReadInt(raw, 0x228), SrcPeriod = ReadInt(raw, 0x230), @@ -170,6 +195,7 @@ internal static class NativeGfxPersistenceCodec private static void WriteScaleMatrix(Span raw, int offset, (double X, double Y, double Z) scale) { + ClearMatrix(raw, offset); WriteFloat(raw, offset, scale.X); WriteFloat(raw, offset + 0x14, scale.Y); WriteFloat(raw, offset + 0x28, scale.Z); @@ -179,6 +205,40 @@ internal static class NativeGfxPersistenceCodec private static (double X, double Y, double Z) ReadScale(ReadOnlySpan raw, int offset) => (ReadFloat(raw, offset), ReadFloat(raw, offset + 0x14), ReadFloat(raw, offset + 0x28)); + private static void WriteRotationMatrix( + Span raw, int offset, (double X, double Y, double Z, double Angle) rotation) + { + ClearMatrix(raw, offset); + double length = Math.Sqrt( + rotation.X * rotation.X + rotation.Y * rotation.Y + rotation.Z * rotation.Z); + if (length <= double.Epsilon || rotation.Angle == 0) + { + WriteFloat(raw, offset, 1); + WriteFloat(raw, offset + 0x14, 1); + WriteFloat(raw, offset + 0x28, 1); + WriteFloat(raw, offset + 0x3c, 1); + return; + } + + double x = rotation.X / length; + double y = rotation.Y / length; + double z = rotation.Z / length; + double radians = rotation.Angle * Math.PI / 180.0; + double cosine = Math.Cos(radians); + double sine = Math.Sin(radians); + double complement = 1.0 - cosine; + WriteFloat(raw, offset, x * x * complement + cosine); + WriteFloat(raw, offset + 0x04, x * y * complement + z * sine); + WriteFloat(raw, offset + 0x08, x * z * complement - y * sine); + WriteFloat(raw, offset + 0x10, x * y * complement - z * sine); + WriteFloat(raw, offset + 0x14, y * y * complement + cosine); + WriteFloat(raw, offset + 0x18, x * sine + y * z * complement); + WriteFloat(raw, offset + 0x20, y * sine + x * z * complement); + WriteFloat(raw, offset + 0x24, y * z * complement - x * sine); + WriteFloat(raw, offset + 0x28, z * z * complement + cosine); + WriteFloat(raw, offset + 0x3c, 1); + } + private static void WriteVector(Span raw, int offset, (long X, long Y, long Z) vector) { WriteInt(raw, offset, unchecked((int)vector.X)); @@ -189,11 +249,55 @@ internal static class NativeGfxPersistenceCodec private static (long X, long Y, long Z) ReadLongVector(ReadOnlySpan raw, int offset) => (ReadInt(raw, offset), ReadInt(raw, offset + 4), ReadInt(raw, offset + 8)); - private static (long X, long Y, long Z) ReadLongFloatVector(ReadOnlySpan raw, int offset) - => ((long)ReadFloat(raw, offset), (long)ReadFloat(raw, offset + 4), (long)ReadFloat(raw, offset + 8)); + private static void WriteTranslationMatrix( + Span raw, int offset, (double X, double Y, double Z) translation) + { + ClearMatrix(raw, offset); + WriteFloat(raw, offset, 1); + WriteFloat(raw, offset + 0x14, 1); + WriteFloat(raw, offset + 0x28, 1); + WriteFloat(raw, offset + 0x30, translation.X); + WriteFloat(raw, offset + 0x34, translation.Y); + WriteFloat(raw, offset + 0x38, translation.Z); + WriteFloat(raw, offset + 0x3c, 1); + } - private static (double X, double Y, double Z) ReadDoubleVector(ReadOnlySpan raw, int offset) - => (ReadFloat(raw, offset), ReadFloat(raw, offset + 4), ReadFloat(raw, offset + 8)); + private static (long X, long Y, long Z) ReadLongMatrixTranslation( + ReadOnlySpan raw, int offset) + => ((long)ReadFloat(raw, offset + 0x30), + (long)ReadFloat(raw, offset + 0x34), + (long)ReadFloat(raw, offset + 0x38)); + + private static (double X, double Y, double Z) ReadMatrixTranslation( + ReadOnlySpan raw, int offset) + => (ReadFloat(raw, offset + 0x30), + ReadFloat(raw, offset + 0x34), + ReadFloat(raw, offset + 0x38)); + + private static void ClearMatrix(Span raw, int offset) + => raw.Slice(offset, 0x40).Clear(); + + private static int EncodeNativeStart(long value) + => value < 0 ? 0 : unchecked((int)value); + + private static long DecodeNativeStart(int value) + => value == 0 ? -1 : value; + + private static byte[] CreateDefaultObjectRecord() + { + byte[] raw = new byte[NativeNumberedSaveState.GfxRecordSize]; + WriteInt(raw, 0x60, -1); + WriteInt(raw, 0x64, -1); + foreach (int matrixOffset in new[] { 0x6c, 0xac, 0xec, 0x12c, 0x16c, 0x1ac }) + { + WriteFloat(raw, matrixOffset, 1); + WriteFloat(raw, matrixOffset + 0x14, 1); + WriteFloat(raw, matrixOffset + 0x28, 1); + WriteFloat(raw, matrixOffset + 0x3c, 1); + } + WriteInt(raw, 0x240, -1); + return raw; + } private static void WriteFloat(Span raw, int offset, double value) => WriteInt(raw, offset, BitConverter.SingleToInt32Bits((float)value)); diff --git a/engine/Age.Engine/Persistence/NativeNumberedSaveCodec.cs b/engine/Age.Engine/Persistence/NativeNumberedSaveCodec.cs index 67e674f..e443827 100644 --- a/engine/Age.Engine/Persistence/NativeNumberedSaveCodec.cs +++ b/engine/Age.Engine/Persistence/NativeNumberedSaveCodec.cs @@ -28,7 +28,8 @@ public sealed record NativeNumberedSaveState( IReadOnlyList GfxObjects, long RangeTransformFirst, int RangeTransformCount, - byte[] RangeTransformRecord) + byte[] RangeTransformRecord, + bool LegacyTightGfxLayout = false) { public const int SoundEffectChannelCount = 10; public const int ResourceRecordsSize = 300 * 4; @@ -45,6 +46,9 @@ public static class NativeNumberedSaveCodec private const int FrameReturnCapacity = 256; private const int GfxAllocationDwords = 0x2d8; private const int GfxConstantDwords = 0x2e1; + // AGE stores a byte-sized 0x2d4 object record at the front of a 0x2d4-DWORD region. + // The handle consumes one preceding DWORD, so consecutive entries start 0x2d5 DWORDs apart. + private const int GfxEntryStrideBytes = (1 + NativeNumberedSaveState.GfxRecordSize) * 4; private static readonly Encoding NativeEncoding = CreateNativeEncoding(); public static byte[] Encode(NativeNumberedSaveState state) @@ -103,7 +107,7 @@ public static class NativeNumberedSaveCodec { WriteInt(payload, at, unchecked((int)gfx.Handle)); gfx.Record.CopyTo(payload, at + 4); - at += 4 + NativeNumberedSaveState.GfxRecordSize; + at += GfxEntryStrideBytes; } WriteInt(payload, at, unchecked((int)state.RangeTransformFirst)); WriteInt(payload, at + 4, state.RangeTransformCount); @@ -154,14 +158,19 @@ public static class NativeNumberedSaveCodec if (gfxRecordSize != NativeNumberedSaveState.GfxRecordSize) throw new InvalidDataException($"Unsupported native gfx record size 0x{gfxRecordSize:x}."); at += 8; - var objects = new NativeSavedGfxObject[gfxCount]; - for (int i = 0; i < objects.Length; i++) + int objectsAt = at; + (NativeSavedGfxObject[] objects, int afterObjects) = + ReadGfxObjects(payload, objectsAt, gfxCount, GfxEntryStrideBytes); + bool legacyTightGfxLayout = false; + // Compatibility for experimental files written by the port before the native DWORD stride was + // understood. A native retained-object map cannot contain duplicate handles. + if (objects.Select(item => item.Handle).Distinct().Count() != objects.Length) { - Require(payload, at, 4 + gfxRecordSize, "numbered-save gfx object"); - long handle = ReadInt(payload, at); - objects[i] = new NativeSavedGfxObject(handle, payload.Slice(at + 4, gfxRecordSize).ToArray()); - at += 4 + gfxRecordSize; + (objects, afterObjects) = + ReadGfxObjects(payload, objectsAt, gfxCount, 4 + gfxRecordSize); + legacyTightGfxLayout = true; } + at = afterObjects; Require(payload, at, 8 + gfxRecordSize, "numbered-save range transform"); long rangeFirst = ReadInt(payload, at); int rangeCount = ReadInt(payload, at + 4); @@ -170,7 +179,25 @@ public static class NativeNumberedSaveCodec return new NativeNumberedSaveState( ReadInt(payload, 4), ReadInt(payload, 8), soundEffects, resources, surfaces, frames, integers, floats, strings, pointers, pointerStrings, localPointerScratch, objects, - rangeFirst, rangeCount, rangeRecord); + rangeFirst, rangeCount, rangeRecord, legacyTightGfxLayout); + } + + private static (NativeSavedGfxObject[] Objects, int After) ReadGfxObjects( + ReadOnlySpan payload, int offset, int count, int strideBytes) + { + var objects = new NativeSavedGfxObject[count]; + int at = offset; + for (int i = 0; i < objects.Length; i++) + { + Require(payload, at, 4 + NativeNumberedSaveState.GfxRecordSize, + "numbered-save gfx object"); + long handle = ReadInt(payload, at); + objects[i] = new NativeSavedGfxObject( + handle, + payload.Slice(at + 4, NativeNumberedSaveState.GfxRecordSize).ToArray()); + at = checked(at + strideBytes); + } + return (objects, at); } public static NativeNumberedSaveState Empty(IReadOnlyList frames) diff --git a/engine/Age.Engine/Sys4/RetainedSurfaceRasterizer.cs b/engine/Age.Engine/Sys4/RetainedSurfaceRasterizer.cs new file mode 100644 index 0000000..fe2fba1 --- /dev/null +++ b/engine/Age.Engine/Sys4/RetainedSurfaceRasterizer.cs @@ -0,0 +1,65 @@ +using Age.Engine.Model; + +namespace Age.Engine.Sys4; + +/// Platform-neutral software publication of retained gfx objects into an AGE surface. +/// Native scripts use the same object list for the backbuffer and selected offscreen render targets. +public static class RetainedSurfaceRasterizer +{ + public static int CompositeRange( + RgbaImage destination, + IReadOnlyList visible, + long firstHandle, + long count, + Func resolveSource) + { + ArgumentNullException.ThrowIfNull(destination); + ArgumentNullException.ThrowIfNull(visible); + ArgumentNullException.ThrowIfNull(resolveSource); + if (destination.Width <= 0 || destination.Height <= 0 + || destination.Pixels.Length != checked(destination.Width * destination.Height * 4) + || count <= 0) + return 0; + + int rendered = 0; + foreach (RenderObject item in visible) + { + if (item.Handle < firstHandle || item.Handle - firstHandle >= count) continue; + + TransformState transform = item.Transform; + Affine2D localToDestination = + Transform2DMath.Build(transform, item.Rotation).FromLocalOrigin(item.DstX, item.DstY); + if (item.RangeTransform is { } rangeTransform) + localToDestination = localToDestination.Then(rangeTransform); + + RgbaImage? source = resolveSource(item); + float opacity = item.Alpha / 255f; + float tintStrength = item.TintStrength / 255f; + if (source == null) + { + if (item.SurfaceResId != 0 || item.Blend == BlendKind.Opaque) continue; + int width = item.W > 0 ? item.W : 800; + int height = item.H > 0 ? item.H : 600; + float fillOpacity = item.MultiplyTint ? opacity : opacity * tintStrength; + SoftwareAffineRasterizer.FillRgba( + destination.Pixels, destination.Width, destination.Height, + width, height, localToDestination, item.Tint, fillOpacity); + rendered++; + continue; + } + + if (item.W <= 0 || item.H <= 0 || item.SrcX < 0 || item.SrcY < 0) continue; + int widthToDraw = System.Math.Min(item.W, source.Width - item.SrcX); + int heightToDraw = System.Math.Min(item.H, source.Height - item.SrcY); + if (widthToDraw <= 0 || heightToDraw <= 0) continue; + SoftwareAffineRasterizer.BlitRgba( + destination.Pixels, destination.Width, destination.Height, + source.Pixels, source.Width, source.Height, + item.SrcX, item.SrcY, widthToDraw, heightToDraw, + localToDestination, item.Tint, tintStrength, opacity, + item.MultiplyTint, item.Blend); + rendered++; + } + return rendered; + } +} diff --git a/engine/Age.Engine/Vm/ExecFrame.cs b/engine/Age.Engine/Vm/ExecFrame.cs index c0e40a1..c0bdf26 100644 --- a/engine/Age.Engine/Vm/ExecFrame.cs +++ b/engine/Age.Engine/Vm/ExecFrame.cs @@ -1,4 +1,5 @@ using Age.Engine.Model; +using Age.Engine.Persistence; namespace Age.Engine.Vm; /// One script activation: the running script, its instruction cursor, its local slots, @@ -10,6 +11,10 @@ internal sealed class ExecFrame public readonly Script Script; public int Pc; // entry instruction index + // While a restored descendant is running, this activation is parked at the synthetic op-0xae + // rendezvous rather than its native T1/T2/T3 coordinate. Retain the serialized coordinates until + // the descendant returns so an intervening save can reproduce the native parent frame. + public NativeSavedScriptFrame? RestoredSaveFrame; public int ReadMessageOffset = -1; // latest op-0x71 code DWORD coordinate public readonly Frame Locals = new(); public readonly List CallStack = new(); // intra-script `call` (op 0x8f) returns diff --git a/engine/Age.Engine/Vm/VirtualMachine.cs b/engine/Age.Engine/Vm/VirtualMachine.cs index 42cbf4a..ee61b91 100644 --- a/engine/Age.Engine/Vm/VirtualMachine.cs +++ b/engine/Age.Engine/Vm/VirtualMachine.cs @@ -595,12 +595,17 @@ public sealed class VirtualMachine Script root = _s; int rootEntry = root.IndexByOffset.TryGetValue(entryOffset, out var idx) ? idx : 0; FrameCause cause = FrameCause.TopScene; + NativeSavedScriptFrame? restoredRootFrame = null; while (true) { FrameOutcome outcome; try { - outcome = RunFrame(new ExecFrame(root, rootEntry), cause); + ExecFrame rootFrame = restoredRootFrame == null + ? new ExecFrame(root, rootEntry) + : CreateRestoredFrame(root, restoredRootFrame); + restoredRootFrame = null; + outcome = RunFrame(rootFrame, cause); } catch (NumberedRestoreRequestedException) { @@ -618,8 +623,9 @@ public sealed class VirtualMachine } } root = ResolveSavedScript(_loadedNumberedState.Frames[0]); - rootEntry = FindRestoreRendezvous(root); + rootEntry = 0; _restoreFrameIndex = 0; + restoredRootFrame = _loadedNumberedState.Frames[0]; cause = FrameCause.SaveRestore; continue; } @@ -800,19 +806,26 @@ public sealed class VirtualMachine for (int i = 0; i <= cutoff; i++) { ExecFrame frame = active[i]; - int[] returns = frame.CallStack - .Where(returnPc => (uint)returnPc < (uint)frame.Script.Instructions.Count) - .Select(returnPc => - { - int returnOffset = frame.Script.Instructions[returnPc].Offset; - return FindTableIndex(frame.Script.LocalCallOffsets, returnOffset - 3); - }) - .Where(index => index >= 0) - .ToArray(); - int resumeIndex = CurrentReadMessageIndex(frame); - int callTargetIndex = i == cutoff || (uint)frame.Pc >= (uint)frame.Script.Instructions.Count + NativeSavedScriptFrame? restored = frame.RestoredSaveFrame; + int[] returns = restored?.ReturnIndices.ToArray() + ?? frame.CallStack + .Where(returnPc => (uint)returnPc < (uint)frame.Script.Instructions.Count) + .Select(returnPc => + { + int returnOffset = frame.Script.Instructions[returnPc].Offset; + return FindTableIndex(frame.Script.LocalCallOffsets, returnOffset - 3); + }) + .Where(index => index >= 0) + .ToArray(); + int resumeIndex = restored?.ResumeIndex ?? CurrentReadMessageIndex(frame); + int callTargetIndex = i == cutoff ? -1 - : FindTableIndex(frame.Script.ScriptCallOffsets, frame.Script.Instructions[frame.Pc].Offset); + : restored?.CallTargetIndex + ?? ((uint)frame.Pc >= (uint)frame.Script.Instructions.Count + ? -1 + : FindTableIndex( + frame.Script.ScriptCallOffsets, + frame.Script.Instructions[frame.Pc].Offset)); frames[i] = new NativeSavedScriptFrame( i - 1, frame.Script.PackedId, returns, resumeIndex, callTargetIndex); } @@ -941,6 +954,29 @@ public sealed class VirtualMachine $"Saved script {script.Name} has no opcode 0xae restore rendezvous."); } + private static ExecFrame CreateRestoredFrame(Script script, NativeSavedScriptFrame saved) + { + // Native creates each saved script context at its ordinary entrypoint. The script runs its + // local-array/constants/resource prologue and rendezvouses at 0xae itself; jumping directly + // to 0xae leaves those frame-local tables zeroed (FIELD then collapses its map zoom to 0%). + _ = FindRestoreRendezvous(script); + int entry = script.IndexByOffset.TryGetValue(0, out int index) ? index : 0; + var frame = new ExecFrame(script, entry) + { + RestoredSaveFrame = saved, + }; + if ((uint)saved.ResumeIndex < (uint)script.ReadMessageOffsets.Count) + frame.ReadMessageOffset = script.ReadMessageOffsets[saved.ResumeIndex]; + foreach (int returnIndex in saved.ReturnIndices) + { + if ((uint)returnIndex >= (uint)script.LocalCallOffsets.Count) continue; + int returnOffset = checked(script.LocalCallOffsets[returnIndex] + 3); + if (script.IndexByOffset.TryGetValue(returnOffset, out int returnPc)) + frame.CallStack.Add(returnPc); + } + return frame; + } + private static int ResolveTableOffset( Script script, IReadOnlyList table, int index, int fallback) { @@ -1039,6 +1075,9 @@ public sealed class VirtualMachine { case "script-entry": Gfx.ClearSurfaceReloadPolicies(); return pc + 1; + case "set-surface-persistence-flags": // 0x258 (slot)(flags): bit 0 = numbered-load reload + Gfx.SetSurfaceReloadOnRestore(unchecked((int)Read(a[0])), (Read(a[1]) & 1) != 0); + return pc + 1; case "add": Write(a[0], Read(a[1]) + Read(a[2])); return pc + 1; case "sub": Write(a[0], Read(a[1]) - Read(a[2])); return pc + 1; case "mul": Write(a[0], Read(a[1]) * Read(a[2])); return pc + 1; @@ -1139,6 +1178,11 @@ public sealed class VirtualMachine bool terminal = _restoreFrameIndex == _loadedNumberedState.Frames.Count - 1; if (terminal) { + // Native restores the saved top frame as the numbered-save boundary selected by + // opcode 0x1ad. Re-establish that identity so a subsequent save excludes transient + // SAVE/menu helper frames instead of serializing the currently open modal stack. + lock (_debugControlLock) _saveResumeFrame = _cur; + _cur.RestoredSaveFrame = null; _loadedNumberedState = null; _restoreFrameIndex = -1; return ResolveTableOffset(_cur.Script, _cur.Script.ReadMessageOffsets, saved.ResumeIndex, pc + 1); @@ -1149,11 +1193,14 @@ public sealed class VirtualMachine Script child = ResolveSavedScript(childSaved); _restoreFrameIndex = parentIndex + 1; FrameOutcome childOutcome = RunFrame( - new ExecFrame(child, FindRestoreRendezvous(child)), FrameCause.SaveRestore, + CreateRestoredFrame(child, childSaved), FrameCause.SaveRestore, childSaved.ScriptId); _restoreFrameIndex = parentIndex; - if (childOutcome is FrameOutcome.Halted or FrameOutcome.ExitRequested) - return HALT; + if (childOutcome == FrameOutcome.Halted) return HALT; + if (childOutcome == FrameOutcome.RootReload) return ROOT_RELOAD; + if (childOutcome == FrameOutcome.ExitRequested) + throw new ProcessExitRequestedException(); + _cur.RestoredSaveFrame = null; return ResolveTableOffset( _cur.Script, _cur.Script.ScriptCallOffsets, saved.CallTargetIndex, pc) + 1; } diff --git a/godot/GodotAdvHost.cs b/godot/GodotAdvHost.cs index a394a7e..5119ec0 100644 --- a/godot/GodotAdvHost.cs +++ b/godot/GodotAdvHost.cs @@ -274,6 +274,11 @@ public sealed class GodotAdvHost : IHost public void PresentObjectRange(GfxState gfx, long firstHandle, long count) { + if (gfx.CurrentRenderTargetSlot >= 0) + { + PublishObjectRangeToSurface(gfx, firstHandle, count); + return; + } System.Threading.Interlocked.Exchange(ref _presentRequested, 1); _timeline?.Event("present-object-range", new() { ["first"] = firstHandle, ["count"] = count }); } @@ -573,6 +578,7 @@ public sealed class GodotAdvHost : IHost { int slot = gfx.CurrentRenderTargetSlot; var snapshot = gfx.SnapshotVisibleObjects(_clock.NowMs); + PublishObjectRangeToSurface(gfx, 0, long.MaxValue, snapshot); lock (_screenTransitionLock) _renderTargetSnapshots[slot] = snapshot; _timeline?.Event("render-target-snapshot", new() { @@ -1110,10 +1116,67 @@ public sealed class GodotAdvHost : IHost // For an offscreen target, discard separately retained text draws so its modeled pixel contents // observe the native D3D clear as well. if (surfaceSlot >= 0) + { lock (_textLock) _surfaceText.Remove(surfaceSlot); + lock (_imageLock) + { + if (_surfaceImages.TryGetValue(surfaceSlot, out var image)) + System.Array.Clear(image.Pixels); + else if (_slotDims.TryGetValue(surfaceSlot, out var dimensions) + && dimensions.W > 0 && dimensions.H > 0) + _surfaceImages[surfaceSlot] = new RgbaImage( + dimensions.W, dimensions.H, + new byte[checked(dimensions.W * dimensions.H * 4)]); + } + } _timeline?.Event("render-target-clear", new() { ["surface"] = surfaceSlot }); } + private void PublishObjectRangeToSurface( + GfxState gfx, long firstHandle, long count, + IReadOnlyList? sampled = null) + { + int targetSlot = gfx.CurrentRenderTargetSlot; + if (targetSlot < 0 || !_slotDims.TryGetValue(targetSlot, out var dimensions) + || dimensions.W <= 0 || dimensions.H <= 0) + return; + + RgbaImage destination; + lock (_imageLock) + destination = _surfaceImages.TryGetValue(targetSlot, out var current) + ? new RgbaImage(current.Width, current.Height, (byte[])current.Pixels.Clone()) + : new RgbaImage(dimensions.W, dimensions.H, + new byte[checked(dimensions.W * dimensions.H * 4)]); + + IReadOnlyList visible = sampled ?? gfx.SnapshotVisibleObjects(_clock.NowMs); + int rendered = RetainedSurfaceRasterizer.CompositeRange( + destination, visible, firstHandle, count, + item => + { + var raw = gfx.TryGet(item.Handle); + var resolved = raw != null + ? ResolveSurfaceTexture(raw.SourceSlot, item.SurfaceResId) + : ResolveResIdTexture(item.SurfaceResId); + return resolved == null + ? null + : RgbaSurfaceOps.WithColorKey(resolved.Value.Image, item.ColorKey); + }); + + lock (_imageLock) _surfaceImages[targetSlot] = destination; + IReadOnlyList retained = visible + .Where(item => item.Handle >= firstHandle && item.Handle - firstHandle < count) + .ToArray(); + lock (_screenTransitionLock) _renderTargetSnapshots[targetSlot] = retained; + _timeline?.Event("render-target-publish", new() + { + ["surface"] = targetSlot, + ["first"] = firstHandle, + ["count"] = count, + ["objects"] = retained.Count, + ["rendered"] = rendered, + }); + } + public void ReleaseSurfaceRange(int firstSlot, int count) { IReadOnlyList stoppedMovies = _movieSurfaces.ReleaseRange(firstSlot, count); diff --git a/tools/age_opcodes_himegari.py b/tools/age_opcodes_himegari.py index c74477c..0d37901 100644 --- a/tools/age_opcodes_himegari.py +++ b/tools/age_opcodes_himegari.py @@ -76,6 +76,5 @@ INFERRED: dict[int, dict] = { 0x239: dict(name='animate-gfx-srcrect-target', category='draw', noop=False, confidence='high', source='investigation', summary='(handle)(delay_ms)(duration_ms)(frame_count)(column_count)(target_frame) — one-shot row-major source-rectangle cell channel. Worker gfx_worker_set_srcrect_cell @0x47ed90 stores timing at obj+0x48/+0x5c, layout at +0x238/+0x23c, and target at +0x234. C# currently retains the endpoint cell immediately.'), 0x23b: dict(name='draw-decimal-glyphs', category='draw', noop=False, confidence='high', source='investigation', summary='Draw an integer as decimal glyph objects from a style registered by opcode 0x13a.'), 0x23f: dict(name='query-surface-stop-time-ms', category='draw', noop=False, confidence='high', source='investigation', summary='(out_stop_time_ms)(surface_slot) — query the DirectShow stop position retained by a loaded movie surface, convert seconds to integer milliseconds by truncating toward zero, and write -1 when the movie slot is empty. Port-only host decoder failure is modeled as an explicitly completed, zero-duration movie.'), - 0x258: dict(name='decl?', category='marker', noop=True, confidence='low', source='harness', summary='2 imm; runs in a chain right after script-entry 0x259, enumerating ids — prologue declaration/registration?'), 0x2c5: dict(name='byte-string-length', category='compute', noop=False, confidence='high', source='investigation', summary="Write the resolved NUL-terminated engine string's raw byte length."), } diff --git a/vm-map/opcodes.toml b/vm-map/opcodes.toml index 8c7d65a..3f8529e 100644 --- a/vm-map/opcodes.toml +++ b/vm-map/opcodes.toml @@ -1724,12 +1724,12 @@ abi_source = "kelebek+decode-validated" name = "continue-save-load-stack-restore" category = "control" summary = "() - during serialized save restoration, replace the current frame PC with its saved resume/call target and advance through the saved script-context stack; otherwise a no-op." -details = "Layout 3 frame d259 indexes SYS4 T1 read-message reset sites, d260 indexes T2 call-script sites, and the saved local return stack indexes T3 local-call sites. Port status (2026-07-24): the active path reconstructs the saved recursive frame chain and resumes the terminal frame at its T1 boundary. The installed SAVE00 continuation gate proves SYSTEM4 -> FORT restoration reaches FORT's CHMENU gameplay poll; the synthetic gate asserts the child frame enters with SaveRestore rather than ordinary CallScript cause." +details = "Layout 3 frame d259 indexes SYS4 T1 read-message reset sites, d260 indexes T2 call-script sites, and the saved local return stack indexes T3 local-call sites. Port status (2026-07-24): each saved script is loaded at its ordinary entry, runs its frame-local prologue, and reaches 0xae itself; the active path then reconstructs the saved recursive frame chain, preserves each synthetic ancestor's original T1/T2/T3 coordinates while its restored child is active, reinstates the terminal restored frame as opcode 0x1ad's save boundary, and resumes it at T1. The installed SAVE00 continuation gate proves SYSTEM4 -> FORT restoration reaches FORT's CHMENU gameplay poll; the unchanged port-authored dungeon slot 006 proves FIELD's pre-rendezvous zoom table is initialized; synthetic gates assert both root/child prologue execution, SaveRestore rather than ordinary CallScript entry, exclusion of nested SAVE helpers from a re-save, and reload of that rewrite without ordinary boot re-entry." noop_headless = false source = "investigation" confidence = "high" depends_on = [] -evidence = "Ghidra /v2: op_0xae_continue_save_load_stack_restore@0x416790 first tests ctx+0x53d24 (set by save_data_deserialize_and_begin_restore@0x40fd10). When clear it returns. When set, it selects the serialized frame layout through set:SaveVersion1/2, restores the current PC from that layout's saved return/call target, advances through contexts with script_frame_restore_saved_layout@0x40f2d0, and clears the restore flag on reaching the saved terminal context. Its 305 corpus sites overwhelmingly follow coroutine-resume/call boundaries, which provide the rendezvous points used while reconstructing the stack." +evidence = "Ghidra /v2: op_0xae_continue_save_load_stack_restore@0x416790 first tests ctx+0x53d24 (set by save_data_deserialize_and_begin_restore@0x40fd10). When clear it returns. When set, it selects the serialized frame layout through set:SaveVersion1/2, restores the current PC from that layout's saved return/call target, advances through contexts with script_frame_restore_saved_layout@0x40f2d0, and clears the restore flag on reaching the saved terminal context. The restore helper calls script_frame_load_resource@0x40e980, which initializes each new frame PC to its script codebase; the script therefore executes its ordinary prologue before reaching 0xae. Native parent contexts retain restored coordinates while the child runs. SAVE00 -> SAVE03 proved the port must retain the terminal 0x1ad boundary; SAVE00 -> SAVE04 proved a managed parent parked at synthetic 0xae must serialize original resume=8/call=8 rather than synthetic -1/-1; slot 006 proved direct entry at 0xae skips FIELD's zoom-table prologue and collapses the dungeon map. Its 305 corpus sites overwhelmingly follow coroutine-resume/call boundaries." [[opcode]] op = 0xb4 @@ -5742,12 +5742,12 @@ abi_source = "kelebek+decode-validated" [opcode.semantics] name = "present-gfx-object-range" category = "draw" -summary = "(first_handle)(count) - flush/present retained graphics objects in the selected handle range and clear their pending update flags." +summary = "(first_handle)(count) - flush/present retained graphics objects in the selected handle range into the currently selected backbuffer or offscreen render target, then clear their pending update flags." noop_headless = false source = "investigation" confidence = "high" depends_on = [] -evidence = "Ghidra /v2: op_0x222_handler@0x4235e0 calls gfx_present_object_range@0x482230. The worker enters the graphics service, walks the retained-object map, processes flagged objects whose handles fall in [first,first+count), clears pending flags, and finalizes the render batch. HISTORY.BIN uses (0,60000) after rebuilding its retained presentation." +evidence = "Ghidra /v2: op_0x222_handler@0x4235e0 calls gfx_present_object_range@0x482230. The worker enters the graphics service, walks the retained-object map, processes flagged objects whose handles fall in [first,first+count), clears pending flags, and finalizes the render batch in the D3D target previously selected by op 0x20d. HISTORY.BIN uses (0,60000) for the backbuffer. SAVE.BIN instead renders [0,0x130b0) into 800x600 slot 2, then handle 0x15f90 at 14% scale into 112x84 slot 192; op 0x1ae writes slot 192 as the numbered .STH thumbnail." [[opcode.semantics.args]] i = 1 @@ -6709,28 +6709,29 @@ observed_types = ["imm"] [[opcode]] op = 0x258 -label = "u00422FE0" +label = "set-surface-persistence-flags" argc = 2 abi_source = "kelebek+decode-validated" [opcode.semantics] -name = "decl?" -category = "marker" -summary = "2 imm; runs in a chain right after script-entry 0x259, enumerating ids — prologue declaration/registration?" -noop_headless = true -source = "harness" -confidence = "low" +name = "set-surface-persistence-flags" +category = "draw" +summary = "(surface_slot)(flags) - replace the native surface record's numbered-save persistence flags; bit 0 controls asset reload on restore and bit 1 controls the adjacent still-unnamed field." +details = "Scripts place declaration chains immediately after opcode 0x259 clears both fields. The port models bit 0 because it is consumed by layout-3 restoration; bit 1 is retained as an identified native field but has no known runtime consumer yet." +noop_headless = false +source = "investigation" +confidence = "high" depends_on = [] -evidence = "" +evidence = "Ghidra /v2: op_0x258_set_surface_persistence_flags@0x4250a0 reads the slot and flags operands and calls gfx_surface_set_persistence_flags@0x4159c0. The worker writes flags&1 to record +0x08 and (flags>>1)&1 to +0x0c in both 1,000-record tables. This corrects the old upstream address/name association: 0x422fe0 is opcode 0x20f movie playback, not 0x258. Corpus declaration chains follow opcode 0x259 at script entry." [[opcode.semantics.args]] i = 1 -role = "" +role = "surface slot" observed_types = ["imm"] [[opcode.semantics.args]] i = 2 -role = "" +role = "persistence flags" observed_types = ["imm"] [[opcode]]