From f94bca4ba5f880e392f878707f15f7dfb00d9c4c Mon Sep 17 00:00:00 2001 From: gamer147 Date: Fri, 10 Jul 2026 21:56:57 -0400 Subject: [PATCH] Refresh retained presentation references --- docs/asset-resolution-re.md | 11 ++-- docs/engine-re.md | 85 ++++++++++++------------- docs/opcode-reference.md | 4 +- docs/phase-a-slice-plan.md | 38 ++++++----- docs/tools-reference.md | 6 +- engine/Age.Engine/Hosting/FrameClock.cs | 20 +++--- godot/GodotAdvHost.cs | 11 ++-- godot/Main.cs | 6 +- tools/frida/probe_frame_cadence.py | 17 ++--- vm-map/opcodes.toml | 4 +- 10 files changed, 102 insertions(+), 100 deletions(-) diff --git a/docs/asset-resolution-re.md b/docs/asset-resolution-re.md index 9bfd753..8019bf5 100644 --- a/docs/asset-resolution-re.md +++ b/docs/asset-resolution-re.md @@ -88,11 +88,12 @@ highest-risk area of the port. This doc is the steering state; it feeds the A2b (Resolve + BMP path) + `GodotAdvHost` texture ops → `TextureRect` compositing behind the dialogue; `IHost.DrawTexture` extended with dst x/y; 800×600 window; `convert_agf.py --scene` pre-converts a scene's manifest AGFs → BMP. The full-screen **event-CG layer renders end-to-end** from the executed - bytecode. **Limitations (next chunk = graphics geometry/blend):** sprites + `BG*` (routed through the - CG-load subroutine) have garbage geometry because native graphics ops are stubbed (`0x208` - get-texture-size + the sprite position/animation chain); fades (`AE*`) draw opaque (no alpha); slot - model approximates the game's immediate-mode blit-onto-slot-0 canvas. See `docs/phase-a-slice-plan.md` - (A2b section) for the full write-up + the graphics-subsystem plan. + bytecode. **Historical limitations at first landing (subsequently resolved in the Phase-A graphics + slices):** sprites + `BG*` (routed through the CG-load subroutine) had garbage geometry because native + graphics ops were stubbed (`0x208` get-texture-size + the sprite position/animation chain); fades + (`AE*`) drew opaque (no alpha); the slot + model approximated the game's immediate-mode blit-onto-slot-0 canvas. See `docs/phase-a-slice-plan.md` + (A2b section) for the implementation history and current retained-object model. 4. **Audio.** **✅ WIRED (2026-07-06) — no Frida needed.** Same rule as textures: `play-bgm(id)`/`play-voice(id)` → `files[section_base(scene)+id]` → OGG. `IHost.PlayBgm/PlayVoice` + VM dispatch (`play-bgm` 0xbf / `play-voice` 0xc4, both argc 1); `ResourceMap.AudioPath` → loose diff --git a/docs/engine-re.md b/docs/engine-re.md index c710ab6..6a85a1d 100644 --- a/docs/engine-re.md +++ b/docs/engine-re.md @@ -341,8 +341,8 @@ inside the loop (it lives in the native video service) — we approximate it hos (138 corpus scripts; all have the same terminal-check shape), synthesizes `G[0xaba5c]=1` on top-level scene entry, forces one setup-body pass even if `G[0x6be]` is stale, then returns the terminal immediate discovered from the following `mov`/`eq` pair. Thus no SC0000 offset is hardcoded. Op `0x7b` retains the -saved handler PCs as frame metadata; op `0x7c` is a host-scheduler marker because `IHost.FrameYield` and -`FrameClock` already own per-frame pacing. `TITLE.BIN`'s unrelated `"BIN","SC????.BIN"` service remains +saved handler PCs as frame metadata; op `0x7c` is a host-scheduler marker because the host already owns +service-boundary suspension and retained presentation. `TITLE.BIN`'s unrelated `"BIN","SC????.BIN"` service remains stubbed. The real video-service timing remains intentionally unmodeled. **Magic-circle retention fixed in the host model (2026-07-09).** The effect is `AE001H.AGF` @@ -409,7 +409,8 @@ Worker functions decoded + annotated in the Ghidra project (updated 2026-07-09): #### The `0x21c–0x243` sprite transform / ANIMATION cluster (2026-07-10, partial implementation) The scene-completeness tracker (`tools/scene_opcode_coverage.py`) flagged a dense band of GAP ops in -`0x21c–0x243` (+ `0x2bd/0x2bf`) — the **largest remaining rendering unknown** in SC0000 (e.g. `0x220`×66, +`0x21c–0x243` (+ `0x2bd/0x2bf`) — at that point the **largest remaining rendering unknown** in SC0000 +(e.g. `0x220`×66, `0x22f`×34, `0x228`×33, `0x21e`×25 static sites). Resolving every one through the dispatch table (`ctx[0x26c93+op]`, read from `FUN_00413860`) shows it is **one coherent subsystem: sprite transform + animation/tween** — and two members were already named in prior RE (`0x234 gfx_op_0x234_anim_start`, @@ -587,24 +588,22 @@ Handler resolved via the dispatch table (`ctx[0x26c93+0xc8]` = `param_1[0x26d5b] thread for `duration` ms while the main-thread compositor (`Main.Recomposite` in `_Process`) keeps presenting is behaviorally equivalent to the native non-blocking timer. This correctly reproduces the **explicit one-shot sleeps** (the dramatic 1000/750/200 ms holds). Headless/CLI hosts no-op `Sleep` (parity). `IHost.Sleep(long)` + -VM `case "sleep"`; see `vm-map/opcodes.toml` 0xc8. **⚠ It does NOT make the rapid opening `AE*` burst animate** -— execution trace shows the back-to-back `set-texture→draw-texture` swaps have no `sleep`/`wait`/`present`/ -coroutine between them, so bare `sleep` was never their pacer; what advances that burst is still unknown (an -earlier claim that "the opening is sleep-paced" was inherited from this doc and never execution-verified — -corrected). Profile the real Godot run (`--trace-histogram`) to find it. +VM `case "sleep"`; see `vm-map/opcodes.toml` 0xc8. **It does not pace ordinary opcode bursts.** The later +native presentation trace proved that back-to-back retained mutations execute within milliseconds and are +first published at the following `0x20c`/`0x21c`, sleep, or stable input boundary. An earlier claim that the +opening was generally sleep-paced was inherited without execution evidence and is superseded. **Related — `present-frame` (op `0x20c`):** dispatch `param_1[0x26e9f] = gfx_op_0x20c_present_frame` → -`gfx_render_frame`@`0x4820b0` (buffer flip). Our compositor presents every frame regardless, so `0x20c` is a VM -no-op (`noop_headless=true`); the Kelebek label `u00416200` was VA-drift. This corrects the earlier open item -("no per-frame present") above — present is host-implicit; only `sleep` timing was missing. +`gfx_render_frame`@`0x4820b0` (buffer flip). The port treats `0x20c` as an explicit retained-state publication +boundary in the interactive host while keeping it `noop_headless=true`; the Kelebek label `u00416200` was +VA-drift. This supersedes the earlier host-implicit/no-op presentation model. ### Frame cadence — the interpreter tick, and why our port "speeds through" (2026-07-08) -Answers the open question the `sleep` section above left ("what advances the rapid opening burst is still -unknown"). The pace is an **engine-level execution cadence**, not any bytecode primitive. Corroborated in-game -by Ctrl fast-forwarding ADV (a speed governor). Motivated by the user's observation that our port visibly -speeds through the opening — which contradicted, and correctly overturned, an earlier same-day overclaim that -"there is no missing pacer" (that was inferred from headless op-counts, which cannot render). +**Historical reconstruction, corrected by the synchronized presentation trace below.** This pass correctly +identified the one-op interpreter tick and ADV Ctrl state, but incorrectly inferred a constant engine-level +opcode cadence. The outer native service invokes that tick in bursts and publishes retained state only at +specific service boundaries. **Confirmed from the engine image (annotated in Ghidra):** @@ -626,7 +625,8 @@ speeds through the opening — which contradicted, and correctly overturned, an and the anim clock (0x238) advance per frame. A CG transition therefore spreads over many real frames. - **Timing source** = the ms-clock function pointer `*DAT_0056f3d4` (`timeGetTime`-class), used throughout. -**Model:** single-threaded, vsync-timed frame loop; each frame it steps opcodes until the context **yields** +**Model proposed at this stage (partially superseded):** single-threaded frame loop; each service pass steps +opcodes until the context **yields** (`sleep` armed / `wait-for-input` 0x72 / active frame-stepped transition/anim / present), renders (`gfx_render_frame`), waits on the clock, continues. Back-to-back draws inside one page compose into a single frame (fine); the opening's CG-to-CG advances are gated by frame-stepped transitions + sleeps, which spread @@ -641,12 +641,10 @@ statement above is a **reconstruction** from those pieces, not a line read from loop + its exact per-frame step budget / vsync wait needs a **live-debugger break** (attach + break in the frame loop), or a wider memory dump that includes the mode object. -**Port relevance (the speed-through root cause).** Our Godot VM runs on a **free-running background thread** -(`Task.Run(() => vm.Run())` in `Main.cs`) with no frame binding — it executes an entire page's ops in -microseconds; only `WaitForInput` and `Sleep` pause it, and the compositor merely samples `GfxState` at 60fps. -So every no-`sleep` CG/state advance collapses to its end state → the speed-through. **Fix shape:** throttle -the VM to a bounded wall-clock op rate (see the measured numbers below) via a per-opcode host yield; retire the -free-running thread. Spec: `docs/superpowers/specs/2026-07-08-frame-stepped-vm-design.md`. +**Historical port conclusion (superseded):** this pass prescribed a bounded wall-clock opcode rate and led to +the frame-stepped implementation documented in the historical spec. The 2026-07-10 native trace disproved that +rate model: ordinary opcode work must remain burst-fast, while `0x20c`, `0x21c`, sleep, and stable input waits +own publication/pacing. Commit `85fc07d` implements the corrected boundary model. ### Frame cadence — live measurement (2026-07-08, Frida read-only) @@ -657,23 +655,19 @@ way:** a first attempt with a **CModule** hook on the hyper-hot `adv_interpreter instantly (bad native callback into the hottest path — *not* anti-tamper; our other scripts hook engine code via plain JS and survive). Use plain-JS hooks on proven addresses + memory polling. -Findings: -- **Execution is rate-limited, not free-running.** Normal active rate ≈ **1,788 operand-fetches/sec** (peak - ~5,796) — far below an unthrottled interpreter (millions/sec), so the engine paces itself. Execution is - bursty (parked at `wait-for-input` prompts, then a bounded burst), confirming per-iteration op-budgeting. -- **Fast-forward (Ctrl) scales the rate ~4×** (≈7,738/sec avg, peak ~15,572), gated by the engine skip bit - **`ctx+0xa0ce4 & 0x8000000`** (set only while fast-forwarding). It runs *more ops per unit time* — it does - not skip content. (Ctrl is **ADV-scoped**; it does not speed up gameplay/menus.) -- **Rendering = Direct3D 9, UNCAPPED.** `ddraw.dll` is not loaded; the game uses `d3d9.dll` (+ `nvd3dum.dll`). - `IDirect3DDevice9::Present` (device vtable slot 17, found by scanning ctx for a d3d9-vtable object with a - full ~119-method table) fires ~**1,908/sec** with **no vsync**; `BeginScene`/`EndScene` never fire → a **2D - StretchRect-style compositor**, not a 3D scene. So there is **no fixed display-frame rate**; `Present` rate - ≈ op rate (~1 op per present). ⇒ the pacing quantity is the **wall-clock op rate**, not a per-frame budget. -- **Corrected implication for the port (2026-07-10):** 1,788/s is an operand-fetch rate, not an opcode rate. - `IHost.FrameYield` runs once per completed opcode, so matching those quantities directly overclocked the - port by roughly ninefold. The native SC0000 transform lifetime pins the corresponding completed-op cadence - at about **200/s**. The limiter must be wall-clock based (not a per-render callback budget), with a future - ADV Ctrl multiplier applied to the same unified clock. +Findings, corrected by the later synchronized presentation trace: +- Normal active capture measured about **1,788 operand fetches/sec** (peak ~5,796). This is neither a + completed-opcode count nor a scheduler budget; it mixes burst execution with native-service parking. +- **Fast-forward (Ctrl)** raised operand-fetch activity about 4× (≈7,738/sec avg, peak ~15,572), gated by + **`ctx+0xa0ce4 & 0x8000000`**. This remains useful evidence that Ctrl is ADV-scoped, but it does not define + a constant opcode cadence. +- Rendering is Direct3D 9 and uncapped. The candidate D3D9 `Present` hook fired around 1,908/sec in the + original probe; there is no fixed display refresh boundary to copy into the port. +- **Final scheduler implication (2026-07-10):** ordinary opcode work is burst-fast between explicit native + service boundaries. AE001D bind + mode + color-target writes completed within ~5 ms with no + `gfx_render_frame`; rendering began only at `0x21c`. The old 200-completed-op/s calibration folded service + waits into script time and is discarded. Godot `FrameYield` is non-blocking; `0x20c`, `0x21c`, sleep, and + input own publication/pacing. ### The render drift's SECOND half: missing system-boot state (2026-07-07, resolved) @@ -688,10 +682,10 @@ the CGs **de-collapse and render correctly** (screenshot-confirmed). **This is t (a)-vs-(b) debate: the drift needed BOTH the native gfx ops (b) AND boot state (a) — specifically INIT2's handle array, never before identified (it is not a story flag).** Note two distinct boots: our Phase-B `--boot` runs the *data* `*INIT` scripts (skills/items/…); this is the *system* boot (`SYSTEM4` prefix) — a -"full boot" should run both. **Residual:** the `AE*` fade/flash effects still draw opaque (alpha/blend -deferred — Phase 2 scope), and some object-slot CGs start with a zero anchor (cold gfx objects vs the real -game's warm ones; default object geometry is confirmed `(0,0)` in `gfx_object_init_default`, so it is not a -missing-default bug). See `docs/phase-a-slice-plan.md` A2b-Geometry. +"full boot" should run both. **Historical residual (subsequently resolved):** this capture still had opaque +`AE*` fade/flash effects and zero-anchor object-slot CGs. The later blend, geometry, animation, and retained- +presentation subsections below supersede that state; default object geometry remains confirmed `(0,0)` in +`gfx_object_init_default`. ### The gfx animation/effects subsystem — the `AE*` fades (2026-07-07) @@ -971,4 +965,5 @@ includes the scene's own early writes (can exclude by codebase for a pure pre-sc - ~~**gfx command-buffer**~~ — **DONE** (the `0x212–0x21a` positioned-object subsystem = the rendering drift): all 14 ops reversed + implemented against a host-side `GfxState`, and the missing INIT2 boot state supplied via `--boot`. CGs render (screenshot-confirmed). See the op `0x215` finding + "The render - drift's SECOND half" above. Remaining: `AE*` alpha/blend (deferred) and cold-object anchors. + drift's SECOND half" above. The then-remaining `AE*` blend and cold-anchor work is resolved by the later + blend, geometry, animation, and retained-presentation sections. diff --git a/docs/opcode-reference.md b/docs/opcode-reference.md index 38e0ba8..f82ee46 100644 --- a/docs/opcode-reference.md +++ b/docs/opcode-reference.md @@ -68,11 +68,11 @@ This also names the whole call graph statically (build/callscript-names.json). - **evidence:** native-RE (Ghidra): handler FUN_0041fba0 (= ctx[0x26c93+0x8f]) sets [frame PC @+0x53d2c] = [frame codebase @+0x53d28] + operand*4 and pushes ((pc-base)>>2)+3 onto the per-frame return stack ([ctx+0x552e8]/[ctx+0x55248]). Target is a code OFFSET within the current script (matches header table T3 tag 0x8F = local call targets), confirming it is a local JSR, not a script load. ### 0xc8 `sleep` (sleep, argc 1) -- **summary:** Pause the script for milliseconds while rendering continues (frame pacing). +- **summary:** Pause the current script for milliseconds while retained presentation continues. - **grounding:** source=investigation, confidence=high - **evidence:** Ghidra: dispatch ctx[0x26c93+0xc8]=0x420ec0; sleep_op_0xc8 + sleep_timer_arm decoded/annotated 2026-07-08. docs/engine-re.md sleep section. -Native handler sleep_op_0xc8 @0x420ec0 is NON-BLOCKING: it arms a timer (sleep_timer_arm @0x44cff0 at ctx+0x5f304 = active flag + start tick + duration) that the engine main loop polls, resuming the script when elapsed. Operand UNIT = MILLISECONDS (start = ms tick source DAT_0056f3d4, timeGetTime/GetTickCount class). duration<10 fast-paths via [0x56f0b8]; all real scene sleeps (100/750/1000) are >=10. The handler also writes gfx cmd-type 3 + runs anti-tamper checks, neither needed host-side. Port equivalent: the Godot host blocks the VM background thread ms while the per-frame compositor keeps presenting -> correctly reproduces the explicit one-shot dramatic pauses. NOTE: does NOT pace the rapid opening AE* burst (those draws have no sleep between them; their real pacer is unknown). Headless hosts no-op it (parity). +Native handler sleep_op_0xc8 @0x420ec0 is NON-BLOCKING: it arms a timer (sleep_timer_arm @0x44cff0 at ctx+0x5f304 = active flag + start tick + duration) that the engine main loop polls, resuming the script when elapsed. Operand UNIT = MILLISECONDS (start = ms tick source DAT_0056f3d4, timeGetTime/GetTickCount class). duration<10 fast-paths via [0x56f0b8]; all real scene sleeps (100/750/1000) are >=10. The handler also writes gfx cmd-type 3 + runs anti-tamper checks, neither needed host-side. Port equivalent: the Godot host parks the VM thread for duration ms while the presentation compositor continues. Sleep is one proven presentation-capable service boundary; ordinary AE setup runs burst-fast to 0x21c and is not paced per opcode. Headless hosts no-op it (parity). ### 0x140 `coroutine-label-yield` (u0041F9C0, argc 4) - **summary:** (out)(name_str)(sub_str)(in) — scene-coroutine LOOP ITERATOR / labeled yield. Handler copies name/sub strings + the int operand and calls the NATIVE video/transition service (*DAT_005c6018)(8, ctx[0x54fe8], &{name,sub,in}); writes the returned PC-like value to operand 1. In SC0000 label_462 'ループ開始' (@0x46d): `out=G[0x6be]=LABEL('J',G[0x6be])`; loop runs the intro-setup body (incl. call label_125bd = slot-table fill G[0x3239..0x324e]=4..11) and jmps back until out==G[0x6c3] (a per-scene exit-PC immediate) → mov aba5c 0 → content. The gate G[0xaba5c]==1 that opens this loop is NATIVE scene-entry state (no script sets it to 1). DAT_005c6018 is runtime-resolved (all xrefs READ) = SAME class as the DirectDraw workers we don't model. PORT = HOST-MODEL IMPLEMENTED: synthesize the ADV scene-entry gate, run the LABEL/J setup body once, then return the structurally discovered per-scene terminal; do not emulate the video service. See engine-re.md §Scene-coroutine framework. diff --git a/docs/phase-a-slice-plan.md b/docs/phase-a-slice-plan.md index fcc9974..5f6e0a4 100644 --- a/docs/phase-a-slice-plan.md +++ b/docs/phase-a-slice-plan.md @@ -391,6 +391,9 @@ The chunk the animation-subsystem note above flagged as "the clearly-scoped next > pauses, the `GfxState` race fix, and the tooling. Related: the "493k sleeps" that confused me were a **headless > artifact** (the name-entry poll loop), since fixed — see the "Headless divergence" note below. +> **Later resolution (2026-07-10):** the matching native presentation trace below established that ordinary +> mutations run burst-fast and first become visible at `0x20c`, `0x21c`, sleep, or stable input boundaries. + **What `sleep` actually is (RE-confirmed, correct):** the Godot compositor (`Main.Recomposite` in `_Process`) presents live `GfxState` every frame; `sleep` (`0xc8`) was a GAP so the VM ran the whole burst in microseconds. Implementing it makes the **explicit one-shot sleeps** (1000/750/200 ms) pause correctly — but those are the @@ -560,6 +563,11 @@ yield/sync point) instead of the free-running thread. Confirm any fix against ** ### A2b — Frame-stepped VM ✅ DONE & MERGED (2026-07-08) +**Historical implementation, superseded on 2026-07-10 by the native retained-presentation trace below.** +The validation recorded here was accurate for that slice, but the inferred native per-op cadence was not: +service waits had been folded into an operand-fetch average. Commit `85fc07d` removed the production opcode +throttle; ordinary work now runs burst-fast to proven publication/pacing boundaries. + Executed the "Next (open)" list above (option 3: a per-frame yield/sync point). Spec `docs/superpowers/specs/2026-07-08-frame-stepped-vm-design.md`, plan `docs/superpowers/plans/2026-07-08-frame-stepped-vm.md`; merged to `main` (`74a4221`). @@ -665,7 +673,7 @@ video service behind op `0x140`. stale prior-scene `out` value and avoids hardcoding SC0000's `0x45e`; all 138 corpus ADV sites share the same shape. 4. Record op `0x7b`'s two saved handler PCs as frame metadata. Consume op `0x7c` as the host-scheduler - resume marker: the port's existing `IHost.FrameYield`/`FrameClock` path supplies per-frame pacing, so it + resume marker: the host supplies service-boundary suspension and retained presentation, so it does not recursively execute the native render/poll/yield bytecode handlers. **Acceptance gates:** a synthetic stale-terminal scene runs its setup body once and reaches content; a @@ -716,19 +724,18 @@ multiplication order, or 2D projection: normal playback still races past these s reliable visual judgment. Treat that math as provisional until the pacing slice enables slow normal playback and a native-versus-port frame comparison. -### A2b — animation pacing + matrix validation ✅ (2026-07-10) +### A2b — animation pacing + matrix validation ✅ (2026-07-10; pacing model superseded) -The remaining race was a unit mismatch at the scheduler boundary. Native -`adv_interpreter_tick` advances one opcode, while the live cadence probe counted calls to -`vm_operand_fetch` (about 1,788 operand reads/s). The port's `FrameYield` runs once per completed -opcode, so using 1,800 there overclocked script teardown by roughly ninefold. `FrameClock` now supplies a -refresh-independent 200 completed-opcode/s allowance; sleep and input waits discard parked-time credit, and -input is ignored unless the VM is actually at `wait-for-input`. `--speed` scales the unified VM, -sleep, and animation clock for inspection without `--shot-sequence` auto-advance. +This slice initially interpreted roughly 1,788 `vm_operand_fetch` calls/s as evidence for a +refresh-independent 200 completed-opcode/s allowance. That scheduler interpretation is **superseded**: +the probe counted operand reads and mixed burst execution with time parked in native services. The matching +presentation trace below proves ordinary work is burst-fast; `--speed` now scales sleeps and retained +presentation clocks, not opcode throughput. Input remains ignored unless the VM is actually at +`wait-for-input`. Live native capture recorded the complete `0xcbc0` scale ramp (1→5 over 1,890 ms), including the exact -composed matrices. The port previously deleted the object at scale 1.44 after 226 ms. At 215/s calibration it -survived 1,798 ms to scale 4.72; the final 200/s replay retained it for 2,014 virtual ms, past the endpoint. +composed matrices. The 215/s and 200/s port replays were useful historical diagnostics showing that object +lifetime had been coupled incorrectly to script progress, but those rates are not native scheduler constants. Native matrix terms and the port's focused tests agree on row-vector `anchor + (point-anchor)*scale + translation`; for base `(0,600)`, anchor `(400,1000)`, scale 5, both project to `(-1600,-1000)`. Exact axis-aligned anchor/order/projection is validated; @@ -738,7 +745,7 @@ Verification: engine **92/92** after the opcode-clock reset test, corpus sweep u **284 exit / 13 STEP-LIMIT**, Godot build and threaded `SELFTEST OK`. The transform capture tool and transform-aware `--gfx-log` are documented in `docs/tools-reference.md`. -### A2b — ADV transition/lifecycle diagnosis plan ⏳ OPEN (2026-07-10) +### A2b — ADV transition/lifecycle diagnosis plan ✅ COMPLETED BELOW (2026-07-10) This is the next SC0000 correctness slice. It is driven by live A/B observations, not by static opcode coverage alone. The direct SC0000 histogram is currently **80/129 distinct ops handled (62.0%)** and @@ -938,7 +945,7 @@ in the object timeline and are not being mistaken for foreground CG transition c apart from the pre-existing nullable warning and threaded `SELFTEST OK`; all six Python test scripts, opcode/ctx lint, 481-script decode validation, RECOVER, and Windows CR-aware `git diff --check` clean. The plain check reports only the generated reference's CRLF on its newly added row. SC0000 coverage is -**85/129 handled (65.9%)**, 44 GAP ops / 598 GAP instructions. No commit was made. +**85/129 handled (65.9%)**, 44 GAP ops / 598 GAP instructions. This slice was later committed as `c9be9c5`. ### A2b — op 0x202 one-shot color/presentation ✅ DONE (2026-07-10) @@ -960,7 +967,8 @@ fill at `(0,373)`; repeated show/hide passages now produce intermediate samples `0.00, 0.26, 0.54, 0.82, 1.00` and the reverse), synchronized to the executing `0x202` sites around `0x12283..0x12342`. -The remaining full-white runs are not a stuck color clock or d2f0 endpoint: pixel windows place them after +**Predecessor diagnosis, resolved by the next subsection.** The remaining full-white runs were not a stuck +color clock or d2f0 endpoint: pixel windows place them after d2f0 is transparent, while object timelines show `AE*` handles first becoming visible in mode 0 with default `0xffffffff` before their later mode-1 initializer. Native call-site disassembly proves op `0x203`'s `obj+0x30` is passed directly as the D3D blend selector and mode 1 uses `SRCALPHA/INVSRCALPHA`; the port now @@ -1007,4 +1015,4 @@ separate slices. threaded `SELFTEST OK`; all seven Python suites, opcode/ctx lint, 481-script decode, and RECOVER clean. Normal-speed windowed capture wrote 220 PNGs and progressed through the complete AE sequence; at the target frame, bind + mode-1 + `0x202` setup share one VM frame and the first published object state is mode 1. -Ghidra `/v2` comments were updated and saved. No commit was made. +Ghidra `/v2` comments were updated and saved. This slice was committed as `85fc07d`. diff --git a/docs/tools-reference.md b/docs/tools-reference.md index 6f4858c..4b826e3 100644 --- a/docs/tools-reference.md +++ b/docs/tools-reference.md @@ -133,9 +133,9 @@ texture ops (no GPU context) — run windowed for real scenes. User args (after - `--seed 0xADDR=VAL` (repeatable) — seed initial global state, e.g. `--seed 0xa57=1` unlocks Lily's form-A voiced dialogue. - `--boot` — run SYSTEM4's state prefix (`INITCONFIG/INIT2/INIT`) via `GameSession` before the scene, so scene-assumed boot state (chiefly INIT2's gfx handle array) is present. **Needed for the gfx CGs to render** (without it the opening event CGs collapse/drift). e.g. `godot --path godot -- --boot`. - `--shot [--shot-page N]` — capture page N to a PNG then quit (dev screenshot). At scene end it also prints the call-scripts executed as nested frames. -- `--shot-sequence [--frames N]` — dump one PNG per rendered frame (`frame_0000.png…`, default N=180 ≈ 3s @60fps) then quit, auto-advancing past input waits. Verifies **time-based (sleep-paced) effects** — e.g. the opening `AE*` burst — as distinct frames, which a single `--shot` cannot. CPU/IO-heavy by design (a PNG every frame); a dev diagnostic, not a normal run. e.g. `godot --path godot -- --boot --shot-sequence out/seq --frames 300`. -- `--sleep-scale ` — multiply every `sleep` (op 0xc8) duration by `f` (default 1.0). The authentic opening burst is only ~2 s, too fast to eyeball live; `--sleep-scale 5` stretches it to ~10 s so the paced sequence (arcane `AE*` → character CGs → settled BG) is watchable. Debug-only; leave at 1.0 for real playback. -- `--speed ` — scale the unified runtime clock (VM cadence, sleeps, and retained animation) without auto-advancing input waits. Values 0.05–8 are accepted; `--speed 0.25` is useful for transform inspection, while 1.0 is normal playback. +- `--shot-sequence [--frames N]` — dump one PNG per rendered frame (`frame_0000.png…`, default N=180 ≈ 3s @60fps) then quit, auto-advancing past input waits. Verifies time-based retained effects and publication boundaries as distinct frames, which a single `--shot` cannot. CPU/IO-heavy by design (a PNG every frame); a dev diagnostic, not a normal run. e.g. `godot --path godot -- --boot --shot-sequence out/seq --frames 300`. +- `--sleep-scale ` — multiply every explicit `sleep` (op 0xc8) duration by `f` (default 1.0). This stretches only script-authored sleep holds; it does not slow ordinary opcode bursts or replace `0x20c`/`0x21c` presentation pacing. Debug-only; leave at 1.0 for real playback. +- `--speed ` — scale sleeps and retained presentation clocks without throttling ordinary opcode bursts or auto-advancing input waits. Values 0.05–8 are accepted; `--speed 0.25` is useful for transform inspection, while 1.0 is normal playback. - `--gfx-log ` — **compositor + op diagnostic** (the tool that root-caused the grey background). Logs, per rendered frame, only the objects whose draw outcome **CHANGED** (drawn↔skip↔gone, resId, resolved file, `slot`, `src`/`dst`, `op`acity, `tintStr`ength) — quiet until something actually changes, so the exact frame a layer drops out (and why) stands out. Also traces every `set-texture`/`create-texture` **slot assignment** (via `GodotAdvHost.TraceOps`). Works live or with `--shot-sequence`. Use it before theorising about layering/blend/geometry: it showed the grey BG = the slot-selecting globals resolving to 0 → every texture collapsing into slot 0 (see engine-re.md §"Grey-background root cause"). e.g. `godot --path godot -- --boot --gfx-log out/gfx.log` then click to the bad page. Matrix-channel outcomes also include `base`, `anchor`, projected `dst`, sampled `scale`/`trans`, and one-shot-plus-cyclic `rot`ation angles. Active op-`0x202` outcomes include packed `color=current->target` diff --git a/engine/Age.Engine/Hosting/FrameClock.cs b/engine/Age.Engine/Hosting/FrameClock.cs index c8a791e..0d65eec 100644 --- a/engine/Age.Engine/Hosting/FrameClock.cs +++ b/engine/Age.Engine/Hosting/FrameClock.cs @@ -1,7 +1,8 @@ namespace Age.Engine.Hosting; -/// Host-owned virtual clock. Godot advances it from real elapsed time; VM pacing, sleeps, and -/// retained graphics all consume this same timebase. Fractional milliseconds are retained so diagnostic +/// Host-owned virtual clock. Godot advances it from real elapsed time; presentation-service waits, +/// sleeps, and retained graphics consume this timebase. Ordinary opcode bursts are not clock-throttled. +/// Fractional milliseconds are retained so diagnostic /// slow motion does not stall on high-refresh displays. public sealed class FrameClock { @@ -11,14 +12,12 @@ public sealed class FrameClock /// Monotonic virtual time in milliseconds (scaled by Speed). public long NowMs => System.Threading.Interlocked.Read(ref _nowMs); - /// Speed multiplier. 1.0 = normal. A lower diagnostic value slows VM progress, sleeps, and - /// graphics together; a future ADV-scoped Ctrl hook can drive the same seam. + /// Speed multiplier. 1.0 = normal. A lower diagnostic value slows sleeps and retained + /// presentation clocks; ordinary opcode bursts still run to the next service boundary. public double Speed = 1.0; - /// Native normal-playback interpreter cadence. The old 1,800 figure counted calls to - /// vm_operand_fetch, not completed opcodes. A live 1,890 ms transform section executes about 407 - /// port opcodes. A normal-speed replay at 215/s retained the object for 1,798 ms; 200/s reaches - /// the native 1,890 ms endpoint before the same teardown path. + /// Legacy calibration retained for the isolated WallClockOpPacer tests. Production Godot no + /// longer uses an opcode-rate limiter; native presentation tracing disproved this as scheduler state. public double OpsPerSecond = 200.0; /// Advance the clock by one rendered frame's real delta (seconds), scaled by Speed. @@ -31,9 +30,8 @@ public sealed class FrameClock } } -/// Pure wall-clock opcode rate limiter. The VM thread records completed opcodes and waits whenever -/// it has consumed the allowance earned from . Reset after a blocking wait so -/// parked time never turns into a catch-up burst. +/// Legacy isolated rate-limiter utility. It is not used by production hosts; ordinary native opcode +/// execution is burst-fast between explicit presentation/sleep/input service boundaries. public sealed class WallClockOpPacer { private readonly FrameClock _clock; diff --git a/godot/GodotAdvHost.cs b/godot/GodotAdvHost.cs index 026f90c..893153e 100644 --- a/godot/GodotAdvHost.cs +++ b/godot/GodotAdvHost.cs @@ -126,21 +126,20 @@ public sealed class GodotAdvHost : IHost _frameSignal.Set(); } - // Main thread, once per rendered frame: releases a VM thread parked in FrameYield/Sleep. + // Main thread, once per rendered frame: releases a VM thread parked in Sleep or a presentation/input wait. public void PulseFrame() => _frameSignal.Set(); // Native presentation trace: ordinary opcode bursts run to the next service boundary in a few // milliseconds and are not frame-paced. Pacing belongs to 0x21c, sleep, and input waits below. public void FrameYield() { } - // op 0xc8: block the VM background thread so the main-thread compositor (Main.Recomposite in _Process) - // presents the current retained GfxState — this is what makes the sleep-paced opening burst animate. + // op 0xc8: block the VM background thread while the main-thread compositor keeps presenting retained state. // Time-based sibling of WaitForInput's suspend. The native op arms a non-blocking main-loop-polled timer; // blocking this throwaway task thread is behaviorally equivalent given our threading model. Operand is // MILLISECONDS (docs/engine-re.md sleep section + opcodes.toml 0xc8). Headless CLI hosts no-op it (parity). - public double SleepScale = 1.0; // --sleep-scale : debug multiplier to slow/speed the paced opening for inspection - // Wait on the unified FrameClock timebase (not Thread.Sleep) so a future Speed multiplier scales - // sleeps together with the throttle and the tween. Main._Process advances the clock + pulses each frame. + public double SleepScale = 1.0; // --sleep-scale : debug multiplier for explicit op-0xc8 holds only + // Wait on the unified FrameClock timebase (not Thread.Sleep) so the Speed multiplier scales + // sleeps together with retained presentation clocks. Main._Process advances the clock + pulses each frame. public void Sleep(long duration) { long ms = (long)System.Math.Clamp(duration * SleepScale, 0, 60_000); // cap so a pathological script can't hang the window diff --git a/godot/Main.cs b/godot/Main.cs index 87160eb..28b28ba 100644 --- a/godot/Main.cs +++ b/godot/Main.cs @@ -96,8 +96,8 @@ public partial class Main : Godot.Control bool boot = System.Array.IndexOf(userArgs, "--boot") >= 0; // run SYSTEM4's state prefix first string scene = "SC0000"; // --scene : which scene to play (default SC0000) var seeds = new List<(int Addr, long Val)>(); // --seed 0xADDR=VAL (repeatable) — initial global state - double sleepScale = 1.0; // --sleep-scale : slow/speed the paced opening for inspection - double speed = 1.0; // --speed : whole-runtime diagnostic speed + double sleepScale = 1.0; // --sleep-scale : scale explicit op-0xc8 holds + double speed = 1.0; // --speed : sleeps + retained presentation clocks long transitionClickMs = -1; // --transition-click-ms : force active transitions after n virtual ms string? histFile = null; // --trace-histogram : op/call-site execution counts of the REAL run for (int i = 0; i < userArgs.Length; i++) @@ -268,7 +268,7 @@ public partial class Main : Godot.Control System.Collections.Generic.Dictionary? decisions = _gfxLogPath != null || _timeline != null ? new() : null; int z = 0; var visible = _vm.Gfx.SnapshotVisibleObjects(_clock.NowMs); // one synchronized sample for objects + ranges - foreach (var v in visible) // interpolate at the throttled clock + foreach (var v in visible) // interpolate at the retained-presentation clock { var t = v.Transform; var affine = Age.Engine.Model.Transform2DMath.Build(t, v.Rotation); diff --git a/tools/frida/probe_frame_cadence.py b/tools/frida/probe_frame_cadence.py index 0d4113d..f89fd77 100644 --- a/tools/frida/probe_frame_cadence.py +++ b/tools/frida/probe_frame_cadence.py @@ -1,7 +1,7 @@ #!/usr/bin/env python3 -"""Live frame-cadence probe (docs/engine-re.md "Frame cadence"). Pins the native cadence — execution -rate vs displayed-frame rate, and frame timing — so the frame-stepped-VM fix picks its mechanism from -data instead of by feel. +"""Live frame-activity probe (docs/engine-re.md "Frame cadence"). Compares operand-fetch activity, +message-pump timing, and Ctrl state. It does not measure completed opcodes or establish an opcode budget; +the later matching presentation trace proved ordinary execution is burst-fast between service boundaries. SAFE pattern (matches capture_gfx_objects.py, which runs without crashing): plain-JS hooks only, no CModule; the only engine-code hook is the PROVEN operand-fetch helper `0x41b940` (fires once per @@ -148,7 +148,7 @@ def report(samples): rows.append(r) tot_ops = samples[-1]["ops"] - samples[0]["ops"] print(f"\n=== frame-cadence report ({len(rows)} intervals, {tot_ops} operand-fetches) ===") - # candidate frame signals: prefer a timing source that sits ~30-120/sec (loop reads clock once/frame) + # Candidate loop/presentation signals only; these do not imply a completed-opcode cadence. for label, want in (("NORMAL", False), ("FAST-FWD (Ctrl / skip-bit)", True)): b = [r for r in rows if r["ff"] == want] # for cadence, use only ACTIVE intervals (ops>0) so parked time doesn't dilute the numbers @@ -170,13 +170,14 @@ def report(samples): ", ".join(f"{nm}={v:.1f}" for nm, v in cand.items())) if frame_name: print(f" -> frame signal ~ {frame_name} @ {frame_rate:.1f}/sec ; " - f"exec/frame = {ops_active/frame_rate:.1f} (peak {peak/frame_rate:.1f})") + f"operand-fetches/signal = {ops_active/frame_rate:.1f} (peak {peak/frame_rate:.1f})") else: - print(" -> no timing source in 20-200/sec band; frame rate still unresolved.") + print(" -> no timing-source candidate in the sampled band; use probe_present.py for presentation.") flg = [f"0x{r['flags']:x}" for r in b if isinstance(r["flags"], int)] print(f" flags seen: {sorted(set(flg))}") - print("\n Stable exec/frame => fixed op-budget cadence; peak >> active-avg with parking => run-until-yield.") - print(" FAST-FWD vs NORMAL exec/frame shows how the ADV governor scales the slice.") + print("\n Counts are operand fetches, not completed opcodes or a scheduler budget.") + print(" Native presentation tracing later proved burst-fast execution between 0x20c/0x21c/sleep/input boundaries.") + print(" FAST-FWD vs NORMAL still measures how Ctrl changes operand-fetch activity.") def main(): diff --git a/vm-map/opcodes.toml b/vm-map/opcodes.toml index 3165c85..0fd0bdf 100644 --- a/vm-map/opcodes.toml +++ b/vm-map/opcodes.toml @@ -2037,8 +2037,8 @@ abi_source = "kelebek+decode-validated" [opcode.semantics] name = "sleep" category = "control" -summary = "Pause the script for milliseconds while rendering continues (frame pacing)." -details = "Native handler sleep_op_0xc8 @0x420ec0 is NON-BLOCKING: it arms a timer (sleep_timer_arm @0x44cff0 at ctx+0x5f304 = active flag + start tick + duration) that the engine main loop polls, resuming the script when elapsed. Operand UNIT = MILLISECONDS (start = ms tick source DAT_0056f3d4, timeGetTime/GetTickCount class). duration<10 fast-paths via [0x56f0b8]; all real scene sleeps (100/750/1000) are >=10. The handler also writes gfx cmd-type 3 + runs anti-tamper checks, neither needed host-side. Port equivalent: the Godot host blocks the VM background thread ms while the per-frame compositor keeps presenting -> correctly reproduces the explicit one-shot dramatic pauses. NOTE: does NOT pace the rapid opening AE* burst (those draws have no sleep between them; their real pacer is unknown). Headless hosts no-op it (parity)." +summary = "Pause the current script for milliseconds while retained presentation continues." +details = "Native handler sleep_op_0xc8 @0x420ec0 is NON-BLOCKING: it arms a timer (sleep_timer_arm @0x44cff0 at ctx+0x5f304 = active flag + start tick + duration) that the engine main loop polls, resuming the script when elapsed. Operand UNIT = MILLISECONDS (start = ms tick source DAT_0056f3d4, timeGetTime/GetTickCount class). duration<10 fast-paths via [0x56f0b8]; all real scene sleeps (100/750/1000) are >=10. The handler also writes gfx cmd-type 3 + runs anti-tamper checks, neither needed host-side. Port equivalent: the Godot host parks the VM thread for duration ms while the presentation compositor continues. Sleep is one proven presentation-capable service boundary; ordinary AE setup runs burst-fast to 0x21c and is not paced per opcode. Headless hosts no-op it (parity)." noop_headless = false source = "investigation" confidence = "high"