fix: align animation pacing and transforms with native

This commit is contained in:
gamer147
2026-07-10 15:41:17 -04:00
parent 29b2dfca27
commit 014d128ccd
16 changed files with 515 additions and 101 deletions

View File

@@ -451,6 +451,16 @@ independent matrix channels, not two encodings of one vec3 property.
delay, linearly interpolates current→target for its duration, then commits the target and clears its own timing.
Neither third component is opacity.
**Exact composition and 2D reduction (live-validated 2026-07-10).** The consumer starts from identity and
right-multiplies `T(-V18) → scale-current → middle/rotation → translation-current → T(+V18)`;
`matrix4_multiply` at `0x4ee2a4` computes `out = left * right`. AGE uses row vectors. With no
rotation/perspective, the screen projection is therefore exactly
`V18.xy + (point.xy - V18.xy) * scale.xy + translation.xy`. The captured SC0000 handle `0xcbc0`
has base `(0,600)`, anchor `(400,1000)`, and final scale `(5,5)`; native matrix translation
terms are `(-1600,-4000)`, projecting the base point to `(-1600,-1000)`. The port's focused
projection test and transform-aware gfx log reproduce those values. Rotation projection and final D3D
raster/rounding details remain deferred; the axis-aligned anchor/order/projection no longer are provisional.
**Port result (2026-07-10):** `GfxState` now retains separate current/target scale and translation
channels with the native shared-start/independent-timing model. Godot scales around `V18` (anchor),
applies translation independently, and never derives opacity from transform Z. The shared `AnimTarget`
@@ -483,6 +493,28 @@ both annotated) and grepping the SC0000 opening settles the animation model and
Op `0x234` is the independent rotation cycle above. Op `0x238` still configures the separate
`ctx+0x51b78/+0x51b7c` animation service used by its own family.
##### `label_1235a` animation-section boundary (2026-07-10)
The section helper computes the maximum configured duration and arms it with `0x238`, then reads
message-skip through `0x1c7` and ADV service state through `0x1cc`. Normal playback (both zero)
executes `0x243` (reset the separate global animation-service clock) followed by `0x20c` present;
the skip/service branch executes `0x21c` (set run-state bit `0x400`). Both branches finish with
`0x224`, which clears the native gfx command queue at `ctx+0x418`. These handlers are now named,
commented, and saved in the Ghidra image.
None of `0x21c/0x224/0x243` waits for the per-object transform duration. The normal-path boundary is
the engine's rate-limited one-op interpreter cadence plus continuous retained compositing. This matters because
the earlier Frida probe hooked `vm_operand_fetch`: its ~1,788/s result counts **operand reads**, commonly
several per completed opcode. Feeding that number to the port's per-completed-opcode `FrameYield` made
the `0xcbc0` section reach only scale 1.44 in 226 ms before teardown.
The corrected host limiter is refresh-independent and runs at 200 completed opcodes/s. It resets accumulated
credit after sleep/input parking, and clicks are accepted only while actually waiting, so clicks during a
visible animation cannot pre-arm the next wait. A normal-clock replay retained `0xcbc0` for 1,798 ms at
the intermediate 215/s calibration; the final 200/s replay kept it alive for 2,014 virtual ms, beyond the native
1,890 ms endpoint. `--speed` scales VM, sleep, and animation clocks together for comparison without
changing these virtual-time relationships or auto-advancing waits.
##### The opening render path is RETAINED, not immediate-mode (2026-07-08, ground-truth correction)
A working note in the animation slice mis-called the SC0000 opening a set of "immediate-mode slot-0 blits." That
@@ -612,10 +644,11 @@ Findings:
full ~119-method table) fires ~**1,908/sec** with **no vsync**; `BeginScene`/`EndScene` never fire → a **2D
StretchRect-style compositor**, not a 3D scene. So there is **no fixed display-frame rate**; `Present` rate
≈ op rate (~1 op per present). ⇒ the pacing quantity is the **wall-clock op rate**, not a per-frame budget.
- **Implication for the port:** throttle our VM to ~**1,800 ops/sec** wall-clock (≈30 ops per 60 fps Godot
`_Process`, tunable), ~4× under a future Ctrl multiplier; Godot's 60 fps compositor + wall-clock tweens then
show the smoothly-advancing state. This *measured* mechanism replaces the earlier present-driven guess
(present is rare in our path and uncapped natively).
- **Corrected implication for the port (2026-07-10):** 1,788/s is an operand-fetch rate, not an opcode rate.
`IHost.FrameYield` runs once per completed opcode, so matching those quantities directly overclocked the
port by roughly ninefold. The native SC0000 transform lifetime pins the corresponding completed-op cadence
at about **200/s**. The limiter must be wall-clock based (not a per-render callback budget), with a future
ADV Ctrl multiplier applied to the same unified clock.
### The render drift's SECOND half: missing system-boot state (2026-07-07, resolved)

View File

@@ -79,6 +79,16 @@ Native handler sleep_op_0xc8 @0x420ec0 is NON-BLOCKING: it arms a timer (sleep_t
- **grounding:** source=investigation, confidence=med
- **evidence:** Ghidra: handler 0x4299c0 (dispatch ctx[0x9b74c]=0x4299c0; created+typed EngineCtx*+annotated; Kelebek u0041F9C0 = VA-drift). Writes gfx cmd-type 9; op2→local_204, op3→local_104, op4→local_208; (*DAT_005c6018)(8, ctx[0x54fe8], &local_210) → FUN_00425fb0(1,ret). DAT_005c6018: 6 xrefs all READ, no static writer; FUN_00405740 (screen-fade) calls it w/ cmd 3, branches on ret 1/2 = transition progress = native video service.
### 0x1cc `get-adv-service-state` (get-adv-service-state, argc 1)
- **summary:** (out) - copy native ADV service state ctx+0x6dbd4; label_1235a ORs it with message-skip to select its yield branch.
- **grounding:** source=investigation, confidence=high
- **evidence:** Ghidra handler 0x427330 calls vm_operand_write(1, ctx+0x6dbd4). Exact service-state producer remains outside this opcode.
### 0x21c `mark-frame-yield` (mark-frame-yield, argc 0)
- **summary:** Set native run-state bit 0x400. Host-implicit: the port already offers a scheduler yield after every completed opcode.
- **grounding:** source=investigation, confidence=high, noop_headless=True
- **evidence:** Ghidra handler 0x417520 sets cmd-type 1 and ORs ctx+0xa0ce4 with 0x400. SC0000 label_1235a reaches it only when op 0x1c7 or 0x1cc is nonzero.
## draw
### 0x1a2 `gfx-cmd-register` (gfx-cmd-register, argc 1)
@@ -188,6 +198,11 @@ Native handler gfx_op_0x20c_present_frame (dispatch ctx[0x26c93+0x20c]) -> gfx_r
- **grounding:** source=investigation, confidence=high
- **evidence:** Ghidra 0x47ecc0 calls matrix builder 0x48afb1 for target obj+0x1ac. Consumer 0x472f00 uses delay obj+0x44, duration obj+0x58, current obj+0x16c, target obj+0x1ac, shared start obj+0x34, and frame-time ctx+0xb550.
### 0x224 `clear-gfx-command-queue` (clear-gfx-command-queue, argc 0)
- **summary:** Clear the native gfx command queue rooted at ctx+0x418. Host-implicit because the port composites retained state directly.
- **grounding:** source=investigation, confidence=high, noop_headless=True
- **evidence:** Ghidra handler 0x417550 -> gfx_command_queue_clear 0x47cb10, which destroys queued nodes and restores the sentinel links/count.
### 0x228 `u00421940` (u00421940, argc 5)
- **summary:** 0x228 query-position (succ)(handle)(outX)(outY)(outZ): read the object's current computed position into vars (worker FUN_0047cdd0). C# VM: writes V24 + success flag. See docs/engine-re.md §SC0000 anim cluster.
- **grounding:** source=kelebek, confidence=low
@@ -225,6 +240,11 @@ Native handler gfx_op_0x20c_present_frame (dispatch ctx[0x26c93+0x20c]) -> gfx_r
- **summary:** 0x23f query-object (out)(handle): return object status (FUN_0042a520; -1 if none). C# VM: 0 if the object exists else -1. See docs/engine-re.md §SC0000 anim cluster.
- **grounding:** source=kelebek, confidence=low
### 0x243 `reset-anim-clock` (reset-anim-clock, argc 0)
- **summary:** Reset the native global animation-service elapsed and duration fields to zero when service flag bit 1 is clear.
- **grounding:** source=investigation, confidence=high
- **evidence:** Ghidra handler 0x4182d0: if !(ctx+0x51b80 & 2), set ctx+0x51b70=1 and zero ctx+0x51b78/+0x51b7c. Normal SC0000 label_1235a calls it before present-frame.
## input
### 0x90 `hotspot-branch` (u0041BEB0, argc 7)
@@ -253,6 +273,11 @@ op 0x90 (u0041BEB0, argc 7): `0x90 x y w h tgt_a tgt_b tgt_c`. Kelebek left it "
- **depends on:** 0x90
- **evidence:** interleaves with 0x90 in the shared ADV-chrome subroutine; trailing imm = action id 0x0/0x7/0x8; same widget cluster as 0x90/0x91/0x92/0x95; confirm via frida
### 0x1c7 `get-message-skip` (get-message-skip, argc 1)
- **summary:** (out) - write 1 iff ADV message-skip run-state bit 0x08000000 is set, otherwise 0.
- **grounding:** source=investigation, confidence=high
- **evidence:** Ghidra handler 0x4272b0 reads ctx+0xa0ce4 bit 0x08000000 and vm_operand_write(1, 1|0). SC0000 label_1235a ORs it with op 0x1cc.
## marker
### 0x1bc `block-mark` (u00415670, argc 0)
@@ -882,10 +907,6 @@ op 0x90 (u0041BEB0, argc 7): `0x90 x y w h tgt_a tgt_b tgt_c`. Kelebek left it "
- **summary:** —
- **grounding:** source=kelebek, confidence=low
### 0x1c7 `u00414F90` (u00414F90, argc 1)
- **summary:** —
- **grounding:** source=kelebek, confidence=low
### 0x1c8 `toString` (toString, argc 2)
- **summary:** —
- **grounding:** source=kelebek, confidence=med
@@ -898,10 +919,6 @@ op 0x90 (u0041BEB0, argc 7): `0x90 x y w h tgt_a tgt_b tgt_c`. Kelebek left it "
- **summary:** —
- **grounding:** source=kelebek, confidence=low
### 0x1cc `u00415010` (u00415010, argc 1)
- **summary:** —
- **grounding:** source=kelebek, confidence=low
### 0x1ce `u0041B9F0` (u0041B9F0, argc 1)
- **summary:** —
- **grounding:** source=kelebek, confidence=low
@@ -970,10 +987,6 @@ op 0x90 (u0041BEB0, argc 7): `0x90 x y w h tgt_a tgt_b tgt_c`. Kelebek left it "
- **summary:** —
- **grounding:** source=kelebek, confidence=low
### 0x21c `u00416270` (u00416270, argc 0)
- **summary:** —
- **grounding:** source=kelebek, confidence=low
### 0x21d `u00421410` (u00421410, argc 2)
- **summary:** —
- **grounding:** source=kelebek, confidence=low
@@ -990,10 +1003,6 @@ op 0x90 (u0041BEB0, argc 7): `0x90 x y w h tgt_a tgt_b tgt_c`. Kelebek left it "
- **summary:** —
- **grounding:** source=kelebek, confidence=low
### 0x224 `u00416290` (u00416290, argc 0)
- **summary:** —
- **grounding:** source=kelebek, confidence=low
### 0x22a `u00421A90` (u00421A90, argc 3)
- **summary:** —
- **grounding:** source=kelebek, confidence=low
@@ -1042,10 +1051,6 @@ op 0x90 (u0041BEB0, argc 7): `0x90 x y w h tgt_a tgt_b tgt_c`. Kelebek left it "
- **summary:** —
- **grounding:** source=kelebek, confidence=low
### 0x243 `u00417070` (u00417070, argc 0)
- **summary:** —
- **grounding:** source=kelebek, confidence=low
### 0x248 `u00422E80` (u00422E80, argc 1)
- **summary:** —
- **grounding:** source=kelebek, confidence=low

View File

@@ -715,3 +715,25 @@ geometry in the capture. It does **not** prove the port's exact matrix calculati
multiplication order, or 2D projection: normal playback still races past these sections too quickly for a
reliable visual judgment. Treat that math as provisional until the pacing slice enables slow normal playback
and a native-versus-port frame comparison.
### A2b — animation pacing + matrix validation ✅ (2026-07-10)
The remaining race was a unit mismatch at the scheduler boundary. Native
`adv_interpreter_tick` advances one opcode, while the live cadence probe counted calls to
`vm_operand_fetch` (about 1,788 operand reads/s). The port's `FrameYield` runs once per completed
opcode, so using 1,800 there overclocked script teardown by roughly ninefold. `FrameClock` now supplies a
refresh-independent 200 completed-opcode/s allowance; sleep and input waits discard parked-time credit, and
input is ignored unless the VM is actually at `wait-for-input`. `--speed` scales the unified VM,
sleep, and animation clock for inspection without `--shot-sequence` auto-advance.
Live native capture recorded the complete `0xcbc0` scale ramp (1→5 over 1,890 ms), including the exact
composed matrices. The port previously deleted the object at scale 1.44 after 226 ms. At 215/s calibration it
survived 1,798 ms to scale 4.72; the final 200/s replay retained it for 2,014 virtual ms, past the endpoint.
Native matrix terms and the port's focused tests agree on row-vector
`anchor + (point-anchor)*scale + translation`; for base `(0,600)`, anchor `(400,1000)`,
scale 5, both project to `(-1600,-1000)`. Exact axis-aligned anchor/order/projection is validated;
cyclic-rotation rasterization remains the next affine-rendering slice.
Verification: engine **92/92** after the opcode-clock reset test, corpus sweep unchanged at
**284 exit / 13 STEP-LIMIT**, Godot build and threaded `SELFTEST OK`. The transform capture tool and
transform-aware `--gfx-log` are documented in `docs/tools-reference.md`.

View File

@@ -135,13 +135,15 @@ texture ops (no GPU context) — run windowed for real scenes. User args (after
- `--shot <png> [--shot-page N]` — capture page N to a PNG then quit (dev screenshot). At scene end it also prints the call-scripts executed as nested frames.
- `--shot-sequence <dir> [--frames N]` — dump one PNG per rendered frame (`frame_0000.png…`, default N=180 ≈ 3s @60fps) then quit, auto-advancing past input waits. Verifies **time-based (sleep-paced) effects** — e.g. the opening `AE*` burst — as distinct frames, which a single `--shot` cannot. CPU/IO-heavy by design (a PNG every frame); a dev diagnostic, not a normal run. e.g. `godot --path godot -- --boot --shot-sequence out/seq --frames 300`.
- `--sleep-scale <f>` — multiply every `sleep` (op 0xc8) duration by `f` (default 1.0). The authentic opening burst is only ~2 s, too fast to eyeball live; `--sleep-scale 5` stretches it to ~10 s so the paced sequence (arcane `AE*` → character CGs → settled BG) is watchable. Debug-only; leave at 1.0 for real playback.
- `--speed <f>` — scale the unified runtime clock (VM cadence, sleeps, and retained animation) without auto-advancing input waits. Values 0.058 are accepted; `--speed 0.25` is useful for transform inspection, while 1.0 is normal playback.
- `--gfx-log <file>`**compositor + op diagnostic** (the tool that root-caused the grey background). Logs, per rendered frame, only the objects whose draw outcome **CHANGED** (drawn↔skip↔gone, resId, resolved file, `slot`, `src`/`dst`, `op`acity, `tintStr`ength) — quiet until something actually changes, so the exact frame a layer drops out (and why) stands out. Also traces every `set-texture`/`create-texture` **slot assignment** (via `GodotAdvHost.TraceOps`). Works live or with `--shot-sequence`. Use it before theorising about layering/blend/geometry: it showed the grey BG = the slot-selecting globals resolving to 0 → every texture collapsing into slot 0 (see engine-re.md §"Grey-background root cause"). e.g. `godot --path godot -- --boot --gfx-log out/gfx.log` then click to the bad page.
Matrix-channel outcomes also include sampled `scale`/`trans` values. Parent directories are created automatically.
Matrix-channel outcomes also include `base`, `anchor`, projected `dst`, and sampled `scale`/`trans` values. Parent directories are created automatically.
## Asset resolution / graphics
| Tool | Purpose | Run | Reads → Writes |
|---|---|---|---|
| `tools/frida/capture_native_transforms.py` | Capture the native retained-object transform input fields and the exact composed 4×4 matrix at `gfx_object_apply_transform_channels`. Optional handle filter; read-only. | `py -3.11 -u -X utf8 tools/frida/capture_native_transforms.py [secs] [pid|AGE.EXE] [--handle 0xHANDLE]` | running game → `build/native-transform-trace.jsonl` |
| `parse_sys4ini.py` | Parse `SYS4INI.BIN` (S4IC422, LZSS-compressed) into the authoritative asset index — name ↔ archive ↔ offset ↔ size for all DATA*.ALF (the `resId→file` answer key). Each entry carries `raw_index` (its 0-based position in the SYS4INI record table incl. `@` placeholders) = the engine's universal file id. Also emits the **`call-script <id> → name`** map (id = `raw_index`; see `engine-re.md`). | `parse_sys4ini.py [--check]` (`--check` validates vs `extracted/` + `.ALF` sizes) | `姫狩り…/SYS4INI.BIN``build/asset-index.json` + `build/callscript-names.json` |
| `resolve_asset.py` | ★ **The static asset resolver.** SYS4INI is sectioned (one per scene: `SCxxxx.BIN` + its cross-archive manifest; `file_number` = index within section). Resolves `resId → files[section_base(scene) + resId]` for graphics AND audio, no capture. | `resolve_asset.py --build` · `resolve_asset.py <SCENE> [resId]` | `build/asset-index.json``build/asset-sections.json`; resolves any (scene, resId) |
| `resolve_frida_reads.py` | Rescue noisy Frida archive-read offsets → asset names via the index (per-archive range search; drops 0x20000 paging reads); recovers the per-scene asset load order. | `resolve_frida_reads.py [reads.log] [-o out.json]` | `build/frida-reads.log` + `build/asset-index.json``build/frida-asset-loads.json` |
@@ -161,7 +163,7 @@ texture ops (no GPU context) — run windowed for real scenes. User args (after
| `tools/frida/map_imports.py` (+ `map_imports_full.py`) | ★ **Name dynamically-resolved Win32 APIs** in the Ghidra image. Read-only: maps live-process module exports → `{addr→dll!Func}`, scans the `0x400000` module for pointer matches → `RVA→name` (ASLR-stable). `--recon` = clustering report (the gate); default writes the map. Applied to `/v2` via a `run_script_inline` pass → 248 `imp_<dll>_<func>` labels at the RVA `0x16f000` IAT (validated: CreateFileA/SetFilePointer/timeGetTime). Pure scan/cluster logic unit-tested (`test_map_imports.py`). | `py -3.11 -u -X utf8 tools/frida/map_imports.py [--recon]` | running game → `build/import-map.json` (+ `-singletons.json`) |
| `tools/frida/probe_handlers.py` | Probe which region the interpreter executes from (module vs heap). Confirmed: **operand-fetch `+0x1b940` fires ~8500/s ⇒ interpreter runs from the module `0x400000`** (handlers hookable by dump address). | `py -3.11 -u -X utf8 tools/frida/probe_handlers.py [pid]` | running game → stdout (per-hook fire counts) |
| `tools/frida/capture_gfx_objects.py` | Capture the native gfx object-manager state: grab engine ctx (`esi` via operand-fetch `ecx`), poll the object-record array `[esi+0x53d64]` (20×120B; `field[0]=0xffffffff`=free, cmd-type at rec+0x24). **⚠ Its "0 CG records ⇒ drift is state-divergence" reading was DISPROVEN** (Ghidra: op 0x215 read settles the drift as a native command-buffer op — `docs/engine-re.md`; the poll observed the record array, not the lookup map that drives the branch, and cmd-buffer records are transient). Kept as a runtime-observation tool. | `py -3.11 -u -X utf8 tools/frida/capture_gfx_objects.py [pid] [secs]` | running game → `build/gfx-objects.jsonl` |
| `tools/frida/probe_frame_cadence.py` | **Frame-cadence probe** (`docs/engine-re.md` "Frame cadence — live measurement"): plain-JS hook on operand-fetch `0x41b940` (grab ctx + count exec rate) + system-DLL message/timing hooks; auto-buckets by Ctrl/skip-bit. Measured: exec **rate-limited** ~1788 ops/sec normal, ~4× fast-forward. **Read-only/import-only — never CModule-hook the hot interpreter (crashes the game).** Play actively during capture; hold Ctrl the back half. | `py -3.11 -u -X utf8 tools/frida/probe_frame_cadence.py [secs] [proc]` | running game → `build/frida-frame-cadence.jsonl` + stdout report |
| `tools/frida/probe_frame_cadence.py` | **Frame-cadence probe** (`docs/engine-re.md` "Frame cadence — live measurement"): plain-JS hook on operand-fetch `0x41b940` (grab ctx + count operand reads) + system-DLL message/timing hooks; auto-buckets by Ctrl/skip-bit. Measured ~1,788 **operand fetches/sec** normal, ~4× fast-forward; this is not an opcode count. **Read-only/import-only — never CModule-hook the hot interpreter (crashes the game).** Play actively during capture; hold Ctrl the back half. | `py -3.11 -u -X utf8 tools/frida/probe_frame_cadence.py [secs] [proc]` | running game → `build/frida-frame-cadence.jsonl` + stdout report |
| `tools/frida/probe_present.py` | **Present-rate probe:** grab ctx, scan it for the D3D9 device (d3d9-vtable object with a full ~119-method table), hook `IDirect3DDevice9::Present`/`EndScene` (+ GDI-blit fallback). Found: **D3D9, UNCAPPED** (`Present` ~1908/sec, no vsync; no `ddraw`; 2D StretchRect compositor) ⇒ no fixed frame rate. Click 23× at start to grab ctx. | `py -3.11 -u -X utf8 tools/frida/probe_present.py [secs]` | running game → `build/frida-present.jsonl` + stdout report |
| `tools/frida/trace_engine_ops.py` | **Engine op-path tracer** for the differential oracle (`docs/engine-re.md` "Differential offset-path oracle"): per executed op, read `cur_ctx_index@0x53d14`/`frame_pc@0x53d2c`/`frame_codebase@0x53d28` → emit `(codebase, offset=(pccodebase)/4)`. **Use `--hook operand` (0x41b940, proven-safe)**`--hook tick` (0x410fb0) sees `ecx≠ctx` (0 entries). Writes `build/tracer-live.flag` when the hook is installed → launch in the background, gate the New-Game trigger on the flag (else the scene-entry burst is missed). | `py -3.11 -u -X utf8 tools/frida/trace_engine_ops.py [--hook operand\|tick] [secs]` | running game → `build/engine-optrace.jsonl` |
| `tools/frida/capture_global_writes.py` | **Scene-entry state capture** → auto-seed for single-scene runs (`docs/engine-re.md` "Scene-entry state snapshot"). Hooks `vm_operand_write@0x425fb0` and logs `(codebase, index, PLAINTEXT value)` for global-ints (the helper sees the value before the obfuscated store — no de-obfuscation needed). **`--spawn` captures from boot** (packer-aware: polls until `0x425fb0` unpacks, then attaches; kills the spawned pid on setup failure so no suspended orphan). `--attach` = partial (misses pre-attach writes). Validated: a real boot→New-Game→SC0000 capture seeds the VM to match the engine's whole opening. | `py -3.11 -u -X utf8 tools/frida/capture_global_writes.py --spawn [secs]` | running/spawned game → `build/global-writes.jsonl` (raw) + `build/scene-entry-state.json` (GameSession snapshot) |