docs: frame-stepped-vm spec+plan + frame-cadence Frida probes
Design artifacts for the merged frame-stepped VM work (throttle the Godot VM to a per-frame op budget). Probes measured the native ~1788 ops/sec cadence and uncapped D3D9 Present that motivated the wall-clock-op-rate approach. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
426
docs/superpowers/plans/2026-07-08-frame-stepped-vm.md
Normal file
426
docs/superpowers/plans/2026-07-08-frame-stepped-vm.md
Normal file
@@ -0,0 +1,426 @@
|
||||
# Frame-stepped VM Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Stop the Godot port's VM from outrunning real time (the SC0000 opening "speeds through") by throttling interpreter execution to a bounded wall-clock op rate, so visible state changes over real time like the native engine.
|
||||
|
||||
**Architecture:** Keep the VM on its background thread (unchanged). Add ONE host call per executed opcode — `IHost.FrameYield()`. Headless hosts no-op it (byte-identical parity). The Godot host throttles: it counts ops and, once a per-frame budget is reached, blocks the VM thread until the main-thread `_Process` advances a host-owned `FrameClock`. That same clock drives `Sleep` and the anim tween, so a future single `Speed` multiplier (Ctrl fast-forward, NOT built now) scales everything coherently.
|
||||
|
||||
**Tech Stack:** C# / .NET 8 (`engine/AgeEngine.sln`), xUnit tests, Godot 4.7 mono (`godot/Himegari.csproj`).
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-07-08-frame-stepped-vm-design.md` (read it — it has the RE evidence: the native interpreter runs ~1,788 ops/sec normal, ~4× under Ctrl; rendering is uncapped D3D9, so the pacing quantity is a wall-clock op rate, not a display-frame budget).
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- **Parity is sacred.** All non-Godot hosts must keep headless output byte-identical. Regression gates that MUST stay green unchanged: the engine test suite (`dotnet test engine/AgeEngine.sln`), `Age.Cli sweep` (reports `exit=284, STEP-LIMIT=13`), and the Godot `--selftest` (`SELFTEST OK`). `FrameYield`/`Sleep` are no-ops in every non-Godot host, which is what guarantees this.
|
||||
- **Seam rule:** `Age.Engine/Vm` may reference only `Model`, `Hosting`, `Diagnostics` — never `Sys4`. `FrameClock` lives in `Age.Engine/Hosting` (used by hosts, not by `Vm`).
|
||||
- **No Ctrl wiring.** `FrameClock.Speed` stays `1.0` (a field, the future hook). Do NOT add a key handler or change Speed. When eventually wired it must be ADV-mode-scoped — do not bake in a global-speed assumption.
|
||||
- **Build/run C#:** `dotnet build engine/AgeEngine.sln -c Debug`; `dotnet test engine/AgeEngine.sln`. Godot: `dotnet build godot/Himegari.csproj -c Debug` then the console exe at `S:\Godot\Godot_v4.7-stable_mono_win64\Godot_v4.7-stable_mono_win64_console.exe` (or `pwsh run-godot.ps1`).
|
||||
- **Python (if needed):** `py -3.11 -X utf8 tools/<name>.py`.
|
||||
- TDD, one deliverable per task, commit at the end of each task. End every commit message with:
|
||||
`Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>`
|
||||
|
||||
---
|
||||
|
||||
## Task 0: Branch
|
||||
|
||||
- [ ] **Step 1: Create the feature branch** (repo root = `age-reimpl/`, git repo; default branch is `main`)
|
||||
|
||||
```bash
|
||||
cd "S:/Game Hacking/Eushully/Himegari/age-reimpl"
|
||||
git checkout -b feat/frame-stepped-vm
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- `engine/Age.Engine/Hosting/FrameClock.cs` — **new**. Pure virtual clock: `NowMs`, `Speed`, `OpsPerFrame`, `Advance()`, `EffectiveBudget`. No threading. Task 1.
|
||||
- `engine/Age.Engine/Hosting/IHost.cs` — **modify**. Add `void FrameYield()`. Task 2.
|
||||
- `engine/Age.Engine/Vm/VirtualMachine.cs` — **modify**. Call `_host.FrameYield()` once per executed opcode. Task 2.
|
||||
- `engine/Age.Engine/Hosting/CaptureHost.cs` + every other non-Godot `IHost` implementer — **modify**. Add empty `FrameYield()`. Task 2.
|
||||
- `godot/GodotAdvHost.cs` — **modify**. Real `FrameYield()` throttle + `Sleep()` on the clock. Task 3.
|
||||
- `godot/Main.cs` — **modify**. Own a `FrameClock`, advance + pulse it each `_Process`, tween reads it. Task 3.
|
||||
- `engine/Age.Engine.Tests/FrameClockTests.cs` — **new**. Task 1.
|
||||
- `engine/Age.Engine.Tests/FrameYieldTests.cs` — **new**. Task 2.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: `FrameClock` — pure virtual clock + op budget
|
||||
|
||||
**Files:**
|
||||
- Create: `engine/Age.Engine/Hosting/FrameClock.cs`
|
||||
- Test: `engine/Age.Engine.Tests/FrameClockTests.cs`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `Age.Engine.Hosting.FrameClock` with `long NowMs { get; }`, `double Speed` (field, default 1.0), `int OpsPerFrame` (field, default 30), `void Advance(double realDeltaSeconds)`, `int EffectiveBudget { get; }`.
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Create `engine/Age.Engine.Tests/FrameClockTests.cs`:
|
||||
|
||||
```csharp
|
||||
using Age.Engine.Hosting;
|
||||
using Xunit;
|
||||
|
||||
public class FrameClockTests
|
||||
{
|
||||
[Fact]
|
||||
public void Advance_AtSpeed1_AddsRealMilliseconds()
|
||||
{
|
||||
var c = new FrameClock(); // Speed defaults to 1.0
|
||||
c.Advance(0.016); // one ~60fps frame
|
||||
Assert.Equal(16, c.NowMs);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Advance_ScalesBySpeed()
|
||||
{
|
||||
var c = new FrameClock { Speed = 4.0 };
|
||||
c.Advance(0.016);
|
||||
Assert.Equal(64, c.NowMs); // 4x virtual time
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void EffectiveBudget_ScalesBySpeed_AndFloorsAtOne()
|
||||
{
|
||||
Assert.Equal(30, new FrameClock { OpsPerFrame = 30, Speed = 1.0 }.EffectiveBudget);
|
||||
Assert.Equal(120, new FrameClock { OpsPerFrame = 30, Speed = 4.0 }.EffectiveBudget);
|
||||
Assert.Equal(1, new FrameClock { OpsPerFrame = 0, Speed = 1.0 }.EffectiveBudget);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `dotnet test engine/AgeEngine.sln --filter FrameClockTests`
|
||||
Expected: FAIL — `FrameClock` does not exist (compile error).
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
Create `engine/Age.Engine/Hosting/FrameClock.cs`:
|
||||
|
||||
```csharp
|
||||
namespace Age.Engine.Hosting;
|
||||
|
||||
/// <summary>Host-owned virtual clock + per-frame op budget. Pure (no threading): the Godot host
|
||||
/// advances it once per rendered frame and consults it to pace the VM. The one <see cref="Speed"/>
|
||||
/// factor is the future (unwired) Ctrl fast-forward multiplier — scaling it scales the throttle
|
||||
/// budget, sleeps, and the anim tween together. See docs/superpowers/specs/2026-07-08-frame-stepped-vm-design.md.</summary>
|
||||
public sealed class FrameClock
|
||||
{
|
||||
/// <summary>Monotonic virtual time in milliseconds (scaled by Speed).</summary>
|
||||
public long NowMs { get; private set; }
|
||||
|
||||
/// <summary>Speed multiplier. 1.0 = normal. The future Ctrl hook (ADV-scoped); leave at 1.0 for now.</summary>
|
||||
public double Speed = 1.0;
|
||||
|
||||
/// <summary>Base per-frame interpreter op budget (tunable by eye; ~30 ≈ 1,800 ops/sec at 60fps).</summary>
|
||||
public int OpsPerFrame = 30;
|
||||
|
||||
/// <summary>Advance the clock by one rendered frame's real delta (seconds), scaled by Speed.</summary>
|
||||
public void Advance(double realDeltaSeconds) => NowMs += (long)(realDeltaSeconds * 1000.0 * Speed);
|
||||
|
||||
/// <summary>Ops the VM may run before yielding a frame, scaled by Speed (min 1).</summary>
|
||||
public int EffectiveBudget => System.Math.Max(1, (int)System.Math.Round(OpsPerFrame * Speed));
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `dotnet test engine/AgeEngine.sln --filter FrameClockTests`
|
||||
Expected: PASS (3 tests).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add engine/Age.Engine/Hosting/FrameClock.cs engine/Age.Engine.Tests/FrameClockTests.cs
|
||||
git commit -m "feat: add FrameClock (virtual clock + per-frame op budget)
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: `IHost.FrameYield()` — per-op hook + headless no-ops + parity
|
||||
|
||||
**Files:**
|
||||
- Modify: `engine/Age.Engine/Hosting/IHost.cs`
|
||||
- Modify: `engine/Age.Engine/Vm/VirtualMachine.cs` (the `RunFrame` loop, ~lines 119-128)
|
||||
- Modify: `engine/Age.Engine/Hosting/CaptureHost.cs` and every other non-Godot `IHost` implementer
|
||||
- Test: `engine/Age.Engine.Tests/FrameYieldTests.cs`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: nothing from Task 1 yet (the Godot host uses `FrameClock` in Task 3).
|
||||
- Produces: `IHost.FrameYield()` (called by the VM exactly once per executed opcode, including the halting/returning op). Headless implementers make it a no-op.
|
||||
|
||||
- [ ] **Step 1: Find every `IHost` implementer** (so the build won't break)
|
||||
|
||||
Run: `py -3.11 -X utf8 -c "import subprocess"` is not needed — just grep:
|
||||
Use the Grep tool for `: IHost` and `IHost` across `engine/` — expected implementers to edit in this task:
|
||||
`engine/Age.Engine/Hosting/CaptureHost.cs`, the test hosts in `engine/Age.Engine.Tests/` (e.g. recording/counting hosts used by existing tests), and the CLI hosts in `engine/Age.Cli/Program.cs` (`AudioTraceHost`, `GfxTraceHost`). **`godot/GodotAdvHost.cs` is handled in Task 3** (separate project, not in `AgeEngine.sln`).
|
||||
|
||||
- [ ] **Step 2: Write the failing test**
|
||||
|
||||
Create `engine/Age.Engine.Tests/FrameYieldTests.cs`. This asserts the VM calls `FrameYield` exactly once per step (`Steps`), proving the per-op wiring, using a tiny counting host. It loads a real boot script that halts cleanly.
|
||||
|
||||
```csharp
|
||||
using Age.Engine.Hosting;
|
||||
using Age.Engine.Model;
|
||||
using Age.Engine.Sys4;
|
||||
using Age.Engine.Vm;
|
||||
using Xunit;
|
||||
|
||||
public class FrameYieldTests
|
||||
{
|
||||
private sealed class CountingHost : IHost
|
||||
{
|
||||
public long Yields;
|
||||
public void FrameYield() => Yields++;
|
||||
public void ShowText(int offset, string text) { }
|
||||
public void WaitForInput() { }
|
||||
public void Sleep(long duration) { }
|
||||
public void CreateTexture(int slot, int width, int height) { }
|
||||
public void SetTexture(long resourceId, int slot) { }
|
||||
public void DrawTexture(int slot, int sx, int sy, int w, int h, int dx, int dy) { }
|
||||
public (int Width, int Height) GetTextureSize(int slot) => (0, 0);
|
||||
public void PlayBgm(long id) { }
|
||||
public void PlayVoice(long id) { }
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void FrameYield_CalledOncePerStep()
|
||||
{
|
||||
var table = OpcodeTableJson.Load(Paths.OpcodesJson);
|
||||
var script = Sys4Loader.Load(Paths.Scripts()["INITCONFIG.BIN"], table);
|
||||
var host = new CountingHost();
|
||||
var vm = new VirtualMachine(script, table, host);
|
||||
vm.Run();
|
||||
Assert.True(vm.Steps > 0);
|
||||
Assert.Equal(vm.Steps, host.Yields); // exactly one FrameYield per executed opcode
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run test to verify it fails**
|
||||
|
||||
Run: `dotnet test engine/AgeEngine.sln --filter FrameYieldTests`
|
||||
Expected: FAIL — `IHost` has no `FrameYield` (compile error in the host), and the VM does not call it.
|
||||
|
||||
- [ ] **Step 4: Add `FrameYield` to the interface**
|
||||
|
||||
In `engine/Age.Engine/Hosting/IHost.cs`, add the method (place it next to `WaitForInput`/`Sleep`):
|
||||
|
||||
```csharp
|
||||
void FrameYield();
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Call it once per opcode in the VM**
|
||||
|
||||
In `engine/Age.Engine/Vm/VirtualMachine.cs`, the `RunFrame` loop currently reads (around lines 119-128):
|
||||
|
||||
```csharp
|
||||
while (pc >= 0 && pc < frame.Script.Instructions.Count)
|
||||
{
|
||||
if (Steps >= _o.MaxSteps) { HaltReason ??= "STEP-LIMIT"; outcome = FrameOutcome.Halted; break; }
|
||||
Steps++;
|
||||
if (_sink.TracingSteps) _sink.Emit(TraceEvent.Step(pc, frame.Script.Instructions[pc], _depth));
|
||||
int next = Step(frame.Script.Instructions[pc], pc);
|
||||
if (next == FRAME_RETURN) { outcome = FrameOutcome.Returned; break; }
|
||||
if (next == HALT) { outcome = FrameOutcome.Halted; break; }
|
||||
pc = next;
|
||||
}
|
||||
```
|
||||
|
||||
Add `_host.FrameYield();` immediately after the `Step(...)` call:
|
||||
|
||||
```csharp
|
||||
int next = Step(frame.Script.Instructions[pc], pc);
|
||||
_host.FrameYield();
|
||||
if (next == FRAME_RETURN) { outcome = FrameOutcome.Returned; break; }
|
||||
```
|
||||
|
||||
(`_host` is the existing `IHost` field the VM already dispatches `ShowText`/`Sleep`/etc. through.)
|
||||
|
||||
- [ ] **Step 6: Add empty `FrameYield()` to every non-Godot host**
|
||||
|
||||
In each implementer found in Step 1, add:
|
||||
|
||||
```csharp
|
||||
public void FrameYield() { }
|
||||
```
|
||||
|
||||
Concretely: `engine/Age.Engine/Hosting/CaptureHost.cs`; each test host class in `engine/Age.Engine.Tests/`; and `AudioTraceHost` + `GfxTraceHost` in `engine/Age.Cli/Program.cs`. (The `CountingHost` in Step 2 already has its own.)
|
||||
|
||||
- [ ] **Step 7: Run the new test — verify it passes**
|
||||
|
||||
Run: `dotnet test engine/AgeEngine.sln --filter FrameYieldTests`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 8: Run the FULL suite — verify parity (nothing else changed)**
|
||||
|
||||
Run: `dotnet test engine/AgeEngine.sln`
|
||||
Expected: PASS — all pre-existing tests still green (FrameYield is a no-op everywhere headless, so traces/steps/emitted are byte-identical).
|
||||
|
||||
- [ ] **Step 9: Verify the corpus oracle is unchanged**
|
||||
|
||||
Run: `dotnet run --project engine/Age.Cli -c Debug -- sweep`
|
||||
Expected: halt distribution includes `exit=284` and `STEP-LIMIT=13` (unchanged from before this task).
|
||||
|
||||
- [ ] **Step 10: Commit**
|
||||
|
||||
```bash
|
||||
git add engine/Age.Engine/Hosting/IHost.cs engine/Age.Engine/Vm/VirtualMachine.cs engine/Age.Engine/Hosting/CaptureHost.cs engine/Age.Cli/Program.cs engine/Age.Engine.Tests/FrameYieldTests.cs
|
||||
# also add any test-host files you edited under engine/Age.Engine.Tests/
|
||||
git commit -m "feat: add IHost.FrameYield per-opcode hook (no-op headless, parity held)
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Godot host throttle + Sleep/tween on the clock
|
||||
|
||||
**Files:**
|
||||
- Modify: `godot/GodotAdvHost.cs`
|
||||
- Modify: `godot/Main.cs`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `Age.Engine.Hosting.FrameClock` (Task 1); `IHost.FrameYield` (Task 2).
|
||||
- Produces: `GodotAdvHost` throttled to `FrameClock.EffectiveBudget` ops per rendered frame; `GodotAdvHost.PulseFrame()` (called by `Main._Process` each frame).
|
||||
|
||||
This task changes only the Godot project, which builds separately (`godot/Himegari.csproj`) and is verified by the Godot `--selftest` + a manual windowed run (not the xUnit suite).
|
||||
|
||||
- [ ] **Step 1: Add the FrameClock + throttle to `GodotAdvHost`**
|
||||
|
||||
In `godot/GodotAdvHost.cs`:
|
||||
|
||||
(a) Add fields (near the existing `_gate`):
|
||||
|
||||
```csharp
|
||||
private readonly Age.Engine.Hosting.FrameClock _clock;
|
||||
private readonly System.Threading.AutoResetEvent _frameSignal = new(false);
|
||||
private int _opsSinceYield;
|
||||
```
|
||||
|
||||
(b) Change the constructor to accept the clock:
|
||||
|
||||
```csharp
|
||||
public GodotAdvHost(Main main, ResourceMap res, string scene, Age.Engine.Hosting.FrameClock clock)
|
||||
{
|
||||
_main = main; _res = res; _scene = scene; _clock = clock;
|
||||
}
|
||||
```
|
||||
|
||||
(c) Add the per-frame pulse (called from `Main._Process`) and the throttle. Add these methods:
|
||||
|
||||
```csharp
|
||||
// Main thread, once per rendered frame: releases a VM thread parked in FrameYield/Sleep.
|
||||
public void PulseFrame() => _frameSignal.Set();
|
||||
|
||||
// Called once per executed opcode (IHost.FrameYield). After a frame's worth of ops (the clock's
|
||||
// budget), block the VM background thread until Main._Process advances the clock — throttling the
|
||||
// interpreter to ~budget ops per rendered frame (the native engine's rate-limited cadence).
|
||||
public void FrameYield()
|
||||
{
|
||||
if (++_opsSinceYield < _clock.EffectiveBudget) return;
|
||||
_opsSinceYield = 0;
|
||||
long start = _clock.NowMs;
|
||||
while (_clock.NowMs == start) // wait until a real _Process advanced the clock
|
||||
if (!_frameSignal.WaitOne(50)) break; // 50ms safety cap: never hang if _Process stalls
|
||||
}
|
||||
```
|
||||
|
||||
(d) Replace the existing `Sleep` body so it waits on the clock (unified timebase) instead of `Thread.Sleep`:
|
||||
|
||||
```csharp
|
||||
public double SleepScale = 1.0; // --sleep-scale <f>: debug multiplier (kept)
|
||||
public void Sleep(long duration)
|
||||
{
|
||||
long ms = (long)System.Math.Clamp(duration * SleepScale, 0, 60_000);
|
||||
long deadline = _clock.NowMs + ms;
|
||||
while (_clock.NowMs < deadline)
|
||||
if (!_frameSignal.WaitOne(2000)) break; // safety cap
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Own + drive the FrameClock in `Main`**
|
||||
|
||||
In `godot/Main.cs`:
|
||||
|
||||
(a) Add a field (near the other Main fields, ~line 23):
|
||||
|
||||
```csharp
|
||||
private readonly Age.Engine.Hosting.FrameClock _clock = new();
|
||||
```
|
||||
|
||||
(b) Pass it when constructing the host. The current line (~122) reads:
|
||||
|
||||
```csharp
|
||||
_host = new GodotAdvHost(this, ResourceMap.Load(), scene) { SleepScale = sleepScale };
|
||||
```
|
||||
|
||||
Change to:
|
||||
|
||||
```csharp
|
||||
_host = new GodotAdvHost(this, ResourceMap.Load(), scene, _clock) { SleepScale = sleepScale };
|
||||
```
|
||||
|
||||
(c) In `_Process(double delta)`, right after the existing `_lastDelta = delta;` (~line 158), advance the clock and pulse the frame:
|
||||
|
||||
```csharp
|
||||
_lastDelta = delta;
|
||||
_clock.Advance(delta);
|
||||
_host?.PulseFrame();
|
||||
```
|
||||
|
||||
(d) Make the anim tween use the clock's (Speed-scaled) time so a future Speed multiplier scales it too. In `AlphaFor` (~line 281), change:
|
||||
|
||||
```csharp
|
||||
tw.Elapsed += _lastDelta;
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```csharp
|
||||
tw.Elapsed += _lastDelta * _clock.Speed; // Speed==1 now => identical; future Ctrl scales the tween
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Build the Godot project**
|
||||
|
||||
Run: `dotnet build godot/Himegari.csproj -c Debug --nologo -v q`
|
||||
Expected: `Build succeeded. 0 Error(s)`.
|
||||
|
||||
- [ ] **Step 4: Verify parity via the headless self-test**
|
||||
|
||||
Run: `& "S:\Godot\Godot_v4.7-stable_mono_win64\Godot_v4.7-stable_mono_win64_console.exe" --headless --path godot -- --selftest`
|
||||
Expected: output ends with `SELFTEST OK` (the synthetic scene still produces identical lines; the throttle changes timing, not output).
|
||||
|
||||
- [ ] **Step 5: Manual visual check (acceptance — user-facing)**
|
||||
|
||||
Run windowed with boot state:
|
||||
`& "S:\Godot\Godot_v4.7-stable_mono_win64\Godot_v4.7-stable_mono_win64_console.exe" --path godot -- --boot`
|
||||
Expected: the SC0000 opening now advances at a readable pace (no instant speed-through). If it's too slow or too fast, tune `FrameClock.OpsPerFrame` (default 30; higher = faster). This is the eyeball calibration step — the user validates and picks the final `OpsPerFrame`.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add godot/GodotAdvHost.cs godot/Main.cs
|
||||
git commit -m "feat: throttle Godot VM to a per-frame op budget on FrameClock (fixes opening speed-through)
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Notes for the executor
|
||||
|
||||
- **If per-op `FrameYield()` overhead shows up** in a slow `dotnet test`/`sweep`, batch it: keep a counter in the VM and call `_host.FrameYield()` every K ops (e.g. 4). Parity is unaffected (still no-op headless); only the Godot throttle granularity coarsens. Not expected to be necessary.
|
||||
- **Do NOT** wire Ctrl / change `Speed`. That's a separate future slice (needs the ADV-scope RE); the seam is ready for it.
|
||||
- **Stretch (only if the opening still looks wrong after tuning):** confirm the 2D-composite assumption by hooking `IDirect3DDevice9::StretchRect`/`Clear` (extend `tools/frida/probe_present.py`). Not required for this plan.
|
||||
|
||||
## Self-review (done while writing)
|
||||
|
||||
- **Spec coverage:** driving model A → Task 3 (thread kept); `IHost.FrameYield` → Task 2; op-budget throttle on host clock → Tasks 1+3; `Sleep` on clock + tween on clock → Task 3; single `Speed` hook (no Ctrl wiring) → FrameClock (Task 1) + tween (Task 3); parity via headless no-op → Task 2 Steps 8-9; present stays no-op → unchanged (no task touches `0x20c`, correct). All covered.
|
||||
- **Placeholder scan:** all code blocks are complete; the only "find these files" step (Task 2 Step 1) is a grep with the expected file list given.
|
||||
- **Type consistency:** `FrameClock.NowMs/Speed/OpsPerFrame/Advance/EffectiveBudget` used identically in Tasks 1 and 3; `FrameYield()` signature identical in IHost (Task 2) and GodotAdvHost (Task 3); `PulseFrame()` defined in Task 3 Step 1c and called in Step 2c.
|
||||
149
docs/superpowers/specs/2026-07-08-frame-stepped-vm-design.md
Normal file
149
docs/superpowers/specs/2026-07-08-frame-stepped-vm-design.md
Normal file
@@ -0,0 +1,149 @@
|
||||
# Frame-stepped VM — design spec
|
||||
|
||||
- **Date:** 2026-07-08
|
||||
- **Status:** design (awaiting review → implementation plan)
|
||||
- **Area:** `Age.Engine` VM/host seam + Godot frontend
|
||||
- **Related RE:** `docs/engine-re.md` — "Frame cadence — the interpreter tick" + "Frame cadence — live measurement"; `docs/phase-a-slice-plan.md` — "opening speed-through / engine-cadence"
|
||||
|
||||
## Problem
|
||||
|
||||
The SC0000 opening **visibly speeds through** in our Godot port. Root cause (confirmed): our VM runs on a
|
||||
**free-running background thread** (`Task.Run(() => vm.Run())` in `godot/Main.cs`) with no binding to real
|
||||
time. Only `WaitForInput` and `Sleep` pause it; the main-thread compositor samples `GfxState` at ~60 fps. So
|
||||
the VM executes a page's opcodes — and any auto-playing stretch — in microseconds, and the compositor only
|
||||
catches the final state. Distinct visible states that aren't separated by a `sleep`/`wait` collapse.
|
||||
|
||||
The native engine is cooperative and **rate-limited**: the interpreter advances a bounded number of opcodes
|
||||
per main-loop iteration; visible state therefore changes over real wall-clock time. Our fix must give the VM
|
||||
the same bounded wall-clock execution rate.
|
||||
|
||||
## Live measurement (Frida, 2026-07-08 — the basis for the mechanism)
|
||||
|
||||
Read-only / import-only instrumentation of the running game (`tools/frida/probe_frame_cadence.py`,
|
||||
`probe_present.py`; see `docs/engine-re.md` "Frame cadence — live measurement"):
|
||||
|
||||
- **Execution is rate-limited, not free-running.** Active interpreter rate ≈ **1,788 operand-fetches/sec**
|
||||
(peak ~5,796) in normal play — orders of magnitude below an unthrottled interpreter, so the engine is
|
||||
pacing itself.
|
||||
- **Fast-forward (Ctrl) scales the rate ~4×** (≈7,738/sec avg, peak ~15,572), gated by the engine's own skip
|
||||
bit `flags & 0x8000000`. It runs *more ops per unit time*, it does not skip content.
|
||||
- **Rendering is Direct3D 9, uncapped.** `ddraw.dll` is not loaded; the game renders via `d3d9.dll`.
|
||||
`IDirect3DDevice9::Present` fires ~**1,908/sec** (no vsync), `BeginScene`/`EndScene` never fire → it's a
|
||||
**2D StretchRect-style compositor**. So there is **no fixed display-frame cadence** to match; `Present`
|
||||
rate ≈ op rate (~1 op per present). The meaningful pacing quantity is the **wall-clock op rate**, not a
|
||||
per-display-frame budget.
|
||||
|
||||
**Correction this supersedes:** an earlier draft of this spec used `present` (op `0x20c`) as the frame
|
||||
boundary. Measurement killed that — `present` occurs only 7× statically / ~2× per opening run in *our*
|
||||
bytecode path, and the native present is uncapped anyway. The mechanism is now a wall-clock op-rate throttle.
|
||||
|
||||
## Goals
|
||||
|
||||
1. The VM advances at a bounded **wall-clock execution rate** (~1,800 ops/sec target, tunable) so visible
|
||||
state changes over real time instead of instantly.
|
||||
2. Keep the host behind `IHost` and the VM engine-agnostic — pacing lives at the seam + in the Godot host.
|
||||
3. Preserve byte-identical headless parity (Steps / emitted lines / halt reasons) across all non-Godot hosts
|
||||
and the existing oracles (`sweep`, `--selftest`, engine tests).
|
||||
4. Route all timing (the throttle, `sleep`, the anim tween) through **one host-owned clock** so a future
|
||||
speed multiplier scales everything coherently.
|
||||
|
||||
## Non-goals (YAGNI)
|
||||
|
||||
- **The Ctrl key / speed multiplier is NOT wired.** Build the single-clock seam so the ~4× multiplier is a
|
||||
trivial later addition; no key handler, no multiplier value now. When added it must be **ADV-mode-scoped**
|
||||
(the native governor does not speed up gameplay/menus), so no global-speed assumption.
|
||||
- **No present/`0x20c` gating** (rare + native present is uncapped), **no transitions/tweens for CG pacing**
|
||||
(CGs hard-swap; back-to-back loads are layers that correctly composite into one image), **no pull-based VM
|
||||
rewrite** (background thread stays), **no automated pacing assertion** (validated by the user on the build).
|
||||
|
||||
## Design
|
||||
|
||||
### Driving model (approach A: background thread + host-timed throttle)
|
||||
|
||||
The VM keeps its background thread and its recursive `Run`/`RunFrame`/`Step` unchanged. The only VM change is
|
||||
**one host call per executed opcode**; all pacing policy lives in the Godot host, mirroring the existing
|
||||
`WaitForInput`/`Sleep` suspend pattern (VM thread blocks on a host primitive; main-thread `_Process` releases
|
||||
it).
|
||||
|
||||
### Interface change (`IHost`)
|
||||
|
||||
Add **one** method:
|
||||
|
||||
```csharp
|
||||
void FrameYield(); // called by the VM after each executed opcode; host paces / no-ops
|
||||
```
|
||||
|
||||
`Sleep(long)` keeps its signature (only its Godot body changes to use the host clock). `present-frame`
|
||||
(`0x20c`) stays a no-op — it is NOT a gate. No other `IHost` changes.
|
||||
|
||||
### VM change
|
||||
|
||||
In `RunFrame`, after each `Step`, call `_host.FrameYield()`. That is the whole VM change — the VM does not
|
||||
know the budget, the rate, or the multiplier; it just offers a yield point per opcode. (If per-op virtual
|
||||
dispatch shows up in headless profiling, batch to every K ops — parity is unaffected either way.)
|
||||
|
||||
### Host-owned scalable clock + op-budget throttle (Godot)
|
||||
|
||||
The Godot host owns a single **`FrameClock`** (a pure, Godot-independent class in `Age.Engine/Hosting`, so it
|
||||
is unit-testable and off the `Vm` seam): monotonic virtual ms + a `Speed` factor (default `1.0`; the future
|
||||
Ctrl hook). `Advance(realDeltaSeconds)` is called once per `_Process`. All host timing reads only this clock:
|
||||
|
||||
- **`FrameYield()` (the throttle):** increment an op counter; when it reaches `opsPerFrame` (the budget,
|
||||
scaled by `Speed`), block the VM thread until the next `_Process` advances the clock, then reset. So the VM
|
||||
runs ~`opsPerFrame` ops per real frame ⇒ ~`opsPerFrame × displayFps` ops/sec. Start `opsPerFrame ≈ 30`
|
||||
(≈1,800 ops/sec at 60 fps) — **tunable**, calibrated by eye.
|
||||
- **`Sleep(ms)`:** block until the clock advances `ms` (replaces the raw `Thread.Sleep`).
|
||||
- **anim tween:** reads `FrameClock` instead of Godot's raw wall clock.
|
||||
|
||||
Because the throttle, sleeps, and tweens share one clock, a future `Speed = 4` scales all three together — the
|
||||
~4× fast-forward — with no desync, and the multiplier stays entirely inside the host.
|
||||
|
||||
### Concurrency
|
||||
|
||||
Same shape as `WaitForInput`: the VM background thread blocks on a host primitive; `_Process` advances the
|
||||
clock and releases waiters. `GfxState` access is already serialized on its re-entrant lock (sleep-race fix);
|
||||
the VM now runs concurrently with the compositor for real durations, so that lock discipline is relied upon
|
||||
(covered by `GfxStateConcurrencyTests`).
|
||||
|
||||
## Parity (hard constraint)
|
||||
|
||||
- All non-Godot hosts implement `FrameYield()` as an **empty method** and keep `Sleep()` a no-op. The VM never
|
||||
blocks headless → `Steps`, emitted lines, halt reasons are byte-identical.
|
||||
- Regression gates unchanged: `Age.Cli sweep` (284 exit / 13 STEP-LIMIT), Godot `--selftest`, engine tests.
|
||||
- A trace-diff on a synthetic scene proves the added per-op `FrameYield()` (no-op headless) changes nothing
|
||||
the oracle observes.
|
||||
|
||||
## Testing (synthesize, don't disable)
|
||||
|
||||
- **`FrameClock` unit tests** — deterministic `Advance` math + budget/deadline crossing at `Speed=1` and other
|
||||
values; no real sleeping.
|
||||
- **Throttle gate test with a fake-clock host** — a synthesized scene (`ScriptAssembler`) run through a test
|
||||
host whose clock is advanced by hand; assert the VM blocks after `opsPerFrame` ops and resumes on the next
|
||||
clock advance, and that `Speed` scales the budget.
|
||||
- **Parity trace-diff** — synthetic scene through the no-op headless path == pre-change baseline.
|
||||
- **Regression** — `sweep` counts unchanged; `--selftest` green.
|
||||
- **Pacing acceptance** — the user validates the finished live build (tunes `opsPerFrame` by eye).
|
||||
|
||||
## Affected components
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `engine/Age.Engine/Hosting/IHost.cs` | add `void FrameYield()` |
|
||||
| `engine/Age.Engine/Hosting/FrameClock.cs` | **new** — pure virtual clock (`NowMs`, `Speed`, `Advance`); the op counter + thread-blocking live in the Godot host, not here |
|
||||
| `engine/Age.Engine/Vm/VirtualMachine.cs` | call `_host.FrameYield()` after each `Step` in `RunFrame` |
|
||||
| `engine/Age.Engine/Hosting/CaptureHost.cs` + all other non-Godot hosts | add empty `FrameYield()` |
|
||||
| `godot/GodotAdvHost.cs` | `FrameYield()` = op-budget throttle on `FrameClock`; `Sleep()` blocks on `FrameClock` |
|
||||
| `godot/Main.cs` | own `FrameClock`; `Advance(delta)` + release waiters each `_Process`; anim tween reads `FrameClock` |
|
||||
| `engine/Age.Engine.Tests/` | new `FrameClock`, throttle-gate, parity trace-diff tests |
|
||||
|
||||
## Risks / open items
|
||||
|
||||
- **`opsPerFrame` tuning:** the initial ~30 (≈1,800 ops/sec) comes from live measurement; final value is
|
||||
dialed in visually. Op *cost* varies (a `call-script` vs a `mov`), so a pure op count is an approximation —
|
||||
acceptable for pacing, tune by eye.
|
||||
- **Per-op host call overhead** in headless (empty `FrameYield`): negligible expected; batch to every K ops if
|
||||
profiling disagrees. Parity holds regardless.
|
||||
- **Concurrency:** the VM now runs concurrently with the compositor for real durations; relies on the existing
|
||||
`GfxState` lock — watch for unlocked shared state during validation.
|
||||
- **Stretch (only if needed):** confirm the 2D-composite model by hooking `StretchRect`/`Clear` — not required
|
||||
for the fix.
|
||||
189
tools/frida/probe_frame_cadence.py
Normal file
189
tools/frida/probe_frame_cadence.py
Normal file
@@ -0,0 +1,189 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Live frame-cadence probe (docs/engine-re.md "Frame cadence"). Pins the native cadence — execution
|
||||
rate vs displayed-frame rate, and frame timing — so the frame-stepped-VM fix picks its mechanism from
|
||||
data instead of by feel.
|
||||
|
||||
SAFE pattern (matches capture_gfx_objects.py, which runs without crashing): plain-JS hooks only, no
|
||||
CModule; the only engine-code hook is the PROVEN operand-fetch helper `0x41b940` (fires per opcode,
|
||||
ecx = context) used to (a) grab the context pointer once and (b) count execution rate. Frame timing
|
||||
comes from SYSTEM-DLL hooks (user32 message pump — never engine code, never anti-tamper). Engine state
|
||||
(coroutine PC, run-state flags, sleep timer) is READ-ONLY polled. Nothing patches engine code beyond the
|
||||
one address our other scripts already prove is safe.
|
||||
(Lesson from the crash: a CModule hook on the hottest engine fn jumped into a bad callback pointer and
|
||||
killed the game instantly. Plain-JS hooks on the proven address are the reliable path here.)
|
||||
|
||||
JS samples counters + Ctrl-key state every 250 ms; Python buckets by Ctrl-held so ONE run captures both
|
||||
the normal and the fast-forward cadence.
|
||||
|
||||
Run (game running, sitting in an ADV scene, e.g. SC0000 line 1):
|
||||
py -3.11 -X utf8 tools/frida/probe_frame_cadence.py [seconds] [proc]
|
||||
Protocol: let it play NORMALLY the first half, then HOLD Ctrl the rest. Auto-buckets by Ctrl state.
|
||||
Writes build/frida-frame-cadence.jsonl.
|
||||
"""
|
||||
import json
|
||||
import statistics
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
OUT = REPO / "build" / "frida-frame-cadence.jsonl"
|
||||
|
||||
OPFETCH_OFF = 0x1b940 # operand-fetch helper (0x41b940); per-op, ecx=ctx. PROVEN-safe hook.
|
||||
IDX_OFF = 0x53d14 # current coroutine index
|
||||
PC_BASE = 0x53d2c # per-coroutine record base; +idx*0x78 holds the PC pointer (deref = opcode)
|
||||
PC_STRIDE = 0x78
|
||||
FLAGS_OFF = 0xa0ce4 # interpreter run-state flags word
|
||||
SLEEP_ACTIVE= 0x5f30c # sleep timer active flag (ctx+0x5f304 + 8)
|
||||
|
||||
JS = r"""
|
||||
const OPFETCH_OFF=%d, IDX_OFF=%d, PC_BASE=%d, PC_STRIDE=%d, FLAGS_OFF=%d, SLEEP_ACTIVE=%d;
|
||||
const mod = Process.getModuleByName('AGE.EXE');
|
||||
let ctx = null, ops = 0;
|
||||
|
||||
// PROVEN-safe engine hook (same address capture_gfx_objects.py uses): grab ctx once + count exec rate.
|
||||
Interceptor.attach(mod.base.add(OPFETCH_OFF), {
|
||||
onEnter(args){ ops++; if (ctx === null) { ctx = this.context.ecx; send({kind:'ctx', ctx: ctx.toString()}); } }
|
||||
});
|
||||
|
||||
// Frame/loop markers + timing sources — SYSTEM DLL exports only (never engine code).
|
||||
const user32 = Process.getModuleByName('user32.dll');
|
||||
const cnt = { peekA:0, peekW:0, getA:0, getW:0, tgt:0, gtc:0, qpc:0 };
|
||||
function hook(mod, name, key){ let p=null; try{ p=mod.findExportByName(name);}catch(e){} if (p) Interceptor.attach(p, { onEnter(){ cnt[key]++; } }); return !!p; }
|
||||
const have = { peekA:hook(user32,'PeekMessageA','peekA'), peekW:hook(user32,'PeekMessageW','peekW'),
|
||||
getA:hook(user32,'GetMessageA','getA'), getW:hook(user32,'GetMessageW','getW') };
|
||||
// timing sources — whichever fires ~60/sec is the frame clock (the loop reads it once per displayed frame)
|
||||
let winmm=null; try { winmm = Process.getModuleByName('winmm.dll'); } catch(e){}
|
||||
const k32 = Process.getModuleByName('kernel32.dll');
|
||||
if (winmm) hook(winmm,'timeGetTime','tgt');
|
||||
hook(k32,'GetTickCount','gtc');
|
||||
hook(k32,'QueryPerformanceCounter','qpc');
|
||||
const GetAsyncKeyState = new NativeFunction(user32.findExportByName('GetAsyncKeyState'), 'int16', ['int']);
|
||||
|
||||
send({kind:'ready', base: mod.base.toString(), have: have, winmm: !!winmm});
|
||||
|
||||
setInterval(() => {
|
||||
let idx=-1, pc=null, flags=null, sleeping=null;
|
||||
if (ctx) {
|
||||
try { idx = ctx.add(IDX_OFF).readS32(); } catch(e){}
|
||||
try { if (idx>=0 && idx<64) pc = ctx.add(PC_BASE + idx*PC_STRIDE).readPointer().toString(); } catch(e){}
|
||||
try { flags = ctx.add(FLAGS_OFF).readU32(); } catch(e){}
|
||||
try { sleeping = ctx.add(SLEEP_ACTIVE).readU32(); } catch(e){}
|
||||
}
|
||||
// fast-forward = physical Ctrl OR the engine's own skip bit (flags & 0x8000000)
|
||||
const ffKey = (GetAsyncKeyState(0x11) & 0x8000) !== 0;
|
||||
const ffBit = (flags !== null) && ((flags & 0x8000000) !== 0);
|
||||
send({ t: Date.now(), ops: ops, idx: idx, pc: pc, flags: flags, sleeping: sleeping,
|
||||
peekA: cnt.peekA, peekW: cnt.peekW, getA: cnt.getA, getW: cnt.getW,
|
||||
tgt: cnt.tgt, gtc: cnt.gtc, qpc: cnt.qpc,
|
||||
ff: ffKey || ffBit, ffKey: ffKey, ffBit: ffBit, hasctx: ctx !== null });
|
||||
}, 250);
|
||||
""" % (OPFETCH_OFF, IDX_OFF, PC_BASE, PC_STRIDE, FLAGS_OFF, SLEEP_ACTIVE)
|
||||
|
||||
|
||||
def capture(seconds, proc):
|
||||
import frida
|
||||
OUT.parent.mkdir(parents=True, exist_ok=True)
|
||||
samples = []
|
||||
|
||||
def on_message(msg, data):
|
||||
if msg.get("type") == "error":
|
||||
print("[frida-error]", msg.get("description")); return
|
||||
if msg.get("type") != "send":
|
||||
return
|
||||
pl = msg["payload"]
|
||||
if pl.get("kind") == "ready":
|
||||
print(f"[frida] hooks live @ base {pl['base']}; message-api present: {pl['have']}"); return
|
||||
if pl.get("kind") == "ctx":
|
||||
print(f"[frida] captured engine ctx = {pl['ctx']}"); return
|
||||
samples.append(pl)
|
||||
|
||||
target = int(proc) if str(proc).isdigit() else proc
|
||||
try:
|
||||
session = frida.attach(target)
|
||||
except frida.ProcessNotFoundError:
|
||||
procs = frida.get_local_device().enumerate_processes()
|
||||
print("[frida] not found. .exe processes:",
|
||||
[(p.pid, p.name) for p in procs if p.name.lower().endswith(".exe")])
|
||||
return 2
|
||||
script = session.create_script(JS)
|
||||
script.on("message", on_message)
|
||||
script.load()
|
||||
print(f"[frida] attached to {proc}; capturing {seconds}s.")
|
||||
print(" >>> Play NORMALLY the first half, then HOLD Ctrl the rest. <<<")
|
||||
try:
|
||||
for i in range(seconds):
|
||||
time.sleep(1)
|
||||
if i == seconds // 2:
|
||||
print(" --- halfway: start holding Ctrl now ---")
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
try:
|
||||
session.detach()
|
||||
except Exception:
|
||||
pass
|
||||
with open(OUT, "w", encoding="utf-8") as f:
|
||||
for s in samples:
|
||||
f.write(json.dumps(s) + "\n")
|
||||
report(samples)
|
||||
return 0
|
||||
|
||||
|
||||
def report(samples):
|
||||
if len(samples) < 3:
|
||||
print(f"[!] only {len(samples)} samples."); return
|
||||
if not samples[-1].get("hasctx"):
|
||||
print("[!!] never captured ctx — the operand-fetch hook did not fire (game idle, or not executing).")
|
||||
keys = ("ops", "peekA", "getA", "tgt", "gtc", "qpc")
|
||||
rows = []
|
||||
for a, b in zip(samples, samples[1:]):
|
||||
dt = (b["t"] - a["t"]) / 1000.0
|
||||
if dt <= 0:
|
||||
continue
|
||||
r = {"dt": dt, "ff": b.get("ff", False), "flags": b.get("flags"),
|
||||
"pc_moved": a.get("pc") != b.get("pc")}
|
||||
for k in keys:
|
||||
r[k] = (b.get(k, 0) - a.get(k, 0)) / dt
|
||||
rows.append(r)
|
||||
tot_ops = samples[-1]["ops"] - samples[0]["ops"]
|
||||
print(f"\n=== frame-cadence report ({len(rows)} intervals, {tot_ops} operand-fetches) ===")
|
||||
# candidate frame signals: prefer a timing source that sits ~30-120/sec (loop reads clock once/frame)
|
||||
for label, want in (("NORMAL", False), ("FAST-FWD (Ctrl / skip-bit)", True)):
|
||||
b = [r for r in rows if r["ff"] == want]
|
||||
# for cadence, use only ACTIVE intervals (ops>0) so parked time doesn't dilute the numbers
|
||||
act = [r for r in b if r["ops"] > 0]
|
||||
if not b:
|
||||
print(f"\n {label}: no samples"); continue
|
||||
def m(rs, k): return statistics.mean(r[k] for r in rs) if rs else 0.0
|
||||
cand = {"timeGetTime": m(act, "tgt"), "GetTickCount": m(act, "gtc"),
|
||||
"QueryPerfCounter": m(act, "qpc"), "PeekMessageA": m(act, "peekA"), "GetMessageA": m(act, "getA")}
|
||||
frame_name, frame_rate = None, 0.0
|
||||
for nm, v in cand.items():
|
||||
if 20 <= v <= 200:
|
||||
frame_name, frame_rate = nm, v; break
|
||||
ops_active = m(act, "ops")
|
||||
peak = max((r["ops"] for r in b), default=0)
|
||||
print(f"\n {label} [{len(b)} intervals, {len(act)} active]")
|
||||
print(f" operand-fetches/sec (active avg / peak): {ops_active:.0f} / {peak:.0f}")
|
||||
print(" timing-source rates (active avg/sec): " +
|
||||
", ".join(f"{nm}={v:.1f}" for nm, v in cand.items()))
|
||||
if frame_name:
|
||||
print(f" -> frame signal ~ {frame_name} @ {frame_rate:.1f}/sec ; "
|
||||
f"exec/frame = {ops_active/frame_rate:.1f} (peak {peak/frame_rate:.1f})")
|
||||
else:
|
||||
print(" -> no timing source in 20-200/sec band; frame rate still unresolved.")
|
||||
flg = [f"0x{r['flags']:x}" for r in b if isinstance(r["flags"], int)]
|
||||
print(f" flags seen: {sorted(set(flg))}")
|
||||
print("\n Stable exec/frame => fixed op-budget cadence; peak >> active-avg with parking => run-until-yield.")
|
||||
print(" FAST-FWD vs NORMAL exec/frame shows how the ADV governor scales the slice.")
|
||||
|
||||
|
||||
def main():
|
||||
args = [a for a in sys.argv[1:] if not a.startswith("-")]
|
||||
seconds = int(args[0]) if args and args[0].isdigit() else 30
|
||||
proc = next((a for a in args if not a.isdigit()), "AGE.EXE")
|
||||
return capture(seconds, proc)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
179
tools/frida/probe_present.py
Normal file
179
tools/frida/probe_present.py
Normal file
@@ -0,0 +1,179 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Pin the displayed-frame rate by hooking the present path (docs/engine-re.md "Frame cadence").
|
||||
|
||||
The frame-cadence probe showed the engine spin-waits on timeGetTime (~118k/sec) and busy-pumps
|
||||
PeekMessageA (~12k/sec), so no message/timing API fires once per frame. The true per-frame signal is
|
||||
the present — a DirectDraw surface Blt/Flip (a COM vtable method, not an export) or, if windowed-GDI,
|
||||
a gdi32 blit. Both live in SYSTEM DLLs, so hooking them is anti-tamper-safe (unlike engine code).
|
||||
|
||||
Approach: grab ctx via the proven operand-fetch hook (0x41b940), scan the engine context for pointers
|
||||
whose vtable lands in ddraw.dll (= surface COM objects), and hook Blt(vtbl[5]) / BltFast(vtbl[7]) /
|
||||
Flip(vtbl[11]) on each distinct vtable found. Also hook gdi32 blits (BitBlt/StretchBlt/StretchDIBits/
|
||||
SetDIBitsToDevice). Report each candidate's call rate; the one that sits at a steady ~display rate
|
||||
(30-120/sec) is the frame signal.
|
||||
|
||||
Run (game running, sitting in / actively playing an ADV scene):
|
||||
py -3.11 -X utf8 tools/frida/probe_present.py [seconds]
|
||||
Just play normally (no Ctrl needed — frame rate is mode-independent). Writes build/frida-present.jsonl.
|
||||
"""
|
||||
import json
|
||||
import statistics
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
OUT = REPO / "build" / "frida-present.jsonl"
|
||||
|
||||
OPFETCH_OFF = 0x1b940 # operand-fetch (0x41b940), ecx=ctx — proven-safe engine hook to grab ctx
|
||||
CTX_SCAN = 0x200000 # bytes of the engine context to scan for the d3d9 device pointer
|
||||
|
||||
JS = r"""
|
||||
const OPFETCH_OFF = %d, CTX_SCAN = %d;
|
||||
const mod = Process.getModuleByName('AGE.EXE');
|
||||
let ctx = null, scanned = false;
|
||||
const cnt = {}; // label -> count
|
||||
function bump(label){ cnt[label] = (cnt[label]||0) + 1; }
|
||||
|
||||
// IDirect3DDevice9 vtable slot indices (0-based): Present=17, BeginScene=41, EndScene=42.
|
||||
// (Present/EndScene fire once per displayed frame -> the frame clock.)
|
||||
const SLOTS = { 17: 'd3d9.Present', 42: 'd3d9.EndScene', 41: 'd3d9.BeginScene' };
|
||||
|
||||
function scanForSurfaces(){
|
||||
let d3d = null; try { d3d = Process.getModuleByName('d3d9.dll'); } catch(e){}
|
||||
if (!d3d) { send({kind:'note', msg:'d3d9.dll not loaded'}); return; }
|
||||
const lo = d3d.base, hi = d3d.base.add(d3d.size);
|
||||
const inD3D = (p) => p.compare(lo) >= 0 && p.compare(hi) < 0;
|
||||
const vtables = new Set();
|
||||
for (let off = 0; off < CTX_SCAN; off += 4) {
|
||||
let obj; try { obj = ctx.add(off).readPointer(); } catch(e){ continue; }
|
||||
if (obj.isNull() || inD3D(obj)) continue; // want a heap COM object whose *obj is a d3d9 vtable
|
||||
let vt; try { vt = obj.readPointer(); } catch(e){ continue; }
|
||||
if (!inD3D(vt)) continue;
|
||||
vtables.add(vt.toString());
|
||||
}
|
||||
// Identify the DEVICE: its vtable has ~119 methods all in d3d9; the factory/textures far fewer.
|
||||
let best = null, bestN = 0;
|
||||
for (const vs of vtables) {
|
||||
const vt = ptr(vs);
|
||||
let n = 0;
|
||||
for (let s = 0; s < 120; s++) {
|
||||
let fn; try { fn = vt.add(s*4).readPointer(); } catch(e){ break; }
|
||||
if (inD3D(fn)) n++; else if (s > 3) break; // stop at first non-d3d slot past IUnknown
|
||||
}
|
||||
if (n > bestN) { bestN = n; best = vt; }
|
||||
}
|
||||
send({kind:'note', msg:'d3d9 vtables in ctx: ' + vtables.size + '; largest method run = ' + bestN
|
||||
+ (bestN >= 60 ? ' (device found)' : ' (no device-sized vtable)')});
|
||||
if (best && bestN >= 60) {
|
||||
for (const slot in SLOTS) {
|
||||
let fn; try { fn = best.add(parseInt(slot)*4).readPointer(); } catch(e){ continue; }
|
||||
if (!inD3D(fn)) continue;
|
||||
const label = SLOTS[slot];
|
||||
try { Interceptor.attach(fn, { onEnter(){ bump(label); } }); send({kind:'note', msg:'hooked '+label+' @ '+fn}); } catch(e){}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Interceptor.attach(mod.base.add(OPFETCH_OFF), {
|
||||
onEnter(){ if (ctx === null) { ctx = this.context.ecx; send({kind:'ctx', ctx: ctx.toString()});
|
||||
if (!scanned) { scanned = true; scanForSurfaces(); } } }
|
||||
});
|
||||
|
||||
// GDI blit fallback (system exports, safe)
|
||||
const gdi = Process.getModuleByName('gdi32.dll');
|
||||
for (const nm of ['BitBlt','StretchBlt','StretchDIBits','SetDIBitsToDevice']) {
|
||||
let p=null; try{ p = gdi.findExportByName(nm); }catch(e){}
|
||||
if (p) Interceptor.attach(p, { onEnter(){ bump('gdi.'+nm); } });
|
||||
}
|
||||
const user32 = Process.getModuleByName('user32.dll');
|
||||
for (const nm of ['UpdateWindow']) {
|
||||
let p=null; try{ p = user32.findExportByName(nm); }catch(e){}
|
||||
if (p) Interceptor.attach(p, { onEnter(){ bump('user32.'+nm); } });
|
||||
}
|
||||
|
||||
let ddrawInfo = null;
|
||||
try { const dd = Process.getModuleByName('ddraw.dll'); ddrawInfo = dd.base.toString() + ' size ' + dd.size; } catch(e){}
|
||||
const gfxMods = Process.enumerateModules().filter(m => /ddraw|d3d|dinput|dsound|d8thunk/i.test(m.name)).map(m => m.name);
|
||||
send({kind:'ready', base: mod.base.toString(), ddraw: ddrawInfo, gfxMods: gfxMods});
|
||||
setInterval(() => { send({ kind:'tick', t: Date.now(), cnt: cnt, hasctx: ctx!==null }); }, 250);
|
||||
""" % (OPFETCH_OFF, CTX_SCAN)
|
||||
|
||||
|
||||
def capture(seconds, proc):
|
||||
import frida
|
||||
OUT.parent.mkdir(parents=True, exist_ok=True)
|
||||
ticks = []
|
||||
|
||||
def on_message(msg, data):
|
||||
if msg.get("type") == "error":
|
||||
print("[frida-error]", msg.get("description")); return
|
||||
if msg.get("type") != "send":
|
||||
return
|
||||
pl = msg["payload"]
|
||||
k = pl.get("kind")
|
||||
if k == "ready":
|
||||
print(f"[frida] hooks live @ {pl['base']}; ddraw.dll: {pl.get('ddraw') or 'NOT loaded'}; "
|
||||
f"gfx modules: {pl.get('gfxMods')}"); return
|
||||
if k == "ctx": print(f"[frida] ctx = {pl['ctx']}"); return
|
||||
if k == "note": print(f"[frida] {pl['msg']}"); return
|
||||
if k == "tick": ticks.append(pl)
|
||||
|
||||
target = int(proc) if str(proc).isdigit() else proc
|
||||
try:
|
||||
session = frida.attach(target)
|
||||
except frida.ProcessNotFoundError:
|
||||
print("[frida] AGE.EXE not found."); return 2
|
||||
script = session.create_script(JS)
|
||||
script.on("message", on_message)
|
||||
script.load()
|
||||
print(f"[frida] attached; capturing {seconds}s — just play normally.")
|
||||
try:
|
||||
for _ in range(seconds):
|
||||
time.sleep(1)
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
try:
|
||||
session.detach()
|
||||
except Exception:
|
||||
pass
|
||||
with open(OUT, "w", encoding="utf-8") as f:
|
||||
for t in ticks:
|
||||
f.write(json.dumps(t) + "\n")
|
||||
report(ticks)
|
||||
return 0
|
||||
|
||||
|
||||
def report(ticks):
|
||||
if len(ticks) < 3:
|
||||
print(f"[!] only {len(ticks)} ticks."); return
|
||||
labels = sorted({k for t in ticks for k in t.get("cnt", {})})
|
||||
if not labels:
|
||||
print("[!] no present-candidate hooks fired — no d3d9 device found in ctx and no GDI blits."); return
|
||||
print(f"\n=== present-rate report ({len(ticks)} ticks) ===")
|
||||
rates = {}
|
||||
for lab in labels:
|
||||
rr = []
|
||||
for a, b in zip(ticks, ticks[1:]):
|
||||
dt = (b["t"] - a["t"]) / 1000.0
|
||||
if dt <= 0:
|
||||
continue
|
||||
rr.append((b["cnt"].get(lab, 0) - a["cnt"].get(lab, 0)) / dt)
|
||||
active = [x for x in rr if x > 0]
|
||||
rates[lab] = (statistics.mean(active) if active else 0.0, max(rr, default=0), len(active))
|
||||
for lab in sorted(labels, key=lambda l: -rates[l][0]):
|
||||
avg, peak, n = rates[lab]
|
||||
flag = " <-- frame signal?" if 20 <= avg <= 130 else ""
|
||||
print(f" {lab:22s} active-avg {avg:8.1f}/sec peak {peak:8.1f} ({n} active){flag}")
|
||||
print("\n The candidate at a steady ~display rate (≈60/sec, maybe ~100) is the present = frame clock.")
|
||||
print(" Combine with the cadence probe's ops/sec to get ops-per-frame.")
|
||||
|
||||
|
||||
def main():
|
||||
args = [a for a in sys.argv[1:] if not a.startswith("-")]
|
||||
seconds = int(args[0]) if args and args[0].isdigit() else 12
|
||||
return capture(seconds, "AGE.EXE")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user