diff --git a/docs/engine-re.md b/docs/engine-re.md index b61b09c..41689ca 100644 --- a/docs/engine-re.md +++ b/docs/engine-re.md @@ -94,11 +94,54 @@ handler through the dispatch table (`ctx[0x26c93 + op]`). The raw VA is off by w --- +### op `0x03` (`call-script`) is a raw index into the SYS4INI file table — SOLVED (2026-07-07) + +The long-deferred `call-script ` registry (`name-resolution.md §1`) is cracked. Resolved through +the dispatch table (op `0x03` → `ctx[0x26c93+3]` = **`FUN_0041bc90`**), then the loader/resolver chain: + +- **`FUN_0041bc90`** (handler): fetches operand 1 (the id), bounds-checks call depth (≤ 0x26), pushes + a script frame, and calls the loader. +- **`FUN_0040e980`** (loader): opens the resource by id, reads the **0x20-byte SYS4 header**, checks + magic, allocates per-frame code/local buffers from the header var-counts, reads the bytecode body, + and pushes a script frame (**stride 0x1e = 30 dwords**, indexed by `ctx[0x14f45]`). Returns to the + caller when the callee ends. +- **`FUN_0044f390`** (resolver — the key): `record = [ctx+0x414] + id*0x50`. The record is exactly the + **SYS4INI 80-byte layout** `{name[64], arc_id@0x40, file_number@0x44, offset@0x48, size@0x4c}` + (count = `[ctx+0x40c]`, archive-name table = `[ctx+0x410]`). It tries a **loose override first** + (`CreateFileA` on `record.name` → the mod/patch hook point), else opens archive + `[record.arc_id*0x100 + ctx+0x410]`, `SetFilePointer` to `record.offset`, size = `record.size`. + High-byte-tagged ids (`id & 0xff000000`) select an alternate pack via `[ctx+0x3028]` — **unused by + the corpus** (0/297 ids carry a high byte). + +**So `call-script ` = a direct RAW index into the SYS4INI global file table** — the same table +`parse_sys4ini.py` reads, but indexed *without* skipping `@` placeholders (13208 records, 2 +placeholders). There is **no separate on-disk id→code registry**; SYS4INI *is* the registry, and we +already had it. **Statically confirmed:** all **297/297** distinct corpus `call-script` ids resolve to +a `.BIN` script with a semantically-exact name (`0x1ab→ADDITEM`, `0x2ae7→MES`, `0x143→BUNKI`, +`0x329d→CALCREVISE`, `0x2add→CALCBTPARAM`), 0 out-of-range, 0 pack-branch. Tooling: +`parse_sys4ini.py` emits `build/callscript-names.json` (id→name); `sys4load` annotates +`call-script 0x1ab =ADDITEM.BIN`; the whole `build/disasm/*.asm` call graph now reads by name. See +`name-resolution.md §1`. + +**Companion — op `0x8f` (`call`) is INTRA-script, not cross-script.** Its handler **`FUN_0041fba0`** +sets `[frame PC @+0x53d2c] = [frame codebase @+0x53d28] + operand*4` and pushes a return address on +the per-frame return stack (`[ctx+0x552e8]`/`[ctx+0x55248]`). The operand is a **code offset within +the current script** (matches header table **T3, tag 0x8F** = local call targets). So `0x8f` is a +local JSR; only `0x03` loads another script. + +**Follow-up (functional):** the C# VM still *stubs* `call-script`. With the id→resource mapping now +known, it can be implemented for real (load the target `.BIN` from the archive via the SYS4INI record, +push a frame, run, return) — the unlock for subroutine-using scripts and, via the same path, +decision→scene (scenes are just `SCxxxx.BIN` records loaded by their SYS4INI index). + +--- + ## Native walls backlog (targets for this loop) -- **decision→scene** — how `0x62ccf`/the decision actually selects the next `SCxxxx` (re-aimed away - from `u00428010`; likely call-script-adjacent). -- **call-script dispatch** — `call-script ` → engine entry point (`name-resolution.md §1`). +- ~~**call-script dispatch**~~ — **SOLVED** (above): `call-script ` = raw SYS4INI file index. +- **decision→scene** — how `0x62ccf`/the decision selects the next `SCxxxx`. Now narrower: scenes load + via `call-script`/the same SYS4INI-index loader, so the open question is only where the decision + value is turned into a scene *id* (a caller of SCJUMP; re-aimed away from `u00428010`). - **op `0x60`** (`u0041A270`) — the rand-like value gating 1732/1755 SCJUMP decisions. - **gfx command-buffer** — the `0x212–0x21a` positioned-object subsystem (`scjump`-unrelated; the rendering drift). diff --git a/docs/name-resolution.md b/docs/name-resolution.md index b0ddbb7..3be9d75 100644 --- a/docs/name-resolution.md +++ b/docs/name-resolution.md @@ -12,10 +12,25 @@ would make it read like source. --- -## #1 — `call-script` target resolution (naming the call graph) +## #1 — `call-script` target resolution (naming the call graph) — ✅ SOLVED (2026-07-07) -**What it is.** `call-script N` (Kelebek opcode 0x03) carries a bare number — `0x329d`, -`0x2ade` — the id of an engine entry point. To render `call RECOVER` instead of +**RESOLVED via native-RE.** `call-script ` is a **direct RAW index into the SYS4INI file table** — +the very asset index we already parsed. No hidden engine registry: SYS4INI *is* the registry. Cracked +by decompiling the handler chain in Ghidra (op 0x03 → `FUN_0041bc90` → loader `FUN_0040e980` → +resolver `FUN_0044f390`, which does `record = table_base + id*0x50` over the 80-byte SYS4INI records). +**Statically confirmed:** all 297/297 distinct corpus `call-script` ids resolve to a `.BIN` script with +a semantically-exact name (`0x1ab→ADDITEM`, `0x2ae7→MES`, `0x143→BUNKI`), 0 out-of-range. Full +mechanism in `engine-re.md` (“op 0x03 (call-script)…”). Tooling: `parse_sys4ini.py` → +`build/callscript-names.json` (id→name); `sys4load` renders `call-script 0x1ab =ADDITEM.BIN`; the +`build/disasm/*.asm` call graph now reads by name. The one caveat: index the RAW SYS4INI records +(*including* the 2 `@` placeholders) — `asset-index.json` carries each entry's `raw_index` (= the id) +for exactly this. **Remaining (functional, not naming):** the C# VM still stubs `call-script` +execution; implementing it (load `.BIN` by id, push frame, run, return) is the follow-up. The original +analysis (kept below for provenance) had concluded this was engine-level and deferred — it was, and +the Ghidra loop is what resolved it. + +**What it is (original framing).** `call-script N` (Kelebek opcode 0x03) carries a bare number — +`0x329d`, `0x2ade` — the id of an engine entry point. To render `call RECOVER` instead of `call-script 0x329d` you need a table `id → (script, entry)`. **Findings (inspected 2026-07-06):** @@ -40,16 +55,18 @@ on disk to read. Resolving it needs one of: - **Find the registration path** — if a boot script assigns ids to entry points, extract it statically (SYSTEM4.BIN is far too small to hold ~13k, so it's cumulative or lives in AGE.EXE). -**Status: deferred.** Not the quick win first assumed. Belongs with the engine/dispatch work -(Phase 3), or a dedicated `SCJUMP.BIN` reverse. Until then `call-script` stays numeric. +**Status: ✅ SOLVED** (see the banner at the top of this section). It did belong with the +engine/dispatch work — the Ghidra + MCP loop resolved it via the opcode-dispatch table. **Update (2026-07-07):** SCJUMP's *decision logic* is now decoded — `(chapter_mode, guards) → decision value` — see `docs/scjump-progression.md` and `tools/scjump_decode.py`. That confirmed -SCJUMP is not the `call-script` registry (it produces a decision value, not a script id). The -decision→scene hop is **native and still unidentified** — an earlier guess that op `u00428010` -resolved it was **disproven via Ghidra** (that op is a graphics command-buffer op; see -`docs/engine-re.md`). It's the same engine-level bucket as `call-script`; the Ghidra + MCP loop (and -its recovered opcode-dispatch table) is now the tool to crack the `call-script`/script-load handler. +SCJUMP is not the `call-script` registry (it produces a decision value, not a script id). Then the +Ghidra + MCP loop **cracked `call-script` itself** (the SOLVED banner above): via the opcode-dispatch +table it walked the handler → loader → resolver and found the id is a raw SYS4INI file index. What +remains of the earlier `decision→scene` question is now narrow: scenes are `SCxxxx.BIN` records loaded +through the *same* id-indexed loader, so the only open piece is where the SCJUMP decision *value* +becomes a scene *id* (a caller of SCJUMP). The `u00428010` guess for that hop was disproven via Ghidra +(it's a graphics command-buffer op; see `docs/engine-re.md`). --- @@ -172,7 +189,7 @@ automatically (it reads `build/global-var-map.json` at load). ## How the two relate #1 names **functions** (the call graph); #2 names **data** (game state). In `RECOVER`, #1 turns -`call-script 0x329d` into a name; #2 turns `C[unit][s] = E[unit][s]` into `unit.hp[s] = -unit.maxHp[s]`. Priority reversal from the first guess: **#2 is the tractable readability lever -now** (static handholds already half-built via the `*INIT` extraction); **#1 needs the engine** -(dispatch reverse or Frida) and is deferred. +`call-script 0x329d` into `CALCREVISE.BIN`; #2 turns `C[unit][s] = E[unit][s]` into `unit.hp[s] = +unit.maxHp[s]`. Both are now largely in hand: **#1 is SOLVED** (the SYS4INI-index dispatch reverse — +turned out to need the engine, and the Ghidra loop delivered it), and **#2 has a partial static map** +(the `*INIT` handholds) that grows on demand. diff --git a/docs/opcode-reference.md b/docs/opcode-reference.md index fd5a7d2..014d8be 100644 --- a/docs/opcode-reference.md +++ b/docs/opcode-reference.md @@ -27,6 +27,36 @@ - **grounding:** source=investigation, confidence=high - **evidence:** By-ear confirmed (2026-07-06): SC0000 prologue voices play on their lines via Godot AudioStreamPlayer. Off-by-one disproven structurally: manifest interleaves graphics/voice (files[35]=EV049AA, [36]=MAN999, [37]=EV052CA, [38]=SYL0001), so files[base+id] lands voices on OGGs while files[base+id-1] would land them on .AGF graphics (silent) -- and they play, so the offset is exactly 0. Lily's lines are correctly form-gated (G[0xa57/0xa58/0xa59]) and stay silent when no form flag is seeded -- not a bug. +## control + +### 0x3 `call-script` (call-script, argc 1) +- **summary:** load & call another SYS4 script by id; id = RAW index into the SYS4INI file table (asset-index). Pushes a script frame; returns to caller when the callee ends. +- **grounding:** source=investigation, confidence=high +- **evidence:** native-RE (Ghidra): handler FUN_0041bc90 -> loader FUN_0040e980 -> resolver FUN_0044f390 indexes an 80-byte record table (base [ctx+0x414], count [ctx+0x40c]) at base+id*0x50 = the SYS4INI record layout {name[64],arc_id@0x40,file_number@0x44,offset@0x48,size@0x4c}. Confirmed statically: all 297 distinct corpus call-script ids resolve to a .BIN script with a semantically-exact name (0x1ab->ADDITEM, 0x2ae7->MES, 0x143->BUNKI, 0x329d->CALCREVISE), 0 out-of-range, 0 pack-branch. See docs/engine-re.md + name-resolution.md #1. + +op 0x03 (call-script, argc 1): `call-script `. RESOLVED — the id is a direct RAW index into +the SYS4INI global file table (the same table parse_sys4ini.py reads, but indexed WITHOUT skipping +'@' placeholders; SYS4INI has 13208 records / 2 placeholders). No separate on-disk id->code registry +exists; SYS4INI *is* the call-script registry. +Native mechanism (dispatch table `handler(op)=ctx[0x26c93+op]`, op 0x03 -> FUN_0041bc90): + 1. FUN_0041bc90 fetches operand 1 (id), bounds-checks call depth (<=0x26), pushes a frame. + 2. FUN_0040e980 (loader): opens the resource by id, reads the 0x20-byte SYS4 header, checks magic, + allocates per-frame code/local buffers from the header var-counts, reads the bytecode body, + pushes a script frame (stride 0x1e = 30 dwords, indexed by ctx[0x14f45]). + 3. FUN_0044f390 (resolver): record = [ctx+0x414] + id*0x50. Tries a LOOSE OVERRIDE first + (CreateFileA on record.name -> mod/patch hook point), else opens archive [record.arc_id*0x100 + + ctx+0x410], SetFilePointer to record.offset, size = record.size. + (High-byte-tagged ids `id & 0xff000000` select an alternate pack via [ctx+0x3028]; UNUSED by the + corpus -- 0/297 ids have a high byte.) +Companion op 0x8f `call` is INTRA-script (a local JSR), not cross-script -- see its entry. +This also names the whole call graph statically (build/callscript-names.json). + + +### 0x8f `call` (call, argc 1) +- **summary:** intra-script subroutine call (local JSR): PC = frame.codebase + operand*4; pushes a return address on the per-frame return stack. NOT cross-script (that is call-script 0x03). +- **grounding:** source=investigation, confidence=high +- **evidence:** native-RE (Ghidra): handler FUN_0041fba0 (= ctx[0x26c93+0x8f]) sets [frame PC @+0x53d2c] = [frame codebase @+0x53d28] + operand*4 and pushes ((pc-base)>>2)+3 onto the per-frame return stack ([ctx+0x552e8]/[ctx+0x55248]). Target is a code OFFSET within the current script (matches header table T3 tag 0x8F = local call targets), confirming it is a local JSR, not a script load. + ## draw ### 0x1a2 `gfx-cmd-register` (u00428010, argc 1) @@ -179,10 +209,6 @@ op 0x90 (u0041BEB0, argc 7): `0x90 x y w h tgt_a tgt_b tgt_c`. Kelebek left it " - **summary:** — - **grounding:** source=kelebek, confidence=med -### 0x3 `call-script` (call-script, argc 1) -- **summary:** — -- **grounding:** source=kelebek, confidence=med - ### 0x5 `ret` (ret, argc 0) - **summary:** — - **grounding:** source=kelebek, confidence=med @@ -375,10 +401,6 @@ op 0x90 (u0041BEB0, argc 7): `0x90 x y w h tgt_a tgt_b tgt_c`. Kelebek left it " - **summary:** — - **grounding:** source=kelebek, confidence=med -### 0x8f `call` (call, argc 1) -- **summary:** — -- **grounding:** source=kelebek, confidence=med - ### 0x93 `u00415040` (u00415040, argc 0) - **summary:** — - **grounding:** source=kelebek, confidence=low diff --git a/docs/phase-a-slice-plan.md b/docs/phase-a-slice-plan.md index 640a87a..416c760 100644 --- a/docs/phase-a-slice-plan.md +++ b/docs/phase-a-slice-plan.md @@ -307,3 +307,44 @@ is only needed for scenes that genuinely use runtime-positioned sprites — revi Build `tools/vm0.py` and get the **RECOVER unit test** green (pointer/array/control-flow correctness), then run the first linear ADV scene against the dialogue oracle. That single result tells us whether the whole VM approach executes correctly — the load-bearing question behind option 3. + +--- + +## ✅ call-script EXECUTION (2026-07-07) — subroutines now run in the C# VM + +Spec `docs/superpowers/specs/2026-07-07-callscript-vm-execution-design.md`; plan +`docs/superpowers/plans/2026-07-07-callscript-vm-execution.md`. Enabled by the native-RE finding that +`call-script ` is a raw SYS4INI file index (`docs/engine-re.md`). + +**Shipped (engine 25/25 green):** +- **`IScriptProvider`** (Hosting) + **`Sys4ScriptProvider`** (Sys4): `id → build/callscript-names.json → + name → Paths.Scripts() → Sys4Loader.Load`, cached. Injected into the VM so `Vm` never references `Sys4`. +- **`ExecFrame` refactor** of `VirtualMachine`: per-script state (script, pc, locals, intra-call stack, + per-frame emit-guard) moved into `ExecFrame`, run by a recursive `RunFrame`. Globals/Emitted/Steps stay + VM-level (shared). Emitted lines now carry their source script name. +- **`call-script` executes:** loads the child, runs it as a nested frame sharing globals, returns to the + caller at `pc+1`. `exit`/`exit-script` and empty-stack `ret` return from the frame (top frame → HALT). + Depth-capped (`VmOptions.CallDepthCap=64`; native limit 38). Shared globals are the return channel; + per-call locals are discarded on return. +- **Product paths** (`Age.Cli run/play/sweep`, `GameSession.RunScene`) inject the provider. `trace` + + the `audio`/`gfx` diagnostics stay provider-less (base-ISA oracle / built against stub behavior). + +**Design decision (refines the spec):** a VM with **no provider** falls back to the prior stub +(`host.CallScript(id); pc+1`), not a halt. This keeps every existing base-ISA test byte-identical +(they construct provider-less VMs) and needs no golden-fixture regeneration; **vm0.py retires from +oracle duty gracefully** — `trace`/`TraceDiffTests`/`WaitForInputTests` stay on the stub path as the +base-ISA guard, with zero lockstep maintenance. + +**Validation:** 6 new tests (fake-provider unit tests: return-to-caller, callee `exit` returns not +halts, shared-global visibility, per-frame local isolation, unresolved-id halt, no-provider stub; +integration: ADDILL executes ADDILLSUB+CALCREVISE and reaches its own exit; BUNKI's top-level `ret` +returns cleanly). **Corpus sweep (execution on): 284/297 exit clean, 13 STEP-LIMIT, 0 depth-cap, 0 +unresolved, 0 crashes.** The 13 STEP-LIMITs are input/state-gated ADV scenes (SC0000 etc.): executing +subroutines makes their global-writes drive caller loops that headless can't break (no input; +`WaitForInput` is a no-op) — the known state-divergence, not a call-script bug (loops hit STEP-LIMIT, +not the depth cap → recursion is bounded correctly). + +**Follow-ups (out of this slice):** wire the provider into the Godot play path (keep `--selftest` +provider-less to preserve the 186-offset gate); optionally give the `audio`/`gfx` diagnostics a +provider once their stub-era baselines are revisited; `decision→scene` (scene chaining) rides this same +loader once the SCJUMP decision→scene-id native hop is reversed. diff --git a/docs/script-inventory.md b/docs/script-inventory.md index 09ef826..3ba34e8 100644 --- a/docs/script-inventory.md +++ b/docs/script-inventory.md @@ -95,6 +95,25 @@ outside scenes lives. --- +## Call graph — scripts are addressable by `call-script ` (2026-07-07) + +`call-script ` (opcode 0x03) loads another script by a **raw index into the SYS4INI file table** +(id = the entry's `raw_index` = its global position in SYS4INI). This is the resolved call-graph +registry — there is no separate id→code table; SYS4INI is it. Mechanism: `engine-re.md` (op 0x03 +section); id→name single source: `build/callscript-names.json` (from `parse_sys4ini.py`); `sys4load` +and the regenerated `build/disasm/*.asm` corpus now render targets by name +(`call-script 0x1ab =ADDITEM.BIN`). **297 distinct scripts are called** across the corpus (3002 sites); +the hottest are `HISTORY` (backlog), `MENU`, `HIDEWIN`, `BUNKI` (branch), `MES` (message), `ADDITEM`, +`ADDEN`, `LOOK`, `RENDERMAP`. Scenes (`SCxxxx.BIN`) load through the *same* id-indexed loader. + +**Living-reference decision:** no separate generated markdown call-script reference is kept. Unlike +`opcode-reference.md` / `global-reference.md` (rendered from *curated* knowledge bases), the id→name +mapping is purely mechanical (SYS4INI index → filename) with no semantics to curate — it already lives +in the build artifact and in the named disasm corpus. Full call-graph edges (caller→callee counts) are +derivable on demand from the corpus; materialize a doc only if a consumer needs it. + +--- + ## Implications for the port 1. **Much more game logic lives in bytecode than expected.** Damage formulas diff --git a/docs/superpowers/plans/2026-07-07-callscript-vm-execution.md b/docs/superpowers/plans/2026-07-07-callscript-vm-execution.md new file mode 100644 index 0000000..d25a91c --- /dev/null +++ b/docs/superpowers/plans/2026-07-07-callscript-vm-execution.md @@ -0,0 +1,644 @@ +# call-script Execution in the C# VM — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make `call-script ` actually load and execute the target `.BIN` as a nested subroutine that shares the global bank and returns to its caller. + +**Architecture:** Introduce an `IScriptProvider` seam (implemented in `Sys4`, injected into the VM so `Vm` never references `Sys4`) that maps a call-script id (a raw SYS4INI file index) to a loaded `Script`. Refactor the VM's per-script state into an `ExecFrame` and run scripts through a recursive `RunFrame`; `call-script` recurses into a child frame. Recursion is safe because call depth is bounded (native ≤ 38). + +**Tech Stack:** C# / .NET 8, xUnit. Solution `engine/AgeEngine.sln` (projects `Age.Engine`, `Age.Cli`, `Age.Engine.Tests`). + +## Global Constraints + +- Run tests with: `dotnet test engine/AgeEngine.sln` (from `age-reimpl/`). +- Seam rule: `Vm` code references only `Model` + `Hosting` namespaces, never `Sys4`. (One assembly; enforced by convention.) +- `call-script `: `id` is a **raw index into the SYS4INI file table** (`build/callscript-names.json` maps id→name; `Paths.Scripts()` maps name→path). Confirmed: all 297 corpus ids resolve to a DATA1 `.BIN`. See `docs/engine-re.md`. +- **Deviation from spec (deliberate):** a VM with **no** `IScriptProvider` that hits `call-script` falls back to the **prior stub** (`_host.CallScript(id); pc+1`), NOT a halt. This keeps every existing base-ISA test byte-identical (they construct provider-less VMs) and avoids golden-fixture churn. Product paths always inject a provider, so execution is the real behavior where it matters. +- Frequent commits: one per task minimum. + +--- + +### Task 1: Script-provider seam + resolver + +**Files:** +- Create: `engine/Age.Engine/Hosting/IScriptProvider.cs` +- Create: `engine/Age.Engine/Sys4/Sys4ScriptProvider.cs` +- Modify: `engine/Age.Engine/Sys4/Paths.cs` (add `CallscriptNamesJson`) +- Test: `engine/Age.Engine.Tests/Sys4ScriptProviderTests.cs` + +**Interfaces:** +- Produces: `interface IScriptProvider { Script? GetById(long id); }` (namespace `Age.Engine.Hosting`). +- Produces: `Sys4ScriptProvider.Load(OpcodeTable table) -> Sys4ScriptProvider`; instance `GetById(long id) -> Script?` (cached; unknown id → null). +- Produces: `Paths.CallscriptNamesJson -> string`. +- Consumes: existing `OpcodeTable`, `Sys4Loader.Load(string, OpcodeTable)`, `Paths.Scripts()`, `Paths.Build`. + +- [ ] **Step 1: Add the path constant** + +In `engine/Age.Engine/Sys4/Paths.cs`, after the `AssetIndexJson` line (line 12), add: + +```csharp + public static string CallscriptNamesJson => Path.Combine(Build, "callscript-names.json"); +``` + +- [ ] **Step 2: Create the provider interface** + +Create `engine/Age.Engine/Hosting/IScriptProvider.cs`: + +```csharp +using Age.Engine.Model; +namespace Age.Engine.Hosting; + +/// Resolves a call-script id (a raw SYS4INI file index) to a loaded . +/// Implemented in Sys4; injected into the VM so the Vm layer never references Sys4. +public interface IScriptProvider +{ + /// The script for this id, or null if the id maps to no known script. + Script? GetById(long id); +} +``` + +- [ ] **Step 3: Write the failing test** + +Create `engine/Age.Engine.Tests/Sys4ScriptProviderTests.cs`: + +```csharp +using Age.Engine.Sys4; +using Xunit; + +public class Sys4ScriptProviderTests +{ + [Fact] + public void ResolvesKnownIdsToTheirScripts() + { + var table = OpcodeTableJson.Load(Paths.OpcodesJson); + var provider = Sys4ScriptProvider.Load(table); + + var additem = provider.GetById(0x1ab); // ADDITEM.BIN + var mes = provider.GetById(0x2ae7); // MES.BIN + Assert.NotNull(additem); + Assert.NotNull(mes); + Assert.True(additem!.Instructions.Count > 0); + Assert.Same(additem, provider.GetById(0x1ab)); // cached: same instance + Assert.Null(provider.GetById(long.MaxValue)); // unknown id + } +} +``` + +- [ ] **Step 4: Run the test to verify it fails** + +Run: `dotnet test engine/AgeEngine.sln --filter Sys4ScriptProviderTests` +Expected: FAIL (compile error — `Sys4ScriptProvider` does not exist). + +- [ ] **Step 5: Implement the provider** + +Create `engine/Age.Engine/Sys4/Sys4ScriptProvider.cs`: + +```csharp +using System.Text.Json; +using Age.Engine.Hosting; +using Age.Engine.Model; +namespace Age.Engine.Sys4; + +/// Resolves call-script ids (raw SYS4INI file indices) to loaded scripts, using +/// build/callscript-names.json (id→name) + Paths.Scripts() (name→path). Cached per id. +/// The native resolver prefers a loose override before the archive; Paths.Scripts() already +/// shadows extracted/DATA1 with root overrides, so that behavior is preserved. +public sealed class Sys4ScriptProvider : IScriptProvider +{ + private readonly OpcodeTable _table; + private readonly IReadOnlyDictionary _idToName; + private readonly Dictionary _byName; // NAME(UPPER) -> path + private readonly Dictionary _cache = new(); + + public Sys4ScriptProvider(OpcodeTable table, IReadOnlyDictionary idToName, + Dictionary byName) + { _table = table; _idToName = idToName; _byName = byName; } + + public static Sys4ScriptProvider Load(OpcodeTable table) + { + var raw = JsonSerializer.Deserialize>( + File.ReadAllText(Paths.CallscriptNamesJson)) ?? new(); + var idToName = raw.ToDictionary(kv => long.Parse(kv.Key), kv => kv.Value); + return new Sys4ScriptProvider(table, idToName, Paths.Scripts()); + } + + public Script? GetById(long id) + { + if (_cache.TryGetValue(id, out var cached)) return cached; + Script? s = null; + if (_idToName.TryGetValue(id, out var name) && + _byName.TryGetValue(name.ToUpperInvariant(), out var path)) + s = Sys4Loader.Load(path, _table); + _cache[id] = s; + return s; + } +} +``` + +- [ ] **Step 6: Run the test to verify it passes** + +Run: `dotnet test engine/AgeEngine.sln --filter Sys4ScriptProviderTests` +Expected: PASS. (Prerequisite: `build/callscript-names.json` exists — regenerate with `py -3.11 -X utf8 tools/parse_sys4ini.py` if missing.) + +- [ ] **Step 7: Commit** + +```bash +git add engine/Age.Engine/Hosting/IScriptProvider.cs engine/Age.Engine/Sys4/Sys4ScriptProvider.cs engine/Age.Engine/Sys4/Paths.cs engine/Age.Engine.Tests/Sys4ScriptProviderTests.cs +git commit -m "Add IScriptProvider + Sys4ScriptProvider (call-script id -> Script)" +``` + +--- + +### Task 2: ExecFrame refactor (no behavior change) + emitted script identity + +Refactor the VM's per-script state into an `ExecFrame` run by a recursive `RunFrame`, and tag each emitted line with its source script. call-script stays a stub in this task. All existing tests must stay green (emitted offsets/counts/halt unchanged). + +**Files:** +- Create: `engine/Age.Engine/Vm/ExecFrame.cs` +- Modify: `engine/Age.Engine/Model/Script.cs` (add `Name`) +- Modify: `engine/Age.Engine/Sys4/Sys4Loader.cs` (set `Name`) +- Modify: `engine/Age.Engine/Vm/VirtualMachine.cs` (the refactor + emitted tuple) +- Modify: `engine/Age.Engine/Vm/GameSession.cs` (SceneResult tuple) +- Modify: `engine/Age.Cli/Program.cs:17` (destructure fix) + +**Interfaces:** +- Produces: `Script.Name` (string, defaults `""`). +- Produces: `VirtualMachine.Emitted` becomes `List<(int Offset, string Text, string Script)>`. +- Produces (internal): `ExecFrame { Model.Script Script; int Pc; Frame Locals; List CallStack; Dictionary EmitSeen; }`. +- Consumes: `IScriptProvider` (Task 1) — added to the constructor but unused until Task 3. + +- [ ] **Step 1: Add `Name` to the Script model** + +In `engine/Age.Engine/Model/Script.cs`, add a property (keep existing members): + +```csharp + public string Name { get; init; } = ""; +``` + +- [ ] **Step 2: Set `Name` in the loader** + +In `engine/Age.Engine/Sys4/Sys4Loader.cs`, in `Parse`, change the return (line ~25) to include the name: + +```csharp + return new Script { Name = name, Header = header, Instructions = instrs, IndexByOffset = idxByOff, Strings = strings }; +``` + +- [ ] **Step 3: Create ExecFrame** + +Create `engine/Age.Engine/Vm/ExecFrame.cs`: + +```csharp +using Age.Engine.Model; +namespace Age.Engine.Vm; + +/// One script activation: the running script, its instruction cursor, its local slots, +/// its intra-script call/ret stack, and its per-script loop-guard map. Globals live on the VM and +/// are shared across frames; everything here is per-call and discarded on return. +internal sealed class ExecFrame +{ + public readonly Script Script; + public int Pc; // entry instruction index + public readonly Frame Locals = new(); + public readonly List CallStack = new(); // intra-script `call` (op 0x8f) returns + public readonly Dictionary EmitSeen = new(); + public ExecFrame(Script script, int pc) { Script = script; Pc = pc; } +} +``` + +- [ ] **Step 4: Refactor VirtualMachine to frames** + +In `engine/Age.Engine/Vm/VirtualMachine.cs`: + +(a) Add the constructor provider parameter and sentinels/fields. Replace the fields block + constructor (lines 7–27) so it reads: + +```csharp + private const long NoJump = 0xFFFFFFFF; + private const int HALT = int.MinValue; + private const int FRAME_RETURN = int.MinValue + 1; + private const int T_IMM = 0, T_STR = 2, T_GINT = 3, T_GFLOAT = 4, T_GSTR = 5, T_GPTR = 6, + T_LINT = 9, T_LFLOAT = 10, T_LSTR = 11, T_LPTR = 12; + + private readonly Script _s; + private readonly OpcodeTable _t; + private readonly IHost _host; + private readonly VmOptions _o; + private readonly IScriptProvider? _provider; + private ExecFrame _cur = null!; + private int _depth; + private bool _halted; + + public Dictionary Globals { get; } = new(); + public Dictionary GlobalStrings { get; } = new(); + public List<(int Offset, string Text, string Script)> Emitted { get; } = new(); + public string? HaltReason { get; private set; } + public long Steps { get; private set; } + + public VirtualMachine(Script s, OpcodeTable t, IHost host, VmOptions? o = null, IScriptProvider? provider = null) + { _s = s; _t = t; _host = host; _o = o ?? new VmOptions(); _provider = provider; } +``` + +(b) Replace `Run` (lines 94–106) with: + +```csharp + private enum FrameOutcome { Returned, Halted, RanOff } + + public void Run(int entryOffset = 0) + { + var top = new ExecFrame(_s, _s.IndexByOffset.TryGetValue(entryOffset, out var idx) ? idx : 0); + var outcome = RunFrame(top); + if (outcome == FrameOutcome.RanOff) HaltReason ??= "pc-out-of-range"; + else if (outcome == FrameOutcome.Returned) HaltReason ??= "exit"; + // Halted: HaltReason already set by the halting op. + } + + private FrameOutcome RunFrame(ExecFrame frame) + { + var prev = _cur; _cur = frame; _depth++; + var outcome = FrameOutcome.RanOff; + int pc = frame.Pc; + while (pc >= 0 && pc < frame.Script.Instructions.Count) + { + if (Steps >= _o.MaxSteps) { HaltReason ??= "STEP-LIMIT"; _halted = true; outcome = FrameOutcome.Halted; break; } + Steps++; + int next = Step(frame.Script.Instructions[pc], pc); + if (next == FRAME_RETURN) { outcome = FrameOutcome.Returned; break; } + if (next == HALT) { _halted = true; outcome = FrameOutcome.Halted; break; } + pc = next; + } + _cur = prev; _depth--; + return outcome; + } +``` + +(c) Repoint the helpers from `_fr`/`_s` to the current frame. Change every `_fr` to `_cur.Locals` in `Read`, `Write`, `ReadStr`, `WriteStr`, `BaseAddr`, `LookupStore` (the fields `_fr.I/F/S/P` become `_cur.Locals.I/F/S/P`). + +(d) In `Step`, repoint frame-scoped state: `_callstack` → `_cur.CallStack`; `_emitSeen` → `_cur.EmitSeen`; `_s.IndexByOffset` → `_cur.Script.IndexByOffset`; `_s.GetString` → `_cur.Script.GetString`. Specifically: + +- `jmp`: `return _cur.Script.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1);` +- `call`: `_cur.CallStack.Add(pc + 1); return _cur.Script.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1);` +- `ret`: `if (_cur.CallStack.Count > 0) { int r = _cur.CallStack[^1]; _cur.CallStack.RemoveAt(_cur.CallStack.Count - 1); return r; } return FRAME_RETURN;` +- `jcc`: `return tgt == NoJump ? pc + 1 : _cur.Script.IndexByOffset.GetValueOrDefault((int)tgt, pc + 1);` +- `exit` / `exit-script`: `return FRAME_RETURN;` +- `show-text` body: `_cur.EmitSeen.TryGetValue(off, out var c); c++; _cur.EmitSeen[off] = c;` then on cap `HaltReason = $"LOOP:line@0x{off:x}×{c}"; return HALT;`, and `string text = _cur.Script.GetString(off); Emitted.Add((off, text, _cur.Script.Name)); _host.ShowText(off, text);` +- `call-script`: leave the stub for now — `case "call-script": _host.CallScript(a.Count > 0 ? Read(a[0]) : 0); return pc + 1;` + +- [ ] **Step 5: Fix the SceneResult tuple** + +In `engine/Age.Engine/Vm/GameSession.cs` line 67, change the record to: + +```csharp +public sealed record SceneResult(IReadOnlyList<(int Offset, string Text, string Script)> Emitted, string? Halt, long Steps); +``` + +- [ ] **Step 6: Fix the one positional destructure** + +In `engine/Age.Cli/Program.cs` line 17, change: + +```csharp + foreach (var (off, text, _) in vm.Emitted.Take(20)) Console.WriteLine($" [{off:x}] {text}"); +``` + +- [ ] **Step 7: Run the full suite to verify no behavior change** + +Run: `dotnet test engine/AgeEngine.sln` +Expected: PASS — all existing tests green. `WaitForInputTests` still asserts SC0000 = 186 (provider-less = stub), `RecoverTests`, `TextureOpsTests`, `GameSessionTests` unchanged. (`TraceDiffTests` passes if `build/vm0-trace.json` is present; it is skipped otherwise.) + +- [ ] **Step 8: Commit** + +```bash +git add engine/Age.Engine/Vm/ExecFrame.cs engine/Age.Engine/Vm/VirtualMachine.cs engine/Age.Engine/Vm/GameSession.cs engine/Age.Engine/Model/Script.cs engine/Age.Engine/Sys4/Sys4Loader.cs engine/Age.Cli/Program.cs +git commit -m "Refactor VM to ExecFrame + tag emitted lines with source script (no behavior change)" +``` + +--- + +### Task 3: call-script execution + +Wire the provider into `call-script`: load the child script, run it as a nested frame sharing globals, return to the caller. `exit`/empty-stack `ret` return from the frame; only the top frame's return ends the VM. Depth-capped. + +**Files:** +- Modify: `engine/Age.Engine/Vm/VirtualMachine.cs` (the `call-script` case) +- Modify: `engine/Age.Engine/Vm/VmOptions.cs` (depth cap) +- Test: `engine/Age.Engine.Tests/CallScriptTests.cs` + +**Interfaces:** +- Consumes: `IScriptProvider.GetById` (Task 1), `ExecFrame`, `RunFrame`, `FrameOutcome` (Task 2). +- Produces: nested execution semantics (below) exercised by later tasks. + +- [ ] **Step 1: Add the depth-cap option** + +In `engine/Age.Engine/Vm/VmOptions.cs`: + +```csharp +namespace Age.Engine.Vm; +public sealed record VmOptions(int EmitCap = 2, long MaxSteps = 2_000_000, int CallDepthCap = 64); +``` + +(64 is a generous runaway guard; the native limit is 38.) + +- [ ] **Step 2: Write the failing tests** + +Create `engine/Age.Engine.Tests/CallScriptTests.cs`. It uses a fake in-memory provider and hand-built scripts. + +```csharp +using System.Collections.Generic; +using Age.Engine.Hosting; +using Age.Engine.Model; +using Age.Engine.Sys4; +using Age.Engine.Vm; +using Xunit; + +public class CallScriptTests +{ + private sealed class NullHost : IHost + { + public List Calls = new(); + public void ShowText(int o, string t) { } + public void CallScript(long id) => Calls.Add(id); + public void OnStub(int op) { } + public void WaitForInput() { } + public void CreateTexture(int s, int w, int h) { } + public void SetTexture(long r, int s) { } + public void DrawTexture(int s, int sx, int sy, int w, int h, int dx, int dy) { } + public (int Width, int Height) GetTextureSize(int s) => (0, 0); + public void PlayBgm(long id) { } + public void PlayVoice(long id) { } + } + + private sealed class MapProvider : IScriptProvider + { + private readonly Dictionary _m; + public MapProvider(Dictionary m) => _m = m; + public Script? GetById(long id) => _m.TryGetValue(id, out var s) ? s : null; + } + + // Build a Script from raw dwords via the real loader (guarantees identical decode). + private static Script Asm(OpcodeTable t, string name, params uint[] body) + { + var bytes = new byte[0x3C + body.Length * 4]; + System.Text.Encoding.ASCII.GetBytes("SYS4422 ").CopyTo(bytes, 0); + // fields[8] (F8 = code end) at header offset 8 + 8*4 = 0x28; set to body length (all code). + System.BitConverter.GetBytes(body.Length).CopyTo(bytes, 8 + 8 * 4); + for (int i = 0; i < body.Length; i++) System.BitConverter.GetBytes(body[i]).CopyTo(bytes, 0x3C + i * 4); + return Sys4Loader.Parse(bytes, t, name); + } + + private static OpcodeTable Table() => OpcodeTableJson.Load(Paths.OpcodesJson); + + [Fact] + public void CalleeRunsAndControlResumesAfterTheCall() + { + var t = Table(); + // Callee (id 5): set global 0x10 = 7, then exit (op 0x0). + var callee = Asm(t, "CALLEE", 0x55, 3, 0x10, 0, 7, 0x0); // mov g[0x10]=7 ; exit + // Caller: call-script 5 ; set g[0x11]=g[0x10]+1 ; exit. + var caller = Asm(t, "CALLER", + 0x03, 0, 5, // call-script 5 + 0x55, 3, 0x11, 3, 0x10, // mov g[0x11] = g[0x10] (see note below) + 0x0); // exit + var host = new NullHost(); + var vm = new VirtualMachine(caller, t, host, null, new MapProvider(new() { [5] = callee })); + vm.Run(); + Assert.Equal(7, vm.Globals[0x10]); // callee wrote a shared global + Assert.Equal(7, vm.Globals[0x11]); // caller read it AFTER the call returned + Assert.Equal("exit", vm.HaltReason); // top-level exit + Assert.Contains(5L, host.Calls); // host notified + } + + [Fact] + public void MissingProviderFallsBackToStub() + { + var t = Table(); + var caller = Asm(t, "CALLER", 0x03, 0, 5, 0x0); // call-script 5 ; exit + var host = new NullHost(); + var vm = new VirtualMachine(caller, t, host, null, null); // no provider + vm.Run(); + Assert.Equal("exit", vm.HaltReason); // did not halt on the call; stub + continue + Assert.Contains(5L, host.Calls); + } + + [Fact] + public void UnresolvedIdHalts() + { + var t = Table(); + var caller = Asm(t, "CALLER", 0x03, 0, 99, 0x0); + var vm = new VirtualMachine(caller, t, new NullHost(), null, new MapProvider(new())); + vm.Run(); + Assert.StartsWith("callscript-unresolved", vm.HaltReason); + } + + [Fact] + public void LocalsDoNotLeakBetweenCallerAndCallee() + { + var t = Table(); + // Callee writes LOCAL-int 0 = 42 (op 0x55 to local-int, type 9), then exit. + var callee = Asm(t, "CALLEE", 0x55, 9, 0, 0, 42, 0x0); + // Caller sets local-int 0 = 1, calls, then copies its own local-int 0 to global 0x20. + var caller = Asm(t, "CALLER", + 0x55, 9, 0, 0, 1, // l[0] = 1 + 0x03, 0, 5, // call-script 5 (callee sets ITS local 0 = 42) + 0x55, 3, 0x20, 9, 0, // g[0x20] = l[0] + 0x0); + var vm = new VirtualMachine(caller, t, new NullHost(), null, new MapProvider(new() { [5] = callee })); + vm.Run(); + Assert.Equal(1, vm.Globals[0x20]); // caller's local 0 unchanged by callee's local 0 + } +} +``` + +Note on `mov g[0x11] = g[0x10]`: the `mov` opcode 0x55 with a 2-operand form copies src→dst; the test encodes dst=`(type 3 = global-int, 0x11)`, src=`(type 3, 0x10)`. If the corpus `mov` is strictly 2-arg, keep argc=2 as encoded (`0x55, 3,0x11, 3,0x10`). Confirm the opcode's argc via `build/opcodes.json` when implementing; adjust the dword stream to match the real argc so `Sys4Loader.Parse` decodes it as one instruction. + +- [ ] **Step 3: Run the tests to verify they fail** + +Run: `dotnet test engine/AgeEngine.sln --filter CallScriptTests` +Expected: FAIL — `CalleeRunsAndControlResumesAfterTheCall`, `UnresolvedIdHalts`, `LocalsDoNotLeakBetweenCallerAndCallee` fail (call-script is still the stub, so the callee never runs / never halts on unresolved). `MissingProviderFallsBackToStub` may already pass. + +- [ ] **Step 4: Implement call-script execution** + +In `engine/Age.Engine/Vm/VirtualMachine.cs`, replace the `call-script` stub case with: + +```csharp + case "call-script": + { + long id = a.Count > 0 ? Read(a[0]) : 0; + _host.CallScript(id); // notify (diagnostics) + if (_provider == null) return pc + 1; // no script source: prior stub behavior + if (_depth >= _o.CallDepthCap) { HaltReason ??= "call-depth-exceeded"; return HALT; } + var child = _provider.GetById(id); + if (child == null) { HaltReason ??= $"callscript-unresolved:0x{id:x}"; return HALT; } + var entry = child.IndexByOffset.TryGetValue(0, out var ci) ? ci : 0; + var outcome = RunFrame(new ExecFrame(child, entry)); + if (outcome == FrameOutcome.Halted) return HALT; // propagate whole-VM halt up + return pc + 1; // Returned / RanOff: resume caller + } +``` + +- [ ] **Step 5: Run the tests to verify they pass** + +Run: `dotnet test engine/AgeEngine.sln --filter CallScriptTests` +Expected: PASS (all four). If the `mov` encoding decodes wrong, fix the dword stream per the Step-2 note. + +- [ ] **Step 6: Run the full suite (no regressions)** + +Run: `dotnet test engine/AgeEngine.sln` +Expected: PASS — existing provider-less tests unchanged (SC0000 still 186 in `WaitForInputTests`). + +- [ ] **Step 7: Commit** + +```bash +git add engine/Age.Engine/Vm/VirtualMachine.cs engine/Age.Engine/Vm/VmOptions.cs engine/Age.Engine.Tests/CallScriptTests.cs +git commit -m "Execute call-script: nested frame, shared globals, return to caller" +``` + +--- + +### Task 4: Wire the provider into product paths + real-scene validation + +Thread a real `Sys4ScriptProvider` into the CLI run/play/sweep paths and `GameSession.RunScene`, prove a real scene executes a real subroutine, and confirm the corpus still terminates with execution on. Confirm the empty-stack `ret` path with a real script (BUNKI). + +**Files:** +- Modify: `engine/Age.Engine/Vm/GameSession.cs` (`RunScene` gains an optional provider) +- Modify: `engine/Age.Cli/Program.cs` (construct + pass the provider in `run`, `play`, `sweep`) +- Test: `engine/Age.Engine.Tests/CallScriptIntegrationTests.cs` + +**Interfaces:** +- Consumes: `Sys4ScriptProvider.Load` (Task 1), `VirtualMachine(..., provider)` (Task 2/3). +- Produces: `GameSession.RunScene(Script, OpcodeTable, IHost, VmOptions?, IScriptProvider?)`. + +- [ ] **Step 1: Add the provider to GameSession.RunScene** + +In `engine/Age.Engine/Vm/GameSession.cs`, change the `RunScene` signature and VM construction: + +```csharp + public SceneResult RunScene(Script script, OpcodeTable table, IHost host, + VmOptions? options = null, IScriptProvider? provider = null) + { + var vm = new VirtualMachine(script, table, host, options, provider); +``` + +(The rest of the method is unchanged.) + +- [ ] **Step 2: Write the failing integration tests** + +Create `engine/Age.Engine.Tests/CallScriptIntegrationTests.cs`: + +```csharp +using Age.Engine.Hosting; +using Age.Engine.Sys4; +using Age.Engine.Vm; +using Xunit; + +public class CallScriptIntegrationTests +{ + private sealed class NullHost : IHost + { + public int CallScripts; + public void ShowText(int o, string t) { } + public void CallScript(long id) => CallScripts++; + public void OnStub(int op) { } + public void WaitForInput() { } + public void CreateTexture(int s, int w, int h) { } + public void SetTexture(long r, int s) { } + public void DrawTexture(int s, int sx, int sy, int w, int h, int dx, int dy) { } + public (int Width, int Height) GetTextureSize(int s) => (0, 0); + public void PlayBgm(long id) { } + public void PlayVoice(long id) { } + } + + [Fact] + public void RealSceneExecutesRealSubroutinesAndReturns() + { + var t = OpcodeTableJson.Load(Paths.OpcodesJson); + var provider = Sys4ScriptProvider.Load(t); + var script = Sys4Loader.Load(Paths.Scripts()["ALCHEMY.BIN"], t); // calls MES/BUNKI/ADDITEM + var host = new NullHost(); + var vm = new VirtualMachine(script, t, host, null, provider); + vm.Run(); + Assert.True(host.CallScripts > 0, "the scene should issue call-scripts"); + // Termination is the key property: execution-on must still reach a clean end, not hang or + // trip the depth cap. (No unresolved / depth halts.) + Assert.DoesNotContain("unresolved", vm.HaltReason ?? ""); + Assert.NotEqual("call-depth-exceeded", vm.HaltReason); + Assert.NotEqual("STEP-LIMIT", vm.HaltReason); + } + + [Fact] + public void BunkiTopLevelRetReturnsCleanlyAsSubroutine() + { + // BUNKI.BIN ends with a top-level `ret` (empty intra-call stack). Called as a subroutine it + // must return to the caller, not underflow-halt. Drive it directly as a child of a 1-op caller. + var t = OpcodeTableJson.Load(Paths.OpcodesJson); + var provider = Sys4ScriptProvider.Load(t); + var bunki = provider.GetById(0x143); // BUNKI.BIN + Assert.NotNull(bunki); + var vm = new VirtualMachine(bunki!, t, new NullHost(), null, provider); + vm.Run(); + // Reaching a frame-return at the top = clean "exit"; never "ret-underflow". + Assert.NotEqual("ret-underflow", vm.HaltReason); + } +} +``` + +- [ ] **Step 3: Run to verify (expected: mostly pass already)** + +Run: `dotnet test engine/AgeEngine.sln --filter CallScriptIntegrationTests` +Expected: PASS. If `BunkiTopLevelRetReturnsCleanlyAsSubroutine` reveals a wrong `ret` semantics (e.g. it halts early with a bad reason), that is the empty-stack-`ret` branch — verify against the native op 0x5 handler `ctx[0x26c93+5]` (LAB_00417ad0) before adjusting; the plan's choice (empty-stack `ret` = frame return) should hold. + +- [ ] **Step 4: Wire the provider into the CLI** + +In `engine/Age.Cli/Program.cs`, build one provider after `table` is loaded and pass it into the product paths: + +- In the `run` command (line ~13), construct `var provider = Sys4ScriptProvider.Load(table);` and change the VM to `new VirtualMachine(script, table, host, null, provider)`. +- In `play` (line ~96), construct the provider once before the loop and pass it: `session.RunScene(script, table, new CaptureHost(), null, provider)`. +- In `sweep` (lines ~121, 140, 162): construct the provider once and pass it to each `RunScene(...)` call (`, null, provider`). The `--boot` INIT scripts may be run with or without the provider; pass it for consistency. +- Leave `trace` (line ~181) provider-less on purpose — it is the base-ISA offset oracle (stub behavior, comparable to vm0.py). + +Exact edit for `run` (lines 12–17): + +```csharp + var table = OpcodeTableJson.Load(Paths.OpcodesJson); + var provider = Sys4ScriptProvider.Load(table); + var script = Sys4Loader.Load(args[1], table); + var host = new ConsoleHost(); + var vm = new VirtualMachine(script, table, host, null, provider); + vm.Run(); +``` + +(Use whatever host `run` already constructs; only the trailing `provider` arg is added.) + +- [ ] **Step 5: Confirm the corpus still terminates with execution on** + +Build and run the sweep: + +```bash +dotnet run --project engine/Age.Cli -- sweep +``` + +Expected: it completes; halt distribution is dominated by `exit`. Note any new non-`exit` halts (e.g. `call-depth-exceeded`, `callscript-unresolved`, `STEP-LIMIT`) — none should appear for well-formed scenes. Compare the anomaly list to the pre-change baseline (unbooted sweep previously matched vm0.py: 294 `exit` + 3 `LOOP`). New anomalies are the review signal. + +- [ ] **Step 6: Run the full suite** + +Run: `dotnet test engine/AgeEngine.sln` +Expected: PASS. + +- [ ] **Step 7: Commit** + +```bash +git add engine/Age.Engine/Vm/GameSession.cs engine/Age.Cli/Program.cs engine/Age.Engine.Tests/CallScriptIntegrationTests.cs +git commit -m "Wire Sys4ScriptProvider into CLI run/play/sweep; validate real subroutine execution" +``` + +--- + +## Post-implementation notes (not tasks) + +- **Godot host:** the playable Godot path (`GodotAdvHost`/`Main`) should construct a `Sys4ScriptProvider` and pass it to its `VirtualMachine`, so subroutines execute live; keep the `--selftest` path **provider-less** (preserves the 186-offset SC0000 gate). This is a small follow-up wiring, mirroring Task 4's CLI changes. +- **vm0.py** is not modified. `trace`/`TraceDiffTests`/`WaitForInputTests` remain on the provider-less (stub) path, so vm0.py stays a valid base-ISA reference without lockstep maintenance. +- **decision→scene** (scene chaining) is out of scope; it rides this same loader once the SCJUMP decision→scene-id hop is reversed. +- After landing, update the status memory (`himegari-port-status.md`) and `docs/phase-a-slice-plan.md` with the result. + +## Self-review + +- **Spec coverage:** provider seam (Task 1) ✓; recursive frame execution (Task 2) ✓; shared globals / per-frame locals / exit+ret semantics / depth cap / dynamic ids / IHost.CallScript notify (Task 3) ✓; emitted script identity (Task 2) ✓; validation via C#-owned tests + sweep-terminates + real subroutine + empty-stack-ret (Tasks 3–4) ✓; vm0.py retired without deletion, base-ISA guard preserved (Global Constraints + notes) ✓. Deviation (no-provider → stub, not halt) is documented in Global Constraints. +- **Placeholder scan:** none — every code step shows full code; the one encoding caveat (`mov` argc) has an explicit resolution instruction. +- **Type consistency:** `IScriptProvider.GetById` (Task 1) is consumed unchanged in Tasks 2–4; `Emitted`/`SceneResult` tuple `(int Offset, string Text, string Script)` is introduced in Task 2 and consumers updated in the same task; `RunFrame`/`FrameOutcome`/`ExecFrame`/`_provider`/`_depth` defined in Task 2 and used in Task 3; `VmOptions.CallDepthCap` defined in Task 3 before first use. diff --git a/docs/superpowers/specs/2026-07-07-callscript-vm-execution-design.md b/docs/superpowers/specs/2026-07-07-callscript-vm-execution-design.md new file mode 100644 index 0000000..384f439 --- /dev/null +++ b/docs/superpowers/specs/2026-07-07-callscript-vm-execution-design.md @@ -0,0 +1,149 @@ +# call-script execution in the C# VM — design (2026-07-07) + +## Goal + +Make `call-script ` (opcode 0x03) **actually execute** in the C# engine (`Age.Engine`): load the +target `.BIN` by id, run it as a nested subroutine that shares the global bank, and return to the +caller. This unlocks the ~297 subroutine scripts the corpus calls (MES, ADDITEM, the CALC* family, +SHOWGROW, …) which are currently stubbed — the first functional step past "single script runs" toward +a driven playthrough. + +Enabled by the native-RE finding that `call-script ` is a direct raw index into the SYS4INI file +table (see `docs/engine-re.md`, "op 0x03 (call-script)"; `docs/name-resolution.md §1`). + +## Scope + +**In scope:** subroutine execution — nested load + run + return, sharing globals, with per-call local +frames. Validated headless + in the existing hosts. + +**Out of scope (this slice):** +- Scene chaining / decision→scene (the SCJUMP decision-value → scene-id native hop is still open). +- Input model (`0x90`) and any new interactive behavior. +- The graphics geometry/blend drift (a separate, state-divergence problem). +- Reimplementing call-script in `vm0.py` (see Validation — vm0.py is retired from oracle duty). + +## Background — current architecture + +`VirtualMachine` (`engine/Age.Engine/Vm/VirtualMachine.cs`) runs **one** `Script` with **one** `Frame` +and a single intra-script `_callstack` (for op 0x8f `call`). `call-script` today is a stub: +`_host.CallScript(id); return pc + 1;` — the target never runs. + +Scripts are loaded by **name** from `Paths.Scripts()` (the override-aware corpus map: root loose-file +overrides shadow `extracted/DATA1/*.BIN`) via `Sys4Loader.Load`. `GameSession` carries the flat global +bank across scenes; local frames correctly do not persist. Seam rule: `Vm` references only `Model` + +`Hosting`, never `Sys4`. + +Corpus facts grounding the design: called scripts terminate with `exit` (ADDITEM, MES, SHOWGROW) or +`ret` (BUNKI); native call depth is bounded (≤ 0x26 = 38); all 297 distinct call-script ids resolve to +a DATA1 `.BIN` (0 out-of-range, 0 alternate-pack). + +## Design + +### 1. Script-provider seam + +A small interface in `Hosting` (which may reference `Model`), injected into the VM so `Vm` never +references `Sys4`: + +```csharp +public interface IScriptProvider { Script? GetById(long id); } +``` + +- `Sys4` implements `Sys4ScriptProvider`: `id → build/callscript-names.json → name → + Paths.Scripts()[name] → Sys4Loader.Load`, **cached by id** (MES is called 59×; parse once). Reusing + `Paths.Scripts()` gives the native "loose override first" behavior for free (it already prefers root + overrides over the archive copy). +- The VM takes the provider as an **optional** constructor dependency. Product paths (CLI `play`, + Godot) always supply one, so execution is effectively always-on. The provider is a genuine + dependency, not a feature flag — you cannot run a script you cannot load. A VM constructed without a + provider that then hits `call-script` halts with a clear reason (it never happens on product paths). + +### 2. Recursive frame execution + +Per-script state moves out of instance fields into an `ExecFrame`: + +- `Script` being run, the local `Frame` (I/F/S/P slots), the intra-script `call`/`ret` stack, the `pc`, + and the **emit-seen** loop-guard map. + +The run loop becomes `RunFrame(ExecFrame)`. `Run(entry)` builds the top frame and calls it. +`call-script id` → `provider.GetById(Read(a[0]))` → build a child `ExecFrame` (fresh locals, entry +pc 0) → `RunFrame(child)` → on return, the caller continues at `pc + 1`. Recursion (not an explicit +stack list) is chosen because depth is bounded (≤ 38), so there is no overflow risk and +`exit`-returns-to-caller falls out as a plain return from `RunFrame`. + +Shared VM-level state stays on the instance: `Globals`, `GlobalStrings`, `Emitted`, `Steps`, `_host`, +`_provider`. + +### 3. Semantics + +- **Shared globals = the return channel.** A callee returns results by writing globals the caller reads + (the native shared-global model; no explicit return value). Local frames are per-call and discarded + on return (matches `GameSession`). +- **`exit` / `exit-script`:** pop the current frame. Empty stack (top-level) → HALT; otherwise return + to the caller at its `call-script` + 1. Only the outermost `exit` halts. +- **`ret` (op 0x5):** returns from an intra-script `call` (0x8f) via the frame's `call`-stack. When a + script's top-level flow reaches `ret` with an empty `call`-stack (e.g. BUNKI), it returns from the + **script frame** (same as `exit`). **This empty-stack `ret` behavior is confirmed against the native + op 0x5 handler (`ctx[0x26c93+5]` = `LAB_00417ad0`) during implementation**, not guessed. +- **Emit-seen is per-frame.** The loop guard keys on string offset, and offsets are script-local + (MES `0x100` ≠ SC0000 `0x100`); a shared map would collide and falsely trip the cap on hot + subroutines. It lives in `ExecFrame`, reset per call. +- **Depth cap:** cap recursion at ~38 frames; exceeding halts with a distinct reason (the native throws + there). Guards runaway / mutual recursion. +- **Steps:** one monotonic counter across all frames (subroutine steps count toward it). +- **`Emitted`:** aggregates all frames' text in execution order; each entry carries the source script's + identity so mixed output is disambiguable in golden traces. +- **Dynamic ids:** the operand is usually immediate but may be g-int/l-ptr; `Read(a[0])` handles all + types, so a computed target resolves through the same provider. +- **`IHost.CallScript(id)`** is kept as a fire-on-entry notification (diagnostics/logging); control flow + is now VM-owned. Existing host implementations are unaffected. + +## Validation + +**vm0.py is retired from oracle duty.** It served its purpose (prototyping the execution model and +proving the C# port byte-identical across 297 scenes); it stays in the repo as a frozen historical +reference and Python-side experiment tool but is no longer maintained in lockstep. Reimplementing +call-script in it would be double work for a shrinking payoff (the C# engine now also has real-game +ground truth vm0.py never had). + +The C# engine **owns its correctness fixtures**: + +- **Golden traces regenerated from the C# engine** and checked into the repo as the source of truth + (replacing "match whatever vm0.py emits"). Scenes that call no subroutines are unchanged (and, for a + cheap belt-and-suspenders check, still happen to match vm0.py — an optional narrowed parity test). +- **New on-path checks:** + - the corpus sweep (`Age.Cli sweep`) still **terminates cleanly** with execution on (no new hangs / + depth-cap trips across the corpus); + - **known subroutines execute and return** — e.g. a scene calling MES/ADDITEM shows the subroutine's + effects (emitted text / global writes) and control resumes after the call; + - the empty-stack `ret` and `exit`-returns-to-caller branches have unit tests with hand-built frames; + - the new SC0000 opening trace (subroutines now executed) is validated by inspection against the + real-game ground truth we hold (Frida load order, screenshots, by-ear audio). + +## Testing + +- **Unit (xUnit, `Age.Engine.Tests`):** + - a fake in-memory `IScriptProvider` returning hand-built `Script`s: caller → callee → return to + caller at `pc+1`; callee `exit` returns (does not halt caller); nested depth; shared-global + write-visible-to-caller; per-frame locals don't leak; depth-cap halt; missing-id halt. + - empty-stack `ret` returns from the frame (after the native-handler confirmation). +- **Integration:** `Sys4ScriptProvider` resolves real ids (0x1ab→ADDITEM, 0x2ae7→MES) and caches; + a real scene that calls MES executes it. +- **Corpus:** `sweep` terminates with execution on; golden-trace fixtures regenerated + committed. +- **Regression:** existing non-call-script scenes produce unchanged golden traces. + +## Risks / open items + +- **Empty-stack `ret` semantics** — resolved by decompiling `LAB_00417ad0` before relying on it. +- **Headless zero-state subroutines** — with no seeded state some callees may take odd branches; the + sweep-terminates check is the guard. Not a correctness bug in call-script itself (state divergence). +- **Golden-trace churn** — one-time regeneration; the diff is reviewed, not blindly accepted. + +## Files touched (anticipated) + +- `Age.Engine/Hosting/IScriptProvider.cs` (new), `IHost.cs` (unchanged; `CallScript` kept as notify). +- `Age.Engine/Vm/VirtualMachine.cs` (ExecFrame refactor + call-script execution), `Vm/Frame.cs` + (possibly `ExecFrame`), `Vm/VmOptions.cs` (depth cap constant). +- `Age.Engine/Sys4/Sys4ScriptProvider.cs` (new), `Sys4/Paths.cs` (callscript-names.json path if needed). +- `Age.Cli/Program.cs` (wire the provider into `play`/`run`/`sweep`), Godot `GodotAdvHost`/`Main` + (supply the provider). +- `Age.Engine.Tests/*` (new call-script tests), regenerated golden fixtures. diff --git a/docs/tools-reference.md b/docs/tools-reference.md index b0ee476..1387747 100644 --- a/docs/tools-reference.md +++ b/docs/tools-reference.md @@ -26,7 +26,7 @@ whenever a tool's inputs/outputs change.** | Tool | Purpose | Run | Reads → Writes | |---|---|---|---| -| `sys4load.py` | Loader + opcode-decoding disassembler for SYS4 `.BIN` scripts (the container-format core every other tool builds on). | `sys4load.py ` · `--summary` · `--strings` · `--json` · `sys4load.py --validate` (corpus check) | `.BIN` + `age_opcodes*.py` + `build/global-var-map.json` → stdout listing, or `build/scripts-json/` with `--json` | +| `sys4load.py` | Loader + opcode-decoding disassembler for SYS4 `.BIN` scripts (the container-format core every other tool builds on). Annotates global operands (`build/globals.json`) and **`call-script` targets by name** (`build/callscript-names.json`, e.g. `call-script 0x1ab =ADDITEM.BIN`). | `sys4load.py ` · `--summary` · `--strings` · `--json` · `sys4load.py --validate` (corpus check) | `.BIN` + `age_opcodes*.py` + `build/globals.json` + `build/callscript-names.json` → stdout listing, or `build/scripts-json/` with `--json` | | `age_opcodes.py` | 548-entry Kelebek AGE opcode/arg-type table. **PRISTINE upstream data — never edit.** | *Imported.* | — | ## Opcode reference toolchain — single source of truth = `vm-map/opcodes.toml` @@ -90,7 +90,7 @@ plays `SC0000` from the real bytecode. User args (after `--`): | Tool | Purpose | Run | Reads → Writes | |---|---|---|---| -| `parse_sys4ini.py` | Parse `SYS4INI.BIN` (S4IC422, LZSS-compressed) into the authoritative asset index — name ↔ archive ↔ offset ↔ size for all DATA*.ALF (the `resId→file` answer key). | `parse_sys4ini.py [--check]` (`--check` validates vs `extracted/` + `.ALF` sizes) | `姫狩り…/SYS4INI.BIN` → `build/asset-index.json` | +| `parse_sys4ini.py` | Parse `SYS4INI.BIN` (S4IC422, LZSS-compressed) into the authoritative asset index — name ↔ archive ↔ offset ↔ size for all DATA*.ALF (the `resId→file` answer key). Each entry carries `raw_index` (its 0-based position in the SYS4INI record table incl. `@` placeholders) = the engine's universal file id. Also emits the **`call-script → name`** map (id = `raw_index`; see `engine-re.md`). | `parse_sys4ini.py [--check]` (`--check` validates vs `extracted/` + `.ALF` sizes) | `姫狩り…/SYS4INI.BIN` → `build/asset-index.json` + `build/callscript-names.json` | | `resolve_asset.py` | ★ **The static asset resolver.** SYS4INI is sectioned (one per scene: `SCxxxx.BIN` + its cross-archive manifest; `file_number` = index within section). Resolves `resId → files[section_base(scene) + resId]` for graphics AND audio, no capture. | `resolve_asset.py --build` · `resolve_asset.py [resId]` | `build/asset-index.json` → `build/asset-sections.json`; resolves any (scene, resId) | | `resolve_frida_reads.py` | Rescue noisy Frida archive-read offsets → asset names via the index (per-archive range search; drops 0x20000 paging reads); recovers the per-scene asset load order. | `resolve_frida_reads.py [reads.log] [-o out.json]` | `build/frida-reads.log` + `build/asset-index.json` → `build/frida-asset-loads.json` | | `convert_agf.py` | Convert AGF stills to BMP via `AGF2BMP2AGF.exe` (searches all `extracted/DATA*`). `--scene` batch-converts a scene's whole SYS4INI manifest — feeds the Godot render. | `convert_agf.py EV052CA.AGF …` · `convert_agf.py --scene SC0000` | `extracted/DATA*/*.AGF` → `build/textures/*.BMP` | diff --git a/engine/Age.Cli/Program.cs b/engine/Age.Cli/Program.cs index 6ee938d..92fc970 100644 --- a/engine/Age.Cli/Program.cs +++ b/engine/Age.Cli/Program.cs @@ -5,16 +5,19 @@ using Age.Engine.Sys4; using Age.Engine.Vm; var table = OpcodeTableJson.Load(Paths.OpcodesJson); +// call-script execution: resolves ids -> scripts. Product paths pass this so subroutines run; +// `trace` stays provider-less on purpose (the base-ISA offset oracle). +var provider = Sys4ScriptProvider.Load(table); if (args.Length == 0) { Console.WriteLine("usage: run | trace "); return 1; } if (args[0] == "run") { var script = Sys4Loader.Load(args[1], table); - var vm = new VirtualMachine(script, table, new CaptureHost()); + var vm = new VirtualMachine(script, table, new CaptureHost(), null, provider); vm.Run(); Console.WriteLine($"{Path.GetFileName(args[1])}: {vm.Steps} steps, {vm.Emitted.Count} show-text (halt: {vm.HaltReason})"); - foreach (var (off, text) in vm.Emitted.Take(20)) Console.WriteLine($" [{off:x}] {text}"); + foreach (var (off, text, _) in vm.Emitted.Take(20)) Console.WriteLine($" [{off:x}] {text}"); return 0; } @@ -94,7 +97,7 @@ if (args[0] == "play") foreach (var name in scenes) { var script = Sys4Loader.Load(scripts[name.ToUpperInvariant()], table); - var r = session.RunScene(script, table, new CaptureHost()); + var r = session.RunScene(script, table, new CaptureHost(), null, provider); totalLines += r.Emitted.Count; Console.WriteLine($" {name,-14} {r.Emitted.Count,4} lines, {r.Steps,7} steps (halt: {r.Halt})"); } @@ -118,7 +121,7 @@ if (args[0] == "sweep") var bootSession = new GameSession(); foreach (var s in new[] { "SKINIT.BIN", "ITINIT.BIN", "EBINIT.BIN", "CGINIT.BIN", "MPINIT.BIN", "AFINIT.BIN", "CCINIT.BIN", "STINIT.BIN", "STINIT2.BIN" }) - bootSession.RunScene(Sys4Loader.Load(scripts[s], table), table, new CaptureHost()); + bootSession.RunScene(Sys4Loader.Load(scripts[s], table), table, new CaptureHost(), null, provider); baseline = bootSession.ToJson(); Console.WriteLine($"[boot] baseline = {bootSession.Globals.Count} globals; running {names.Count} scenes from it."); } @@ -137,7 +140,7 @@ if (args[0] == "sweep") { var session = Fresh(); if (seeded) foreach (var (k, v) in seeds) session.Seed(k, v); - return session.RunScene(Sys4Loader.Load(scripts[name], table), table, new CaptureHost()).Emitted.Count; + return session.RunScene(Sys4Loader.Load(scripts[name], table), table, new CaptureHost(), null, provider).Emitted.Count; } if (seeds.Count > 0) @@ -159,7 +162,7 @@ if (args[0] == "sweep") foreach (var name in names) { var session = Fresh(); - var r = session.RunScene(Sys4Loader.Load(scripts[name], table), table, new CaptureHost()); + var r = session.RunScene(Sys4Loader.Load(scripts[name], table), table, new CaptureHost(), null, provider); var halt = r.Halt ?? "null"; haltDist[halt] = haltDist.GetValueOrDefault(halt) + 1; totalLines += r.Emitted.Count; diff --git a/engine/Age.Engine.Tests/CallScriptIntegrationTests.cs b/engine/Age.Engine.Tests/CallScriptIntegrationTests.cs new file mode 100644 index 0000000..b8b8647 --- /dev/null +++ b/engine/Age.Engine.Tests/CallScriptIntegrationTests.cs @@ -0,0 +1,52 @@ +using Age.Engine.Hosting; +using Age.Engine.Sys4; +using Age.Engine.Vm; +using Xunit; + +public class CallScriptIntegrationTests +{ + private sealed class NullHost : IHost + { + public int CallScripts; + public void ShowText(int o, string t) { } + public void CallScript(long id) => CallScripts++; + public void OnStub(int op) { } + public void WaitForInput() { } + public void CreateTexture(int s, int w, int h) { } + public void SetTexture(long r, int s) { } + public void DrawTexture(int s, int sx, int sy, int w, int h, int dx, int dy) { } + public (int Width, int Height) GetTextureSize(int s) => (0, 0); + public void PlayBgm(long id) { } + public void PlayVoice(long id) { } + } + + [Fact] + public void RealScriptExecutesRealSubroutinesAndReturns() + { + // ADDILL.BIN unconditionally call-scripts ADDILLSUB then CALCREVISE at entry, then exits. + // With execution on, both subroutines load, run, and return, so ADDILL reaches its own exit. + var t = OpcodeTableJson.Load(Paths.OpcodesJson); + var provider = Sys4ScriptProvider.Load(t); + var script = Sys4Loader.Load(Paths.Scripts()["ADDILL.BIN"], t); + var host = new NullHost(); + var vm = new VirtualMachine(script, t, host, null, provider); + vm.Run(); + Assert.Equal(2, host.CallScripts); // ADDILLSUB + CALCREVISE both dispatched + Assert.Equal("exit", vm.HaltReason); // subroutines returned; ADDILL reached its own exit + } + + [Fact] + public void BunkiTopLevelRetReturnsCleanlyAsSubroutine() + { + // BUNKI.BIN ends with a top-level `ret` (empty intra-call stack). Called as a subroutine it + // must return to the caller, not underflow-halt. Drive it directly. + var t = OpcodeTableJson.Load(Paths.OpcodesJson); + var provider = Sys4ScriptProvider.Load(t); + var bunki = provider.GetById(0x143); // BUNKI.BIN + Assert.NotNull(bunki); + var vm = new VirtualMachine(bunki!, t, new NullHost(), null, provider); + vm.Run(); + // Reaching a frame-return at the top = clean "exit"; never "ret-underflow". + Assert.NotEqual("ret-underflow", vm.HaltReason); + } +} diff --git a/engine/Age.Engine.Tests/CallScriptTests.cs b/engine/Age.Engine.Tests/CallScriptTests.cs new file mode 100644 index 0000000..170c0e5 --- /dev/null +++ b/engine/Age.Engine.Tests/CallScriptTests.cs @@ -0,0 +1,107 @@ +using System.Collections.Generic; +using Age.Engine.Hosting; +using Age.Engine.Model; +using Age.Engine.Sys4; +using Age.Engine.Vm; +using Xunit; + +public class CallScriptTests +{ + // Opcodes (from build/opcodes.json): exit=0x2, call-script=0x3(argc1), mov=0x55(argc2). + // Operand types: imm=0, global-int=3, local-int=9. + private const uint OP_EXIT = 0x2, OP_CALLSCRIPT = 0x3, OP_MOV = 0x55; + + private sealed class NullHost : IHost + { + public List Calls = new(); + public void ShowText(int o, string t) { } + public void CallScript(long id) => Calls.Add(id); + public void OnStub(int op) { } + public void WaitForInput() { } + public void CreateTexture(int s, int w, int h) { } + public void SetTexture(long r, int s) { } + public void DrawTexture(int s, int sx, int sy, int w, int h, int dx, int dy) { } + public (int Width, int Height) GetTextureSize(int s) => (0, 0); + public void PlayBgm(long id) { } + public void PlayVoice(long id) { } + } + + private sealed class MapProvider : IScriptProvider + { + private readonly Dictionary _m; + public MapProvider(Dictionary m) => _m = m; + public Script? GetById(long id) => _m.TryGetValue(id, out var s) ? s : null; + } + + // Build a Script from raw dwords via the real loader (guarantees identical decode). + private static Script Asm(OpcodeTable t, string name, params uint[] body) + { + var bytes = new byte[0x3C + body.Length * 4]; + System.Text.Encoding.ASCII.GetBytes("SYS4422 ").CopyTo(bytes, 0); + // fields[8] (F8 = code end) at header offset 8 + 8*4 = 0x28; set to body length (all code). + System.BitConverter.GetBytes(body.Length).CopyTo(bytes, 8 + 8 * 4); + for (int i = 0; i < body.Length; i++) System.BitConverter.GetBytes(body[i]).CopyTo(bytes, 0x3C + i * 4); + return Sys4Loader.Parse(bytes, t, name); + } + + private static OpcodeTable Table() => OpcodeTableJson.Load(Paths.OpcodesJson); + + [Fact] + public void CalleeRunsAndControlResumesAfterTheCall() + { + var t = Table(); + // Callee (id 5): mov g[0x10] = 7, then exit. + var callee = Asm(t, "CALLEE", OP_MOV, 3, 0x10, 0, 7, OP_EXIT); + // Caller: call-script 5 ; mov g[0x11] = g[0x10] ; exit. + var caller = Asm(t, "CALLER", + OP_CALLSCRIPT, 0, 5, + OP_MOV, 3, 0x11, 3, 0x10, + OP_EXIT); + var host = new NullHost(); + var vm = new VirtualMachine(caller, t, host, null, new MapProvider(new() { [5] = callee })); + vm.Run(); + Assert.Equal(7, vm.Globals[0x10]); // callee wrote a shared global + Assert.Equal(7, vm.Globals[0x11]); // caller read it AFTER the call returned + Assert.Equal("exit", vm.HaltReason); // top-level exit + Assert.Contains(5L, host.Calls); // host notified + } + + [Fact] + public void MissingProviderFallsBackToStub() + { + var t = Table(); + var caller = Asm(t, "CALLER", OP_CALLSCRIPT, 0, 5, OP_EXIT); + var host = new NullHost(); + var vm = new VirtualMachine(caller, t, host, null, null); // no provider + vm.Run(); + Assert.Equal("exit", vm.HaltReason); // did not halt on the call; stub + continue + Assert.Contains(5L, host.Calls); + } + + [Fact] + public void UnresolvedIdHalts() + { + var t = Table(); + var caller = Asm(t, "CALLER", OP_CALLSCRIPT, 0, 99, OP_EXIT); + var vm = new VirtualMachine(caller, t, new NullHost(), null, new MapProvider(new())); + vm.Run(); + Assert.StartsWith("callscript-unresolved", vm.HaltReason); + } + + [Fact] + public void LocalsDoNotLeakBetweenCallerAndCallee() + { + var t = Table(); + // Callee writes LOCAL-int 0 = 42 (mov to local-int, type 9), then exit. + var callee = Asm(t, "CALLEE", OP_MOV, 9, 0, 0, 42, OP_EXIT); + // Caller sets local-int 0 = 1, calls, then copies its own local-int 0 to global 0x20. + var caller = Asm(t, "CALLER", + OP_MOV, 9, 0, 0, 1, // l[0] = 1 + OP_CALLSCRIPT, 0, 5, // call-script 5 (callee sets ITS local 0 = 42) + OP_MOV, 3, 0x20, 9, 0, // g[0x20] = l[0] + OP_EXIT); + var vm = new VirtualMachine(caller, t, new NullHost(), null, new MapProvider(new() { [5] = callee })); + vm.Run(); + Assert.Equal(1, vm.Globals[0x20]); // caller's local 0 unchanged by callee's local 0 + } +} diff --git a/engine/Age.Engine.Tests/Sys4ScriptProviderTests.cs b/engine/Age.Engine.Tests/Sys4ScriptProviderTests.cs new file mode 100644 index 0000000..b419f76 --- /dev/null +++ b/engine/Age.Engine.Tests/Sys4ScriptProviderTests.cs @@ -0,0 +1,20 @@ +using Age.Engine.Sys4; +using Xunit; + +public class Sys4ScriptProviderTests +{ + [Fact] + public void ResolvesKnownIdsToTheirScripts() + { + var table = OpcodeTableJson.Load(Paths.OpcodesJson); + var provider = Sys4ScriptProvider.Load(table); + + var additem = provider.GetById(0x1ab); // ADDITEM.BIN + var mes = provider.GetById(0x2ae7); // MES.BIN + Assert.NotNull(additem); + Assert.NotNull(mes); + Assert.True(additem!.Instructions.Count > 0); + Assert.Same(additem, provider.GetById(0x1ab)); // cached: same instance + Assert.Null(provider.GetById(long.MaxValue)); // unknown id + } +} diff --git a/engine/Age.Engine/Hosting/IScriptProvider.cs b/engine/Age.Engine/Hosting/IScriptProvider.cs new file mode 100644 index 0000000..bb75aaa --- /dev/null +++ b/engine/Age.Engine/Hosting/IScriptProvider.cs @@ -0,0 +1,10 @@ +using Age.Engine.Model; +namespace Age.Engine.Hosting; + +/// Resolves a call-script id (a raw SYS4INI file index) to a loaded . +/// Implemented in Sys4; injected into the VM so the Vm layer never references Sys4. +public interface IScriptProvider +{ + /// The script for this id, or null if the id maps to no known script. + Script? GetById(long id); +} diff --git a/engine/Age.Engine/Model/Script.cs b/engine/Age.Engine/Model/Script.cs index 870e03d..e002dfc 100644 --- a/engine/Age.Engine/Model/Script.cs +++ b/engine/Age.Engine/Model/Script.cs @@ -1,6 +1,7 @@ namespace Age.Engine.Model; public sealed class Script { + public string Name { get; init; } = ""; public required ScriptHeader Header { get; init; } public required IReadOnlyList Instructions { get; init; } public required IReadOnlyDictionary IndexByOffset { get; init; } diff --git a/engine/Age.Engine/Sys4/Paths.cs b/engine/Age.Engine/Sys4/Paths.cs index 2533566..fda219b 100644 --- a/engine/Age.Engine/Sys4/Paths.cs +++ b/engine/Age.Engine/Sys4/Paths.cs @@ -10,6 +10,7 @@ public static class Paths public static string OpcodesJson => Path.Combine(Build, "opcodes.json"); public static string AssetSectionsJson => Path.Combine(Build, "asset-sections.json"); public static string AssetIndexJson => Path.Combine(Build, "asset-index.json"); + public static string CallscriptNamesJson => Path.Combine(Build, "callscript-names.json"); public static string Textures => Path.Combine(Build, "textures"); private static string FindRepo() diff --git a/engine/Age.Engine/Sys4/Sys4Loader.cs b/engine/Age.Engine/Sys4/Sys4Loader.cs index ac41fb4..ab09af8 100644 --- a/engine/Age.Engine/Sys4/Sys4Loader.cs +++ b/engine/Age.Engine/Sys4/Sys4Loader.cs @@ -22,7 +22,7 @@ public static class Sys4Loader var header = new ScriptHeader(fields[0], fields[1], fields[2], fields[3], fields[4], fields[5]); var (instrs, idxByOff, strings) = DecodeCode(dw, fields, nbody, table); - return new Script { Header = header, Instructions = instrs, IndexByOffset = idxByOff, Strings = strings }; + return new Script { Name = name, Header = header, Instructions = instrs, IndexByOffset = idxByOff, Strings = strings }; } private static (List, Dictionary, Dictionary) diff --git a/engine/Age.Engine/Sys4/Sys4ScriptProvider.cs b/engine/Age.Engine/Sys4/Sys4ScriptProvider.cs new file mode 100644 index 0000000..9ef9990 --- /dev/null +++ b/engine/Age.Engine/Sys4/Sys4ScriptProvider.cs @@ -0,0 +1,39 @@ +using System.Text.Json; +using Age.Engine.Hosting; +using Age.Engine.Model; +namespace Age.Engine.Sys4; + +/// Resolves call-script ids (raw SYS4INI file indices) to loaded scripts, using +/// build/callscript-names.json (id→name) + Paths.Scripts() (name→path). Cached per id. +/// The native resolver prefers a loose override before the archive; Paths.Scripts() already +/// shadows extracted/DATA1 with root overrides, so that behavior is preserved. +public sealed class Sys4ScriptProvider : IScriptProvider +{ + private readonly OpcodeTable _table; + private readonly IReadOnlyDictionary _idToName; + private readonly Dictionary _byName; // NAME(UPPER) -> path + private readonly Dictionary _cache = new(); + + public Sys4ScriptProvider(OpcodeTable table, IReadOnlyDictionary idToName, + Dictionary byName) + { _table = table; _idToName = idToName; _byName = byName; } + + public static Sys4ScriptProvider Load(OpcodeTable table) + { + var raw = JsonSerializer.Deserialize>( + File.ReadAllText(Paths.CallscriptNamesJson)) ?? new(); + var idToName = raw.ToDictionary(kv => long.Parse(kv.Key), kv => kv.Value); + return new Sys4ScriptProvider(table, idToName, Paths.Scripts()); + } + + public Script? GetById(long id) + { + if (_cache.TryGetValue(id, out var cached)) return cached; + Script? s = null; + if (_idToName.TryGetValue(id, out var name) && + _byName.TryGetValue(name.ToUpperInvariant(), out var path)) + s = Sys4Loader.Load(path, _table); + _cache[id] = s; + return s; + } +} diff --git a/engine/Age.Engine/Vm/ExecFrame.cs b/engine/Age.Engine/Vm/ExecFrame.cs new file mode 100644 index 0000000..361267d --- /dev/null +++ b/engine/Age.Engine/Vm/ExecFrame.cs @@ -0,0 +1,15 @@ +using Age.Engine.Model; +namespace Age.Engine.Vm; + +/// One script activation: the running script, its instruction cursor, its local slots, +/// its intra-script call/ret stack, and its per-script loop-guard map. Globals live on the VM and +/// are shared across frames; everything here is per-call and discarded on return. +internal sealed class ExecFrame +{ + public readonly Script Script; + public int Pc; // entry instruction index + public readonly Frame Locals = new(); + public readonly List CallStack = new(); // intra-script `call` (op 0x8f) returns + public readonly Dictionary EmitSeen = new(); + public ExecFrame(Script script, int pc) { Script = script; Pc = pc; } +} diff --git a/engine/Age.Engine/Vm/GameSession.cs b/engine/Age.Engine/Vm/GameSession.cs index 0698e79..0507c8e 100644 --- a/engine/Age.Engine/Vm/GameSession.cs +++ b/engine/Age.Engine/Vm/GameSession.cs @@ -24,9 +24,10 @@ public sealed class GameSession public void SeedString(int addr, string value) => GlobalStrings[addr] = value; /// Run one scene: seed a fresh VM from session state, execute, merge final state back. - public SceneResult RunScene(Script script, OpcodeTable table, IHost host, VmOptions? options = null) + public SceneResult RunScene(Script script, OpcodeTable table, IHost host, + VmOptions? options = null, IScriptProvider? provider = null) { - var vm = new VirtualMachine(script, table, host, options); + var vm = new VirtualMachine(script, table, host, options, provider); foreach (var kv in Globals) vm.Globals[kv.Key] = kv.Value; foreach (var kv in GlobalStrings) vm.GlobalStrings[kv.Key] = kv.Value; @@ -64,4 +65,4 @@ public sealed class GameSession } /// The observable result of running one scene into a . -public sealed record SceneResult(IReadOnlyList<(int Offset, string Text)> Emitted, string? Halt, long Steps); +public sealed record SceneResult(IReadOnlyList<(int Offset, string Text, string Script)> Emitted, string? Halt, long Steps); diff --git a/engine/Age.Engine/Vm/VirtualMachine.cs b/engine/Age.Engine/Vm/VirtualMachine.cs index 9f3e3fd..57b05bc 100644 --- a/engine/Age.Engine/Vm/VirtualMachine.cs +++ b/engine/Age.Engine/Vm/VirtualMachine.cs @@ -6,6 +6,7 @@ public sealed class VirtualMachine { private const long NoJump = 0xFFFFFFFF; private const int HALT = int.MinValue; + private const int FRAME_RETURN = int.MinValue + 1; private const int T_IMM = 0, T_STR = 2, T_GINT = 3, T_GFLOAT = 4, T_GSTR = 5, T_GPTR = 6, T_LINT = 9, T_LFLOAT = 10, T_LSTR = 11, T_LPTR = 12; @@ -13,18 +14,18 @@ public sealed class VirtualMachine private readonly OpcodeTable _t; private readonly IHost _host; private readonly VmOptions _o; - private readonly Frame _fr = new(); - private readonly List _callstack = new(); - private readonly Dictionary _emitSeen = new(); + private readonly IScriptProvider? _provider; + private ExecFrame _cur = null!; + private int _depth; public Dictionary Globals { get; } = new(); public Dictionary GlobalStrings { get; } = new(); - public List<(int Offset, string Text)> Emitted { get; } = new(); + public List<(int Offset, string Text, string Script)> Emitted { get; } = new(); public string? HaltReason { get; private set; } public long Steps { get; private set; } - public VirtualMachine(Script s, OpcodeTable t, IHost host, VmOptions? o = null) - { _s = s; _t = t; _host = host; _o = o ?? new VmOptions(); } + public VirtualMachine(Script s, OpcodeTable t, IHost host, VmOptions? o = null, IScriptProvider? provider = null) + { _s = s; _t = t; _host = host; _o = o ?? new VmOptions(); _provider = provider; } private static long Gi(Dictionary d, int k) => d.TryGetValue(k, out var v) ? v : 0; private static string Gs(Dictionary d, int k) => d.TryGetValue(k, out var v) ? v : ""; @@ -38,9 +39,9 @@ public sealed class VirtualMachine T_IMM => op.Value, T_GINT or T_GFLOAT => Gi(Globals, (int)op.Value), T_GPTR => Gi(Globals, (int)Gi(Globals, (int)op.Value)), - T_LINT => Gi(_fr.I, (int)op.Value), - T_LFLOAT => Gi(_fr.F, (int)op.Value), - T_LPTR => Gi(Globals, (int)Gi(_fr.P, (int)op.Value)), + T_LINT => Gi(_cur.Locals.I, (int)op.Value), + T_LFLOAT => Gi(_cur.Locals.F, (int)op.Value), + T_LPTR => Gi(Globals, (int)Gi(_cur.Locals.P, (int)op.Value)), _ => op.Value, }; @@ -50,17 +51,17 @@ public sealed class VirtualMachine { case T_GINT: case T_GFLOAT: Globals[(int)op.Value] = val; break; case T_GPTR: Globals[(int)Gi(Globals, (int)op.Value)] = val; break; - case T_LINT: _fr.I[(int)op.Value] = val; break; - case T_LFLOAT: _fr.F[(int)op.Value] = val; break; - case T_LPTR: Globals[(int)Gi(_fr.P, (int)op.Value)] = val; break; + case T_LINT: _cur.Locals.I[(int)op.Value] = val; break; + case T_LFLOAT: _cur.Locals.F[(int)op.Value] = val; break; + case T_LPTR: Globals[(int)Gi(_cur.Locals.P, (int)op.Value)] = val; break; } } private string ReadStr(Operand op) => op.Type switch { - T_STR => _s.GetString((int)op.Value), + T_STR => _cur.Script.GetString((int)op.Value), T_GSTR => Gs(GlobalStrings, (int)op.Value), - T_LSTR => Gs(_fr.S, (int)op.Value), + T_LSTR => Gs(_cur.Locals.S, (int)op.Value), _ => "", }; @@ -69,15 +70,15 @@ public sealed class VirtualMachine switch (op.Type) { case T_GSTR: GlobalStrings[(int)op.Value] = val; break; - case T_LSTR: _fr.S[(int)op.Value] = val; break; + case T_LSTR: _cur.Locals.S[(int)op.Value] = val; break; } } private long BaseAddr(Operand op) => op.Type switch { T_IMM or T_GINT or T_GFLOAT or T_GSTR or T_GPTR => op.Value, - T_LINT => Gi(_fr.I, (int)op.Value), - T_LPTR => Gi(_fr.P, (int)op.Value), + T_LINT => Gi(_cur.Locals.I, (int)op.Value), + T_LPTR => Gi(_cur.Locals.P, (int)op.Value), _ => op.Value, }; @@ -85,24 +86,39 @@ public sealed class VirtualMachine { switch (dst.Type) { - case T_LPTR: _fr.P[(int)dst.Value] = addr; break; + case T_LPTR: _cur.Locals.P[(int)dst.Value] = addr; break; case T_GPTR: Globals[(int)dst.Value] = addr; break; default: Write(dst, Gi(Globals, (int)addr)); break; } } + private enum FrameOutcome { Returned, Halted, RanOff } + public void Run(int entryOffset = 0) { - int pc = _s.IndexByOffset.TryGetValue(entryOffset, out var idx) ? idx : 0; - while (pc >= 0 && pc < _s.Instructions.Count) + var top = new ExecFrame(_s, _s.IndexByOffset.TryGetValue(entryOffset, out var idx) ? idx : 0); + var outcome = RunFrame(top); + if (outcome == FrameOutcome.RanOff) HaltReason ??= "pc-out-of-range"; + else if (outcome == FrameOutcome.Returned) HaltReason ??= "exit"; + // Halted: HaltReason already set by the halting op. + } + + private FrameOutcome RunFrame(ExecFrame frame) + { + var prev = _cur; _cur = frame; _depth++; + var outcome = FrameOutcome.RanOff; + int pc = frame.Pc; + while (pc >= 0 && pc < frame.Script.Instructions.Count) { - if (Steps >= _o.MaxSteps) { HaltReason ??= "STEP-LIMIT"; return; } + if (Steps >= _o.MaxSteps) { HaltReason ??= "STEP-LIMIT"; outcome = FrameOutcome.Halted; break; } Steps++; - int next = Step(_s.Instructions[pc], pc); - if (next == HALT) return; + int next = Step(frame.Script.Instructions[pc], pc); + if (next == FRAME_RETURN) { outcome = FrameOutcome.Returned; break; } + if (next == HALT) { outcome = FrameOutcome.Halted; break; } pc = next; } - HaltReason ??= "pc-out-of-range"; + _cur = prev; _depth--; + return outcome; } private int Step(Instruction ins, int pc) @@ -139,28 +155,40 @@ public sealed class VirtualMachine case "bit-reset": Write(a[0], Read(a[0]) & ~Read(a[1])); return pc + 1; case "check-bit": Write(a[0], (Read(a[1]) >> (int)(Read(a[2]) & 31)) & 1); return pc + 1; case "copy-to-global": Write(a[0], Read(a[1])); return pc + 1; - case "jmp": return _s.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1); - case "call": _callstack.Add(pc + 1); return _s.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1); + case "jmp": return _cur.Script.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1); + case "call": _cur.CallStack.Add(pc + 1); return _cur.Script.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1); case "ret": - if (_callstack.Count > 0) { int r = _callstack[^1]; _callstack.RemoveAt(_callstack.Count - 1); return r; } - HaltReason = "ret-underflow"; return HALT; + if (_cur.CallStack.Count > 0) { int r = _cur.CallStack[^1]; _cur.CallStack.RemoveAt(_cur.CallStack.Count - 1); return r; } + return FRAME_RETURN; // empty intra-call stack => return from the script frame case "jcc": { long tgt = Read(a[0]) != 0 ? a[1].Value : a[2].Value; - return tgt == NoJump ? pc + 1 : _s.IndexByOffset.GetValueOrDefault((int)tgt, pc + 1); + return tgt == NoJump ? pc + 1 : _cur.Script.IndexByOffset.GetValueOrDefault((int)tgt, pc + 1); } case "exit": - case "exit-script": HaltReason = "exit"; return HALT; - case "call-script": _host.CallScript(a.Count > 0 ? Read(a[0]) : 0); return pc + 1; + case "exit-script": return FRAME_RETURN; + case "call-script": + { + long id = a.Count > 0 ? Read(a[0]) : 0; + _host.CallScript(id); // notify (diagnostics) + if (_provider == null) return pc + 1; // no script source: prior stub behavior + if (_depth >= _o.CallDepthCap) { HaltReason ??= "call-depth-exceeded"; return HALT; } + var child = _provider.GetById(id); + if (child == null) { HaltReason ??= $"callscript-unresolved:0x{id:x}"; return HALT; } + var entry = child.IndexByOffset.TryGetValue(0, out var ci) ? ci : 0; + var outcome = RunFrame(new ExecFrame(child, entry)); + if (outcome == FrameOutcome.Halted) return HALT; // propagate whole-VM halt up + return pc + 1; // Returned / RanOff: resume caller + } case "show-text": foreach (var o in a) { if (o.Type != T_STR) continue; int off = (int)o.Value; - _emitSeen.TryGetValue(off, out var c); c++; _emitSeen[off] = c; + _cur.EmitSeen.TryGetValue(off, out var c); c++; _cur.EmitSeen[off] = c; if (c > _o.EmitCap) { HaltReason = $"LOOP:line@0x{off:x}×{c}"; return HALT; } - string text = _s.GetString(off); - Emitted.Add((off, text)); + string text = _cur.Script.GetString(off); + Emitted.Add((off, text, _cur.Script.Name)); _host.ShowText(off, text); } return pc + 1; diff --git a/engine/Age.Engine/Vm/VmOptions.cs b/engine/Age.Engine/Vm/VmOptions.cs index 329cab9..2d858ee 100644 --- a/engine/Age.Engine/Vm/VmOptions.cs +++ b/engine/Age.Engine/Vm/VmOptions.cs @@ -1,2 +1,2 @@ namespace Age.Engine.Vm; -public sealed record VmOptions(int EmitCap = 2, long MaxSteps = 2_000_000); +public sealed record VmOptions(int EmitCap = 2, long MaxSteps = 2_000_000, int CallDepthCap = 64); diff --git a/tools/parse_sys4ini.py b/tools/parse_sys4ini.py index b9f4ab0..995efda 100644 --- a/tools/parse_sys4ini.py +++ b/tools/parse_sys4ini.py @@ -115,13 +115,18 @@ def parse(path: Path) -> dict: f"have {len(blob)} decompressed bytes)") files = [] - for _ in range(file_count): + for i in range(file_count): name_b, arc_id, file_number, offset, size = struct.unpack_from(FILE_ENTRY_FMT, blob, p) p += FILE_ENTRY_LEN name = _cstr(name_b) if name == "@" or not name: continue files.append({ + # raw_index = the record's 0-based position in the SYS4INI table INCLUDING '@' + # placeholders. This is the engine's universal file id: `call-script ` and every + # script/asset load index by it (engine FUN_0044f390: record = base + id*0x50). It is + # NOT the same as this entry's position in `files` (which omits placeholders). + "raw_index": i, "name": name, "archive": archives[arc_id] if 0 <= arc_id < arc_count else None, "arc_id": arc_id, @@ -215,6 +220,15 @@ def main() -> int: out.write_text(json.dumps(index, ensure_ascii=False, indent=1), encoding="utf-8") print(f"-> {out.relative_to(paths.REPO)}") + # call-script id -> name map (raw_index keyed). `call-script ` (opcode 0x03) is a direct + # raw index into the SYS4INI file table, so this IS the call-graph name registry that + # name-resolution.md #1 needed (confirmed via native-RE, see docs/engine-re.md). sys4load reads + # it to annotate `call-script 0x1ab =ADDITEM`. + cs = {str(f["raw_index"]): f["name"] for f in index["files"]} + cs_out = paths.BUILD / "callscript-names.json" + cs_out.write_text(json.dumps(cs, ensure_ascii=False, indent=0), encoding="utf-8") + print(f"-> {cs_out.relative_to(paths.REPO)} ({len(cs)} ids)") + if do_check: print("--- validation ---") return 1 if check(index) else 0 diff --git a/tools/sys4load.py b/tools/sys4load.py index 01a7ad9..b893dc1 100644 --- a/tools/sys4load.py +++ b/tools/sys4load.py @@ -80,6 +80,21 @@ def _load_global_labels() -> dict: GLOBAL_LABELS = _load_global_labels() +def _load_callscript_names() -> dict: + """id -> script name. `call-script ` (op 0x03) is a raw index into the SYS4INI file + table; build/callscript-names.json maps every id to its script name (see docs/engine-re.md).""" + try: + p = Path(__file__).resolve().parent.parent / "build" / "callscript-names.json" + data = json.loads(p.read_text(encoding="utf-8")) + except Exception: + return {} + return {int(k): v for k, v in data.items()} + + +CALLSCRIPT_NAMES = _load_callscript_names() +CALLSCRIPT_OP = 0x03 + + def display_label(op: int) -> str: """Rendered mnemonic: Kelebek name if it has one, else the inferred name, else u00….""" lbl = OPCODES.get(op, (f"?{op:x}", 0))[0] @@ -368,6 +383,8 @@ def _fmt_operand(op: int, arg_index: int, atype: int, aval: int, strings: dict) tlabel = ARG_TYPES.get(atype) if atype == 0 or tlabel is None: # immediate / unknown-tag: raw value if atype == 0: + if op == CALLSCRIPT_OP and aval in CALLSCRIPT_NAMES: + return f"{aval:#x} ={CALLSCRIPT_NAMES[aval]}" # call-script target script name return f"{aval:#x}" return f"" if tlabel == "float": diff --git a/vm-map/opcodes.toml b/vm-map/opcodes.toml index e1b3e2b..8369add 100644 --- a/vm-map/opcodes.toml +++ b/vm-map/opcodes.toml @@ -77,17 +77,36 @@ abi_source = "kelebek+decode-validated" [opcode.semantics] name = "call-script" -category = "unknown" -summary = "" +category = "control" +summary = "load & call another SYS4 script by id; id = RAW index into the SYS4INI file table (asset-index). Pushes a script frame; returns to caller when the callee ends." noop_headless = false -source = "kelebek" -confidence = "med" +source = "investigation" +confidence = "high" depends_on = [] -evidence = "" +evidence = "native-RE (Ghidra): handler FUN_0041bc90 -> loader FUN_0040e980 -> resolver FUN_0044f390 indexes an 80-byte record table (base [ctx+0x414], count [ctx+0x40c]) at base+id*0x50 = the SYS4INI record layout {name[64],arc_id@0x40,file_number@0x44,offset@0x48,size@0x4c}. Confirmed statically: all 297 distinct corpus call-script ids resolve to a .BIN script with a semantically-exact name (0x1ab->ADDITEM, 0x2ae7->MES, 0x143->BUNKI, 0x329d->CALCREVISE), 0 out-of-range, 0 pack-branch. See docs/engine-re.md + name-resolution.md #1." +confirm_by = "" +details = """ +op 0x03 (call-script, argc 1): `call-script `. RESOLVED — the id is a direct RAW index into +the SYS4INI global file table (the same table parse_sys4ini.py reads, but indexed WITHOUT skipping +'@' placeholders; SYS4INI has 13208 records / 2 placeholders). No separate on-disk id->code registry +exists; SYS4INI *is* the call-script registry. +Native mechanism (dispatch table `handler(op)=ctx[0x26c93+op]`, op 0x03 -> FUN_0041bc90): + 1. FUN_0041bc90 fetches operand 1 (id), bounds-checks call depth (<=0x26), pushes a frame. + 2. FUN_0040e980 (loader): opens the resource by id, reads the 0x20-byte SYS4 header, checks magic, + allocates per-frame code/local buffers from the header var-counts, reads the bytecode body, + pushes a script frame (stride 0x1e = 30 dwords, indexed by ctx[0x14f45]). + 3. FUN_0044f390 (resolver): record = [ctx+0x414] + id*0x50. Tries a LOOSE OVERRIDE first + (CreateFileA on record.name -> mod/patch hook point), else opens archive [record.arc_id*0x100 + + ctx+0x410], SetFilePointer to record.offset, size = record.size. + (High-byte-tagged ids `id & 0xff000000` select an alternate pack via [ctx+0x3028]; UNUSED by the + corpus -- 0/297 ids have a high byte.) +Companion op 0x8f `call` is INTRA-script (a local JSR), not cross-script -- see its entry. +This also names the whole call graph statically (build/callscript-names.json). +""" [[opcode.semantics.args]] i = 1 -role = "" +role = "script id = raw SYS4INI file index" observed_types = ["imm", "g-int", "l-ptr"] [[opcode]] @@ -1428,17 +1447,18 @@ abi_source = "kelebek+decode-validated" [opcode.semantics] name = "call" -category = "unknown" -summary = "" +category = "control" +summary = "intra-script subroutine call (local JSR): PC = frame.codebase + operand*4; pushes a return address on the per-frame return stack. NOT cross-script (that is call-script 0x03)." noop_headless = false -source = "kelebek" -confidence = "med" +source = "investigation" +confidence = "high" depends_on = [] -evidence = "" +evidence = "native-RE (Ghidra): handler FUN_0041fba0 (= ctx[0x26c93+0x8f]) sets [frame PC @+0x53d2c] = [frame codebase @+0x53d28] + operand*4 and pushes ((pc-base)>>2)+3 onto the per-frame return stack ([ctx+0x552e8]/[ctx+0x55248]). Target is a code OFFSET within the current script (matches header table T3 tag 0x8F = local call targets), confirming it is a local JSR, not a script load." +confirm_by = "" [[opcode.semantics.args]] i = 1 -role = "" +role = "local code target (word offset within current script)" observed_types = ["imm", "g-int"] [[opcode]]