Merge feat/callscript-resolution: call-script resolution + C# VM execution
- Native-RE: call-script id = raw SYS4INI file index (name-resolution #1 solved) - C# VM: call-script now executes (IScriptProvider + ExecFrame + nested run) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
gamer147
@@ -94,11 +94,54 @@ handler through the dispatch table (`ctx[0x26c93 + op]`). The raw VA is off by w
|
||||
|
||||
---
|
||||
|
||||
### op `0x03` (`call-script`) is a raw index into the SYS4INI file table — SOLVED (2026-07-07)
|
||||
|
||||
The long-deferred `call-script <id>` registry (`name-resolution.md §1`) is cracked. Resolved through
|
||||
the dispatch table (op `0x03` → `ctx[0x26c93+3]` = **`FUN_0041bc90`**), then the loader/resolver chain:
|
||||
|
||||
- **`FUN_0041bc90`** (handler): fetches operand 1 (the id), bounds-checks call depth (≤ 0x26), pushes
|
||||
a script frame, and calls the loader.
|
||||
- **`FUN_0040e980`** (loader): opens the resource by id, reads the **0x20-byte SYS4 header**, checks
|
||||
magic, allocates per-frame code/local buffers from the header var-counts, reads the bytecode body,
|
||||
and pushes a script frame (**stride 0x1e = 30 dwords**, indexed by `ctx[0x14f45]`). Returns to the
|
||||
caller when the callee ends.
|
||||
- **`FUN_0044f390`** (resolver — the key): `record = [ctx+0x414] + id*0x50`. The record is exactly the
|
||||
**SYS4INI 80-byte layout** `{name[64], arc_id@0x40, file_number@0x44, offset@0x48, size@0x4c}`
|
||||
(count = `[ctx+0x40c]`, archive-name table = `[ctx+0x410]`). It tries a **loose override first**
|
||||
(`CreateFileA` on `record.name` → the mod/patch hook point), else opens archive
|
||||
`[record.arc_id*0x100 + ctx+0x410]`, `SetFilePointer` to `record.offset`, size = `record.size`.
|
||||
High-byte-tagged ids (`id & 0xff000000`) select an alternate pack via `[ctx+0x3028]` — **unused by
|
||||
the corpus** (0/297 ids carry a high byte).
|
||||
|
||||
**So `call-script <id>` = a direct RAW index into the SYS4INI global file table** — the same table
|
||||
`parse_sys4ini.py` reads, but indexed *without* skipping `@` placeholders (13208 records, 2
|
||||
placeholders). There is **no separate on-disk id→code registry**; SYS4INI *is* the registry, and we
|
||||
already had it. **Statically confirmed:** all **297/297** distinct corpus `call-script` ids resolve to
|
||||
a `.BIN` script with a semantically-exact name (`0x1ab→ADDITEM`, `0x2ae7→MES`, `0x143→BUNKI`,
|
||||
`0x329d→CALCREVISE`, `0x2add→CALCBTPARAM`), 0 out-of-range, 0 pack-branch. Tooling:
|
||||
`parse_sys4ini.py` emits `build/callscript-names.json` (id→name); `sys4load` annotates
|
||||
`call-script 0x1ab =ADDITEM.BIN`; the whole `build/disasm/*.asm` call graph now reads by name. See
|
||||
`name-resolution.md §1`.
|
||||
|
||||
**Companion — op `0x8f` (`call`) is INTRA-script, not cross-script.** Its handler **`FUN_0041fba0`**
|
||||
sets `[frame PC @+0x53d2c] = [frame codebase @+0x53d28] + operand*4` and pushes a return address on
|
||||
the per-frame return stack (`[ctx+0x552e8]`/`[ctx+0x55248]`). The operand is a **code offset within
|
||||
the current script** (matches header table **T3, tag 0x8F** = local call targets). So `0x8f` is a
|
||||
local JSR; only `0x03` loads another script.
|
||||
|
||||
**Follow-up (functional):** the C# VM still *stubs* `call-script`. With the id→resource mapping now
|
||||
known, it can be implemented for real (load the target `.BIN` from the archive via the SYS4INI record,
|
||||
push a frame, run, return) — the unlock for subroutine-using scripts and, via the same path,
|
||||
decision→scene (scenes are just `SCxxxx.BIN` records loaded by their SYS4INI index).
|
||||
|
||||
---
|
||||
|
||||
## Native walls backlog (targets for this loop)
|
||||
|
||||
- **decision→scene** — how `0x62ccf`/the decision actually selects the next `SCxxxx` (re-aimed away
|
||||
from `u00428010`; likely call-script-adjacent).
|
||||
- **call-script dispatch** — `call-script <id>` → engine entry point (`name-resolution.md §1`).
|
||||
- ~~**call-script dispatch**~~ — **SOLVED** (above): `call-script <id>` = raw SYS4INI file index.
|
||||
- **decision→scene** — how `0x62ccf`/the decision selects the next `SCxxxx`. Now narrower: scenes load
|
||||
via `call-script`/the same SYS4INI-index loader, so the open question is only where the decision
|
||||
value is turned into a scene *id* (a caller of SCJUMP; re-aimed away from `u00428010`).
|
||||
- **op `0x60`** (`u0041A270`) — the rand-like value gating 1732/1755 SCJUMP decisions.
|
||||
- **gfx command-buffer** — the `0x212–0x21a` positioned-object subsystem (`scjump`-unrelated; the
|
||||
rendering drift).
|
||||
|
||||
@@ -12,10 +12,25 @@ would make it read like source.
|
||||
|
||||
---
|
||||
|
||||
## #1 — `call-script` target resolution (naming the call graph)
|
||||
## #1 — `call-script` target resolution (naming the call graph) — ✅ SOLVED (2026-07-07)
|
||||
|
||||
**What it is.** `call-script N` (Kelebek opcode 0x03) carries a bare number — `0x329d`,
|
||||
`0x2ade` — the id of an engine entry point. To render `call RECOVER` instead of
|
||||
**RESOLVED via native-RE.** `call-script <id>` is a **direct RAW index into the SYS4INI file table** —
|
||||
the very asset index we already parsed. No hidden engine registry: SYS4INI *is* the registry. Cracked
|
||||
by decompiling the handler chain in Ghidra (op 0x03 → `FUN_0041bc90` → loader `FUN_0040e980` →
|
||||
resolver `FUN_0044f390`, which does `record = table_base + id*0x50` over the 80-byte SYS4INI records).
|
||||
**Statically confirmed:** all 297/297 distinct corpus `call-script` ids resolve to a `.BIN` script with
|
||||
a semantically-exact name (`0x1ab→ADDITEM`, `0x2ae7→MES`, `0x143→BUNKI`), 0 out-of-range. Full
|
||||
mechanism in `engine-re.md` (“op 0x03 (call-script)…”). Tooling: `parse_sys4ini.py` →
|
||||
`build/callscript-names.json` (id→name); `sys4load` renders `call-script 0x1ab =ADDITEM.BIN`; the
|
||||
`build/disasm/*.asm` call graph now reads by name. The one caveat: index the RAW SYS4INI records
|
||||
(*including* the 2 `@` placeholders) — `asset-index.json` carries each entry's `raw_index` (= the id)
|
||||
for exactly this. **Remaining (functional, not naming):** the C# VM still stubs `call-script`
|
||||
execution; implementing it (load `.BIN` by id, push frame, run, return) is the follow-up. The original
|
||||
analysis (kept below for provenance) had concluded this was engine-level and deferred — it was, and
|
||||
the Ghidra loop is what resolved it.
|
||||
|
||||
**What it is (original framing).** `call-script N` (Kelebek opcode 0x03) carries a bare number —
|
||||
`0x329d`, `0x2ade` — the id of an engine entry point. To render `call RECOVER` instead of
|
||||
`call-script 0x329d` you need a table `id → (script, entry)`.
|
||||
|
||||
**Findings (inspected 2026-07-06):**
|
||||
@@ -40,16 +55,18 @@ on disk to read. Resolving it needs one of:
|
||||
- **Find the registration path** — if a boot script assigns ids to entry points, extract it
|
||||
statically (SYSTEM4.BIN is far too small to hold ~13k, so it's cumulative or lives in AGE.EXE).
|
||||
|
||||
**Status: deferred.** Not the quick win first assumed. Belongs with the engine/dispatch work
|
||||
(Phase 3), or a dedicated `SCJUMP.BIN` reverse. Until then `call-script` stays numeric.
|
||||
**Status: ✅ SOLVED** (see the banner at the top of this section). It did belong with the
|
||||
engine/dispatch work — the Ghidra + MCP loop resolved it via the opcode-dispatch table.
|
||||
|
||||
**Update (2026-07-07):** SCJUMP's *decision logic* is now decoded — `(chapter_mode, guards) →
|
||||
decision value` — see `docs/scjump-progression.md` and `tools/scjump_decode.py`. That confirmed
|
||||
SCJUMP is not the `call-script` registry (it produces a decision value, not a script id). The
|
||||
decision→scene hop is **native and still unidentified** — an earlier guess that op `u00428010`
|
||||
resolved it was **disproven via Ghidra** (that op is a graphics command-buffer op; see
|
||||
`docs/engine-re.md`). It's the same engine-level bucket as `call-script`; the Ghidra + MCP loop (and
|
||||
its recovered opcode-dispatch table) is now the tool to crack the `call-script`/script-load handler.
|
||||
SCJUMP is not the `call-script` registry (it produces a decision value, not a script id). Then the
|
||||
Ghidra + MCP loop **cracked `call-script` itself** (the SOLVED banner above): via the opcode-dispatch
|
||||
table it walked the handler → loader → resolver and found the id is a raw SYS4INI file index. What
|
||||
remains of the earlier `decision→scene` question is now narrow: scenes are `SCxxxx.BIN` records loaded
|
||||
through the *same* id-indexed loader, so the only open piece is where the SCJUMP decision *value*
|
||||
becomes a scene *id* (a caller of SCJUMP). The `u00428010` guess for that hop was disproven via Ghidra
|
||||
(it's a graphics command-buffer op; see `docs/engine-re.md`).
|
||||
|
||||
---
|
||||
|
||||
@@ -172,7 +189,7 @@ automatically (it reads `build/global-var-map.json` at load).
|
||||
|
||||
## How the two relate
|
||||
#1 names **functions** (the call graph); #2 names **data** (game state). In `RECOVER`, #1 turns
|
||||
`call-script 0x329d` into a name; #2 turns `C[unit][s] = E[unit][s]` into `unit.hp[s] =
|
||||
unit.maxHp[s]`. Priority reversal from the first guess: **#2 is the tractable readability lever
|
||||
now** (static handholds already half-built via the `*INIT` extraction); **#1 needs the engine**
|
||||
(dispatch reverse or Frida) and is deferred.
|
||||
`call-script 0x329d` into `CALCREVISE.BIN`; #2 turns `C[unit][s] = E[unit][s]` into `unit.hp[s] =
|
||||
unit.maxHp[s]`. Both are now largely in hand: **#1 is SOLVED** (the SYS4INI-index dispatch reverse —
|
||||
turned out to need the engine, and the Ghidra loop delivered it), and **#2 has a partial static map**
|
||||
(the `*INIT` handholds) that grows on demand.
|
||||
|
||||
@@ -27,6 +27,36 @@
|
||||
- **grounding:** source=investigation, confidence=high
|
||||
- **evidence:** By-ear confirmed (2026-07-06): SC0000 prologue voices play on their lines via Godot AudioStreamPlayer. Off-by-one disproven structurally: manifest interleaves graphics/voice (files[35]=EV049AA, [36]=MAN999, [37]=EV052CA, [38]=SYL0001), so files[base+id] lands voices on OGGs while files[base+id-1] would land them on .AGF graphics (silent) -- and they play, so the offset is exactly 0. Lily's lines are correctly form-gated (G[0xa57/0xa58/0xa59]) and stay silent when no form flag is seeded -- not a bug.
|
||||
|
||||
## control
|
||||
|
||||
### 0x3 `call-script` (call-script, argc 1)
|
||||
- **summary:** load & call another SYS4 script by id; id = RAW index into the SYS4INI file table (asset-index). Pushes a script frame; returns to caller when the callee ends.
|
||||
- **grounding:** source=investigation, confidence=high
|
||||
- **evidence:** native-RE (Ghidra): handler FUN_0041bc90 -> loader FUN_0040e980 -> resolver FUN_0044f390 indexes an 80-byte record table (base [ctx+0x414], count [ctx+0x40c]) at base+id*0x50 = the SYS4INI record layout {name[64],arc_id@0x40,file_number@0x44,offset@0x48,size@0x4c}. Confirmed statically: all 297 distinct corpus call-script ids resolve to a .BIN script with a semantically-exact name (0x1ab->ADDITEM, 0x2ae7->MES, 0x143->BUNKI, 0x329d->CALCREVISE), 0 out-of-range, 0 pack-branch. See docs/engine-re.md + name-resolution.md #1.
|
||||
|
||||
op 0x03 (call-script, argc 1): `call-script <id>`. RESOLVED — the id is a direct RAW index into
|
||||
the SYS4INI global file table (the same table parse_sys4ini.py reads, but indexed WITHOUT skipping
|
||||
'@' placeholders; SYS4INI has 13208 records / 2 placeholders). No separate on-disk id->code registry
|
||||
exists; SYS4INI *is* the call-script registry.
|
||||
Native mechanism (dispatch table `handler(op)=ctx[0x26c93+op]`, op 0x03 -> FUN_0041bc90):
|
||||
1. FUN_0041bc90 fetches operand 1 (id), bounds-checks call depth (<=0x26), pushes a frame.
|
||||
2. FUN_0040e980 (loader): opens the resource by id, reads the 0x20-byte SYS4 header, checks magic,
|
||||
allocates per-frame code/local buffers from the header var-counts, reads the bytecode body,
|
||||
pushes a script frame (stride 0x1e = 30 dwords, indexed by ctx[0x14f45]).
|
||||
3. FUN_0044f390 (resolver): record = [ctx+0x414] + id*0x50. Tries a LOOSE OVERRIDE first
|
||||
(CreateFileA on record.name -> mod/patch hook point), else opens archive [record.arc_id*0x100 +
|
||||
ctx+0x410], SetFilePointer to record.offset, size = record.size.
|
||||
(High-byte-tagged ids `id & 0xff000000` select an alternate pack via [ctx+0x3028]; UNUSED by the
|
||||
corpus -- 0/297 ids have a high byte.)
|
||||
Companion op 0x8f `call` is INTRA-script (a local JSR), not cross-script -- see its entry.
|
||||
This also names the whole call graph statically (build/callscript-names.json).
|
||||
|
||||
|
||||
### 0x8f `call` (call, argc 1)
|
||||
- **summary:** intra-script subroutine call (local JSR): PC = frame.codebase + operand*4; pushes a return address on the per-frame return stack. NOT cross-script (that is call-script 0x03).
|
||||
- **grounding:** source=investigation, confidence=high
|
||||
- **evidence:** native-RE (Ghidra): handler FUN_0041fba0 (= ctx[0x26c93+0x8f]) sets [frame PC @+0x53d2c] = [frame codebase @+0x53d28] + operand*4 and pushes ((pc-base)>>2)+3 onto the per-frame return stack ([ctx+0x552e8]/[ctx+0x55248]). Target is a code OFFSET within the current script (matches header table T3 tag 0x8F = local call targets), confirming it is a local JSR, not a script load.
|
||||
|
||||
## draw
|
||||
|
||||
### 0x1a2 `gfx-cmd-register` (u00428010, argc 1)
|
||||
@@ -179,10 +209,6 @@ op 0x90 (u0041BEB0, argc 7): `0x90 x y w h tgt_a tgt_b tgt_c`. Kelebek left it "
|
||||
- **summary:** —
|
||||
- **grounding:** source=kelebek, confidence=med
|
||||
|
||||
### 0x3 `call-script` (call-script, argc 1)
|
||||
- **summary:** —
|
||||
- **grounding:** source=kelebek, confidence=med
|
||||
|
||||
### 0x5 `ret` (ret, argc 0)
|
||||
- **summary:** —
|
||||
- **grounding:** source=kelebek, confidence=med
|
||||
@@ -375,10 +401,6 @@ op 0x90 (u0041BEB0, argc 7): `0x90 x y w h tgt_a tgt_b tgt_c`. Kelebek left it "
|
||||
- **summary:** —
|
||||
- **grounding:** source=kelebek, confidence=med
|
||||
|
||||
### 0x8f `call` (call, argc 1)
|
||||
- **summary:** —
|
||||
- **grounding:** source=kelebek, confidence=med
|
||||
|
||||
### 0x93 `u00415040` (u00415040, argc 0)
|
||||
- **summary:** —
|
||||
- **grounding:** source=kelebek, confidence=low
|
||||
|
||||
@@ -307,3 +307,44 @@ is only needed for scenes that genuinely use runtime-positioned sprites — revi
|
||||
Build `tools/vm0.py` and get the **RECOVER unit test** green (pointer/array/control-flow correctness),
|
||||
then run the first linear ADV scene against the dialogue oracle. That single result tells us whether
|
||||
the whole VM approach executes correctly — the load-bearing question behind option 3.
|
||||
|
||||
---
|
||||
|
||||
## ✅ call-script EXECUTION (2026-07-07) — subroutines now run in the C# VM
|
||||
|
||||
Spec `docs/superpowers/specs/2026-07-07-callscript-vm-execution-design.md`; plan
|
||||
`docs/superpowers/plans/2026-07-07-callscript-vm-execution.md`. Enabled by the native-RE finding that
|
||||
`call-script <id>` is a raw SYS4INI file index (`docs/engine-re.md`).
|
||||
|
||||
**Shipped (engine 25/25 green):**
|
||||
- **`IScriptProvider`** (Hosting) + **`Sys4ScriptProvider`** (Sys4): `id → build/callscript-names.json →
|
||||
name → Paths.Scripts() → Sys4Loader.Load`, cached. Injected into the VM so `Vm` never references `Sys4`.
|
||||
- **`ExecFrame` refactor** of `VirtualMachine`: per-script state (script, pc, locals, intra-call stack,
|
||||
per-frame emit-guard) moved into `ExecFrame`, run by a recursive `RunFrame`. Globals/Emitted/Steps stay
|
||||
VM-level (shared). Emitted lines now carry their source script name.
|
||||
- **`call-script` executes:** loads the child, runs it as a nested frame sharing globals, returns to the
|
||||
caller at `pc+1`. `exit`/`exit-script` and empty-stack `ret` return from the frame (top frame → HALT).
|
||||
Depth-capped (`VmOptions.CallDepthCap=64`; native limit 38). Shared globals are the return channel;
|
||||
per-call locals are discarded on return.
|
||||
- **Product paths** (`Age.Cli run/play/sweep`, `GameSession.RunScene`) inject the provider. `trace` +
|
||||
the `audio`/`gfx` diagnostics stay provider-less (base-ISA oracle / built against stub behavior).
|
||||
|
||||
**Design decision (refines the spec):** a VM with **no provider** falls back to the prior stub
|
||||
(`host.CallScript(id); pc+1`), not a halt. This keeps every existing base-ISA test byte-identical
|
||||
(they construct provider-less VMs) and needs no golden-fixture regeneration; **vm0.py retires from
|
||||
oracle duty gracefully** — `trace`/`TraceDiffTests`/`WaitForInputTests` stay on the stub path as the
|
||||
base-ISA guard, with zero lockstep maintenance.
|
||||
|
||||
**Validation:** 6 new tests (fake-provider unit tests: return-to-caller, callee `exit` returns not
|
||||
halts, shared-global visibility, per-frame local isolation, unresolved-id halt, no-provider stub;
|
||||
integration: ADDILL executes ADDILLSUB+CALCREVISE and reaches its own exit; BUNKI's top-level `ret`
|
||||
returns cleanly). **Corpus sweep (execution on): 284/297 exit clean, 13 STEP-LIMIT, 0 depth-cap, 0
|
||||
unresolved, 0 crashes.** The 13 STEP-LIMITs are input/state-gated ADV scenes (SC0000 etc.): executing
|
||||
subroutines makes their global-writes drive caller loops that headless can't break (no input;
|
||||
`WaitForInput` is a no-op) — the known state-divergence, not a call-script bug (loops hit STEP-LIMIT,
|
||||
not the depth cap → recursion is bounded correctly).
|
||||
|
||||
**Follow-ups (out of this slice):** wire the provider into the Godot play path (keep `--selftest`
|
||||
provider-less to preserve the 186-offset gate); optionally give the `audio`/`gfx` diagnostics a
|
||||
provider once their stub-era baselines are revisited; `decision→scene` (scene chaining) rides this same
|
||||
loader once the SCJUMP decision→scene-id native hop is reversed.
|
||||
|
||||
@@ -95,6 +95,25 @@ outside scenes lives.
|
||||
|
||||
---
|
||||
|
||||
## Call graph — scripts are addressable by `call-script <id>` (2026-07-07)
|
||||
|
||||
`call-script <id>` (opcode 0x03) loads another script by a **raw index into the SYS4INI file table**
|
||||
(id = the entry's `raw_index` = its global position in SYS4INI). This is the resolved call-graph
|
||||
registry — there is no separate id→code table; SYS4INI is it. Mechanism: `engine-re.md` (op 0x03
|
||||
section); id→name single source: `build/callscript-names.json` (from `parse_sys4ini.py`); `sys4load`
|
||||
and the regenerated `build/disasm/*.asm` corpus now render targets by name
|
||||
(`call-script 0x1ab =ADDITEM.BIN`). **297 distinct scripts are called** across the corpus (3002 sites);
|
||||
the hottest are `HISTORY` (backlog), `MENU`, `HIDEWIN`, `BUNKI` (branch), `MES` (message), `ADDITEM`,
|
||||
`ADDEN`, `LOOK`, `RENDERMAP`. Scenes (`SCxxxx.BIN`) load through the *same* id-indexed loader.
|
||||
|
||||
**Living-reference decision:** no separate generated markdown call-script reference is kept. Unlike
|
||||
`opcode-reference.md` / `global-reference.md` (rendered from *curated* knowledge bases), the id→name
|
||||
mapping is purely mechanical (SYS4INI index → filename) with no semantics to curate — it already lives
|
||||
in the build artifact and in the named disasm corpus. Full call-graph edges (caller→callee counts) are
|
||||
derivable on demand from the corpus; materialize a doc only if a consumer needs it.
|
||||
|
||||
---
|
||||
|
||||
## Implications for the port
|
||||
|
||||
1. **Much more game logic lives in bytecode than expected.** Damage formulas
|
||||
|
||||
644
docs/superpowers/plans/2026-07-07-callscript-vm-execution.md
Normal file
644
docs/superpowers/plans/2026-07-07-callscript-vm-execution.md
Normal file
@@ -0,0 +1,644 @@
|
||||
# call-script Execution in the C# VM — Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Make `call-script <id>` actually load and execute the target `.BIN` as a nested subroutine that shares the global bank and returns to its caller.
|
||||
|
||||
**Architecture:** Introduce an `IScriptProvider` seam (implemented in `Sys4`, injected into the VM so `Vm` never references `Sys4`) that maps a call-script id (a raw SYS4INI file index) to a loaded `Script`. Refactor the VM's per-script state into an `ExecFrame` and run scripts through a recursive `RunFrame`; `call-script` recurses into a child frame. Recursion is safe because call depth is bounded (native ≤ 38).
|
||||
|
||||
**Tech Stack:** C# / .NET 8, xUnit. Solution `engine/AgeEngine.sln` (projects `Age.Engine`, `Age.Cli`, `Age.Engine.Tests`).
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Run tests with: `dotnet test engine/AgeEngine.sln` (from `age-reimpl/`).
|
||||
- Seam rule: `Vm` code references only `Model` + `Hosting` namespaces, never `Sys4`. (One assembly; enforced by convention.)
|
||||
- `call-script <id>`: `id` is a **raw index into the SYS4INI file table** (`build/callscript-names.json` maps id→name; `Paths.Scripts()` maps name→path). Confirmed: all 297 corpus ids resolve to a DATA1 `.BIN`. See `docs/engine-re.md`.
|
||||
- **Deviation from spec (deliberate):** a VM with **no** `IScriptProvider` that hits `call-script` falls back to the **prior stub** (`_host.CallScript(id); pc+1`), NOT a halt. This keeps every existing base-ISA test byte-identical (they construct provider-less VMs) and avoids golden-fixture churn. Product paths always inject a provider, so execution is the real behavior where it matters.
|
||||
- Frequent commits: one per task minimum.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Script-provider seam + resolver
|
||||
|
||||
**Files:**
|
||||
- Create: `engine/Age.Engine/Hosting/IScriptProvider.cs`
|
||||
- Create: `engine/Age.Engine/Sys4/Sys4ScriptProvider.cs`
|
||||
- Modify: `engine/Age.Engine/Sys4/Paths.cs` (add `CallscriptNamesJson`)
|
||||
- Test: `engine/Age.Engine.Tests/Sys4ScriptProviderTests.cs`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `interface IScriptProvider { Script? GetById(long id); }` (namespace `Age.Engine.Hosting`).
|
||||
- Produces: `Sys4ScriptProvider.Load(OpcodeTable table) -> Sys4ScriptProvider`; instance `GetById(long id) -> Script?` (cached; unknown id → null).
|
||||
- Produces: `Paths.CallscriptNamesJson -> string`.
|
||||
- Consumes: existing `OpcodeTable`, `Sys4Loader.Load(string, OpcodeTable)`, `Paths.Scripts()`, `Paths.Build`.
|
||||
|
||||
- [ ] **Step 1: Add the path constant**
|
||||
|
||||
In `engine/Age.Engine/Sys4/Paths.cs`, after the `AssetIndexJson` line (line 12), add:
|
||||
|
||||
```csharp
|
||||
public static string CallscriptNamesJson => Path.Combine(Build, "callscript-names.json");
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Create the provider interface**
|
||||
|
||||
Create `engine/Age.Engine/Hosting/IScriptProvider.cs`:
|
||||
|
||||
```csharp
|
||||
using Age.Engine.Model;
|
||||
namespace Age.Engine.Hosting;
|
||||
|
||||
/// <summary>Resolves a call-script id (a raw SYS4INI file index) to a loaded <see cref="Script"/>.
|
||||
/// Implemented in Sys4; injected into the VM so the Vm layer never references Sys4.</summary>
|
||||
public interface IScriptProvider
|
||||
{
|
||||
/// <summary>The script for this id, or null if the id maps to no known script.</summary>
|
||||
Script? GetById(long id);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Write the failing test**
|
||||
|
||||
Create `engine/Age.Engine.Tests/Sys4ScriptProviderTests.cs`:
|
||||
|
||||
```csharp
|
||||
using Age.Engine.Sys4;
|
||||
using Xunit;
|
||||
|
||||
public class Sys4ScriptProviderTests
|
||||
{
|
||||
[Fact]
|
||||
public void ResolvesKnownIdsToTheirScripts()
|
||||
{
|
||||
var table = OpcodeTableJson.Load(Paths.OpcodesJson);
|
||||
var provider = Sys4ScriptProvider.Load(table);
|
||||
|
||||
var additem = provider.GetById(0x1ab); // ADDITEM.BIN
|
||||
var mes = provider.GetById(0x2ae7); // MES.BIN
|
||||
Assert.NotNull(additem);
|
||||
Assert.NotNull(mes);
|
||||
Assert.True(additem!.Instructions.Count > 0);
|
||||
Assert.Same(additem, provider.GetById(0x1ab)); // cached: same instance
|
||||
Assert.Null(provider.GetById(long.MaxValue)); // unknown id
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the test to verify it fails**
|
||||
|
||||
Run: `dotnet test engine/AgeEngine.sln --filter Sys4ScriptProviderTests`
|
||||
Expected: FAIL (compile error — `Sys4ScriptProvider` does not exist).
|
||||
|
||||
- [ ] **Step 5: Implement the provider**
|
||||
|
||||
Create `engine/Age.Engine/Sys4/Sys4ScriptProvider.cs`:
|
||||
|
||||
```csharp
|
||||
using System.Text.Json;
|
||||
using Age.Engine.Hosting;
|
||||
using Age.Engine.Model;
|
||||
namespace Age.Engine.Sys4;
|
||||
|
||||
/// <summary>Resolves call-script ids (raw SYS4INI file indices) to loaded scripts, using
|
||||
/// build/callscript-names.json (id→name) + Paths.Scripts() (name→path). Cached per id.
|
||||
/// The native resolver prefers a loose override before the archive; Paths.Scripts() already
|
||||
/// shadows extracted/DATA1 with root overrides, so that behavior is preserved.</summary>
|
||||
public sealed class Sys4ScriptProvider : IScriptProvider
|
||||
{
|
||||
private readonly OpcodeTable _table;
|
||||
private readonly IReadOnlyDictionary<long, string> _idToName;
|
||||
private readonly Dictionary<string, string> _byName; // NAME(UPPER) -> path
|
||||
private readonly Dictionary<long, Script?> _cache = new();
|
||||
|
||||
public Sys4ScriptProvider(OpcodeTable table, IReadOnlyDictionary<long, string> idToName,
|
||||
Dictionary<string, string> byName)
|
||||
{ _table = table; _idToName = idToName; _byName = byName; }
|
||||
|
||||
public static Sys4ScriptProvider Load(OpcodeTable table)
|
||||
{
|
||||
var raw = JsonSerializer.Deserialize<Dictionary<string, string>>(
|
||||
File.ReadAllText(Paths.CallscriptNamesJson)) ?? new();
|
||||
var idToName = raw.ToDictionary(kv => long.Parse(kv.Key), kv => kv.Value);
|
||||
return new Sys4ScriptProvider(table, idToName, Paths.Scripts());
|
||||
}
|
||||
|
||||
public Script? GetById(long id)
|
||||
{
|
||||
if (_cache.TryGetValue(id, out var cached)) return cached;
|
||||
Script? s = null;
|
||||
if (_idToName.TryGetValue(id, out var name) &&
|
||||
_byName.TryGetValue(name.ToUpperInvariant(), out var path))
|
||||
s = Sys4Loader.Load(path, _table);
|
||||
_cache[id] = s;
|
||||
return s;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Run the test to verify it passes**
|
||||
|
||||
Run: `dotnet test engine/AgeEngine.sln --filter Sys4ScriptProviderTests`
|
||||
Expected: PASS. (Prerequisite: `build/callscript-names.json` exists — regenerate with `py -3.11 -X utf8 tools/parse_sys4ini.py` if missing.)
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add engine/Age.Engine/Hosting/IScriptProvider.cs engine/Age.Engine/Sys4/Sys4ScriptProvider.cs engine/Age.Engine/Sys4/Paths.cs engine/Age.Engine.Tests/Sys4ScriptProviderTests.cs
|
||||
git commit -m "Add IScriptProvider + Sys4ScriptProvider (call-script id -> Script)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: ExecFrame refactor (no behavior change) + emitted script identity
|
||||
|
||||
Refactor the VM's per-script state into an `ExecFrame` run by a recursive `RunFrame`, and tag each emitted line with its source script. call-script stays a stub in this task. All existing tests must stay green (emitted offsets/counts/halt unchanged).
|
||||
|
||||
**Files:**
|
||||
- Create: `engine/Age.Engine/Vm/ExecFrame.cs`
|
||||
- Modify: `engine/Age.Engine/Model/Script.cs` (add `Name`)
|
||||
- Modify: `engine/Age.Engine/Sys4/Sys4Loader.cs` (set `Name`)
|
||||
- Modify: `engine/Age.Engine/Vm/VirtualMachine.cs` (the refactor + emitted tuple)
|
||||
- Modify: `engine/Age.Engine/Vm/GameSession.cs` (SceneResult tuple)
|
||||
- Modify: `engine/Age.Cli/Program.cs:17` (destructure fix)
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `Script.Name` (string, defaults `""`).
|
||||
- Produces: `VirtualMachine.Emitted` becomes `List<(int Offset, string Text, string Script)>`.
|
||||
- Produces (internal): `ExecFrame { Model.Script Script; int Pc; Frame Locals; List<int> CallStack; Dictionary<int,int> EmitSeen; }`.
|
||||
- Consumes: `IScriptProvider` (Task 1) — added to the constructor but unused until Task 3.
|
||||
|
||||
- [ ] **Step 1: Add `Name` to the Script model**
|
||||
|
||||
In `engine/Age.Engine/Model/Script.cs`, add a property (keep existing members):
|
||||
|
||||
```csharp
|
||||
public string Name { get; init; } = "";
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Set `Name` in the loader**
|
||||
|
||||
In `engine/Age.Engine/Sys4/Sys4Loader.cs`, in `Parse`, change the return (line ~25) to include the name:
|
||||
|
||||
```csharp
|
||||
return new Script { Name = name, Header = header, Instructions = instrs, IndexByOffset = idxByOff, Strings = strings };
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Create ExecFrame**
|
||||
|
||||
Create `engine/Age.Engine/Vm/ExecFrame.cs`:
|
||||
|
||||
```csharp
|
||||
using Age.Engine.Model;
|
||||
namespace Age.Engine.Vm;
|
||||
|
||||
/// <summary>One script activation: the running script, its instruction cursor, its local slots,
|
||||
/// its intra-script call/ret stack, and its per-script loop-guard map. Globals live on the VM and
|
||||
/// are shared across frames; everything here is per-call and discarded on return.</summary>
|
||||
internal sealed class ExecFrame
|
||||
{
|
||||
public readonly Script Script;
|
||||
public int Pc; // entry instruction index
|
||||
public readonly Frame Locals = new();
|
||||
public readonly List<int> CallStack = new(); // intra-script `call` (op 0x8f) returns
|
||||
public readonly Dictionary<int, int> EmitSeen = new();
|
||||
public ExecFrame(Script script, int pc) { Script = script; Pc = pc; }
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Refactor VirtualMachine to frames**
|
||||
|
||||
In `engine/Age.Engine/Vm/VirtualMachine.cs`:
|
||||
|
||||
(a) Add the constructor provider parameter and sentinels/fields. Replace the fields block + constructor (lines 7–27) so it reads:
|
||||
|
||||
```csharp
|
||||
private const long NoJump = 0xFFFFFFFF;
|
||||
private const int HALT = int.MinValue;
|
||||
private const int FRAME_RETURN = int.MinValue + 1;
|
||||
private const int T_IMM = 0, T_STR = 2, T_GINT = 3, T_GFLOAT = 4, T_GSTR = 5, T_GPTR = 6,
|
||||
T_LINT = 9, T_LFLOAT = 10, T_LSTR = 11, T_LPTR = 12;
|
||||
|
||||
private readonly Script _s;
|
||||
private readonly OpcodeTable _t;
|
||||
private readonly IHost _host;
|
||||
private readonly VmOptions _o;
|
||||
private readonly IScriptProvider? _provider;
|
||||
private ExecFrame _cur = null!;
|
||||
private int _depth;
|
||||
private bool _halted;
|
||||
|
||||
public Dictionary<int, long> Globals { get; } = new();
|
||||
public Dictionary<int, string> GlobalStrings { get; } = new();
|
||||
public List<(int Offset, string Text, string Script)> Emitted { get; } = new();
|
||||
public string? HaltReason { get; private set; }
|
||||
public long Steps { get; private set; }
|
||||
|
||||
public VirtualMachine(Script s, OpcodeTable t, IHost host, VmOptions? o = null, IScriptProvider? provider = null)
|
||||
{ _s = s; _t = t; _host = host; _o = o ?? new VmOptions(); _provider = provider; }
|
||||
```
|
||||
|
||||
(b) Replace `Run` (lines 94–106) with:
|
||||
|
||||
```csharp
|
||||
private enum FrameOutcome { Returned, Halted, RanOff }
|
||||
|
||||
public void Run(int entryOffset = 0)
|
||||
{
|
||||
var top = new ExecFrame(_s, _s.IndexByOffset.TryGetValue(entryOffset, out var idx) ? idx : 0);
|
||||
var outcome = RunFrame(top);
|
||||
if (outcome == FrameOutcome.RanOff) HaltReason ??= "pc-out-of-range";
|
||||
else if (outcome == FrameOutcome.Returned) HaltReason ??= "exit";
|
||||
// Halted: HaltReason already set by the halting op.
|
||||
}
|
||||
|
||||
private FrameOutcome RunFrame(ExecFrame frame)
|
||||
{
|
||||
var prev = _cur; _cur = frame; _depth++;
|
||||
var outcome = FrameOutcome.RanOff;
|
||||
int pc = frame.Pc;
|
||||
while (pc >= 0 && pc < frame.Script.Instructions.Count)
|
||||
{
|
||||
if (Steps >= _o.MaxSteps) { HaltReason ??= "STEP-LIMIT"; _halted = true; outcome = FrameOutcome.Halted; break; }
|
||||
Steps++;
|
||||
int next = Step(frame.Script.Instructions[pc], pc);
|
||||
if (next == FRAME_RETURN) { outcome = FrameOutcome.Returned; break; }
|
||||
if (next == HALT) { _halted = true; outcome = FrameOutcome.Halted; break; }
|
||||
pc = next;
|
||||
}
|
||||
_cur = prev; _depth--;
|
||||
return outcome;
|
||||
}
|
||||
```
|
||||
|
||||
(c) Repoint the helpers from `_fr`/`_s` to the current frame. Change every `_fr` to `_cur.Locals` in `Read`, `Write`, `ReadStr`, `WriteStr`, `BaseAddr`, `LookupStore` (the fields `_fr.I/F/S/P` become `_cur.Locals.I/F/S/P`).
|
||||
|
||||
(d) In `Step`, repoint frame-scoped state: `_callstack` → `_cur.CallStack`; `_emitSeen` → `_cur.EmitSeen`; `_s.IndexByOffset` → `_cur.Script.IndexByOffset`; `_s.GetString` → `_cur.Script.GetString`. Specifically:
|
||||
|
||||
- `jmp`: `return _cur.Script.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1);`
|
||||
- `call`: `_cur.CallStack.Add(pc + 1); return _cur.Script.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1);`
|
||||
- `ret`: `if (_cur.CallStack.Count > 0) { int r = _cur.CallStack[^1]; _cur.CallStack.RemoveAt(_cur.CallStack.Count - 1); return r; } return FRAME_RETURN;`
|
||||
- `jcc`: `return tgt == NoJump ? pc + 1 : _cur.Script.IndexByOffset.GetValueOrDefault((int)tgt, pc + 1);`
|
||||
- `exit` / `exit-script`: `return FRAME_RETURN;`
|
||||
- `show-text` body: `_cur.EmitSeen.TryGetValue(off, out var c); c++; _cur.EmitSeen[off] = c;` then on cap `HaltReason = $"LOOP:line@0x{off:x}×{c}"; return HALT;`, and `string text = _cur.Script.GetString(off); Emitted.Add((off, text, _cur.Script.Name)); _host.ShowText(off, text);`
|
||||
- `call-script`: leave the stub for now — `case "call-script": _host.CallScript(a.Count > 0 ? Read(a[0]) : 0); return pc + 1;`
|
||||
|
||||
- [ ] **Step 5: Fix the SceneResult tuple**
|
||||
|
||||
In `engine/Age.Engine/Vm/GameSession.cs` line 67, change the record to:
|
||||
|
||||
```csharp
|
||||
public sealed record SceneResult(IReadOnlyList<(int Offset, string Text, string Script)> Emitted, string? Halt, long Steps);
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Fix the one positional destructure**
|
||||
|
||||
In `engine/Age.Cli/Program.cs` line 17, change:
|
||||
|
||||
```csharp
|
||||
foreach (var (off, text, _) in vm.Emitted.Take(20)) Console.WriteLine($" [{off:x}] {text}");
|
||||
```
|
||||
|
||||
- [ ] **Step 7: Run the full suite to verify no behavior change**
|
||||
|
||||
Run: `dotnet test engine/AgeEngine.sln`
|
||||
Expected: PASS — all existing tests green. `WaitForInputTests` still asserts SC0000 = 186 (provider-less = stub), `RecoverTests`, `TextureOpsTests`, `GameSessionTests` unchanged. (`TraceDiffTests` passes if `build/vm0-trace.json` is present; it is skipped otherwise.)
|
||||
|
||||
- [ ] **Step 8: Commit**
|
||||
|
||||
```bash
|
||||
git add engine/Age.Engine/Vm/ExecFrame.cs engine/Age.Engine/Vm/VirtualMachine.cs engine/Age.Engine/Vm/GameSession.cs engine/Age.Engine/Model/Script.cs engine/Age.Engine/Sys4/Sys4Loader.cs engine/Age.Cli/Program.cs
|
||||
git commit -m "Refactor VM to ExecFrame + tag emitted lines with source script (no behavior change)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: call-script execution
|
||||
|
||||
Wire the provider into `call-script`: load the child script, run it as a nested frame sharing globals, return to the caller. `exit`/empty-stack `ret` return from the frame; only the top frame's return ends the VM. Depth-capped.
|
||||
|
||||
**Files:**
|
||||
- Modify: `engine/Age.Engine/Vm/VirtualMachine.cs` (the `call-script` case)
|
||||
- Modify: `engine/Age.Engine/Vm/VmOptions.cs` (depth cap)
|
||||
- Test: `engine/Age.Engine.Tests/CallScriptTests.cs`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `IScriptProvider.GetById` (Task 1), `ExecFrame`, `RunFrame`, `FrameOutcome` (Task 2).
|
||||
- Produces: nested execution semantics (below) exercised by later tasks.
|
||||
|
||||
- [ ] **Step 1: Add the depth-cap option**
|
||||
|
||||
In `engine/Age.Engine/Vm/VmOptions.cs`:
|
||||
|
||||
```csharp
|
||||
namespace Age.Engine.Vm;
|
||||
public sealed record VmOptions(int EmitCap = 2, long MaxSteps = 2_000_000, int CallDepthCap = 64);
|
||||
```
|
||||
|
||||
(64 is a generous runaway guard; the native limit is 38.)
|
||||
|
||||
- [ ] **Step 2: Write the failing tests**
|
||||
|
||||
Create `engine/Age.Engine.Tests/CallScriptTests.cs`. It uses a fake in-memory provider and hand-built scripts.
|
||||
|
||||
```csharp
|
||||
using System.Collections.Generic;
|
||||
using Age.Engine.Hosting;
|
||||
using Age.Engine.Model;
|
||||
using Age.Engine.Sys4;
|
||||
using Age.Engine.Vm;
|
||||
using Xunit;
|
||||
|
||||
public class CallScriptTests
|
||||
{
|
||||
private sealed class NullHost : IHost
|
||||
{
|
||||
public List<long> Calls = new();
|
||||
public void ShowText(int o, string t) { }
|
||||
public void CallScript(long id) => Calls.Add(id);
|
||||
public void OnStub(int op) { }
|
||||
public void WaitForInput() { }
|
||||
public void CreateTexture(int s, int w, int h) { }
|
||||
public void SetTexture(long r, int s) { }
|
||||
public void DrawTexture(int s, int sx, int sy, int w, int h, int dx, int dy) { }
|
||||
public (int Width, int Height) GetTextureSize(int s) => (0, 0);
|
||||
public void PlayBgm(long id) { }
|
||||
public void PlayVoice(long id) { }
|
||||
}
|
||||
|
||||
private sealed class MapProvider : IScriptProvider
|
||||
{
|
||||
private readonly Dictionary<long, Script> _m;
|
||||
public MapProvider(Dictionary<long, Script> m) => _m = m;
|
||||
public Script? GetById(long id) => _m.TryGetValue(id, out var s) ? s : null;
|
||||
}
|
||||
|
||||
// Build a Script from raw dwords via the real loader (guarantees identical decode).
|
||||
private static Script Asm(OpcodeTable t, string name, params uint[] body)
|
||||
{
|
||||
var bytes = new byte[0x3C + body.Length * 4];
|
||||
System.Text.Encoding.ASCII.GetBytes("SYS4422 ").CopyTo(bytes, 0);
|
||||
// fields[8] (F8 = code end) at header offset 8 + 8*4 = 0x28; set to body length (all code).
|
||||
System.BitConverter.GetBytes(body.Length).CopyTo(bytes, 8 + 8 * 4);
|
||||
for (int i = 0; i < body.Length; i++) System.BitConverter.GetBytes(body[i]).CopyTo(bytes, 0x3C + i * 4);
|
||||
return Sys4Loader.Parse(bytes, t, name);
|
||||
}
|
||||
|
||||
private static OpcodeTable Table() => OpcodeTableJson.Load(Paths.OpcodesJson);
|
||||
|
||||
[Fact]
|
||||
public void CalleeRunsAndControlResumesAfterTheCall()
|
||||
{
|
||||
var t = Table();
|
||||
// Callee (id 5): set global 0x10 = 7, then exit (op 0x0).
|
||||
var callee = Asm(t, "CALLEE", 0x55, 3, 0x10, 0, 7, 0x0); // mov g[0x10]=7 ; exit
|
||||
// Caller: call-script 5 ; set g[0x11]=g[0x10]+1 ; exit.
|
||||
var caller = Asm(t, "CALLER",
|
||||
0x03, 0, 5, // call-script 5
|
||||
0x55, 3, 0x11, 3, 0x10, // mov g[0x11] = g[0x10] (see note below)
|
||||
0x0); // exit
|
||||
var host = new NullHost();
|
||||
var vm = new VirtualMachine(caller, t, host, null, new MapProvider(new() { [5] = callee }));
|
||||
vm.Run();
|
||||
Assert.Equal(7, vm.Globals[0x10]); // callee wrote a shared global
|
||||
Assert.Equal(7, vm.Globals[0x11]); // caller read it AFTER the call returned
|
||||
Assert.Equal("exit", vm.HaltReason); // top-level exit
|
||||
Assert.Contains(5L, host.Calls); // host notified
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void MissingProviderFallsBackToStub()
|
||||
{
|
||||
var t = Table();
|
||||
var caller = Asm(t, "CALLER", 0x03, 0, 5, 0x0); // call-script 5 ; exit
|
||||
var host = new NullHost();
|
||||
var vm = new VirtualMachine(caller, t, host, null, null); // no provider
|
||||
vm.Run();
|
||||
Assert.Equal("exit", vm.HaltReason); // did not halt on the call; stub + continue
|
||||
Assert.Contains(5L, host.Calls);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void UnresolvedIdHalts()
|
||||
{
|
||||
var t = Table();
|
||||
var caller = Asm(t, "CALLER", 0x03, 0, 99, 0x0);
|
||||
var vm = new VirtualMachine(caller, t, new NullHost(), null, new MapProvider(new()));
|
||||
vm.Run();
|
||||
Assert.StartsWith("callscript-unresolved", vm.HaltReason);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void LocalsDoNotLeakBetweenCallerAndCallee()
|
||||
{
|
||||
var t = Table();
|
||||
// Callee writes LOCAL-int 0 = 42 (op 0x55 to local-int, type 9), then exit.
|
||||
var callee = Asm(t, "CALLEE", 0x55, 9, 0, 0, 42, 0x0);
|
||||
// Caller sets local-int 0 = 1, calls, then copies its own local-int 0 to global 0x20.
|
||||
var caller = Asm(t, "CALLER",
|
||||
0x55, 9, 0, 0, 1, // l[0] = 1
|
||||
0x03, 0, 5, // call-script 5 (callee sets ITS local 0 = 42)
|
||||
0x55, 3, 0x20, 9, 0, // g[0x20] = l[0]
|
||||
0x0);
|
||||
var vm = new VirtualMachine(caller, t, new NullHost(), null, new MapProvider(new() { [5] = callee }));
|
||||
vm.Run();
|
||||
Assert.Equal(1, vm.Globals[0x20]); // caller's local 0 unchanged by callee's local 0
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Note on `mov g[0x11] = g[0x10]`: the `mov` opcode 0x55 with a 2-operand form copies src→dst; the test encodes dst=`(type 3 = global-int, 0x11)`, src=`(type 3, 0x10)`. If the corpus `mov` is strictly 2-arg, keep argc=2 as encoded (`0x55, 3,0x11, 3,0x10`). Confirm the opcode's argc via `build/opcodes.json` when implementing; adjust the dword stream to match the real argc so `Sys4Loader.Parse` decodes it as one instruction.
|
||||
|
||||
- [ ] **Step 3: Run the tests to verify they fail**
|
||||
|
||||
Run: `dotnet test engine/AgeEngine.sln --filter CallScriptTests`
|
||||
Expected: FAIL — `CalleeRunsAndControlResumesAfterTheCall`, `UnresolvedIdHalts`, `LocalsDoNotLeakBetweenCallerAndCallee` fail (call-script is still the stub, so the callee never runs / never halts on unresolved). `MissingProviderFallsBackToStub` may already pass.
|
||||
|
||||
- [ ] **Step 4: Implement call-script execution**
|
||||
|
||||
In `engine/Age.Engine/Vm/VirtualMachine.cs`, replace the `call-script` stub case with:
|
||||
|
||||
```csharp
|
||||
case "call-script":
|
||||
{
|
||||
long id = a.Count > 0 ? Read(a[0]) : 0;
|
||||
_host.CallScript(id); // notify (diagnostics)
|
||||
if (_provider == null) return pc + 1; // no script source: prior stub behavior
|
||||
if (_depth >= _o.CallDepthCap) { HaltReason ??= "call-depth-exceeded"; return HALT; }
|
||||
var child = _provider.GetById(id);
|
||||
if (child == null) { HaltReason ??= $"callscript-unresolved:0x{id:x}"; return HALT; }
|
||||
var entry = child.IndexByOffset.TryGetValue(0, out var ci) ? ci : 0;
|
||||
var outcome = RunFrame(new ExecFrame(child, entry));
|
||||
if (outcome == FrameOutcome.Halted) return HALT; // propagate whole-VM halt up
|
||||
return pc + 1; // Returned / RanOff: resume caller
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run the tests to verify they pass**
|
||||
|
||||
Run: `dotnet test engine/AgeEngine.sln --filter CallScriptTests`
|
||||
Expected: PASS (all four). If the `mov` encoding decodes wrong, fix the dword stream per the Step-2 note.
|
||||
|
||||
- [ ] **Step 6: Run the full suite (no regressions)**
|
||||
|
||||
Run: `dotnet test engine/AgeEngine.sln`
|
||||
Expected: PASS — existing provider-less tests unchanged (SC0000 still 186 in `WaitForInputTests`).
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add engine/Age.Engine/Vm/VirtualMachine.cs engine/Age.Engine/Vm/VmOptions.cs engine/Age.Engine.Tests/CallScriptTests.cs
|
||||
git commit -m "Execute call-script: nested frame, shared globals, return to caller"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Wire the provider into product paths + real-scene validation
|
||||
|
||||
Thread a real `Sys4ScriptProvider` into the CLI run/play/sweep paths and `GameSession.RunScene`, prove a real scene executes a real subroutine, and confirm the corpus still terminates with execution on. Confirm the empty-stack `ret` path with a real script (BUNKI).
|
||||
|
||||
**Files:**
|
||||
- Modify: `engine/Age.Engine/Vm/GameSession.cs` (`RunScene` gains an optional provider)
|
||||
- Modify: `engine/Age.Cli/Program.cs` (construct + pass the provider in `run`, `play`, `sweep`)
|
||||
- Test: `engine/Age.Engine.Tests/CallScriptIntegrationTests.cs`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `Sys4ScriptProvider.Load` (Task 1), `VirtualMachine(..., provider)` (Task 2/3).
|
||||
- Produces: `GameSession.RunScene(Script, OpcodeTable, IHost, VmOptions?, IScriptProvider?)`.
|
||||
|
||||
- [ ] **Step 1: Add the provider to GameSession.RunScene**
|
||||
|
||||
In `engine/Age.Engine/Vm/GameSession.cs`, change the `RunScene` signature and VM construction:
|
||||
|
||||
```csharp
|
||||
public SceneResult RunScene(Script script, OpcodeTable table, IHost host,
|
||||
VmOptions? options = null, IScriptProvider? provider = null)
|
||||
{
|
||||
var vm = new VirtualMachine(script, table, host, options, provider);
|
||||
```
|
||||
|
||||
(The rest of the method is unchanged.)
|
||||
|
||||
- [ ] **Step 2: Write the failing integration tests**
|
||||
|
||||
Create `engine/Age.Engine.Tests/CallScriptIntegrationTests.cs`:
|
||||
|
||||
```csharp
|
||||
using Age.Engine.Hosting;
|
||||
using Age.Engine.Sys4;
|
||||
using Age.Engine.Vm;
|
||||
using Xunit;
|
||||
|
||||
public class CallScriptIntegrationTests
|
||||
{
|
||||
private sealed class NullHost : IHost
|
||||
{
|
||||
public int CallScripts;
|
||||
public void ShowText(int o, string t) { }
|
||||
public void CallScript(long id) => CallScripts++;
|
||||
public void OnStub(int op) { }
|
||||
public void WaitForInput() { }
|
||||
public void CreateTexture(int s, int w, int h) { }
|
||||
public void SetTexture(long r, int s) { }
|
||||
public void DrawTexture(int s, int sx, int sy, int w, int h, int dx, int dy) { }
|
||||
public (int Width, int Height) GetTextureSize(int s) => (0, 0);
|
||||
public void PlayBgm(long id) { }
|
||||
public void PlayVoice(long id) { }
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void RealSceneExecutesRealSubroutinesAndReturns()
|
||||
{
|
||||
var t = OpcodeTableJson.Load(Paths.OpcodesJson);
|
||||
var provider = Sys4ScriptProvider.Load(t);
|
||||
var script = Sys4Loader.Load(Paths.Scripts()["ALCHEMY.BIN"], t); // calls MES/BUNKI/ADDITEM
|
||||
var host = new NullHost();
|
||||
var vm = new VirtualMachine(script, t, host, null, provider);
|
||||
vm.Run();
|
||||
Assert.True(host.CallScripts > 0, "the scene should issue call-scripts");
|
||||
// Termination is the key property: execution-on must still reach a clean end, not hang or
|
||||
// trip the depth cap. (No unresolved / depth halts.)
|
||||
Assert.DoesNotContain("unresolved", vm.HaltReason ?? "");
|
||||
Assert.NotEqual("call-depth-exceeded", vm.HaltReason);
|
||||
Assert.NotEqual("STEP-LIMIT", vm.HaltReason);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void BunkiTopLevelRetReturnsCleanlyAsSubroutine()
|
||||
{
|
||||
// BUNKI.BIN ends with a top-level `ret` (empty intra-call stack). Called as a subroutine it
|
||||
// must return to the caller, not underflow-halt. Drive it directly as a child of a 1-op caller.
|
||||
var t = OpcodeTableJson.Load(Paths.OpcodesJson);
|
||||
var provider = Sys4ScriptProvider.Load(t);
|
||||
var bunki = provider.GetById(0x143); // BUNKI.BIN
|
||||
Assert.NotNull(bunki);
|
||||
var vm = new VirtualMachine(bunki!, t, new NullHost(), null, provider);
|
||||
vm.Run();
|
||||
// Reaching a frame-return at the top = clean "exit"; never "ret-underflow".
|
||||
Assert.NotEqual("ret-underflow", vm.HaltReason);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run to verify (expected: mostly pass already)**
|
||||
|
||||
Run: `dotnet test engine/AgeEngine.sln --filter CallScriptIntegrationTests`
|
||||
Expected: PASS. If `BunkiTopLevelRetReturnsCleanlyAsSubroutine` reveals a wrong `ret` semantics (e.g. it halts early with a bad reason), that is the empty-stack-`ret` branch — verify against the native op 0x5 handler `ctx[0x26c93+5]` (LAB_00417ad0) before adjusting; the plan's choice (empty-stack `ret` = frame return) should hold.
|
||||
|
||||
- [ ] **Step 4: Wire the provider into the CLI**
|
||||
|
||||
In `engine/Age.Cli/Program.cs`, build one provider after `table` is loaded and pass it into the product paths:
|
||||
|
||||
- In the `run` command (line ~13), construct `var provider = Sys4ScriptProvider.Load(table);` and change the VM to `new VirtualMachine(script, table, host, null, provider)`.
|
||||
- In `play` (line ~96), construct the provider once before the loop and pass it: `session.RunScene(script, table, new CaptureHost(), null, provider)`.
|
||||
- In `sweep` (lines ~121, 140, 162): construct the provider once and pass it to each `RunScene(...)` call (`, null, provider`). The `--boot` INIT scripts may be run with or without the provider; pass it for consistency.
|
||||
- Leave `trace` (line ~181) provider-less on purpose — it is the base-ISA offset oracle (stub behavior, comparable to vm0.py).
|
||||
|
||||
Exact edit for `run` (lines 12–17):
|
||||
|
||||
```csharp
|
||||
var table = OpcodeTableJson.Load(Paths.OpcodesJson);
|
||||
var provider = Sys4ScriptProvider.Load(table);
|
||||
var script = Sys4Loader.Load(args[1], table);
|
||||
var host = new ConsoleHost();
|
||||
var vm = new VirtualMachine(script, table, host, null, provider);
|
||||
vm.Run();
|
||||
```
|
||||
|
||||
(Use whatever host `run` already constructs; only the trailing `provider` arg is added.)
|
||||
|
||||
- [ ] **Step 5: Confirm the corpus still terminates with execution on**
|
||||
|
||||
Build and run the sweep:
|
||||
|
||||
```bash
|
||||
dotnet run --project engine/Age.Cli -- sweep
|
||||
```
|
||||
|
||||
Expected: it completes; halt distribution is dominated by `exit`. Note any new non-`exit` halts (e.g. `call-depth-exceeded`, `callscript-unresolved`, `STEP-LIMIT`) — none should appear for well-formed scenes. Compare the anomaly list to the pre-change baseline (unbooted sweep previously matched vm0.py: 294 `exit` + 3 `LOOP`). New anomalies are the review signal.
|
||||
|
||||
- [ ] **Step 6: Run the full suite**
|
||||
|
||||
Run: `dotnet test engine/AgeEngine.sln`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add engine/Age.Engine/Vm/GameSession.cs engine/Age.Cli/Program.cs engine/Age.Engine.Tests/CallScriptIntegrationTests.cs
|
||||
git commit -m "Wire Sys4ScriptProvider into CLI run/play/sweep; validate real subroutine execution"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Post-implementation notes (not tasks)
|
||||
|
||||
- **Godot host:** the playable Godot path (`GodotAdvHost`/`Main`) should construct a `Sys4ScriptProvider` and pass it to its `VirtualMachine`, so subroutines execute live; keep the `--selftest` path **provider-less** (preserves the 186-offset SC0000 gate). This is a small follow-up wiring, mirroring Task 4's CLI changes.
|
||||
- **vm0.py** is not modified. `trace`/`TraceDiffTests`/`WaitForInputTests` remain on the provider-less (stub) path, so vm0.py stays a valid base-ISA reference without lockstep maintenance.
|
||||
- **decision→scene** (scene chaining) is out of scope; it rides this same loader once the SCJUMP decision→scene-id hop is reversed.
|
||||
- After landing, update the status memory (`himegari-port-status.md`) and `docs/phase-a-slice-plan.md` with the result.
|
||||
|
||||
## Self-review
|
||||
|
||||
- **Spec coverage:** provider seam (Task 1) ✓; recursive frame execution (Task 2) ✓; shared globals / per-frame locals / exit+ret semantics / depth cap / dynamic ids / IHost.CallScript notify (Task 3) ✓; emitted script identity (Task 2) ✓; validation via C#-owned tests + sweep-terminates + real subroutine + empty-stack-ret (Tasks 3–4) ✓; vm0.py retired without deletion, base-ISA guard preserved (Global Constraints + notes) ✓. Deviation (no-provider → stub, not halt) is documented in Global Constraints.
|
||||
- **Placeholder scan:** none — every code step shows full code; the one encoding caveat (`mov` argc) has an explicit resolution instruction.
|
||||
- **Type consistency:** `IScriptProvider.GetById` (Task 1) is consumed unchanged in Tasks 2–4; `Emitted`/`SceneResult` tuple `(int Offset, string Text, string Script)` is introduced in Task 2 and consumers updated in the same task; `RunFrame`/`FrameOutcome`/`ExecFrame`/`_provider`/`_depth` defined in Task 2 and used in Task 3; `VmOptions.CallDepthCap` defined in Task 3 before first use.
|
||||
@@ -0,0 +1,149 @@
|
||||
# call-script execution in the C# VM — design (2026-07-07)
|
||||
|
||||
## Goal
|
||||
|
||||
Make `call-script <id>` (opcode 0x03) **actually execute** in the C# engine (`Age.Engine`): load the
|
||||
target `.BIN` by id, run it as a nested subroutine that shares the global bank, and return to the
|
||||
caller. This unlocks the ~297 subroutine scripts the corpus calls (MES, ADDITEM, the CALC* family,
|
||||
SHOWGROW, …) which are currently stubbed — the first functional step past "single script runs" toward
|
||||
a driven playthrough.
|
||||
|
||||
Enabled by the native-RE finding that `call-script <id>` is a direct raw index into the SYS4INI file
|
||||
table (see `docs/engine-re.md`, "op 0x03 (call-script)"; `docs/name-resolution.md §1`).
|
||||
|
||||
## Scope
|
||||
|
||||
**In scope:** subroutine execution — nested load + run + return, sharing globals, with per-call local
|
||||
frames. Validated headless + in the existing hosts.
|
||||
|
||||
**Out of scope (this slice):**
|
||||
- Scene chaining / decision→scene (the SCJUMP decision-value → scene-id native hop is still open).
|
||||
- Input model (`0x90`) and any new interactive behavior.
|
||||
- The graphics geometry/blend drift (a separate, state-divergence problem).
|
||||
- Reimplementing call-script in `vm0.py` (see Validation — vm0.py is retired from oracle duty).
|
||||
|
||||
## Background — current architecture
|
||||
|
||||
`VirtualMachine` (`engine/Age.Engine/Vm/VirtualMachine.cs`) runs **one** `Script` with **one** `Frame`
|
||||
and a single intra-script `_callstack` (for op 0x8f `call`). `call-script` today is a stub:
|
||||
`_host.CallScript(id); return pc + 1;` — the target never runs.
|
||||
|
||||
Scripts are loaded by **name** from `Paths.Scripts()` (the override-aware corpus map: root loose-file
|
||||
overrides shadow `extracted/DATA1/*.BIN`) via `Sys4Loader.Load`. `GameSession` carries the flat global
|
||||
bank across scenes; local frames correctly do not persist. Seam rule: `Vm` references only `Model` +
|
||||
`Hosting`, never `Sys4`.
|
||||
|
||||
Corpus facts grounding the design: called scripts terminate with `exit` (ADDITEM, MES, SHOWGROW) or
|
||||
`ret` (BUNKI); native call depth is bounded (≤ 0x26 = 38); all 297 distinct call-script ids resolve to
|
||||
a DATA1 `.BIN` (0 out-of-range, 0 alternate-pack).
|
||||
|
||||
## Design
|
||||
|
||||
### 1. Script-provider seam
|
||||
|
||||
A small interface in `Hosting` (which may reference `Model`), injected into the VM so `Vm` never
|
||||
references `Sys4`:
|
||||
|
||||
```csharp
|
||||
public interface IScriptProvider { Script? GetById(long id); }
|
||||
```
|
||||
|
||||
- `Sys4` implements `Sys4ScriptProvider`: `id → build/callscript-names.json → name →
|
||||
Paths.Scripts()[name] → Sys4Loader.Load`, **cached by id** (MES is called 59×; parse once). Reusing
|
||||
`Paths.Scripts()` gives the native "loose override first" behavior for free (it already prefers root
|
||||
overrides over the archive copy).
|
||||
- The VM takes the provider as an **optional** constructor dependency. Product paths (CLI `play`,
|
||||
Godot) always supply one, so execution is effectively always-on. The provider is a genuine
|
||||
dependency, not a feature flag — you cannot run a script you cannot load. A VM constructed without a
|
||||
provider that then hits `call-script` halts with a clear reason (it never happens on product paths).
|
||||
|
||||
### 2. Recursive frame execution
|
||||
|
||||
Per-script state moves out of instance fields into an `ExecFrame`:
|
||||
|
||||
- `Script` being run, the local `Frame` (I/F/S/P slots), the intra-script `call`/`ret` stack, the `pc`,
|
||||
and the **emit-seen** loop-guard map.
|
||||
|
||||
The run loop becomes `RunFrame(ExecFrame)`. `Run(entry)` builds the top frame and calls it.
|
||||
`call-script id` → `provider.GetById(Read(a[0]))` → build a child `ExecFrame` (fresh locals, entry
|
||||
pc 0) → `RunFrame(child)` → on return, the caller continues at `pc + 1`. Recursion (not an explicit
|
||||
stack list) is chosen because depth is bounded (≤ 38), so there is no overflow risk and
|
||||
`exit`-returns-to-caller falls out as a plain return from `RunFrame`.
|
||||
|
||||
Shared VM-level state stays on the instance: `Globals`, `GlobalStrings`, `Emitted`, `Steps`, `_host`,
|
||||
`_provider`.
|
||||
|
||||
### 3. Semantics
|
||||
|
||||
- **Shared globals = the return channel.** A callee returns results by writing globals the caller reads
|
||||
(the native shared-global model; no explicit return value). Local frames are per-call and discarded
|
||||
on return (matches `GameSession`).
|
||||
- **`exit` / `exit-script`:** pop the current frame. Empty stack (top-level) → HALT; otherwise return
|
||||
to the caller at its `call-script` + 1. Only the outermost `exit` halts.
|
||||
- **`ret` (op 0x5):** returns from an intra-script `call` (0x8f) via the frame's `call`-stack. When a
|
||||
script's top-level flow reaches `ret` with an empty `call`-stack (e.g. BUNKI), it returns from the
|
||||
**script frame** (same as `exit`). **This empty-stack `ret` behavior is confirmed against the native
|
||||
op 0x5 handler (`ctx[0x26c93+5]` = `LAB_00417ad0`) during implementation**, not guessed.
|
||||
- **Emit-seen is per-frame.** The loop guard keys on string offset, and offsets are script-local
|
||||
(MES `0x100` ≠ SC0000 `0x100`); a shared map would collide and falsely trip the cap on hot
|
||||
subroutines. It lives in `ExecFrame`, reset per call.
|
||||
- **Depth cap:** cap recursion at ~38 frames; exceeding halts with a distinct reason (the native throws
|
||||
there). Guards runaway / mutual recursion.
|
||||
- **Steps:** one monotonic counter across all frames (subroutine steps count toward it).
|
||||
- **`Emitted`:** aggregates all frames' text in execution order; each entry carries the source script's
|
||||
identity so mixed output is disambiguable in golden traces.
|
||||
- **Dynamic ids:** the operand is usually immediate but may be g-int/l-ptr; `Read(a[0])` handles all
|
||||
types, so a computed target resolves through the same provider.
|
||||
- **`IHost.CallScript(id)`** is kept as a fire-on-entry notification (diagnostics/logging); control flow
|
||||
is now VM-owned. Existing host implementations are unaffected.
|
||||
|
||||
## Validation
|
||||
|
||||
**vm0.py is retired from oracle duty.** It served its purpose (prototyping the execution model and
|
||||
proving the C# port byte-identical across 297 scenes); it stays in the repo as a frozen historical
|
||||
reference and Python-side experiment tool but is no longer maintained in lockstep. Reimplementing
|
||||
call-script in it would be double work for a shrinking payoff (the C# engine now also has real-game
|
||||
ground truth vm0.py never had).
|
||||
|
||||
The C# engine **owns its correctness fixtures**:
|
||||
|
||||
- **Golden traces regenerated from the C# engine** and checked into the repo as the source of truth
|
||||
(replacing "match whatever vm0.py emits"). Scenes that call no subroutines are unchanged (and, for a
|
||||
cheap belt-and-suspenders check, still happen to match vm0.py — an optional narrowed parity test).
|
||||
- **New on-path checks:**
|
||||
- the corpus sweep (`Age.Cli sweep`) still **terminates cleanly** with execution on (no new hangs /
|
||||
depth-cap trips across the corpus);
|
||||
- **known subroutines execute and return** — e.g. a scene calling MES/ADDITEM shows the subroutine's
|
||||
effects (emitted text / global writes) and control resumes after the call;
|
||||
- the empty-stack `ret` and `exit`-returns-to-caller branches have unit tests with hand-built frames;
|
||||
- the new SC0000 opening trace (subroutines now executed) is validated by inspection against the
|
||||
real-game ground truth we hold (Frida load order, screenshots, by-ear audio).
|
||||
|
||||
## Testing
|
||||
|
||||
- **Unit (xUnit, `Age.Engine.Tests`):**
|
||||
- a fake in-memory `IScriptProvider` returning hand-built `Script`s: caller → callee → return to
|
||||
caller at `pc+1`; callee `exit` returns (does not halt caller); nested depth; shared-global
|
||||
write-visible-to-caller; per-frame locals don't leak; depth-cap halt; missing-id halt.
|
||||
- empty-stack `ret` returns from the frame (after the native-handler confirmation).
|
||||
- **Integration:** `Sys4ScriptProvider` resolves real ids (0x1ab→ADDITEM, 0x2ae7→MES) and caches;
|
||||
a real scene that calls MES executes it.
|
||||
- **Corpus:** `sweep` terminates with execution on; golden-trace fixtures regenerated + committed.
|
||||
- **Regression:** existing non-call-script scenes produce unchanged golden traces.
|
||||
|
||||
## Risks / open items
|
||||
|
||||
- **Empty-stack `ret` semantics** — resolved by decompiling `LAB_00417ad0` before relying on it.
|
||||
- **Headless zero-state subroutines** — with no seeded state some callees may take odd branches; the
|
||||
sweep-terminates check is the guard. Not a correctness bug in call-script itself (state divergence).
|
||||
- **Golden-trace churn** — one-time regeneration; the diff is reviewed, not blindly accepted.
|
||||
|
||||
## Files touched (anticipated)
|
||||
|
||||
- `Age.Engine/Hosting/IScriptProvider.cs` (new), `IHost.cs` (unchanged; `CallScript` kept as notify).
|
||||
- `Age.Engine/Vm/VirtualMachine.cs` (ExecFrame refactor + call-script execution), `Vm/Frame.cs`
|
||||
(possibly `ExecFrame`), `Vm/VmOptions.cs` (depth cap constant).
|
||||
- `Age.Engine/Sys4/Sys4ScriptProvider.cs` (new), `Sys4/Paths.cs` (callscript-names.json path if needed).
|
||||
- `Age.Cli/Program.cs` (wire the provider into `play`/`run`/`sweep`), Godot `GodotAdvHost`/`Main`
|
||||
(supply the provider).
|
||||
- `Age.Engine.Tests/*` (new call-script tests), regenerated golden fixtures.
|
||||
@@ -26,7 +26,7 @@ whenever a tool's inputs/outputs change.**
|
||||
|
||||
| Tool | Purpose | Run | Reads → Writes |
|
||||
|---|---|---|---|
|
||||
| `sys4load.py` | Loader + opcode-decoding disassembler for SYS4 `.BIN` scripts (the container-format core every other tool builds on). | `sys4load.py <file.BIN>` · `--summary` · `--strings` · `--json` · `sys4load.py <dir> --validate` (corpus check) | `.BIN` + `age_opcodes*.py` + `build/global-var-map.json` → stdout listing, or `build/scripts-json/` with `--json` |
|
||||
| `sys4load.py` | Loader + opcode-decoding disassembler for SYS4 `.BIN` scripts (the container-format core every other tool builds on). Annotates global operands (`build/globals.json`) and **`call-script` targets by name** (`build/callscript-names.json`, e.g. `call-script 0x1ab =ADDITEM.BIN`). | `sys4load.py <file.BIN>` · `--summary` · `--strings` · `--json` · `sys4load.py <dir> --validate` (corpus check) | `.BIN` + `age_opcodes*.py` + `build/globals.json` + `build/callscript-names.json` → stdout listing, or `build/scripts-json/` with `--json` |
|
||||
| `age_opcodes.py` | 548-entry Kelebek AGE opcode/arg-type table. **PRISTINE upstream data — never edit.** | *Imported.* | — |
|
||||
|
||||
## Opcode reference toolchain — single source of truth = `vm-map/opcodes.toml`
|
||||
@@ -90,7 +90,7 @@ plays `SC0000` from the real bytecode. User args (after `--`):
|
||||
|
||||
| Tool | Purpose | Run | Reads → Writes |
|
||||
|---|---|---|---|
|
||||
| `parse_sys4ini.py` | Parse `SYS4INI.BIN` (S4IC422, LZSS-compressed) into the authoritative asset index — name ↔ archive ↔ offset ↔ size for all DATA*.ALF (the `resId→file` answer key). | `parse_sys4ini.py [--check]` (`--check` validates vs `extracted/` + `.ALF` sizes) | `姫狩り…/SYS4INI.BIN` → `build/asset-index.json` |
|
||||
| `parse_sys4ini.py` | Parse `SYS4INI.BIN` (S4IC422, LZSS-compressed) into the authoritative asset index — name ↔ archive ↔ offset ↔ size for all DATA*.ALF (the `resId→file` answer key). Each entry carries `raw_index` (its 0-based position in the SYS4INI record table incl. `@` placeholders) = the engine's universal file id. Also emits the **`call-script <id> → name`** map (id = `raw_index`; see `engine-re.md`). | `parse_sys4ini.py [--check]` (`--check` validates vs `extracted/` + `.ALF` sizes) | `姫狩り…/SYS4INI.BIN` → `build/asset-index.json` + `build/callscript-names.json` |
|
||||
| `resolve_asset.py` | ★ **The static asset resolver.** SYS4INI is sectioned (one per scene: `SCxxxx.BIN` + its cross-archive manifest; `file_number` = index within section). Resolves `resId → files[section_base(scene) + resId]` for graphics AND audio, no capture. | `resolve_asset.py --build` · `resolve_asset.py <SCENE> [resId]` | `build/asset-index.json` → `build/asset-sections.json`; resolves any (scene, resId) |
|
||||
| `resolve_frida_reads.py` | Rescue noisy Frida archive-read offsets → asset names via the index (per-archive range search; drops 0x20000 paging reads); recovers the per-scene asset load order. | `resolve_frida_reads.py [reads.log] [-o out.json]` | `build/frida-reads.log` + `build/asset-index.json` → `build/frida-asset-loads.json` |
|
||||
| `convert_agf.py` | Convert AGF stills to BMP via `AGF2BMP2AGF.exe` (searches all `extracted/DATA*`). `--scene` batch-converts a scene's whole SYS4INI manifest — feeds the Godot render. | `convert_agf.py EV052CA.AGF …` · `convert_agf.py --scene SC0000` | `extracted/DATA*/*.AGF` → `build/textures/*.BMP` |
|
||||
|
||||
@@ -5,16 +5,19 @@ using Age.Engine.Sys4;
|
||||
using Age.Engine.Vm;
|
||||
|
||||
var table = OpcodeTableJson.Load(Paths.OpcodesJson);
|
||||
// call-script execution: resolves ids -> scripts. Product paths pass this so subroutines run;
|
||||
// `trace` stays provider-less on purpose (the base-ISA offset oracle).
|
||||
var provider = Sys4ScriptProvider.Load(table);
|
||||
|
||||
if (args.Length == 0) { Console.WriteLine("usage: run <file> | trace <out.json>"); return 1; }
|
||||
|
||||
if (args[0] == "run")
|
||||
{
|
||||
var script = Sys4Loader.Load(args[1], table);
|
||||
var vm = new VirtualMachine(script, table, new CaptureHost());
|
||||
var vm = new VirtualMachine(script, table, new CaptureHost(), null, provider);
|
||||
vm.Run();
|
||||
Console.WriteLine($"{Path.GetFileName(args[1])}: {vm.Steps} steps, {vm.Emitted.Count} show-text (halt: {vm.HaltReason})");
|
||||
foreach (var (off, text) in vm.Emitted.Take(20)) Console.WriteLine($" [{off:x}] {text}");
|
||||
foreach (var (off, text, _) in vm.Emitted.Take(20)) Console.WriteLine($" [{off:x}] {text}");
|
||||
return 0;
|
||||
}
|
||||
|
||||
@@ -94,7 +97,7 @@ if (args[0] == "play")
|
||||
foreach (var name in scenes)
|
||||
{
|
||||
var script = Sys4Loader.Load(scripts[name.ToUpperInvariant()], table);
|
||||
var r = session.RunScene(script, table, new CaptureHost());
|
||||
var r = session.RunScene(script, table, new CaptureHost(), null, provider);
|
||||
totalLines += r.Emitted.Count;
|
||||
Console.WriteLine($" {name,-14} {r.Emitted.Count,4} lines, {r.Steps,7} steps (halt: {r.Halt})");
|
||||
}
|
||||
@@ -118,7 +121,7 @@ if (args[0] == "sweep")
|
||||
var bootSession = new GameSession();
|
||||
foreach (var s in new[] { "SKINIT.BIN", "ITINIT.BIN", "EBINIT.BIN", "CGINIT.BIN", "MPINIT.BIN",
|
||||
"AFINIT.BIN", "CCINIT.BIN", "STINIT.BIN", "STINIT2.BIN" })
|
||||
bootSession.RunScene(Sys4Loader.Load(scripts[s], table), table, new CaptureHost());
|
||||
bootSession.RunScene(Sys4Loader.Load(scripts[s], table), table, new CaptureHost(), null, provider);
|
||||
baseline = bootSession.ToJson();
|
||||
Console.WriteLine($"[boot] baseline = {bootSession.Globals.Count} globals; running {names.Count} scenes from it.");
|
||||
}
|
||||
@@ -137,7 +140,7 @@ if (args[0] == "sweep")
|
||||
{
|
||||
var session = Fresh();
|
||||
if (seeded) foreach (var (k, v) in seeds) session.Seed(k, v);
|
||||
return session.RunScene(Sys4Loader.Load(scripts[name], table), table, new CaptureHost()).Emitted.Count;
|
||||
return session.RunScene(Sys4Loader.Load(scripts[name], table), table, new CaptureHost(), null, provider).Emitted.Count;
|
||||
}
|
||||
|
||||
if (seeds.Count > 0)
|
||||
@@ -159,7 +162,7 @@ if (args[0] == "sweep")
|
||||
foreach (var name in names)
|
||||
{
|
||||
var session = Fresh();
|
||||
var r = session.RunScene(Sys4Loader.Load(scripts[name], table), table, new CaptureHost());
|
||||
var r = session.RunScene(Sys4Loader.Load(scripts[name], table), table, new CaptureHost(), null, provider);
|
||||
var halt = r.Halt ?? "null";
|
||||
haltDist[halt] = haltDist.GetValueOrDefault(halt) + 1;
|
||||
totalLines += r.Emitted.Count;
|
||||
|
||||
52
engine/Age.Engine.Tests/CallScriptIntegrationTests.cs
Normal file
52
engine/Age.Engine.Tests/CallScriptIntegrationTests.cs
Normal file
@@ -0,0 +1,52 @@
|
||||
using Age.Engine.Hosting;
|
||||
using Age.Engine.Sys4;
|
||||
using Age.Engine.Vm;
|
||||
using Xunit;
|
||||
|
||||
public class CallScriptIntegrationTests
|
||||
{
|
||||
private sealed class NullHost : IHost
|
||||
{
|
||||
public int CallScripts;
|
||||
public void ShowText(int o, string t) { }
|
||||
public void CallScript(long id) => CallScripts++;
|
||||
public void OnStub(int op) { }
|
||||
public void WaitForInput() { }
|
||||
public void CreateTexture(int s, int w, int h) { }
|
||||
public void SetTexture(long r, int s) { }
|
||||
public void DrawTexture(int s, int sx, int sy, int w, int h, int dx, int dy) { }
|
||||
public (int Width, int Height) GetTextureSize(int s) => (0, 0);
|
||||
public void PlayBgm(long id) { }
|
||||
public void PlayVoice(long id) { }
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void RealScriptExecutesRealSubroutinesAndReturns()
|
||||
{
|
||||
// ADDILL.BIN unconditionally call-scripts ADDILLSUB then CALCREVISE at entry, then exits.
|
||||
// With execution on, both subroutines load, run, and return, so ADDILL reaches its own exit.
|
||||
var t = OpcodeTableJson.Load(Paths.OpcodesJson);
|
||||
var provider = Sys4ScriptProvider.Load(t);
|
||||
var script = Sys4Loader.Load(Paths.Scripts()["ADDILL.BIN"], t);
|
||||
var host = new NullHost();
|
||||
var vm = new VirtualMachine(script, t, host, null, provider);
|
||||
vm.Run();
|
||||
Assert.Equal(2, host.CallScripts); // ADDILLSUB + CALCREVISE both dispatched
|
||||
Assert.Equal("exit", vm.HaltReason); // subroutines returned; ADDILL reached its own exit
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void BunkiTopLevelRetReturnsCleanlyAsSubroutine()
|
||||
{
|
||||
// BUNKI.BIN ends with a top-level `ret` (empty intra-call stack). Called as a subroutine it
|
||||
// must return to the caller, not underflow-halt. Drive it directly.
|
||||
var t = OpcodeTableJson.Load(Paths.OpcodesJson);
|
||||
var provider = Sys4ScriptProvider.Load(t);
|
||||
var bunki = provider.GetById(0x143); // BUNKI.BIN
|
||||
Assert.NotNull(bunki);
|
||||
var vm = new VirtualMachine(bunki!, t, new NullHost(), null, provider);
|
||||
vm.Run();
|
||||
// Reaching a frame-return at the top = clean "exit"; never "ret-underflow".
|
||||
Assert.NotEqual("ret-underflow", vm.HaltReason);
|
||||
}
|
||||
}
|
||||
107
engine/Age.Engine.Tests/CallScriptTests.cs
Normal file
107
engine/Age.Engine.Tests/CallScriptTests.cs
Normal file
@@ -0,0 +1,107 @@
|
||||
using System.Collections.Generic;
|
||||
using Age.Engine.Hosting;
|
||||
using Age.Engine.Model;
|
||||
using Age.Engine.Sys4;
|
||||
using Age.Engine.Vm;
|
||||
using Xunit;
|
||||
|
||||
public class CallScriptTests
|
||||
{
|
||||
// Opcodes (from build/opcodes.json): exit=0x2, call-script=0x3(argc1), mov=0x55(argc2).
|
||||
// Operand types: imm=0, global-int=3, local-int=9.
|
||||
private const uint OP_EXIT = 0x2, OP_CALLSCRIPT = 0x3, OP_MOV = 0x55;
|
||||
|
||||
private sealed class NullHost : IHost
|
||||
{
|
||||
public List<long> Calls = new();
|
||||
public void ShowText(int o, string t) { }
|
||||
public void CallScript(long id) => Calls.Add(id);
|
||||
public void OnStub(int op) { }
|
||||
public void WaitForInput() { }
|
||||
public void CreateTexture(int s, int w, int h) { }
|
||||
public void SetTexture(long r, int s) { }
|
||||
public void DrawTexture(int s, int sx, int sy, int w, int h, int dx, int dy) { }
|
||||
public (int Width, int Height) GetTextureSize(int s) => (0, 0);
|
||||
public void PlayBgm(long id) { }
|
||||
public void PlayVoice(long id) { }
|
||||
}
|
||||
|
||||
private sealed class MapProvider : IScriptProvider
|
||||
{
|
||||
private readonly Dictionary<long, Script> _m;
|
||||
public MapProvider(Dictionary<long, Script> m) => _m = m;
|
||||
public Script? GetById(long id) => _m.TryGetValue(id, out var s) ? s : null;
|
||||
}
|
||||
|
||||
// Build a Script from raw dwords via the real loader (guarantees identical decode).
|
||||
private static Script Asm(OpcodeTable t, string name, params uint[] body)
|
||||
{
|
||||
var bytes = new byte[0x3C + body.Length * 4];
|
||||
System.Text.Encoding.ASCII.GetBytes("SYS4422 ").CopyTo(bytes, 0);
|
||||
// fields[8] (F8 = code end) at header offset 8 + 8*4 = 0x28; set to body length (all code).
|
||||
System.BitConverter.GetBytes(body.Length).CopyTo(bytes, 8 + 8 * 4);
|
||||
for (int i = 0; i < body.Length; i++) System.BitConverter.GetBytes(body[i]).CopyTo(bytes, 0x3C + i * 4);
|
||||
return Sys4Loader.Parse(bytes, t, name);
|
||||
}
|
||||
|
||||
private static OpcodeTable Table() => OpcodeTableJson.Load(Paths.OpcodesJson);
|
||||
|
||||
[Fact]
|
||||
public void CalleeRunsAndControlResumesAfterTheCall()
|
||||
{
|
||||
var t = Table();
|
||||
// Callee (id 5): mov g[0x10] = 7, then exit.
|
||||
var callee = Asm(t, "CALLEE", OP_MOV, 3, 0x10, 0, 7, OP_EXIT);
|
||||
// Caller: call-script 5 ; mov g[0x11] = g[0x10] ; exit.
|
||||
var caller = Asm(t, "CALLER",
|
||||
OP_CALLSCRIPT, 0, 5,
|
||||
OP_MOV, 3, 0x11, 3, 0x10,
|
||||
OP_EXIT);
|
||||
var host = new NullHost();
|
||||
var vm = new VirtualMachine(caller, t, host, null, new MapProvider(new() { [5] = callee }));
|
||||
vm.Run();
|
||||
Assert.Equal(7, vm.Globals[0x10]); // callee wrote a shared global
|
||||
Assert.Equal(7, vm.Globals[0x11]); // caller read it AFTER the call returned
|
||||
Assert.Equal("exit", vm.HaltReason); // top-level exit
|
||||
Assert.Contains(5L, host.Calls); // host notified
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void MissingProviderFallsBackToStub()
|
||||
{
|
||||
var t = Table();
|
||||
var caller = Asm(t, "CALLER", OP_CALLSCRIPT, 0, 5, OP_EXIT);
|
||||
var host = new NullHost();
|
||||
var vm = new VirtualMachine(caller, t, host, null, null); // no provider
|
||||
vm.Run();
|
||||
Assert.Equal("exit", vm.HaltReason); // did not halt on the call; stub + continue
|
||||
Assert.Contains(5L, host.Calls);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void UnresolvedIdHalts()
|
||||
{
|
||||
var t = Table();
|
||||
var caller = Asm(t, "CALLER", OP_CALLSCRIPT, 0, 99, OP_EXIT);
|
||||
var vm = new VirtualMachine(caller, t, new NullHost(), null, new MapProvider(new()));
|
||||
vm.Run();
|
||||
Assert.StartsWith("callscript-unresolved", vm.HaltReason);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void LocalsDoNotLeakBetweenCallerAndCallee()
|
||||
{
|
||||
var t = Table();
|
||||
// Callee writes LOCAL-int 0 = 42 (mov to local-int, type 9), then exit.
|
||||
var callee = Asm(t, "CALLEE", OP_MOV, 9, 0, 0, 42, OP_EXIT);
|
||||
// Caller sets local-int 0 = 1, calls, then copies its own local-int 0 to global 0x20.
|
||||
var caller = Asm(t, "CALLER",
|
||||
OP_MOV, 9, 0, 0, 1, // l[0] = 1
|
||||
OP_CALLSCRIPT, 0, 5, // call-script 5 (callee sets ITS local 0 = 42)
|
||||
OP_MOV, 3, 0x20, 9, 0, // g[0x20] = l[0]
|
||||
OP_EXIT);
|
||||
var vm = new VirtualMachine(caller, t, new NullHost(), null, new MapProvider(new() { [5] = callee }));
|
||||
vm.Run();
|
||||
Assert.Equal(1, vm.Globals[0x20]); // caller's local 0 unchanged by callee's local 0
|
||||
}
|
||||
}
|
||||
20
engine/Age.Engine.Tests/Sys4ScriptProviderTests.cs
Normal file
20
engine/Age.Engine.Tests/Sys4ScriptProviderTests.cs
Normal file
@@ -0,0 +1,20 @@
|
||||
using Age.Engine.Sys4;
|
||||
using Xunit;
|
||||
|
||||
public class Sys4ScriptProviderTests
|
||||
{
|
||||
[Fact]
|
||||
public void ResolvesKnownIdsToTheirScripts()
|
||||
{
|
||||
var table = OpcodeTableJson.Load(Paths.OpcodesJson);
|
||||
var provider = Sys4ScriptProvider.Load(table);
|
||||
|
||||
var additem = provider.GetById(0x1ab); // ADDITEM.BIN
|
||||
var mes = provider.GetById(0x2ae7); // MES.BIN
|
||||
Assert.NotNull(additem);
|
||||
Assert.NotNull(mes);
|
||||
Assert.True(additem!.Instructions.Count > 0);
|
||||
Assert.Same(additem, provider.GetById(0x1ab)); // cached: same instance
|
||||
Assert.Null(provider.GetById(long.MaxValue)); // unknown id
|
||||
}
|
||||
}
|
||||
10
engine/Age.Engine/Hosting/IScriptProvider.cs
Normal file
10
engine/Age.Engine/Hosting/IScriptProvider.cs
Normal file
@@ -0,0 +1,10 @@
|
||||
using Age.Engine.Model;
|
||||
namespace Age.Engine.Hosting;
|
||||
|
||||
/// <summary>Resolves a call-script id (a raw SYS4INI file index) to a loaded <see cref="Script"/>.
|
||||
/// Implemented in Sys4; injected into the VM so the Vm layer never references Sys4.</summary>
|
||||
public interface IScriptProvider
|
||||
{
|
||||
/// <summary>The script for this id, or null if the id maps to no known script.</summary>
|
||||
Script? GetById(long id);
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
namespace Age.Engine.Model;
|
||||
public sealed class Script
|
||||
{
|
||||
public string Name { get; init; } = "";
|
||||
public required ScriptHeader Header { get; init; }
|
||||
public required IReadOnlyList<Instruction> Instructions { get; init; }
|
||||
public required IReadOnlyDictionary<int, int> IndexByOffset { get; init; }
|
||||
|
||||
@@ -10,6 +10,7 @@ public static class Paths
|
||||
public static string OpcodesJson => Path.Combine(Build, "opcodes.json");
|
||||
public static string AssetSectionsJson => Path.Combine(Build, "asset-sections.json");
|
||||
public static string AssetIndexJson => Path.Combine(Build, "asset-index.json");
|
||||
public static string CallscriptNamesJson => Path.Combine(Build, "callscript-names.json");
|
||||
public static string Textures => Path.Combine(Build, "textures");
|
||||
|
||||
private static string FindRepo()
|
||||
|
||||
@@ -22,7 +22,7 @@ public static class Sys4Loader
|
||||
|
||||
var header = new ScriptHeader(fields[0], fields[1], fields[2], fields[3], fields[4], fields[5]);
|
||||
var (instrs, idxByOff, strings) = DecodeCode(dw, fields, nbody, table);
|
||||
return new Script { Header = header, Instructions = instrs, IndexByOffset = idxByOff, Strings = strings };
|
||||
return new Script { Name = name, Header = header, Instructions = instrs, IndexByOffset = idxByOff, Strings = strings };
|
||||
}
|
||||
|
||||
private static (List<Instruction>, Dictionary<int, int>, Dictionary<int, string>)
|
||||
|
||||
39
engine/Age.Engine/Sys4/Sys4ScriptProvider.cs
Normal file
39
engine/Age.Engine/Sys4/Sys4ScriptProvider.cs
Normal file
@@ -0,0 +1,39 @@
|
||||
using System.Text.Json;
|
||||
using Age.Engine.Hosting;
|
||||
using Age.Engine.Model;
|
||||
namespace Age.Engine.Sys4;
|
||||
|
||||
/// <summary>Resolves call-script ids (raw SYS4INI file indices) to loaded scripts, using
|
||||
/// build/callscript-names.json (id→name) + Paths.Scripts() (name→path). Cached per id.
|
||||
/// The native resolver prefers a loose override before the archive; Paths.Scripts() already
|
||||
/// shadows extracted/DATA1 with root overrides, so that behavior is preserved.</summary>
|
||||
public sealed class Sys4ScriptProvider : IScriptProvider
|
||||
{
|
||||
private readonly OpcodeTable _table;
|
||||
private readonly IReadOnlyDictionary<long, string> _idToName;
|
||||
private readonly Dictionary<string, string> _byName; // NAME(UPPER) -> path
|
||||
private readonly Dictionary<long, Script?> _cache = new();
|
||||
|
||||
public Sys4ScriptProvider(OpcodeTable table, IReadOnlyDictionary<long, string> idToName,
|
||||
Dictionary<string, string> byName)
|
||||
{ _table = table; _idToName = idToName; _byName = byName; }
|
||||
|
||||
public static Sys4ScriptProvider Load(OpcodeTable table)
|
||||
{
|
||||
var raw = JsonSerializer.Deserialize<Dictionary<string, string>>(
|
||||
File.ReadAllText(Paths.CallscriptNamesJson)) ?? new();
|
||||
var idToName = raw.ToDictionary(kv => long.Parse(kv.Key), kv => kv.Value);
|
||||
return new Sys4ScriptProvider(table, idToName, Paths.Scripts());
|
||||
}
|
||||
|
||||
public Script? GetById(long id)
|
||||
{
|
||||
if (_cache.TryGetValue(id, out var cached)) return cached;
|
||||
Script? s = null;
|
||||
if (_idToName.TryGetValue(id, out var name) &&
|
||||
_byName.TryGetValue(name.ToUpperInvariant(), out var path))
|
||||
s = Sys4Loader.Load(path, _table);
|
||||
_cache[id] = s;
|
||||
return s;
|
||||
}
|
||||
}
|
||||
15
engine/Age.Engine/Vm/ExecFrame.cs
Normal file
15
engine/Age.Engine/Vm/ExecFrame.cs
Normal file
@@ -0,0 +1,15 @@
|
||||
using Age.Engine.Model;
|
||||
namespace Age.Engine.Vm;
|
||||
|
||||
/// <summary>One script activation: the running script, its instruction cursor, its local slots,
|
||||
/// its intra-script call/ret stack, and its per-script loop-guard map. Globals live on the VM and
|
||||
/// are shared across frames; everything here is per-call and discarded on return.</summary>
|
||||
internal sealed class ExecFrame
|
||||
{
|
||||
public readonly Script Script;
|
||||
public int Pc; // entry instruction index
|
||||
public readonly Frame Locals = new();
|
||||
public readonly List<int> CallStack = new(); // intra-script `call` (op 0x8f) returns
|
||||
public readonly Dictionary<int, int> EmitSeen = new();
|
||||
public ExecFrame(Script script, int pc) { Script = script; Pc = pc; }
|
||||
}
|
||||
@@ -24,9 +24,10 @@ public sealed class GameSession
|
||||
public void SeedString(int addr, string value) => GlobalStrings[addr] = value;
|
||||
|
||||
/// <summary>Run one scene: seed a fresh VM from session state, execute, merge final state back.</summary>
|
||||
public SceneResult RunScene(Script script, OpcodeTable table, IHost host, VmOptions? options = null)
|
||||
public SceneResult RunScene(Script script, OpcodeTable table, IHost host,
|
||||
VmOptions? options = null, IScriptProvider? provider = null)
|
||||
{
|
||||
var vm = new VirtualMachine(script, table, host, options);
|
||||
var vm = new VirtualMachine(script, table, host, options, provider);
|
||||
foreach (var kv in Globals) vm.Globals[kv.Key] = kv.Value;
|
||||
foreach (var kv in GlobalStrings) vm.GlobalStrings[kv.Key] = kv.Value;
|
||||
|
||||
@@ -64,4 +65,4 @@ public sealed class GameSession
|
||||
}
|
||||
|
||||
/// <summary>The observable result of running one scene into a <see cref="GameSession"/>.</summary>
|
||||
public sealed record SceneResult(IReadOnlyList<(int Offset, string Text)> Emitted, string? Halt, long Steps);
|
||||
public sealed record SceneResult(IReadOnlyList<(int Offset, string Text, string Script)> Emitted, string? Halt, long Steps);
|
||||
|
||||
@@ -6,6 +6,7 @@ public sealed class VirtualMachine
|
||||
{
|
||||
private const long NoJump = 0xFFFFFFFF;
|
||||
private const int HALT = int.MinValue;
|
||||
private const int FRAME_RETURN = int.MinValue + 1;
|
||||
private const int T_IMM = 0, T_STR = 2, T_GINT = 3, T_GFLOAT = 4, T_GSTR = 5, T_GPTR = 6,
|
||||
T_LINT = 9, T_LFLOAT = 10, T_LSTR = 11, T_LPTR = 12;
|
||||
|
||||
@@ -13,18 +14,18 @@ public sealed class VirtualMachine
|
||||
private readonly OpcodeTable _t;
|
||||
private readonly IHost _host;
|
||||
private readonly VmOptions _o;
|
||||
private readonly Frame _fr = new();
|
||||
private readonly List<int> _callstack = new();
|
||||
private readonly Dictionary<int, int> _emitSeen = new();
|
||||
private readonly IScriptProvider? _provider;
|
||||
private ExecFrame _cur = null!;
|
||||
private int _depth;
|
||||
|
||||
public Dictionary<int, long> Globals { get; } = new();
|
||||
public Dictionary<int, string> GlobalStrings { get; } = new();
|
||||
public List<(int Offset, string Text)> Emitted { get; } = new();
|
||||
public List<(int Offset, string Text, string Script)> Emitted { get; } = new();
|
||||
public string? HaltReason { get; private set; }
|
||||
public long Steps { get; private set; }
|
||||
|
||||
public VirtualMachine(Script s, OpcodeTable t, IHost host, VmOptions? o = null)
|
||||
{ _s = s; _t = t; _host = host; _o = o ?? new VmOptions(); }
|
||||
public VirtualMachine(Script s, OpcodeTable t, IHost host, VmOptions? o = null, IScriptProvider? provider = null)
|
||||
{ _s = s; _t = t; _host = host; _o = o ?? new VmOptions(); _provider = provider; }
|
||||
|
||||
private static long Gi(Dictionary<int, long> d, int k) => d.TryGetValue(k, out var v) ? v : 0;
|
||||
private static string Gs(Dictionary<int, string> d, int k) => d.TryGetValue(k, out var v) ? v : "";
|
||||
@@ -38,9 +39,9 @@ public sealed class VirtualMachine
|
||||
T_IMM => op.Value,
|
||||
T_GINT or T_GFLOAT => Gi(Globals, (int)op.Value),
|
||||
T_GPTR => Gi(Globals, (int)Gi(Globals, (int)op.Value)),
|
||||
T_LINT => Gi(_fr.I, (int)op.Value),
|
||||
T_LFLOAT => Gi(_fr.F, (int)op.Value),
|
||||
T_LPTR => Gi(Globals, (int)Gi(_fr.P, (int)op.Value)),
|
||||
T_LINT => Gi(_cur.Locals.I, (int)op.Value),
|
||||
T_LFLOAT => Gi(_cur.Locals.F, (int)op.Value),
|
||||
T_LPTR => Gi(Globals, (int)Gi(_cur.Locals.P, (int)op.Value)),
|
||||
_ => op.Value,
|
||||
};
|
||||
|
||||
@@ -50,17 +51,17 @@ public sealed class VirtualMachine
|
||||
{
|
||||
case T_GINT: case T_GFLOAT: Globals[(int)op.Value] = val; break;
|
||||
case T_GPTR: Globals[(int)Gi(Globals, (int)op.Value)] = val; break;
|
||||
case T_LINT: _fr.I[(int)op.Value] = val; break;
|
||||
case T_LFLOAT: _fr.F[(int)op.Value] = val; break;
|
||||
case T_LPTR: Globals[(int)Gi(_fr.P, (int)op.Value)] = val; break;
|
||||
case T_LINT: _cur.Locals.I[(int)op.Value] = val; break;
|
||||
case T_LFLOAT: _cur.Locals.F[(int)op.Value] = val; break;
|
||||
case T_LPTR: Globals[(int)Gi(_cur.Locals.P, (int)op.Value)] = val; break;
|
||||
}
|
||||
}
|
||||
|
||||
private string ReadStr(Operand op) => op.Type switch
|
||||
{
|
||||
T_STR => _s.GetString((int)op.Value),
|
||||
T_STR => _cur.Script.GetString((int)op.Value),
|
||||
T_GSTR => Gs(GlobalStrings, (int)op.Value),
|
||||
T_LSTR => Gs(_fr.S, (int)op.Value),
|
||||
T_LSTR => Gs(_cur.Locals.S, (int)op.Value),
|
||||
_ => "",
|
||||
};
|
||||
|
||||
@@ -69,15 +70,15 @@ public sealed class VirtualMachine
|
||||
switch (op.Type)
|
||||
{
|
||||
case T_GSTR: GlobalStrings[(int)op.Value] = val; break;
|
||||
case T_LSTR: _fr.S[(int)op.Value] = val; break;
|
||||
case T_LSTR: _cur.Locals.S[(int)op.Value] = val; break;
|
||||
}
|
||||
}
|
||||
|
||||
private long BaseAddr(Operand op) => op.Type switch
|
||||
{
|
||||
T_IMM or T_GINT or T_GFLOAT or T_GSTR or T_GPTR => op.Value,
|
||||
T_LINT => Gi(_fr.I, (int)op.Value),
|
||||
T_LPTR => Gi(_fr.P, (int)op.Value),
|
||||
T_LINT => Gi(_cur.Locals.I, (int)op.Value),
|
||||
T_LPTR => Gi(_cur.Locals.P, (int)op.Value),
|
||||
_ => op.Value,
|
||||
};
|
||||
|
||||
@@ -85,24 +86,39 @@ public sealed class VirtualMachine
|
||||
{
|
||||
switch (dst.Type)
|
||||
{
|
||||
case T_LPTR: _fr.P[(int)dst.Value] = addr; break;
|
||||
case T_LPTR: _cur.Locals.P[(int)dst.Value] = addr; break;
|
||||
case T_GPTR: Globals[(int)dst.Value] = addr; break;
|
||||
default: Write(dst, Gi(Globals, (int)addr)); break;
|
||||
}
|
||||
}
|
||||
|
||||
private enum FrameOutcome { Returned, Halted, RanOff }
|
||||
|
||||
public void Run(int entryOffset = 0)
|
||||
{
|
||||
int pc = _s.IndexByOffset.TryGetValue(entryOffset, out var idx) ? idx : 0;
|
||||
while (pc >= 0 && pc < _s.Instructions.Count)
|
||||
var top = new ExecFrame(_s, _s.IndexByOffset.TryGetValue(entryOffset, out var idx) ? idx : 0);
|
||||
var outcome = RunFrame(top);
|
||||
if (outcome == FrameOutcome.RanOff) HaltReason ??= "pc-out-of-range";
|
||||
else if (outcome == FrameOutcome.Returned) HaltReason ??= "exit";
|
||||
// Halted: HaltReason already set by the halting op.
|
||||
}
|
||||
|
||||
private FrameOutcome RunFrame(ExecFrame frame)
|
||||
{
|
||||
var prev = _cur; _cur = frame; _depth++;
|
||||
var outcome = FrameOutcome.RanOff;
|
||||
int pc = frame.Pc;
|
||||
while (pc >= 0 && pc < frame.Script.Instructions.Count)
|
||||
{
|
||||
if (Steps >= _o.MaxSteps) { HaltReason ??= "STEP-LIMIT"; return; }
|
||||
if (Steps >= _o.MaxSteps) { HaltReason ??= "STEP-LIMIT"; outcome = FrameOutcome.Halted; break; }
|
||||
Steps++;
|
||||
int next = Step(_s.Instructions[pc], pc);
|
||||
if (next == HALT) return;
|
||||
int next = Step(frame.Script.Instructions[pc], pc);
|
||||
if (next == FRAME_RETURN) { outcome = FrameOutcome.Returned; break; }
|
||||
if (next == HALT) { outcome = FrameOutcome.Halted; break; }
|
||||
pc = next;
|
||||
}
|
||||
HaltReason ??= "pc-out-of-range";
|
||||
_cur = prev; _depth--;
|
||||
return outcome;
|
||||
}
|
||||
|
||||
private int Step(Instruction ins, int pc)
|
||||
@@ -139,28 +155,40 @@ public sealed class VirtualMachine
|
||||
case "bit-reset": Write(a[0], Read(a[0]) & ~Read(a[1])); return pc + 1;
|
||||
case "check-bit": Write(a[0], (Read(a[1]) >> (int)(Read(a[2]) & 31)) & 1); return pc + 1;
|
||||
case "copy-to-global": Write(a[0], Read(a[1])); return pc + 1;
|
||||
case "jmp": return _s.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1);
|
||||
case "call": _callstack.Add(pc + 1); return _s.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1);
|
||||
case "jmp": return _cur.Script.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1);
|
||||
case "call": _cur.CallStack.Add(pc + 1); return _cur.Script.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1);
|
||||
case "ret":
|
||||
if (_callstack.Count > 0) { int r = _callstack[^1]; _callstack.RemoveAt(_callstack.Count - 1); return r; }
|
||||
HaltReason = "ret-underflow"; return HALT;
|
||||
if (_cur.CallStack.Count > 0) { int r = _cur.CallStack[^1]; _cur.CallStack.RemoveAt(_cur.CallStack.Count - 1); return r; }
|
||||
return FRAME_RETURN; // empty intra-call stack => return from the script frame
|
||||
case "jcc":
|
||||
{
|
||||
long tgt = Read(a[0]) != 0 ? a[1].Value : a[2].Value;
|
||||
return tgt == NoJump ? pc + 1 : _s.IndexByOffset.GetValueOrDefault((int)tgt, pc + 1);
|
||||
return tgt == NoJump ? pc + 1 : _cur.Script.IndexByOffset.GetValueOrDefault((int)tgt, pc + 1);
|
||||
}
|
||||
case "exit":
|
||||
case "exit-script": HaltReason = "exit"; return HALT;
|
||||
case "call-script": _host.CallScript(a.Count > 0 ? Read(a[0]) : 0); return pc + 1;
|
||||
case "exit-script": return FRAME_RETURN;
|
||||
case "call-script":
|
||||
{
|
||||
long id = a.Count > 0 ? Read(a[0]) : 0;
|
||||
_host.CallScript(id); // notify (diagnostics)
|
||||
if (_provider == null) return pc + 1; // no script source: prior stub behavior
|
||||
if (_depth >= _o.CallDepthCap) { HaltReason ??= "call-depth-exceeded"; return HALT; }
|
||||
var child = _provider.GetById(id);
|
||||
if (child == null) { HaltReason ??= $"callscript-unresolved:0x{id:x}"; return HALT; }
|
||||
var entry = child.IndexByOffset.TryGetValue(0, out var ci) ? ci : 0;
|
||||
var outcome = RunFrame(new ExecFrame(child, entry));
|
||||
if (outcome == FrameOutcome.Halted) return HALT; // propagate whole-VM halt up
|
||||
return pc + 1; // Returned / RanOff: resume caller
|
||||
}
|
||||
case "show-text":
|
||||
foreach (var o in a)
|
||||
{
|
||||
if (o.Type != T_STR) continue;
|
||||
int off = (int)o.Value;
|
||||
_emitSeen.TryGetValue(off, out var c); c++; _emitSeen[off] = c;
|
||||
_cur.EmitSeen.TryGetValue(off, out var c); c++; _cur.EmitSeen[off] = c;
|
||||
if (c > _o.EmitCap) { HaltReason = $"LOOP:line@0x{off:x}×{c}"; return HALT; }
|
||||
string text = _s.GetString(off);
|
||||
Emitted.Add((off, text));
|
||||
string text = _cur.Script.GetString(off);
|
||||
Emitted.Add((off, text, _cur.Script.Name));
|
||||
_host.ShowText(off, text);
|
||||
}
|
||||
return pc + 1;
|
||||
|
||||
@@ -1,2 +1,2 @@
|
||||
namespace Age.Engine.Vm;
|
||||
public sealed record VmOptions(int EmitCap = 2, long MaxSteps = 2_000_000);
|
||||
public sealed record VmOptions(int EmitCap = 2, long MaxSteps = 2_000_000, int CallDepthCap = 64);
|
||||
|
||||
@@ -115,13 +115,18 @@ def parse(path: Path) -> dict:
|
||||
f"have {len(blob)} decompressed bytes)")
|
||||
|
||||
files = []
|
||||
for _ in range(file_count):
|
||||
for i in range(file_count):
|
||||
name_b, arc_id, file_number, offset, size = struct.unpack_from(FILE_ENTRY_FMT, blob, p)
|
||||
p += FILE_ENTRY_LEN
|
||||
name = _cstr(name_b)
|
||||
if name == "@" or not name:
|
||||
continue
|
||||
files.append({
|
||||
# raw_index = the record's 0-based position in the SYS4INI table INCLUDING '@'
|
||||
# placeholders. This is the engine's universal file id: `call-script <id>` and every
|
||||
# script/asset load index by it (engine FUN_0044f390: record = base + id*0x50). It is
|
||||
# NOT the same as this entry's position in `files` (which omits placeholders).
|
||||
"raw_index": i,
|
||||
"name": name,
|
||||
"archive": archives[arc_id] if 0 <= arc_id < arc_count else None,
|
||||
"arc_id": arc_id,
|
||||
@@ -215,6 +220,15 @@ def main() -> int:
|
||||
out.write_text(json.dumps(index, ensure_ascii=False, indent=1), encoding="utf-8")
|
||||
print(f"-> {out.relative_to(paths.REPO)}")
|
||||
|
||||
# call-script id -> name map (raw_index keyed). `call-script <id>` (opcode 0x03) is a direct
|
||||
# raw index into the SYS4INI file table, so this IS the call-graph name registry that
|
||||
# name-resolution.md #1 needed (confirmed via native-RE, see docs/engine-re.md). sys4load reads
|
||||
# it to annotate `call-script 0x1ab =ADDITEM`.
|
||||
cs = {str(f["raw_index"]): f["name"] for f in index["files"]}
|
||||
cs_out = paths.BUILD / "callscript-names.json"
|
||||
cs_out.write_text(json.dumps(cs, ensure_ascii=False, indent=0), encoding="utf-8")
|
||||
print(f"-> {cs_out.relative_to(paths.REPO)} ({len(cs)} ids)")
|
||||
|
||||
if do_check:
|
||||
print("--- validation ---")
|
||||
return 1 if check(index) else 0
|
||||
|
||||
@@ -80,6 +80,21 @@ def _load_global_labels() -> dict:
|
||||
GLOBAL_LABELS = _load_global_labels()
|
||||
|
||||
|
||||
def _load_callscript_names() -> dict:
|
||||
"""id -> script name. `call-script <id>` (op 0x03) is a raw index into the SYS4INI file
|
||||
table; build/callscript-names.json maps every id to its script name (see docs/engine-re.md)."""
|
||||
try:
|
||||
p = Path(__file__).resolve().parent.parent / "build" / "callscript-names.json"
|
||||
data = json.loads(p.read_text(encoding="utf-8"))
|
||||
except Exception:
|
||||
return {}
|
||||
return {int(k): v for k, v in data.items()}
|
||||
|
||||
|
||||
CALLSCRIPT_NAMES = _load_callscript_names()
|
||||
CALLSCRIPT_OP = 0x03
|
||||
|
||||
|
||||
def display_label(op: int) -> str:
|
||||
"""Rendered mnemonic: Kelebek name if it has one, else the inferred name, else u00…."""
|
||||
lbl = OPCODES.get(op, (f"?{op:x}", 0))[0]
|
||||
@@ -368,6 +383,8 @@ def _fmt_operand(op: int, arg_index: int, atype: int, aval: int, strings: dict)
|
||||
tlabel = ARG_TYPES.get(atype)
|
||||
if atype == 0 or tlabel is None: # immediate / unknown-tag: raw value
|
||||
if atype == 0:
|
||||
if op == CALLSCRIPT_OP and aval in CALLSCRIPT_NAMES:
|
||||
return f"{aval:#x} ={CALLSCRIPT_NAMES[aval]}" # call-script target script name
|
||||
return f"{aval:#x}"
|
||||
return f"<t{atype:#x} {aval:#x}>"
|
||||
if tlabel == "float":
|
||||
|
||||
@@ -77,17 +77,36 @@ abi_source = "kelebek+decode-validated"
|
||||
|
||||
[opcode.semantics]
|
||||
name = "call-script"
|
||||
category = "unknown"
|
||||
summary = ""
|
||||
category = "control"
|
||||
summary = "load & call another SYS4 script by id; id = RAW index into the SYS4INI file table (asset-index). Pushes a script frame; returns to caller when the callee ends."
|
||||
noop_headless = false
|
||||
source = "kelebek"
|
||||
confidence = "med"
|
||||
source = "investigation"
|
||||
confidence = "high"
|
||||
depends_on = []
|
||||
evidence = ""
|
||||
evidence = "native-RE (Ghidra): handler FUN_0041bc90 -> loader FUN_0040e980 -> resolver FUN_0044f390 indexes an 80-byte record table (base [ctx+0x414], count [ctx+0x40c]) at base+id*0x50 = the SYS4INI record layout {name[64],arc_id@0x40,file_number@0x44,offset@0x48,size@0x4c}. Confirmed statically: all 297 distinct corpus call-script ids resolve to a .BIN script with a semantically-exact name (0x1ab->ADDITEM, 0x2ae7->MES, 0x143->BUNKI, 0x329d->CALCREVISE), 0 out-of-range, 0 pack-branch. See docs/engine-re.md + name-resolution.md #1."
|
||||
confirm_by = ""
|
||||
details = """
|
||||
op 0x03 (call-script, argc 1): `call-script <id>`. RESOLVED — the id is a direct RAW index into
|
||||
the SYS4INI global file table (the same table parse_sys4ini.py reads, but indexed WITHOUT skipping
|
||||
'@' placeholders; SYS4INI has 13208 records / 2 placeholders). No separate on-disk id->code registry
|
||||
exists; SYS4INI *is* the call-script registry.
|
||||
Native mechanism (dispatch table `handler(op)=ctx[0x26c93+op]`, op 0x03 -> FUN_0041bc90):
|
||||
1. FUN_0041bc90 fetches operand 1 (id), bounds-checks call depth (<=0x26), pushes a frame.
|
||||
2. FUN_0040e980 (loader): opens the resource by id, reads the 0x20-byte SYS4 header, checks magic,
|
||||
allocates per-frame code/local buffers from the header var-counts, reads the bytecode body,
|
||||
pushes a script frame (stride 0x1e = 30 dwords, indexed by ctx[0x14f45]).
|
||||
3. FUN_0044f390 (resolver): record = [ctx+0x414] + id*0x50. Tries a LOOSE OVERRIDE first
|
||||
(CreateFileA on record.name -> mod/patch hook point), else opens archive [record.arc_id*0x100 +
|
||||
ctx+0x410], SetFilePointer to record.offset, size = record.size.
|
||||
(High-byte-tagged ids `id & 0xff000000` select an alternate pack via [ctx+0x3028]; UNUSED by the
|
||||
corpus -- 0/297 ids have a high byte.)
|
||||
Companion op 0x8f `call` is INTRA-script (a local JSR), not cross-script -- see its entry.
|
||||
This also names the whole call graph statically (build/callscript-names.json).
|
||||
"""
|
||||
|
||||
[[opcode.semantics.args]]
|
||||
i = 1
|
||||
role = ""
|
||||
role = "script id = raw SYS4INI file index"
|
||||
observed_types = ["imm", "g-int", "l-ptr"]
|
||||
|
||||
[[opcode]]
|
||||
@@ -1428,17 +1447,18 @@ abi_source = "kelebek+decode-validated"
|
||||
|
||||
[opcode.semantics]
|
||||
name = "call"
|
||||
category = "unknown"
|
||||
summary = ""
|
||||
category = "control"
|
||||
summary = "intra-script subroutine call (local JSR): PC = frame.codebase + operand*4; pushes a return address on the per-frame return stack. NOT cross-script (that is call-script 0x03)."
|
||||
noop_headless = false
|
||||
source = "kelebek"
|
||||
confidence = "med"
|
||||
source = "investigation"
|
||||
confidence = "high"
|
||||
depends_on = []
|
||||
evidence = ""
|
||||
evidence = "native-RE (Ghidra): handler FUN_0041fba0 (= ctx[0x26c93+0x8f]) sets [frame PC @+0x53d2c] = [frame codebase @+0x53d28] + operand*4 and pushes ((pc-base)>>2)+3 onto the per-frame return stack ([ctx+0x552e8]/[ctx+0x55248]). Target is a code OFFSET within the current script (matches header table T3 tag 0x8F = local call targets), confirming it is a local JSR, not a script load."
|
||||
confirm_by = ""
|
||||
|
||||
[[opcode.semantics.args]]
|
||||
i = 1
|
||||
role = ""
|
||||
role = "local code target (word offset within current script)"
|
||||
observed_types = ["imm", "g-int"]
|
||||
|
||||
[[opcode]]
|
||||
|
||||
Reference in New Issue
Block a user