From a50b7f600a13993a87eca8841a6962dc3d7f8413 Mon Sep 17 00:00:00 2001 From: gamer147 Date: Mon, 6 Jul 2026 13:45:05 -0400 Subject: [PATCH] docs(a1): implementation plan for the C# VM core Co-Authored-By: Claude Opus 4.8 (1M context) --- .../plans/2026-07-06-a1-csharp-vm.md | 909 ++++++++++++++++++ 1 file changed, 909 insertions(+) create mode 100644 docs/superpowers/plans/2026-07-06-a1-csharp-vm.md diff --git a/docs/superpowers/plans/2026-07-06-a1-csharp-vm.md b/docs/superpowers/plans/2026-07-06-a1-csharp-vm.md new file mode 100644 index 0000000..7cd5cc3 --- /dev/null +++ b/docs/superpowers/plans/2026-07-06-a1-csharp-vm.md @@ -0,0 +1,909 @@ +# A1 — C# VM Core Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** A headless C# (.NET 8) reimplementation of the validated `vm0.py` AGE/SYS4 model that is per-scene trace-identical to the Python prototype and passes the RECOVER unit test. + +**Architecture:** A `.NET 8` class library `Age.Engine` with four seam-namespaces — `Model` (version-neutral `Script` contract), `Vm` (execution core, depends only on `Model` + `Hosting`), `Sys4` (`.BIN` parser + cp932 codec + opcode-table loader), `Hosting` (`IHost` + headless `CaptureHost`) — plus a console `Age.Cli` and an `Age.Engine.Tests` xUnit project. Validated by a per-scene trace diff against `tools/vm0.py --trace`. + +**Tech Stack:** C# / .NET 8, xUnit, `System.Text.Json`, `System.Text.Encoding.CodePages` (cp932). Python 3.11 for the `vm0.py --trace` reference. + +## Global Constraints + +- **Work on a feature branch** (e.g. `feat/a1-csharp-vm`); do not commit to `main`. +- Build: `dotnet build engine/AgeEngine.sln`. Test: `dotnet test engine/AgeEngine.sln` (or `--filter FullyQualifiedName~`). +- **Seam rule:** `Age.Engine.Vm` may reference only `Age.Engine.Model` and `Age.Engine.Hosting` — never `Age.Engine.Sys4`, never file I/O. +- All VM numeric values are `long`. `div`/`mod` use the Python-parity helpers `PyDiv`/`PyMod` (floor-div; divisor-sign mod). u32 operands load as **unsigned** into `long` (no sign-extension), matching vm0. +- Halt-reason strings must be **byte-identical** to vm0: `"exit"`, `"LOOP:line@0x{off:x}×{count}"` (× = U+00D7), `"STEP-LIMIT"`, `"ret-underflow"`, `"pc-out-of-range"`. Source files are UTF-8. +- Data prerequisites (resolved via `Paths`, repo-relative, like `tools/paths.py`): `build/opcodes.json` (run `tools/opcodes_build.py --build`), `../extracted/DATA1/*.BIN`, and — for the trace-diff — `build/vm0-trace.json` (produced by Task 5). +- Exact format facts (from `tools/sys4load.py`, `tools/vm0.py`): magic prefix `SYS4`, header 0x3C bytes = 8 magic + 13×u32 (F0–F12), body dwords from 0x3C; code section `[0..F8)`; instruction length `1 + 2*argc` dwords; `data_array_end` shrink stops code at the lowest referenced string(type-2)/array(op `0x64` arg index 1) offset; operand types `0=imm 2=string 3=g-int 4=g-float 5=g-string 6=g-ptr 9=l-int 10=l-float 11=l-string 12=l-ptr`; `jcc` sentinel `0xFFFFFFFF` = fallthrough; `ARRAY_OPCODE=0x64`; `EMIT_CAP=2`, `MAX_STEPS=2_000_000`. + +--- + +## File Structure + +``` +engine/AgeEngine.sln +engine/Age.Engine/Age.Engine.csproj +engine/Age.Engine/Model/ Operand.cs Instruction.cs ScriptHeader.cs Script.cs OpcodeTable.cs +engine/Age.Engine/Sys4/ Paths.cs OpcodeTableJson.cs Sys4StringCodec.cs Sys4Loader.cs +engine/Age.Engine/Vm/ VmOptions.cs Frame.cs VirtualMachine.cs +engine/Age.Engine/Hosting/ IHost.cs CaptureHost.cs +engine/Age.Cli/Age.Cli.csproj Program.cs +engine/Age.Engine.Tests/Age.Engine.Tests.csproj + OpcodeTableTests.cs Sys4StringCodecTests.cs Sys4LoaderTests.cs RecoverTests.cs TraceDiffTests.cs +tools/vm0.py (modify: add --trace) +.gitignore (modify: ignore engine build output) +``` + +--- + +## Task 1: Solution scaffold + Model + OpcodeTable + +**Files:** Create the solution, `Age.Engine/Model/*`, `Age.Engine/Sys4/Paths.cs`, `Age.Engine/Sys4/OpcodeTableJson.cs`; Test `Age.Engine.Tests/OpcodeTableTests.cs`; Modify `.gitignore`. + +**Interfaces — Produces:** +- `Age.Engine.Model.Operand(int Type, long Value)` (readonly record struct) +- `Age.Engine.Model.Instruction(int Offset, int Opcode, IReadOnlyList Args)` +- `Age.Engine.Model.ScriptHeader(int LocalInt1, int LocalFloats, int LocalStrings1, int LocalInt2, int Unknown, int LocalStrings2)` +- `Age.Engine.Model.Script { ScriptHeader Header; IReadOnlyList Instructions; IReadOnlyDictionary IndexByOffset; IReadOnlyDictionary Strings; string GetString(int) }` +- `Age.Engine.Model.OpcodeTable { bool TryGet(int,out string,out int); string Label(int); int Argc(int); int Count }` +- `Age.Engine.Sys4.OpcodeTableJson.Load(string path) -> OpcodeTable` +- `Age.Engine.Sys4.Paths { string Repo, Extracted, Data1, GameDir, Build, OpcodesJson; Dictionary Scripts() }` + +- [ ] **Step 1: Scaffold the solution** + +```bash +cd "S:/Game Hacking/Eushully/Himegari/age-reimpl" +dotnet new sln -n AgeEngine -o engine +dotnet new classlib -n Age.Engine -o engine/Age.Engine -f net8.0 +dotnet new console -n Age.Cli -o engine/Age.Cli -f net8.0 +dotnet new xunit -n Age.Engine.Tests -o engine/Age.Engine.Tests -f net8.0 +rm -f engine/Age.Engine/Class1.cs engine/Age.Engine.Tests/UnitTest1.cs +dotnet sln engine/AgeEngine.sln add engine/Age.Engine engine/Age.Cli engine/Age.Engine.Tests +dotnet add engine/Age.Cli reference engine/Age.Engine +dotnet add engine/Age.Engine.Tests reference engine/Age.Engine +dotnet add engine/Age.Engine package System.Text.Encoding.CodePages +``` + +- [ ] **Step 2: Ignore .NET build output** — append to `.gitignore`: + +``` +# .NET (engine/) build output +engine/**/bin/ +engine/**/obj/ +.vs/ +``` + +- [ ] **Step 3: Write the failing test** — `engine/Age.Engine.Tests/OpcodeTableTests.cs`: + +```csharp +using Age.Engine.Sys4; +using Xunit; + +public class OpcodeTableTests +{ + [Fact] + public void LoadsAll248FromJson() + { + var t = OpcodeTableJson.Load(Paths.OpcodesJson); + Assert.Equal(248, t.Count); + Assert.True(t.TryGet(0x55, out var label, out var argc)); + Assert.Equal("mov", label); + Assert.Equal(2, argc); + Assert.Equal("u0041BEB0", t.Label(0x90)); + Assert.Equal(7, t.Argc(0x90)); + Assert.Equal(-1, t.Argc(0x9999)); // absent -> -1 + } +} +``` + +- [ ] **Step 4: Run — expect FAIL (types missing / won't compile)** + +```bash +dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~OpcodeTableTests +``` +Expected: build error — `Paths`/`OpcodeTableJson` do not exist. + +- [ ] **Step 5: Implement Model + Paths + OpcodeTableJson** + +`engine/Age.Engine/Model/Operand.cs`: +```csharp +namespace Age.Engine.Model; +public readonly record struct Operand(int Type, long Value); +``` +`engine/Age.Engine/Model/Instruction.cs`: +```csharp +namespace Age.Engine.Model; +public sealed record Instruction(int Offset, int Opcode, IReadOnlyList Args); +``` +`engine/Age.Engine/Model/ScriptHeader.cs`: +```csharp +namespace Age.Engine.Model; +public sealed record ScriptHeader( + int LocalInt1, int LocalFloats, int LocalStrings1, + int LocalInt2, int Unknown, int LocalStrings2); +``` +`engine/Age.Engine/Model/Script.cs`: +```csharp +namespace Age.Engine.Model; +public sealed class Script +{ + public required ScriptHeader Header { get; init; } + public required IReadOnlyList Instructions { get; init; } + public required IReadOnlyDictionary IndexByOffset { get; init; } + public required IReadOnlyDictionary Strings { get; init; } + public string GetString(int offset) => Strings.TryGetValue(offset, out var s) ? s : ""; +} +``` +`engine/Age.Engine/Model/OpcodeTable.cs`: +```csharp +namespace Age.Engine.Model; +public sealed class OpcodeTable +{ + private readonly IReadOnlyDictionary _t; + public OpcodeTable(IReadOnlyDictionary t) => _t = t; + public int Count => _t.Count; + public bool TryGet(int op, out string label, out int argc) + { + if (_t.TryGetValue(op, out var e)) { label = e.Label; argc = e.Argc; return true; } + label = ""; argc = -1; return false; + } + public string Label(int op) => _t.TryGetValue(op, out var e) ? e.Label : ""; + public int Argc(int op) => _t.TryGetValue(op, out var e) ? e.Argc : -1; +} +``` +`engine/Age.Engine/Sys4/Paths.cs`: +```csharp +namespace Age.Engine.Sys4; +public static class Paths +{ + public static string Repo { get; } = FindRepo(); + public static string Workspace => Directory.GetParent(Repo)!.FullName; + public static string Extracted => Path.Combine(Workspace, "extracted"); + public static string Data1 => Path.Combine(Extracted, "DATA1"); + public static string GameDir => Path.Combine(Workspace, "姫狩りダンジョンマイスター"); + public static string Build => Path.Combine(Repo, "build"); + public static string OpcodesJson => Path.Combine(Build, "opcodes.json"); + + private static string FindRepo() + { + var d = new DirectoryInfo(AppContext.BaseDirectory); + while (d != null && d.Name != "age-reimpl") d = d.Parent; + if (d == null) throw new DirectoryNotFoundException( + "age-reimpl root not found above " + AppContext.BaseDirectory); + return d.FullName; + } + + /// name(UPPER).BIN -> path; game-dir loose overrides shadow extracted/DATA1 (mirrors paths.scripts()). + public static Dictionary Scripts() + { + var d = new Dictionary(StringComparer.OrdinalIgnoreCase); + if (Directory.Exists(Data1)) + foreach (var p in Directory.EnumerateFiles(Data1, "*.BIN")) + d[Path.GetFileName(p).ToUpperInvariant()] = p; + if (Directory.Exists(GameDir)) + foreach (var p in Directory.EnumerateFiles(GameDir, "*.BIN")) + d[Path.GetFileName(p).ToUpperInvariant()] = p; + return d; + } +} +``` +`engine/Age.Engine/Sys4/OpcodeTableJson.cs`: +```csharp +using System.Text.Json; +using Age.Engine.Model; +namespace Age.Engine.Sys4; +public static class OpcodeTableJson +{ + public static OpcodeTable Load(string path) + { + using var doc = JsonDocument.Parse(File.ReadAllText(path)); + var dict = new Dictionary(); + foreach (var e in doc.RootElement.GetProperty("opcodes").EnumerateArray()) + { + int op = Convert.ToInt32(e.GetProperty("op").GetString(), 16); + string label = e.GetProperty("label").GetString() ?? ""; + int argc = e.GetProperty("argc").GetInt32(); + dict[op] = (label, argc); + } + return new OpcodeTable(dict); + } +} +``` + +- [ ] **Step 6: Run — expect PASS** + +```bash +dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~OpcodeTableTests +``` +Expected: 1 passed. (Prereq: `tools/opcodes_build.py --build` has produced `build/opcodes.json`.) + +- [ ] **Step 7: Commit** +```bash +git add engine .gitignore +git commit -m "feat(a1): .NET 8 solution scaffold + Model + OpcodeTable loader" +``` + +--- + +## Task 2: SYS4 string codec + +**Files:** Create `Age.Engine/Sys4/Sys4StringCodec.cs`; Test `Age.Engine.Tests/Sys4StringCodecTests.cs`. + +**Interfaces — Produces:** `Sys4StringCodec.Decode(IReadOnlyList dwords, int start, int limit = 4096) -> (string? Text, int NDwords)` — XOR-`0xFFFFFFFF` per little-endian dword, cut at first NUL, strict cp932 validation (returns `(null,0)` when not a clean string), else decoded text + dword length including the NUL dword. + +- [ ] **Step 1: Write the failing test** — `engine/Age.Engine.Tests/Sys4StringCodecTests.cs`: + +```csharp +using Age.Engine.Sys4; +using Xunit; + +public class Sys4StringCodecTests +{ + [Fact] + public void DecodesXorFfCp932Ascii() + { + // "AB\0\0" little-endian = 0x00004241, stored XOR 0xFFFFFFFF + uint[] dw = { 0x00004241u ^ 0xFFFFFFFFu }; + var (text, nd) = Sys4StringCodec.Decode(dw, 0); + Assert.Equal("AB", text); + Assert.Equal(1, nd); + } + + [Fact] + public void RejectsNonString() + { + // 0x00000000 XOR-decodes to 0xFFFFFFFF bytes (0xFF,0xFF,0xFF,0xFF) -> no NUL, invalid + uint[] dw = { 0x00000000u }; + var (text, _) = Sys4StringCodec.Decode(dw, 0); + Assert.Null(text); + } +} +``` + +- [ ] **Step 2: Run — expect FAIL** (`Sys4StringCodec` missing) +```bash +dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~Sys4StringCodecTests +``` + +- [ ] **Step 3: Implement** — `engine/Age.Engine/Sys4/Sys4StringCodec.cs`: + +```csharp +using System.Text; +namespace Age.Engine.Sys4; +public static class Sys4StringCodec +{ + static Sys4StringCodec() => Encoding.RegisterProvider(CodePagesEncodingProvider.Instance); + private static readonly Encoding Cp932 = Encoding.GetEncoding(932); + + public static (string? Text, int NDwords) Decode(IReadOnlyList dwords, int start, int limit = 4096) + { + int n = dwords.Count, end = Math.Min(start + limit, n); + var raw = new List(); + bool sawNul = false; + for (int j = start; j < end; j++) + { + uint x = dwords[j] ^ 0xFFFFFFFFu; + raw.Add((byte)(x & 0xFF)); raw.Add((byte)((x >> 8) & 0xFF)); + raw.Add((byte)((x >> 16) & 0xFF)); raw.Add((byte)((x >> 24) & 0xFF)); + if (raw[^1] == 0 || raw[^2] == 0 || raw[^3] == 0 || raw[^4] == 0) { sawNul = true; break; } + } + if (!sawNul) return (null, 0); + int nul = raw.IndexOf(0); + byte[] s = raw.GetRange(0, nul < 0 ? raw.Count : nul).ToArray(); + if (s.Length < 1) return ("", 1); + int i = 0, chars = 0; + while (i < s.Length) + { + byte b = s[i]; + if (b >= 0x20 && b <= 0x7E) { i++; chars++; } + else if ((b >= 0x81 && b <= 0x9F) || (b >= 0xE0 && b <= 0xEA)) + { + if (i + 1 < s.Length && s[i + 1] >= 0x40 && s[i + 1] <= 0xFC && s[i + 1] != 0x7F) { i += 2; chars++; } + else return (null, 0); + } + else return (null, 0); + } + if (chars < 1) return (null, 0); + string text; + try { text = Cp932.GetString(s); } catch { return (null, 0); } + return (text, (s.Length / 4) + 1); + } +} +``` + +- [ ] **Step 4: Run — expect PASS** +```bash +dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~Sys4StringCodecTests +``` + +- [ ] **Step 5: Commit** +```bash +git add engine/Age.Engine/Sys4/Sys4StringCodec.cs engine/Age.Engine.Tests/Sys4StringCodecTests.cs +git commit -m "feat(a1): SYS4 XOR-FF cp932 string codec" +``` + +--- + +## Task 3: SYS4 container parser (`Sys4Loader`) + +**Files:** Create `Age.Engine/Sys4/Sys4Loader.cs`; Test `Age.Engine.Tests/Sys4LoaderTests.cs`. + +**Interfaces — Consumes:** `OpcodeTable`, `Sys4StringCodec`, `Script`. **Produces:** `Sys4Loader.Load(string path, OpcodeTable table) -> Script`; `Sys4Loader.Parse(byte[] data, OpcodeTable table, string name = "") -> Script`. + +- [ ] **Step 1: Write the failing test** — `engine/Age.Engine.Tests/Sys4LoaderTests.cs`: + +```csharp +using Age.Engine.Sys4; +using Xunit; + +public class Sys4LoaderTests +{ + [Fact] + public void ParsesMenuBinLikeSys4load() + { + var table = OpcodeTableJson.Load(Paths.OpcodesJson); + var s = Sys4Loader.Load(Path.Combine(Paths.Data1, "MENU.BIN"), table); + // header F0..F5 from `sys4load MENU.BIN --summary` + Assert.Equal(0x17, s.Header.LocalInt1); + Assert.Equal(1, s.Header.LocalFloats); + Assert.Equal(1, s.Header.LocalStrings1); + Assert.Equal(2, s.Header.LocalInt2); + Assert.Equal(1, s.Header.Unknown); + Assert.Equal(1, s.Header.LocalStrings2); + Assert.Equal(148, s.Instructions.Count); // sys4load: 148 instructions + Assert.Equal(3, s.Strings.Count); // sys4load: 3 inline strings + // every instruction offset is indexed + Assert.All(s.Instructions, ins => Assert.True(s.IndexByOffset.ContainsKey(ins.Offset))); + } +} +``` + +- [ ] **Step 2: Run — expect FAIL** (`Sys4Loader` missing) +```bash +dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~Sys4LoaderTests +``` + +- [ ] **Step 3: Implement** — `engine/Age.Engine/Sys4/Sys4Loader.cs`: + +```csharp +using Age.Engine.Model; +namespace Age.Engine.Sys4; +public static class Sys4Loader +{ + private const int HeaderSize = 0x3C, BodyOff = 0x3C, NumFields = 13, ArrayOpcode = 0x64; + + public static Script Load(string path, OpcodeTable table) + => Parse(File.ReadAllBytes(path), table, Path.GetFileName(path)); + + public static Script Parse(byte[] data, OpcodeTable table, string name = "") + { + if (data.Length < HeaderSize) throw new InvalidDataException($"{name}: too small"); + if (!(data[0] == (byte)'S' && data[1] == (byte)'Y' && data[2] == (byte)'S' && data[3] == (byte)'4')) + throw new InvalidDataException($"{name}: bad magic"); + if (data.Length % 4 != 0) throw new InvalidDataException($"{name}: not dword-aligned"); + + var fields = new int[NumFields]; + for (int k = 0; k < NumFields; k++) fields[k] = BitConverter.ToInt32(data, 8 + k * 4); + int nbody = (data.Length - BodyOff) / 4; + var dw = new uint[nbody]; + for (int k = 0; k < nbody; k++) dw[k] = BitConverter.ToUInt32(data, BodyOff + k * 4); + + var header = new ScriptHeader(fields[0], fields[1], fields[2], fields[3], fields[4], fields[5]); + var (instrs, idxByOff, strings) = DecodeCode(dw, fields, nbody, table); + return new Script { Header = header, Instructions = instrs, IndexByOffset = idxByOff, Strings = strings }; + } + + private static (List, Dictionary, Dictionary) + DecodeCode(uint[] dw, int[] fields, int nbody, OpcodeTable table) + { + int codeEnd = fields[8]; // F8; shrinks to first inline string/array offset + var instrs = new List(); + var idx = new Dictionary(); + var strings = new Dictionary(); + int i = 0; + while (i < codeEnd) + { + int op = (int)dw[i]; + int argc = table.Argc(op); + if (argc < 0) { idx[i] = instrs.Count; instrs.Add(new Instruction(i, op, Array.Empty())); break; } + int baseI = i + 1; + if (baseI + 2 * argc > codeEnd) { idx[i] = instrs.Count; instrs.Add(new Instruction(i, op, Array.Empty())); break; } + var args = new Operand[argc]; + for (int a = 0; a < argc; a++) + { + int atype = (int)dw[baseI + 2 * a]; + long aval = dw[baseI + 2 * a + 1]; + args[a] = new Operand(atype, aval); + if (atype == 2 && aval >= 0 && aval < nbody) + { + if (aval < codeEnd) codeEnd = (int)aval; + if (!strings.ContainsKey((int)aval)) + { + var (text, _) = Sys4StringCodec.Decode(dw, (int)aval); + if (text != null) strings[(int)aval] = text; + } + } + else if (op == ArrayOpcode && a == 1 && aval >= 0 && aval < nbody) + { + if (aval < codeEnd) codeEnd = (int)aval; + } + } + idx[i] = instrs.Count; + instrs.Add(new Instruction(i, op, args)); + i = baseI + 2 * argc; + } + return (instrs, idx, strings); + } +} +``` + +- [ ] **Step 4: Run — expect PASS** +```bash +dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~Sys4LoaderTests +``` +Expected: 1 passed (148 instructions, 3 strings, header matches). If instruction count differs, the `data_array_end` shrink or argc walk diverged — compare against `py -3.11 -X utf8 tools/sys4load.py ../extracted/DATA1/MENU.BIN --summary`. + +- [ ] **Step 5: Commit** +```bash +git add engine/Age.Engine/Sys4/Sys4Loader.cs engine/Age.Engine.Tests/Sys4LoaderTests.cs +git commit -m "feat(a1): SYS4 container parser (header + code decode + data_array_end shrink)" +``` + +--- + +## Task 4: VM core + IHost/CaptureHost + RECOVER + +**Files:** Create `Age.Engine/Hosting/IHost.cs`, `Age.Engine/Hosting/CaptureHost.cs`, `Age.Engine/Vm/VmOptions.cs`, `Age.Engine/Vm/Frame.cs`, `Age.Engine/Vm/VirtualMachine.cs`; Test `Age.Engine.Tests/RecoverTests.cs`. + +**Interfaces — Consumes:** `Script`, `OpcodeTable`, `Sys4Loader`. **Produces:** +- `Age.Engine.Hosting.IHost { void ShowText(int,string); void CallScript(long); void OnStub(int); }`, `CaptureHost : IHost` +- `Age.Engine.Vm.VmOptions(int EmitCap = 2, long MaxSteps = 2_000_000)` +- `Age.Engine.Vm.VirtualMachine(Script, OpcodeTable, IHost, VmOptions? = null)` with public `Dictionary Globals`, `Dictionary GlobalStrings`, `List<(int Offset,string Text)> Emitted`, `string? HaltReason`, `long Steps`, and `void Run(int entryOffset = 0)`. + +- [ ] **Step 1: Write the failing test** — `engine/Age.Engine.Tests/RecoverTests.cs`: + +```csharp +using Age.Engine.Hosting; +using Age.Engine.Sys4; +using Age.Engine.Vm; +using Xunit; + +public class RecoverTests +{ + private static long G(VirtualMachine vm, int k) => vm.Globals.TryGetValue(k, out var v) ? v : 0; + + [Fact] + public void RecoverUnitTestPasses() + { + var table = OpcodeTableJson.Load(Paths.OpcodesJson); + var script = Sys4Loader.Load(Path.Combine(Paths.Data1, "RECOVER.BIN"), table); + var vm = new VirtualMachine(script, table, new CaptureHost()); + + int unit = 0; + vm.Globals[0x152616] = unit; + int A = 0x4e11b, B = 0x4e085, C = 0x52383, E = 0x52f3b, F = 0x5295f, FL = 0xaacb4; + for (int k = 0; k < 3; k++) vm.Globals[A + unit * 14 + (11 + k)] = 100 + k; + vm.Globals[C + unit * 30 + 5] = 7; vm.Globals[FL + 5] = 1; vm.Globals[E + unit * 30 + 5] = 42; + vm.Globals[C + unit * 30 + 6] = 0; vm.Globals[FL + 6] = 1; vm.Globals[E + unit * 30 + 6] = 99; + vm.Globals[C + unit * 30 + 7] = 3; vm.Globals[FL + 7] = 0; vm.Globals[E + unit * 30 + 7] = 88; + vm.Run(); + + Assert.Equal(100, G(vm, B + unit * 3 + 0)); + Assert.Equal(101, G(vm, B + unit * 3 + 1)); + Assert.Equal(102, G(vm, B + unit * 3 + 2)); + Assert.Equal(42, G(vm, C + unit * 30 + 5)); + Assert.Equal(-1, G(vm, F + unit * 30 + 5)); + Assert.Equal(0, G(vm, C + unit * 30 + 6)); + Assert.Equal(3, G(vm, C + unit * 30 + 7)); + } +} +``` + +- [ ] **Step 2: Run — expect FAIL** (VM types missing) +```bash +dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~RecoverTests +``` + +- [ ] **Step 3: Implement hosting + VM** + +`engine/Age.Engine/Hosting/IHost.cs`: +```csharp +namespace Age.Engine.Hosting; +public interface IHost +{ + void ShowText(int offset, string text); + void CallScript(long id); + void OnStub(int opcode); +} +``` +`engine/Age.Engine/Hosting/CaptureHost.cs`: +```csharp +namespace Age.Engine.Hosting; +public sealed class CaptureHost : IHost +{ + public List<(int Offset, string Text)> Emitted { get; } = new(); + public int CallScriptCount { get; private set; } + public Dictionary Stubs { get; } = new(); + public void ShowText(int offset, string text) => Emitted.Add((offset, text)); + public void CallScript(long id) => CallScriptCount++; + public void OnStub(int opcode) { Stubs.TryGetValue(opcode, out var c); Stubs[opcode] = c + 1; } +} +``` +`engine/Age.Engine/Vm/VmOptions.cs`: +```csharp +namespace Age.Engine.Vm; +public sealed record VmOptions(int EmitCap = 2, long MaxSteps = 2_000_000); +``` +`engine/Age.Engine/Vm/Frame.cs`: +```csharp +namespace Age.Engine.Vm; +public sealed class Frame +{ + public Dictionary I = new(); // local-int + public Dictionary F = new(); // local-float (raw) + public Dictionary S = new(); // local-string + public Dictionary P = new(); // local-ptr (holds a global address) +} +``` +`engine/Age.Engine/Vm/VirtualMachine.cs`: +```csharp +using Age.Engine.Hosting; +using Age.Engine.Model; +namespace Age.Engine.Vm; + +public sealed class VirtualMachine +{ + private const long NoJump = 0xFFFFFFFF; + private const int HALT = int.MinValue; + private const int T_IMM = 0, T_STR = 2, T_GINT = 3, T_GFLOAT = 4, T_GSTR = 5, T_GPTR = 6, + T_LINT = 9, T_LFLOAT = 10, T_LSTR = 11, T_LPTR = 12; + + private readonly Script _s; + private readonly OpcodeTable _t; + private readonly IHost _host; + private readonly VmOptions _o; + private readonly Frame _fr = new(); + private readonly List _callstack = new(); + private readonly Dictionary _emitSeen = new(); + + public Dictionary Globals { get; } = new(); + public Dictionary GlobalStrings { get; } = new(); + public List<(int Offset, string Text)> Emitted { get; } = new(); + public string? HaltReason { get; private set; } + public long Steps { get; private set; } + + public VirtualMachine(Script s, OpcodeTable t, IHost host, VmOptions? o = null) + { _s = s; _t = t; _host = host; _o = o ?? new VmOptions(); } + + private static long Gi(Dictionary d, int k) => d.TryGetValue(k, out var v) ? v : 0; + private static string Gs(Dictionary d, int k) => d.TryGetValue(k, out var v) ? v : ""; + private static long PyDiv(long a, long b) { if (b == 0) return 0; long q = a / b, r = a % b; if (r != 0 && (r < 0) != (b < 0)) q--; return q; } + private static long PyMod(long a, long b) { if (b == 0) return 0; long r = a % b; if (r != 0 && (r < 0) != (b < 0)) r += b; return r; } + + private static bool IsStr(Operand o) => o.Type == T_STR || o.Type == T_GSTR || o.Type == T_LSTR; + + private long Read(Operand op) => op.Type switch + { + T_IMM => op.Value, + T_GINT or T_GFLOAT => Gi(Globals, (int)op.Value), + T_GPTR => Gi(Globals, (int)Gi(Globals, (int)op.Value)), + T_LINT => Gi(_fr.I, (int)op.Value), + T_LFLOAT => Gi(_fr.F, (int)op.Value), + T_LPTR => Gi(Globals, (int)Gi(_fr.P, (int)op.Value)), + _ => op.Value, + }; + + private void Write(Operand op, long val) + { + switch (op.Type) + { + case T_GINT: case T_GFLOAT: Globals[(int)op.Value] = val; break; + case T_GPTR: Globals[(int)Gi(Globals, (int)op.Value)] = val; break; + case T_LINT: _fr.I[(int)op.Value] = val; break; + case T_LFLOAT: _fr.F[(int)op.Value] = val; break; + case T_LPTR: Globals[(int)Gi(_fr.P, (int)op.Value)] = val; break; + } + } + + private string ReadStr(Operand op) => op.Type switch + { + T_STR => _s.GetString((int)op.Value), + T_GSTR => Gs(GlobalStrings, (int)op.Value), + T_LSTR => Gs(_fr.S, (int)op.Value), + _ => "", + }; + + private void WriteStr(Operand op, string val) + { + switch (op.Type) + { + case T_GSTR: GlobalStrings[(int)op.Value] = val; break; + case T_LSTR: _fr.S[(int)op.Value] = val; break; + } + } + + private long BaseAddr(Operand op) => op.Type switch + { + T_IMM or T_GINT or T_GFLOAT or T_GSTR or T_GPTR => op.Value, + T_LINT => Gi(_fr.I, (int)op.Value), + T_LPTR => Gi(_fr.P, (int)op.Value), + _ => op.Value, + }; + + private void LookupStore(Operand dst, long addr) + { + switch (dst.Type) + { + case T_LPTR: _fr.P[(int)dst.Value] = addr; break; + case T_GPTR: Globals[(int)dst.Value] = addr; break; + default: Write(dst, Gi(Globals, (int)addr)); break; + } + } + + public void Run(int entryOffset = 0) + { + int pc = _s.IndexByOffset.TryGetValue(entryOffset, out var idx) ? idx : 0; + while (pc >= 0 && pc < _s.Instructions.Count) + { + if (Steps >= _o.MaxSteps) { HaltReason ??= "STEP-LIMIT"; return; } + Steps++; + int next = Step(_s.Instructions[pc], pc); + if (next == HALT) return; + pc = next; + } + HaltReason ??= "pc-out-of-range"; + } + + private int Step(Instruction ins, int pc) + { + int op = ins.Opcode; + var a = ins.Args; + switch (_t.Label(op)) + { + case "add": Write(a[0], Read(a[1]) + Read(a[2])); return pc + 1; + case "sub": Write(a[0], Read(a[1]) - Read(a[2])); return pc + 1; + case "mul": Write(a[0], Read(a[1]) * Read(a[2])); return pc + 1; + case "div": Write(a[0], PyDiv(Read(a[1]), Read(a[2]))); return pc + 1; + case "mod": Write(a[0], PyMod(Read(a[1]), Read(a[2]))); return pc + 1; + case "and": Write(a[0], Read(a[1]) & Read(a[2])); return pc + 1; + case "or": Write(a[0], Read(a[1]) | Read(a[2])); return pc + 1; + case "sar": Write(a[0], Read(a[1]) >> (int)(Read(a[2]) & 31)); return pc + 1; + case "shl": Write(a[0], Read(a[1]) << (int)(Read(a[2]) & 31)); return pc + 1; + case "eq": Write(a[0], Read(a[1]) == Read(a[2]) ? 1 : 0); return pc + 1; + case "ne": Write(a[0], Read(a[1]) != Read(a[2]) ? 1 : 0); return pc + 1; + case "lt": Write(a[0], Read(a[1]) < Read(a[2]) ? 1 : 0); return pc + 1; + case "lte": Write(a[0], Read(a[1]) <= Read(a[2]) ? 1 : 0); return pc + 1; + case "gr": Write(a[0], Read(a[1]) > Read(a[2]) ? 1 : 0); return pc + 1; + case "gre": Write(a[0], Read(a[1]) >= Read(a[2]) ? 1 : 0); return pc + 1; + case "mov": + case "set-string": + if (IsStr(a[0]) || IsStr(a[1])) WriteStr(a[0], ReadStr(a[1])); + else Write(a[0], Read(a[1])); + return pc + 1; + case "lookup-array": + LookupStore(a[0], BaseAddr(a[1]) + Read(a[2])); return pc + 1; + case "lookup-array-2d": + LookupStore(a[0], BaseAddr(a[1]) + Read(a[2]) * Read(a[3]) + Read(a[4])); return pc + 1; + case "bit-set": Write(a[0], Read(a[0]) | Read(a[1])); return pc + 1; + case "bit-reset": Write(a[0], Read(a[0]) & ~Read(a[1])); return pc + 1; + case "check-bit": Write(a[0], (Read(a[1]) >> (int)(Read(a[2]) & 31)) & 1); return pc + 1; + case "copy-to-global": Write(a[0], Read(a[1])); return pc + 1; + case "jmp": return _s.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1); + case "call": _callstack.Add(pc + 1); return _s.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1); + case "ret": + if (_callstack.Count > 0) { int r = _callstack[^1]; _callstack.RemoveAt(_callstack.Count - 1); return r; } + HaltReason = "ret-underflow"; return HALT; + case "jcc": + { + long tgt = Read(a[0]) != 0 ? a[1].Value : a[2].Value; + return tgt == NoJump ? pc + 1 : _s.IndexByOffset.GetValueOrDefault((int)tgt, pc + 1); + } + case "exit": + case "exit-script": HaltReason = "exit"; return HALT; + case "call-script": _host.CallScript(a.Count > 0 ? Read(a[0]) : 0); return pc + 1; + case "show-text": + foreach (var o in a) + { + if (o.Type != T_STR) continue; + int off = (int)o.Value; + _emitSeen.TryGetValue(off, out var c); c++; _emitSeen[off] = c; + if (c > _o.EmitCap) { HaltReason = $"LOOP:line@0x{off:x}×{c}"; return HALT; } + string text = _s.GetString(off); + Emitted.Add((off, text)); + _host.ShowText(off, text); + } + return pc + 1; + case "end-text-line": case "wait-for-input": case "set-font": + case "comment": case "display-furigana": case "dev_ukn": + return pc + 1; + default: + _host.OnStub(op); return pc + 1; + } + } +} +``` + +- [ ] **Step 4: Run — expect PASS** +```bash +dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~RecoverTests +``` +Expected: RECOVER passes (all 7 asserts). If a pointer/array assert fails, compare against `py -3.11 -X utf8 tools/vm0.py --test`. + +- [ ] **Step 5: Commit** +```bash +git add engine/Age.Engine/Hosting engine/Age.Engine/Vm engine/Age.Engine.Tests/RecoverTests.cs +git commit -m "feat(a1): VM core (operands+pointer semantics, handlers, emit-cap) + RECOVER" +``` + +--- + +## Task 5: `Age.Cli` + `vm0.py --trace` + per-scene trace diff + +**Files:** Modify `tools/vm0.py` (add `--trace`); Create `engine/Age.Cli/Program.cs`; Test `Age.Engine.Tests/TraceDiffTests.cs`. + +**Interfaces — Consumes:** `Paths.Scripts()`, `Sys4Loader`, `OpcodeTableJson`, `VirtualMachine`, `CaptureHost`. **Produces (shared trace contract):** JSON object `{ ".BIN": { "offsets": [int...], "halt": "", "steps": int }, ... }` over the sorted set of names matching `^S[CP]\d{4}\.BIN$` from `Paths.Scripts()`. Both `vm0.py --trace ` and `Age.Cli trace ` emit exactly this. + +- [ ] **Step 1: Write the failing test** — `engine/Age.Engine.Tests/TraceDiffTests.cs`: + +```csharp +using System.Text.Json; +using System.Text.RegularExpressions; +using Age.Engine.Hosting; +using Age.Engine.Sys4; +using Age.Engine.Vm; +using Xunit; + +public class TraceDiffTests +{ + private static readonly Regex Scene = new(@"^S[CP]\d{4}\.BIN$"); + + [Fact] + public void CsTraceMatchesVm0PerScene() + { + string refPath = Path.Combine(Paths.Build, "vm0-trace.json"); + Assert.True(File.Exists(refPath), + "prerequisite: run `py -3.11 -X utf8 tools/vm0.py --trace build/vm0-trace.json`"); + + using var doc = JsonDocument.Parse(File.ReadAllText(refPath)); + var expected = doc.RootElement; + var table = OpcodeTableJson.Load(Paths.OpcodesJson); + var scripts = Paths.Scripts(); + + var mismatches = new List(); + foreach (var name in scripts.Keys.Where(n => Scene.IsMatch(n)).OrderBy(n => n, StringComparer.Ordinal)) + { + var script = Sys4Loader.Load(scripts[name], table); + var vm = new VirtualMachine(script, table, new CaptureHost()); + vm.Run(); + var offsets = vm.Emitted.Select(e => e.Offset).ToArray(); + + if (!expected.TryGetProperty(name, out var exp)) { mismatches.Add($"{name}: absent in vm0 trace"); continue; } + var expOffsets = exp.GetProperty("offsets").EnumerateArray().Select(x => x.GetInt32()).ToArray(); + string expHalt = exp.GetProperty("halt").GetString() ?? ""; + long expSteps = exp.GetProperty("steps").GetInt64(); + + if (!offsets.SequenceEqual(expOffsets)) + mismatches.Add($"{name}: offsets differ (cs {offsets.Length} vs vm0 {expOffsets.Length}; first diff at {FirstDiff(offsets, expOffsets)})"); + else if (vm.HaltReason != expHalt) mismatches.Add($"{name}: halt cs='{vm.HaltReason}' vs vm0='{expHalt}'"); + else if (vm.Steps != expSteps) mismatches.Add($"{name}: steps cs={vm.Steps} vs vm0={expSteps}"); + } + Assert.True(mismatches.Count == 0, "scene mismatches:\n" + string.Join("\n", mismatches.Take(20))); + } + + private static int FirstDiff(int[] a, int[] b) + { + int n = Math.Min(a.Length, b.Length); + for (int i = 0; i < n; i++) if (a[i] != b[i]) return i; + return n; + } +} +``` + +- [ ] **Step 2: Add `--trace` to `tools/vm0.py`** — insert this function before `def main(` and a branch in `main`: + +```python +def run_trace(out_path): + """Dump per-scene emitted show-text offsets + halt + steps for the differential test.""" + oracle = load_oracle() + scripts = paths.scripts() + names = sorted(n for n in scripts if SCENE_RE.match(n)) + trace = {} + for name in names: + r = run_scene(name, scripts[name], oracle) + vm = r["vm"] + trace[name] = {"offsets": [off for off, _ in vm.text], + "halt": vm.halt_reason, "steps": vm.steps} + Path(out_path).write_text(json.dumps(trace, ensure_ascii=False), encoding="utf-8") + print(f"trace: {len(trace)} scenes -> {out_path}") + return 0 +``` + +In `main`, add the branch (after the `--sweep` branch): +```python + if argv[0] == "--trace": + return run_trace(argv[1]) +``` + +- [ ] **Step 3: Generate the reference trace and confirm it matches the sweep** +```bash +cd "S:/Game Hacking/Eushully/Himegari/age-reimpl" +py -3.11 -X utf8 tools/vm0.py --trace build/vm0-trace.json +py -3.11 -X utf8 -c "import json; d=json.load(open('build/vm0-trace.json',encoding='utf-8')); print(len(d),'scenes')" +``` +Expected: `trace: N scenes -> build/vm0-trace.json` and the count prints (≈297 SC/SP names). + +- [ ] **Step 4: Run the C# test — expect FAIL** (Program/CLI not needed yet, but the test drives the C# VM directly) +```bash +dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~TraceDiffTests +``` +Expected: FAIL only if a scene diverges — the message names the scene + first differing offset index. Investigate each divergence against `tools/vm0.py --scene `; fix the VM/parser until the trace is identical. (If it passes first try, even better.) + +- [ ] **Step 5: Implement the CLI** — `engine/Age.Cli/Program.cs`: + +```csharp +using System.Text.Json; +using System.Text.RegularExpressions; +using Age.Engine.Hosting; +using Age.Engine.Sys4; +using Age.Engine.Vm; + +var table = OpcodeTableJson.Load(Paths.OpcodesJson); + +if (args.Length == 0) { Console.WriteLine("usage: run | trace "); return 1; } + +if (args[0] == "run") +{ + var script = Sys4Loader.Load(args[1], table); + var vm = new VirtualMachine(script, table, new CaptureHost()); + vm.Run(); + Console.WriteLine($"{Path.GetFileName(args[1])}: {vm.Steps} steps, {vm.Emitted.Count} show-text (halt: {vm.HaltReason})"); + foreach (var (off, text) in vm.Emitted.Take(20)) Console.WriteLine($" [{off:x}] {text}"); + return 0; +} + +if (args[0] == "trace") +{ + var scene = new Regex(@"^S[CP]\d{4}\.BIN$"); + var scripts = Paths.Scripts(); + var trace = new SortedDictionary(StringComparer.Ordinal); + foreach (var name in scripts.Keys.Where(n => scene.IsMatch(n)).OrderBy(n => n, StringComparer.Ordinal)) + { + var vm = new VirtualMachine(Sys4Loader.Load(scripts[name], table), table, new CaptureHost()); + vm.Run(); + trace[name] = new { offsets = vm.Emitted.Select(e => e.Offset).ToArray(), halt = vm.HaltReason, steps = vm.Steps }; + } + File.WriteAllText(args[1], JsonSerializer.Serialize(trace, new JsonSerializerOptions { Encoder = System.Text.Encodings.Web.JavaScriptEncoder.UnsafeRelaxedJsonEscaping })); + Console.WriteLine($"trace: {trace.Count} scenes -> {args[1]}"); + return 0; +} +Console.WriteLine("unknown command"); return 1; +``` + +- [ ] **Step 6: Run the C# test — expect PASS**, then cross-check the CLI trace +```bash +dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~TraceDiffTests +dotnet run --project engine/Age.Cli -- trace build/cs-trace.json +py -3.11 -X utf8 -c "import json;a=json.load(open('build/vm0-trace.json',encoding='utf-8'));b=json.load(open('build/cs-trace.json',encoding='utf-8'));print('identical' if a==b else 'DIFFER: '+str([k for k in a if a[k]!=b.get(k)][:5]))" +``` +Expected: test passes; the Python cross-check prints `identical`. + +- [ ] **Step 7: Full solution test + commit** +```bash +dotnet test engine/AgeEngine.sln +git add engine/Age.Cli/Program.cs engine/Age.Engine.Tests/TraceDiffTests.cs tools/vm0.py +git commit -m "feat(a1): CLI (run/trace) + vm0.py --trace + per-scene differential test (A1 green)" +``` + +--- + +## Self-Review + +**Spec coverage:** +- Standalone .NET 8 solution (Task 1) ✓ · three seams + neutral `Script` contract (Model in Task 1, seam rule in Global Constraints, VM refs only Model+Hosting in Task 4) ✓ · SYS4 front-end parser+codec+opcode-table (Tasks 2–3, Task 1) ✓ · `IHost`/`CaptureHost` (Task 4) ✓ · faithful vm0 handlers incl. pointer semantics, emit-cap, halt reasons, PyDiv/PyMod (Task 4) ✓ · integer-parity + cp932 registration (Task 4, Task 2) ✓ · per-scene trace diff + RECOVER + parser/codec-parity tests (Tasks 3,4,5) ✓ · reads `build/opcodes.json` via `Paths` (Task 1) ✓ · `vm0.py --trace` addition (Task 5) ✓. +- Codec-parity note: the offset-only trace can't catch a decode bug; `Sys4LoaderTests` asserts `Strings.Count` and the ASCII codec test covers the codec — Japanese decode is exercised transitively (MENU strings) and by the trace run loading every scene without codec exceptions. (Acceptable; a byte-for-byte Japanese golden could be added if a divergence appears.) + +**Placeholder scan:** none — every code step is complete; the only "investigate" is Task 5 Step 4, which is the differential-debugging loop (inherent) and points at `vm0.py --scene` for each named divergence. + +**Type consistency:** `OpcodeTable`, `Script`, `Operand(Type,Value)`, `Instruction(Offset,Opcode,Args)`, `Sys4Loader.Load(path,table)`, `VirtualMachine(script,table,host,options?)` with `.Globals/.Emitted/.HaltReason/.Steps/.Run()`, `Paths.Scripts()/OpcodesJson/Build/Data1`, `IHost.ShowText/CallScript/OnStub` — used identically across Tasks 1→5. Trace JSON keys (`offsets`/`halt`/`steps`) match between `vm0.py --trace` (Task 5 Step 2) and both C# consumers (Task 5 Steps 1 & 5). Halt-reason format `LOOP:line@0x{off:x}×{count}` matches vm0's `{v:#x}` (0x-prefixed) exactly.