Files
OpenMaidEngine/docs/superpowers/plans/2026-07-06-a1-csharp-vm.md
gamer147 a50b7f600a docs(a1): implementation plan for the C# VM core
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 13:45:05 -04:00

910 lines
40 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# A1 — C# VM Core Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** A headless C# (.NET 8) reimplementation of the validated `vm0.py` AGE/SYS4 model that is per-scene trace-identical to the Python prototype and passes the RECOVER unit test.
**Architecture:** A `.NET 8` class library `Age.Engine` with four seam-namespaces — `Model` (version-neutral `Script` contract), `Vm` (execution core, depends only on `Model` + `Hosting`), `Sys4` (`.BIN` parser + cp932 codec + opcode-table loader), `Hosting` (`IHost` + headless `CaptureHost`) — plus a console `Age.Cli` and an `Age.Engine.Tests` xUnit project. Validated by a per-scene trace diff against `tools/vm0.py --trace`.
**Tech Stack:** C# / .NET 8, xUnit, `System.Text.Json`, `System.Text.Encoding.CodePages` (cp932). Python 3.11 for the `vm0.py --trace` reference.
## Global Constraints
- **Work on a feature branch** (e.g. `feat/a1-csharp-vm`); do not commit to `main`.
- Build: `dotnet build engine/AgeEngine.sln`. Test: `dotnet test engine/AgeEngine.sln` (or `--filter FullyQualifiedName~<Class>`).
- **Seam rule:** `Age.Engine.Vm` may reference only `Age.Engine.Model` and `Age.Engine.Hosting` — never `Age.Engine.Sys4`, never file I/O.
- All VM numeric values are `long`. `div`/`mod` use the Python-parity helpers `PyDiv`/`PyMod` (floor-div; divisor-sign mod). u32 operands load as **unsigned** into `long` (no sign-extension), matching vm0.
- Halt-reason strings must be **byte-identical** to vm0: `"exit"`, `"LOOP:line@0x{off:x}×{count}"` (× = U+00D7), `"STEP-LIMIT"`, `"ret-underflow"`, `"pc-out-of-range"`. Source files are UTF-8.
- Data prerequisites (resolved via `Paths`, repo-relative, like `tools/paths.py`): `build/opcodes.json` (run `tools/opcodes_build.py --build`), `../extracted/DATA1/*.BIN`, and — for the trace-diff — `build/vm0-trace.json` (produced by Task 5).
- Exact format facts (from `tools/sys4load.py`, `tools/vm0.py`): magic prefix `SYS4`, header 0x3C bytes = 8 magic + 13×u32 (F0F12), body dwords from 0x3C; code section `[0..F8)`; instruction length `1 + 2*argc` dwords; `data_array_end` shrink stops code at the lowest referenced string(type-2)/array(op `0x64` arg index 1) offset; operand types `0=imm 2=string 3=g-int 4=g-float 5=g-string 6=g-ptr 9=l-int 10=l-float 11=l-string 12=l-ptr`; `jcc` sentinel `0xFFFFFFFF` = fallthrough; `ARRAY_OPCODE=0x64`; `EMIT_CAP=2`, `MAX_STEPS=2_000_000`.
---
## File Structure
```
engine/AgeEngine.sln
engine/Age.Engine/Age.Engine.csproj
engine/Age.Engine/Model/ Operand.cs Instruction.cs ScriptHeader.cs Script.cs OpcodeTable.cs
engine/Age.Engine/Sys4/ Paths.cs OpcodeTableJson.cs Sys4StringCodec.cs Sys4Loader.cs
engine/Age.Engine/Vm/ VmOptions.cs Frame.cs VirtualMachine.cs
engine/Age.Engine/Hosting/ IHost.cs CaptureHost.cs
engine/Age.Cli/Age.Cli.csproj Program.cs
engine/Age.Engine.Tests/Age.Engine.Tests.csproj
OpcodeTableTests.cs Sys4StringCodecTests.cs Sys4LoaderTests.cs RecoverTests.cs TraceDiffTests.cs
tools/vm0.py (modify: add --trace)
.gitignore (modify: ignore engine build output)
```
---
## Task 1: Solution scaffold + Model + OpcodeTable
**Files:** Create the solution, `Age.Engine/Model/*`, `Age.Engine/Sys4/Paths.cs`, `Age.Engine/Sys4/OpcodeTableJson.cs`; Test `Age.Engine.Tests/OpcodeTableTests.cs`; Modify `.gitignore`.
**Interfaces — Produces:**
- `Age.Engine.Model.Operand(int Type, long Value)` (readonly record struct)
- `Age.Engine.Model.Instruction(int Offset, int Opcode, IReadOnlyList<Operand> Args)`
- `Age.Engine.Model.ScriptHeader(int LocalInt1, int LocalFloats, int LocalStrings1, int LocalInt2, int Unknown, int LocalStrings2)`
- `Age.Engine.Model.Script { ScriptHeader Header; IReadOnlyList<Instruction> Instructions; IReadOnlyDictionary<int,int> IndexByOffset; IReadOnlyDictionary<int,string> Strings; string GetString(int) }`
- `Age.Engine.Model.OpcodeTable { bool TryGet(int,out string,out int); string Label(int); int Argc(int); int Count }`
- `Age.Engine.Sys4.OpcodeTableJson.Load(string path) -> OpcodeTable`
- `Age.Engine.Sys4.Paths { string Repo, Extracted, Data1, GameDir, Build, OpcodesJson; Dictionary<string,string> Scripts() }`
- [ ] **Step 1: Scaffold the solution**
```bash
cd "S:/Game Hacking/Eushully/Himegari/age-reimpl"
dotnet new sln -n AgeEngine -o engine
dotnet new classlib -n Age.Engine -o engine/Age.Engine -f net8.0
dotnet new console -n Age.Cli -o engine/Age.Cli -f net8.0
dotnet new xunit -n Age.Engine.Tests -o engine/Age.Engine.Tests -f net8.0
rm -f engine/Age.Engine/Class1.cs engine/Age.Engine.Tests/UnitTest1.cs
dotnet sln engine/AgeEngine.sln add engine/Age.Engine engine/Age.Cli engine/Age.Engine.Tests
dotnet add engine/Age.Cli reference engine/Age.Engine
dotnet add engine/Age.Engine.Tests reference engine/Age.Engine
dotnet add engine/Age.Engine package System.Text.Encoding.CodePages
```
- [ ] **Step 2: Ignore .NET build output** — append to `.gitignore`:
```
# .NET (engine/) build output
engine/**/bin/
engine/**/obj/
.vs/
```
- [ ] **Step 3: Write the failing test**`engine/Age.Engine.Tests/OpcodeTableTests.cs`:
```csharp
using Age.Engine.Sys4;
using Xunit;
public class OpcodeTableTests
{
[Fact]
public void LoadsAll248FromJson()
{
var t = OpcodeTableJson.Load(Paths.OpcodesJson);
Assert.Equal(248, t.Count);
Assert.True(t.TryGet(0x55, out var label, out var argc));
Assert.Equal("mov", label);
Assert.Equal(2, argc);
Assert.Equal("u0041BEB0", t.Label(0x90));
Assert.Equal(7, t.Argc(0x90));
Assert.Equal(-1, t.Argc(0x9999)); // absent -> -1
}
}
```
- [ ] **Step 4: Run — expect FAIL (types missing / won't compile)**
```bash
dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~OpcodeTableTests
```
Expected: build error — `Paths`/`OpcodeTableJson` do not exist.
- [ ] **Step 5: Implement Model + Paths + OpcodeTableJson**
`engine/Age.Engine/Model/Operand.cs`:
```csharp
namespace Age.Engine.Model;
public readonly record struct Operand(int Type, long Value);
```
`engine/Age.Engine/Model/Instruction.cs`:
```csharp
namespace Age.Engine.Model;
public sealed record Instruction(int Offset, int Opcode, IReadOnlyList<Operand> Args);
```
`engine/Age.Engine/Model/ScriptHeader.cs`:
```csharp
namespace Age.Engine.Model;
public sealed record ScriptHeader(
int LocalInt1, int LocalFloats, int LocalStrings1,
int LocalInt2, int Unknown, int LocalStrings2);
```
`engine/Age.Engine/Model/Script.cs`:
```csharp
namespace Age.Engine.Model;
public sealed class Script
{
public required ScriptHeader Header { get; init; }
public required IReadOnlyList<Instruction> Instructions { get; init; }
public required IReadOnlyDictionary<int, int> IndexByOffset { get; init; }
public required IReadOnlyDictionary<int, string> Strings { get; init; }
public string GetString(int offset) => Strings.TryGetValue(offset, out var s) ? s : "";
}
```
`engine/Age.Engine/Model/OpcodeTable.cs`:
```csharp
namespace Age.Engine.Model;
public sealed class OpcodeTable
{
private readonly IReadOnlyDictionary<int, (string Label, int Argc)> _t;
public OpcodeTable(IReadOnlyDictionary<int, (string, int)> t) => _t = t;
public int Count => _t.Count;
public bool TryGet(int op, out string label, out int argc)
{
if (_t.TryGetValue(op, out var e)) { label = e.Label; argc = e.Argc; return true; }
label = ""; argc = -1; return false;
}
public string Label(int op) => _t.TryGetValue(op, out var e) ? e.Label : "";
public int Argc(int op) => _t.TryGetValue(op, out var e) ? e.Argc : -1;
}
```
`engine/Age.Engine/Sys4/Paths.cs`:
```csharp
namespace Age.Engine.Sys4;
public static class Paths
{
public static string Repo { get; } = FindRepo();
public static string Workspace => Directory.GetParent(Repo)!.FullName;
public static string Extracted => Path.Combine(Workspace, "extracted");
public static string Data1 => Path.Combine(Extracted, "DATA1");
public static string GameDir => Path.Combine(Workspace, "姫狩りダンジョンマイスター");
public static string Build => Path.Combine(Repo, "build");
public static string OpcodesJson => Path.Combine(Build, "opcodes.json");
private static string FindRepo()
{
var d = new DirectoryInfo(AppContext.BaseDirectory);
while (d != null && d.Name != "age-reimpl") d = d.Parent;
if (d == null) throw new DirectoryNotFoundException(
"age-reimpl root not found above " + AppContext.BaseDirectory);
return d.FullName;
}
/// name(UPPER).BIN -> path; game-dir loose overrides shadow extracted/DATA1 (mirrors paths.scripts()).
public static Dictionary<string, string> Scripts()
{
var d = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
if (Directory.Exists(Data1))
foreach (var p in Directory.EnumerateFiles(Data1, "*.BIN"))
d[Path.GetFileName(p).ToUpperInvariant()] = p;
if (Directory.Exists(GameDir))
foreach (var p in Directory.EnumerateFiles(GameDir, "*.BIN"))
d[Path.GetFileName(p).ToUpperInvariant()] = p;
return d;
}
}
```
`engine/Age.Engine/Sys4/OpcodeTableJson.cs`:
```csharp
using System.Text.Json;
using Age.Engine.Model;
namespace Age.Engine.Sys4;
public static class OpcodeTableJson
{
public static OpcodeTable Load(string path)
{
using var doc = JsonDocument.Parse(File.ReadAllText(path));
var dict = new Dictionary<int, (string, int)>();
foreach (var e in doc.RootElement.GetProperty("opcodes").EnumerateArray())
{
int op = Convert.ToInt32(e.GetProperty("op").GetString(), 16);
string label = e.GetProperty("label").GetString() ?? "";
int argc = e.GetProperty("argc").GetInt32();
dict[op] = (label, argc);
}
return new OpcodeTable(dict);
}
}
```
- [ ] **Step 6: Run — expect PASS**
```bash
dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~OpcodeTableTests
```
Expected: 1 passed. (Prereq: `tools/opcodes_build.py --build` has produced `build/opcodes.json`.)
- [ ] **Step 7: Commit**
```bash
git add engine .gitignore
git commit -m "feat(a1): .NET 8 solution scaffold + Model + OpcodeTable loader"
```
---
## Task 2: SYS4 string codec
**Files:** Create `Age.Engine/Sys4/Sys4StringCodec.cs`; Test `Age.Engine.Tests/Sys4StringCodecTests.cs`.
**Interfaces — Produces:** `Sys4StringCodec.Decode(IReadOnlyList<uint> dwords, int start, int limit = 4096) -> (string? Text, int NDwords)` — XOR-`0xFFFFFFFF` per little-endian dword, cut at first NUL, strict cp932 validation (returns `(null,0)` when not a clean string), else decoded text + dword length including the NUL dword.
- [ ] **Step 1: Write the failing test**`engine/Age.Engine.Tests/Sys4StringCodecTests.cs`:
```csharp
using Age.Engine.Sys4;
using Xunit;
public class Sys4StringCodecTests
{
[Fact]
public void DecodesXorFfCp932Ascii()
{
// "AB\0\0" little-endian = 0x00004241, stored XOR 0xFFFFFFFF
uint[] dw = { 0x00004241u ^ 0xFFFFFFFFu };
var (text, nd) = Sys4StringCodec.Decode(dw, 0);
Assert.Equal("AB", text);
Assert.Equal(1, nd);
}
[Fact]
public void RejectsNonString()
{
// 0x00000000 XOR-decodes to 0xFFFFFFFF bytes (0xFF,0xFF,0xFF,0xFF) -> no NUL, invalid
uint[] dw = { 0x00000000u };
var (text, _) = Sys4StringCodec.Decode(dw, 0);
Assert.Null(text);
}
}
```
- [ ] **Step 2: Run — expect FAIL** (`Sys4StringCodec` missing)
```bash
dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~Sys4StringCodecTests
```
- [ ] **Step 3: Implement**`engine/Age.Engine/Sys4/Sys4StringCodec.cs`:
```csharp
using System.Text;
namespace Age.Engine.Sys4;
public static class Sys4StringCodec
{
static Sys4StringCodec() => Encoding.RegisterProvider(CodePagesEncodingProvider.Instance);
private static readonly Encoding Cp932 = Encoding.GetEncoding(932);
public static (string? Text, int NDwords) Decode(IReadOnlyList<uint> dwords, int start, int limit = 4096)
{
int n = dwords.Count, end = Math.Min(start + limit, n);
var raw = new List<byte>();
bool sawNul = false;
for (int j = start; j < end; j++)
{
uint x = dwords[j] ^ 0xFFFFFFFFu;
raw.Add((byte)(x & 0xFF)); raw.Add((byte)((x >> 8) & 0xFF));
raw.Add((byte)((x >> 16) & 0xFF)); raw.Add((byte)((x >> 24) & 0xFF));
if (raw[^1] == 0 || raw[^2] == 0 || raw[^3] == 0 || raw[^4] == 0) { sawNul = true; break; }
}
if (!sawNul) return (null, 0);
int nul = raw.IndexOf(0);
byte[] s = raw.GetRange(0, nul < 0 ? raw.Count : nul).ToArray();
if (s.Length < 1) return ("", 1);
int i = 0, chars = 0;
while (i < s.Length)
{
byte b = s[i];
if (b >= 0x20 && b <= 0x7E) { i++; chars++; }
else if ((b >= 0x81 && b <= 0x9F) || (b >= 0xE0 && b <= 0xEA))
{
if (i + 1 < s.Length && s[i + 1] >= 0x40 && s[i + 1] <= 0xFC && s[i + 1] != 0x7F) { i += 2; chars++; }
else return (null, 0);
}
else return (null, 0);
}
if (chars < 1) return (null, 0);
string text;
try { text = Cp932.GetString(s); } catch { return (null, 0); }
return (text, (s.Length / 4) + 1);
}
}
```
- [ ] **Step 4: Run — expect PASS**
```bash
dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~Sys4StringCodecTests
```
- [ ] **Step 5: Commit**
```bash
git add engine/Age.Engine/Sys4/Sys4StringCodec.cs engine/Age.Engine.Tests/Sys4StringCodecTests.cs
git commit -m "feat(a1): SYS4 XOR-FF cp932 string codec"
```
---
## Task 3: SYS4 container parser (`Sys4Loader`)
**Files:** Create `Age.Engine/Sys4/Sys4Loader.cs`; Test `Age.Engine.Tests/Sys4LoaderTests.cs`.
**Interfaces — Consumes:** `OpcodeTable`, `Sys4StringCodec`, `Script`. **Produces:** `Sys4Loader.Load(string path, OpcodeTable table) -> Script`; `Sys4Loader.Parse(byte[] data, OpcodeTable table, string name = "") -> Script`.
- [ ] **Step 1: Write the failing test**`engine/Age.Engine.Tests/Sys4LoaderTests.cs`:
```csharp
using Age.Engine.Sys4;
using Xunit;
public class Sys4LoaderTests
{
[Fact]
public void ParsesMenuBinLikeSys4load()
{
var table = OpcodeTableJson.Load(Paths.OpcodesJson);
var s = Sys4Loader.Load(Path.Combine(Paths.Data1, "MENU.BIN"), table);
// header F0..F5 from `sys4load MENU.BIN --summary`
Assert.Equal(0x17, s.Header.LocalInt1);
Assert.Equal(1, s.Header.LocalFloats);
Assert.Equal(1, s.Header.LocalStrings1);
Assert.Equal(2, s.Header.LocalInt2);
Assert.Equal(1, s.Header.Unknown);
Assert.Equal(1, s.Header.LocalStrings2);
Assert.Equal(148, s.Instructions.Count); // sys4load: 148 instructions
Assert.Equal(3, s.Strings.Count); // sys4load: 3 inline strings
// every instruction offset is indexed
Assert.All(s.Instructions, ins => Assert.True(s.IndexByOffset.ContainsKey(ins.Offset)));
}
}
```
- [ ] **Step 2: Run — expect FAIL** (`Sys4Loader` missing)
```bash
dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~Sys4LoaderTests
```
- [ ] **Step 3: Implement**`engine/Age.Engine/Sys4/Sys4Loader.cs`:
```csharp
using Age.Engine.Model;
namespace Age.Engine.Sys4;
public static class Sys4Loader
{
private const int HeaderSize = 0x3C, BodyOff = 0x3C, NumFields = 13, ArrayOpcode = 0x64;
public static Script Load(string path, OpcodeTable table)
=> Parse(File.ReadAllBytes(path), table, Path.GetFileName(path));
public static Script Parse(byte[] data, OpcodeTable table, string name = "")
{
if (data.Length < HeaderSize) throw new InvalidDataException($"{name}: too small");
if (!(data[0] == (byte)'S' && data[1] == (byte)'Y' && data[2] == (byte)'S' && data[3] == (byte)'4'))
throw new InvalidDataException($"{name}: bad magic");
if (data.Length % 4 != 0) throw new InvalidDataException($"{name}: not dword-aligned");
var fields = new int[NumFields];
for (int k = 0; k < NumFields; k++) fields[k] = BitConverter.ToInt32(data, 8 + k * 4);
int nbody = (data.Length - BodyOff) / 4;
var dw = new uint[nbody];
for (int k = 0; k < nbody; k++) dw[k] = BitConverter.ToUInt32(data, BodyOff + k * 4);
var header = new ScriptHeader(fields[0], fields[1], fields[2], fields[3], fields[4], fields[5]);
var (instrs, idxByOff, strings) = DecodeCode(dw, fields, nbody, table);
return new Script { Header = header, Instructions = instrs, IndexByOffset = idxByOff, Strings = strings };
}
private static (List<Instruction>, Dictionary<int, int>, Dictionary<int, string>)
DecodeCode(uint[] dw, int[] fields, int nbody, OpcodeTable table)
{
int codeEnd = fields[8]; // F8; shrinks to first inline string/array offset
var instrs = new List<Instruction>();
var idx = new Dictionary<int, int>();
var strings = new Dictionary<int, string>();
int i = 0;
while (i < codeEnd)
{
int op = (int)dw[i];
int argc = table.Argc(op);
if (argc < 0) { idx[i] = instrs.Count; instrs.Add(new Instruction(i, op, Array.Empty<Operand>())); break; }
int baseI = i + 1;
if (baseI + 2 * argc > codeEnd) { idx[i] = instrs.Count; instrs.Add(new Instruction(i, op, Array.Empty<Operand>())); break; }
var args = new Operand[argc];
for (int a = 0; a < argc; a++)
{
int atype = (int)dw[baseI + 2 * a];
long aval = dw[baseI + 2 * a + 1];
args[a] = new Operand(atype, aval);
if (atype == 2 && aval >= 0 && aval < nbody)
{
if (aval < codeEnd) codeEnd = (int)aval;
if (!strings.ContainsKey((int)aval))
{
var (text, _) = Sys4StringCodec.Decode(dw, (int)aval);
if (text != null) strings[(int)aval] = text;
}
}
else if (op == ArrayOpcode && a == 1 && aval >= 0 && aval < nbody)
{
if (aval < codeEnd) codeEnd = (int)aval;
}
}
idx[i] = instrs.Count;
instrs.Add(new Instruction(i, op, args));
i = baseI + 2 * argc;
}
return (instrs, idx, strings);
}
}
```
- [ ] **Step 4: Run — expect PASS**
```bash
dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~Sys4LoaderTests
```
Expected: 1 passed (148 instructions, 3 strings, header matches). If instruction count differs, the `data_array_end` shrink or argc walk diverged — compare against `py -3.11 -X utf8 tools/sys4load.py ../extracted/DATA1/MENU.BIN --summary`.
- [ ] **Step 5: Commit**
```bash
git add engine/Age.Engine/Sys4/Sys4Loader.cs engine/Age.Engine.Tests/Sys4LoaderTests.cs
git commit -m "feat(a1): SYS4 container parser (header + code decode + data_array_end shrink)"
```
---
## Task 4: VM core + IHost/CaptureHost + RECOVER
**Files:** Create `Age.Engine/Hosting/IHost.cs`, `Age.Engine/Hosting/CaptureHost.cs`, `Age.Engine/Vm/VmOptions.cs`, `Age.Engine/Vm/Frame.cs`, `Age.Engine/Vm/VirtualMachine.cs`; Test `Age.Engine.Tests/RecoverTests.cs`.
**Interfaces — Consumes:** `Script`, `OpcodeTable`, `Sys4Loader`. **Produces:**
- `Age.Engine.Hosting.IHost { void ShowText(int,string); void CallScript(long); void OnStub(int); }`, `CaptureHost : IHost`
- `Age.Engine.Vm.VmOptions(int EmitCap = 2, long MaxSteps = 2_000_000)`
- `Age.Engine.Vm.VirtualMachine(Script, OpcodeTable, IHost, VmOptions? = null)` with public `Dictionary<int,long> Globals`, `Dictionary<int,string> GlobalStrings`, `List<(int Offset,string Text)> Emitted`, `string? HaltReason`, `long Steps`, and `void Run(int entryOffset = 0)`.
- [ ] **Step 1: Write the failing test**`engine/Age.Engine.Tests/RecoverTests.cs`:
```csharp
using Age.Engine.Hosting;
using Age.Engine.Sys4;
using Age.Engine.Vm;
using Xunit;
public class RecoverTests
{
private static long G(VirtualMachine vm, int k) => vm.Globals.TryGetValue(k, out var v) ? v : 0;
[Fact]
public void RecoverUnitTestPasses()
{
var table = OpcodeTableJson.Load(Paths.OpcodesJson);
var script = Sys4Loader.Load(Path.Combine(Paths.Data1, "RECOVER.BIN"), table);
var vm = new VirtualMachine(script, table, new CaptureHost());
int unit = 0;
vm.Globals[0x152616] = unit;
int A = 0x4e11b, B = 0x4e085, C = 0x52383, E = 0x52f3b, F = 0x5295f, FL = 0xaacb4;
for (int k = 0; k < 3; k++) vm.Globals[A + unit * 14 + (11 + k)] = 100 + k;
vm.Globals[C + unit * 30 + 5] = 7; vm.Globals[FL + 5] = 1; vm.Globals[E + unit * 30 + 5] = 42;
vm.Globals[C + unit * 30 + 6] = 0; vm.Globals[FL + 6] = 1; vm.Globals[E + unit * 30 + 6] = 99;
vm.Globals[C + unit * 30 + 7] = 3; vm.Globals[FL + 7] = 0; vm.Globals[E + unit * 30 + 7] = 88;
vm.Run();
Assert.Equal(100, G(vm, B + unit * 3 + 0));
Assert.Equal(101, G(vm, B + unit * 3 + 1));
Assert.Equal(102, G(vm, B + unit * 3 + 2));
Assert.Equal(42, G(vm, C + unit * 30 + 5));
Assert.Equal(-1, G(vm, F + unit * 30 + 5));
Assert.Equal(0, G(vm, C + unit * 30 + 6));
Assert.Equal(3, G(vm, C + unit * 30 + 7));
}
}
```
- [ ] **Step 2: Run — expect FAIL** (VM types missing)
```bash
dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~RecoverTests
```
- [ ] **Step 3: Implement hosting + VM**
`engine/Age.Engine/Hosting/IHost.cs`:
```csharp
namespace Age.Engine.Hosting;
public interface IHost
{
void ShowText(int offset, string text);
void CallScript(long id);
void OnStub(int opcode);
}
```
`engine/Age.Engine/Hosting/CaptureHost.cs`:
```csharp
namespace Age.Engine.Hosting;
public sealed class CaptureHost : IHost
{
public List<(int Offset, string Text)> Emitted { get; } = new();
public int CallScriptCount { get; private set; }
public Dictionary<int, int> Stubs { get; } = new();
public void ShowText(int offset, string text) => Emitted.Add((offset, text));
public void CallScript(long id) => CallScriptCount++;
public void OnStub(int opcode) { Stubs.TryGetValue(opcode, out var c); Stubs[opcode] = c + 1; }
}
```
`engine/Age.Engine/Vm/VmOptions.cs`:
```csharp
namespace Age.Engine.Vm;
public sealed record VmOptions(int EmitCap = 2, long MaxSteps = 2_000_000);
```
`engine/Age.Engine/Vm/Frame.cs`:
```csharp
namespace Age.Engine.Vm;
public sealed class Frame
{
public Dictionary<int, long> I = new(); // local-int
public Dictionary<int, long> F = new(); // local-float (raw)
public Dictionary<int, string> S = new(); // local-string
public Dictionary<int, long> P = new(); // local-ptr (holds a global address)
}
```
`engine/Age.Engine/Vm/VirtualMachine.cs`:
```csharp
using Age.Engine.Hosting;
using Age.Engine.Model;
namespace Age.Engine.Vm;
public sealed class VirtualMachine
{
private const long NoJump = 0xFFFFFFFF;
private const int HALT = int.MinValue;
private const int T_IMM = 0, T_STR = 2, T_GINT = 3, T_GFLOAT = 4, T_GSTR = 5, T_GPTR = 6,
T_LINT = 9, T_LFLOAT = 10, T_LSTR = 11, T_LPTR = 12;
private readonly Script _s;
private readonly OpcodeTable _t;
private readonly IHost _host;
private readonly VmOptions _o;
private readonly Frame _fr = new();
private readonly List<int> _callstack = new();
private readonly Dictionary<int, int> _emitSeen = new();
public Dictionary<int, long> Globals { get; } = new();
public Dictionary<int, string> GlobalStrings { get; } = new();
public List<(int Offset, string Text)> Emitted { get; } = new();
public string? HaltReason { get; private set; }
public long Steps { get; private set; }
public VirtualMachine(Script s, OpcodeTable t, IHost host, VmOptions? o = null)
{ _s = s; _t = t; _host = host; _o = o ?? new VmOptions(); }
private static long Gi(Dictionary<int, long> d, int k) => d.TryGetValue(k, out var v) ? v : 0;
private static string Gs(Dictionary<int, string> d, int k) => d.TryGetValue(k, out var v) ? v : "";
private static long PyDiv(long a, long b) { if (b == 0) return 0; long q = a / b, r = a % b; if (r != 0 && (r < 0) != (b < 0)) q--; return q; }
private static long PyMod(long a, long b) { if (b == 0) return 0; long r = a % b; if (r != 0 && (r < 0) != (b < 0)) r += b; return r; }
private static bool IsStr(Operand o) => o.Type == T_STR || o.Type == T_GSTR || o.Type == T_LSTR;
private long Read(Operand op) => op.Type switch
{
T_IMM => op.Value,
T_GINT or T_GFLOAT => Gi(Globals, (int)op.Value),
T_GPTR => Gi(Globals, (int)Gi(Globals, (int)op.Value)),
T_LINT => Gi(_fr.I, (int)op.Value),
T_LFLOAT => Gi(_fr.F, (int)op.Value),
T_LPTR => Gi(Globals, (int)Gi(_fr.P, (int)op.Value)),
_ => op.Value,
};
private void Write(Operand op, long val)
{
switch (op.Type)
{
case T_GINT: case T_GFLOAT: Globals[(int)op.Value] = val; break;
case T_GPTR: Globals[(int)Gi(Globals, (int)op.Value)] = val; break;
case T_LINT: _fr.I[(int)op.Value] = val; break;
case T_LFLOAT: _fr.F[(int)op.Value] = val; break;
case T_LPTR: Globals[(int)Gi(_fr.P, (int)op.Value)] = val; break;
}
}
private string ReadStr(Operand op) => op.Type switch
{
T_STR => _s.GetString((int)op.Value),
T_GSTR => Gs(GlobalStrings, (int)op.Value),
T_LSTR => Gs(_fr.S, (int)op.Value),
_ => "",
};
private void WriteStr(Operand op, string val)
{
switch (op.Type)
{
case T_GSTR: GlobalStrings[(int)op.Value] = val; break;
case T_LSTR: _fr.S[(int)op.Value] = val; break;
}
}
private long BaseAddr(Operand op) => op.Type switch
{
T_IMM or T_GINT or T_GFLOAT or T_GSTR or T_GPTR => op.Value,
T_LINT => Gi(_fr.I, (int)op.Value),
T_LPTR => Gi(_fr.P, (int)op.Value),
_ => op.Value,
};
private void LookupStore(Operand dst, long addr)
{
switch (dst.Type)
{
case T_LPTR: _fr.P[(int)dst.Value] = addr; break;
case T_GPTR: Globals[(int)dst.Value] = addr; break;
default: Write(dst, Gi(Globals, (int)addr)); break;
}
}
public void Run(int entryOffset = 0)
{
int pc = _s.IndexByOffset.TryGetValue(entryOffset, out var idx) ? idx : 0;
while (pc >= 0 && pc < _s.Instructions.Count)
{
if (Steps >= _o.MaxSteps) { HaltReason ??= "STEP-LIMIT"; return; }
Steps++;
int next = Step(_s.Instructions[pc], pc);
if (next == HALT) return;
pc = next;
}
HaltReason ??= "pc-out-of-range";
}
private int Step(Instruction ins, int pc)
{
int op = ins.Opcode;
var a = ins.Args;
switch (_t.Label(op))
{
case "add": Write(a[0], Read(a[1]) + Read(a[2])); return pc + 1;
case "sub": Write(a[0], Read(a[1]) - Read(a[2])); return pc + 1;
case "mul": Write(a[0], Read(a[1]) * Read(a[2])); return pc + 1;
case "div": Write(a[0], PyDiv(Read(a[1]), Read(a[2]))); return pc + 1;
case "mod": Write(a[0], PyMod(Read(a[1]), Read(a[2]))); return pc + 1;
case "and": Write(a[0], Read(a[1]) & Read(a[2])); return pc + 1;
case "or": Write(a[0], Read(a[1]) | Read(a[2])); return pc + 1;
case "sar": Write(a[0], Read(a[1]) >> (int)(Read(a[2]) & 31)); return pc + 1;
case "shl": Write(a[0], Read(a[1]) << (int)(Read(a[2]) & 31)); return pc + 1;
case "eq": Write(a[0], Read(a[1]) == Read(a[2]) ? 1 : 0); return pc + 1;
case "ne": Write(a[0], Read(a[1]) != Read(a[2]) ? 1 : 0); return pc + 1;
case "lt": Write(a[0], Read(a[1]) < Read(a[2]) ? 1 : 0); return pc + 1;
case "lte": Write(a[0], Read(a[1]) <= Read(a[2]) ? 1 : 0); return pc + 1;
case "gr": Write(a[0], Read(a[1]) > Read(a[2]) ? 1 : 0); return pc + 1;
case "gre": Write(a[0], Read(a[1]) >= Read(a[2]) ? 1 : 0); return pc + 1;
case "mov":
case "set-string":
if (IsStr(a[0]) || IsStr(a[1])) WriteStr(a[0], ReadStr(a[1]));
else Write(a[0], Read(a[1]));
return pc + 1;
case "lookup-array":
LookupStore(a[0], BaseAddr(a[1]) + Read(a[2])); return pc + 1;
case "lookup-array-2d":
LookupStore(a[0], BaseAddr(a[1]) + Read(a[2]) * Read(a[3]) + Read(a[4])); return pc + 1;
case "bit-set": Write(a[0], Read(a[0]) | Read(a[1])); return pc + 1;
case "bit-reset": Write(a[0], Read(a[0]) & ~Read(a[1])); return pc + 1;
case "check-bit": Write(a[0], (Read(a[1]) >> (int)(Read(a[2]) & 31)) & 1); return pc + 1;
case "copy-to-global": Write(a[0], Read(a[1])); return pc + 1;
case "jmp": return _s.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1);
case "call": _callstack.Add(pc + 1); return _s.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1);
case "ret":
if (_callstack.Count > 0) { int r = _callstack[^1]; _callstack.RemoveAt(_callstack.Count - 1); return r; }
HaltReason = "ret-underflow"; return HALT;
case "jcc":
{
long tgt = Read(a[0]) != 0 ? a[1].Value : a[2].Value;
return tgt == NoJump ? pc + 1 : _s.IndexByOffset.GetValueOrDefault((int)tgt, pc + 1);
}
case "exit":
case "exit-script": HaltReason = "exit"; return HALT;
case "call-script": _host.CallScript(a.Count > 0 ? Read(a[0]) : 0); return pc + 1;
case "show-text":
foreach (var o in a)
{
if (o.Type != T_STR) continue;
int off = (int)o.Value;
_emitSeen.TryGetValue(off, out var c); c++; _emitSeen[off] = c;
if (c > _o.EmitCap) { HaltReason = $"LOOP:line@0x{off:x}×{c}"; return HALT; }
string text = _s.GetString(off);
Emitted.Add((off, text));
_host.ShowText(off, text);
}
return pc + 1;
case "end-text-line": case "wait-for-input": case "set-font":
case "comment": case "display-furigana": case "dev_ukn":
return pc + 1;
default:
_host.OnStub(op); return pc + 1;
}
}
}
```
- [ ] **Step 4: Run — expect PASS**
```bash
dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~RecoverTests
```
Expected: RECOVER passes (all 7 asserts). If a pointer/array assert fails, compare against `py -3.11 -X utf8 tools/vm0.py --test`.
- [ ] **Step 5: Commit**
```bash
git add engine/Age.Engine/Hosting engine/Age.Engine/Vm engine/Age.Engine.Tests/RecoverTests.cs
git commit -m "feat(a1): VM core (operands+pointer semantics, handlers, emit-cap) + RECOVER"
```
---
## Task 5: `Age.Cli` + `vm0.py --trace` + per-scene trace diff
**Files:** Modify `tools/vm0.py` (add `--trace`); Create `engine/Age.Cli/Program.cs`; Test `Age.Engine.Tests/TraceDiffTests.cs`.
**Interfaces — Consumes:** `Paths.Scripts()`, `Sys4Loader`, `OpcodeTableJson`, `VirtualMachine`, `CaptureHost`. **Produces (shared trace contract):** JSON object `{ "<SCENE>.BIN": { "offsets": [int...], "halt": "<reason>", "steps": int }, ... }` over the sorted set of names matching `^S[CP]\d{4}\.BIN$` from `Paths.Scripts()`. Both `vm0.py --trace <out>` and `Age.Cli trace <out>` emit exactly this.
- [ ] **Step 1: Write the failing test**`engine/Age.Engine.Tests/TraceDiffTests.cs`:
```csharp
using System.Text.Json;
using System.Text.RegularExpressions;
using Age.Engine.Hosting;
using Age.Engine.Sys4;
using Age.Engine.Vm;
using Xunit;
public class TraceDiffTests
{
private static readonly Regex Scene = new(@"^S[CP]\d{4}\.BIN$");
[Fact]
public void CsTraceMatchesVm0PerScene()
{
string refPath = Path.Combine(Paths.Build, "vm0-trace.json");
Assert.True(File.Exists(refPath),
"prerequisite: run `py -3.11 -X utf8 tools/vm0.py --trace build/vm0-trace.json`");
using var doc = JsonDocument.Parse(File.ReadAllText(refPath));
var expected = doc.RootElement;
var table = OpcodeTableJson.Load(Paths.OpcodesJson);
var scripts = Paths.Scripts();
var mismatches = new List<string>();
foreach (var name in scripts.Keys.Where(n => Scene.IsMatch(n)).OrderBy(n => n, StringComparer.Ordinal))
{
var script = Sys4Loader.Load(scripts[name], table);
var vm = new VirtualMachine(script, table, new CaptureHost());
vm.Run();
var offsets = vm.Emitted.Select(e => e.Offset).ToArray();
if (!expected.TryGetProperty(name, out var exp)) { mismatches.Add($"{name}: absent in vm0 trace"); continue; }
var expOffsets = exp.GetProperty("offsets").EnumerateArray().Select(x => x.GetInt32()).ToArray();
string expHalt = exp.GetProperty("halt").GetString() ?? "";
long expSteps = exp.GetProperty("steps").GetInt64();
if (!offsets.SequenceEqual(expOffsets))
mismatches.Add($"{name}: offsets differ (cs {offsets.Length} vs vm0 {expOffsets.Length}; first diff at {FirstDiff(offsets, expOffsets)})");
else if (vm.HaltReason != expHalt) mismatches.Add($"{name}: halt cs='{vm.HaltReason}' vs vm0='{expHalt}'");
else if (vm.Steps != expSteps) mismatches.Add($"{name}: steps cs={vm.Steps} vs vm0={expSteps}");
}
Assert.True(mismatches.Count == 0, "scene mismatches:\n" + string.Join("\n", mismatches.Take(20)));
}
private static int FirstDiff(int[] a, int[] b)
{
int n = Math.Min(a.Length, b.Length);
for (int i = 0; i < n; i++) if (a[i] != b[i]) return i;
return n;
}
}
```
- [ ] **Step 2: Add `--trace` to `tools/vm0.py`** — insert this function before `def main(` and a branch in `main`:
```python
def run_trace(out_path):
"""Dump per-scene emitted show-text offsets + halt + steps for the differential test."""
oracle = load_oracle()
scripts = paths.scripts()
names = sorted(n for n in scripts if SCENE_RE.match(n))
trace = {}
for name in names:
r = run_scene(name, scripts[name], oracle)
vm = r["vm"]
trace[name] = {"offsets": [off for off, _ in vm.text],
"halt": vm.halt_reason, "steps": vm.steps}
Path(out_path).write_text(json.dumps(trace, ensure_ascii=False), encoding="utf-8")
print(f"trace: {len(trace)} scenes -> {out_path}")
return 0
```
In `main`, add the branch (after the `--sweep` branch):
```python
if argv[0] == "--trace":
return run_trace(argv[1])
```
- [ ] **Step 3: Generate the reference trace and confirm it matches the sweep**
```bash
cd "S:/Game Hacking/Eushully/Himegari/age-reimpl"
py -3.11 -X utf8 tools/vm0.py --trace build/vm0-trace.json
py -3.11 -X utf8 -c "import json; d=json.load(open('build/vm0-trace.json',encoding='utf-8')); print(len(d),'scenes')"
```
Expected: `trace: N scenes -> build/vm0-trace.json` and the count prints (≈297 SC/SP names).
- [ ] **Step 4: Run the C# test — expect FAIL** (Program/CLI not needed yet, but the test drives the C# VM directly)
```bash
dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~TraceDiffTests
```
Expected: FAIL only if a scene diverges — the message names the scene + first differing offset index. Investigate each divergence against `tools/vm0.py --scene <NAME>`; fix the VM/parser until the trace is identical. (If it passes first try, even better.)
- [ ] **Step 5: Implement the CLI**`engine/Age.Cli/Program.cs`:
```csharp
using System.Text.Json;
using System.Text.RegularExpressions;
using Age.Engine.Hosting;
using Age.Engine.Sys4;
using Age.Engine.Vm;
var table = OpcodeTableJson.Load(Paths.OpcodesJson);
if (args.Length == 0) { Console.WriteLine("usage: run <file> | trace <out.json>"); return 1; }
if (args[0] == "run")
{
var script = Sys4Loader.Load(args[1], table);
var vm = new VirtualMachine(script, table, new CaptureHost());
vm.Run();
Console.WriteLine($"{Path.GetFileName(args[1])}: {vm.Steps} steps, {vm.Emitted.Count} show-text (halt: {vm.HaltReason})");
foreach (var (off, text) in vm.Emitted.Take(20)) Console.WriteLine($" [{off:x}] {text}");
return 0;
}
if (args[0] == "trace")
{
var scene = new Regex(@"^S[CP]\d{4}\.BIN$");
var scripts = Paths.Scripts();
var trace = new SortedDictionary<string, object>(StringComparer.Ordinal);
foreach (var name in scripts.Keys.Where(n => scene.IsMatch(n)).OrderBy(n => n, StringComparer.Ordinal))
{
var vm = new VirtualMachine(Sys4Loader.Load(scripts[name], table), table, new CaptureHost());
vm.Run();
trace[name] = new { offsets = vm.Emitted.Select(e => e.Offset).ToArray(), halt = vm.HaltReason, steps = vm.Steps };
}
File.WriteAllText(args[1], JsonSerializer.Serialize(trace, new JsonSerializerOptions { Encoder = System.Text.Encodings.Web.JavaScriptEncoder.UnsafeRelaxedJsonEscaping }));
Console.WriteLine($"trace: {trace.Count} scenes -> {args[1]}");
return 0;
}
Console.WriteLine("unknown command"); return 1;
```
- [ ] **Step 6: Run the C# test — expect PASS**, then cross-check the CLI trace
```bash
dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~TraceDiffTests
dotnet run --project engine/Age.Cli -- trace build/cs-trace.json
py -3.11 -X utf8 -c "import json;a=json.load(open('build/vm0-trace.json',encoding='utf-8'));b=json.load(open('build/cs-trace.json',encoding='utf-8'));print('identical' if a==b else 'DIFFER: '+str([k for k in a if a[k]!=b.get(k)][:5]))"
```
Expected: test passes; the Python cross-check prints `identical`.
- [ ] **Step 7: Full solution test + commit**
```bash
dotnet test engine/AgeEngine.sln
git add engine/Age.Cli/Program.cs engine/Age.Engine.Tests/TraceDiffTests.cs tools/vm0.py
git commit -m "feat(a1): CLI (run/trace) + vm0.py --trace + per-scene differential test (A1 green)"
```
---
## Self-Review
**Spec coverage:**
- Standalone .NET 8 solution (Task 1) ✓ · three seams + neutral `Script` contract (Model in Task 1, seam rule in Global Constraints, VM refs only Model+Hosting in Task 4) ✓ · SYS4 front-end parser+codec+opcode-table (Tasks 23, Task 1) ✓ · `IHost`/`CaptureHost` (Task 4) ✓ · faithful vm0 handlers incl. pointer semantics, emit-cap, halt reasons, PyDiv/PyMod (Task 4) ✓ · integer-parity + cp932 registration (Task 4, Task 2) ✓ · per-scene trace diff + RECOVER + parser/codec-parity tests (Tasks 3,4,5) ✓ · reads `build/opcodes.json` via `Paths` (Task 1) ✓ · `vm0.py --trace` addition (Task 5) ✓.
- Codec-parity note: the offset-only trace can't catch a decode bug; `Sys4LoaderTests` asserts `Strings.Count` and the ASCII codec test covers the codec — Japanese decode is exercised transitively (MENU strings) and by the trace run loading every scene without codec exceptions. (Acceptable; a byte-for-byte Japanese golden could be added if a divergence appears.)
**Placeholder scan:** none — every code step is complete; the only "investigate" is Task 5 Step 4, which is the differential-debugging loop (inherent) and points at `vm0.py --scene` for each named divergence.
**Type consistency:** `OpcodeTable`, `Script`, `Operand(Type,Value)`, `Instruction(Offset,Opcode,Args)`, `Sys4Loader.Load(path,table)`, `VirtualMachine(script,table,host,options?)` with `.Globals/.Emitted/.HaltReason/.Steps/.Run()`, `Paths.Scripts()/OpcodesJson/Build/Data1`, `IHost.ShowText/CallScript/OnStub` — used identically across Tasks 1→5. Trace JSON keys (`offsets`/`halt`/`steps`) match between `vm0.py --trace` (Task 5 Step 2) and both C# consumers (Task 5 Steps 1 & 5). Halt-reason format `LOOP:line@0x{off:x}×{count}` matches vm0's `{v:#x}` (0x-prefixed) exactly.