Files

gamer147 d2ec834fc3 feat(startup): add configurable game root

2026-07-28 23:05:25 -04:00

40 KiB

Raw Permalink Blame History

A1 — C# VM Core Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: A headless C# (.NET 8) reimplementation of the validated vm0.py AGE/SYS4 model that is per-scene trace-identical to the Python prototype and passes the RECOVER unit test.

Architecture: A .NET 8 class library Age.Engine with four seam-namespaces — Model (version-neutral Script contract), Vm (execution core, depends only on Model + Hosting), Sys4 (.BIN parser + cp932 codec + opcode-table loader), Hosting (IHost + headless CaptureHost) — plus a console Age.Cli and an Age.Engine.Tests xUnit project. Validated by a per-scene trace diff against tools/vm0.py --trace.

Tech Stack: C# / .NET 8, xUnit, System.Text.Json, System.Text.Encoding.CodePages (cp932). Python 3.11 for the vm0.py --trace reference.

Global Constraints

Work on a feature branch (e.g. feat/a1-csharp-vm); do not commit to main.
Build: dotnet build engine/AgeEngine.sln. Test: dotnet test engine/AgeEngine.sln (or --filter FullyQualifiedName~<Class>).
Seam rule: Age.Engine.Vm may reference only Age.Engine.Model and Age.Engine.Hosting — never Age.Engine.Sys4, never file I/O.
All VM numeric values are long. div/mod use the Python-parity helpers PyDiv/PyMod (floor-div; divisor-sign mod). u32 operands load as unsigned into long (no sign-extension), matching vm0.
Halt-reason strings must be byte-identical to vm0: "exit", "LOOP:line@0x{off:x}×{count}" (× = U+00D7), "STEP-LIMIT", "ret-underflow", "pc-out-of-range". Source files are UTF-8.
Data prerequisites (resolved via Paths, repo-relative, like tools/paths.py): build/opcodes.json (run tools/opcodes_build.py --build), ../extracted/DATA1/*.BIN, and — for the trace-diff — build/vm0-trace.json (produced by Task 5).
Exact format facts (from tools/sys4load.py, tools/vm0.py): magic prefix SYS4, header 0x3C bytes = 8 magic + 13×u32 (F0–F12), body dwords from 0x3C; code section [0..F8); instruction length 1 + 2*argc dwords; data_array_end shrink stops code at the lowest referenced string(type-2)/array(op 0x64 arg index 1) offset; operand types 0=imm 2=string 3=g-int 4=g-float 5=g-string 6=g-ptr 9=l-int 10=l-float 11=l-string 12=l-ptr; jcc sentinel 0xFFFFFFFF = fallthrough; ARRAY_OPCODE=0x64; EMIT_CAP=2, MAX_STEPS=2_000_000.

File Structure

engine/AgeEngine.sln
engine/Age.Engine/Age.Engine.csproj
engine/Age.Engine/Model/    Operand.cs Instruction.cs ScriptHeader.cs Script.cs OpcodeTable.cs
engine/Age.Engine/Sys4/     Paths.cs OpcodeTableJson.cs Sys4StringCodec.cs Sys4Loader.cs
engine/Age.Engine/Vm/       VmOptions.cs Frame.cs VirtualMachine.cs
engine/Age.Engine/Hosting/  IHost.cs CaptureHost.cs
engine/Age.Cli/Age.Cli.csproj  Program.cs
engine/Age.Engine.Tests/Age.Engine.Tests.csproj
   OpcodeTableTests.cs Sys4StringCodecTests.cs Sys4LoaderTests.cs RecoverTests.cs TraceDiffTests.cs
tools/vm0.py  (modify: add --trace)
.gitignore    (modify: ignore engine build output)

Task 1: Solution scaffold + Model + OpcodeTable

Files: Create the solution, Age.Engine/Model/*, Age.Engine/Sys4/Paths.cs, Age.Engine/Sys4/OpcodeTableJson.cs; Test Age.Engine.Tests/OpcodeTableTests.cs; Modify .gitignore.

Interfaces — Produces:

Age.Engine.Model.Operand(int Type, long Value) (readonly record struct)
Age.Engine.Model.Instruction(int Offset, int Opcode, IReadOnlyList<Operand> Args)
Age.Engine.Model.ScriptHeader(int LocalInt1, int LocalFloats, int LocalStrings1, int LocalInt2, int Unknown, int LocalStrings2)
Age.Engine.Model.Script { ScriptHeader Header; IReadOnlyList<Instruction> Instructions; IReadOnlyDictionary<int,int> IndexByOffset; IReadOnlyDictionary<int,string> Strings; string GetString(int) }
Age.Engine.Model.OpcodeTable { bool TryGet(int,out string,out int); string Label(int); int Argc(int); int Count }
Age.Engine.Sys4.OpcodeTableJson.Load(string path) -> OpcodeTable
Age.Engine.Sys4.Paths { string Repo, Extracted, Data1, GameDir, Build, OpcodesJson; Dictionary<string,string> Scripts() }
Step 1: Scaffold the solution

cd "S:/Game Hacking/Eushully/Himegari/age-reimpl"
dotnet new sln -n AgeEngine -o engine
dotnet new classlib -n Age.Engine -o engine/Age.Engine -f net8.0
dotnet new console  -n Age.Cli    -o engine/Age.Cli    -f net8.0
dotnet new xunit    -n Age.Engine.Tests -o engine/Age.Engine.Tests -f net8.0
rm -f engine/Age.Engine/Class1.cs engine/Age.Engine.Tests/UnitTest1.cs
dotnet sln engine/AgeEngine.sln add engine/Age.Engine engine/Age.Cli engine/Age.Engine.Tests
dotnet add engine/Age.Cli reference engine/Age.Engine
dotnet add engine/Age.Engine.Tests reference engine/Age.Engine
dotnet add engine/Age.Engine package System.Text.Encoding.CodePages

Step 2: Ignore .NET build output — append to .gitignore:

# .NET (engine/) build output
engine/**/bin/
engine/**/obj/
.vs/

Step 3: Write the failing test — engine/Age.Engine.Tests/OpcodeTableTests.cs:

using Age.Engine.Sys4;
using Xunit;

public class OpcodeTableTests
{
    [Fact]
    public void LoadsAll248FromJson()
    {
        var t = OpcodeTableJson.Load(Paths.OpcodesJson);
        Assert.Equal(248, t.Count);
        Assert.True(t.TryGet(0x55, out var label, out var argc));
        Assert.Equal("mov", label);
        Assert.Equal(2, argc);
        Assert.Equal("u0041BEB0", t.Label(0x90));
        Assert.Equal(7, t.Argc(0x90));
        Assert.Equal(-1, t.Argc(0x9999)); // absent -> -1
    }
}

Step 4: Run — expect FAIL (types missing / won't compile)

dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~OpcodeTableTests

Expected: build error — Paths/OpcodeTableJson do not exist.

Step 5: Implement Model + Paths + OpcodeTableJson

engine/Age.Engine/Model/Operand.cs:

namespace Age.Engine.Model;
public readonly record struct Operand(int Type, long Value);

engine/Age.Engine/Model/Instruction.cs:

namespace Age.Engine.Model;
public sealed record Instruction(int Offset, int Opcode, IReadOnlyList<Operand> Args);

engine/Age.Engine/Model/ScriptHeader.cs:

namespace Age.Engine.Model;
public sealed record ScriptHeader(
    int LocalInt1, int LocalFloats, int LocalStrings1,
    int LocalInt2, int Unknown, int LocalStrings2);

engine/Age.Engine/Model/Script.cs:

namespace Age.Engine.Model;
public sealed class Script
{
    public required ScriptHeader Header { get; init; }
    public required IReadOnlyList<Instruction> Instructions { get; init; }
    public required IReadOnlyDictionary<int, int> IndexByOffset { get; init; }
    public required IReadOnlyDictionary<int, string> Strings { get; init; }
    public string GetString(int offset) => Strings.TryGetValue(offset, out var s) ? s : "";
}

engine/Age.Engine/Model/OpcodeTable.cs:

namespace Age.Engine.Model;
public sealed class OpcodeTable
{
    private readonly IReadOnlyDictionary<int, (string Label, int Argc)> _t;
    public OpcodeTable(IReadOnlyDictionary<int, (string, int)> t) => _t = t;
    public int Count => _t.Count;
    public bool TryGet(int op, out string label, out int argc)
    {
        if (_t.TryGetValue(op, out var e)) { label = e.Label; argc = e.Argc; return true; }
        label = ""; argc = -1; return false;
    }
    public string Label(int op) => _t.TryGetValue(op, out var e) ? e.Label : "";
    public int Argc(int op) => _t.TryGetValue(op, out var e) ? e.Argc : -1;
}

engine/Age.Engine/Sys4/Paths.cs:

namespace Age.Engine.Sys4;
public static class Paths
{
    public static string Repo { get; } = FindRepo();
    public static string Workspace => Directory.GetParent(Repo)!.FullName;
    public static string Extracted => Path.Combine(Workspace, "extracted");
    public static string Data1 => Path.Combine(Extracted, "DATA1");
    public static string GameDir => Path.Combine(Workspace, "Himegari_Game");
    public static string Build => Path.Combine(Repo, "build");
    public static string OpcodesJson => Path.Combine(Build, "opcodes.json");

    private static string FindRepo()
    {
        var d = new DirectoryInfo(AppContext.BaseDirectory);
        while (d != null && d.Name != "age-reimpl") d = d.Parent;
        if (d == null) throw new DirectoryNotFoundException(
            "age-reimpl root not found above " + AppContext.BaseDirectory);
        return d.FullName;
    }

    /// name(UPPER).BIN -> path; game-dir loose overrides shadow extracted/DATA1 (mirrors paths.scripts()).
    public static Dictionary<string, string> Scripts()
    {
        var d = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
        if (Directory.Exists(Data1))
            foreach (var p in Directory.EnumerateFiles(Data1, "*.BIN"))
                d[Path.GetFileName(p).ToUpperInvariant()] = p;
        if (Directory.Exists(GameDir))
            foreach (var p in Directory.EnumerateFiles(GameDir, "*.BIN"))
                d[Path.GetFileName(p).ToUpperInvariant()] = p;
        return d;
    }
}

engine/Age.Engine/Sys4/OpcodeTableJson.cs:

using System.Text.Json;
using Age.Engine.Model;
namespace Age.Engine.Sys4;
public static class OpcodeTableJson
{
    public static OpcodeTable Load(string path)
    {
        using var doc = JsonDocument.Parse(File.ReadAllText(path));
        var dict = new Dictionary<int, (string, int)>();
        foreach (var e in doc.RootElement.GetProperty("opcodes").EnumerateArray())
        {
            int op = Convert.ToInt32(e.GetProperty("op").GetString(), 16);
            string label = e.GetProperty("label").GetString() ?? "";
            int argc = e.GetProperty("argc").GetInt32();
            dict[op] = (label, argc);
        }
        return new OpcodeTable(dict);
    }
}

Step 6: Run — expect PASS

dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~OpcodeTableTests

Expected: 1 passed. (Prereq: tools/opcodes_build.py --build has produced build/opcodes.json.)

Step 7: Commit

git add engine .gitignore
git commit -m "feat(a1): .NET 8 solution scaffold + Model + OpcodeTable loader"

Task 2: SYS4 string codec

Files: Create Age.Engine/Sys4/Sys4StringCodec.cs; Test Age.Engine.Tests/Sys4StringCodecTests.cs.

Interfaces — Produces: Sys4StringCodec.Decode(IReadOnlyList<uint> dwords, int start, int limit = 4096) -> (string? Text, int NDwords) — XOR-0xFFFFFFFF per little-endian dword, cut at first NUL, strict cp932 validation (returns (null,0) when not a clean string), else decoded text + dword length including the NUL dword.

Step 1: Write the failing test — engine/Age.Engine.Tests/Sys4StringCodecTests.cs:

using Age.Engine.Sys4;
using Xunit;

public class Sys4StringCodecTests
{
    [Fact]
    public void DecodesXorFfCp932Ascii()
    {
        // "AB\0\0" little-endian = 0x00004241, stored XOR 0xFFFFFFFF
        uint[] dw = { 0x00004241u ^ 0xFFFFFFFFu };
        var (text, nd) = Sys4StringCodec.Decode(dw, 0);
        Assert.Equal("AB", text);
        Assert.Equal(1, nd);
    }

    [Fact]
    public void RejectsNonString()
    {
        // 0x00000000 XOR-decodes to 0xFFFFFFFF bytes (0xFF,0xFF,0xFF,0xFF) -> no NUL, invalid
        uint[] dw = { 0x00000000u };
        var (text, _) = Sys4StringCodec.Decode(dw, 0);
        Assert.Null(text);
    }
}

Step 2: Run — expect FAIL (Sys4StringCodec missing)

dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~Sys4StringCodecTests

Step 3: Implement — engine/Age.Engine/Sys4/Sys4StringCodec.cs:

using System.Text;
namespace Age.Engine.Sys4;
public static class Sys4StringCodec
{
    static Sys4StringCodec() => Encoding.RegisterProvider(CodePagesEncodingProvider.Instance);
    private static readonly Encoding Cp932 = Encoding.GetEncoding(932);

    public static (string? Text, int NDwords) Decode(IReadOnlyList<uint> dwords, int start, int limit = 4096)
    {
        int n = dwords.Count, end = Math.Min(start + limit, n);
        var raw = new List<byte>();
        bool sawNul = false;
        for (int j = start; j < end; j++)
        {
            uint x = dwords[j] ^ 0xFFFFFFFFu;
            raw.Add((byte)(x & 0xFF)); raw.Add((byte)((x >> 8) & 0xFF));
            raw.Add((byte)((x >> 16) & 0xFF)); raw.Add((byte)((x >> 24) & 0xFF));
            if (raw[^1] == 0 || raw[^2] == 0 || raw[^3] == 0 || raw[^4] == 0) { sawNul = true; break; }
        }
        if (!sawNul) return (null, 0);
        int nul = raw.IndexOf(0);
        byte[] s = raw.GetRange(0, nul < 0 ? raw.Count : nul).ToArray();
        if (s.Length < 1) return ("", 1);
        int i = 0, chars = 0;
        while (i < s.Length)
        {
            byte b = s[i];
            if (b >= 0x20 && b <= 0x7E) { i++; chars++; }
            else if ((b >= 0x81 && b <= 0x9F) || (b >= 0xE0 && b <= 0xEA))
            {
                if (i + 1 < s.Length && s[i + 1] >= 0x40 && s[i + 1] <= 0xFC && s[i + 1] != 0x7F) { i += 2; chars++; }
                else return (null, 0);
            }
            else return (null, 0);
        }
        if (chars < 1) return (null, 0);
        string text;
        try { text = Cp932.GetString(s); } catch { return (null, 0); }
        return (text, (s.Length / 4) + 1);
    }
}

Step 4: Run — expect PASS

dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~Sys4StringCodecTests

Step 5: Commit

git add engine/Age.Engine/Sys4/Sys4StringCodec.cs engine/Age.Engine.Tests/Sys4StringCodecTests.cs
git commit -m "feat(a1): SYS4 XOR-FF cp932 string codec"

Task 3: SYS4 container parser (`Sys4Loader`)

Files: Create Age.Engine/Sys4/Sys4Loader.cs; Test Age.Engine.Tests/Sys4LoaderTests.cs.

Interfaces — Consumes: OpcodeTable, Sys4StringCodec, Script. Produces: Sys4Loader.Load(string path, OpcodeTable table) -> Script; Sys4Loader.Parse(byte[] data, OpcodeTable table, string name = "") -> Script.

Step 1: Write the failing test — engine/Age.Engine.Tests/Sys4LoaderTests.cs:

using Age.Engine.Sys4;
using Xunit;

public class Sys4LoaderTests
{
    [Fact]
    public void ParsesMenuBinLikeSys4load()
    {
        var table = OpcodeTableJson.Load(Paths.OpcodesJson);
        var s = Sys4Loader.Load(Path.Combine(Paths.Data1, "MENU.BIN"), table);
        // header F0..F5 from `sys4load MENU.BIN --summary`
        Assert.Equal(0x17, s.Header.LocalInt1);
        Assert.Equal(1, s.Header.LocalFloats);
        Assert.Equal(1, s.Header.LocalStrings1);
        Assert.Equal(2, s.Header.LocalInt2);
        Assert.Equal(1, s.Header.Unknown);
        Assert.Equal(1, s.Header.LocalStrings2);
        Assert.Equal(148, s.Instructions.Count);   // sys4load: 148 instructions
        Assert.Equal(3, s.Strings.Count);           // sys4load: 3 inline strings
        // every instruction offset is indexed
        Assert.All(s.Instructions, ins => Assert.True(s.IndexByOffset.ContainsKey(ins.Offset)));
    }
}

Step 2: Run — expect FAIL (Sys4Loader missing)

dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~Sys4LoaderTests

Step 3: Implement — engine/Age.Engine/Sys4/Sys4Loader.cs:

using Age.Engine.Model;
namespace Age.Engine.Sys4;
public static class Sys4Loader
{
    private const int HeaderSize = 0x3C, BodyOff = 0x3C, NumFields = 13, ArrayOpcode = 0x64;

    public static Script Load(string path, OpcodeTable table)
        => Parse(File.ReadAllBytes(path), table, Path.GetFileName(path));

    public static Script Parse(byte[] data, OpcodeTable table, string name = "")
    {
        if (data.Length < HeaderSize) throw new InvalidDataException($"{name}: too small");
        if (!(data[0] == (byte)'S' && data[1] == (byte)'Y' && data[2] == (byte)'S' && data[3] == (byte)'4'))
            throw new InvalidDataException($"{name}: bad magic");
        if (data.Length % 4 != 0) throw new InvalidDataException($"{name}: not dword-aligned");

        var fields = new int[NumFields];
        for (int k = 0; k < NumFields; k++) fields[k] = BitConverter.ToInt32(data, 8 + k * 4);
        int nbody = (data.Length - BodyOff) / 4;
        var dw = new uint[nbody];
        for (int k = 0; k < nbody; k++) dw[k] = BitConverter.ToUInt32(data, BodyOff + k * 4);

        var header = new ScriptHeader(fields[0], fields[1], fields[2], fields[3], fields[4], fields[5]);
        var (instrs, idxByOff, strings) = DecodeCode(dw, fields, nbody, table);
        return new Script { Header = header, Instructions = instrs, IndexByOffset = idxByOff, Strings = strings };
    }

    private static (List<Instruction>, Dictionary<int, int>, Dictionary<int, string>)
        DecodeCode(uint[] dw, int[] fields, int nbody, OpcodeTable table)
    {
        int codeEnd = fields[8];                       // F8; shrinks to first inline string/array offset
        var instrs = new List<Instruction>();
        var idx = new Dictionary<int, int>();
        var strings = new Dictionary<int, string>();
        int i = 0;
        while (i < codeEnd)
        {
            int op = (int)dw[i];
            int argc = table.Argc(op);
            if (argc < 0) { idx[i] = instrs.Count; instrs.Add(new Instruction(i, op, Array.Empty<Operand>())); break; }
            int baseI = i + 1;
            if (baseI + 2 * argc > codeEnd) { idx[i] = instrs.Count; instrs.Add(new Instruction(i, op, Array.Empty<Operand>())); break; }
            var args = new Operand[argc];
            for (int a = 0; a < argc; a++)
            {
                int atype = (int)dw[baseI + 2 * a];
                long aval = dw[baseI + 2 * a + 1];
                args[a] = new Operand(atype, aval);
                if (atype == 2 && aval >= 0 && aval < nbody)
                {
                    if (aval < codeEnd) codeEnd = (int)aval;
                    if (!strings.ContainsKey((int)aval))
                    {
                        var (text, _) = Sys4StringCodec.Decode(dw, (int)aval);
                        if (text != null) strings[(int)aval] = text;
                    }
                }
                else if (op == ArrayOpcode && a == 1 && aval >= 0 && aval < nbody)
                {
                    if (aval < codeEnd) codeEnd = (int)aval;
                }
            }
            idx[i] = instrs.Count;
            instrs.Add(new Instruction(i, op, args));
            i = baseI + 2 * argc;
        }
        return (instrs, idx, strings);
    }
}

Step 4: Run — expect PASS

dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~Sys4LoaderTests

Expected: 1 passed (148 instructions, 3 strings, header matches). If instruction count differs, the data_array_end shrink or argc walk diverged — compare against py -3.11 -X utf8 tools/sys4load.py ../extracted/DATA1/MENU.BIN --summary.

Step 5: Commit

git add engine/Age.Engine/Sys4/Sys4Loader.cs engine/Age.Engine.Tests/Sys4LoaderTests.cs
git commit -m "feat(a1): SYS4 container parser (header + code decode + data_array_end shrink)"

Task 4: VM core + IHost/CaptureHost + RECOVER

Files: Create Age.Engine/Hosting/IHost.cs, Age.Engine/Hosting/CaptureHost.cs, Age.Engine/Vm/VmOptions.cs, Age.Engine/Vm/Frame.cs, Age.Engine/Vm/VirtualMachine.cs; Test Age.Engine.Tests/RecoverTests.cs.

Interfaces — Consumes: Script, OpcodeTable, Sys4Loader. Produces:

Age.Engine.Hosting.IHost { void ShowText(int,string); void CallScript(long); void OnStub(int); }, CaptureHost : IHost
Age.Engine.Vm.VmOptions(int EmitCap = 2, long MaxSteps = 2_000_000)
Age.Engine.Vm.VirtualMachine(Script, OpcodeTable, IHost, VmOptions? = null) with public Dictionary<int,long> Globals, Dictionary<int,string> GlobalStrings, List<(int Offset,string Text)> Emitted, string? HaltReason, long Steps, and void Run(int entryOffset = 0).
Step 1: Write the failing test — engine/Age.Engine.Tests/RecoverTests.cs:

using Age.Engine.Hosting;
using Age.Engine.Sys4;
using Age.Engine.Vm;
using Xunit;

public class RecoverTests
{
    private static long G(VirtualMachine vm, int k) => vm.Globals.TryGetValue(k, out var v) ? v : 0;

    [Fact]
    public void RecoverUnitTestPasses()
    {
        var table = OpcodeTableJson.Load(Paths.OpcodesJson);
        var script = Sys4Loader.Load(Path.Combine(Paths.Data1, "RECOVER.BIN"), table);
        var vm = new VirtualMachine(script, table, new CaptureHost());

        int unit = 0;
        vm.Globals[0x152616] = unit;
        int A = 0x4e11b, B = 0x4e085, C = 0x52383, E = 0x52f3b, F = 0x5295f, FL = 0xaacb4;
        for (int k = 0; k < 3; k++) vm.Globals[A + unit * 14 + (11 + k)] = 100 + k;
        vm.Globals[C + unit * 30 + 5] = 7; vm.Globals[FL + 5] = 1; vm.Globals[E + unit * 30 + 5] = 42;
        vm.Globals[C + unit * 30 + 6] = 0; vm.Globals[FL + 6] = 1; vm.Globals[E + unit * 30 + 6] = 99;
        vm.Globals[C + unit * 30 + 7] = 3; vm.Globals[FL + 7] = 0; vm.Globals[E + unit * 30 + 7] = 88;
        vm.Run();

        Assert.Equal(100, G(vm, B + unit * 3 + 0));
        Assert.Equal(101, G(vm, B + unit * 3 + 1));
        Assert.Equal(102, G(vm, B + unit * 3 + 2));
        Assert.Equal(42, G(vm, C + unit * 30 + 5));
        Assert.Equal(-1, G(vm, F + unit * 30 + 5));
        Assert.Equal(0, G(vm, C + unit * 30 + 6));
        Assert.Equal(3, G(vm, C + unit * 30 + 7));
    }
}

Step 2: Run — expect FAIL (VM types missing)

dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~RecoverTests

Step 3: Implement hosting + VM

engine/Age.Engine/Hosting/IHost.cs:

namespace Age.Engine.Hosting;
public interface IHost
{
    void ShowText(int offset, string text);
    void CallScript(long id);
    void OnStub(int opcode);
}

engine/Age.Engine/Hosting/CaptureHost.cs:

namespace Age.Engine.Hosting;
public sealed class CaptureHost : IHost
{
    public List<(int Offset, string Text)> Emitted { get; } = new();
    public int CallScriptCount { get; private set; }
    public Dictionary<int, int> Stubs { get; } = new();
    public void ShowText(int offset, string text) => Emitted.Add((offset, text));
    public void CallScript(long id) => CallScriptCount++;
    public void OnStub(int opcode) { Stubs.TryGetValue(opcode, out var c); Stubs[opcode] = c + 1; }
}

engine/Age.Engine/Vm/VmOptions.cs:

namespace Age.Engine.Vm;
public sealed record VmOptions(int EmitCap = 2, long MaxSteps = 2_000_000);

engine/Age.Engine/Vm/Frame.cs:

namespace Age.Engine.Vm;
public sealed class Frame
{
    public Dictionary<int, long> I = new();   // local-int
    public Dictionary<int, long> F = new();   // local-float (raw)
    public Dictionary<int, string> S = new(); // local-string
    public Dictionary<int, long> P = new();   // local-ptr (holds a global address)
}

engine/Age.Engine/Vm/VirtualMachine.cs:

using Age.Engine.Hosting;
using Age.Engine.Model;
namespace Age.Engine.Vm;

public sealed class VirtualMachine
{
    private const long NoJump = 0xFFFFFFFF;
    private const int HALT = int.MinValue;
    private const int T_IMM = 0, T_STR = 2, T_GINT = 3, T_GFLOAT = 4, T_GSTR = 5, T_GPTR = 6,
                      T_LINT = 9, T_LFLOAT = 10, T_LSTR = 11, T_LPTR = 12;

    private readonly Script _s;
    private readonly OpcodeTable _t;
    private readonly IHost _host;
    private readonly VmOptions _o;
    private readonly Frame _fr = new();
    private readonly List<int> _callstack = new();
    private readonly Dictionary<int, int> _emitSeen = new();

    public Dictionary<int, long> Globals { get; } = new();
    public Dictionary<int, string> GlobalStrings { get; } = new();
    public List<(int Offset, string Text)> Emitted { get; } = new();
    public string? HaltReason { get; private set; }
    public long Steps { get; private set; }

    public VirtualMachine(Script s, OpcodeTable t, IHost host, VmOptions? o = null)
    { _s = s; _t = t; _host = host; _o = o ?? new VmOptions(); }

    private static long Gi(Dictionary<int, long> d, int k) => d.TryGetValue(k, out var v) ? v : 0;
    private static string Gs(Dictionary<int, string> d, int k) => d.TryGetValue(k, out var v) ? v : "";
    private static long PyDiv(long a, long b) { if (b == 0) return 0; long q = a / b, r = a % b; if (r != 0 && (r < 0) != (b < 0)) q--; return q; }
    private static long PyMod(long a, long b) { if (b == 0) return 0; long r = a % b; if (r != 0 && (r < 0) != (b < 0)) r += b; return r; }

    private static bool IsStr(Operand o) => o.Type == T_STR || o.Type == T_GSTR || o.Type == T_LSTR;

    private long Read(Operand op) => op.Type switch
    {
        T_IMM => op.Value,
        T_GINT or T_GFLOAT => Gi(Globals, (int)op.Value),
        T_GPTR => Gi(Globals, (int)Gi(Globals, (int)op.Value)),
        T_LINT => Gi(_fr.I, (int)op.Value),
        T_LFLOAT => Gi(_fr.F, (int)op.Value),
        T_LPTR => Gi(Globals, (int)Gi(_fr.P, (int)op.Value)),
        _ => op.Value,
    };

    private void Write(Operand op, long val)
    {
        switch (op.Type)
        {
            case T_GINT: case T_GFLOAT: Globals[(int)op.Value] = val; break;
            case T_GPTR: Globals[(int)Gi(Globals, (int)op.Value)] = val; break;
            case T_LINT: _fr.I[(int)op.Value] = val; break;
            case T_LFLOAT: _fr.F[(int)op.Value] = val; break;
            case T_LPTR: Globals[(int)Gi(_fr.P, (int)op.Value)] = val; break;
        }
    }

    private string ReadStr(Operand op) => op.Type switch
    {
        T_STR => _s.GetString((int)op.Value),
        T_GSTR => Gs(GlobalStrings, (int)op.Value),
        T_LSTR => Gs(_fr.S, (int)op.Value),
        _ => "",
    };

    private void WriteStr(Operand op, string val)
    {
        switch (op.Type)
        {
            case T_GSTR: GlobalStrings[(int)op.Value] = val; break;
            case T_LSTR: _fr.S[(int)op.Value] = val; break;
        }
    }

    private long BaseAddr(Operand op) => op.Type switch
    {
        T_IMM or T_GINT or T_GFLOAT or T_GSTR or T_GPTR => op.Value,
        T_LINT => Gi(_fr.I, (int)op.Value),
        T_LPTR => Gi(_fr.P, (int)op.Value),
        _ => op.Value,
    };

    private void LookupStore(Operand dst, long addr)
    {
        switch (dst.Type)
        {
            case T_LPTR: _fr.P[(int)dst.Value] = addr; break;
            case T_GPTR: Globals[(int)dst.Value] = addr; break;
            default: Write(dst, Gi(Globals, (int)addr)); break;
        }
    }

    public void Run(int entryOffset = 0)
    {
        int pc = _s.IndexByOffset.TryGetValue(entryOffset, out var idx) ? idx : 0;
        while (pc >= 0 && pc < _s.Instructions.Count)
        {
            if (Steps >= _o.MaxSteps) { HaltReason ??= "STEP-LIMIT"; return; }
            Steps++;
            int next = Step(_s.Instructions[pc], pc);
            if (next == HALT) return;
            pc = next;
        }
        HaltReason ??= "pc-out-of-range";
    }

    private int Step(Instruction ins, int pc)
    {
        int op = ins.Opcode;
        var a = ins.Args;
        switch (_t.Label(op))
        {
            case "add": Write(a[0], Read(a[1]) + Read(a[2])); return pc + 1;
            case "sub": Write(a[0], Read(a[1]) - Read(a[2])); return pc + 1;
            case "mul": Write(a[0], Read(a[1]) * Read(a[2])); return pc + 1;
            case "div": Write(a[0], PyDiv(Read(a[1]), Read(a[2]))); return pc + 1;
            case "mod": Write(a[0], PyMod(Read(a[1]), Read(a[2]))); return pc + 1;
            case "and": Write(a[0], Read(a[1]) & Read(a[2])); return pc + 1;
            case "or":  Write(a[0], Read(a[1]) | Read(a[2])); return pc + 1;
            case "sar": Write(a[0], Read(a[1]) >> (int)(Read(a[2]) & 31)); return pc + 1;
            case "shl": Write(a[0], Read(a[1]) << (int)(Read(a[2]) & 31)); return pc + 1;
            case "eq":  Write(a[0], Read(a[1]) == Read(a[2]) ? 1 : 0); return pc + 1;
            case "ne":  Write(a[0], Read(a[1]) != Read(a[2]) ? 1 : 0); return pc + 1;
            case "lt":  Write(a[0], Read(a[1]) <  Read(a[2]) ? 1 : 0); return pc + 1;
            case "lte": Write(a[0], Read(a[1]) <= Read(a[2]) ? 1 : 0); return pc + 1;
            case "gr":  Write(a[0], Read(a[1]) >  Read(a[2]) ? 1 : 0); return pc + 1;
            case "gre": Write(a[0], Read(a[1]) >= Read(a[2]) ? 1 : 0); return pc + 1;
            case "mov":
            case "set-string":
                if (IsStr(a[0]) || IsStr(a[1])) WriteStr(a[0], ReadStr(a[1]));
                else Write(a[0], Read(a[1]));
                return pc + 1;
            case "lookup-array":
                LookupStore(a[0], BaseAddr(a[1]) + Read(a[2])); return pc + 1;
            case "lookup-array-2d":
                LookupStore(a[0], BaseAddr(a[1]) + Read(a[2]) * Read(a[3]) + Read(a[4])); return pc + 1;
            case "bit-set": Write(a[0], Read(a[0]) | Read(a[1])); return pc + 1;
            case "bit-reset": Write(a[0], Read(a[0]) & ~Read(a[1])); return pc + 1;
            case "check-bit": Write(a[0], (Read(a[1]) >> (int)(Read(a[2]) & 31)) & 1); return pc + 1;
            case "copy-to-global": Write(a[0], Read(a[1])); return pc + 1;
            case "jmp": return _s.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1);
            case "call": _callstack.Add(pc + 1); return _s.IndexByOffset.GetValueOrDefault((int)a[0].Value, pc + 1);
            case "ret":
                if (_callstack.Count > 0) { int r = _callstack[^1]; _callstack.RemoveAt(_callstack.Count - 1); return r; }
                HaltReason = "ret-underflow"; return HALT;
            case "jcc":
            {
                long tgt = Read(a[0]) != 0 ? a[1].Value : a[2].Value;
                return tgt == NoJump ? pc + 1 : _s.IndexByOffset.GetValueOrDefault((int)tgt, pc + 1);
            }
            case "exit":
            case "exit-script": HaltReason = "exit"; return HALT;
            case "call-script": _host.CallScript(a.Count > 0 ? Read(a[0]) : 0); return pc + 1;
            case "show-text":
                foreach (var o in a)
                {
                    if (o.Type != T_STR) continue;
                    int off = (int)o.Value;
                    _emitSeen.TryGetValue(off, out var c); c++; _emitSeen[off] = c;
                    if (c > _o.EmitCap) { HaltReason = $"LOOP:line@0x{off:x}×{c}"; return HALT; }
                    string text = _s.GetString(off);
                    Emitted.Add((off, text));
                    _host.ShowText(off, text);
                }
                return pc + 1;
            case "end-text-line": case "wait-for-input": case "set-font":
            case "comment": case "display-furigana": case "dev_ukn":
                return pc + 1;
            default:
                _host.OnStub(op); return pc + 1;
        }
    }
}

Step 4: Run — expect PASS

dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~RecoverTests

Expected: RECOVER passes (all 7 asserts). If a pointer/array assert fails, compare against py -3.11 -X utf8 tools/vm0.py --test.

Step 5: Commit

git add engine/Age.Engine/Hosting engine/Age.Engine/Vm engine/Age.Engine.Tests/RecoverTests.cs
git commit -m "feat(a1): VM core (operands+pointer semantics, handlers, emit-cap) + RECOVER"

Task 5: `Age.Cli` + `vm0.py --trace` + per-scene trace diff

Files: Modify tools/vm0.py (add --trace); Create engine/Age.Cli/Program.cs; Test Age.Engine.Tests/TraceDiffTests.cs.

Interfaces — Consumes: Paths.Scripts(), Sys4Loader, OpcodeTableJson, VirtualMachine, CaptureHost. Produces (shared trace contract): JSON object { "<SCENE>.BIN": { "offsets": [int...], "halt": "<reason>", "steps": int }, ... } over the sorted set of names matching ^S[CP]\d{4}\.BIN$ from Paths.Scripts(). Both vm0.py --trace <out> and Age.Cli trace <out> emit exactly this.

Step 1: Write the failing test — engine/Age.Engine.Tests/TraceDiffTests.cs:

using System.Text.Json;
using System.Text.RegularExpressions;
using Age.Engine.Hosting;
using Age.Engine.Sys4;
using Age.Engine.Vm;
using Xunit;

public class TraceDiffTests
{
    private static readonly Regex Scene = new(@"^S[CP]\d{4}\.BIN$");

    [Fact]
    public void CsTraceMatchesVm0PerScene()
    {
        string refPath = Path.Combine(Paths.Build, "vm0-trace.json");
        Assert.True(File.Exists(refPath),
            "prerequisite: run `py -3.11 -X utf8 tools/vm0.py --trace build/vm0-trace.json`");

        using var doc = JsonDocument.Parse(File.ReadAllText(refPath));
        var expected = doc.RootElement;
        var table = OpcodeTableJson.Load(Paths.OpcodesJson);
        var scripts = Paths.Scripts();

        var mismatches = new List<string>();
        foreach (var name in scripts.Keys.Where(n => Scene.IsMatch(n)).OrderBy(n => n, StringComparer.Ordinal))
        {
            var script = Sys4Loader.Load(scripts[name], table);
            var vm = new VirtualMachine(script, table, new CaptureHost());
            vm.Run();
            var offsets = vm.Emitted.Select(e => e.Offset).ToArray();

            if (!expected.TryGetProperty(name, out var exp)) { mismatches.Add($"{name}: absent in vm0 trace"); continue; }
            var expOffsets = exp.GetProperty("offsets").EnumerateArray().Select(x => x.GetInt32()).ToArray();
            string expHalt = exp.GetProperty("halt").GetString() ?? "";
            long expSteps = exp.GetProperty("steps").GetInt64();

            if (!offsets.SequenceEqual(expOffsets))
                mismatches.Add($"{name}: offsets differ (cs {offsets.Length} vs vm0 {expOffsets.Length}; first diff at {FirstDiff(offsets, expOffsets)})");
            else if (vm.HaltReason != expHalt) mismatches.Add($"{name}: halt cs='{vm.HaltReason}' vs vm0='{expHalt}'");
            else if (vm.Steps != expSteps) mismatches.Add($"{name}: steps cs={vm.Steps} vs vm0={expSteps}");
        }
        Assert.True(mismatches.Count == 0, "scene mismatches:\n" + string.Join("\n", mismatches.Take(20)));
    }

    private static int FirstDiff(int[] a, int[] b)
    {
        int n = Math.Min(a.Length, b.Length);
        for (int i = 0; i < n; i++) if (a[i] != b[i]) return i;
        return n;
    }
}

Step 2: Add --trace to tools/vm0.py — insert this function before def main( and a branch in main:

def run_trace(out_path):
    """Dump per-scene emitted show-text offsets + halt + steps for the differential test."""
    oracle = load_oracle()
    scripts = paths.scripts()
    names = sorted(n for n in scripts if SCENE_RE.match(n))
    trace = {}
    for name in names:
        r = run_scene(name, scripts[name], oracle)
        vm = r["vm"]
        trace[name] = {"offsets": [off for off, _ in vm.text],
                       "halt": vm.halt_reason, "steps": vm.steps}
    Path(out_path).write_text(json.dumps(trace, ensure_ascii=False), encoding="utf-8")
    print(f"trace: {len(trace)} scenes -> {out_path}")
    return 0

In main, add the branch (after the --sweep branch):

    if argv[0] == "--trace":
        return run_trace(argv[1])

Step 3: Generate the reference trace and confirm it matches the sweep

cd "S:/Game Hacking/Eushully/Himegari/age-reimpl"
py -3.11 -X utf8 tools/vm0.py --trace build/vm0-trace.json
py -3.11 -X utf8 -c "import json; d=json.load(open('build/vm0-trace.json',encoding='utf-8')); print(len(d),'scenes')"

Expected: trace: N scenes -> build/vm0-trace.json and the count prints (≈297 SC/SP names).

Step 4: Run the C# test — expect FAIL (Program/CLI not needed yet, but the test drives the C# VM directly)

dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~TraceDiffTests

Expected: FAIL only if a scene diverges — the message names the scene + first differing offset index. Investigate each divergence against tools/vm0.py --scene <NAME>; fix the VM/parser until the trace is identical. (If it passes first try, even better.)

Step 5: Implement the CLI — engine/Age.Cli/Program.cs:

using System.Text.Json;
using System.Text.RegularExpressions;
using Age.Engine.Hosting;
using Age.Engine.Sys4;
using Age.Engine.Vm;

var table = OpcodeTableJson.Load(Paths.OpcodesJson);

if (args.Length == 0) { Console.WriteLine("usage: run <file> | trace <out.json>"); return 1; }

if (args[0] == "run")
{
    var script = Sys4Loader.Load(args[1], table);
    var vm = new VirtualMachine(script, table, new CaptureHost());
    vm.Run();
    Console.WriteLine($"{Path.GetFileName(args[1])}: {vm.Steps} steps, {vm.Emitted.Count} show-text (halt: {vm.HaltReason})");
    foreach (var (off, text) in vm.Emitted.Take(20)) Console.WriteLine($"  [{off:x}] {text}");
    return 0;
}

if (args[0] == "trace")
{
    var scene = new Regex(@"^S[CP]\d{4}\.BIN$");
    var scripts = Paths.Scripts();
    var trace = new SortedDictionary<string, object>(StringComparer.Ordinal);
    foreach (var name in scripts.Keys.Where(n => scene.IsMatch(n)).OrderBy(n => n, StringComparer.Ordinal))
    {
        var vm = new VirtualMachine(Sys4Loader.Load(scripts[name], table), table, new CaptureHost());
        vm.Run();
        trace[name] = new { offsets = vm.Emitted.Select(e => e.Offset).ToArray(), halt = vm.HaltReason, steps = vm.Steps };
    }
    File.WriteAllText(args[1], JsonSerializer.Serialize(trace, new JsonSerializerOptions { Encoder = System.Text.Encodings.Web.JavaScriptEncoder.UnsafeRelaxedJsonEscaping }));
    Console.WriteLine($"trace: {trace.Count} scenes -> {args[1]}");
    return 0;
}
Console.WriteLine("unknown command"); return 1;

Step 6: Run the C# test — expect PASS, then cross-check the CLI trace

dotnet test engine/Age.Engine.Tests --filter FullyQualifiedName~TraceDiffTests
dotnet run --project engine/Age.Cli -- trace build/cs-trace.json
py -3.11 -X utf8 -c "import json;a=json.load(open('build/vm0-trace.json',encoding='utf-8'));b=json.load(open('build/cs-trace.json',encoding='utf-8'));print('identical' if a==b else 'DIFFER: '+str([k for k in a if a[k]!=b.get(k)][:5]))"

Expected: test passes; the Python cross-check prints identical.

Step 7: Full solution test + commit

dotnet test engine/AgeEngine.sln
git add engine/Age.Cli/Program.cs engine/Age.Engine.Tests/TraceDiffTests.cs tools/vm0.py
git commit -m "feat(a1): CLI (run/trace) + vm0.py --trace + per-scene differential test (A1 green)"

Self-Review

Spec coverage:

Standalone .NET 8 solution (Task 1) ✓ · three seams + neutral Script contract (Model in Task 1, seam rule in Global Constraints, VM refs only Model+Hosting in Task 4) ✓ · SYS4 front-end parser+codec+opcode-table (Tasks 2–3, Task 1) ✓ · IHost/CaptureHost (Task 4) ✓ · faithful vm0 handlers incl. pointer semantics, emit-cap, halt reasons, PyDiv/PyMod (Task 4) ✓ · integer-parity + cp932 registration (Task 4, Task 2) ✓ · per-scene trace diff + RECOVER + parser/codec-parity tests (Tasks 3,4,5) ✓ · reads build/opcodes.json via Paths (Task 1) ✓ · vm0.py --trace addition (Task 5) ✓.
Codec-parity note: the offset-only trace can't catch a decode bug; Sys4LoaderTests asserts Strings.Count and the ASCII codec test covers the codec — Japanese decode is exercised transitively (MENU strings) and by the trace run loading every scene without codec exceptions. (Acceptable; a byte-for-byte Japanese golden could be added if a divergence appears.)

Placeholder scan: none — every code step is complete; the only "investigate" is Task 5 Step 4, which is the differential-debugging loop (inherent) and points at vm0.py --scene for each named divergence.

Type consistency: OpcodeTable, Script, Operand(Type,Value), Instruction(Offset,Opcode,Args), Sys4Loader.Load(path,table), VirtualMachine(script,table,host,options?) with .Globals/.Emitted/.HaltReason/.Steps/.Run(), Paths.Scripts()/OpcodesJson/Build/Data1, IHost.ShowText/CallScript/OnStub — used identically across Tasks 1→5. Trace JSON keys (offsets/halt/steps) match between vm0.py --trace (Task 5 Step 2) and both C# consumers (Task 5 Steps 1 & 5). Halt-reason format LOOP:line@0x{off:x}×{count} matches vm0's {v:#x} (0x-prefixed) exactly.

40 KiB Raw Permalink Blame History Unescape Escape