OpenMaidEngine/docs/superpowers/plans/2026-07-09-engine-ctx-struct.md

EngineCtx Struct — Implementation Plan

Status: completed 2026-07-09 in 3826064 and 0ecd776. This is the archived execution plan; checkboxes below are preserved as originally authored.

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Type the engine context as a Ghidra EngineCtx struct sourced from a canonical vm-map/engine-ctx.toml, and retype the dispatch handlers' this so they decompile ctx->field instead of magic offsets.

Architecture: Single source of truth (engine-ctx.toml) → generator (engine_ctx_build.py) emits build/engine-ctx.json + docs/engine-ctx-reference.md → a run_script_inline pass builds the EngineCtx struct and retypes handler this params on /v2. Pure model/lint/emit logic is unit-tested; the applied result is validated by decompiling two handlers.

Tech Stack: Python 3.11 (py -3.11 -X utf8, tomllib), ghidra-mcp run_script_inline (Java; GHIDRA_MCP_ALLOW_SCRIPTS=1).

Spec: docs/superpowers/specs/2026-07-09-engine-ctx-struct-design.md.

Global Constraints

• Run Python as py -3.11 -X utf8 tools/<name>.py …. Tools compute paths via paths.py; build/ is disposable.
• Never hand-edit generated files (build/engine-ctx.json, docs/engine-ctx-reference.md): edit engine-ctx.toml, re-run --build.
• Scope = engine ctx (esi) fields ONLY; the VM global bank G[…] is a separate space (globals.toml), out of scope.
• Ghidra: confirm the active program is /v2/range_00400000.bin (base 0x400000, 4400+ fns — the two-program gotcha) before any script; end mutation with save_program.
• Seed fields are 4-byte scalars (int/uint/void*) — names the offset without span-management; arrays/sub-structs are a later refinement.

---

Task 1: Canonical source + generator (engine-ctx.toml → JSON + doc)

Files:

• Create: vm-map/engine-ctx.toml (canonical source)
• Create: tools/engine_ctx_build.py (model + lint + emit + CLI)
• Create: tools/test_engine_ctx.py (unit tests for lint + emit)
• Create: build/engine-ctx.json (generated), docs/engine-ctx-reference.md (generated)

Interfaces:

• Produces: load(toml_text) -> dict ({"meta": {...}, "fields": [{"offset":int,"name","type","note"}...]}); lint(model) -> list[str] (errors); emit_json(model) -> dict; emit_reference_md(model) -> str.

• TYPE_SIZES = {"int":4, "uint":4, "void*":4} (all seed fields 4 bytes).

• Step 1: Create the canonical source vm-map/engine-ctx.toml

# vm-map/engine-ctx.toml -- CANONICAL source for the EngineCtx struct (hand-edited).
# Generated: build/engine-ctx.json + docs/engine-ctx-reference.md via tools/engine_ctx_build.py --build.
# Applied to the Ghidra /v2 image via run_script_inline (see the plan). ctx = engine context (esi/thiscall this).
# The VM global bank G[...] is a SEPARATE space (vm-map/globals.toml), NOT ctx offsets.
[meta]
struct_name = "EngineCtx"
size = 0xa1000

[[field]]
offset = 0x3028
name = "alt_pack_table"
type = "int"
note = "call-script high-byte alternate pack table (unused by corpus)"
[[field]]
offset = 0x408
name = "gfx_obj_registry"
type = "int"
note = "retained gfx-object map (std::map handle->object); geometry/draw get-or-create, 0x215 returns obj+4 source slot, 0x1f7 erases"
[[field]]
offset = 0x40c
name = "sys4ini_count"
type = "int"
note = "SYS4INI record count"
[[field]]
offset = 0x410
name = "archive_name_table"
type = "void*"
note = "archive-name table base (arc_id*0x100 indexes it)"
[[field]]
offset = 0x414
name = "sys4ini_records"
type = "void*"
note = "SYS4INI 80-byte record base {name[64],arc_id,file_number,offset,size}; record = base + id*0x50"
[[field]]
offset = 0xb558
name = "gfx_dirty_a"
type = "int"
note = "gfx dirty flag (anim set raises)"
[[field]]
offset = 0xb560
name = "gfx_dirty_b"
type = "int"
note = "gfx dirty flag"
[[field]]
offset = 0x14d54
name = "gfx_obj_ptr_table"
type = "void*"
note = "per-object pointer table (ops 0x212/0x213 write obj+0x64/0x68/0x6c)"
[[field]]
offset = 0x14f45
name = "script_frame_index"
type = "int"
note = "call-script frame index (0x1e-dword frames)"
[[field]]
offset = 0x46d14
name = "query_table_46d14"
type = "void*"
note = "stride-0x14 table read by op 0x216"
[[field]]
offset = 0x51b64
name = "frame_timer"
type = "int"
note = "frame timer (present updates 0x51b64/0x51b68)"
[[field]]
offset = 0x51b78
name = "anim_clock_elapsed"
type = "int"
note = "global anim clock elapsed (op 0x238 zeroes)"
[[field]]
offset = 0x51b7c
name = "anim_clock_duration"
type = "int"
note = "global anim clock total duration (op 0x238 sets)"
[[field]]
offset = 0x52bd4
name = "surfaces"
type = "void*"
note = "surface array base [~1000 slots]; create/set-texture (0x1f8/0x1f9) allocate"
[[field]]
offset = 0x53d14
name = "cur_ctx_index"
type = "uint"
note = "current gfx-object / script-context index (curCtx); indexes 0x78-byte records"
[[field]]
offset = 0x53d28
name = "frame_codebase"
type = "void*"
note = "current frame codebase (PC = codebase + off*4)"
[[field]]
offset = 0x53d2c
name = "frame_pc"
type = "int"
note = "current frame PC column (op = *(0x53d2c + curCtx*0x78))"
[[field]]
offset = 0x53d60
name = "ctx_record_base"
type = "void*"
note = "0x78-byte context-record array base (coroutine/script contexts)"
[[field]]
offset = 0x53d64
name = "gfx_obj_record_array"
type = "void*"
note = "gfx object-record array (field[0]=0xffffffff free; cmd-type at rec+0x24)"
[[field]]
offset = 0x53d88
name = "cmd_type_table"
type = "int"
note = "per-object cmd-type column base (write *(0x53d88 + curCtx*0x78))"
[[field]]
offset = 0x55120
name = "anti_tamper_a"
type = "int"
note = "anti-tamper checksum operand"
[[field]]
offset = 0x55124
name = "anti_tamper_b"
type = "int"
note = "anti-tamper checksum operand"
[[field]]
offset = 0x5512c
name = "anti_tamper_fp"
type = "int"
note = "anti-tamper (import fn ptr / result)"
[[field]]
offset = 0x55248
name = "ret_stack_a"
type = "void*"
note = "per-frame return stack (op 0x8f call pushes)"
[[field]]
offset = 0x552e8
name = "ret_stack_b"
type = "void*"
note = "per-frame return stack (companion)"
[[field]]
offset = 0x5f304
name = "sleep_timer"
type = "int"
note = "sleep timer object (op 0xc8; +8 active, +0x14 start-ms, +0x18 duration)"
[[field]]
offset = 0x6da88
name = "coroutine_yield_a"
type = "void*"
note = "op 0x7b yield-state save (op1 -> +ctxidx*4)"
[[field]]
offset = 0x6db28
name = "coroutine_yield_b"
type = "void*"
note = "op 0x7b yield-state save (op2 -> +ctxidx*4)"
[[field]]
offset = 0x6dbc8
name = "coroutine_runstate"
type = "int"
note = "op 0x7c resume gate (run-state bit 0x2000000)"
[[field]]
offset = 0x6dbcc
name = "coroutine_resume_off"
type = "int"
note = "op 0x7c resume PC offset"
[[field]]
offset = 0x9b24c
name = "dispatch_table"
type = "void*"
note = "opcode->handler table base [0x400]; handler(op) = *(0x9b24c + op*4)"
[[field]]
offset = 0xa0cc0
name = "screen_w"
type = "int"
note = "screen width (640)"
[[field]]
offset = 0xa0cc4
name = "screen_h"
type = "int"
note = "screen height (480)"
[[field]]
offset = 0xa0cc8
name = "screen_bpp"
type = "int"
note = "screen bpp (8)"
[[field]]
offset = 0xa0ce4
name = "run_state_flags"
type = "uint"
note = "interpreter run-state flags (bit1 sleeping; 0x8000000 skip/fast-forward)"

• Step 2: Write the failing tests

# tools/test_engine_ctx.py  (plain runner, no pytest)
import sys
from engine_ctx_build import load, lint, emit_json

FAILS = []
def check(c, m):
    (FAILS.append(m) or print("FAIL:", m)) if not c else print("ok:", m)

GOOD = '[meta]\nstruct_name="EngineCtx"\nsize=0x1000\n' \
       '[[field]]\noffset=0x10\nname="a"\ntype="int"\nnote="x"\n' \
       '[[field]]\noffset=0x20\nname="b"\ntype="void*"\nnote="y"\n'

def test_load_and_emit():
    m = load(GOOD)
    j = emit_json(m)
    check(j["fields"]["0x10"]["name"] == "a" and j["meta"]["struct_name"] == "EngineCtx",
          "emit_json keys fields by hex offset + carries meta")

def test_lint_clean():
    check(lint(load(GOOD)) == [], "clean model lints with no errors")

def test_lint_catches_overlap():
    bad = '[meta]\nstruct_name="E"\nsize=0x1000\n' \
          '[[field]]\noffset=0x10\nname="a"\ntype="int"\nnote=""\n' \
          '[[field]]\noffset=0x12\nname="b"\ntype="int"\nnote=""\n'   # 0x10+4 > 0x12 -> overlap
    check(any("overlap" in e.lower() for e in lint(load(bad))), "lint flags overlapping fields")

def test_lint_catches_oob_and_dupname():
    bad = '[meta]\nstruct_name="E"\nsize=0x14\n' \
          '[[field]]\noffset=0x10\nname="a"\ntype="int"\nnote=""\n' \
          '[[field]]\noffset=0x40\nname="a"\ntype="int"\nnote=""\n'   # 0x40 > size AND dup name
    errs = lint(load(bad))
    check(any("out of bounds" in e.lower() for e in errs) and any("duplicate" in e.lower() for e in errs),
          "lint flags out-of-bounds offset and duplicate name")

def main():
    test_load_and_emit(); test_lint_clean(); test_lint_catches_overlap(); test_lint_catches_oob_and_dupname()
    print("FAILURES:", len(FAILS)); return 1 if FAILS else 0

if __name__ == "__main__":
    sys.exit(main())

• Step 3: Run tests to verify they fail

Run: py -3.11 -X utf8 tools/test_engine_ctx.py Expected: FAIL — ModuleNotFoundError: No module named 'engine_ctx_build'.

• Step 4: Implement tools/engine_ctx_build.py

"""Build the EngineCtx struct artifacts from vm-map/engine-ctx.toml (single source of truth).

  py -3.11 -X utf8 tools/engine_ctx_build.py --build   # -> build/engine-ctx.json + docs/engine-ctx-reference.md
  py -3.11 -X utf8 tools/engine_ctx_build.py --lint     # checks only

Apply to the Ghidra /v2 image via run_script_inline (see docs/superpowers/plans/2026-07-09-engine-ctx-struct.md).
"""
import json, sys, tomllib
from pathlib import Path

REPO = Path(__file__).resolve().parent.parent
TYPE_SIZES = {"int": 4, "uint": 4, "void*": 4}


def load(toml_text):
    d = tomllib.loads(toml_text)
    fields = [{"offset": int(f["offset"]), "name": f["name"], "type": f["type"], "note": f.get("note", "")}
              for f in d.get("field", [])]
    fields.sort(key=lambda f: f["offset"])
    return {"meta": d["meta"], "fields": fields}


def lint(model):
    errs, size = [], int(model["meta"]["size"])
    seen_names, prev = {}, None
    for f in model["fields"]:
        sz = TYPE_SIZES.get(f["type"])
        if sz is None:
            errs.append(f"unknown type {f['type']!r} for {f['name']}")
            sz = 4
        if f["offset"] + sz > size:
            errs.append(f"field {f['name']} @0x{f['offset']:x} out of bounds (size 0x{size:x})")
        if f["name"] in seen_names:
            errs.append(f"duplicate name {f['name']!r}")
        seen_names[f["name"]] = True
        if prev is not None and f["offset"] < prev["end"]:
            errs.append(f"overlap: {f['name']} @0x{f['offset']:x} into {prev['name']} (ends 0x{prev['end']:x})")
        prev = {"name": f["name"], "end": f["offset"] + sz}
    return errs


def emit_json(model):
    return {"meta": model["meta"],
            "fields": {hex(f["offset"]): {"name": f["name"], "type": f["type"]} for f in model["fields"]}}


def emit_reference_md(model):
    lines = ["# EngineCtx field reference", "",
             "> Generated from `vm-map/engine-ctx.toml` by `tools/engine_ctx_build.py --build`. Do not edit.",
             "", f"Struct `{model['meta']['struct_name']}`, size `0x{int(model['meta']['size']):x}`.", "",
             "| offset | name | type | note |", "|---|---|---|---|"]
    for f in model["fields"]:
        lines.append(f"| `0x{f['offset']:x}` | `{f['name']}` | `{f['type']}` | {f['note']} |")
    return "\n".join(lines) + "\n"


def main():
    text = (REPO / "vm-map" / "engine-ctx.toml").read_text(encoding="utf-8")
    model = load(text)
    errs = lint(model)
    if errs:
        print("LINT ERRORS:")
        for e in errs:
            print("  " + e)
        return 1
    if "--lint" in sys.argv[1:]:
        print(f"lint clean: {len(model['fields'])} fields")
        return 0
    if "--build" in sys.argv[1:]:
        (REPO / "build" / "engine-ctx.json").write_text(json.dumps(emit_json(model), indent=2) + "\n", encoding="utf-8")
        (REPO / "docs" / "engine-ctx-reference.md").write_text(emit_reference_md(model), encoding="utf-8")
        print(f"built {len(model['fields'])} fields -> build/engine-ctx.json + docs/engine-ctx-reference.md")
        return 0
    print(__doc__)
    return 2


if __name__ == "__main__":
    sys.exit(main())

• Step 5: Run tests to verify they pass

Run: py -3.11 -X utf8 tools/test_engine_ctx.py Expected: PASS (4 tests, FAILURES: 0).

• Step 6: Build the real artifacts and lint-check the seed

Run: py -3.11 -X utf8 tools/engine_ctx_build.py --build Expected: built 35 fields -> build/engine-ctx.json + docs/engine-ctx-reference.md, no lint errors. Eyeball docs/engine-ctx-reference.md for the field table.

• Step 7: Commit

git add vm-map/engine-ctx.toml tools/engine_ctx_build.py tools/test_engine_ctx.py docs/engine-ctx-reference.md
git commit -m "re: EngineCtx field registry (engine-ctx.toml) + builder"

---

Task 2: Apply the struct + retype handlers in Ghidra, validate

Precondition: build/engine-ctx.json + build/op-handler-map.json exist; active program /v2.

Files: none in-repo — driven via run_script_inline (Java).

• Step 1: Confirm the correct program + capture BEFORE snippet

Via MCP get_current_program_info — assert base 0x400000, 4400+ fns (else open_program/switch_program /v2/range_00400000.bin). Then decompile_function 0x44cff0 (sleep_timer_arm) and note the current (*DAT_...) / param_1 + 0x5f304 rendering for the before/after. Expected: confirmed program + a BEFORE snippet showing raw offsets.

• Step 2: Create the EngineCtx struct from build/engine-ctx.json

run_script_inline (Java): read the JSON; get/create a StructureDataType("EngineCtx", size) in the program's data type manager; for each offset → {name,type} map type to a Ghidra DataType (int/uint → IntegerDataType/UnsignedIntegerDataType; void* → PointerDataType) and struct.replaceAtOffset(offset, dt, dt.getLength(), name, note); add via dataTypeManager.addDataType(struct, REPLACE_HANDLER). One transaction. Print field count placed.

import java.nio.file.*; import ghidra.program.model.data.*;
String txt = new String(Files.readAllBytes(Paths.get("S:/Game Hacking/Eushully/Himegari/age-reimpl/build/engine-ctx.json")));
// parse meta.size + each "0xoff":{"name":..,"type":..} with a regex; build the struct.
// map type: "void*" -> new PointerDataType(); "uint" -> UnsignedIntegerDataType.dataType; else IntegerDataType.dataType
// struct.replaceAtOffset(off, dt, 4, name, note);

Expected: EngineCtx struct created with 35 fields.

• Step 3: Retype dispatch-handler this to EngineCtx *

run_script_inline (Java): read build/op-handler-map.json; resolve EngineCtx * (new PointerDataType(struct)); for each handler VA, get the Function; if it has a this/first param whose usage is ctx (thiscall or first param), set that param's type to EngineCtx * (func.getParameter(0).setDataType(ptr, SourceType.USER_DEFINED), or func.setCallingConvention("__thiscall") + this-type where needed). Count retyped; collect + print the VAs skipped (no first param / not ctx-shaped) for review. One transaction; save_program. Expected: most handlers retyped; a small skip list.

• Step 4: Validate — AFTER snippet

decompile_function 0x44cff0 (sleep_timer_arm) and 0x42a0b0 (gfx_op_0x215_query_source_slot). Expected: sleep_timer_arm renders ctx->sleep_timer (was param_1 + 0x5f304); the 0x215 handler renders ctx->cur_ctx_index / ctx->cmd_type_table at 0x53d14/0x53d88. Record before/after. If fields don't render, diagnose the this-type application (Step 3) before proceeding.

---

Task 3: Docs + memory + close

Files:

• Modify: CLAUDE.md (canonical-documents map + single-source-of-truth table: add the engine-ctx.toml row)

• Modify: docs/tools-reference.md (add engine_ctx_build.py)

• Modify: docs/engine-re.md (link docs/engine-ctx-reference.md from the ctx-offsets discussion)

• Modify: ~/.claude/…/memory/himegari-port-status.md (milestone)

• Step 1: Update CLAUDE.md

Add to the canonical-documents map: | Engine ctx struct fields | age-reimpl/vm-map/engine-ctx.toml (generated → build/engine-ctx.json, docs/engine-ctx-reference.md) |. Add to the single-source table: | vm-map/engine-ctx.toml | py -3.11 -X utf8 tools/engine_ctx_build.py --build | build/engine-ctx.json, docs/engine-ctx-reference.md |.

• Step 2: Update tools-reference.md + engine-re.md

tools-reference.md: add an engine_ctx_build.py row (source → JSON+doc; --build/--lint; applied to Ghidra via run_script_inline). engine-re.md: in the dispatch/ctx-offsets area, add a line pointing to docs/engine-ctx-reference.md as the field map, noting the struct is applied to the /v2 image.

• Step 3: Update the status memory

Record: EngineCtx struct DONE — N fields, applied to /v2 (M handlers retyped), before/after validated; source engine-ctx.toml; grows incrementally.

• Step 4: Commit

git add CLAUDE.md docs/tools-reference.md docs/engine-re.md
git commit -m "re: apply EngineCtx struct to /v2 handlers; doc + canonical-map wiring"

---

Self-Review

Spec coverage: source engine-ctx.toml → Task 1 Step 1; builder + lint + JSON + doc → Task 1 Steps 4/6; struct creation → Task 2 Step 2; handler this retype → Task 2 Step 3; validation decompiles → Task 2 Step 4; CLAUDE.md/tools-reference/memory → Task 3. Covered.

Placeholder scan: Task 2 Java bodies are described with the key API calls (replaceAtOffset, setDataType, PointerDataType) rather than full source — intentional: the exact DataTypeManager calls are pinned at implementation against the live API (same as Task A's iterative script bring-up), and the logic is fully specified. No TBDs in the Python (Task 1 is complete code).

Type consistency: load returns {"meta","fields":[{offset,name,type,note}]}; lint/emit_json/emit_reference_md all consume that shape; emit_json outputs {meta, fields:{hex:{name,type}}} which the Task 2 Step 2 Ghidra parser reads. TYPE_SIZES keys (int/uint/void*) match the toml type values and the Task 2 Ghidra type mapping. Consistent.