OpenMaidEngine/docs/superpowers/plans/2026-07-07-globals-registry-and-story-flags.md

Globals Registry + Story-Flag Miner Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Build a hand-curated, single-source semantic registry for VM global variables (vm-map/globals.toml), generated into machine + human views and wired into sys4load labeling, plus a static miner that discovers story-state flags and proposes skeleton entries.

Architecture: Mirror the existing opcode-reference toolchain. vm-map/globals.toml is the only hand-edited source (like opcodes.toml). tools/globals_build.py merges curated entries over the auto-generated shape map (build/global-var-map.json) into build/globals.json + generated docs/global-reference.md. tools/story_flags.py statically scans the 481-script corpus for globals that feed branch conditions, emits ranked candidates, and can bootstrap skeletons into globals.toml. sys4load switches its operand-label source from the raw auto map to the merged build/globals.json.

Tech Stack: Python 3.11 (py -3.11 -X utf8), stdlib tomllib/json/argparse/re. Reuses tools/paths.py, tools/sys4load.py, tools/age_opcodes.py. Standalone test scripts (no pytest), matching tools/test_opcodes.py.

Global Constraints

• Run every tool as py -3.11 -X utf8 tools/<name>.py … (the -X utf8 is mandatory on Windows for cp932/Shift-JIS text). Source text is cp932; everything we generate is UTF-8.
• Tools never hard-code paths — always import paths (tools/paths.py) and derive from it.
• Never hand-edit a generated file. Generated files carry a "DO NOT EDIT — generated from …" header. build/globals.json and docs/global-reference.md are generated from vm-map/globals.toml; build/global-var-map.json is generated by tools/global_map.py (do not edit either).
• tools/age_opcodes.py is pristine upstream Kelebek data — never edit it.
• Provenance vocabulary is fixed: category ∈ {story-flag, index-pointer, data-table, string-table, ui-toggle, choice-output, counter, unknown}; source ∈ {investigation, harness, inference, auto-shape}; confidence ∈ {low, med, high}. An auto-shape source may never claim high confidence.
• Git repo root is age-reimpl/. Commit from there. End commit messages with the Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> trailer.
• Tests are standalone scripts using a check(cond, msg) helper that records failures and exits nonzero if any fail (pattern: tools/test_opcodes.py). Run: py -3.11 -X utf8 tools/test_globals.py.

Reference facts (verified against the corpus, 2026-07-07):

• Comparison opcodes (result→arg0, operands at arg indices 1 & 2): 0x5a eq, 0x5b ne, 0x5c lt, 0x5d lte, 0x5e gr, 0x5f gre. Logical (operands at 1 & 2): 0x56 and, 0x57 or. 0xa0 jcc reads its condition at arg index 0 (args 1 & 2 are jump targets). 0x55 mov writes arg0.
• Operand shape: Instruction.args is a list of (atype, value) tuples. Global atypes = {3,4,5,6,8} (int/float/string/ptr/string-ptr). Immediate atype = 0.
• sys4load.load(path) → object with .instructions; each Instruction has .opcode, .label, .args, .offset.
• build/global-var-map.json top-level keys: globals (dict "0xADDR" → {type, uses, roles, stride_candidates, label, confidence, kind, table}) and current_entity_index_candidates (list of {addr, used_as_row_index, also_scalar, purity}).
• Corpus scan of the two known flags confirms the heuristic: 0x3234 (chapter) → scene-reach 0, total branch-reads exist, compared consts {1..8}, writer FIELD.BIN; 0xa57 (Lily form A) → scene-reach 78, compared const {1}, no mov writer. The miner must track total-reach (not only scene-reach) and must not require a static writer.

---

Task 1: globals.toml source + loader + lint

Create the hand-edited registry with first-pass curated content, plus the load/lint core of globals_build.py.

Files:

• Create: vm-map/globals.toml
• Create: tools/globals_build.py
• Create: tools/test_globals.py

Interfaces:

• Produces (consumed by Tasks 2, 4, 5):

  • globals_build.load_toml(path: Path) -> tuple[dict[int, dict], dict] — returns (entries_by_addr, meta); each entry is the raw TOML table dict with an added _addr: int key.
  • globals_build.lint(entries: dict[int, dict], all_addrs: set[int]) -> tuple[list[str], list[str]] — (errors, warnings). all_addrs is the set of addresses valid for depends_on resolution (curated ∪ auto).
  • Module constants CATEGORIES: set[str], SOURCES: set[str], CONFIDENCE: dict[str,int].

• Step 1: Write the failing test

Create tools/test_globals.py:

#!/usr/bin/env python3
"""Standalone tests for the globals registry tooling. Run: py -3.11 -X utf8 tools/test_globals.py"""
import os, sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import paths
import globals_build as G

FAILS = []
def check(cond, msg):
    print(("  ok  " if cond else " FAIL ") + msg)
    if not cond: FAILS.append(msg)

def test_load_and_lint():
    entries, meta = G.load_toml(paths.VM_MAP / "globals.toml")
    check(0xa57 in entries, "0xa57 present in globals.toml")
    check(entries[0xa57]["category"] == "story-flag", "0xa57 is category story-flag")
    check(entries[0x3234]["category"] == "story-flag", "0x3234 is category story-flag")
    check({0xa57, 0xa58, 0xa59} <= set(entries), "Lily form flags A/B/C all present")
    # lint clean against a permissive address universe (curated addrs are self-consistent)
    errors, warnings = G.lint(entries, set(entries))
    check(errors == [], f"globals.toml lints clean (errors={errors})")

def test_lint_catches_bad_vocab():
    bad = {0x1: {"_addr": 0x1, "name": "x", "category": "bogus",
                 "source": "auto-shape", "confidence": "high"}}
    errors, _ = G.lint(bad, {0x1})
    check(any("category" in e for e in errors), "lint flags bad category")
    check(any("confidence" in e for e in errors), "lint flags auto-shape claiming high confidence")

if __name__ == "__main__":
    test_load_and_lint()
    test_lint_catches_bad_vocab()
    print(f"\n{len(FAILS)} failures")
    sys.exit(1 if FAILS else 0)

• Step 2: Run test to verify it fails

Run: py -3.11 -X utf8 tools/test_globals.py Expected: FAIL — ModuleNotFoundError: No module named 'globals_build' (or, once the module exists but globals.toml doesn't, a load error).

• Step 3: Create globals.toml with schema + first-pass curated content

Create vm-map/globals.toml:

# vm-map/globals.toml -- CANONICAL living global-variable registry (hand-edited).
# Generated artifacts (build/globals.json, docs/global-reference.md) come from this file via
# tools/globals_build.py --build. Do not edit those. Skeletons are appended by
# tools/story_flags.py --bootstrap; enrich each entry as we investigate.
# See docs/superpowers/specs/2026-07-07-globals-registry-and-story-flags-design.md.
[meta]
note = "Curated global addresses override the auto shape-inference map (build/global-var-map.json)."

[[global]]
address = "0x3234"
name = "chapter_mode"
category = "story-flag"
type = "int"
value_domain = "1..9"
usage = "Progression chapter/mode selector. SCJUMP's top-level switch keys on it; branch-read by progression scripts (FIELD etc.), not directly by SC/SP scenes."
source = "investigation"
confidence = "high"
depends_on = ["0x62ccf"]

[[global]]
address = "0xa57"
name = "lily_form_a"
category = "story-flag"
type = "int"
value_domain = "{0,1}"
usage = "Lily current-form flag A. Exactly one of form A/B/C is 1; gates form-specific voiced dialogue (seeding 0xa57=1 -> SC0000 186->229 lines). Set externally (menu/save), no static writer."
source = "investigation"
confidence = "high"
depends_on = ["0xa58", "0xa59"]

[[global]]
address = "0xa58"
name = "lily_form_b"
category = "story-flag"
type = "int"
value_domain = "{0,1}"
usage = "Lily current-form flag B. See lily_form_a."
source = "investigation"
confidence = "high"
depends_on = ["0xa57", "0xa59"]

[[global]]
address = "0xa59"
name = "lily_form_c"
category = "story-flag"
type = "int"
value_domain = "{0,1}"
usage = "Lily current-form flag C. See lily_form_a."
source = "investigation"
confidence = "high"
depends_on = ["0xa57", "0xa58"]

[[global]]
address = "0x6c9"
name = "ui_toggle_0"
category = "ui-toggle"
type = "int"
value_domain = "{0,1}"
usage = "ADV-chrome hotspot button toggle (op 0x90 site, near-universal across scenes)."
source = "investigation"
confidence = "med"
depends_on = []

[[global]]
address = "0x6ca"
name = "ui_toggle_1"
category = "ui-toggle"
type = "int"
value_domain = "{0,1}"
usage = "ADV-chrome hotspot button toggle. See ui_toggle_0."
source = "investigation"
confidence = "med"
depends_on = []

[[global]]
address = "0x6cb"
name = "ui_toggle_2"
category = "ui-toggle"
type = "int"
value_domain = "{0,1}"
usage = "ADV-chrome hotspot button toggle. See ui_toggle_0."
source = "investigation"
confidence = "med"
depends_on = []

[[global]]
address = "0x6cc"
name = "ui_toggle_3"
category = "ui-toggle"
type = "int"
value_domain = "{0,1}"
usage = "ADV-chrome hotspot button toggle. See ui_toggle_0."
source = "investigation"
confidence = "med"
depends_on = []

[[global]]
address = "0x6cd"
name = "ui_toggle_4"
category = "ui-toggle"
type = "int"
value_domain = "{0,1}"
usage = "ADV-chrome hotspot button toggle. See ui_toggle_0."
source = "investigation"
confidence = "med"
depends_on = []

[[global]]
address = "0x62ccf"
name = "scjump_decision_out"
category = "choice-output"
type = "int"
value_domain = "?"
usage = "One of SCJUMP's output/decision globals (progression state machine writes it). Related to chapter_mode."
source = "inference"
confidence = "low"
depends_on = []

[[global]]
address = "0xeff75"
name = "current_entity_index_hi"
category = "index-pointer"
type = "int"
value_domain = "row index"
usage = "High-purity current-entity row index (purity 0.95 in the auto shape map); dominant 2D-table row selector."
source = "inference"
confidence = "med"
depends_on = []

[[global]]
address = "0x152616"
name = "current_entity_index"
category = "index-pointer"
type = "int"
value_domain = "row index"
usage = "Primary current-entity row index (RECOVER-confirmed; purity 0.51, 363 row-index uses)."
source = "investigation"
confidence = "med"
depends_on = []

• Step 4: Implement load_toml + lint in globals_build.py

Create tools/globals_build.py:

#!/usr/bin/env python3
"""Generator + linter for the living global-variable registry (vm-map/globals.toml).
  --build   merge curated over build/global-var-map.json -> build/globals.json + docs/global-reference.md
  --lint    run the linter, print errors/warnings, exit nonzero on errors
See docs/superpowers/specs/2026-07-07-globals-registry-and-story-flags-design.md."""
from __future__ import annotations
import os, sys, json, argparse, collections, tomllib
from pathlib import Path
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import paths

TOML_DEFAULT = paths.VM_MAP / "globals.toml"
AUTO_MAP     = paths.BUILD / "global-var-map.json"

CATEGORIES = {"story-flag", "index-pointer", "data-table", "string-table",
              "ui-toggle", "choice-output", "counter", "unknown"}
SOURCES    = {"investigation", "harness", "inference", "auto-shape"}
CONFIDENCE = {"low": 1, "med": 2, "high": 3}


def _parse_addr(v) -> int:
    return int(v, 16) if isinstance(v, str) else int(v)


def load_toml(path) -> tuple[dict[int, dict], dict]:
    """Return ({addr:int -> entry-dict (with _addr set)}, meta-dict)."""
    data = tomllib.loads(Path(path).read_text(encoding="utf-8"))
    out: dict[int, dict] = {}
    for e in data.get("global", []):
        addr = _parse_addr(e["address"])
        e = dict(e)
        e["_addr"] = addr
        out[addr] = e
    return out, data.get("meta", {})


def lint(entries: dict[int, dict], all_addrs: set[int]) -> tuple[list[str], list[str]]:
    """Errors: bad vocabulary, auto-shape@high, dangling depends_on, dup handled by dict.
    Warnings: (reserved)."""
    errors: list[str] = []
    warnings: list[str] = []
    for addr, e in sorted(entries.items()):
        tag = f"0x{addr:x}"
        cat, src, conf = e.get("category"), e.get("source"), e.get("confidence")
        if cat not in CATEGORIES:
            errors.append(f"{tag}: bad category {cat!r}")
        if src not in SOURCES:
            errors.append(f"{tag}: bad source {src!r}")
        if conf not in CONFIDENCE:
            errors.append(f"{tag}: bad confidence {conf!r}")
        if src == "auto-shape" and conf == "high":
            errors.append(f"{tag}: auto-shape source may not claim high confidence")
        for dep in e.get("depends_on", []):
            if _parse_addr(dep) not in all_addrs:
                errors.append(f"{tag}: depends_on missing address {dep}")
    return errors, warnings


def main(argv=None):
    ap = argparse.ArgumentParser()
    ap.add_argument("--build", action="store_true")
    ap.add_argument("--lint", action="store_true")
    ap.add_argument("--toml", default=str(TOML_DEFAULT))
    args = ap.parse_args(argv)
    tp = Path(args.toml)
    if args.lint:
        entries, _ = load_toml(tp)
        errors, warnings = lint(entries, set(entries))
        for m in warnings: print("warn:", m)
        for m in errors: print("error:", m)
        print(f"lint: {len(errors)} errors, {len(warnings)} warnings")
        return 1 if errors else 0
    if args.build:
        return build(tp)          # implemented in Task 2
    ap.error("no action (expected --build/--lint)")


if __name__ == "__main__":
    sys.exit(main())

Note: main references build() for --build, added in Task 2. --lint and the loader are fully functional now; do not call --build until Task 2.

• Step 5: Run tests to verify they pass

Run: py -3.11 -X utf8 tools/test_globals.py Expected: PASS — all checks ok, 0 failures, exit 0.

Also run the linter directly: Run: py -3.11 -X utf8 tools/globals_build.py --lint Expected: lint: 0 errors, 0 warnings, exit 0.

• Step 6: Commit

git add vm-map/globals.toml tools/globals_build.py tools/test_globals.py
git commit -m "feat: globals.toml registry source + loader/lint (Task 1)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"

---

Task 2: globals_build.py --build — merge + generate

Merge curated entries over the auto shape map and emit the machine JSON + human MD.

Files:

• Modify: tools/globals_build.py (add build(), merge(), emit_json(), emit_reference_md(), _auto_category())
• Modify: tools/test_globals.py (add merge-precedence test)

Interfaces:

• Consumes: load_toml, lint, AUTO_MAP, CATEGORIES from Task 1.

• Produces (consumed by Task 3 + Task 6):

  • globals_build.merge(curated: dict[int,dict], auto: dict) -> dict[int, dict] — merged registry keyed by addr; each value has keys address, name, category, type, value_domain, usage, source, confidence, depends_on, provenance where provenance ∈ {"curated","auto"}.
  • build/globals.json — {"meta":…, "generated_from":"vm-map/globals.toml", "globals": {"0xADDR": entry}}.
  • docs/global-reference.md — generated human view.

• Step 1: Write the failing test

Add to tools/test_globals.py (call it from __main__ after the existing tests):

def test_merge_precedence():
    curated, _ = G.load_toml(paths.VM_MAP / "globals.toml")
    auto = G.load_auto(paths.BUILD / "global-var-map.json")
    merged = G.merge(curated, auto)
    check(merged[0xa57]["name"] == "lily_form_a", "curated 0xa57 name wins over auto label")
    check(merged[0xa57]["category"] == "story-flag", "curated 0xa57 category overrides auto string-table")
    check(merged[0xa57]["provenance"] == "curated", "0xa57 marked curated")
    # an address only in the auto map falls through as provenance=auto
    auto_only = next((a for a in auto.get("globals", {})
                      if int(a, 16) not in curated and auto["globals"][a].get("label")), None)
    check(auto_only is not None and merged[int(auto_only, 16)]["provenance"] == "auto",
          "auto-only address retained with provenance=auto")

Add test_merge_precedence() to the __main__ block.

• Step 2: Run test to verify it fails

Run: py -3.11 -X utf8 tools/test_globals.py Expected: FAIL — AttributeError: module 'globals_build' has no attribute 'load_auto' (and merge).

• Step 3: Implement load_auto, merge, emitters, and build in globals_build.py

Add to tools/globals_build.py (above main):

GEN_JSON_META = {"generated_from": "vm-map/globals.toml",
                 "note": "DO NOT EDIT -- generated by tools/globals_build.py --build"}

def load_auto(path) -> dict:
    try:
        return json.loads(Path(path).read_text(encoding="utf-8"))
    except Exception:
        return {}

def _auto_category(e: dict) -> str:
    """Best-effort category for an auto shape-map entry (unreliable; curation overrides)."""
    kind = e.get("kind") or ""
    if kind == "record-table":
        return "data-table"
    if kind == "array-1d":
        return "data-table"
    lbl = e.get("label") or ""
    if lbl.startswith("string-table"):
        return "string-table"
    return "unknown"

def merge(curated: dict[int, dict], auto: dict) -> dict[int, dict]:
    out: dict[int, dict] = {}
    for addr_s, e in auto.get("globals", {}).items():
        if not e.get("label"):
            continue
        addr = int(addr_s, 16)
        out[addr] = {"address": f"0x{addr:x}", "name": None, "category": _auto_category(e),
                     "type": e.get("type"), "value_domain": None, "usage": e.get("label"),
                     "source": "auto-shape", "confidence": e.get("confidence") or "low",
                     "depends_on": [], "provenance": "auto"}
    for addr, e in curated.items():
        out[addr] = {"address": f"0x{addr:x}", "name": e.get("name"),
                     "category": e.get("category", "unknown"), "type": e.get("type"),
                     "value_domain": e.get("value_domain"), "usage": e.get("usage", ""),
                     "source": e.get("source", "inference"), "confidence": e.get("confidence", "low"),
                     "depends_on": [f"0x{_parse_addr(d):x}" for d in e.get("depends_on", [])],
                     "provenance": "curated"}
    return out

def emit_json(merged: dict[int, dict], meta: dict) -> str:
    out = {"meta": {**GEN_JSON_META, **meta},
           "globals": {e["address"]: e for _, e in sorted(merged.items())}}
    return json.dumps(out, ensure_ascii=False, indent=2) + "\n"

def emit_reference_md(merged: dict[int, dict]) -> str:
    L = ["<!-- DO NOT EDIT -- generated from vm-map/globals.toml by tools/globals_build.py --build -->",
         "# Global Variable Reference (generated)", "",
         f"{len(merged)} globals ({sum(1 for e in merged.values() if e['provenance']=='curated')} curated, "
         f"{sum(1 for e in merged.values() if e['provenance']=='auto')} auto shape-inferred). "
         "Source of truth: `vm-map/globals.toml`.", ""]
    by_cat = collections.defaultdict(list)
    for addr, e in merged.items():
        by_cat[e["category"]].append(addr)
    for cat in sorted(by_cat):
        L += [f"## {cat}", "", "| address | name | conf | source | usage |", "|---|---|---|---|---|"]
        # curated first, then by address
        for addr in sorted(by_cat[cat], key=lambda a: (merged[a]["provenance"] != "curated", a)):
            e = merged[addr]
            name = e["name"] or "—"
            usage = (e["usage"] or "").replace("|", "\\|").replace("\n", " ")
            L.append(f"| `{e['address']}` | {name} | {e['confidence']} | {e['source']} | {usage} |")
        L.append("")
    return "\n".join(L) + "\n"

def build(tp: Path) -> int:
    curated, meta = load_toml(tp)
    auto = load_auto(AUTO_MAP)
    all_addrs = set(curated) | {int(a, 16) for a in auto.get("globals", {})}
    errors, warnings = lint(curated, all_addrs)
    for m in warnings: print("warn:", m)
    if errors:
        for m in errors: print("error:", m)
        return 1
    merged = merge(curated, auto)
    paths.BUILD.mkdir(parents=True, exist_ok=True)
    (paths.BUILD / "globals.json").write_text(emit_json(merged, meta), encoding="utf-8")
    (paths.REPO / "docs" / "global-reference.md").write_text(emit_reference_md(merged), encoding="utf-8")
    print(f"build: {len(merged)} globals -> build/globals.json, docs/global-reference.md")
    return 0

main already calls build(tp) for --build (the line was written in Task 1). No further change to main is needed.

• Step 4: Run tests + build

Run: py -3.11 -X utf8 tools/globals_build.py --build Expected: build: N globals -> build/globals.json, docs/global-reference.md (N in the low thousands — every auto entry with a label plus curated), exit 0.

Run: py -3.11 -X utf8 tools/test_globals.py Expected: PASS — all checks ok, 0 failures.

• Step 5: Sanity-check the generated JSON

Run: py -3.11 -X utf8 -c "import json; d=json.load(open('build/globals.json',encoding='utf-8')); print(d['globals']['0xa57'])" Expected: shows name: lily_form_a, category: story-flag, provenance: curated.

• Step 6: Commit

git add tools/globals_build.py tools/test_globals.py build/globals.json docs/global-reference.md
git commit -m "feat: globals_build --build merge + generated JSON/MD (Task 2)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"

---

Task 3: Wire sys4load to the merged build/globals.json

Switch the disassembler's global-operand labels from the raw auto map to the merged registry, so curated names/categories show.

Files:

• Modify: tools/sys4load.py:48-76 (the _load_global_labels block)
• Modify: tools/test_globals.py (add a sys4load-label test)

Interfaces:

• Consumes: build/globals.json from Task 2 (globals dict, entries with name, category, usage).

• Produces: sys4load.GLOBAL_LABELS: dict[int, str] now sourced from the merged registry (curated names win; auto tail kept for high/med only). No signature change — downstream renderers unaffected.

• Step 1: Write the failing test

Add to tools/test_globals.py:

def test_sys4load_labels_from_registry():
    import importlib, sys4load
    importlib.reload(sys4load)   # re-run _load_global_labels against current build/globals.json
    lbl = sys4load.GLOBAL_LABELS.get(0x3234, "")
    check("chapter_mode" in lbl, f"sys4load labels 0x3234 with curated name (got {lbl!r})")
    lbl2 = sys4load.GLOBAL_LABELS.get(0xa57, "")
    check("lily_form_a" in lbl2, f"sys4load labels 0xa57 with curated name (got {lbl2!r})")

Add test_sys4load_labels_from_registry() to __main__.

• Step 2: Run test to verify it fails

Run: py -3.11 -X utf8 tools/test_globals.py Expected: FAIL — sys4load labels 0x3234 with curated name fails (current GLOBAL_LABELS comes from global-var-map.json, which has no chapter_mode).

• Step 3: Rewrite _load_global_labels in sys4load.py

Replace the block at tools/sys4load.py:48-76 (from the # Global-variable labels comment through GLOBAL_LABELS = _load_global_labels()) with:

# Global-variable labels (optional): annotate global operands from the merged registry
# build/globals.json (curated vm-map/globals.toml over the auto shape map), produced by
# tools/globals_build.py --build. Curated entries show their name+category; the auto tail
# keeps only high/med confidence to stay readable. Degrades to {} if the file is absent.
GLOBAL_ATYPES = {3, 4, 5, 6, 8}          # global-int/float/string/ptr/string-ptr


def _short_global_label(lbl: str) -> str:
    if lbl.startswith("record-table[stride "):
        return "rec[s" + lbl[len("record-table[stride "):-1] + "]"
    if lbl.startswith("string-table (written by "):
        return "str<" + lbl[len("string-table (written by "):-1] + ">"
    return lbl


def _load_global_labels() -> dict:
    try:
        p = Path(__file__).resolve().parent.parent / "build" / "globals.json"
        data = json.loads(p.read_text(encoding="utf-8"))
    except Exception:
        return {}
    out = {}
    for addr_s, e in data.get("globals", {}).items():
        addr = int(addr_s, 16)
        if e.get("provenance") == "curated" and e.get("name"):
            cat = e.get("category")
            out[addr] = f"{e['name']}({cat})" if cat and cat != "unknown" else e["name"]
        elif e.get("usage") and e.get("confidence") in ("high", "med"):
            out[addr] = _short_global_label(e["usage"])
    return out


GLOBAL_LABELS = _load_global_labels()

• Step 4: Run test to verify it passes

Run: py -3.11 -X utf8 tools/test_globals.py Expected: PASS — the two label checks ok.

• Step 5: Verify the loader still decodes the whole corpus

Run: py -3.11 -X utf8 tools/sys4load.py --validate Expected: 481/481 container-clean AND opcode-decode clean (unchanged from before — labels are cosmetic).

Spot-check a curated label renders in a listing: Run: py -3.11 -X utf8 -c "import sys; sys.path.insert(0,'tools'); import sys4load,paths; print(sys4load.GLOBAL_LABELS[0x3234], '|', sys4load.GLOBAL_LABELS[0xa57])" Expected: prints chapter_mode(story-flag) | lily_form_a(story-flag).

• Step 6: Commit

git add tools/sys4load.py tools/test_globals.py
git commit -m "feat: sys4load labels globals from merged build/globals.json (Task 3)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"

---

Task 4: story_flags.py — static miner → candidates JSON

Scan the corpus for globals that feed branch conditions; emit ranked candidates with evidence.

Files:

• Create: tools/story_flags.py
• Modify: tools/test_globals.py (add miner regression-anchor test)

Interfaces:

• Consumes: sys4load.load, paths.scripts, build/global-var-map.json (for shape exclusion), globals_build.load_toml (Task 5).

• Produces (consumed by Task 5 + Task 6):

  • story_flags.mine() -> dict[int, dict] — addr -> evidence with keys: address(str), reach_total(int), reach_scenes(int), consts(sorted list[int]), writers(sorted list[str]), writers_progression(sorted list[str]), atypes(sorted list[int]), near_universal(bool), category(str), confidence(str).
  • build/story-flags-candidates.json — {"meta":…, "candidates":[evidence…]} sorted by (reach_scenes desc, reach_total desc).
  • story_flags.assign_category(ev) -> tuple[str,str] — returns (category, confidence).

• Step 1: Write the failing test

Add to tools/test_globals.py:

def test_miner_finds_known_flags():
    import story_flags
    cands = story_flags.mine()
    check(0x3234 in cands, "miner surfaces chapter flag 0x3234")
    check(set(range(1, 9)) <= set(cands[0x3234]["consts"]), "0x3234 compared against 1..8 enum")
    check(cands[0x3234]["category"] == "story-flag", "0x3234 classified story-flag")
    check(0xa57 in cands, "miner surfaces Lily form flag 0xa57")
    check(cands[0xa57]["reach_scenes"] >= 70, "0xa57 high scene reach")
    check(cands[0xa57]["category"] == "story-flag", "0xa57 classified story-flag")

Add test_miner_finds_known_flags() to __main__.

• Step 2: Run test to verify it fails

Run: py -3.11 -X utf8 tools/test_globals.py Expected: FAIL — ModuleNotFoundError: No module named 'story_flags'.

• Step 3: Implement tools/story_flags.py

Create tools/story_flags.py:

#!/usr/bin/env python3
"""Static story-state flag miner. Scans the 481-script corpus for global variables that feed
branch conditions (comparisons / jcc), gathers evidence, classifies candidates, and emits a
ranked review surface. 100% static -- no runtime, no sweep. See
docs/superpowers/specs/2026-07-07-globals-registry-and-story-flags-design.md.
  (no flag)     -> build/story-flags-candidates.json
  --bootstrap   append skeleton [[global]] entries to vm-map/globals.toml (Task 5)"""
from __future__ import annotations
import os, sys, re, json, argparse, collections
from pathlib import Path
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import paths
import sys4load

CMP_OPS   = {0x5a, 0x5b, 0x5c, 0x5d, 0x5e, 0x5f}   # eq ne lt lte gr gre  (operands at arg idx 1,2)
LOGIC_OPS = {0x56, 0x57}                            # and or               (operands at arg idx 1,2)
JCC_OP    = 0xa0                                    # condition at arg idx 0
ASSIGN_OP = 0x55                                    # mov -> writes arg0
GLOBAL_ATYPES = {3, 4, 5, 6, 8}
SCENE_RE = re.compile(r"^S[CP]\d{4}\.BIN$")
KNOWN_UI_TOGGLES = {0x6c9, 0x6ca, 0x6cb, 0x6cd, 0x6cc}
NEAR_UNIVERSAL = 250       # scene-reach at/above this = ADV-chrome-wide, not a story flag


def _excluded_addrs() -> set[int]:
    """Addresses the auto shape map classifies as genuine tables/index pointers -- excluded from
    story-flag candidacy. Deliberately does NOT exclude the auto 'string-table' *label* guesses
    (those are unreliable: 0xa57, a real story flag, is mislabelled string-table)."""
    try:
        data = json.loads((paths.BUILD / "global-var-map.json").read_text(encoding="utf-8"))
    except Exception:
        return set()
    excl = {int(c["addr"], 16) for c in data.get("current_entity_index_candidates", [])}
    for addr_s, e in data.get("globals", {}).items():
        if e.get("kind") == "record-table" and e.get("stride_candidates"):
            excl.add(int(addr_s, 16))
    return excl


def assign_category(ev: dict) -> tuple[str, str]:
    """(category, confidence) from evidence. Never 'high' -- auto-shape."""
    addr = int(ev["address"], 16)
    consts = set(ev["consts"])
    domain_bool = consts <= {0, 1}
    enum_like = len(consts) >= 3 and (max(consts) if consts else 0) <= 32
    has_prog_writer = bool(ev["writers_progression"])
    scene_only_writers = bool(ev["writers"]) and not has_prog_writer
    if addr in KNOWN_UI_TOGGLES or (ev["near_universal"] and domain_bool):
        cat = "ui-toggle"
    elif enum_like and has_prog_writer:
        cat = "story-flag"                       # chapter-like (0x3234)
    elif has_prog_writer and ev["reach_total"] >= 1:
        cat = "story-flag"
    elif not ev["writers"] and ev["reach_scenes"] >= 3 and domain_bool:
        cat = "story-flag"                       # externally/natively set form-like (0xa57)
    elif scene_only_writers:
        cat = "choice-output"
    elif ev["reach_scenes"] >= 2 or ev["reach_total"] >= 2:
        cat = "story-flag"
    else:
        cat = "unknown"
    conf = "med" if (has_prog_writer and ev["reach_total"] >= 3) or ev["reach_scenes"] >= 10 else "low"
    return cat, conf


def mine() -> dict[int, dict]:
    reach_total = collections.defaultdict(set)
    reach_scenes = collections.defaultdict(set)
    consts = collections.defaultdict(set)
    writers = collections.defaultdict(set)
    atypes = collections.defaultdict(set)
    for name, path in paths.scripts().items():
        try:
            scr = sys4load.load(path)
        except Exception:
            continue
        is_scene = bool(SCENE_RE.match(name))
        for ins in scr.instructions:
            op, a = ins.opcode, ins.args
            if op in CMP_OPS or op in LOGIC_OPS:
                idxs = [1, 2]
            elif op == JCC_OP:
                idxs = [0]
            else:
                idxs = []
            for i in idxs:
                if i < len(a) and a[i][0] in GLOBAL_ATYPES:
                    addr = a[i][1]
                    reach_total[addr].add(name)
                    atypes[addr].add(a[i][0])
                    if is_scene:
                        reach_scenes[addr].add(name)
                    for j in idxs:                      # compared-against immediates
                        if j != i and j < len(a) and a[j][0] == 0:
                            consts[addr].add(a[j][1])
            if op == ASSIGN_OP and len(a) >= 1 and a[0][0] in GLOBAL_ATYPES:
                writers[a[0][1]].add(name)
    excl = _excluded_addrs()
    out: dict[int, dict] = {}
    for addr in reach_total:
        if addr in excl:
            continue
        w = writers.get(addr, set())
        w_prog = {n for n in w if not SCENE_RE.match(n)}
        ev = {"address": f"0x{addr:x}",
              "reach_total": len(reach_total[addr]),
              "reach_scenes": len(reach_scenes.get(addr, set())),
              "consts": sorted(consts.get(addr, set())),
              "writers": sorted(w),
              "writers_progression": sorted(w_prog),
              "atypes": sorted(atypes[addr]),
              "near_universal": len(reach_scenes.get(addr, set())) >= NEAR_UNIVERSAL}
        ev["category"], ev["confidence"] = assign_category(ev)
        out[addr] = ev
    return out


def write_candidates(cands: dict[int, dict]) -> Path:
    ranked = sorted(cands.values(), key=lambda e: (-e["reach_scenes"], -e["reach_total"]))
    paths.BUILD.mkdir(parents=True, exist_ok=True)
    p = paths.BUILD / "story-flags-candidates.json"
    p.write_text(json.dumps({"meta": {"note": "static branch-condition mining; review surface, "
                                              "not ground truth", "count": len(ranked)},
                             "candidates": ranked}, ensure_ascii=False, indent=2) + "\n",
                 encoding="utf-8")
    return p


def main(argv=None):
    ap = argparse.ArgumentParser()
    ap.add_argument("--bootstrap", action="store_true")   # implemented in Task 5
    args = ap.parse_args(argv)
    cands = mine()
    if args.bootstrap:
        return bootstrap(cands)                           # Task 5
    p = write_candidates(cands)
    story = sum(1 for e in cands.values() if e["category"] == "story-flag")
    print(f"mined {len(cands)} branch-read globals ({story} story-flag candidates) -> {p}")
    return 0


if __name__ == "__main__":
    sys.exit(main())

Note: main references bootstrap() for --bootstrap, added in Task 5. Running with no flag is fully functional now.

• Step 4: Run test + miner

Run: py -3.11 -X utf8 tools/test_globals.py Expected: PASS — the six miner checks ok.

Run: py -3.11 -X utf8 tools/story_flags.py Expected: mined N branch-read globals (M story-flag candidates) -> …/build/story-flags-candidates.json.

• Step 5: Eyeball the top candidates

Run: py -3.11 -X utf8 -c "import json; d=json.load(open('build/story-flags-candidates.json',encoding='utf-8')); [print(c['address'], c['category'], 'scenes=%d'%c['reach_scenes'], 'consts=%s'%c['consts'][:6]) for c in d['candidates'][:15]]" Expected: 0xa57/0xa58/0xa59 near the top with high reach_scenes; entries look plausible (booleans and small enums).

• Step 6: Commit

git add tools/story_flags.py tools/test_globals.py build/story-flags-candidates.json
git commit -m "feat: static story-flag miner -> candidates JSON (Task 4)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"

---

Task 5: story_flags.py --bootstrap — seed skeletons into globals.toml

Append skeleton entries for newly discovered addresses, preserving all hand edits.

Files:

• Modify: tools/story_flags.py (add bootstrap(), skeleton_toml())
• Modify: tools/test_globals.py (add bootstrap test using a temp toml)

Interfaces:

• Consumes: mine() (Task 4), globals_build.load_toml (Task 1).

• Produces: story_flags.bootstrap(cands, toml_path=…) -> int — appends only new-address skeletons (source="auto-shape", confidence from evidence, usage="TODO: …"); idempotent; never rewrites existing entries.

• Step 1: Write the failing test

Add to tools/test_globals.py:

def test_bootstrap_is_additive_and_idempotent():
    import tempfile, pathlib, story_flags
    # start from a copy of the real toml so curated entries are present
    src = (paths.VM_MAP / "globals.toml").read_text(encoding="utf-8")
    with tempfile.TemporaryDirectory() as d:
        tp = pathlib.Path(d) / "globals.toml"
        tp.write_text(src, encoding="utf-8")
        before, _ = G.load_toml(tp)
        cands = story_flags.mine()
        story_flags.bootstrap(cands, toml_path=tp)
        after, _ = G.load_toml(tp)
        check(len(after) > len(before), "bootstrap adds new skeleton entries")
        check(after[0xa57]["name"] == "lily_form_a", "bootstrap preserves curated 0xa57")
        # idempotent: second run adds nothing
        n1 = len(after)
        story_flags.bootstrap(story_flags.mine(), toml_path=tp)
        after2, _ = G.load_toml(tp)
        check(len(after2) == n1, "second bootstrap is a no-op (idempotent)")
        # every skeleton is auto-shape and not high-confidence
        added = set(after) - set(before)
        check(all(after[a]["source"] == "auto-shape" for a in added), "skeletons are source=auto-shape")
        check(all(after[a]["confidence"] != "high" for a in added), "skeletons never high confidence")

Add test_bootstrap_is_additive_and_idempotent() to __main__. (Also add import globals_build as G is already at top; story_flags imported inside the test.)

• Step 2: Run test to verify it fails

Run: py -3.11 -X utf8 tools/test_globals.py Expected: FAIL — AttributeError: module 'story_flags' has no attribute 'bootstrap'.

• Step 3: Implement bootstrap + skeleton_toml in story_flags.py

Add to tools/story_flags.py (above main):

import globals_build as _gb

def skeleton_toml(ev: dict) -> str:
    consts = ", ".join(str(c) for c in ev["consts"][:8])
    domain = "{0,1}" if set(ev["consts"]) <= {0, 1} and ev["consts"] else (f"one of {{{consts}}}" if consts else "?")
    usage = (f"TODO: confirm. Branch-read in {ev['reach_scenes']} scenes / {ev['reach_total']} scripts; "
             f"compared against [{consts}]; "
             f"writers={ev['writers'][:4] or 'none (external/native?)'}.")
    lines = ["[[global]]",
             f'address = "{ev["address"]}"',
             f'name = ""',
             f'category = "{ev["category"]}"',
             'type = "int"',
             f'value_domain = "{domain}"',
             f'usage = "{usage}"',
             'source = "auto-shape"',
             f'confidence = "{ev["confidence"]}"',
             "depends_on = []"]
    return "\n".join(lines) + "\n"

def bootstrap(cands: dict[int, dict], toml_path=None) -> int:
    toml_path = Path(toml_path) if toml_path else (paths.VM_MAP / "globals.toml")
    present = set(_gb.load_toml(toml_path)[0]) if toml_path.exists() else set()
    # only story-flag / ui-toggle / choice-output candidates are worth seeding for curation
    seedable = {a: e for a, e in cands.items()
                if e["category"] in ("story-flag", "ui-toggle", "choice-output") and a not in present}
    if not seedable:
        print(f"bootstrap: nothing new to add ({len(present)} already present).")
        return 0
    blocks = [skeleton_toml(seedable[a]) for a in sorted(seedable)]
    with toml_path.open("a", encoding="utf-8") as f:
        f.write("\n" + "\n".join(blocks))
    print(f"bootstrap: appended {len(blocks)} skeletons -> {toml_path}")
    return 0

The name = "" skeletons parse fine but should be named before they add value; leaving name empty means sys4load keeps the auto label until curated (curated-with-empty-name is treated as auto by the label rule in Task 3, which requires name). That's intended: a skeleton is a to-do, not a finished label.

• Step 4: Run test to verify it passes

Run: py -3.11 -X utf8 tools/test_globals.py Expected: PASS — all bootstrap checks ok, 0 failures.

• Step 5: Verify against the real file is non-destructive (dry check)

Run: py -3.11 -X utf8 -c "import sys; sys.path.insert(0,'tools'); import story_flags,globals_build as g; before=set(g.load_toml('vm-map/globals.toml')[0]); print('curated now:', len(before))" Expected: prints the curated count (13). Do not run --bootstrap on the real file yet — that happens deliberately in Task 6 under review.

• Step 6: Commit

git add tools/story_flags.py tools/test_globals.py
git commit -m "feat: story_flags --bootstrap seeds skeletons into globals.toml (Task 5)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"

---

Task 6: Curate strong candidates + docs + status memory

Run the miner against the real registry, promote the confident findings to named entries, rebuild, and update all documentation homes.

Files:

• Modify: vm-map/globals.toml (bootstrap + hand-name the strong candidates)
• Regenerate: build/globals.json, docs/global-reference.md, build/story-flags-candidates.json
• Modify: docs/name-resolution.md (§2 registry description + new "Story-state flags" subsection)
• Modify: CLAUDE.md (canonical-documents map row + single-source table row)
• Modify: docs/tools-reference.md (add globals_build.py, story_flags.py, test_globals.py)
• Modify: ~/.claude/…/memory/himegari-port-status.md + MEMORY.md (status entry)

Interfaces:

• Consumes: all prior tasks.

• Produces: a populated registry + current docs. No new code interfaces.

• Step 1: Bootstrap the real registry

Run: py -3.11 -X utf8 tools/story_flags.py --bootstrap Expected: bootstrap: appended K skeletons -> …/vm-map/globals.toml.

Run: py -3.11 -X utf8 tools/globals_build.py --lint Expected: lint: 0 errors, … (skeletons are auto-shape + non-high, so they lint clean).

• Step 2: Hand-name the confident candidates

Open build/story-flags-candidates.json. For each candidate with confidence == "med" (strong reach and/or a progression writer), edit its skeleton in vm-map/globals.toml: set a name (snake_case), tighten value_domain, replace the TODO: usage with a real description, and bump source/confidence if you are confident (e.g. inference/med). Leave weak low-confidence skeletons with empty name (they stay to-dos). Do not invent meaning you cannot support — an honest unknown-ish skeleton is fine.

Guidance: cross-reference the reader scripts in a listing to infer purpose, e.g. py -3.11 -X utf8 tools/sys4load.py <SCRIPT.BIN> and search for the address. Use Age.Cli sweep 0xADDR=1 (the separate dynamic reach tool) to confirm a flag actually moves dialogue before naming it a story-flag.

• Step 3: Rebuild generated artifacts + full test

Run: py -3.11 -X utf8 tools/globals_build.py --build Expected: build: N globals -> build/globals.json, docs/global-reference.md.

Run: py -3.11 -X utf8 tools/story_flags.py Expected: refreshed candidates JSON.

Run: py -3.11 -X utf8 tools/test_globals.py Expected: PASS, 0 failures.

Run: py -3.11 -X utf8 tools/sys4load.py --validate Expected: 481/481 clean.

• Step 4: Update docs/name-resolution.md

In §2 ("The global-variable map"), after the existing "Partial map — BUILT (v1)" content, add a subsection describing the curated registry. Insert this text:

### The curated registry — `vm-map/globals.toml` (2026-07-07)

The v1 auto map (`build/global-var-map.json`) infers *shapes* but cannot recover branch-flag
*meaning* — and is sometimes wrong (it labels `0xa57`, the Lily form-A story flag, as a
"string-table"). The curated registry fixes this, modelled exactly on `vm-map/opcodes.toml`:

- **`vm-map/globals.toml`** — the only hand-edited source. One `[[global]]` per known address:
  `name`, `category` (`story-flag`/`index-pointer`/`data-table`/`string-table`/`ui-toggle`/
  `choice-output`/`counter`/`unknown`), `type`, `value_domain`, `usage`, and provenance
  (`source`/`confidence`/`depends_on`).
- **`tools/globals_build.py --build`** merges curated entries *over* the auto map →
  `build/globals.json` (machine) + `docs/global-reference.md` (generated human view). `--lint`
  checks vocabulary, the auto-shape≠high rule, and dangling `depends_on`. `sys4load` reads
  `build/globals.json` for operand labels (curated names win; auto tail kept at high/med).

#### Story-state flags (the first populated category)

Story flags are scalar globals that ADV/progression logic *branches on* (chapter, character
forms, choices) — a category the auto shape map never enumerated. **`tools/story_flags.py`** is a
100% static miner: it flags a global as a candidate when it feeds a comparison (`eq`/`ne`/`lt`/
`lte`/`gr`/`gre`), a logical (`and`/`or`), or a `jcc` condition, and is not a genuine table/index
in the shape map. Per candidate it records compared-against constants (→ value domain), the writer
set (progression-written but scene-read = strong story flag), total- and scene-reach, and
near-universal (ADV-chrome) status → an auto category + confidence. Output:
`build/story-flags-candidates.json` (review surface); `--bootstrap` appends skeletons to
`globals.toml` for human naming. Dynamic confirmation of a flag's reach stays separate —
`Age.Cli sweep 0xADDR=VAL`. Known anchors: `0x3234` chapter (enum 1..9, progression-read,
scene-reach 0), `0xa57/8/9` Lily forms (boolean, scene-read, externally set).

• Step 5: Update CLAUDE.md (two tables)

In the canonical-documents map table, add a row after the opcode row:

| Global variable semantics / addresses (names, story-flags) | `age-reimpl/vm-map/globals.toml` *(source; generated → `build/globals.json`, `docs/global-reference.md`)* |

In the single source of truth table, add a row:

| `vm-map/globals.toml` | `py -3.11 -X utf8 tools/globals_build.py --build` | `build/globals.json`, `docs/global-reference.md` |

• Step 6: Update docs/tools-reference.md

Add rows (match the existing table's column shape — tool / purpose / reads / writes):

| `globals_build.py` | Merge curated globals.toml over the auto shape map; generate registry. | `vm-map/globals.toml`, `build/global-var-map.json` | `build/globals.json`, `docs/global-reference.md` |
| `story_flags.py` | Static story-flag miner (branch-condition mining) + `--bootstrap`. | corpus, `build/global-var-map.json` | `build/story-flags-candidates.json`, appends `vm-map/globals.toml` |
| `test_globals.py` | Unit tests for the globals registry + miner. | — | — |

• Step 7: Commit code + docs

git add vm-map/globals.toml build/globals.json docs/global-reference.md build/story-flags-candidates.json docs/name-resolution.md CLAUDE.md docs/tools-reference.md
git commit -m "feat: curate story-flags into registry + docs (Task 6)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"

• Step 8: Update the status memory

Edit ~/.claude/projects/S--Game-Hacking-Eushully-Himegari/memory/himegari-port-status.md: add a bullet under Phase B recording that the globals registry (vm-map/globals.toml) + static story-flag miner (tools/story_flags.py) landed — the curated home for global semantics (opcodes.toml analogue), story-flags its first category, sys4load now labels from the merged build/globals.json; N story-flag candidates catalogued, K named. Update the MEMORY.md one-line index entry to mention story-flag mapping is underway/done. Convert "2026-07-07" to the absolute date. (Memory files are outside the git repo — no commit needed.)

---

Notes for the implementer

• Ordering matters: Task 2's build() is referenced by Task 1's main but only defined in Task 2 — do not run --build until Task 2 is complete. Likewise story_flags.bootstrap is referenced in Task 4's main but defined in Task 5 — do not run --bootstrap until Task 5.
• py -3.11 -X utf8 always — omitting -X utf8 will corrupt cp932 script text on Windows.
• The candidates JSON is a review surface, not ground truth. Curation (naming) is a human judgment step (Task 6 Step 2); it is fine to leave most low-confidence skeletons unnamed.
• If tomllib is missing (Python < 3.11), the tools fail loudly — this project standardizes on 3.11.