Decode STINIT map and object placement data

This commit is contained in:
gamer147
2026-07-23 00:22:03 -04:00
parent 86a7cb1750
commit 3fb3d3cfd8
9 changed files with 328 additions and 31 deletions

View File

@@ -1,7 +1,7 @@
<!-- DO NOT EDIT -- generated from vm-map/globals.toml by tools/globals_build.py --build -->
# Global Variable Reference (generated)
5026 globals (155 curated, 4871 auto shape-inferred). Source of truth: `vm-map/globals.toml`.
5030 globals (167 curated, 4863 auto shape-inferred). Source of truth: `vm-map/globals.toml`.
## choice-output
@@ -26,6 +26,7 @@
| address | name | conf | source | usage |
|---|---|---|---|---|
| `0x6d3` | story_event_flags | high | investigation | Shared one-based story/event flag bank. Progression, recruitment, item, and stage logic read or write individual cells; STINIT object prerequisite ids are decremented by one before SETOBJ tests this array. |
| `0x2e49` | character_voice_suppressed | high | investigation | Base of the per-character voice enable/suppress settings. INITCONFIG zero-fills all 13 cells and registers each with the shared profile service; LOADCONFIG restores them. CONFIG indexes the table to preview a character voice and write 0/1. ROOM reads cell 0 before assigning its selected greeter's greeting/farewell voice ids, so the port's former scalar interpretation of zero-int-range (writing 13 into the base cell) suppressed those voices on every natural boot. This names the script-visible setting array without choosing a persistence backend for op 0x1a2/0x1a3. |
| `0x65ce` | skill_acquired_flags | high | investigation | Persistent acquired-skill flags. ADDSKILL sets the selected skill after resolving the unit's equipped-skill slots; FORT checks the flag before granting a skill; CHMENU combines it with skill_change_catalog_eligible to build the available skill-change catalog. |
| `0x673c` | party_slot_flags | high | investigation | Per-party-slot state flags for slots 0..99. UNITECH creates the initial unit by setting slot 2 to 0x13; CALCARR counts slots whose flags intersect 0x6, and CHMENU includes slots with bit 1 set. Exact meanings of the remaining bits are not yet classified. |
@@ -102,8 +103,18 @@
| `0xaaa1e` | skill_battle_animation_id | high | investigation | Populated for 101 combat skills. BTL and CALCDMG place this value in the battle-animation selector before calling BTANINIT; most skills reuse their own id, while related skills deliberately share an animation and passive reactions use ids 801..808. |
| `0xaab4a` | skill_handler_script_id | high | investigation | SKINIT field for all 131 skills. CHMENU and INFOIT look it up and pass it directly to call-script; packed id 0x31ca resolves to SKMES.BIN, the shared per-skill text/behavior dispatcher. |
| `0xe7302` | stage_bgm_id | high | investigation | STINIT's per-stage scalar loaded for all 74 records. FIELD passes the value directly to play-bgm when starting the stage. |
| `0xe7303` | stage_target_clear_turns | high | investigation | STINIT's target/par turn count. STAGECLEAR divides elapsed turns by this value to derive a performance multiplier; when replaying an already-cleared ordinary stage, FIELD also uses it as the forced-retreat turn limit. |
| `0xe7304` | stage_clear_performance_bonus | med | investigation | Base stage-clear reward increment. STAGECLEAR multiplies it by the turn-performance percentage derived from stage_target_clear_turns, divides by 100, and adds the result to the capped persistent reward counter at 0x6719. The counter's player-facing resource name remains unresolved. |
| `0xe730b` | stage_replay_rules_disabled | high | investigation | FIELD's already-cleared-stage override gate. For a cleared stage with value 0, FIELD replaces the mission with an all-party retreat objective and a stage_target_clear_turns forced-retreat limit; value 1 suppresses that replay conversion and related revisit handling. |
| `0xe730c` | stage_turn_limit | high | investigation | STINIT's per-stage turn limit. DRAWCHP presents the value in the stage information, while FIELD compares the current turn against it when checking stage completion. |
| `0xe730d` | stage_turn_limit_outcome | high | investigation | STINIT mode paired with stage_turn_limit. Stage 1 stores 0 and describes 50-turn expiry as defeat; stage 2 stores 1 and explicitly describes 15-turn expiry as a forced-retreat clear. |
| `0xe7311` | stage_map_texture_asset_overrides | high | investigation | Twenty current-stage map texture slots. FIELD loads positive resource ids into tiled surface slots 0x52+index, skips zero, and substitutes the shared per-index fallback for -1; DRAWMAP selects and draws those surfaces through the terrain-to-texture-slot map. |
| `0xe7325` | stage_object_tile_x | high | investigation | X coordinate for each current-stage object slot. DRAWOBJ converts it to centered map-space pixels; FIELD and CALCOCC use it with stage_object_tile_y for object interaction and occupancy. |
| `0xe7357` | stage_object_tile_y | high | investigation | Y coordinate for each current-stage object slot. DRAWOBJ converts it to centered map-space pixels; FIELD and CALCOCC use it with stage_object_tile_x for object interaction and occupancy. |
| `0xe7389` | stage_object_type_id | high | investigation | Object-definition id for each current-stage object slot. SETOBJ decides whether the slot exists, while DRAWOBJ and FIELD use the id to select shared object graphics, dimensions, animation, collision, and behavior metadata. |
| `0xe7483` | stage_object_difficulty_mask | high | investigation | Per-object difficulty inclusion mask. SETOBJ uses check-bit with difficulty_index and rejects the object slot when the selected difficulty bit is absent. |
| `0xe74b5` | stage_object_required_story_flags | high | investigation | Seven positive prerequisites per stage object. SETOBJ subtracts one from every populated id and suppresses the object unless the corresponding story_event_flags cell equals 1. Columns: 0=required_flag_1, 1=required_flag_2, 2=required_flag_3, 3=required_flag_4, 4=required_flag_5, 5=required_flag_6, 6=required_flag_7. |
| `0xe7613` | stage_object_forbidden_story_flags | high | investigation | Five negative prerequisites per stage object. SETOBJ subtracts one from every populated id and suppresses the object when the corresponding story_event_flags cell equals 1. Columns: 0=forbidden_flag_1, 1=forbidden_flag_2, 2=forbidden_flag_3, 3=forbidden_flag_4, 4=forbidden_flag_5. |
| `0x5` | — | low | auto-shape | array |
| `0xd2` | — | low | auto-shape | array |
| `0xd7` | — | low | auto-shape | array |
@@ -116,7 +127,6 @@
| `0x4f1` | — | low | auto-shape | array |
| `0x5c3` | — | low | auto-shape | array |
| `0x69e` | — | low | auto-shape | array |
| `0x6d3` | — | low | auto-shape | array |
| `0x6fe` | — | low | auto-shape | array |
| `0x712` | — | low | auto-shape | array |
| `0x7db` | — | low | auto-shape | array |
@@ -250,15 +260,10 @@
| `0xe7172` | — | low | auto-shape | array |
| `0xe7305` | — | low | auto-shape | array |
| `0xe730e` | — | low | auto-shape | array |
| `0xe7311` | — | low | auto-shape | array |
| `0xe7325` | — | low | auto-shape | array |
| `0xe7357` | — | low | auto-shape | array |
| `0xe7389` | — | low | auto-shape | array |
| `0xe73bb` | — | low | auto-shape | array |
| `0xe73ed` | — | low | auto-shape | array |
| `0xe741f` | — | low | auto-shape | array |
| `0xe7451` | — | low | auto-shape | array |
| `0xe7483` | — | low | auto-shape | array |
| `0xe770d` | — | low | auto-shape | array |
| `0xe773f` | — | low | auto-shape | array |
| `0xe775d` | — | low | auto-shape | array |
@@ -356,6 +361,7 @@
| address | name | conf | source | usage |
|---|---|---|---|---|
| `0x32f1` | difficulty_index | high | investigation | GAMESTART's three-way difficulty selection. SETEN uses the zero-based index to select difficulty stat adjustments; FIELD indexes three-column stage-enemy arrays with it, and SETOBJ tests it against each object's difficulty mask. |
| `0x6718` | selected_party_slot | high | investigation | Current/selected slot in the 100-entry party-unit arrays. UNITECH chooses a free slot here before populating it; CHMENU replaces it with the selected sorted roster slot, then uses it to index party_slot_flags, party_slot_character_id, and companion per-slot tables. A natural New Game enters SC0000 with slot 2 selected. |
| `0x8c877` | current_item_id | high | investigation | Shared item-id argument/selection slot. Item menus and gameplay scripts write a chosen item id, use it to index ITINIT arrays, and dispatch through item_handler_script_id; ITMES compares it against all 287 item ids to select the matching player-facing title and description. |
| `0xa6e59` | current_skill_id | high | investigation | Shared skill-id argument/selection slot. Skill menus and combat scripts write the chosen skill id and use it to index SKINIT arrays; SKMES compares it against all 131 skill ids to select the matching player-facing title and description. |
@@ -5013,8 +5019,6 @@
| `0xe6afe` | — | med | auto-shape | record-table[stride 10] |
| `0xe6fe2` | — | med | auto-shape | record-table[stride 3] |
| `0xe71d6` | — | med | auto-shape | record-table[stride 3] |
| `0xe74b5` | — | med | auto-shape | record-table[stride 7] |
| `0xe7613` | — | med | auto-shape | record-table[stride 5] |
| `0xe7889` | — | med | auto-shape | record-table[stride 3] |
| `0xe78e3` | — | med | auto-shape | record-table[stride 3] |
| `0xe793d` | — | med | auto-shape | record-table[stride 7] |

View File

@@ -255,12 +255,30 @@ cell destinations, and 37 footer-array destinations across the 74 records. The r
the first nonempty victory-condition string, making consumer/value correlations readable without inventing
a stage-name field.
The strongest consumer-backed meanings are curated in `globals.toml`: `0x27b9..0x27bc` are the two
victory and two defeat-condition lines rendered by AIM/FIELD; `0xe7302` is passed by FIELD to `play-bgm`;
`0xe730c` is the turn limit displayed by DRAWCHP and checked by FIELD; and `0xe730d` selects defeat versus
forced-retreat clear when that limit expires. The remaining stage scalars and buffer families stay raw until
their FIELD/SETEN/SETOBJ/DRAWMAP consumers support stable names. In particular, repeated asset-like values
inside `0xe7311` are not promoted merely because they resolve through SYS4INI.
The strongest header meanings are curated in `globals.toml`: `0x27b9..0x27bc` are the two victory and two
defeat-condition lines rendered by AIM/FIELD; `0xe7302` is passed by FIELD to `play-bgm`; `0xe730c` is the
turn limit displayed by DRAWCHP and checked by FIELD; and `0xe730d` selects defeat versus forced-retreat
clear when that limit expires. STAGECLEAR establishes `0xe7303` as the target/par turn count and scales
`0xe7304`'s persistent reward increment by performance against that target. FIELD establishes `0xe730b` as
the gate that disables its already-cleared-stage retreat/replay conversion.
The first map/object pass resolves seven more buffer families. FIELD loads `0xe7311[1..19]` into tiled
surface slots and DRAWMAP selects those surfaces through terrain metadata, proving it is the current stage's
map-texture override list: positive values are SYS4INI resource ids, zero disables a slot, and -1 selects
the shared fallback. DRAWOBJ converts `0xe7325` and `0xe7357` to map-space coordinates, while
SETOBJ/DRAWOBJ/FIELD use `0xe7389` to index shared object definitions. They are object tile X, tile Y, and
type id. SETOBJ tests the `{3,4,7}` masks in `0xe7483` against GAMESTART's three-way `difficulty_index`,
then applies seven required and five forbidden one-based ids from `0xe74b5`/`0xe7613` against the shared
`story_event_flags` bank.
Generated INIT records now retain their raw `fields`/`record_fields`/buffer keys and additionally expose a
flat `semantic_fields` projection joined through the top-level `field_semantics` map. For STINIT, the four
confirmed parallel buffers plus both prerequisite tables are also assembled into 2,312
`object_placements` across 66 stages. Each placement contains its slot, type, tile coordinates, difficulty
mask, and populated positive/negative story prerequisites. The four still-unresolved object-parameter
buffers remain attached under `unknown_fields`, so this convenience view loses no evidence or invents names.
The next structural batch is the separate 30-slot enemy-spawn schema and its difficulty-dependent footer
arrays.
### The curated registry — `vm-map/globals.toml` (2026-07-07)
@@ -307,11 +325,11 @@ are *not* story flags — the miner over-tags them; they are recategorized `unkn
The v1 map labels *shapes and tables*; the next increments add *meaning*, cheapest first:
1. **Continue INIT semantics by evidence density.** ITINIT/SKINIT, the confirmed EBINIT row layouts, and
STINIT's mixed stage records now have machine-readable investigation surfaces. Next trace STINIT's
highest-coverage object/enemy/map buffers through FIELD, SETOBJ, SETEN, and DRAWMAP; isolate EBINIT's
remaining voice/action slots only when their routed consumers make them distinguishable. Investigate
unread enums and signed classes only when consumer or native evidence appears. Preserve explicit joins
and do not infer meaning from column position alone.
STINIT's mixed stage records now have machine-readable investigation surfaces. Next assemble STINIT's
30-slot enemy-spawn schema through FIELD/SETEN/ADDEN, including its three-difficulty footer arrays; trace
the four remaining specialized object parameters only when their type-specific branches distinguish
them. Isolate EBINIT's remaining voice/action slots under the same rule. Preserve explicit joins and do
not infer meaning from column position alone.
2. **Extend message-table joins beyond the completed ITMES/SKMES pair** (`VIMES`, other id dispatchers, …)
and fold in other `set-string`/`copy-to-global` writers not covered by the `*INIT` set.
3. **Label 2D record tables by their readers** — cross-reference which scripts read each

View File

@@ -749,7 +749,11 @@ The remaining EBINIT unknowns are now the unread `0x7843e` enum and the signed m
enemy AI appears to live outside the static EBINIT schema. STINIT's separate mixed parser is now complete:
all 74 sparse stage ids retain their victory/defeat strings, six scalars, fixed-buffer cells, and 1,396
footer-array copies. AIM/FIELD/DRAWCHP consumers establish the four condition slots, stage BGM, turn limit,
and turn-limit outcome; the object/enemy/map buffer families remain an evidence-driven follow-up.
and turn-limit outcome. The next pass establishes target clear turns, performance reward, replay behavior,
the 20-slot map-texture override list, object type/tile/difficulty fields, and seven required plus five
forbidden story prerequisites. Generated records now carry a joined `semantic_fields` view and assemble
2,312 object placements across 66 stages while retaining four unresolved type-specific parameters by raw
address. The separate 30-slot enemy-spawn family remains the next evidence-driven batch.
Once the natural spine and first gameplay loop are trustworthy, broaden in independent tracks:
@@ -759,8 +763,8 @@ Once the natural spine and first gameplay loop are trustworthy, broaden in indep
- Unit statistics, equipment, inventory, skills, magic, heroine forms, and progression.
- Combat resolution presentation, enemy turns/AI services, and win/loss transitions.
- A full chapter of ADV and the scene types encountered between gameplay segments.
- Remaining STINIT object/enemy/map buffer semantics and other data schemas when their runtime consumers make
them necessary.
- Remaining STINIT enemy-spawn and specialized object-parameter semantics, plus other data schemas when their
runtime consumers make them necessary.
The high-level Phase B direction remains canonical in `docs/remake-architecture-and-roadmap.md`; this file
only provides the execution framework.

View File

@@ -70,7 +70,9 @@ contains only genuine parallel arrays.
STINIT uses the separate mixed shape: 74 sparse `scjump_progress_a` branches load one current-stage work
buffer rather than parallel per-id arrays. Its generated records preserve four mission-condition strings,
six scalars, cells in 29 preallocated buffers, and all 1,396 length-prefixed footer-array copies. Confirmed
stage field meanings and the evidence workflow live in `docs/name-resolution.md`.
stage field meanings and the evidence workflow live in `docs/name-resolution.md`. The first semantic join
now assembles the confirmed type/X/Y/difficulty and story-prerequisite buffers into 2,312 object placements
across 66 stages; unknown type-specific parameters remain raw, and the enemy-spawn buffers remain separate.
### Message/string tables (`*MES`)
`ITMES` (64 KB — item text), `VIMES` (43 KB), `EIMES` (37 KB), `SKMES` (31 KB — skill

View File

@@ -55,7 +55,7 @@ All opcode knowledge (ABI, semantics, provenance, `depends_on`) is hand-edited *
|---|---|---|---|
| `extract_phase2.py` | Batch: disassembly + text corpora for every script. | `extract_phase2.py` | corpus → `build/disasm/*.asm`, `build/text/{dialogue.jsonl,strings.jsonl,*.strings.txt}`, `build/manifest.json` |
| `extract_message_table.py` | Discover a repeated global-id dispatch chain such as ITMES/SKMES, reconstruct player-facing title/description lines (including furigana surface text and readings), and emit an ID-keyed message table with bytecode provenance. | `extract_message_table.py <MES> [OUTNAME]` | `<MES>.BIN``build/data/<OUTNAME>.json` |
| `extract_init.py` | Parse a `*INIT` data table (auto-detects name / numeric / footer / mixed shape). Name tables infer their reserved record span, preserve sparse one-based runtime ids, distinguish lookup bases from first written cells, statically evaluate direct and negative-value writes, and separate parallel `fields` from linked row-major `record_fields`. Mixed tables recover selector-dispatched records, condition strings, scalars, preallocated buffer cells, consumer-confirmed strides, and length-prefixed footer arrays; STINIT is the first such table. ITINIT and SKINIT join ITMES/SKMES messages; top-level `field_semantics` maps raw keys to canonical global/column names. Refreshes the generated data index. | `extract_init.py <TABLE> [OUTNAME] [--mode …]` | `<TABLE>.BIN` plus matching `<MES>.BIN` when supported + `build/globals.json``build/data/<OUTNAME>.json`, `build/data/README.md` |
| `extract_init.py` | Parse a `*INIT` data table (auto-detects name / numeric / footer / mixed shape). Name tables infer their reserved record span, preserve sparse one-based runtime ids, distinguish lookup bases from first written cells, statically evaluate direct and negative-value writes, and separate parallel `fields` from linked row-major `record_fields`. Mixed tables recover selector-dispatched records, condition strings, scalars, preallocated buffer cells, consumer-confirmed strides, and length-prefixed footer arrays; STINIT additionally assembles confirmed buffers into `object_placements`. ITINIT and SKINIT join ITMES/SKMES messages. Top-level `field_semantics` maps raw keys to canonical global/column names, while each record's `semantic_fields` is the generated name-keyed join; raw keys remain provenance. OUTNAME accepts a stem or one `.json` suffix and rejects paths. Refreshes the generated data index. | `extract_init.py <TABLE> [OUTNAME] [--mode …]` | `<TABLE>.BIN` plus matching `<MES>.BIN` when supported + `build/globals.json``build/data/<OUTNAME>.json`, `build/data/README.md` |
| `init_table_profile.py` | Build the static investigation surface for an extracted name/numeric/mixed table: message coverage, per-scalar/string/array-cell/footer-array population and value distributions, representative records, and direct opcode/script consumers. `--message-query REGEX` shows every matching name/message beside all populated fields for semantic correlation. Findings are evidence only; confirmed meanings go in `vm-map/globals.toml`. | `init_table_profile.py <TABLE> [--build] [--limit N] [--message-query REGEX]` | `build/data/<TABLE>.json` + corpus → stdout; with `--build`, `build/data/<TABLE>-field-profile.{json,md}` |
| `test_extract_init.py`, `test_init_table_profile.py` | Regression checks for sparse one-based and mixed selector-dispatched INIT extraction, MES reconstruction/joins, footer-array accounting, and field/message profiling. | run each directly | — |
| `global_map.py` | Build the partial global-variable name map from static evidence. | `global_map.py` | corpus + `build/data/``build/global-var-map.{json,md}` |

View File

@@ -54,7 +54,7 @@
- [x] **2.0 — Project structure.** Established `docs/`, `build/{disasm,text,data,scripts-json}/`, `godot/`; game install stays read-only in place. Also relaxed the loader magic check to the `SYS4` family (`SYS4424` patch scripts now parse — was silently skipping 5 scripts).
- [x] **2.1 — Text corpora.** `tools/extract_phase2.py` → 481/481 scripts: full disassembly (`build/disasm/*.asm`), per-script strings, `build/text/dialogue.jsonl` (**30,057 show-text lines** — the translation corpus), `build/text/strings.jsonl` (38,449 strings tagged by source opcode), `build/manifest.json`.
- [x] **2.2 — `*INIT` data tables → JSON.** `tools/extract_init.py` auto-detects table shape (`name`/`numeric`/`footer`/`mixed`) → **SKINIT (131 skills), ITINIT (287 items), EBINIT (277 units)** [name: sparse one-based name/description/fields], **CGINIT (379 CG entries)** [numeric: index-keyed columns], **MPINIT (1472 map records)** [footer: length-prefixed arrays], and **STINIT (74 stages)** [mixed: selector-dispatched strings/scalars/buffer cells/footer arrays]. Validated; see `build/data/README.md`. Raw addresses remain bytecode provenance; confirmed semantics come from `vm-map/globals.toml`.
- [x] **2.3 — Extract `STINIT`'s 74 sparse stage records.** The mixed mode identifies the dominant `scjump_progress_a` dispatch, recovers 29 preallocated buffer layouts (including six consumer-confirmed row strides), and keeps four condition strings, six scalars, fixed-buffer writes, and all 1,396 footer-array copies separated by stage id. The generated profile supplies population/value and direct-consumer evidence; victory/defeat lines, stage BGM, turn limit, and expiry outcome are curated.
- [x] **2.3 — Extract `STINIT`'s 74 sparse stage records.** The mixed mode identifies the dominant `scjump_progress_a` dispatch, recovers 29 preallocated buffer layouts (including six consumer-confirmed row strides), and keeps four condition strings, six scalars, fixed-buffer writes, and all 1,396 footer-array copies separated by stage id. Profiles supply population/value and direct-consumer evidence. Header/map semantics now cover conditions, BGM, turn/replay/clear-reward settings, and map texture overrides; four parallel object fields plus 7 required/5 forbidden story prerequisites assemble into 2,312 object placements. Raw address views remain alongside generated `semantic_fields`.
- [x] **2.4 — Partial global-var map BUILT + wired into the disassembler.** `tools/global_map.py``build/global-var-map.{json,md}` (16,354/49,435 globals labelled: string tables, `*INIT` field arrays, 122 record tables w/ strides, current-entity index pointers). `sys4load` renders the labels inline (`=rec[s30]`, `=current-entity-index?`). See `docs/name-resolution.md`.
- [ ] **2.5 — Grow the global-var map (future, incremental).** Static first: fold in `*MES` writers; label 2D record tables by their reader scripts. Then Frida to name *which stat* each field is. Full detail: `docs/name-resolution.md` → "Future step — growing the map". Also deferred: `call-script` id→name resolution (engine-level — SCJUMP.BIN decode or Frida; see `docs/name-resolution.md` #1).

View File

@@ -54,6 +54,16 @@ def resolve(name: str) -> Path:
raise SystemExit(f"not found: {name}.BIN")
def normalize_outname(value: str) -> str:
"""Accept a generated-file stem, not a path; tolerate one `.json` suffix."""
if not value or Path(value).name != value or "/" in value or "\\" in value:
raise ValueError("OUTNAME must be a file stem, not a path")
outname = value.removesuffix(".json")
if not outname or outname in {".", ".."}:
raise ValueError("OUTNAME must be a nonempty file stem")
return outname
def _val(arg):
"""Render an operand as an int (immediate) or a {type,value} ref."""
t, v = arg
@@ -470,7 +480,9 @@ def _global_registry() -> dict:
return {}
def field_semantics(records: list[dict]) -> dict[str, str]:
def field_semantics(
records: list[dict], array_layouts: dict[str, dict] | None = None
) -> dict[str, str]:
"""Map raw extracted field keys to canonical semantic names when available."""
keys = {
key
@@ -494,7 +506,17 @@ def field_semantics(records: list[dict]) -> dict[str, str]:
if not name:
continue
if len(parts) == 2:
name = f"{name}.index_{parts[1]}"
index = int(parts[1])
layout = (array_layouts or {}).get(f"0x{int(parts[0], 16):x}", {})
if stride := layout.get("stride"):
row, column = divmod(index, stride)
column_name = entry.get("columns", {}).get(
str(column), f"column_{column}"
)
name = f"{name}.row_{row}.{column_name}"
else:
index_name = entry.get("columns", {}).get(str(index), f"index_{index}")
name = f"{name}.{index_name}"
elif len(parts) == 3:
column = parts[2]
column_name = entry.get("columns", {}).get(column, f"column_{column}")
@@ -503,6 +525,74 @@ def field_semantics(records: list[dict]) -> dict[str, str]:
return semantics
def attach_semantic_fields(records: list[dict], semantics: dict[str, str]) -> None:
"""Add a generated name-keyed view while retaining raw address provenance."""
containers = (
"string_fields", "fields", "array_fields", "footer_arrays", "record_fields"
)
for record in records:
semantic_fields = {}
for container in containers:
for key, value in record.get(container, {}).items():
if semantic_name := semantics.get(key):
if semantic_name in semantic_fields:
raise ValueError(
f"record {record['id']}: duplicate semantic field {semantic_name}"
)
semantic_fields[semantic_name] = value
if semantic_fields:
record["semantic_fields"] = semantic_fields
else:
record.pop("semantic_fields", None)
def attach_stage_object_placements(records: list[dict]) -> None:
"""Assemble STINIT's parallel object buffers into modder-facing slot records."""
known_fields = {
"type_id": "0xe7389",
"tile_x": "0xe7325",
"tile_y": "0xe7357",
"difficulty_mask": "0xe7483",
}
unknown_bases = ("0xe73bb", "0xe73ed", "0xe741f", "0xe7451")
for record in records:
fields = record.get("array_fields", {})
objects = []
for slot in range(1, 50):
type_key = f"{known_fields['type_id']}/{slot}"
if type_key not in fields:
continue
obj = {"slot": slot}
for semantic_name, base in known_fields.items():
if (key := f"{base}/{slot}") in fields:
obj[semantic_name] = fields[key]
required = [
fields[key]
for column in range(7)
if (key := f"0xe74b5/{slot * 7 + column}") in fields
and fields[key] > 0
]
forbidden = [
fields[key]
for column in range(5)
if (key := f"0xe7613/{slot * 5 + column}") in fields
and fields[key] > 0
]
if required:
obj["required_story_flags"] = required
if forbidden:
obj["forbidden_story_flags"] = forbidden
unknown = {
base: fields[f"{base}/{slot}"]
for base in unknown_bases
if f"{base}/{slot}" in fields
}
if unknown:
obj["unknown_fields"] = unknown
objects.append(obj)
record["object_placements"] = objects
def write_data_index(data_dir: Path) -> None:
"""Regenerate the disposable build/data index from current table JSONs."""
tables = []
@@ -549,10 +639,13 @@ def write_data_index(data_dir: Path) -> None:
"title, description, furigana, and bytecode dispatch offset separately from the",
"short description stored by the INIT script.",
"Top-level `field_semantics` maps raw array/row-column keys to canonical machine-readable",
"names from `vm-map/globals.toml`; raw keys remain intact as bytecode provenance.",
"names from `vm-map/globals.toml`; each record's generated `semantic_fields` is the joined",
"name-keyed convenience view. Raw keys remain intact as bytecode provenance.",
"",
"Mixed-mode tables preserve the sparse selector id, branch offset, condition strings,",
"scalar fields, cells within preallocated buffers, and length-prefixed footer arrays.",
"STINIT additionally joins the confirmed parallel object buffers into per-slot",
"`object_placements`; unresolved type-specific parameters remain in `unknown_fields`.",
"Use `tools/init_table_profile.py <TABLE> --build` to generate value/population and",
"direct-consumer evidence.",
"",
@@ -579,7 +672,10 @@ def main() -> int:
if not argv:
raise SystemExit(__doc__)
name = argv[0].upper().removesuffix(".BIN")
outname = argv[1] if len(argv) > 1 else name
try:
outname = normalize_outname(argv[1]) if len(argv) > 1 else name
except ValueError as error:
raise SystemExit(str(error)) from error
scr = sys4load.load(resolve(name))
mode = mode_arg or detect_mode(scr)
@@ -597,7 +693,10 @@ def main() -> int:
)
cols = sorted({c for r in recs for c in r.get("fields", {})}, key=lambda h: int(h, 16))
semantics = field_semantics(recs)
semantics = field_semantics(recs, meta.get("array_layouts"))
attach_semantic_fields(recs, semantics)
if mode == "mixed" and name == "STINIT":
attach_stage_object_placements(recs)
out = {"table": name, "source": scr.path.name, "magic": scr.magic, "mode": mode,
"record_count": len(recs), **meta,
"field_columns": cols if mode != "footer" else None,

View File

@@ -66,6 +66,18 @@ def test_static_negative_write() -> None:
"INIT subtraction writes preserve negative values")
def test_output_name_validation() -> None:
check(extract_init.normalize_outname("ITINIT.json") == "ITINIT",
"INIT output names tolerate one JSON suffix")
try:
extract_init.normalize_outname("build/data/ITINIT.json")
except ValueError:
rejected = True
else:
rejected = False
check(rejected, "INIT output names reject nested paths")
def test_real_mixed_table() -> None:
script = sys4load.load(extract_init.resolve("STINIT"))
check(extract_init.detect_mode(script) == "mixed",
@@ -93,6 +105,21 @@ def test_real_mixed_table() -> None:
"STINIT length-prefixed footer arrays retain their destination")
check(sum(len(record.get("footer_arrays", {})) for record in records) == 1396,
"STINIT accounts for every footer-array copy")
extract_init.attach_stage_object_placements(records)
first_object = records[0]["object_placements"][0]
check(first_object == {
"slot": 1,
"type_id": 1,
"tile_x": 13,
"tile_y": 1,
"difficulty_mask": 7,
"unknown_fields": {"0xe73bb": 1},
}, "STINIT joins confirmed object buffers into one placement record")
stage2_slot3 = next(
obj for obj in records[1]["object_placements"] if obj["slot"] == 3
)
check(stage2_slot3["required_story_flags"] == [902],
"STINIT object placements join positive story prerequisites")
def test_real_message_tables() -> None:
@@ -167,11 +194,29 @@ def test_field_semantics() -> None:
"parallel INIT fields expose canonical semantic names")
check(semantics["0x9f541/14/8"] == "item_stat_modifiers.critical_chance",
"row-table columns expose canonical semantic names")
extract_init.attach_semantic_fields(items, semantics)
check(items[0]["semantic_fields"]["item_sort_key"] == 10,
"records expose a joined semantic field view")
stages, meta = extract_init.extract_mixed(
sys4load.load(extract_init.resolve("STINIT"))
)
stage_semantics = extract_init.field_semantics(
stages, meta["array_layouts"]
)
check(stage_semantics["0xe7325/1"] == "stage_object_tile_x.index_1",
"mixed buffer cells expose canonical semantic names")
check(
stage_semantics["0xe74b5/21"]
== "stage_object_required_story_flags.row_3.required_flag_1",
"mixed row buffers expose row and column semantics",
)
if __name__ == "__main__":
test_real_name_tables()
test_static_negative_write()
test_output_name_validation()
test_real_mixed_table()
test_real_message_tables()
test_message_join()

View File

@@ -1671,3 +1671,128 @@ value_domain = "{0,1}; 0 defeat on expiry, 1 forced-retreat clear on expiry"
usage = "STINIT mode paired with stage_turn_limit. Stage 1 stores 0 and describes 50-turn expiry as defeat; stage 2 stores 1 and explicitly describes 15-turn expiry as a forced-retreat clear."
source = "investigation"
confidence = "high"
[[global]]
address = "0x32f1"
name = "difficulty_index"
category = "index-pointer"
type = "int"
value_domain = "{0,1,2}"
usage = "GAMESTART's three-way difficulty selection. SETEN uses the zero-based index to select difficulty stat adjustments; FIELD indexes three-column stage-enemy arrays with it, and SETOBJ tests it against each object's difficulty mask."
source = "investigation"
confidence = "high"
[[global]]
address = "0x6d3"
name = "story_event_flags"
category = "data-table"
type = "int[]"
value_domain = "mostly {0,1}"
usage = "Shared one-based story/event flag bank. Progression, recruitment, item, and stage logic read or write individual cells; STINIT object prerequisite ids are decremented by one before SETOBJ tests this array."
source = "investigation"
confidence = "high"
[[global]]
address = "0xe7303"
name = "stage_target_clear_turns"
category = "data-table"
type = "int"
value_domain = "0..50; 0 disables turn-performance scaling"
usage = "STINIT's target/par turn count. STAGECLEAR divides elapsed turns by this value to derive a performance multiplier; when replaying an already-cleared ordinary stage, FIELD also uses it as the forced-retreat turn limit."
source = "investigation"
confidence = "high"
[[global]]
address = "0xe7304"
name = "stage_clear_performance_bonus"
category = "data-table"
type = "int"
value_domain = "0..20"
usage = "Base stage-clear reward increment. STAGECLEAR multiplies it by the turn-performance percentage derived from stage_target_clear_turns, divides by 100, and adds the result to the capped persistent reward counter at 0x6719. The counter's player-facing resource name remains unresolved."
source = "investigation"
confidence = "med"
[[global]]
address = "0xe730b"
name = "stage_replay_rules_disabled"
category = "data-table"
type = "int"
value_domain = "{0,1}; 1 preserves the original mission rules after clear"
usage = "FIELD's already-cleared-stage override gate. For a cleared stage with value 0, FIELD replaces the mission with an all-party retreat objective and a stage_target_clear_turns forced-retreat limit; value 1 suppresses that replay conversion and related revisit handling."
source = "investigation"
confidence = "high"
[[global]]
address = "0xe7311"
name = "stage_map_texture_asset_overrides"
category = "data-table"
type = "int[20]"
value_domain = "SYS4INI resource id when positive, 0 disables the slot, -1 selects the shared fallback asset"
usage = "Twenty current-stage map texture slots. FIELD loads positive resource ids into tiled surface slots 0x52+index, skips zero, and substitutes the shared per-index fallback for -1; DRAWMAP selects and draws those surfaces through the terrain-to-texture-slot map."
source = "investigation"
confidence = "high"
[[global]]
address = "0xe7325"
name = "stage_object_tile_x"
category = "data-table"
type = "int[50]"
value_domain = "map tile X coordinate, 1..25 in the shipped stages"
usage = "X coordinate for each current-stage object slot. DRAWOBJ converts it to centered map-space pixels; FIELD and CALCOCC use it with stage_object_tile_y for object interaction and occupancy."
source = "investigation"
confidence = "high"
[[global]]
address = "0xe7357"
name = "stage_object_tile_y"
category = "data-table"
type = "int[50]"
value_domain = "map tile Y coordinate, 1..800 in the shipped stages"
usage = "Y coordinate for each current-stage object slot. DRAWOBJ converts it to centered map-space pixels; FIELD and CALCOCC use it with stage_object_tile_x for object interaction and occupancy."
source = "investigation"
confidence = "high"
[[global]]
address = "0xe7389"
name = "stage_object_type_id"
category = "data-table"
type = "int[50]"
value_domain = "object definition id 1..46"
usage = "Object-definition id for each current-stage object slot. SETOBJ decides whether the slot exists, while DRAWOBJ and FIELD use the id to select shared object graphics, dimensions, animation, collision, and behavior metadata."
source = "investigation"
confidence = "high"
[[global]]
address = "0xe7483"
name = "stage_object_difficulty_mask"
category = "data-table"
type = "int[50]"
value_domain = "{3,4,7}; bit 0/1/2 enables the object on the corresponding difficulty"
usage = "Per-object difficulty inclusion mask. SETOBJ uses check-bit with difficulty_index and rejects the object slot when the selected difficulty bit is absent."
source = "investigation"
confidence = "high"
depends_on = ["0x32f1"]
[[global]]
address = "0xe74b5"
name = "stage_object_required_story_flags"
category = "data-table"
type = "int[50][7]"
columns = { "0" = "required_flag_1", "1" = "required_flag_2", "2" = "required_flag_3", "3" = "required_flag_4", "4" = "required_flag_5", "5" = "required_flag_6", "6" = "required_flag_7" }
value_domain = "one-based story_event_flags ids; zero means no requirement"
usage = "Seven positive prerequisites per stage object. SETOBJ subtracts one from every populated id and suppresses the object unless the corresponding story_event_flags cell equals 1."
source = "investigation"
confidence = "high"
depends_on = ["0x6d3"]
[[global]]
address = "0xe7613"
name = "stage_object_forbidden_story_flags"
category = "data-table"
type = "int[50][5]"
columns = { "0" = "forbidden_flag_1", "1" = "forbidden_flag_2", "2" = "forbidden_flag_3", "3" = "forbidden_flag_4", "4" = "forbidden_flag_5" }
value_domain = "one-based story_event_flags ids; zero means no exclusion"
usage = "Five negative prerequisites per stage object. SETOBJ subtracts one from every populated id and suppresses the object when the corresponding story_event_flags cell equals 1."
source = "investigation"
confidence = "high"
depends_on = ["0x6d3"]