conco/OpenMaidEngine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Decode stage definition tables

Browse Source
This commit is contained in:
gamer147
2026-07-23 23:36:02 -04:00
parent 35080a086e
commit 796b0e850e
12 changed files with 959 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
docs/tools-reference.md
View File

@@ -55,9 +55,9 @@ All opcode knowledge (ABI, semantics, provenance, `depends_on`) is hand-edited *
|---|---|---|---|
| `extract_phase2.py` | Batch: disassembly + text corpora for every script. | `extract_phase2.py` | corpus → `build/disasm/*.asm`, `build/text/{dialogue.jsonl,strings.jsonl,*.strings.txt}`, `build/manifest.json` |
| `extract_message_table.py` | Discover repeated global-id dispatch chains in either fallthrough-body (ITMES/SKMES/MAMES) or branch-target-body (VIMES/EIMES/CIMES) form, reconstruct player-facing text and furigana, preserve guard/body offsets, and emit an ID-keyed message table. Title-bearing tables expose `title`/`description`; EIMES exposes its untitled lines as `summary`/`strategy`; CIMES exposes one untitled multiline `biography`; MAMES exposes untitled `description` text. | `extract_message_table.py <MES> [OUTNAME]` | `<MES>.BIN` → `build/data/<OUTNAME>.json` |
| `extract_init.py` | Parse a `*INIT` data source (auto-detects name / numeric / footer / mixed / rules / dispatch / banked shape). Name tables infer their reserved record span, preserve sparse one-based runtime ids, distinguish lookup bases from first written cells, statically evaluate direct and negative-value writes, and separate parallel `fields` from linked row-major `record_fields`; when flat global ranges overlap, a parallel base established by other records wins over a coincidental row-table cell calculation. Dedicated name-mode schemas recover VIINIT's pre-title glossary prerequisites and CIINIT's profile name, backing-unit, optional portrait, and reserved placement columns. Footer-mode MPINIT recovers its sparse 53-column doubled-coordinate terrain atlas, joins 66 STINIT2 stage rectangles, and resolves terrain ids through LAINIT. Mixed tables recover selector-dispatched records, condition strings, scalars, preallocated buffer cells, consumer-confirmed strides, and length-prefixed footer arrays; STINIT additionally assembles confirmed buffers into `object_placements` and `enemy_spawns`, joins object type names and available descriptions from OBINIT, and exposes consumer-proven tagged payloads (teleport, treasure, card, initial faction, non-triggering hazard/barrier faction, and OBINIT-gated initial object state), preserves engine-dead type-27 writes under `ignored_payload_fields`, and exposes reinforcement schedules and first-clear-only enemy gates. Rules mode decodes CCINIT's source-ordered class-change predicates and output accumulators, joining EBINIT unit names, SKINIT skill names, titles, named stat bonuses, cost deltas, and applied-state slots. Dispatch mode decodes SCINIT's paired decision-to-packed-scene and authored-chapter arrays, preserving source-order overwrites, resolving SYS4INI script names, and cross-checking chapter tags against decoded SCJUMP paths. Banked mode decodes RTINIT's twenty parallel 1000-by-20 movement/battle routine banks, retains all assignments and overwrites, assembles final per-slot steps, resolves provider selectors to RTN_M/RTN_B scripts, and applies selector-scoped schemas where proven. Current schemas cover all nineteen used providers, RTN_M001/002/003/004/005/006/007/008/009/010/011/012/013/014/015/017/051/052/061, including progress-only, randomized roaming, object-slot, coordinate, enemy/ally, treasure and Magic Pillar search, Healing Feather, waypoint, faction-terrain, retreat, reachable normal-attack routing, immediate offensive target/action selection, and immediate allied healing. Parameterless behavior schemas publish an empty `parameter_fields` mapping while still joining `provider_behavior`, target selection, action selection, and completion rules. Provider defaults such as M004's unwritten slot 0, M010's zero/HP resource index, and M013's any-foreign-faction filter are projected explicitly and counted separately from populated source cells. Authored cells proven unread by M001/M008 remain under per-step `ignored_movement_parameters` and contribute to top-level `ignored_movement_parameter_count`; they are not silently discarded or assigned invented meanings. Raw provider banks remain intact, and top-level `movement_provider_parameter_schemas` documents the projection. ITINIT, SKINIT, VIINIT, EBINIT, and CIINIT join their matching MES text. Top-level `field_semantics` maps raw keys to canonical global/column names, while each record's `semantic_fields` is the generated name-keyed join; complete footer copies map to a row and expose its values without the raw provenance wrapper. Raw keys and unresolved tagged payload cells remain provenance. OUTNAME accepts a stem or one `.json` suffix and rejects paths. Refreshes the generated data index. | `extract_init.py <TABLE> [OUTNAME] [--mode …]` | `<TABLE>.BIN` plus EBINIT/SKINIT for CCINIT, OBINIT for STINIT, STINIT2/LAINIT for MPINIT, CDINIT2/STINIT for CDINIT, ITINIT/SCINIT/ILINIT and `build/callscript-names.json` for CDINIT2, BTANINIT/SKINIT/ITINIT and `build/callscript-names.json` for BTANINIT2, BTANINIT2 and `build/callscript-names.json` for BTANINIT, `build/callscript-names.json` for RTINIT/SCINIT, `build/scjump-decisions.json` for SCINIT, matching `<MES>.BIN` when supported, and `build/globals.json` → `build/data/<OUTNAME>.json`, `build/data/README.md` |
| `init_table_profile.py` | Build the static investigation surface for an extracted name/numeric/footer/mixed/rules/dispatch/banked table: per-scalar/string/array-cell/footer-array population and value distributions, representative records, and direct opcode/script consumers. MPINIT's footer specialization summarizes atlas geometry, implicit-zero rows, terrain ids, stage-rectangle joins, sharing, and border cells; CDINIT summarizes selector/list sizes, CDINIT2 joins and story gates, STINIT references, and the runtime scan/clear geometry; CDINIT2 summarizes card-type and effect populations plus item/event/condition/visual join coverage; BTANINIT/BTANINIT2 summarize effect modes/resources, six-slot timeline geometry, delays, duration, skill/weapon joins, and unreferenced authoring. Name tables report message coverage for title/description, summary/strategy, biography, and description-only layouts; rule tables summarize covered units, titled and level-independent rules, threshold/slot distributions, and skill awards; dispatch tables summarize assignments, overwrites, script-name resolution, and SCJUMP chapter agreement; banked tables summarize populated/reserved banks, movement/battle steps and providers, selector-specific semantic coverage, and overwrite conflicts. `--message-query REGEX` searches every supported message field beside all populated fields for semantic correlation. Findings are evidence only; confirmed meanings go in `vm-map/globals.toml`. | `init_table_profile.py <TABLE> [--build] [--limit N] [--message-query REGEX]` | `build/data/<TABLE>.json` + corpus → stdout; with `--build`, `build/data/<TABLE>-field-profile.{json,md}` |
| `test_extract_init.py`, `test_init_table_profile.py` | Regression checks for sparse one-based, mixed selector-dispatched, CCINIT rule, SCINIT dispatch, RTINIT banked, ILINIT condition-matrix, CNINIT unit-name/voice-family, CGINIT gallery, ALINIT alchemy-recipe, AFINIT affinity/progression, CTINIT name-entry-palette, CVINIT character-voice, TRINIT training-action, CDINIT card-generation, CDINIT2 card-effect, BTANINIT/BTANINIT2 battle-animation, and MPINIT terrain-atlas extraction; RECOVER ABI validation; MES reconstruction/joins; footer-array accounting and semantic projection; EBINIT/SKINIT/OBINIT definition joins; STINIT object/enemy joins and tagged-payload behavior; overwrite/provider joins; and field/message/rule/dispatch/banked/map/training/card/battle-animation profiling. | run each directly | — |
| `extract_init.py` | Parse a `*INIT` data source (auto-detects name / numeric / footer / mixed / rules / dispatch / banked shape). Name tables infer their reserved record span, preserve sparse one-based runtime ids, distinguish lookup bases from first written cells, statically evaluate direct and negative-value writes, and separate parallel `fields` from linked row-major `record_fields`; when flat global ranges overlap, a parallel base established by other records wins over a coincidental row-table cell calculation. Dedicated name-mode schemas recover VIINIT's pre-title glossary prerequisites, CIINIT's profile registry, and STINIT2's corrected 74-row stage catalog plus six-line pre/post-clear text matrix, availability gates, geometry, rewards, SCJUMP joins, and loader resources. Footer-mode MPINIT recovers its sparse 53-column doubled-coordinate terrain atlas, joins 66 STINIT2 stage rectangles, and resolves terrain ids through LAINIT. Mixed tables recover selector-dispatched records, condition strings, scalars, preallocated buffer cells, consumer-confirmed strides, and length-prefixed footer arrays; STINIT additionally assembles confirmed buffers into `object_placements` and `enemy_spawns`, joins object type names and available descriptions from OBINIT, and exposes consumer-proven tagged payloads (teleport, treasure, card, initial faction, non-triggering hazard/barrier faction, and OBINIT-gated initial object state), preserves engine-dead type-27 writes under `ignored_payload_fields`, and exposes reinforcement schedules and first-clear-only enemy gates. Rules mode decodes CCINIT's source-ordered class-change predicates and output accumulators, joining EBINIT unit names, SKINIT skill names, titles, named stat bonuses, cost deltas, and applied-state slots. Dispatch mode decodes SCINIT's paired decision-to-packed-scene and authored-chapter arrays, preserving source-order overwrites, resolving SYS4INI script names, and cross-checking chapter tags against decoded SCJUMP paths. Banked mode decodes RTINIT's twenty parallel 1000-by-20 movement/battle routine banks, retains all assignments and overwrites, assembles final per-slot steps, resolves provider selectors to RTN_M/RTN_B scripts, and applies selector-scoped schemas where proven. Current schemas cover all nineteen used providers, RTN_M001/002/003/004/005/006/007/008/009/010/011/012/013/014/015/017/051/052/061, including progress-only, randomized roaming, object-slot, coordinate, enemy/ally, treasure and Magic Pillar search, Healing Feather, waypoint, faction-terrain, retreat, reachable normal-attack routing, immediate offensive target/action selection, and immediate allied healing. Parameterless behavior schemas publish an empty `parameter_fields` mapping while still joining `provider_behavior`, target selection, action selection, and completion rules. Provider defaults such as M004's unwritten slot 0, M010's zero/HP resource index, and M013's any-foreign-faction filter are projected explicitly and counted separately from populated source cells. Authored cells proven unread by M001/M008 remain under per-step `ignored_movement_parameters` and contribute to top-level `ignored_movement_parameter_count`; they are not silently discarded or assigned invented meanings. Raw provider banks remain intact, and top-level `movement_provider_parameter_schemas` documents the projection. ITINIT, SKINIT, VIINIT, EBINIT, and CIINIT join their matching MES text. Top-level `field_semantics` maps raw keys to canonical global/column names, while each record's `semantic_fields` is the generated name-keyed join; complete footer copies map to a row and expose its values without the raw provenance wrapper. Raw keys and unresolved tagged payload cells remain provenance. OUTNAME accepts a stem or one `.json` suffix and rejects paths. Refreshes the generated data index. | `extract_init.py <TABLE> [OUTNAME] [--mode …]` | `<TABLE>.BIN` plus EBINIT/SKINIT for CCINIT, OBINIT for STINIT, ITINIT/SCINIT and `build/callscript-names.json` for STINIT2, STINIT2/LAINIT for MPINIT, CDINIT2/STINIT for CDINIT, ITINIT/SCINIT/ILINIT and `build/callscript-names.json` for CDINIT2, BTANINIT/SKINIT/ITINIT and `build/callscript-names.json` for BTANINIT2, BTANINIT2 and `build/callscript-names.json` for BTANINIT, `build/callscript-names.json` for RTINIT/SCINIT, `build/scjump-decisions.json` for SCINIT, matching `<MES>.BIN` when supported, and `build/globals.json` → `build/data/<OUTNAME>.json`, `build/data/README.md` |
| `init_table_profile.py` | Build the static investigation surface for an extracted name/numeric/footer/mixed/rules/dispatch/banked table: per-scalar/string/array-cell/footer-array population and value distributions, representative records, and direct opcode/script consumers. MPINIT's footer specialization summarizes atlas geometry, implicit-zero rows, terrain ids, stage-rectangle joins, sharing, and border cells; STINIT2 summarizes corrected record/text geometry, mapped/event/main/EX populations, story gates, SCJUMP/loader joins, rewards, and its unresolved column; CDINIT summarizes selector/list sizes, CDINIT2 joins and story gates, STINIT references, and the runtime scan/clear geometry; CDINIT2 summarizes card-type and effect populations plus item/event/condition/visual join coverage; BTANINIT/BTANINIT2 summarize effect modes/resources, six-slot timeline geometry, delays, duration, skill/weapon joins, and unreferenced authoring. Name tables report message coverage for title/description, summary/strategy, biography, and description-only layouts; rule tables summarize covered units, titled and level-independent rules, threshold/slot distributions, and skill awards; dispatch tables summarize assignments, overwrites, script-name resolution, and SCJUMP chapter agreement; banked tables summarize populated/reserved banks, movement/battle steps and providers, selector-specific semantic coverage, and overwrite conflicts. `--message-query REGEX` searches every supported message field beside all populated fields for semantic correlation. Findings are evidence only; confirmed meanings go in `vm-map/globals.toml`. | `init_table_profile.py <TABLE> [--build] [--limit N] [--message-query REGEX]` | `build/data/<TABLE>.json` + corpus → stdout; with `--build`, `build/data/<TABLE>-field-profile.{json,md}` |
| `test_extract_init.py`, `test_init_table_profile.py` | Regression checks for sparse one-based, mixed selector-dispatched, CCINIT rule, SCINIT dispatch, RTINIT banked, ILINIT condition-matrix, CNINIT unit-name/voice-family, CGINIT gallery, ALINIT alchemy-recipe, AFINIT affinity/progression, CTINIT name-entry-palette, CVINIT character-voice, TRINIT training-action, CDINIT card-generation, CDINIT2 card-effect, BTANINIT/BTANINIT2 battle-animation, STINIT2 stage-definition, and MPINIT terrain-atlas extraction; RECOVER ABI validation; MES reconstruction/joins; footer-array accounting and semantic projection; EBINIT/SKINIT/OBINIT definition joins; STINIT object/enemy joins and tagged-payload behavior; overwrite/provider joins; and field/message/rule/dispatch/banked/map/training/card/battle-animation/stage profiling. | run each directly | — |
| `global_map.py` | Build the partial global-variable name map from static evidence. | `global_map.py` | corpus + `build/data/` → `build/global-var-map.{json,md}` |
`extract_init.py` also has name-mode specializations for VIINIT's sparse 200-row glossary table,
@@ -149,6 +149,14 @@ resolve through SYS4INI. The schema records BTL's movie/sprite playback and acto
contract, the engine-dead authored atlas-row count, and sixteen definitions not referenced by
BTANINIT2.
Name-mode STINIT2 emits 74 sparse stage definitions rather than the generic view's 321 apparent
records. It recognizes the separate `string[1000]` title array and `string[1000][6]` description
matrix, retaining three uncleared and three cleared lines per stage. The schema joins numbered,
event-only, and EX presentation; main-progression, unlock-group, and seven-column story gates;
tile/doubled-grid/minimap geometry; point and bronze/silver/gold coin rewards; all 174
entry/clear/failure decisions through SCINIT; and all 74 loader ids to STINIT.BIN. The 66 populated
cells at `0xedc4d` remain raw and explicitly unresolved because no script reads the column.
Footer-mode MPINIT classifies all 1,472 copies as fifty-cell rows within one stride-53 terrain atlas.
It derives grid Y from each destination, retains 127 implicit-zero gaps, and joins STINIT2's inclusive
tile bounds after the consumer-proven two-times coordinate conversion. The output contains 66 named