Join CIINIT and CIMES character profiles

This commit is contained in:
gamer147
2026-07-23 17:57:21 -04:00
parent e84f61d670
commit 29aa9a4da5
13 changed files with 318 additions and 39 deletions

View File

@@ -1,7 +1,7 @@
<!-- DO NOT EDIT -- generated from vm-map/globals.toml by tools/globals_build.py --build -->
# Global Variable Reference (generated)
5041 globals (262 curated, 4779 auto shape-inferred). Source of truth: `vm-map/globals.toml`.
5042 globals (268 curated, 4774 auto shape-inferred). Source of truth: `vm-map/globals.toml`.
## choice-output
@@ -193,6 +193,10 @@
| `0x14cbd8` | battle_routine_forbidden_story_flag_ids | high | investigation | RTINIT battle bank 19. BTRTN subtracts one and rejects the step when the referenced story flag is set. |
| `0x152877` | battle_entity_indices | high | investigation | The two runtime entity rows participating in BTL. battle_actor_side_index and battle_target_side_index select these cells before BTL and CALCDMG access HP, unit definitions, skills, animation state, and voice banks. Columns: 0=side_0, 1=side_1. |
| `0x152879` | battle_selected_skill_ids | high | investigation | Selected skill for each battle side. BTL displays a positive entry through the skill-name table and selects skill-use voice columns; CALCDMG uses the same id for skill parameters. A zero entry follows the equipped/default ordinary-attack path. Columns: 0=side_0, 1=side_1. |
| `0x15a118` | character_profile_unit_ids | high | investigation | CIINIT's profile-to-unit join. INFOCH uses the selected unit id for reveal state, map sprite, short unit descriptions, species, and sex while CIMES remains keyed by the enclosing profile id. |
| `0x15a17c` | character_profile_portrait_asset_ids | high | investigation | Optional CIINIT portrait resource indexed by character profile id. INFOCH draws it when nonzero and otherwise falls back to the backing unit's map-sprite asset. |
| `0x15a1e0` | character_profile_portrait_x_offsets | high | investigation | INFOCH adds the selected cell to the centered portrait x coordinate. The 100-cell spacing to the adjacent CIINIT arrays and the direct indexed reader establish the reserved per-profile placement column. |
| `0x15a244` | character_profile_portrait_y_offsets | high | investigation | INFOCH adds the selected cell to the bottom-aligned portrait y coordinate. The 100-cell spacing to the adjacent CIINIT arrays and the direct indexed reader establish the reserved per-profile placement column. |
| `0x15a2a9` | glossary_topic_unlock_seen_decision_ids | high | investigation | VIINIT's three unlock prerequisites per glossary topic. INFOVO subtracts one from each positive value and exposes the topic when any referenced scene_decision_seen_flags cell is set; debug modes expose the same populated topics unconditionally. Columns: 0=unlock_seen_decision_1, 1=unlock_seen_decision_2, 2=unlock_seen_decision_3. |
| `0x5` | — | low | auto-shape | array |
| `0xd2` | — | low | auto-shape | array |
@@ -225,7 +229,6 @@
| `0x3ad6` | — | low | auto-shape | array |
| `0x4315` | — | low | auto-shape | array |
| `0x4379` | — | low | auto-shape | array |
| `0x45d7` | — | low | auto-shape | array |
| `0x671c` | — | low | auto-shape | array |
| `0x671f` | — | low | auto-shape | array |
| `0x6727` | — | low | auto-shape | array |
@@ -380,10 +383,6 @@
| `0x156214` | — | low | auto-shape | array |
| `0x1565fc` | — | low | auto-shape | array |
| `0x157d6c` | — | low | auto-shape | array |
| `0x15a118` | — | low | auto-shape | array |
| `0x15a17c` | — | low | auto-shape | array |
| `0x15a1e0` | — | low | auto-shape | array |
| `0x15a244` | — | low | auto-shape | array |
| `0x15a501` | — | low | auto-shape | array |
| `0x15a5c9` | — | low | auto-shape | array |
| `0x15a691` | — | low | auto-shape | array |
@@ -405,6 +404,7 @@
| `0x152617` | current_stage_enemy_spawn_slot | high | investigation | Current STINIT enemy-template slot. FIELD selects slots 1..29 while materializing stage units, SETEN records the slot on the runtime entity and reads every parallel enemy buffer through it, and ADDEN uses slot 0 for its special generated unit. |
| `0x15261a` | battle_actor_side_index | high | investigation | Index of the acting side in BTL's two-entry battle arrays. CALCDMG derives battle_target_side_index as 1 minus this value; BTL uses the actor's unit-definition row for attack, skill, critical, and finishing-blow voices. |
| `0x15261b` | battle_target_side_index | high | investigation | Index of the target side in BTL's two-entry battle arrays. CALCDMG sets it to 1 - battle_actor_side_index; BTL applies battle_hp_delta to this side and uses its unit-definition row for damage and defeated voices. |
| `0x15a117` | current_character_profile_id | high | investigation | INFOCH's selected character-information row. It indexes CIINIT's name, unit, portrait, and placement arrays; CIMES compares the same id against all 24 profiles to dispatch the character biography. |
| `0x15a2a8` | current_glossary_topic_id | high | investigation | INFOVO's selected glossary/help topic. It indexes the VIINIT title and presentation arrays, then VIMES compares it against all 65 shipped topic ids to dispatch the full player-facing explanation. |
| `0x15a759` | current_enemy_encyclopedia_unit_id | high | investigation | INFOEN's selected enemy-encyclopedia unit id. It indexes EBINIT presentation and reveal arrays, seeds a temporary runtime entity for stat display, and EIMES compares it against 192 sparse unit ids to dispatch summary and strategy lines. |
@@ -464,6 +464,7 @@
| `0x27ba` | stage_victory_condition_2 | high | investigation | STINIT writes one value for each of its 74 stage records. AIM renders nonempty values after stage_victory_condition_1, and FIELD copies the slot into the current mission-condition display. |
| `0x27bb` | stage_defeat_condition_1 | high | investigation | STINIT writes one value for each of its 74 stage records. AIM renders this line first in the defeat-condition section, and FIELD copies it into the current mission-condition display. |
| `0x27bc` | stage_defeat_condition_2 | high | investigation | STINIT writes one value for each of its 74 stage records. AIM renders nonempty values after stage_defeat_condition_1, and FIELD copies the slot into the current mission-condition display. |
| `0x45d7` | character_profile_names | high | investigation | CIINIT writes the character-information screen's 24 display names. INFOCH lists them by profile id, masks unrevealed rows, and uses the same id to select the backing unit metadata and CIMES biography. |
| `0x463b` | glossary_topic_titles | high | investigation | VIINIT writes the sparse glossary title table. INFOVO renders the selected title, and extract_init joins all 65 populated ids to VIMES's full help text. |
| `0x276` | — | med | auto-shape | string-table (written by SC0130) |
| `0x277` | — | med | auto-shape | string-table (written by SC0130) |

View File

@@ -221,11 +221,12 @@ after the hit check and selects the critical-result state on success. Column 8 i
for both `item_stat_modifiers` and `skill_combat_stat_deltas`; the skill descriptions and matching item
columns also confirm evasion, magic defense, and speed.
ITMES, SKMES, VIMES, and EIMES are now joined back to their INIT records by a reusable id-dispatch
extractor. It recognizes both the ITMES/SKMES fallthrough layout and the VIMES/EIMES layout whose compact
guard block branches forward to separately stored message bodies. All 287 item ids, 131 skill ids, and 65
glossary-topic ids match their INIT definitions exactly; all 192 sparse EIMES ids resolve to EBINIT unit
definitions. `init_table_profile.py --message-query REGEX` puts the complete player-facing text beside
ITMES, SKMES, VIMES, EIMES, and CIMES are now joined back to their INIT records by a reusable id-dispatch
extractor. It recognizes both the ITMES/SKMES fallthrough layout and the VIMES/EIMES/CIMES layout whose
compact guard block branches forward to separately stored message bodies. All 287 item ids, 131 skill ids,
65 glossary-topic ids, and 24 character-profile ids match their INIT definitions exactly; all 192 sparse
EIMES ids resolve to EBINIT unit definitions. `init_table_profile.py --message-query REGEX` puts the
complete player-facing text beside
every populated field, which confirms item/skill condition, resource, range, combat-stat, and restriction
mappings without relying on column position. The same CHMENU trace identifies SKINIT `0xa70b2` as
`skill_change_catalog_eligible`, distinguishes persistent `skill_acquired_flags` from broader
@@ -243,6 +244,14 @@ corresponding `enemy_encyclopedia_revealed_flags` cell is set, and uses the sele
dispatch key. Nine authored EIMES boss variants are valid definitions but are not enabled in that shipped
list.
CIINIT is the character-information definition table. Its 100-cell reserved columns contain 24 populated
profile ids: a displayed name, a backing EBINIT unit id for unit metadata and fallback art, an optional
portrait resource for 19 profiles, and reserved x/y portrait offsets which the shipped initializer leaves
at zero. INFOCH enumerates the nonzero unit-id rows, uses profile reveal state to mask names and details,
draws the selected unit metadata and optional portrait, and then reaches CIMES through INFOMES's handler
router. The selected `current_character_profile_id` is therefore the exact join key, not the backing unit
id. CIMES supplies one untitled multiline `biography` for every one of the 24 CIINIT profiles.
Confirmed row-column meanings are no longer prose-only. The relevant `globals.toml` entries carry a
machine-readable `columns` map; `globals_build.py` preserves it in `build/globals.json`, and
`extract_init.py` emits a top-level `field_semantics` mapping while retaining raw address/stride/column keys
@@ -646,11 +655,11 @@ The v1 map labels *shapes and tables*; the next increments add *meaning*, cheape
and effects. SCINIT closes the progression decision-to-scene join, and RTINIT's twenty movement/battle
banks are structurally decoded with every populated movement-parameter cell classified and all 1,043
shipped movement steps joined to provider behavior. With RTINIT's used-provider surface closed and
ITMES/SKMES/VIMES/EIMES joined, resume the remaining `*MES` writers and other reader-proven 2D tables;
ITMES/SKMES/VIMES/EIMES/CIMES joined, resume the remaining `*MES` writers and other reader-proven 2D tables;
never assign one universal meaning to a parameter bank whose meaning varies by provider selector.
2. **Extend message-table joins beyond the completed ITMES/SKMES/VIMES/EIMES set**, starting with
CIMES's matching branch-target shape, then classify the smaller non-ID-dispatch message scripts and
other `set-string`/`copy-to-global` writers not covered by the `*INIT` set.
2. **Classify the remaining message infrastructure beyond the completed ITMES/SKMES/VIMES/EIMES/CIMES
joins**, starting with the smaller non-ID-dispatch MAMES/INFOMES/MES scripts and then other
`set-string`/`copy-to-global` writers not covered by the `*INIT` set.
3. **Label 2D record tables by their readers** — cross-reference which scripts read each
`rec[sN]` table and infer purpose from context (e.g. RECOVER's 30-wide tables ↔ a
status/recovery system). Static, medium effort.

View File

@@ -729,12 +729,13 @@ state flags, per-action unlock requirements, SCJUMP event ids, normal/brainwashe
per-unit bonus item. Combat tracing also names the shared column-8 item/skill modifier as critical chance.
The message-table tranche adds a reusable extractor for both shipped global-id control-flow layouts and
joins player-facing ITMES/SKMES/VIMES/EIMES text back to INIT records. All 287 item ids, 131 skill ids,
and 65 glossary topic ids match exactly in both directions; all 192 sparse EIMES ids resolve to EBINIT
unit definitions. Joined records retain dispatch and message-body offsets, rendered text, and furigana.
Title-bearing tables expose title/description, while EIMES preserves its untitled lines as enemy summary
and strategy. The dispatch keys establish `current_item_id`, `current_skill_id`,
`current_glossary_topic_id`, and `current_enemy_encyclopedia_unit_id` as high-confidence shared index
joins player-facing ITMES/SKMES/VIMES/EIMES/CIMES text back to INIT records. All 287 item ids, 131 skill
ids, 65 glossary topic ids, and 24 character-profile ids match exactly in both directions; all 192 sparse
EIMES ids resolve to EBINIT unit definitions. Joined records retain dispatch and message-body offsets,
rendered text, and furigana. Title-bearing tables expose title/description, EIMES preserves its untitled
lines as enemy summary and strategy, and CIMES preserves its untitled multiline body as `biography`.
The dispatch keys establish `current_item_id`, `current_skill_id`, `current_glossary_topic_id`,
`current_enemy_encyclopedia_unit_id`, and `current_character_profile_id` as high-confidence shared index
slots.
VIINIT is now extracted as a sparse 200-by-3 glossary definition table: 65 populated names, one required
@@ -743,6 +744,9 @@ INFOVO evaluates those prerequisites against persistent scene-decision seen flag
INFOEN likewise gates EBINIT's enabled enemy-information rows through persistent encounter/reveal flags.
The generated VIINIT profile reports 65/65 VIMES coverage, while EBINIT reports 192/277 EIMES coverage;
the remaining EBINIT rows are valid player/unused unit definitions rather than missing EIMES records.
CIINIT defines 24 rows in four reserved 100-cell columns: displayed names, backing EBINIT unit ids,
optional portrait resources, and default-zero portrait placement offsets. INFOCH consumes that registry,
and the generated CIINIT profile reports complete 24/24 CIMES biography coverage.
The follow-up correlation pass makes that evidence directly queryable with
`init_table_profile.py --message-query REGEX` and moves confirmed item/skill row-column meanings into

View File

@@ -116,12 +116,14 @@ instantiated steps to join.
`ITMES` (64 KB — item text), `VIMES` (43 KB), `EIMES` (37 KB), `SKMES` (31 KB — skill
text), `CIMES` (15 KB), `MAMES`, `INFOMES`, `MES` — where most translatable text
outside scenes lives. The generated message join covers ITMES's 287 item records, SKMES's 131 skill
records, VIMES's complete 65-topic VIINIT glossary, and EIMES's 192 sparse EBINIT enemy-commentary
records. ITMES/SKMES use fallthrough message bodies; VIMES/EIMES place all guards first and branch to
later bodies. VIMES records retain title plus multiline description, while EIMES's untitled two-line
records expose `summary` and `strategy`. CIMES has the same branch-target dispatcher shape and is the
next unjoined message table; the smaller MAMES/INFOMES/MES scripts need separate structural
classification.
records, VIMES's complete 65-topic VIINIT glossary, EIMES's 192 sparse EBINIT enemy-commentary
records, and CIMES's complete 24-profile CIINIT character registry. ITMES/SKMES use fallthrough message
bodies; VIMES/EIMES/CIMES place all guards first and branch to later bodies. VIMES records retain title
plus multiline description, EIMES's untitled two-line records expose `summary` and `strategy`, and
CIMES exposes one untitled multiline `biography`. INFOCH uses each CIINIT profile id to join its displayed
name, backing unit definition, optional portrait, and CIMES text. The smaller MAMES/INFOMES/MES scripts
need separate structural classification; INFOMES is already known to be the information-screen handler
router rather than a text body.
### Battle system
`BTL` (61 KB — main battle loop), `BTRTN`, `ROUND`, `AIM`, `ATSEEK`, `MVSEEK`,

View File

@@ -54,15 +54,16 @@ All opcode knowledge (ABI, semantics, provenance, `depends_on`) is hand-edited *
| Tool | Purpose | Run | Reads → Writes |
|---|---|---|---|
| `extract_phase2.py` | Batch: disassembly + text corpora for every script. | `extract_phase2.py` | corpus → `build/disasm/*.asm`, `build/text/{dialogue.jsonl,strings.jsonl,*.strings.txt}`, `build/manifest.json` |
| `extract_message_table.py` | Discover repeated global-id dispatch chains in either fallthrough-body (ITMES/SKMES) or branch-target-body (VIMES/EIMES) form, reconstruct player-facing text and furigana, preserve guard/body offsets, and emit an ID-keyed message table. Title-bearing tables expose `title`/`description`; EIMES exposes its untitled lines as `summary`/`strategy`. | `extract_message_table.py <MES> [OUTNAME]` | `<MES>.BIN``build/data/<OUTNAME>.json` |
| `extract_init.py` | Parse a `*INIT` data source (auto-detects name / numeric / footer / mixed / rules / dispatch / banked shape). Name tables infer their reserved record span, preserve sparse one-based runtime ids, distinguish lookup bases from first written cells, statically evaluate direct and negative-value writes, and separate parallel `fields` from linked row-major `record_fields`; when flat global ranges overlap, a parallel base established by other records wins over a coincidental row-table cell calculation. Mixed tables recover selector-dispatched records, condition strings, scalars, preallocated buffer cells, consumer-confirmed strides, and length-prefixed footer arrays; STINIT additionally assembles confirmed buffers into `object_placements` and `enemy_spawns`, joins object type names and available descriptions from OBINIT, and exposes consumer-proven tagged payloads (teleport, treasure, card, initial faction, non-triggering hazard/barrier faction, and OBINIT-gated initial object state), preserves engine-dead type-27 writes under `ignored_payload_fields`, and exposes reinforcement schedules and first-clear-only enemy gates. Rules mode decodes CCINIT's source-ordered class-change predicates and output accumulators, joining EBINIT unit names, SKINIT skill names, titles, named stat bonuses, cost deltas, and applied-state slots. Dispatch mode decodes SCINIT's paired decision-to-packed-scene and authored-chapter arrays, preserving source-order overwrites, resolving SYS4INI script names, and cross-checking chapter tags against decoded SCJUMP paths. Banked mode decodes RTINIT's twenty parallel 1000-by-20 movement/battle routine banks, retains all assignments and overwrites, assembles final per-slot steps, resolves provider selectors to RTN_M/RTN_B scripts, and applies selector-scoped schemas where proven. Current schemas cover all nineteen used providers, RTN_M001/002/003/004/005/006/007/008/009/010/011/012/013/014/015/017/051/052/061, including progress-only, randomized roaming, object-slot, coordinate, enemy/ally, treasure and Magic Pillar search, Healing Feather, waypoint, faction-terrain, retreat, reachable normal-attack routing, immediate offensive target/action selection, and immediate allied healing. Parameterless behavior schemas publish an empty `parameter_fields` mapping while still joining `provider_behavior`, target selection, action selection, and completion rules. Provider defaults such as M004's unwritten slot 0, M010's zero/HP resource index, and M013's any-foreign-faction filter are projected explicitly and counted separately from populated source cells. Authored cells proven unread by M001/M008 remain under per-step `ignored_movement_parameters` and contribute to top-level `ignored_movement_parameter_count`; they are not silently discarded or assigned invented meanings. Raw provider banks remain intact, and top-level `movement_provider_parameter_schemas` documents the projection. ITINIT and SKINIT join ITMES/SKMES messages. Top-level `field_semantics` maps raw keys to canonical global/column names, while each record's `semantic_fields` is the generated name-keyed join; complete footer copies map to a row and expose its values without the raw provenance wrapper. Raw keys and unresolved tagged payload cells remain provenance. OUTNAME accepts a stem or one `.json` suffix and rejects paths. Refreshes the generated data index. | `extract_init.py <TABLE> [OUTNAME] [--mode …]` | `<TABLE>.BIN` plus EBINIT/SKINIT for CCINIT, OBINIT for STINIT, `build/callscript-names.json` for RTINIT/SCINIT, `build/scjump-decisions.json` for SCINIT, matching `<MES>.BIN` when supported, and `build/globals.json``build/data/<OUTNAME>.json`, `build/data/README.md` |
| `init_table_profile.py` | Build the static investigation surface for an extracted name/numeric/mixed/rules/dispatch/banked table: per-scalar/string/array-cell/footer-array population and value distributions, representative records, and direct opcode/script consumers. Name tables report message coverage for both title/description and summary/strategy layouts; rule tables summarize covered units, titled and level-independent rules, threshold/slot distributions, and skill awards; dispatch tables summarize assignments, overwrites, script-name resolution, and SCJUMP chapter agreement; banked tables summarize populated/reserved banks, movement/battle steps and providers, selector-specific semantic coverage, and overwrite conflicts. `--message-query REGEX` searches every supported message field beside all populated fields for semantic correlation. Findings are evidence only; confirmed meanings go in `vm-map/globals.toml`. | `init_table_profile.py <TABLE> [--build] [--limit N] [--message-query REGEX]` | `build/data/<TABLE>.json` + corpus → stdout; with `--build`, `build/data/<TABLE>-field-profile.{json,md}` |
| `extract_message_table.py` | Discover repeated global-id dispatch chains in either fallthrough-body (ITMES/SKMES) or branch-target-body (VIMES/EIMES/CIMES) form, reconstruct player-facing text and furigana, preserve guard/body offsets, and emit an ID-keyed message table. Title-bearing tables expose `title`/`description`; EIMES exposes its untitled lines as `summary`/`strategy`; CIMES exposes one untitled multiline `biography`. | `extract_message_table.py <MES> [OUTNAME]` | `<MES>.BIN``build/data/<OUTNAME>.json` |
| `extract_init.py` | Parse a `*INIT` data source (auto-detects name / numeric / footer / mixed / rules / dispatch / banked shape). Name tables infer their reserved record span, preserve sparse one-based runtime ids, distinguish lookup bases from first written cells, statically evaluate direct and negative-value writes, and separate parallel `fields` from linked row-major `record_fields`; when flat global ranges overlap, a parallel base established by other records wins over a coincidental row-table cell calculation. Dedicated name-mode schemas recover VIINIT's pre-title glossary prerequisites and CIINIT's profile name, backing-unit, optional portrait, and reserved placement columns. Mixed tables recover selector-dispatched records, condition strings, scalars, preallocated buffer cells, consumer-confirmed strides, and length-prefixed footer arrays; STINIT additionally assembles confirmed buffers into `object_placements` and `enemy_spawns`, joins object type names and available descriptions from OBINIT, and exposes consumer-proven tagged payloads (teleport, treasure, card, initial faction, non-triggering hazard/barrier faction, and OBINIT-gated initial object state), preserves engine-dead type-27 writes under `ignored_payload_fields`, and exposes reinforcement schedules and first-clear-only enemy gates. Rules mode decodes CCINIT's source-ordered class-change predicates and output accumulators, joining EBINIT unit names, SKINIT skill names, titles, named stat bonuses, cost deltas, and applied-state slots. Dispatch mode decodes SCINIT's paired decision-to-packed-scene and authored-chapter arrays, preserving source-order overwrites, resolving SYS4INI script names, and cross-checking chapter tags against decoded SCJUMP paths. Banked mode decodes RTINIT's twenty parallel 1000-by-20 movement/battle routine banks, retains all assignments and overwrites, assembles final per-slot steps, resolves provider selectors to RTN_M/RTN_B scripts, and applies selector-scoped schemas where proven. Current schemas cover all nineteen used providers, RTN_M001/002/003/004/005/006/007/008/009/010/011/012/013/014/015/017/051/052/061, including progress-only, randomized roaming, object-slot, coordinate, enemy/ally, treasure and Magic Pillar search, Healing Feather, waypoint, faction-terrain, retreat, reachable normal-attack routing, immediate offensive target/action selection, and immediate allied healing. Parameterless behavior schemas publish an empty `parameter_fields` mapping while still joining `provider_behavior`, target selection, action selection, and completion rules. Provider defaults such as M004's unwritten slot 0, M010's zero/HP resource index, and M013's any-foreign-faction filter are projected explicitly and counted separately from populated source cells. Authored cells proven unread by M001/M008 remain under per-step `ignored_movement_parameters` and contribute to top-level `ignored_movement_parameter_count`; they are not silently discarded or assigned invented meanings. Raw provider banks remain intact, and top-level `movement_provider_parameter_schemas` documents the projection. ITINIT, SKINIT, VIINIT, EBINIT, and CIINIT join their matching MES text. Top-level `field_semantics` maps raw keys to canonical global/column names, while each record's `semantic_fields` is the generated name-keyed join; complete footer copies map to a row and expose its values without the raw provenance wrapper. Raw keys and unresolved tagged payload cells remain provenance. OUTNAME accepts a stem or one `.json` suffix and rejects paths. Refreshes the generated data index. | `extract_init.py <TABLE> [OUTNAME] [--mode …]` | `<TABLE>.BIN` plus EBINIT/SKINIT for CCINIT, OBINIT for STINIT, `build/callscript-names.json` for RTINIT/SCINIT, `build/scjump-decisions.json` for SCINIT, matching `<MES>.BIN` when supported, and `build/globals.json``build/data/<OUTNAME>.json`, `build/data/README.md` |
| `init_table_profile.py` | Build the static investigation surface for an extracted name/numeric/mixed/rules/dispatch/banked table: per-scalar/string/array-cell/footer-array population and value distributions, representative records, and direct opcode/script consumers. Name tables report message coverage for title/description, summary/strategy, and biography layouts; rule tables summarize covered units, titled and level-independent rules, threshold/slot distributions, and skill awards; dispatch tables summarize assignments, overwrites, script-name resolution, and SCJUMP chapter agreement; banked tables summarize populated/reserved banks, movement/battle steps and providers, selector-specific semantic coverage, and overwrite conflicts. `--message-query REGEX` searches every supported message field beside all populated fields for semantic correlation. Findings are evidence only; confirmed meanings go in `vm-map/globals.toml`. | `init_table_profile.py <TABLE> [--build] [--limit N] [--message-query REGEX]` | `build/data/<TABLE>.json` + corpus → stdout; with `--build`, `build/data/<TABLE>-field-profile.{json,md}` |
| `test_extract_init.py`, `test_init_table_profile.py` | Regression checks for sparse one-based, mixed selector-dispatched, CCINIT rule, SCINIT dispatch, and RTINIT banked extraction; MES reconstruction/joins; footer-array accounting and semantic projection; EBINIT/SKINIT/OBINIT definition joins; STINIT object/enemy joins and tagged-payload behavior; overwrite/provider joins; and field/message/rule/dispatch/banked profiling. | run each directly | — |
| `global_map.py` | Build the partial global-variable name map from static evidence. | `global_map.py` | corpus + `build/data/``build/global-var-map.{json,md}` |
`extract_init.py` also has a VIINIT name-mode specialization for its sparse 200-row glossary table,
whose up-to-three unlock writes precede each title rather than follow it. Matching message joins are
ITINIT→ITMES, SKINIT→SKMES, VIINIT→VIMES, and EBINIT→EIMES; the last is intentionally sparse.
`extract_init.py` also has name-mode specializations for VIINIT's sparse 200-row glossary table,
whose up-to-three unlock writes precede each title, and CIINIT's 24 populated character-information
profiles within four reserved 100-cell columns. Matching message joins are ITINIT→ITMES,
SKINIT→SKMES, VIINIT→VIMES, EBINIT→EIMES, and CIINIT→CIMES; only the EBINIT join is intentionally sparse.
## VM

View File

@@ -53,13 +53,13 @@
- [x] **2.0 — Project structure.** Established `docs/`, `build/{disasm,text,data,scripts-json}/`, `godot/`; game install stays read-only in place. Also relaxed the loader magic check to the `SYS4` family (`SYS4424` patch scripts now parse — was silently skipping 5 scripts).
- [x] **2.1 — Text corpora.** `tools/extract_phase2.py` → 481/481 scripts: full disassembly (`build/disasm/*.asm`), per-script strings, `build/text/dialogue.jsonl` (**30,057 show-text lines** — the translation corpus), `build/text/strings.jsonl` (38,449 strings tagged by source opcode), `build/manifest.json`.
- [x] **2.2 — `*INIT` data and rule sources → JSON.** `tools/extract_init.py` auto-detects shape (`name`/`numeric`/`footer`/`mixed`/`rules`/`dispatch`/`banked`) → **SKINIT (131 skills), ITINIT (287 items), EBINIT (277 units), OBINIT (46 object definitions)** [name: sparse one-based name/description/fields], **CGINIT (379 CG entries)** [numeric: index-keyed columns], **MPINIT (1472 map records)** [footer: length-prefixed arrays], **STINIT (74 stages)** [mixed: selector-dispatched strings/scalars/buffer cells/footer arrays], **CCINIT (71 class-change rules over 33 units)** [rules: unit/level/state predicates plus title/cost/stat/skill effects], **SCINIT (1,209 final decision rows)** [dispatch: scene resource plus authored chapter metadata and overwrite history], and **RTINIT (172 routine sets)** [banked: twenty movement/battle step banks, provider joins, and overwrite history]. Validated; see `build/data/README.md`. Raw addresses remain bytecode provenance; confirmed semantics come from `vm-map/globals.toml`.
- [x] **2.2 — `*INIT` data and rule sources → JSON.** `tools/extract_init.py` auto-detects shape (`name`/`numeric`/`footer`/`mixed`/`rules`/`dispatch`/`banked`) → **SKINIT (131 skills), ITINIT (287 items), EBINIT (277 units), OBINIT (46 object definitions), CIINIT (24 character-information profiles)** [name: sparse one-based name/description/fields, with CIINIT's dedicated profile registry], **CGINIT (379 CG entries)** [numeric: index-keyed columns], **MPINIT (1472 map records)** [footer: length-prefixed arrays], **STINIT (74 stages)** [mixed: selector-dispatched strings/scalars/buffer cells/footer arrays], **CCINIT (71 class-change rules over 33 units)** [rules: unit/level/state predicates plus title/cost/stat/skill effects], **SCINIT (1,209 final decision rows)** [dispatch: scene resource plus authored chapter metadata and overwrite history], and **RTINIT (172 routine sets)** [banked: twenty movement/battle step banks, provider joins, and overwrite history]. Validated; see `build/data/README.md`. Raw addresses remain bytecode provenance; confirmed semantics come from `vm-map/globals.toml`.
- [x] **2.3 — Extract `STINIT`'s 74 sparse stage records.** The mixed mode identifies the dominant `scjump_progress_a` dispatch, recovers 29 preallocated buffer layouts (including six consumer-confirmed row strides), and keeps four condition strings, six scalars, fixed-buffer writes, and all 1,396 footer-array copies separated by stage id. Profiles supply population/value and direct-consumer evidence. Header/map semantics cover conditions, BGM, turn/replay/clear-reward settings, and map texture overrides. The object join assembles 2,312 placements with type/position/gates, OBINIT names and available descriptions, 604 reinforcement schedules, and typed initial-faction, teleport, treasure, card-list, non-triggering-faction, and initial-object-state payloads; FIELD's dedicated special-spawn path proves the final three populated type-27 tagged writes are engine-dead, so they remain visible as ignored provenance rather than unresolved semantics. The 30-cell enemy family contributes 1,378 joined spawns with unit/faction, placement, difficulty/story gates, level scaling, weighted selection, difficulty-specific movement/battle routine sets, and 485 first-clear-only gates. Raw address views remain alongside generated `semantic_fields`.
- [x] **2.3a — Extract `CCINIT`'s 71 class-change rules.** Rules mode preserves source-order guards and detects the unit id, minimum level, clear applied-state slot, selected title/level, deployment-cost delta, named fourteen-stat bonuses, awarded SKINIT skills, and state slot set by each rule. EBINIT/SKINIT definition joins and the common global semantic projection coexist with raw addresses. CALCCC/ADDEXP establish the apply/report ABI; EVOLVE explains the two level-independent empty-title Lily rules. The generated profile covers 33 units, 69 titled rules, 30 skill awards, three used promotion slots, and 19 populated output fields.
- [x] **2.3b — Extract `SCINIT`'s scene-dispatch registry.** Dispatch mode recognizes 2,179 alternating writes to two 10,000-cell arrays, preserves all 710 overwritten decision ids, and emits 1,209 final rows. The primary column maps decisions to 135 packed SYS4INI ids, all resolved to numbered SC scripts; the parallel column is authored chapter metadata. All 847 live SCJUMP decisions join to rows and 844 final chapter tags agree with the independently decoded paths; three legacy/stale mismatches remain explicit.
- [x] **2.3c — Extract `RTINIT`'s routine-step banks.** Banked mode recognizes twenty parallel 1000-by-20 tables and emits 172 sparse routine-set rows with all 3,336 source assignments and 3,307 final cells. Movement banks join all 1,043 steps to all 19 used RTN_M providers (22 dispatchable); battle banks join fourteen steps to all four RTN_B providers. Activation percentages, progress gates, and required/forbidden story flags are consumer-proven; six empty banks remain explicit. RTN_M001/002/003/004/005/006/007/008/009/010/011/012/013/014/015/017/051/052/061 add selector-specific progress, randomized roaming, object-slot, coordinate, enemy/ally, treasure and object search, Healing Feather, waypoint, faction-terrain, retreat, normal-attack routing, offensive target/action selection, and immediate allied-healing semantics. All 977 populated movement-parameter cells are classified: 974 semantic inputs and three M001/M008 cells proven unread; thirteen unwritten defaults are projected separately while the generic raw banks remain intact.
- [x] **2.4 — Partial global-var map BUILT + wired into the disassembler.** `tools/global_map.py``build/global-var-map.{json,md}` (16,354/49,435 globals labelled: string tables, `*INIT` field arrays, 122 record tables w/ strides, current-entity index pointers). `sys4load` renders the labels inline (`=rec[s30]`, `=current-entity-index?`). See `docs/name-resolution.md`.
- [ ] **2.5 — Grow the global-var map (future, incremental).** Static first: with RTINIT closed at 1,043/1,043 movement steps and ITMES/SKMES/VIMES/EIMES joined to their definitions, decode CIMES's matching branch-target dispatcher, classify the smaller remaining `*MES` scripts, and label other 2D record tables by their readers. Then use Frida only for semantics that static consumers cannot settle. Full detail: `docs/name-resolution.md` → "Future step — growing the map". Packed `call-script` ids, SCJUMP decision-to-scene dispatch, the shipped RTINIT movement-provider join, and the four completed message joins are resolved.
- [ ] **2.5 — Grow the global-var map (future, incremental).** Static first: with RTINIT closed at 1,043/1,043 movement steps and ITMES/SKMES/VIMES/EIMES/CIMES joined to their definitions, classify the smaller remaining `*MES` scripts and label other 2D record tables by their readers. Then use Frida only for semantics that static consumers cannot settle. Full detail: `docs/name-resolution.md` → "Future step — growing the map". Packed `call-script` ids, SCJUMP decision-to-scene dispatch, the shipped RTINIT movement-provider join, and the five completed message joins are resolved.
## Phase 3 — Name the unnamed opcodes *(top ~20 BEFORE Phase 4; the rest on demand)*

View File

@@ -493,12 +493,20 @@ UNIT_STAT_COLUMNS = (
)
MESSAGE_TABLES = {
"CIINIT": "CIMES",
"EBINIT": "EIMES",
"ITINIT": "ITMES",
"SKINIT": "SKMES",
"VIINIT": "VIMES",
}
CHARACTER_PROFILE_NAME_ARRAY_BASE = 0x45D7
CHARACTER_PROFILE_UNIT_ARRAY_BASE = 0x15A118
CHARACTER_PROFILE_PORTRAIT_ARRAY_BASE = 0x15A17C
CHARACTER_PROFILE_PORTRAIT_X_ARRAY_BASE = 0x15A1E0
CHARACTER_PROFILE_PORTRAIT_Y_ARRAY_BASE = 0x15A244
CHARACTER_PROFILE_RECORD_SPAN = 100
VOCABULARY_NAME_ARRAY_BASE = 0x463B
VOCABULARY_RECORD_TABLE_BASE = 0x15A2A9
VOCABULARY_RECORD_STRIDE = 3
@@ -1174,6 +1182,84 @@ def extract_vocabulary(scr):
}
def extract_character_profiles(scr):
"""Extract CIINIT's profile-id keyed character-information registry."""
records = []
by_id = {}
for ins in scr.instructions:
if (
ins.opcode != SET_STRING
or len(ins.args) < 2
or ins.args[0][0] != T_GLOBAL_STRING
):
continue
record_id = ins.args[0][1] - CHARACTER_PROFILE_NAME_ARRAY_BASE
if not (1 <= record_id < CHARACTER_PROFILE_RECORD_SPAN):
raise ValueError(
f"{scr.path.name}: character name outside reserved id span: "
f"0x{ins.args[0][1]:x}"
)
text = scr.strings.get(ins.args[1][1], (None,))[0]
record = {"id": record_id, "name": text, "fields": {}}
records.append(record)
by_id[record_id] = record
integer_arrays = (
CHARACTER_PROFILE_UNIT_ARRAY_BASE,
CHARACTER_PROFILE_PORTRAIT_ARRAY_BASE,
CHARACTER_PROFILE_PORTRAIT_X_ARRAY_BASE,
CHARACTER_PROFILE_PORTRAIT_Y_ARRAY_BASE,
)
for ins in scr.instructions:
write = _static_global_write(ins)
if write is None:
continue
destination, value = write
matched = False
for base in integer_arrays:
relative = destination - base
if 0 <= relative < CHARACTER_PROFILE_RECORD_SPAN:
if relative not in by_id:
raise ValueError(
f"{scr.path.name}: integer write for unnamed character "
f"profile id {relative}"
)
_store_unique(
by_id[relative]["fields"],
f"0x{base:x}",
value,
relative,
)
matched = True
break
if not matched:
raise ValueError(
f"{scr.path.name}: unexpected integer write 0x{destination:x}"
)
return records, {
"schema": "character-information-profiles",
"name_array_base": f"0x{CHARACTER_PROFILE_NAME_ARRAY_BASE:x}",
"name_write_base": f"0x{CHARACTER_PROFILE_NAME_ARRAY_BASE + 1:x}",
"first_record_id": 1,
"record_span": CHARACTER_PROFILE_RECORD_SPAN,
"unit_id_array_base": f"0x{CHARACTER_PROFILE_UNIT_ARRAY_BASE:x}",
"portrait_asset_array_base": (
f"0x{CHARACTER_PROFILE_PORTRAIT_ARRAY_BASE:x}"
),
"portrait_x_offset_array_base": (
f"0x{CHARACTER_PROFILE_PORTRAIT_X_ARRAY_BASE:x}"
),
"portrait_y_offset_array_base": (
f"0x{CHARACTER_PROFILE_PORTRAIT_Y_ARRAY_BASE:x}"
),
"implicit_defaults": {
f"0x{CHARACTER_PROFILE_PORTRAIT_X_ARRAY_BASE:x}": 0,
f"0x{CHARACTER_PROFILE_PORTRAIT_Y_ARRAY_BASE:x}": 0,
},
}
@cache
def object_type_definitions() -> dict[int, dict]:
"""Load OBINIT's authoritative display and state-row metadata by object type id."""
@@ -1913,7 +1999,8 @@ def write_data_index(data_dir: Path) -> None:
"Linked row-major fields are stored separately in `record_fields`, keyed as",
"`base/stride/column` from corpus-observed `lookup-array-2d` consumers.",
"Where a matching `*MES` dispatcher exists, `message` preserves its player-facing",
"title, description, furigana, and bytecode dispatch offset separately from the",
"layout-specific text (title/description, summary/strategy, or biography), furigana,",
"and bytecode dispatch offset separately from the",
"short description stored by the INIT script.",
"Top-level `field_semantics` maps raw array/row-column keys to canonical machine-readable",
"names from `vm-map/globals.toml`; each record's generated `semantic_fields` is the joined",
@@ -1987,6 +2074,8 @@ def main() -> int:
}[mode]
if mode == "name" and name == "VIINIT":
extractor = extract_vocabulary
elif mode == "name" and name == "CIINIT":
extractor = extract_character_profiles
recs, meta = extractor(scr)
if mode == "name" and name in MESSAGE_TABLES:
message_name = MESSAGE_TABLES[name]

View File

@@ -18,6 +18,7 @@ Usage:
py -3.11 -X utf8 tools/extract_message_table.py SKMES [OUTNAME]
py -3.11 -X utf8 tools/extract_message_table.py VIMES
py -3.11 -X utf8 tools/extract_message_table.py EIMES
py -3.11 -X utf8 tools/extract_message_table.py CIMES
"""
from __future__ import annotations
@@ -38,6 +39,7 @@ T_GLOBAL_INT = 3
BRANCH_SENTINEL = 0xFFFFFFFF
MESSAGE_LAYOUTS = {
"CIMES.BIN": "character-biography",
"EIMES.BIN": "enemy-commentary",
}
@@ -111,7 +113,11 @@ def _message_body(
lines.append("".join(fragments))
if not lines:
return None
if layout == "enemy-commentary":
if layout == "character-biography":
message = {
"biography": "\n".join(lines),
}
elif layout == "enemy-commentary":
message = {
"summary": lines[0],
"strategy": "\n".join(lines[1:]),

View File

@@ -36,7 +36,9 @@ def message_heading_body(message: dict) -> tuple[str, str]:
"""Return presentation-neutral heading/body text for supported MES layouts."""
return (
message.get("title", message.get("summary", "")),
message.get("description", message.get("strategy", "")),
message.get(
"description", message.get("strategy", message.get("biography", ""))
),
)
@@ -283,7 +285,9 @@ def profile_messages(data: dict) -> dict:
"description": body,
"message_fields": {
key: record["message"][key]
for key in ("title", "description", "summary", "strategy")
for key in (
"title", "description", "summary", "strategy", "biography"
)
if key in record["message"]
},
})
@@ -307,6 +311,7 @@ def find_message_matches(data: dict, pattern: str) -> list[dict]:
record.get("message", {}).get("description", ""),
record.get("message", {}).get("summary", ""),
record.get("message", {}).get("strategy", ""),
record.get("message", {}).get("biography", ""),
]))
]

View File

@@ -105,6 +105,37 @@ def test_real_name_tables() -> None:
)
def test_character_profiles() -> None:
scripts = paths.scripts()
records, meta = extract_init.extract_character_profiles(
sys4load.load(scripts["CIINIT.BIN"])
)
by_id = {record["id"]: record for record in records}
check(len(records) == 24, "CIINIT extracts all 24 character profiles")
check(
meta["record_span"] == 100
and meta["unit_id_array_base"] == "0x15a118"
and meta["portrait_asset_array_base"] == "0x15a17c",
"CIINIT exposes the four 100-cell profile columns",
)
check(
meta["implicit_defaults"]
== {"0x15a1e0": 0, "0x15a244": 0},
"CIINIT records its two unwritten portrait-placement defaults",
)
check(
by_id[1]["name"] == "エミリオ"
and by_id[1]["fields"]["0x15a118"] == 1
and by_id[1]["fields"]["0x15a17c"] == 0x2C88,
"CIINIT profile 1 joins Emilio to unit and portrait resources",
)
check(
by_id[16]["fields"]["0x15a118"] == 0x5F
and "0x15a17c" not in by_id[16]["fields"],
"CIINIT preserves the portrait-fallback profiles",
)
def test_static_negative_write() -> None:
class Instruction:
opcode = extract_init.SUB
@@ -581,6 +612,7 @@ def test_real_message_tables() -> None:
"SKMES.BIN": (0xA6E59, 131, "fallthrough"),
"VIMES.BIN": (0x15A2A8, 65, "branch-target"),
"EIMES.BIN": (0x15A759, 192, "branch-target"),
"CIMES.BIN": (0x15A117, 24, "branch-target"),
}
for name, (selector, count, dispatch_layout) in expected.items():
records, meta = extract_message_table.extract_messages(
@@ -639,6 +671,20 @@ def test_real_message_tables() -> None:
and "title" not in enemies[101],
"EIMES does not mislabel its first commentary line as a title")
character_messages, character_meta = extract_message_table.extract_messages(
sys4load.load(scripts["CIMES.BIN"])
)
characters = {record["id"]: record for record in character_messages}
check(
characters[1]["biography"].startswith("かつては人々を恐怖に陥れた")
and "title" not in characters[1],
"CIMES exposes its complete untitled character biography",
)
check(
character_meta["message_layout"] == "character-biography",
"CIMES records the biography-only message layout",
)
def test_message_join() -> None:
scripts = paths.scripts()
@@ -691,6 +737,26 @@ def test_message_join() -> None:
"EBINIT records expose EIMES strategy text by unit id",
)
characters, _ = extract_init.extract_character_profiles(
sys4load.load(scripts["CIINIT.BIN"])
)
character_meta = extract_init.join_messages(
characters, sys4load.load(scripts["CIMES.BIN"])
)
character_by_id = {record["id"]: record for record in characters}
check(
character_meta["joined_count"] == 24
and not character_meta["init_ids_without_message"]
and not character_meta["message_ids_without_init"],
"CIINIT and CIMES form a complete 24-profile runtime-id join",
)
check(
character_by_id[1]["message"]["biography"].startswith(
"かつては人々を恐怖に陥れた"
),
"CIINIT records expose CIMES biography text by profile id",
)
def test_field_semantics() -> None:
scripts = paths.scripts()
@@ -798,6 +864,7 @@ def test_field_semantics() -> None:
if __name__ == "__main__":
test_real_name_tables()
test_character_profiles()
test_static_negative_write()
test_output_name_validation()
test_real_mixed_table()

View File

@@ -67,6 +67,12 @@ def test_load_and_lint():
and entries[0x56b85]["name"]
== "enemy_encyclopedia_revealed_flags",
"EBINIT/EIMES enemy-encyclopedia state is curated")
check(entries[0x15a117]["name"] == "current_character_profile_id"
and entries[0x15a118]["name"] == "character_profile_unit_ids"
and entries[0x15a17c]["name"]
== "character_profile_portrait_asset_ids"
and entries[0x45d7]["name"] == "character_profile_names",
"CIINIT/CIMES character-profile state is curated")
check(entries[0x5f0ed]["name"] == "scene_decision_seen_flags",
"glossary prerequisite seen-state is curated")
check(entries[0x20543]["name"] == "tile_faction_traversal_masks",

View File

@@ -200,6 +200,29 @@ def main() -> int:
"strategy": "Avoid the first encounter",
}
assert profile.find_message_matches(enemy_fixture, "first encounter")
character_fixture = {
"table": "CHARACTER",
"records": [
{
"id": 1,
"name": "Emilio",
"message": {
"biography": "Former demon king\nNow a familiar",
},
}
],
}
character_messages = profile.profile_messages(character_fixture)
assert character_messages["examples"][0]["title"] == ""
assert (
character_messages["examples"][0]["description"]
== "Former demon king\nNow a familiar"
)
assert character_messages["examples"][0]["message_fields"] == {
"biography": "Former demon king\nNow a familiar",
}
assert profile.find_message_matches(character_fixture, "familiar")
print("all init_table_profile checks passed")
return 0

View File

@@ -242,6 +242,72 @@ usage = "Shared skill-id argument/selection slot. Skill menus and combat scripts
source = "investigation"
confidence = "high"
[[global]]
address = "0x45d7"
name = "character_profile_names"
category = "string-table"
type = "string[100]"
value_domain = "CIINIT display name indexed by character profile id; 24 populated shipped ids"
usage = "CIINIT writes the character-information screen's 24 display names. INFOCH lists them by profile id, masks unrevealed rows, and uses the same id to select the backing unit metadata and CIMES biography."
source = "investigation"
confidence = "high"
depends_on = ["0x15a117", "0x15a118"]
[[global]]
address = "0x15a117"
name = "current_character_profile_id"
category = "index-pointer"
type = "int"
value_domain = "CIINIT character profile id in 1..24"
usage = "INFOCH's selected character-information row. It indexes CIINIT's name, unit, portrait, and placement arrays; CIMES compares the same id against all 24 profiles to dispatch the character biography."
source = "investigation"
confidence = "high"
depends_on = ["0x45d7", "0x15a118", "0x15a17c"]
[[global]]
address = "0x15a118"
name = "character_profile_unit_ids"
category = "data-table"
type = "int[100]"
value_domain = "EBINIT unit definition id indexed by CIINIT character profile id"
usage = "CIINIT's profile-to-unit join. INFOCH uses the selected unit id for reveal state, map sprite, short unit descriptions, species, and sex while CIMES remains keyed by the enclosing profile id."
source = "investigation"
confidence = "high"
depends_on = ["0x15a117", "0x66715"]
[[global]]
address = "0x15a17c"
name = "character_profile_portrait_asset_ids"
category = "data-table"
type = "int[100]"
value_domain = "SYS4 graphics resource id; zero means use the unit map sprite fallback"
usage = "Optional CIINIT portrait resource indexed by character profile id. INFOCH draws it when nonzero and otherwise falls back to the backing unit's map-sprite asset."
source = "investigation"
confidence = "high"
depends_on = ["0x15a117", "0x15a118", "0x15a1e0", "0x15a244"]
[[global]]
address = "0x15a1e0"
name = "character_profile_portrait_x_offsets"
category = "data-table"
type = "int[100]"
value_domain = "signed horizontal draw offset in pixels; shipped CIINIT leaves all cells at zero"
usage = "INFOCH adds the selected cell to the centered portrait x coordinate. The 100-cell spacing to the adjacent CIINIT arrays and the direct indexed reader establish the reserved per-profile placement column."
source = "investigation"
confidence = "high"
depends_on = ["0x15a117", "0x15a17c"]
[[global]]
address = "0x15a244"
name = "character_profile_portrait_y_offsets"
category = "data-table"
type = "int[100]"
value_domain = "signed vertical draw offset in pixels; shipped CIINIT leaves all cells at zero"
usage = "INFOCH adds the selected cell to the bottom-aligned portrait y coordinate. The 100-cell spacing to the adjacent CIINIT arrays and the direct indexed reader establish the reserved per-profile placement column."
source = "investigation"
confidence = "high"
depends_on = ["0x15a117", "0x15a17c"]
[[global]]
address = "0x463b"
name = "glossary_topic_titles"