Fix EBINIT overlapping field ownership
This commit is contained in:
@@ -172,7 +172,7 @@ writes) unambiguously to six row-major tables and 44 populated columns:
|
|||||||
| `0xa5301` | 3 | 17 | HP/SP/FS recovery amounts |
|
| `0xa5301` | 3 | 17 | HP/SP/FS recovery amounts |
|
||||||
|
|
||||||
`extract_init.py` now records these as `record_fields["base/stride/column"]` rather than inventing a
|
`extract_init.py` now records these as `record_fields["base/stride/column"]` rather than inventing a
|
||||||
one-off `fields` base for every row. Applying the same rule exposes 18 linked SKINIT columns and 84 linked
|
one-off `fields` base for every row. Applying the same rule exposes 18 linked SKINIT columns and 82 linked
|
||||||
EBINIT columns. This correction reduces the auto map's false INIT-field labels from 12,311 to 1,353; the raw
|
EBINIT columns. This correction reduces the auto map's false INIT-field labels from 12,311 to 1,353; the raw
|
||||||
write addresses were valid, but their former ownership model and omission of negative writes were not.
|
write addresses were valid, but their former ownership model and omission of negative writes were not.
|
||||||
|
|
||||||
@@ -243,9 +243,9 @@ records use the shared accuracy-through-max-FS vocabulary; starting skills, drop
|
|||||||
brainwashed roster forms, battle portraits/cut-ins, and health-selected status art all expose named fields.
|
brainwashed roster forms, battle portraits/cut-ins, and health-selected status art all expose named fields.
|
||||||
Consumer control flow further divides the five CP sprite assets into normal/alternate compact and directional
|
Consumer control flow further divides the five CP sprite assets into normal/alternate compact and directional
|
||||||
sheets plus the special compact sheet, and SHOWGROW proves voice column 24 is the level-up reaction. Of
|
sheets plus the special compact sheet, and SHOWGROW proves voice column 24 is the level-up reaction. Of
|
||||||
EBINIT's 110 populated profile fields, 107 now have specific semantic names. SALLY's SO012 button atlas and
|
EBINIT's 108 genuine populated profile fields, 107 now have specific semantic names. SALLY's SO012 button
|
||||||
action dispatch resolve all four unlock columns as contract, brainwash, a reserved/unreachable slot, and
|
atlas and action dispatch resolve all four unlock columns as contract, brainwash, a reserved/unreachable
|
||||||
sex magic. The paired eight event columns are contract, brainwash, the same reserved slot, three
|
slot, and sex magic. The paired eight event columns are contract, brainwash, the same reserved slot, three
|
||||||
form-dependent Lily sex-magic events, sacrifice, and release. The reserved slot has a switch arm but is
|
form-dependent Lily sex-magic events, sacrifice, and release. The reserved slot has a switch arm but is
|
||||||
deliberately skipped by both drawing and input; its event ids also lack SCJUMP mappings, so it is recorded
|
deliberately skipped by both drawing and input; its event ids also lack SCJUMP mappings, so it is recorded
|
||||||
as unreachable rather than assigned a speculative action.
|
as unreachable rather than assigned a speculative action.
|
||||||
@@ -263,8 +263,15 @@ essence yield, level cap, and base statistics. Lily's three forms are exactly ti
|
|||||||
heroine boss definitions generally rise with their later, stronger appearances. The static corpus contains
|
heroine boss definitions generally rise with their later, stronger appearances. The static corpus contains
|
||||||
no read of this array, and the `/v2` native image contains neither its global index as an instruction operand
|
no read of this array, and the `/v2` native image contains neither its global index as an instruction operand
|
||||||
nor as a little-endian constant. `unit_power_tier` is therefore curated at medium confidence as descriptive
|
nor as a little-endian constant. `unit_power_tier` is therefore curated at medium confidence as descriptive
|
||||||
authoring metadata, not a runtime behavior claim. EBINIT now has specific names for 108 of 110 populated
|
authoring metadata, not a runtime behavior claim.
|
||||||
profile fields; only two suspicious one-record writes into runtime table `0x4e693/300` remain anonymous.
|
|
||||||
|
The apparent final two anonymous EBINIT writes were address-ownership collisions, not new fields. The flat
|
||||||
|
global range occupied by stride-300 runtime table `0x4e693` overlaps established EBINIT parallel arrays.
|
||||||
|
Address `0x6fd4e` can be expressed as row 456, column 91 of that table, but it is also exactly
|
||||||
|
`unit_battle_sprite_asset_id[456]`; its value 12585 resolves to `CB456A.AGF`. Likewise `0x7a5d6` can be
|
||||||
|
expressed as row 600, column 35, but is exactly `unit_starting_level[600] = 80`. The extractor now gives an
|
||||||
|
already-established parallel base precedence over a coincidental row-table range match. EBINIT therefore
|
||||||
|
has 82 genuine linked columns and specific semantic names for all 108 populated profile fields.
|
||||||
|
|
||||||
### STINIT mixed stage records (2026-07-23)
|
### STINIT mixed stage records (2026-07-23)
|
||||||
|
|
||||||
|
|||||||
@@ -724,7 +724,7 @@ SKINIT contributes skill category/order/icon/handler, encoded range, element, st
|
|||||||
proc chance, and battle animation. EBINIT now contributes unit order/icon, sex and provisional species,
|
proc chance, and battle animation. EBINIT now contributes unit order/icon, sex and provisional species,
|
||||||
elements, attacks/equipment/skills, level/cost/base/growth data, canonical variants, CP/CA/CB/CS/cut-in and
|
elements, attacks/equipment/skills, level/cost/base/growth data, canonical variants, CP/CA/CB/CS/cut-in and
|
||||||
voice assets, XP/drops, capture and compendium flags, summoning economy, essence yield, and level scaling.
|
voice assets, XP/drops, capture and compendium flags, summoning economy, essence yield, and level scaling.
|
||||||
Its linked-table shapes are 18 SKINIT and 84 EBINIT populated columns. The next EB tranche identifies roster
|
Its linked-table shapes are 18 SKINIT and 82 EBINIT populated columns. The next EB tranche identifies roster
|
||||||
state flags, per-action unlock requirements, SCJUMP event ids, normal/brainwashed unit variants, and SALLY's
|
state flags, per-action unlock requirements, SCJUMP event ids, normal/brainwashed unit variants, and SALLY's
|
||||||
per-unit bonus item. Combat tracing also names the shared column-8 item/skill modifier as critical chance.
|
per-unit bonus item. Combat tracing also names the shared column-8 item/skill modifier as critical chance.
|
||||||
|
|
||||||
@@ -745,8 +745,10 @@ SKINIT parallel field as the skill-change catalog inclusion flag and separates a
|
|||||||
skill-information visibility. The remaining item/skill work is refinement rather than an unnamed populated
|
skill-information visibility. The remaining item/skill work is refinement rather than an unnamed populated
|
||||||
schema.
|
schema.
|
||||||
|
|
||||||
The remaining EBINIT unknowns are two isolated sparse writes into a runtime table; enemy AI appears to live
|
EBINIT's populated schema is now fully named. Its apparent final two sparse runtime-table writes were flat-
|
||||||
outside the static EBINIT schema. The signed boss class is now behaviorally separated: all nonzero values
|
address overlap collisions: they are the already-established battle-sprite asset for unit 456 and starting
|
||||||
|
level for unit 600, not distinct stride-300 fields. Enemy AI appears to live outside the static EBINIT
|
||||||
|
schema. The signed boss class is now behaviorally separated: all nonzero values
|
||||||
receive boss protections and targeting treatment, positive values alone count as required targets under
|
receive boss protections and targeting treatment, positive values alone count as required targets under
|
||||||
FIELD's defeat-boss clear rule, negative values are boss-treated adds/decoys/hazards that do not block
|
FIELD's defeat-boss clear rule, negative values are boss-treated adds/decoys/hazards that do not block
|
||||||
victory, and either sign of class 4 selects final-boss music and tactical-map presentation. Classes 1--3
|
victory, and either sign of class 4 selects final-boss music and tactical-map presentation. Classes 1--3
|
||||||
|
|||||||
@@ -62,7 +62,7 @@ described by the workflow in `docs/name-resolution.md`.
|
|||||||
|
|
||||||
The same audit found that linked row-major writes must not be normalized as independent parallel arrays.
|
The same audit found that linked row-major writes must not be normalized as independent parallel arrays.
|
||||||
Corpus `lookup-array-2d` bases and strides assign every such ITINIT write unambiguously to six tables (44
|
Corpus `lookup-array-2d` bases and strides assign every such ITINIT write unambiguously to six tables (44
|
||||||
populated columns), while SKINIT and EBINIT expose 18 and 84 linked columns respectively. The extractor also
|
populated columns), while SKINIT and EBINIT expose 18 and 82 linked columns respectively. The extractor also
|
||||||
preserves the scripts' zero-minus-immediate negative writes, which carry item penalties, condition cures, and
|
preserves the scripts' zero-minus-immediate negative writes, which carry item penalties, condition cures, and
|
||||||
skill SP costs. Generated records keep linked cells under `record_fields[base/stride/column]`; `fields`
|
skill SP costs. Generated records keep linked cells under `record_fields[base/stride/column]`; `fields`
|
||||||
contains only genuine parallel arrays.
|
contains only genuine parallel arrays.
|
||||||
|
|||||||
@@ -55,7 +55,7 @@ All opcode knowledge (ABI, semantics, provenance, `depends_on`) is hand-edited *
|
|||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| `extract_phase2.py` | Batch: disassembly + text corpora for every script. | `extract_phase2.py` | corpus → `build/disasm/*.asm`, `build/text/{dialogue.jsonl,strings.jsonl,*.strings.txt}`, `build/manifest.json` |
|
| `extract_phase2.py` | Batch: disassembly + text corpora for every script. | `extract_phase2.py` | corpus → `build/disasm/*.asm`, `build/text/{dialogue.jsonl,strings.jsonl,*.strings.txt}`, `build/manifest.json` |
|
||||||
| `extract_message_table.py` | Discover a repeated global-id dispatch chain such as ITMES/SKMES, reconstruct player-facing title/description lines (including furigana surface text and readings), and emit an ID-keyed message table with bytecode provenance. | `extract_message_table.py <MES> [OUTNAME]` | `<MES>.BIN` → `build/data/<OUTNAME>.json` |
|
| `extract_message_table.py` | Discover a repeated global-id dispatch chain such as ITMES/SKMES, reconstruct player-facing title/description lines (including furigana surface text and readings), and emit an ID-keyed message table with bytecode provenance. | `extract_message_table.py <MES> [OUTNAME]` | `<MES>.BIN` → `build/data/<OUTNAME>.json` |
|
||||||
| `extract_init.py` | Parse a `*INIT` data table (auto-detects name / numeric / footer / mixed shape). Name tables infer their reserved record span, preserve sparse one-based runtime ids, distinguish lookup bases from first written cells, statically evaluate direct and negative-value writes, and separate parallel `fields` from linked row-major `record_fields`. Mixed tables recover selector-dispatched records, condition strings, scalars, preallocated buffer cells, consumer-confirmed strides, and length-prefixed footer arrays; STINIT additionally assembles confirmed buffers into `object_placements` and `enemy_spawns`, joins object type names and available descriptions from OBINIT, and exposes consumer-proven tagged payloads (teleport, treasure, card, initial faction, non-triggering hazard/barrier faction, and OBINIT-gated initial object state), preserves engine-dead type-27 writes under `ignored_payload_fields`, and exposes reinforcement schedules and first-clear-only enemy gates. ITINIT and SKINIT join ITMES/SKMES messages. Top-level `field_semantics` maps raw keys to canonical global/column names, while each record's `semantic_fields` is the generated name-keyed join; complete footer copies map to a row and expose its values without the raw provenance wrapper. Raw keys and unresolved tagged payload cells remain provenance. OUTNAME accepts a stem or one `.json` suffix and rejects paths. Refreshes the generated data index. | `extract_init.py <TABLE> [OUTNAME] [--mode …]` | `<TABLE>.BIN` plus OBINIT for STINIT, matching `<MES>.BIN` when supported, and `build/globals.json` → `build/data/<OUTNAME>.json`, `build/data/README.md` |
|
| `extract_init.py` | Parse a `*INIT` data table (auto-detects name / numeric / footer / mixed shape). Name tables infer their reserved record span, preserve sparse one-based runtime ids, distinguish lookup bases from first written cells, statically evaluate direct and negative-value writes, and separate parallel `fields` from linked row-major `record_fields`; when flat global ranges overlap, a parallel base established by other records wins over a coincidental row-table cell calculation. Mixed tables recover selector-dispatched records, condition strings, scalars, preallocated buffer cells, consumer-confirmed strides, and length-prefixed footer arrays; STINIT additionally assembles confirmed buffers into `object_placements` and `enemy_spawns`, joins object type names and available descriptions from OBINIT, and exposes consumer-proven tagged payloads (teleport, treasure, card, initial faction, non-triggering hazard/barrier faction, and OBINIT-gated initial object state), preserves engine-dead type-27 writes under `ignored_payload_fields`, and exposes reinforcement schedules and first-clear-only enemy gates. ITINIT and SKINIT join ITMES/SKMES messages. Top-level `field_semantics` maps raw keys to canonical global/column names, while each record's `semantic_fields` is the generated name-keyed join; complete footer copies map to a row and expose its values without the raw provenance wrapper. Raw keys and unresolved tagged payload cells remain provenance. OUTNAME accepts a stem or one `.json` suffix and rejects paths. Refreshes the generated data index. | `extract_init.py <TABLE> [OUTNAME] [--mode …]` | `<TABLE>.BIN` plus OBINIT for STINIT, matching `<MES>.BIN` when supported, and `build/globals.json` → `build/data/<OUTNAME>.json`, `build/data/README.md` |
|
||||||
| `init_table_profile.py` | Build the static investigation surface for an extracted name/numeric/mixed table: message coverage, per-scalar/string/array-cell/footer-array population and value distributions, representative records, and direct opcode/script consumers. `--message-query REGEX` shows every matching name/message beside all populated fields for semantic correlation. Findings are evidence only; confirmed meanings go in `vm-map/globals.toml`. | `init_table_profile.py <TABLE> [--build] [--limit N] [--message-query REGEX]` | `build/data/<TABLE>.json` + corpus → stdout; with `--build`, `build/data/<TABLE>-field-profile.{json,md}` |
|
| `init_table_profile.py` | Build the static investigation surface for an extracted name/numeric/mixed table: message coverage, per-scalar/string/array-cell/footer-array population and value distributions, representative records, and direct opcode/script consumers. `--message-query REGEX` shows every matching name/message beside all populated fields for semantic correlation. Findings are evidence only; confirmed meanings go in `vm-map/globals.toml`. | `init_table_profile.py <TABLE> [--build] [--limit N] [--message-query REGEX]` | `build/data/<TABLE>.json` + corpus → stdout; with `--build`, `build/data/<TABLE>-field-profile.{json,md}` |
|
||||||
| `test_extract_init.py`, `test_init_table_profile.py` | Regression checks for sparse one-based and mixed selector-dispatched INIT extraction, MES reconstruction/joins, footer-array accounting and semantic projection, OBINIT object-definition extraction, STINIT object/enemy joins and tagged-payload behavior, and field/message profiling. | run each directly | — |
|
| `test_extract_init.py`, `test_init_table_profile.py` | Regression checks for sparse one-based and mixed selector-dispatched INIT extraction, MES reconstruction/joins, footer-array accounting and semantic projection, OBINIT object-definition extraction, STINIT object/enemy joins and tagged-payload behavior, and field/message profiling. | run each directly | — |
|
||||||
| `global_map.py` | Build the partial global-variable name map from static evidence. | `global_map.py` | corpus + `build/data/` → `build/global-var-map.{json,md}` |
|
| `global_map.py` | Build the partial global-variable name map from static evidence. | `global_map.py` | corpus + `build/data/` → `build/global-var-map.{json,md}` |
|
||||||
|
|||||||
@@ -343,6 +343,36 @@ def _record_table_cell(destination, record_id):
|
|||||||
return matches[0] if matches else None
|
return matches[0] if matches else None
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_parallel_record_overlaps(records):
|
||||||
|
"""Prefer an established parallel column over a row-table range collision.
|
||||||
|
|
||||||
|
The global bank is flat, so a sufficiently large row-major table can
|
||||||
|
contain an address that another INIT schema reaches as `base + entity_id`.
|
||||||
|
A parallel base repeated by other records is stronger ownership evidence
|
||||||
|
than one accidental in-range row/column calculation.
|
||||||
|
"""
|
||||||
|
parallel_records = {}
|
||||||
|
for record in records:
|
||||||
|
for key in record.get("fields", {}):
|
||||||
|
parallel_records.setdefault(int(key, 0), set()).add(record["id"])
|
||||||
|
for record in records:
|
||||||
|
retained = {}
|
||||||
|
for key, value in record.get("record_fields", {}).items():
|
||||||
|
base, stride, column = (int(part, 0) for part in key.split("/"))
|
||||||
|
destination = base + record["id"] * stride + column
|
||||||
|
parallel_base = destination - record["id"]
|
||||||
|
if any(
|
||||||
|
other_id != record["id"]
|
||||||
|
for other_id in parallel_records.get(parallel_base, ())
|
||||||
|
):
|
||||||
|
_store_unique(
|
||||||
|
record["fields"], f"0x{parallel_base:x}", value, record["id"]
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
retained[key] = value
|
||||||
|
record["record_fields"] = retained
|
||||||
|
|
||||||
|
|
||||||
def extract_name(scr):
|
def extract_name(scr):
|
||||||
string_addrs = [
|
string_addrs = [
|
||||||
ins.args[0][1]
|
ins.args[0][1]
|
||||||
@@ -379,6 +409,7 @@ def extract_name(scr):
|
|||||||
else:
|
else:
|
||||||
base, stride, column = cell
|
base, stride, column = cell
|
||||||
cur["record_fields"][f"0x{base:x}/{stride}/{column}"] = value
|
cur["record_fields"][f"0x{base:x}/{stride}/{column}"] = value
|
||||||
|
_resolve_parallel_record_overlaps(records)
|
||||||
for record in records:
|
for record in records:
|
||||||
if not record["record_fields"]:
|
if not record["record_fields"]:
|
||||||
del record["record_fields"]
|
del record["record_fields"]
|
||||||
|
|||||||
@@ -56,6 +56,30 @@ def test_real_name_tables() -> None:
|
|||||||
check(by_id[108]["record_fields"]["0x906f9/30/8"] == -5,
|
check(by_id[108]["record_fields"]["0x906f9/30/8"] == -5,
|
||||||
"ITINIT item 108 preserves its paralysis-removal delta")
|
"ITINIT item 108 preserves its paralysis-removal delta")
|
||||||
|
|
||||||
|
units, _ = extract_init.extract_name(sys4load.load(scripts["EBINIT.BIN"]))
|
||||||
|
unit_by_id = {record["id"]: record for record in units}
|
||||||
|
check(len({
|
||||||
|
key
|
||||||
|
for record in units
|
||||||
|
for key in {
|
||||||
|
**record.get("fields", {}),
|
||||||
|
**record.get("record_fields", {}),
|
||||||
|
}
|
||||||
|
}) == 108, "EBINIT exposes 108 genuine populated fields")
|
||||||
|
check(len({
|
||||||
|
key for record in units for key in record.get("record_fields", {})
|
||||||
|
}) == 82, "EBINIT exposes 82 genuine linked row-major columns")
|
||||||
|
check(
|
||||||
|
unit_by_id[456]["fields"]["0x6fb86"] == 12585
|
||||||
|
and "0x4e693/300/91" not in unit_by_id[456].get("record_fields", {}),
|
||||||
|
"EBINIT unit 456 keeps its battle sprite in the parallel asset column",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
unit_by_id[600]["fields"]["0x7a37e"] == 80
|
||||||
|
and "0x4e693/300/35" not in unit_by_id[600].get("record_fields", {}),
|
||||||
|
"EBINIT unit 600 keeps its starting level in the parallel level column",
|
||||||
|
)
|
||||||
|
|
||||||
objects, _ = extract_init.extract_name(sys4load.load(scripts["OBINIT.BIN"]))
|
objects, _ = extract_init.extract_name(sys4load.load(scripts["OBINIT.BIN"]))
|
||||||
object_by_id = {record["id"]: record for record in objects}
|
object_by_id = {record["id"]: record for record in objects}
|
||||||
check(object_by_id[17]["name"] == "針"
|
check(object_by_id[17]["name"] == "針"
|
||||||
@@ -374,6 +398,11 @@ def test_field_semantics() -> None:
|
|||||||
] == [1, -1, 4, -4],
|
] == [1, -1, 4, -4],
|
||||||
"paired story and final-boss records preserve victory-target sign",
|
"paired story and final-boss records preserve victory-target sign",
|
||||||
)
|
)
|
||||||
|
check(
|
||||||
|
unit_by_id[456]["semantic_fields"]["unit_battle_sprite_asset_id"] == 12585
|
||||||
|
and unit_by_id[600]["semantic_fields"]["unit_starting_level"] == 80,
|
||||||
|
"overlapping EBINIT writes retain their established parallel semantics",
|
||||||
|
)
|
||||||
|
|
||||||
stages, meta = extract_init.extract_mixed(
|
stages, meta = extract_init.extract_mixed(
|
||||||
sys4load.load(extract_init.resolve("STINIT"))
|
sys4load.load(extract_init.resolve("STINIT"))
|
||||||
|
|||||||
Reference in New Issue
Block a user