Fix EBINIT overlapping field ownership
This commit is contained in:
@@ -172,7 +172,7 @@ writes) unambiguously to six row-major tables and 44 populated columns:
|
||||
| `0xa5301` | 3 | 17 | HP/SP/FS recovery amounts |
|
||||
|
||||
`extract_init.py` now records these as `record_fields["base/stride/column"]` rather than inventing a
|
||||
one-off `fields` base for every row. Applying the same rule exposes 18 linked SKINIT columns and 84 linked
|
||||
one-off `fields` base for every row. Applying the same rule exposes 18 linked SKINIT columns and 82 linked
|
||||
EBINIT columns. This correction reduces the auto map's false INIT-field labels from 12,311 to 1,353; the raw
|
||||
write addresses were valid, but their former ownership model and omission of negative writes were not.
|
||||
|
||||
@@ -243,9 +243,9 @@ records use the shared accuracy-through-max-FS vocabulary; starting skills, drop
|
||||
brainwashed roster forms, battle portraits/cut-ins, and health-selected status art all expose named fields.
|
||||
Consumer control flow further divides the five CP sprite assets into normal/alternate compact and directional
|
||||
sheets plus the special compact sheet, and SHOWGROW proves voice column 24 is the level-up reaction. Of
|
||||
EBINIT's 110 populated profile fields, 107 now have specific semantic names. SALLY's SO012 button atlas and
|
||||
action dispatch resolve all four unlock columns as contract, brainwash, a reserved/unreachable slot, and
|
||||
sex magic. The paired eight event columns are contract, brainwash, the same reserved slot, three
|
||||
EBINIT's 108 genuine populated profile fields, 107 now have specific semantic names. SALLY's SO012 button
|
||||
atlas and action dispatch resolve all four unlock columns as contract, brainwash, a reserved/unreachable
|
||||
slot, and sex magic. The paired eight event columns are contract, brainwash, the same reserved slot, three
|
||||
form-dependent Lily sex-magic events, sacrifice, and release. The reserved slot has a switch arm but is
|
||||
deliberately skipped by both drawing and input; its event ids also lack SCJUMP mappings, so it is recorded
|
||||
as unreachable rather than assigned a speculative action.
|
||||
@@ -263,8 +263,15 @@ essence yield, level cap, and base statistics. Lily's three forms are exactly ti
|
||||
heroine boss definitions generally rise with their later, stronger appearances. The static corpus contains
|
||||
no read of this array, and the `/v2` native image contains neither its global index as an instruction operand
|
||||
nor as a little-endian constant. `unit_power_tier` is therefore curated at medium confidence as descriptive
|
||||
authoring metadata, not a runtime behavior claim. EBINIT now has specific names for 108 of 110 populated
|
||||
profile fields; only two suspicious one-record writes into runtime table `0x4e693/300` remain anonymous.
|
||||
authoring metadata, not a runtime behavior claim.
|
||||
|
||||
The apparent final two anonymous EBINIT writes were address-ownership collisions, not new fields. The flat
|
||||
global range occupied by stride-300 runtime table `0x4e693` overlaps established EBINIT parallel arrays.
|
||||
Address `0x6fd4e` can be expressed as row 456, column 91 of that table, but it is also exactly
|
||||
`unit_battle_sprite_asset_id[456]`; its value 12585 resolves to `CB456A.AGF`. Likewise `0x7a5d6` can be
|
||||
expressed as row 600, column 35, but is exactly `unit_starting_level[600] = 80`. The extractor now gives an
|
||||
already-established parallel base precedence over a coincidental row-table range match. EBINIT therefore
|
||||
has 82 genuine linked columns and specific semantic names for all 108 populated profile fields.
|
||||
|
||||
### STINIT mixed stage records (2026-07-23)
|
||||
|
||||
|
||||
@@ -724,7 +724,7 @@ SKINIT contributes skill category/order/icon/handler, encoded range, element, st
|
||||
proc chance, and battle animation. EBINIT now contributes unit order/icon, sex and provisional species,
|
||||
elements, attacks/equipment/skills, level/cost/base/growth data, canonical variants, CP/CA/CB/CS/cut-in and
|
||||
voice assets, XP/drops, capture and compendium flags, summoning economy, essence yield, and level scaling.
|
||||
Its linked-table shapes are 18 SKINIT and 84 EBINIT populated columns. The next EB tranche identifies roster
|
||||
Its linked-table shapes are 18 SKINIT and 82 EBINIT populated columns. The next EB tranche identifies roster
|
||||
state flags, per-action unlock requirements, SCJUMP event ids, normal/brainwashed unit variants, and SALLY's
|
||||
per-unit bonus item. Combat tracing also names the shared column-8 item/skill modifier as critical chance.
|
||||
|
||||
@@ -745,8 +745,10 @@ SKINIT parallel field as the skill-change catalog inclusion flag and separates a
|
||||
skill-information visibility. The remaining item/skill work is refinement rather than an unnamed populated
|
||||
schema.
|
||||
|
||||
The remaining EBINIT unknowns are two isolated sparse writes into a runtime table; enemy AI appears to live
|
||||
outside the static EBINIT schema. The signed boss class is now behaviorally separated: all nonzero values
|
||||
EBINIT's populated schema is now fully named. Its apparent final two sparse runtime-table writes were flat-
|
||||
address overlap collisions: they are the already-established battle-sprite asset for unit 456 and starting
|
||||
level for unit 600, not distinct stride-300 fields. Enemy AI appears to live outside the static EBINIT
|
||||
schema. The signed boss class is now behaviorally separated: all nonzero values
|
||||
receive boss protections and targeting treatment, positive values alone count as required targets under
|
||||
FIELD's defeat-boss clear rule, negative values are boss-treated adds/decoys/hazards that do not block
|
||||
victory, and either sign of class 4 selects final-boss music and tactical-map presentation. Classes 1--3
|
||||
|
||||
@@ -62,7 +62,7 @@ described by the workflow in `docs/name-resolution.md`.
|
||||
|
||||
The same audit found that linked row-major writes must not be normalized as independent parallel arrays.
|
||||
Corpus `lookup-array-2d` bases and strides assign every such ITINIT write unambiguously to six tables (44
|
||||
populated columns), while SKINIT and EBINIT expose 18 and 84 linked columns respectively. The extractor also
|
||||
populated columns), while SKINIT and EBINIT expose 18 and 82 linked columns respectively. The extractor also
|
||||
preserves the scripts' zero-minus-immediate negative writes, which carry item penalties, condition cures, and
|
||||
skill SP costs. Generated records keep linked cells under `record_fields[base/stride/column]`; `fields`
|
||||
contains only genuine parallel arrays.
|
||||
|
||||
@@ -55,7 +55,7 @@ All opcode knowledge (ABI, semantics, provenance, `depends_on`) is hand-edited *
|
||||
|---|---|---|---|
|
||||
| `extract_phase2.py` | Batch: disassembly + text corpora for every script. | `extract_phase2.py` | corpus → `build/disasm/*.asm`, `build/text/{dialogue.jsonl,strings.jsonl,*.strings.txt}`, `build/manifest.json` |
|
||||
| `extract_message_table.py` | Discover a repeated global-id dispatch chain such as ITMES/SKMES, reconstruct player-facing title/description lines (including furigana surface text and readings), and emit an ID-keyed message table with bytecode provenance. | `extract_message_table.py <MES> [OUTNAME]` | `<MES>.BIN` → `build/data/<OUTNAME>.json` |
|
||||
| `extract_init.py` | Parse a `*INIT` data table (auto-detects name / numeric / footer / mixed shape). Name tables infer their reserved record span, preserve sparse one-based runtime ids, distinguish lookup bases from first written cells, statically evaluate direct and negative-value writes, and separate parallel `fields` from linked row-major `record_fields`. Mixed tables recover selector-dispatched records, condition strings, scalars, preallocated buffer cells, consumer-confirmed strides, and length-prefixed footer arrays; STINIT additionally assembles confirmed buffers into `object_placements` and `enemy_spawns`, joins object type names and available descriptions from OBINIT, and exposes consumer-proven tagged payloads (teleport, treasure, card, initial faction, non-triggering hazard/barrier faction, and OBINIT-gated initial object state), preserves engine-dead type-27 writes under `ignored_payload_fields`, and exposes reinforcement schedules and first-clear-only enemy gates. ITINIT and SKINIT join ITMES/SKMES messages. Top-level `field_semantics` maps raw keys to canonical global/column names, while each record's `semantic_fields` is the generated name-keyed join; complete footer copies map to a row and expose its values without the raw provenance wrapper. Raw keys and unresolved tagged payload cells remain provenance. OUTNAME accepts a stem or one `.json` suffix and rejects paths. Refreshes the generated data index. | `extract_init.py <TABLE> [OUTNAME] [--mode …]` | `<TABLE>.BIN` plus OBINIT for STINIT, matching `<MES>.BIN` when supported, and `build/globals.json` → `build/data/<OUTNAME>.json`, `build/data/README.md` |
|
||||
| `extract_init.py` | Parse a `*INIT` data table (auto-detects name / numeric / footer / mixed shape). Name tables infer their reserved record span, preserve sparse one-based runtime ids, distinguish lookup bases from first written cells, statically evaluate direct and negative-value writes, and separate parallel `fields` from linked row-major `record_fields`; when flat global ranges overlap, a parallel base established by other records wins over a coincidental row-table cell calculation. Mixed tables recover selector-dispatched records, condition strings, scalars, preallocated buffer cells, consumer-confirmed strides, and length-prefixed footer arrays; STINIT additionally assembles confirmed buffers into `object_placements` and `enemy_spawns`, joins object type names and available descriptions from OBINIT, and exposes consumer-proven tagged payloads (teleport, treasure, card, initial faction, non-triggering hazard/barrier faction, and OBINIT-gated initial object state), preserves engine-dead type-27 writes under `ignored_payload_fields`, and exposes reinforcement schedules and first-clear-only enemy gates. ITINIT and SKINIT join ITMES/SKMES messages. Top-level `field_semantics` maps raw keys to canonical global/column names, while each record's `semantic_fields` is the generated name-keyed join; complete footer copies map to a row and expose its values without the raw provenance wrapper. Raw keys and unresolved tagged payload cells remain provenance. OUTNAME accepts a stem or one `.json` suffix and rejects paths. Refreshes the generated data index. | `extract_init.py <TABLE> [OUTNAME] [--mode …]` | `<TABLE>.BIN` plus OBINIT for STINIT, matching `<MES>.BIN` when supported, and `build/globals.json` → `build/data/<OUTNAME>.json`, `build/data/README.md` |
|
||||
| `init_table_profile.py` | Build the static investigation surface for an extracted name/numeric/mixed table: message coverage, per-scalar/string/array-cell/footer-array population and value distributions, representative records, and direct opcode/script consumers. `--message-query REGEX` shows every matching name/message beside all populated fields for semantic correlation. Findings are evidence only; confirmed meanings go in `vm-map/globals.toml`. | `init_table_profile.py <TABLE> [--build] [--limit N] [--message-query REGEX]` | `build/data/<TABLE>.json` + corpus → stdout; with `--build`, `build/data/<TABLE>-field-profile.{json,md}` |
|
||||
| `test_extract_init.py`, `test_init_table_profile.py` | Regression checks for sparse one-based and mixed selector-dispatched INIT extraction, MES reconstruction/joins, footer-array accounting and semantic projection, OBINIT object-definition extraction, STINIT object/enemy joins and tagged-payload behavior, and field/message profiling. | run each directly | — |
|
||||
| `global_map.py` | Build the partial global-variable name map from static evidence. | `global_map.py` | corpus + `build/data/` → `build/global-var-map.{json,md}` |
|
||||
|
||||
@@ -343,6 +343,36 @@ def _record_table_cell(destination, record_id):
|
||||
return matches[0] if matches else None
|
||||
|
||||
|
||||
def _resolve_parallel_record_overlaps(records):
|
||||
"""Prefer an established parallel column over a row-table range collision.
|
||||
|
||||
The global bank is flat, so a sufficiently large row-major table can
|
||||
contain an address that another INIT schema reaches as `base + entity_id`.
|
||||
A parallel base repeated by other records is stronger ownership evidence
|
||||
than one accidental in-range row/column calculation.
|
||||
"""
|
||||
parallel_records = {}
|
||||
for record in records:
|
||||
for key in record.get("fields", {}):
|
||||
parallel_records.setdefault(int(key, 0), set()).add(record["id"])
|
||||
for record in records:
|
||||
retained = {}
|
||||
for key, value in record.get("record_fields", {}).items():
|
||||
base, stride, column = (int(part, 0) for part in key.split("/"))
|
||||
destination = base + record["id"] * stride + column
|
||||
parallel_base = destination - record["id"]
|
||||
if any(
|
||||
other_id != record["id"]
|
||||
for other_id in parallel_records.get(parallel_base, ())
|
||||
):
|
||||
_store_unique(
|
||||
record["fields"], f"0x{parallel_base:x}", value, record["id"]
|
||||
)
|
||||
else:
|
||||
retained[key] = value
|
||||
record["record_fields"] = retained
|
||||
|
||||
|
||||
def extract_name(scr):
|
||||
string_addrs = [
|
||||
ins.args[0][1]
|
||||
@@ -379,6 +409,7 @@ def extract_name(scr):
|
||||
else:
|
||||
base, stride, column = cell
|
||||
cur["record_fields"][f"0x{base:x}/{stride}/{column}"] = value
|
||||
_resolve_parallel_record_overlaps(records)
|
||||
for record in records:
|
||||
if not record["record_fields"]:
|
||||
del record["record_fields"]
|
||||
|
||||
@@ -56,6 +56,30 @@ def test_real_name_tables() -> None:
|
||||
check(by_id[108]["record_fields"]["0x906f9/30/8"] == -5,
|
||||
"ITINIT item 108 preserves its paralysis-removal delta")
|
||||
|
||||
units, _ = extract_init.extract_name(sys4load.load(scripts["EBINIT.BIN"]))
|
||||
unit_by_id = {record["id"]: record for record in units}
|
||||
check(len({
|
||||
key
|
||||
for record in units
|
||||
for key in {
|
||||
**record.get("fields", {}),
|
||||
**record.get("record_fields", {}),
|
||||
}
|
||||
}) == 108, "EBINIT exposes 108 genuine populated fields")
|
||||
check(len({
|
||||
key for record in units for key in record.get("record_fields", {})
|
||||
}) == 82, "EBINIT exposes 82 genuine linked row-major columns")
|
||||
check(
|
||||
unit_by_id[456]["fields"]["0x6fb86"] == 12585
|
||||
and "0x4e693/300/91" not in unit_by_id[456].get("record_fields", {}),
|
||||
"EBINIT unit 456 keeps its battle sprite in the parallel asset column",
|
||||
)
|
||||
check(
|
||||
unit_by_id[600]["fields"]["0x7a37e"] == 80
|
||||
and "0x4e693/300/35" not in unit_by_id[600].get("record_fields", {}),
|
||||
"EBINIT unit 600 keeps its starting level in the parallel level column",
|
||||
)
|
||||
|
||||
objects, _ = extract_init.extract_name(sys4load.load(scripts["OBINIT.BIN"]))
|
||||
object_by_id = {record["id"]: record for record in objects}
|
||||
check(object_by_id[17]["name"] == "針"
|
||||
@@ -374,6 +398,11 @@ def test_field_semantics() -> None:
|
||||
] == [1, -1, 4, -4],
|
||||
"paired story and final-boss records preserve victory-target sign",
|
||||
)
|
||||
check(
|
||||
unit_by_id[456]["semantic_fields"]["unit_battle_sprite_asset_id"] == 12585
|
||||
and unit_by_id[600]["semantic_fields"]["unit_starting_level"] == 80,
|
||||
"overlapping EBINIT writes retain their established parallel semantics",
|
||||
)
|
||||
|
||||
stages, meta = extract_init.extract_mixed(
|
||||
sys4load.load(extract_init.resolve("STINIT"))
|
||||
|
||||
Reference in New Issue
Block a user