Document native persistence file formats
This commit is contained in:
gamer147
@@ -1683,86 +1683,24 @@ the analogous `$$RT.DAT` → `RT.DAT` / `RT.BAK` update for ReadTextDB. `shared_
|
||||
falls back from `SAVE.DAT` to `SAVE.BAK` and loads `RT.DAT` independently. Numbered `.DAT` writes do not
|
||||
use this backup transaction, but a successful numbered serialization flushes the shared profile too.
|
||||
|
||||
After the common container decode, the shared logical payload is:
|
||||
The complete shared-payload byte structure, installed-file counts, and its separation from `RT.DAT` and
|
||||
numbered slot state are canonical in `sys4-format-notes.md` under "Native persistence files." The
|
||||
selected-cell address meanings are canonical in `vm-map/globals.toml`. Native writer/reader behavior
|
||||
establishes that the fixed integer/string maps are profile-wide services rather than whole-bank snapshots;
|
||||
their opaque catalog and extended sections are preserved by the port.
|
||||
|
||||
1. a DWORD catalog/compatibility count followed by that many DWORDs;
|
||||
2. a DWORD integer-entry count, then fixed 16-byte entries `{typed_key[12], raw_value_u32}`;
|
||||
3. a DWORD string-entry count and a string-blob **DWORD count**, then concatenated
|
||||
`typed_key\0value\0` pairs with DWORD padding;
|
||||
4. for shared save version at least 3.10, a 256-DWORD selector/count table, a DWORD extra-count, and that
|
||||
many extra DWORDs;
|
||||
5. one explicit trailing zero DWORD plus eight historical allocation-slack DWORDs (nine reserved tail
|
||||
DWORDs in decoded native Himegari files).
|
||||
|
||||
The key prefix is binary, not the ASCII character suggested by shorthand such as `3%08x`:
|
||||
integer-cell ops `0x1a2`/`0x1a3` use raw byte `0x03` followed by eight lowercase ASCII hex digits;
|
||||
string-cell ops `0x1a9`/`0x1aa` use raw byte `0x05` plus the same address text. In both cases the
|
||||
hexadecimal portion is the VM lvalue's resolved global-bank index. Integer fixed-key bytes 10 and 11 are
|
||||
unused native stack padding and cannot be required to be zero when importing an original file. The string
|
||||
table is therefore not an incidental settings blob: it is the paired profile-wide selected-cell service
|
||||
for string globals, with insert-or-assign and empty-on-miss behavior.
|
||||
|
||||
The installed native Himegari `SAVE.DAT` provides a direct compatibility oracle for this shape:
|
||||
version 3.10 decodes to 13,210 opaque catalog values, 24,070 integer cells, 605 string cells, selector
|
||||
entry 1 equal to 81, 83 extended values, and the nine-DWORD reserved tail. The port decoder imports that
|
||||
file exactly; catalog/extended/tail sections remain opaque and are preserved losslessly rather than
|
||||
reinterpreted as mod state.
|
||||
|
||||
`SAVE.BIN` also proves why a numbered `.DAT`/`.STH` pair alone cannot reproduce a complete menu row.
|
||||
The numbered fixed header supplies only timestamp and accumulated playtime, and `.STH` supplies the
|
||||
112x84 screenshot. The remaining preview record is maintained through selected-cell ops in shared
|
||||
`SAVE.DAT`: three parallel 210-cell string banks hold chapter/title (`GStr[0x0+]`), location
|
||||
(`GStr[0xd2+]`), and protagonist name (`GStr[0x1a4+]`); eight 210-cell integer banks beginning at
|
||||
`GInt[0x5]`, `0xd7`, `0x1a9`, `0x27b`, `0x34d`, `0x41f`, `0x4f1`, and `0x5c3` hold portrait entity,
|
||||
level, training count, growth, personality, difficulty/mode badge, cleared-ending mask, and installed
|
||||
append mask respectively. Save and stage-select paths snapshot these cells with `0x1a2`/`0x1a9`;
|
||||
the menu restores them with `0x1a3`/`0x1aa`. For the NG+ history display, `SAVE.BIN` scans bits 0..14
|
||||
of the restored `GInt[0x4f1 + slot]` mask and draws one 17x17 ending badge for every set bit.
|
||||
`SAVE.BIN` provides the game-level consumer proof for the menu split: it reads timestamp/playtime from
|
||||
the numbered header, loads the `.STH` screenshot, restores the remaining row fields with selected-cell
|
||||
ops, and scans the restored cleared-ending mask to draw the NG+ badges. Save and stage-select paths
|
||||
snapshot the same preview cells before persistence.
|
||||
|
||||
#### Common `.DAT` container
|
||||
|
||||
Both shared and numbered `.DAT` payloads use the same native container. The fixed header is exactly
|
||||
`0x124` bytes:
|
||||
|
||||
| Offset | Size | Field |
|
||||
|---:|---:|---|
|
||||
| `0x000` | 4 | little-endian magic `S3SD` or `S4SD` |
|
||||
| `0x004` | 4 | compatibility id |
|
||||
| `0x008` | `0x100` | NUL-terminated game id area |
|
||||
| `0x108` | `0x10` | Win32 `SYSTEMTIME` |
|
||||
| `0x118` | 4 | accumulated playtime seconds |
|
||||
| `0x11c` | 4 | `SaveVersion1` / logical state-layout version |
|
||||
| `0x120` | 4 | `SaveVersion2` / payload-codec subversion |
|
||||
|
||||
The wrapper algorithm is shared, but Himegari's identities are domain-specific: shared `SAVE.DAT` and
|
||||
`RT.DAT` use compatibility id `0x4a343234`, while installed numbered `SAVE##.DAT` files use
|
||||
`0x42323234`. Both carry game id `姫狩りダンジョンマイスター` and versions 3.10.
|
||||
|
||||
`save_container_read_and_validate_header@0x4306f0` is the metadata-only path used by `0x1a0`.
|
||||
The full writer/reader are `save_container_encode_and_write@0x42fac0` and
|
||||
`save_container_read_and_decode@0x42ff80`. Immediately after the header is this exact 20-byte codec frame:
|
||||
|
||||
| Offset | Size | Field |
|
||||
|---:|---:|---|
|
||||
| `+0x00` | 4 | encoded DWORD count |
|
||||
| `+0x04` | 4 | MSB-first CRC-32 of encoded bytes |
|
||||
| `+0x08` | 4 | reflected CRC-32 of encoded bytes |
|
||||
| `+0x0c` | 4 | random rolling XOR seed |
|
||||
| `+0x10` | 4 | random odd multiplier (low 16 bits used) |
|
||||
|
||||
The encoded byte length is `encoded_dword_count * 4`. Each source DWORD is XORed with the current seed;
|
||||
its high and low 16-bit halves are independently multiplied by the current odd multiplier and stored as two
|
||||
DWORD products. Per source DWORD the seed advances by `0x0b0b0b0b` and the multiplier by `0x0b02`.
|
||||
The inverse requires both products to divide exactly, providing another corruption check. The decoded logical
|
||||
payload itself starts with two more DWORDs: MSB-first and reflected CRC-32 values over all following logical
|
||||
bytes.
|
||||
|
||||
For `SaveVersion2 < 2`, that checked logical buffer is transformed directly. For version 2 or later, the
|
||||
buffer is first passed through the native 4 KiB LZSS codec: a zero-filled 4096-byte ring starting at `0xfee`,
|
||||
groups of eight tokens under an LSB-first flag byte (`1=literal`), and two-byte matches containing a 12-bit
|
||||
offset plus a four-bit `length-3`. Incompressible data is stored verbatim. The transformed inner buffer begins
|
||||
with three DWORDs recording original byte length, consumed byte length, and stored byte length, followed by
|
||||
the compressed/verbatim bytes and native padding. The mapped helpers are `lzss_4k_compress@0x42ed30`,
|
||||
`save_container_read_and_decode@0x42ff80`. The canonical header, codec-frame, CRC, LZSS, and reversible
|
||||
DWORD-transform specification is in `sys4-format-notes.md`. The mapped helpers are
|
||||
`lzss_4k_compress@0x42ed30`,
|
||||
`lzss_4k_decompress@0x42f050`, `save_payload_expand_multiply_transform@0x42f400`,
|
||||
`save_payload_inverse_multiply_transform@0x42f4a0`, `crc32_msb_first@0x42f360`, and
|
||||
`crc32_reflected@0x42f300`.
|
||||
@@ -1772,30 +1710,21 @@ the compressed/verbatim bytes and native padding. The mapped helpers are `lzss_4
|
||||
`context_state_serialize@0x40d320` chooses numbered logical layout 1, 2, or 3 from `SaveVersion1`;
|
||||
layout 1 retains legacy `SaveVersion2` sublayouts 10 and 20. Layouts 2 and 3 serialize script contexts
|
||||
`0..save_frame_boundary_index` inclusive (falling back to the current context), clear the terminal frame's
|
||||
return target, and append `text_history_serialize`. Their frame record is `0x414` bytes (`0x105` DWORDs);
|
||||
layout 2's frame-zero/fixed prefix is `0x8f8` bytes and layout 3's is `0x5718` bytes. Layout 3's larger
|
||||
prefix adds a 20,000-byte surface/resource state block and retained graphics-object state; each retained
|
||||
object record carries its handle plus the native `0xb5`-DWORD object record. The state also carries six
|
||||
global-bank counts, raw integer banks, packed strings/other banks, and resource registrations (100 in the
|
||||
modern layouts). The matching `save_data_deserialize_and_begin_restore@0x40fd10` reconstructs those banks,
|
||||
resources, retained state, history, and—when requested by `0x1a1`—the active frame chain consumed by `0xae`.
|
||||
return target, and append `text_history_serialize`. The matching
|
||||
`save_data_deserialize_and_begin_restore@0x40fd10` reconstructs the banks, resources, retained state,
|
||||
history, and—when requested by `0x1a1`—the active frame chain consumed by `0xae`.
|
||||
|
||||
The thumbnail is a separate file, never part of that logical state. Both renderer paths prove `.STH` is an
|
||||
ordinary bottom-up 24-bit BMP under a nonstandard extension: `BM`, pixel offset `0x36`, a 40-byte info
|
||||
header, BGR pixels, and four-byte row padding. Installed Himegari thumbnails are 112x84 and 28,278 bytes.
|
||||
The native writer records `bfSize=28,264`—DIB header plus pixel bytes, omitting the 14-byte BMP file header
|
||||
even though that header and the `0x36` pixel offset are physically present. The handle-based path is
|
||||
`gfx_surface_write_bmp24_to_handle@0x434bf0`; backend 1 reads back supported D3D surface formats and passes
|
||||
them to `gfx_surface_write_bmp24_to_path@0x475420`. The load side uses the renderer image decoder. A
|
||||
compatibility implementation should therefore preserve the paired-file lifecycle and BMP payload rather than
|
||||
inventing a second save container.
|
||||
The complete Himegari layout-3 body, frame record, six-bank sequence, retained-gfx region, appended
|
||||
text-history tail, and `.STH` BMP format are canonical in `sys4-format-notes.md`. Native thumbnail
|
||||
helpers are `gfx_surface_write_bmp24_to_handle@0x434bf0` and
|
||||
`gfx_surface_write_bmp24_to_path@0x475420`; the load side uses the renderer image decoder.
|
||||
|
||||
**1.0 implementation boundary:** reproduce these native binary domains and lifecycle first: shared
|
||||
`SAVE.DAT`/`SAVE.BAK`, `RT.DAT`/`RT.BAK`, numbered `.DAT`, and paired BMP `.STH`. Keep the ownership behind
|
||||
a profile/save service so extended mode can later add JSON inspection/export, namespaced mod state, migrations,
|
||||
or a friendlier editor without changing compatibility-mode opcode semantics or the native import/export path.
|
||||
|
||||
**Port correspondence (2026-07-24, numbered pair/thumbnail layer implemented):**
|
||||
**Port correspondence (2026-07-24):**
|
||||
`Age.Engine.Persistence.NativeSaveContainerCodec` now reads and writes the common header, Shift-JIS game id,
|
||||
SYSTEMTIME/playtime/version metadata, both CRC layers, version-2 compression wrapper, and exact reversible
|
||||
DWORD transform. `Sys4.LzssEncoder` emits the same 4 KiB-ring token dialect already consumed by
|
||||
@@ -1811,37 +1740,14 @@ global pointers. `RT.DAT` is now implemented as the separate S3RT layer describe
|
||||
validates its distinct compatibility id, queries metadata without decoding payloads, and performs exact
|
||||
paired `.DAT`/`.STH` copy/delete status layering. `NumberedThumbnailCodec` reads and writes the native BMP
|
||||
dialect through host surface capture/replacement, and ops `0x1a0`, `0x1ab`–`0x1af` (including `0x1ad`'s
|
||||
frame marker) are wired. Godot injects the store at `user://SAVE`. Full layout-3 global/frame/history/
|
||||
`NativeNumberedSaveCodec`, `NativeTextHistoryCodec`, and `NativeGfxPersistenceCodec` now own the complete
|
||||
layout-3 numbered body and appended history tail. The VM wires `0x19e`, data-only `0x19f`, full load
|
||||
frame marker) are wired. Godot injects the store at `user://SAVE`.
|
||||
`NativeNumberedSaveCodec`, `NativeTextHistoryCodec`, and `NativeGfxPersistenceCodec` own the complete
|
||||
layout-3 numbered body, global/frame/gfx state, and appended history tail. The VM wires `0x19e`,
|
||||
data-only `0x19f`, full load
|
||||
`0x1a1`, and the active branch of `0xae`; the existing `GameSession` JSON snapshot remains a separate
|
||||
diagnostic/extended-mode surface.
|
||||
|
||||
#### Layout-3 byte map and port correspondence (2026-07-24)
|
||||
|
||||
The decoded layout-3 body begins with `cutoff` at `+0x00`; its fixed length is
|
||||
`0x5718 + cutoff*0x414`. The fixed prefix contains two saved engine/context words, ten state DWORDs,
|
||||
100 three-DWORD resource reload records, 1,000 20-byte surface reload records, then frames beginning at
|
||||
`+0x5304`. Each `0x414`-byte frame contains parent context, packed script id, up to 256 saved local-return
|
||||
indices, and two terminal control indices. The three SYS4 footer tables now have explicit runtime roles:
|
||||
|
||||
- d259 indexes T1/F7-F8, the op-`0x71` read-message reset sites;
|
||||
- d260 indexes T2/F9-F10, resumable `call-script` sites (forced to `-1` on the cutoff frame);
|
||||
- each local return index addresses T3/F11-F12, whose stored target plus three is the live return offset.
|
||||
|
||||
After the fixed bytes are six bank counts followed by the ordinary (de-protected) integer bank, float
|
||||
storage, a DWORD-sized CP932 string blob, the pointer-family banks, and retained graphics. Graphics begin
|
||||
with record size `0x2d4` and object count; every object is `{handle,0x2d4 raw record}`. The range-transform
|
||||
first/count and another `0x2d4` record follow. AGE overallocates this region
|
||||
(`0x2e1 + objectCount*0x2d8` DWORDs) relative to the records it writes; the codec preserves the native
|
||||
logical sizing and zeroes the unused allocation slack.
|
||||
|
||||
The text-history tail is outside the S4SD container. Its 12-byte header is
|
||||
`{logicalBytes,logicalBytes,storedBytes}` followed by the same 4 KiB-ring LZSS/verbatim dialect. Logical
|
||||
history is entry count + 8-byte entries, record count + the first 11 DWORDs of each retained record, then
|
||||
a DWORD-counted CP932 string blob whose DWORDs are bitwise inverted. Installed `SAVE00.DAT` validates the
|
||||
whole boundary read-only: cutoff 1, six counts `[402459,1,789,1,1,1]`, 211 gfx objects, a 21-byte history
|
||||
tail, and exact consumption of the S4SD container before that tail.
|
||||
#### Layout-3 restore mechanics and port correspondence (2026-07-24)
|
||||
|
||||
The port tracks the native global banks separately at runtime, captures the marked frame chain, serializes
|
||||
live surfaces and retained objects, and reloads host textures from the 20-byte surface records. Full load
|
||||
|
||||
@@ -3529,6 +3529,23 @@ extended-mode work.
|
||||
Validation: all 386 engine tests pass, global/opcode lint reports zero errors/warnings, the Godot C#
|
||||
build has zero warnings, and the threaded headless run reports `SELFTEST OK`.
|
||||
|
||||
### Persistence documentation closeout — canonical file-format specification (2026-07-24)
|
||||
|
||||
`docs/sys4-format-notes.md` now contains the canonical byte-level persistence specification for
|
||||
Himegari 3.10. It separates structural ownership from game semantics and documents the common
|
||||
S3SD/S4SD container and transform, shared `SAVE.DAT` selected-cell payload, standalone `RT.DAT`
|
||||
ReadTextDB, numbered layout-3 fixed/dynamic state and appended History stream, and paired `.STH` BMP.
|
||||
Installed-file oracle counts and exact known limits are recorded alongside each layout.
|
||||
|
||||
Overlapping byte-layout prose was removed from `docs/engine-re.md`; that reference now retains native
|
||||
function provenance, opcode behavior, save-root/lifecycle policy, and restoration mechanics while linking
|
||||
to the format specification. Game-global meanings remain sourced from `vm-map/globals.toml` and its
|
||||
generated reference. The spec explicitly leaves legacy numbered layouts 1/2, opaque shared sections,
|
||||
and unnamed retained-gfx fields outside the confirmed Himegari compatibility claim.
|
||||
|
||||
This documentation-only closeout does not change runtime persistence. The next functional check remains
|
||||
continuing from the installed slot through the restored gameplay frame.
|
||||
|
||||
## Data-semantics sidebar: focused append EBINIT inspection (2026-07-24)
|
||||
|
||||
The static INIT surface now accepts a universal packed script id for focused append inspection.
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
# SYS4 Script Format — Reverse-Engineering Notes (hex-first)
|
||||
# SYS4 Binary Format Notes (hex-first)
|
||||
|
||||
Derived purely from byte-pattern analysis of the 481 DATA1 `.BIN` scripts, before
|
||||
any Ghidra work on `AGE.EXE`. Confidence levels flagged per finding. Probe scripts
|
||||
live in `tools/probe_*.py`.
|
||||
The script sections were derived initially from byte-pattern analysis of the 481 DATA1 `.BIN`
|
||||
files, before Ghidra work on `AGE.EXE`; their probe scripts live in `tools/probe_*.py`. The
|
||||
persistence sections combine native-code reconstruction, shipped-script consumers, installed
|
||||
read-only files, and port codec round trips. Confidence and remaining semantic limits are stated
|
||||
per format.
|
||||
|
||||
> **UPDATE 2026-07-05 — opcode set solved via Kelebek1's table.** Everything below
|
||||
> under "Instruction stream — PARTIAL" is now resolved: code = instructions of
|
||||
@@ -31,6 +33,287 @@ The same Himegari trailer also contains `USEAPPDATAFOLDER=1` and
|
||||
settings to select `%LOCALAPPDATA%` plus that relative path on modern Windows. Filename patterns remain
|
||||
native policy rather than trailer or script strings; see `engine-re.md` under "Save-root resolution."
|
||||
|
||||
## Native persistence files — Himegari 3.10
|
||||
|
||||
This section is the canonical binary-format specification for the persistence files used by Himegari.
|
||||
Native function provenance, opcode behavior, filesystem policy, and restore control flow remain in
|
||||
`engine-re.md`; game-global meanings remain sourced from `vm-map/globals.toml` and its generated
|
||||
`global-reference.md`.
|
||||
|
||||
Unless noted otherwise, offsets are byte offsets, integers are little-endian DWORDs, and strings are
|
||||
NUL-terminated CP932. Himegari separates four physical domains:
|
||||
|
||||
| File | Scope | Structural owner | Semantic owner |
|
||||
|---|---|---|---|
|
||||
| `SAVE.DAT` / `SAVE.BAK` | whole profile | AGE's typed selected-cell payload inside the common save container | scripts choose which integer/string global cells to persist |
|
||||
| `RT.DAT` / `RT.BAK` | whole profile | AGE's standalone `S3RT` ReadTextDB | script resource ids and T1 message boundaries identify the flags |
|
||||
| `SAVE%02d.DAT` | one numbered slot | AGE's common container, full VM-state layout, and appended text-history tail | scripts populate live state, choose the frame cutoff, and request a slot save |
|
||||
| `SAVE%02d.STH` | one numbered slot | AGE's BMP writer/reader | scripts choose the captured surface and slot |
|
||||
|
||||
The menu-preview record is not read from `RT.DAT`. Its timestamp/playtime comes from the numbered
|
||||
`.DAT` header, its screenshot comes from `.STH`, and its title/location/name/stats/difficulty/ending
|
||||
history come from slot-indexed selected cells in shared `SAVE.DAT`. The numbered `.DAT` still contains
|
||||
the full restorable VM state; `SAVE.DAT` holds presentation-oriented copies so `SAVE.BIN` can list slots
|
||||
without decoding every numbered body.
|
||||
|
||||
### Common `S3SD` / `S4SD` `.DAT` container — CONFIRMED
|
||||
|
||||
Shared `SAVE.DAT` and numbered `SAVE%02d.DAT` use the same wrapper. `RT.DAT` does not: it has the
|
||||
standalone `S3RT` layout documented below. The fixed `.DAT` header is `0x124` bytes:
|
||||
|
||||
| Offset | Size | Field |
|
||||
|---:|---:|---|
|
||||
| `0x000` | 4 | ASCII magic `S3SD` or `S4SD` |
|
||||
| `0x004` | 4 | compatibility id |
|
||||
| `0x008` | `0x100` | NUL-terminated CP932 game id area |
|
||||
| `0x108` | `0x10` | Win32 `SYSTEMTIME`: eight little-endian WORDs |
|
||||
| `0x118` | 4 | accumulated playtime in seconds |
|
||||
| `0x11c` | 4 | signed `SaveVersion1`, selecting the logical state layout |
|
||||
| `0x120` | 4 | signed `SaveVersion2`, selecting the payload-codec subversion |
|
||||
|
||||
Himegari uses game id `姫狩りダンジョンマイスター` and versions 3.10. Shared `SAVE.DAT` uses
|
||||
compatibility id `0x4a343234`; installed numbered files use the distinct id `0x42323234`.
|
||||
|
||||
A `0x14`-byte codec frame begins at file offset `0x124`, and expanded data begins at `0x138`:
|
||||
|
||||
| Frame offset | File offset | Size | Field |
|
||||
|---:|---:|---:|---|
|
||||
| `+0x00` | `0x124` | 4 | expanded-data DWORD count; always even |
|
||||
| `+0x04` | `0x128` | 4 | MSB-first CRC-32 of expanded bytes |
|
||||
| `+0x08` | `0x12c` | 4 | reflected CRC-32 of expanded bytes |
|
||||
| `+0x0c` | `0x130` | 4 | rolling XOR seed |
|
||||
| `+0x10` | `0x134` | 4 | nonzero odd multiplier; only the low 16 bits are valid |
|
||||
|
||||
The logical payload is protected in this order:
|
||||
|
||||
1. Prefix the payload with two DWORD CRCs over the payload: MSB-first CRC-32, then reflected CRC-32.
|
||||
2. For `SaveVersion2 >= 2`, optionally compress that checked buffer with the 4 KiB LZSS dialect and
|
||||
wrap it as `{original_bytes, consumed_bytes, stored_bytes, stored_data, padding}`. The first two
|
||||
lengths agree. If compression does not shrink the data, `stored_bytes == original_bytes` and the
|
||||
bytes are verbatim.
|
||||
3. XOR each source DWORD with the current seed. Multiply its high and low 16-bit halves separately by
|
||||
the current odd multiplier, emitting two DWORD products. Advance the seed by `0x0b0b0b0b` and the
|
||||
16-bit multiplier by `0x0b02` for each source DWORD.
|
||||
4. Store CRCs of the expanded products in the outer codec frame.
|
||||
|
||||
The inverse requires both products to divide exactly by the current multiplier and both quotients to
|
||||
fit 16 bits. This supplies an integrity check in addition to the two outer and two inner CRCs.
|
||||
|
||||
The LZSS stream uses a zero-filled 4096-byte ring starting at `0xfee`. Each LSB-first flag byte controls
|
||||
eight tokens (`1=literal`); a match token stores a 12-bit ring offset and four-bit `length-3`, allowing
|
||||
lengths 3 through 18. The container's expanded-data count determines where it ends. Numbered files may
|
||||
append another domain after that boundary, so a reader must retain the consumed byte count rather than
|
||||
assuming the container occupies the entire physical file.
|
||||
|
||||
### Shared `SAVE.DAT` logical payload — CONFIRMED
|
||||
|
||||
After decoding the common container, the shared payload is sequential:
|
||||
|
||||
| Order | Field |
|
||||
|---:|---|
|
||||
| 1 | `catalog_count:u32`, then `catalog_values[catalog_count]:u32` |
|
||||
| 2 | `integer_count:u32`, then `integer_entries[integer_count]`, 16 bytes each |
|
||||
| 3 | `string_count:u32`, `string_blob_dwords:u32`, then the DWORD-padded string blob |
|
||||
| 4 | for version 3.10 or later, `selector_counts[256]:u32` |
|
||||
| 5 | for version 3.10 or later, `extended_count:u32`, then `extended_values[extended_count]:u32` |
|
||||
| 6 | `reserved_tail[9]:u32` |
|
||||
|
||||
An integer entry is:
|
||||
|
||||
| Entry offset | Size | Field |
|
||||
|---:|---:|---|
|
||||
| `+0x00` | `0x0c` | typed key field |
|
||||
| `+0x0c` | 4 | raw integer value |
|
||||
|
||||
The meaningful key bytes are `{type_tag, address_hex[8], NUL}`. Integer cells use raw type byte
|
||||
`0x03`; the address is eight ASCII hexadecimal digits naming the resolved VM global-bank index.
|
||||
Bytes 10 and 11 in the fixed integer key field are unused native stack residue and cannot be required
|
||||
to be zero when importing original data.
|
||||
|
||||
The string blob repeats `{typed_key_cstring, value_cstring}` `string_count` times. String keys use raw
|
||||
type byte `0x05` plus the same eight-digit address and NUL; values are CP932. The stored blob length is
|
||||
a DWORD count. AGE rounds with `(unpadded_bytes / 4) + 1`, deliberately adding a whole zero DWORD when
|
||||
the last value already ends on a DWORD boundary.
|
||||
|
||||
The nine-DWORD tail consists of one explicit terminator plus eight DWORDs produced by the native
|
||||
allocation formula. Catalog, selector, extended, and tail sections are structurally bounded but not
|
||||
fully assigned game-level meanings, so compatibility import/export preserves them rather than treating
|
||||
them as mod storage.
|
||||
|
||||
The installed Himegari profile is a complete oracle: version 3.10 decodes to 13,210 catalog values,
|
||||
24,070 integer cells, 605 string cells, selector entry 1 equal to 81, 83 extended values, and nine tail
|
||||
DWORDs.
|
||||
|
||||
The integer/string maps are AGE's generic selected-cell service, not copies of whole global banks.
|
||||
Himegari scripts assign their meanings through opcodes `0x1a2`/`0x1a3` and `0x1a9`/`0x1aa`. Known
|
||||
slot-preview banks include chapter/title, location, protagonist name, portrait entity, level, training
|
||||
count, growth, personality, difficulty, cleared-ending mask, and installed-append mask. Their canonical
|
||||
addresses and semantics are generated in `global-reference.md` from `vm-map/globals.toml`; the
|
||||
cleared-ending value is a 15-bit NG+/inheritance mask whose set bits select the displayed ending badges.
|
||||
|
||||
### Shared `RT.DAT` (`S3RT`) — CONFIRMED
|
||||
|
||||
`RT.DAT` is not wrapped, transformed, compressed, or checksummed like the `.DAT` save container. It is
|
||||
a standalone ReadTextDB with a `0x114`-byte header:
|
||||
|
||||
| Offset | Size | Field |
|
||||
|---:|---:|---|
|
||||
| `0x000` | 4 | ASCII magic `S3RT` |
|
||||
| `0x004` | 4 | compatibility id (`0x4a343234` for Himegari) |
|
||||
| `0x008` | `0x100` | NUL-terminated CP932 game id area |
|
||||
| `0x108` | 4 | major version (`1`) |
|
||||
| `0x10c` | 4 | minor version (`0`) |
|
||||
| `0x110` | 4 | script-record count |
|
||||
|
||||
At `0x114` are all 12-byte script records:
|
||||
|
||||
| Record offset | Size | Field |
|
||||
|---:|---:|---|
|
||||
| `+0x00` | 4 | raw packed SYS4/AAI script resource id |
|
||||
| `+0x04` | 4 | message count |
|
||||
| `+0x08` | 4 | serialized native heap pointer residue |
|
||||
|
||||
After the complete record table, each record's `message_count` DWORD flags follows in record order.
|
||||
Zero means unread and nonzero means read. The third record word is process-local residue rather than a
|
||||
portable offset; AGE allocates and replaces it on load. Readers accept original nonzero values, while
|
||||
portable writers emit zero.
|
||||
|
||||
Message indices are positions in that script's T1/F7 table, whose entries point to op-`0x71`
|
||||
read-message boundaries. Thus the file is profile-wide engine state derived from game scripts, not an
|
||||
arbitrary script-selected persistence table and not numbered-slot state.
|
||||
|
||||
The installed Himegari file is exactly 76,752 bytes: 192 records plus 18,543 flag DWORDs consume it
|
||||
without trailing data. Base script `SC0000` has packed id `0x22` and 320 flags, exactly matching its
|
||||
T1 count.
|
||||
|
||||
### Numbered `SAVE%02d.DAT`, logical layout 3 — CONFIRMED FOR HIMEGARI
|
||||
|
||||
Himegari's numbered file begins with the common container using compatibility id `0x42323234`,
|
||||
`SaveVersion1=3`, and `SaveVersion2=10`. The header supplies the save-menu timestamp and accumulated
|
||||
playtime without decoding the body.
|
||||
|
||||
The decoded layout-3 body begins:
|
||||
|
||||
| Offset | Size | Field |
|
||||
|---:|---:|---|
|
||||
| `0x000` | 4 | terminal saved-frame index, called `cutoff`; frame count is `cutoff + 1` |
|
||||
| `0x004` | 4 | saved frame-owner/context word |
|
||||
| `0x008` | 4 | engine-state word |
|
||||
| `0x00c` | `0x28` | ten engine/context state DWORDs |
|
||||
| `0x034` | `0x4b0` | 100 resource-reload records of three DWORDs |
|
||||
| `0x4e4` | `0x4e20` | 1,000 surface-reload records of 20 bytes |
|
||||
| `0x5304` | `(cutoff + 1) * 0x414` | saved script-frame records |
|
||||
| `0x5718 + cutoff * 0x414` | variable | global banks and retained graphics |
|
||||
|
||||
Each `0x414`-byte frame is:
|
||||
|
||||
| Frame offset | Size | Field |
|
||||
|---:|---:|---|
|
||||
| `+0x000` | 4 | parent context |
|
||||
| `+0x004` | 4 | packed script resource id |
|
||||
| `+0x008` | 4 | saved local-return count, at most 256 |
|
||||
| `+0x00c` | up to `0x400` | T3 local-return indices |
|
||||
| `+0x40c` | 4 | T1 resume/message-boundary index |
|
||||
| `+0x410` | 4 | T2 resumable call-script index; forced to `-1` on the cutoff frame |
|
||||
|
||||
The T1/T2/T3 indices refer to the three footer tables in the saved SYS4 script. T1 restores the active
|
||||
message boundary, T2 restores the inter-script call site, and T3 entries restore local returns as
|
||||
`T3[index] + 3`.
|
||||
|
||||
The variable section starts with six DWORD counts in this order:
|
||||
|
||||
1. integer globals;
|
||||
2. raw float storage;
|
||||
3. string globals;
|
||||
4. integer-pointer globals;
|
||||
5. string-pointer globals;
|
||||
6. local pointer scratch.
|
||||
|
||||
The integer and float arrays follow their counts. String count comes from bank count 3, but its storage
|
||||
begins with an additional `string_blob_dwords` followed by that many bytes of concatenated NUL-terminated
|
||||
CP932 strings and DWORD padding. The three pointer-family DWORD arrays follow.
|
||||
|
||||
Retained graphics then uses:
|
||||
|
||||
```text
|
||||
0x00 gfx_record_size:u32 # 0x2d4
|
||||
+0x04 gfx_object_count:u32
|
||||
+0x08 repeated { handle:u32, record[0x2d4] }
|
||||
... range_first:u32
|
||||
... range_count:i32
|
||||
... range_transform_record[0x2d4]
|
||||
... native allocation slack
|
||||
```
|
||||
|
||||
The 1,000 surface records preserve resource reload state; mapped fields include resource id at `+0x00`,
|
||||
packed color key at `+0x04`, and presence at `+0x08`. A retained `0x2d4` record is structurally complete
|
||||
but not every internal graphics field is semantically named. The native allocation is larger than the
|
||||
records actually written (`0x2e1 + object_count * 0x2d8` DWORDs in the graphics sizing term), leaving
|
||||
zero/slack bytes after the meaningful range record.
|
||||
|
||||
The installed `SAVE00.DAT` validates the complete layout-3 decode: cutoff 1, global-bank counts
|
||||
`[402459,1,789,1,1,1]`, and 211 retained graphics objects.
|
||||
|
||||
#### Appended text-history tail
|
||||
|
||||
The common container's expanded-data count ends before the numbered file ends. Layout 3 appends a
|
||||
separate text-history stream immediately at that consumed boundary:
|
||||
|
||||
| Tail offset | Size | Field |
|
||||
|---:|---:|---|
|
||||
| `+0x00` | 4 | logical byte length |
|
||||
| `+0x04` | 4 | duplicate logical byte length |
|
||||
| `+0x08` | 4 | stored byte length |
|
||||
| `+0x0c` | variable | the same 4 KiB-ring LZSS stream, or verbatim bytes when lengths match |
|
||||
|
||||
The decoded history data is:
|
||||
|
||||
1. `entry_count:u32`, then `entry_count` pairs `{layout_slot:i32, first_record_index:i32}`;
|
||||
2. `record_count:u32`, then `record_count` records of 11 DWORDs;
|
||||
3. `string_blob_dwords:u32`, then a DWORD-aligned CP932 string blob whose DWORDs are bitwise inverted.
|
||||
|
||||
Each 11-DWORD record stores layout slot, origin X/Y, width/height, primary value, auxiliary value,
|
||||
primary font size, text color, cursor Y, and flags. The string blob contains one NUL-terminated string
|
||||
per record. The installed save has a 21-byte physical history tail, and container plus tail consume the
|
||||
file exactly.
|
||||
|
||||
### Numbered `SAVE%02d.STH` thumbnail — CONFIRMED
|
||||
|
||||
The thumbnail is a separate ordinary BMP under a nonstandard extension:
|
||||
|
||||
- `BM` file signature and `0x36` pixel offset;
|
||||
- 14-byte bitmap file header plus 40-byte `BITMAPINFOHEADER`;
|
||||
- uncompressed 24-bit BGR pixels;
|
||||
- bottom-up positive height;
|
||||
- rows padded to four-byte boundaries.
|
||||
|
||||
Installed Himegari thumbnails are 112x84 and 28,278 physical bytes. The native writer stores
|
||||
`bfSize=28,264`, omitting the physically present 14-byte file header from that field while retaining the
|
||||
correct physical header and pixel offset. Compatibility writers reproduce this harmless historical quirk.
|
||||
|
||||
Copy, move, and delete operations treat `.DAT` and `.STH` as a pair, but neither file contains the other.
|
||||
|
||||
### Scope and remaining uncertainty
|
||||
|
||||
The following are complete enough for compatible import/export:
|
||||
|
||||
- the common S3SD/S4SD header, CRCs, optional LZSS wrapper, and DWORD transform;
|
||||
- Himegari's shared `SAVE.DAT` 3.10 section boundaries and typed selected cells;
|
||||
- `RT.DAT` 1.0 records and flag arrays;
|
||||
- Himegari's numbered layout-3 fixed state, global banks, frames, graphics region, and history tail;
|
||||
- `.STH` BMP encoding.
|
||||
|
||||
The remaining uncertainty is deliberately narrower:
|
||||
|
||||
- numbered layouts 1 and 2 are structurally identified but are not the implemented or installed
|
||||
Himegari compatibility target;
|
||||
- the shared catalog/extended arrays and some selected cells are still opaque at the game-semantic level;
|
||||
- the `0x2d4` retained-graphics record is byte-bounded, but not every field is named.
|
||||
|
||||
This distinction is important: AGE owns most numbered-save and ReadTextDB structure, while Himegari
|
||||
scripts primarily own the values in live VM globals, the save cutoff/invocation, and the semantic choice
|
||||
of selected `SAVE.DAT` cells.
|
||||
|
||||
## Header — CONFIRMED
|
||||
|
||||
Fixed 60-byte (0x3C) header: 8-byte magic + thirteen little-endian u32 fields.
|
||||
|
||||
Reference in New Issue
Block a user