Implement native shared profile persistence
This commit is contained in:
gamer147
@@ -103,7 +103,8 @@ S:\Game Hacking\Eushully\Himegari\ ← workspace root (three siblings)
|
||||
├── engine/ DELIVERABLE — the .NET VM core (AgeEngine.sln: Age.Engine / Age.Cli / tests)
|
||||
│ ├── Age.Engine/Sys4/ runtime catalog parser, loose-first bounded ALF asset store,
|
||||
│ script provider, AGF/LZSS and Windows CUR decoders, and resource facade
|
||||
│ └── Age.Engine/Persistence/ native S3SD/S4SD container codec and profile/numbered DAT lifecycle seam
|
||||
│ └── Age.Engine/Persistence/ native S3SD/S4SD container + shared-payload codecs,
|
||||
│ profile-owned selected cells, and shared/numbered DAT lifecycle seam
|
||||
├── native/ authored native runtime boundaries
|
||||
│ └── age_movie_ffmpeg/ project-owned FFmpeg C ABI, immutable Windows dependency manifest,
|
||||
│ and bootstrap/build scripts (outputs stay under disposable build/)
|
||||
|
||||
@@ -198,14 +198,12 @@ and pinning the real handler plus its paired reader resolves the service:
|
||||
globals with consecutive loads; `SYSTEM4.BIN` stores its initialized-config flag; gameplay and ADV
|
||||
scripts store selected array cells rather than the whole VM global bank.
|
||||
|
||||
**Port verdict: semantics solved, persistence implementation deferred.** The old unread `GfxState`
|
||||
`HashSet` and legacy VM handler have been removed, so both opcodes now appear as effectful gaps rather than
|
||||
false implementations. A faithful service needs a profile-owned `global-cell-index → raw-int32` map shared
|
||||
across VM/script lifetimes, with `0x1a2` upsert and `0x1a3` load-or-zero, then a deliberate persistence
|
||||
boundary. The current `GameSession` JSON serializes the entire global bank, which can accidentally preserve
|
||||
some values but cannot reproduce AGE's selected-cell restore/reset lifecycle. Do not add another ad-hoc JSON
|
||||
field until the unified shared `SAVE.DAT`/`RT.DAT`/numbered-save architecture chooses ownership and migration.
|
||||
This deferral is now explicit in opcode coverage; it is not a safe-noop claim.
|
||||
**Port verdict (implemented 2026-07-24):** `SharedProfile` now owns the selected
|
||||
`global-cell-index → raw-int32` map across VM/script lifetimes. Opcode `0x1a2` upserts and `0x1a3`
|
||||
loads-or-zero for direct globals and local pointers resolved into the global bank. The profile is shared by
|
||||
fresh scene VMs through `GameSession`, while `GameSession` JSON remains intentionally unchanged and does not
|
||||
silently alias this native profile domain. The paired string service and native `SAVE.DAT` payload/store
|
||||
lifecycle are implemented with it; see the persistence-family section below.
|
||||
|
||||
The FIELD snippet `lookup(0x5f0ed, 0x62ccf); mov(ptr,1); lookup(...); 0x1a2(ptr)` therefore persists that
|
||||
selected global array cell to the shared profile. It does not resolve decision→scene; scene dispatch remains
|
||||
@@ -1648,6 +1646,32 @@ Thumbnail I/O uses `0=success`, `1=open/create failure`, and `2=codec failure`.
|
||||
expects its caller to pre-seed status zero: on success it starts the resume sequence without rewriting that
|
||||
operand, while a missing file writes one.
|
||||
|
||||
#### Save-root resolution
|
||||
|
||||
Save paths are native engine policy, not script-provided strings. Every numbered/shared handler first calls
|
||||
`save_root_resolve@0x40b880`, then formats one of the engine-owned names (`SAVE%2.2d.DAT`,
|
||||
`SAVE%2.2d.STH`, `SAVE.DAT`, `SAVE.BAK`, `RT.DAT`, and their `$$` temporary names) under that root.
|
||||
The script operands select the operation and numbered slot only.
|
||||
|
||||
The resolver reads two per-game settings from the SYS4INI-backed settings registry:
|
||||
`set:UseAppDataFolder` and `set:SavePath`. When `UseAppDataFolder == 1`, the modern-Windows branch
|
||||
loads `SHGetFolderPathA` and requests `CSIDL_LOCAL_APPDATA | CSIDL_FLAG_CREATE` (`0x801c`); it then
|
||||
appends `SavePath`. Otherwise the configured `SavePath` is used directly. It probes
|
||||
`<root>\SAVE.DAT`; if that file does not exist, it walks the root and creates missing directory
|
||||
components before returning it.
|
||||
|
||||
Himegari's SYS4INI trailer supplies:
|
||||
|
||||
```text
|
||||
USEAPPDATAFOLDER=1
|
||||
SAVEPATH=Eushully\姫狩りダンジョンマイスター\SAVE
|
||||
```
|
||||
|
||||
Thus its normal modern-Windows root is
|
||||
`%LOCALAPPDATA%\Eushully\姫狩りダンジョンマイスター\SAVE`. The complete filenames and backup
|
||||
lifecycle are fixed by AGE after that root has been selected; no persistence opcode accepts an arbitrary
|
||||
path or filename.
|
||||
|
||||
#### Shared `SAVE.DAT`
|
||||
|
||||
`shared_profile_save@0x40c950` writes `$$SAVE.DAT`, replaces `SAVE.DAT`, and keeps `SAVE.BAK`; it performs
|
||||
@@ -1658,17 +1682,27 @@ use this backup transaction, but a successful numbered serialization flushes the
|
||||
After the common container decode, the shared logical payload is:
|
||||
|
||||
1. a DWORD catalog/compatibility count followed by that many DWORDs;
|
||||
2. a DWORD integer-entry count, then fixed 16-byte entries `{ascii_key[12], raw_value_u32}`;
|
||||
3. a DWORD string-entry count and DWORD padded string-blob length, then concatenated
|
||||
`ascii_key\0value\0` pairs with DWORD padding;
|
||||
2. a DWORD integer-entry count, then fixed 16-byte entries `{typed_key[12], raw_value_u32}`;
|
||||
3. a DWORD string-entry count and a string-blob **DWORD count**, then concatenated
|
||||
`typed_key\0value\0` pairs with DWORD padding;
|
||||
4. for shared save version at least 3.10, a 256-DWORD selector/count table, a DWORD extra-count, and that
|
||||
many extra DWORDs;
|
||||
5. a trailing zero DWORD.
|
||||
5. one explicit trailing zero DWORD plus eight historical allocation-slack DWORDs (nine reserved tail
|
||||
DWORDs in decoded native Himegari files).
|
||||
|
||||
Integer-cell ops `0x1a2`/`0x1a3` use keys `3%08x`; string-cell ops `0x1a9`/`0x1aa` use `5%08x`. In both
|
||||
cases the hexadecimal portion is the VM lvalue's resolved global-bank index. The string table therefore
|
||||
is not an incidental settings blob: it is the paired profile-wide selected-cell service for string globals,
|
||||
with insert-or-assign and empty-on-miss behavior.
|
||||
The key prefix is binary, not the ASCII character suggested by shorthand such as `3%08x`:
|
||||
integer-cell ops `0x1a2`/`0x1a3` use raw byte `0x03` followed by eight lowercase ASCII hex digits;
|
||||
string-cell ops `0x1a9`/`0x1aa` use raw byte `0x05` plus the same address text. In both cases the
|
||||
hexadecimal portion is the VM lvalue's resolved global-bank index. Integer fixed-key bytes 10 and 11 are
|
||||
unused native stack padding and cannot be required to be zero when importing an original file. The string
|
||||
table is therefore not an incidental settings blob: it is the paired profile-wide selected-cell service
|
||||
for string globals, with insert-or-assign and empty-on-miss behavior.
|
||||
|
||||
The installed native Himegari `SAVE.DAT` provides a direct compatibility oracle for this shape:
|
||||
version 3.10 decodes to 13,210 opaque catalog values, 24,070 integer cells, 605 string cells, selector
|
||||
entry 1 equal to 81, 83 extended values, and the nine-DWORD reserved tail. The port decoder imports that
|
||||
file exactly; catalog/extended/tail sections remain opaque and are preserved losslessly rather than
|
||||
reinterpreted as mod state.
|
||||
|
||||
#### Common `.DAT` container
|
||||
|
||||
@@ -1740,15 +1774,20 @@ inventing a second save container.
|
||||
a profile/save service so extended mode can later add JSON inspection/export, namespaced mod state, migrations,
|
||||
or a friendlier editor without changing compatibility-mode opcode semantics or the native import/export path.
|
||||
|
||||
**Port correspondence (2026-07-24, codec foundation implemented):**
|
||||
**Port correspondence (2026-07-24, shared payload and selected cells implemented):**
|
||||
`Age.Engine.Persistence.NativeSaveContainerCodec` now reads and writes the common header, Shift-JIS game id,
|
||||
SYSTEMTIME/playtime/version metadata, both CRC layers, version-2 compression wrapper, and exact reversible
|
||||
DWORD transform. `Sys4.LzssEncoder` emits the same 4 KiB-ring token dialect already consumed by
|
||||
`LzssDecoder`, falling back to native verbatim storage when compression does not shrink. The
|
||||
`INativeDatStore` boundary and `DirectoryNativeDatStore` own shared `$$SAVE.DAT` → `SAVE.DAT` /
|
||||
`SAVE.BAK` replacement/fallback and direct numbered `SAVE##.DAT` writes. Payload schemas, `RT.DAT`,
|
||||
thumbnails, and opcode wiring remain deliberately above or after this layer; the existing `GameSession`
|
||||
JSON snapshot is unchanged.
|
||||
`SAVE.BAK` replacement/fallback and direct numbered `SAVE##.DAT` writes.
|
||||
|
||||
`SharedProfilePayloadCodec` owns the typed shared logical layout, including native CP932 strings and
|
||||
lossless opaque-section preservation. `SharedProfile` owns selected integer/string maps and explicit
|
||||
load/save lifecycle; `GameSession` injects it into every fresh VM. Opcodes `0x1a2`/`0x1a3` and
|
||||
`0x1a9`/`0x1aa` now implement native upsert and missing-value defaults for direct cells and resolved
|
||||
global pointers. `RT.DAT`, thumbnails, and numbered active-frame payloads remain later layers; the existing
|
||||
`GameSession` JSON snapshot is unchanged.
|
||||
|
||||
### Opcode `0xae` continues numbered-save stack restoration (2026-07-20)
|
||||
|
||||
@@ -2880,9 +2919,9 @@ These handlers and their newly understood workers are renamed/commented in Ghidr
|
||||
The exact source metadata lives in `vm-map/opcodes.toml`. The port now implements this cluster against the
|
||||
shared Godot frame clock, retained graphics state, movie decoder state, mutable RGBA surfaces, and delayed
|
||||
voice service. `0x207` copies colorkey-baked source pixels into immutable published snapshots so compositor
|
||||
reads cannot race VM-side mutations. The first complete player attack remains the manual acceptance test;
|
||||
the implementation deliberately leaves BTL's two `0x1a2` shared-profile writes deferred because they do
|
||||
not feed same-exchange combat state.
|
||||
reads cannot race VM-side mutations. The first complete player attack remains the manual acceptance test.
|
||||
BTL's two `0x1a2` shared-profile writes were initially deferred because they do not feed same-exchange
|
||||
combat state; the unified profile service now handles them through the ordinary opcode dispatch.
|
||||
|
||||
### Movement/attack flood-fill FIFO -- opcodes `0x132`-`0x134` (2026-07-21)
|
||||
|
||||
|
||||
@@ -496,32 +496,32 @@ Status 0 means valid metadata was written, 1 means the file could not be opened,
|
||||
The caller pre-seeds status to zero. A missing/open failure writes 1; success starts the asynchronous native stack-restoration rendezvous and does not overwrite that zero. This is the ordinary load-game path, unlike data-only opcode 0x19f.
|
||||
|
||||
### 0x1a2 `store-shared-profile-int` (store-shared-profile-int, argc 1)
|
||||
- **summary:** 0x1a2 (cell) — snapshot the selected global integer cell into AGE's shared SAVE.DAT profile table. The key is `3%08x`, where the address is resolved through global/local pointer operands; the stored value is the cell's current raw 32-bit value. Insert-or-assign semantics replace an existing entry.
|
||||
- **summary:** 0x1a2 (cell) — snapshot the selected global integer cell into AGE's shared SAVE.DAT profile table. Its native key is raw type byte `0x03` followed by eight lowercase ASCII hex digits for the lvalue's resolved global-bank index; the stored value is the cell's current raw 32-bit value. Insert-or-assign semantics replace an existing entry.
|
||||
- **grounding:** source=investigation, confidence=high
|
||||
- **evidence:** Ghidra /v2: op_0x1a2_store_shared_profile_int@0x42d360 reads operand 1 with vm_operand_fetch, resolves its cell index with vm_operand_lvalue (accepted types 3/6/0xc), formats `3%08x`, and calls hash_table_insert_or_assign with ECX=ctx+0x5190. shared_profile_payload_write@0x430a20 serializes each 12-byte key plus 32-bit value to SAVE.DAT. Corpus: 17,585 calls in 315 scripts; 17,539 use local-ptr after lookup-array and 46 use global-int.
|
||||
|
||||
Paired reader 0x1a3 restores the same cell, returning zero when the key is absent. The table is embedded at ctx+0x5190, explicitly enumerated by shared_profile_payload_write into shared SAVE.DAT, and reconstructed by shared_profile_payload_read. It is not the retained gfx map, numbered SAVE##.DAT state, or RT.DAT ReadTextDB. Port status (2026-07-20): deliberately unimplemented pending the unified shared-profile storage boundary; the former inert GfxState HashSet approximation was removed so coverage reports the gap.
|
||||
Paired reader 0x1a3 restores the same cell, returning zero when the key is absent. The table is embedded at ctx+0x5190, explicitly enumerated by shared_profile_payload_write into shared SAVE.DAT, and reconstructed by shared_profile_payload_read. It is not the retained gfx map, numbered SAVE##.DAT state, or RT.DAT ReadTextDB. Port status (2026-07-24): implemented through profile-owned SharedProfile state shared across fresh scene VMs and the native shared-payload/store lifecycle; direct globals and resolved global local-pointers are covered.
|
||||
|
||||
### 0x1a3 `load-shared-profile-int` (load-shared-profile-int, argc 1)
|
||||
- **summary:** 0x1a3 (cell) — restore the selected global integer cell from AGE's shared SAVE.DAT profile table. It resolves the same `3%08x` cell-address key as 0x1a2 and overwrites the operand with the stored raw 32-bit value, or zero when the key is absent.
|
||||
- **summary:** 0x1a3 (cell) — restore the selected global integer cell from AGE's shared SAVE.DAT profile table. It resolves the same raw `0x03` plus eight-hex-digit cell-address key as 0x1a2 and overwrites the operand with the stored raw 32-bit value, or zero when the key is absent.
|
||||
- **grounding:** source=investigation, confidence=high
|
||||
- **evidence:** Ghidra /v2: op_0x1a3_load_shared_profile_int@0x427e90 resolves operand 1 with vm_operand_lvalue, calls shared_profile_int_lookup@0x4199d0 with the shared-profile object at ctx+0x4d7c, then vm_operand_write stores the result. The wrapper searches the table at object+0x414 = ctx+0x5190 and returns zero on a miss. Current override-aware corpus: 77 calls in 12 scripts.
|
||||
|
||||
This is the read half of the shared-profile integer service, not a string operation. LOADCONFIG.BIN restores configuration globals with it; GAMESTART/GAMECLEAR and array-pointer call sites restore other selected profile-wide values. Port status (2026-07-20): deliberately unimplemented with 0x1a2 pending the unified shared-profile storage boundary.
|
||||
This is the read half of the shared-profile integer service, not a string operation. LOADCONFIG.BIN restores configuration globals with it; GAMESTART/GAMECLEAR and array-pointer call sites restore other selected profile-wide values. Port status (2026-07-24): implemented with 0x1a2 through the shared profile service and native payload codec.
|
||||
|
||||
### 0x1a9 `store-shared-profile-string` (store-shared-profile-string, argc 1)
|
||||
- **summary:** (cell) - snapshot the selected global string cell into AGE's shared `SAVE.DAT` profile table under key `5%08x`, replacing any earlier value.
|
||||
- **summary:** (cell) - snapshot the selected global string cell into AGE's shared `SAVE.DAT` profile table under a raw `0x05` type byte plus eight lowercase ASCII hex digits for the resolved global-bank index, replacing any earlier value.
|
||||
- **grounding:** source=investigation, confidence=high
|
||||
- **evidence:** Ghidra /v2: op_0x1a9_store_shared_profile_string@0x42d3e0 fetches operand 1's string, resolves the lvalue cell index, and calls shared_profile_store_string_by_typed_key@0x42d2b0 with type prefix 5. Corpus: 17 calls in GAMECLEAR, INPUTNAME, SAVE, and SELSTAGE.
|
||||
|
||||
This is the string counterpart to integer-store opcode 0x1a2. The value is profile-wide rather than numbered-slot-local and is written by the shared profile lifecycle.
|
||||
This is the string counterpart to integer-store opcode 0x1a2. The value is profile-wide rather than numbered-slot-local and is written by the shared profile lifecycle. Port status (2026-07-24): implemented for direct global strings and resolved global string-pointers.
|
||||
|
||||
### 0x1aa `load-shared-profile-string` (load-shared-profile-string, argc 1)
|
||||
- **summary:** (cell) - restore the selected global string cell from AGE's shared `SAVE.DAT` profile table using key `5%08x`; a missing key yields the native empty-string default.
|
||||
- **summary:** (cell) - restore the selected global string cell from AGE's shared `SAVE.DAT` profile table using the raw `0x05` plus eight-hex-digit key; a missing key yields the native empty-string default.
|
||||
- **grounding:** source=investigation, confidence=high
|
||||
- **evidence:** Ghidra /v2: op_0x1aa_load_shared_profile_string@0x42bd90 resolves operand 1's lvalue index, calls shared_profile_load_string_by_typed_key@0x419ca0 with type prefix 5, and writes the result back through the VM string lvalue. Corpus: seven calls in GAMESTART, INIT2, INPUTNAME, and SAVE.
|
||||
|
||||
This is the paired reader for opcode 0x1a9 and the string counterpart to integer-load opcode 0x1a3.
|
||||
This is the paired reader for opcode 0x1a9 and the string counterpart to integer-load opcode 0x1a3. Port status (2026-07-24): implemented through the shared profile service and native payload codec.
|
||||
|
||||
### 0x1ab `delete-numbered-save` (delete-numbered-save, argc 2)
|
||||
- **summary:** (status_out)(slot) - attempt to delete both `SAVE%02d.DAT` and its `SAVE%02d.STH` thumbnail.
|
||||
|
||||
@@ -3381,6 +3381,30 @@ integer/string cell maps, then connect `0x1a2`/`0x1a3` and `0x1a9`/`0x1aa`. Add
|
||||
read-message lifecycle after that shared ownership is live; numbered active-frame layouts remain the later,
|
||||
larger payload.
|
||||
|
||||
### Persistence implementation step 2 — shared SAVE.DAT payload and selected cells (2026-07-24)
|
||||
|
||||
The typed shared payload and profile ownership are now implemented. `SharedProfilePayloadCodec` reads and
|
||||
writes the catalog block, fixed integer entries, DWORD-counted CP932 string blob, version-3.10 selector and
|
||||
extra arrays, and native reserved tail. `SharedProfile` preserves the catalog/extended sections opaquely,
|
||||
owns selected raw-int32 and string maps, and explicitly loads/saves through `INativeDatStore`.
|
||||
`GameSession` shares that profile across fresh scene VMs without adding it to the existing whole-bank JSON
|
||||
diagnostic snapshot.
|
||||
|
||||
The original AppData `SAVE.DAT` served as a read-only compatibility oracle and decodes completely:
|
||||
13,210 catalog values, 24,070 integer cells, 605 string cells, selector 1 = 81, 83 extended values, and
|
||||
nine reserved tail DWORDs. That check corrected three reconnaissance details before implementation: typed
|
||||
keys begin with raw bytes `0x03`/`0x05` rather than ASCII digits; the string length is a DWORD count; and
|
||||
the native writer's explicit terminator plus historical allocation slack leaves nine tail DWORDs.
|
||||
|
||||
VM opcodes `0x1a2`/`0x1a3` and `0x1a9`/`0x1aa` now implement insert-or-assign plus native zero/empty
|
||||
miss defaults. Direct global cells and the shipped local-pointer-to-global idiom are covered, including
|
||||
fresh-VM continuity through `GameSession`. Five focused shared-profile tests plus the seven container/store
|
||||
tests cover binary shape, malformed input, native lifecycle, pointer lvalues, miss behavior, and JSON-domain
|
||||
separation.
|
||||
|
||||
**Next persistence step:** add native `RT.DAT` and connect the already-mapped ReadTextDB queue/commit/query
|
||||
lifecycle. Numbered active-frame saves and `.STH` remain the later, larger payload slice.
|
||||
|
||||
## Data-semantics sidebar: focused append EBINIT inspection (2026-07-24)
|
||||
|
||||
The static INIT surface now accepts a universal packed script id for focused append inspection.
|
||||
|
||||
@@ -466,8 +466,10 @@ domains and lifecycle—shared `SAVE.DAT`/`SAVE.BAK`, `RT.DAT`/`RT.BAK`, numbere
|
||||
paired BMP `.STH` thumbnails. Keep the codec behind a profile/save-service boundary. Human-readable
|
||||
JSON inspection/export, migrations, and namespaced mod state are additive extended-mode work, not a
|
||||
replacement for compatibility-mode import/export. The recovered native contract lives in
|
||||
`docs/engine-re.md`. The common container codec and payload-agnostic shared/numbered DAT store boundary
|
||||
landed on 2026-07-24; logical payload services and opcode wiring remain Phase B work.
|
||||
`docs/engine-re.md`. The common container codec/store and typed shared `SAVE.DAT` payload landed on
|
||||
2026-07-24. Profile-owned selected integer/string cells now survive across scene VMs and are wired to
|
||||
their four native opcodes while opaque catalog/version sections round-trip unchanged. `RT.DAT` is the
|
||||
next profile domain; numbered active-frame state and thumbnails remain later Phase B work.
|
||||
|
||||
### Phase C — Externalize & modding foundation
|
||||
- Add **editable named data overlays** mapped explicitly onto the VM's `*INIT`-produced state; external
|
||||
|
||||
@@ -26,6 +26,11 @@ The Himegari trailer contains `SCREENX=800` and `SCREENY=600`; Kamidori's indepe
|
||||
initialization and the executable's `640x480` fallback are documented in `engine-re.md`;
|
||||
fullscreen/display-mode settings are a separate concern from these logical dimensions.
|
||||
|
||||
The same Himegari trailer also contains `USEAPPDATAFOLDER=1` and
|
||||
`SAVEPATH=Eushully\姫狩りダンジョンマイスター\SAVE`. AGE's native save-root resolver consumes those
|
||||
settings to select `%LOCALAPPDATA%` plus that relative path on modern Windows. Filename patterns remain
|
||||
native policy rather than trailer or script strings; see `engine-re.md` under "Save-root resolution."
|
||||
|
||||
## Header — CONFIRMED
|
||||
|
||||
Fixed 60-byte (0x3C) header: 8-byte magic + thirteen little-endian u32 fields.
|
||||
|
||||
Reference in New Issue
Block a user