diff --git a/docs/engine-re.md b/docs/engine-re.md index 009f1de..b5a915b 100644 --- a/docs/engine-re.md +++ b/docs/engine-re.md @@ -1720,9 +1720,29 @@ generic save operation: | `0x1ae(status,slot,surface)` | write `SAVE%02d.STH` from a surface | | `0x1af(status,slot,surface)` | load `SAVE%02d.STH` into a surface | -Opcode `0x19d`, adjacent in number and used by CGMODE/ED/HMODE/MMODE, is not persistence: its handler is a -resource/compatibility lookup. It is deliberately excluded rather than named from proximity. The actual -persistence cluster has calls in `SAVE`, `SELSTAGE`, `GAMESTART`, `GAMECLEAR`, `INPUTNAME`, and INIT2. +Opcode `0x19d`, adjacent in number and used by CGMODE/ED/HMODE/MMODE, queries the shared-profile catalog +unlock database: + +- `op_0x19d_is_catalog_resource_unlocked@0x427810` fetches `(out, packed_resource_id)`. Packed append + ids return zero when the configured save version is older than 3.10; otherwise it calls + `asset_catalog_is_resource_unlocked@0x415920` on `EngineCtx+0x9c24c`. +- Base ids use the marker table at asset-catalog `+0x41c`. A nonzero high byte selects the append + pointer at `+0x3844 + selector*4`; the low 24 bits are the table index. +- The marker is valid exactly when its low word equals + `low16(index * 0x053d6f99 + 0xb0b0b0b0)`. `asset_catalog_mark_resource_opened@0x44e410`, called only + after `asset_open_indexed_entry` succeeds, writes the plain marker and its modular-exponent encrypted + SAVE.DAT counterpart. +- `shared_profile_load` decrypts base and flattened append marker arrays through + `modular_exponent_u32@0x471e60`, then + `asset_catalog_restore_resource_unlock_marker@0x404ca0` installs and re-encrypts them under the + current session key. The two leading table DWORDs are private-exponent XOR `0x87912345` and modulus. + +CGMODE passes each CGINIT full-image AGF id; HMODE passes each SPINIT scene-script id. The copied port +profile validates all 851 gallery images and all 118 H-scene scripts. The former all-locked presentation +was therefore an opcode/runtime integration defect, not missing save data. The port now decodes and +queries the native markers, marks successful VFS opens, and writes a native-decodable encrypted table. +The actual numbered/selected-cell persistence cluster remains the adjacent `0x19e` through `0x1af` +family described above. **Numbered operation status contracts.** Save/load open failure is `1`; metadata uses `0=valid`, `1=absent/open failure`, and `2=invalid/incompatible`. Delete/copy attempt both members of the pair and use diff --git a/docs/opcode-reference.md b/docs/opcode-reference.md index 46e49df..76f3b5c 100644 --- a/docs/opcode-reference.md +++ b/docs/opcode-reference.md @@ -323,6 +323,13 @@ Implemented as FIFO TryDequeue: nonempty writes success=1 plus the signed dword; This is the logical inverse of op 0x194 string-equals. Skipping it is stateful: the destination is not cleared. At BUNKI@0x905, local 0x99 still contains a nonzero graphics handle, so the missing write falsely reserves a 30-pixel title row and shifts every choice down. The C# VM implements ordinal inequality through the shared string resolver; focused tests cover literal/global/local/pointer operands and the exact empty-title stale-handle overwrite. +### 0x19d `is-catalog-resource-unlocked` (is-catalog-resource-unlocked, argc 2) +- **summary:** (out)(packed_resource_id) - return 1 when the shared SAVE.DAT catalog marker for the selected base or append resource decodes to AGE's deterministic per-index unlock stamp, otherwise 0. +- **grounding:** source=investigation, confidence=high +- **evidence:** Ghidra /v2: op_0x19d_handler@0x427810 fetches operand 2, rejects packed append ids when `set:SaveVersion1/2` is older than 3.10, calls asset_catalog_is_resource_unlocked@0x415920 on ctx+0x9c24c, and writes the bool to operand 1. The helper selects base table +0x41c or append pointer table +0x3844 and compares the stored low word to `(short)index * 0x6f99 - 0x4f50`. asset_catalog_mark_resource_opened@0x44e410 writes the equivalent full stamp and encrypted profile marker after asset_open_indexed_entry succeeds. shared_profile_load decodes the catalog arrays with modular exponentiation before populating those tables. Corpus: four calls, in CGMODE, HMODE, MMODE, and ED. The installed port SAVE.DAT decodes to 7,966/13,208 base markers and 75/81 append markers; all 851 CGINIT gallery images and all 118 SPINIT H-scene scripts validate unlocked. + +Base ids use the low 24-bit index in the primary catalog marker table. A nonzero high-byte selector addresses that append catalog's independent table; selectors require SaveVersion 3.10 or later, and AGE's signed selector lookup only supports 1..127. Successful native catalog opens write the stamp `low16(index * 0x053d6f99 + 0xb0b0b0b0)` and its encrypted SAVE.DAT form. CGMODE tests CGINIT image ids and HMODE tests SPINIT scene-script ids through this predicate. Port status (2026-07-28): shared-profile import decodes both catalog sections, successful VFS opens mark resources, shared SAVE.DAT writes native-decodable encrypted markers, and the opcode returns the live profile predicate. + ### 0x1a6 `half-byte-string-length` (halve-strlen, argc 2) - **summary:** Write half the resolved string's byte length, using integer truncation. - **grounding:** source=investigation, confidence=high @@ -1332,10 +1339,6 @@ Port status (2026-07-24): implemented through the same profile-lifetime setting - **summary:** — - **grounding:** source=kelebek, confidence=med -### 0x19d `u0041C680` (u0041C680, argc 2) -- **summary:** — -- **grounding:** source=kelebek, confidence=low - ### 0x1a5 `set-font` (set-font, argc 1) - **summary:** — - **grounding:** source=kelebek, confidence=med diff --git a/docs/phase-b-framework.md b/docs/phase-b-framework.md index cd9ee66..33326bd 100644 --- a/docs/phase-b-framework.md +++ b/docs/phase-b-framework.md @@ -272,6 +272,22 @@ voice ids were reaching `play-voice` but failing the port's SC-section-only look section. The initial compatibility fix added a raw fallback, but later native RE proved the broader rule: voice and frontend texture operands are already universal packed catalog ids and never receive a scene base. +**Extra Room CG/H-scene unlock database (`CGMODE.BIN` / `HMODE.BIN`, 2026-07-28).** The copied +`user://SAVE/SAVE.DAT` was intact: native decode finds 7,966/13,208 valid base-catalog markers and +75/81 append markers, including all 851 CGINIT gallery-image ids and all 118 SPINIT H-scene script ids. +The encyclopedia and New Game settings looked correct because they restore selected integer cells through +`0x1a3`; CGMODE/HMODE instead filter their entries through opcode `0x19d`, which the port still stubbed and +therefore left as zero/locked. + +Native RE identifies `0x19d` as the profile catalog-resource unlock predicate. Successful catalog opens +write a deterministic per-index stamp into a live marker table and its modular-exponent encrypted +SAVE.DAT table; shared-profile load decrypts base and selector-keyed append arrays before the query runs. +`SharedProfile` now owns that decoded set, the VM implements the query, and a shared VFS wrapper marks only +successful script/texture/audio/movie/cursor opens. Saving emits native-decodable base and append tables, +so imported unlocks display immediately and newly encountered resources persist. Focused regressions cover +exact native ciphertexts, base/append queries, successful-open timing, round-trip import/export, and known +installed CG/H-scene entries. + **Natural Game Start diagnostic gate (2026-07-21; captured).** The opt-in `-StartupDiagnostics` route kept the persistent `SYSTEM4` root and native exit semantics and added no seeds, boot prefix, timing changes, or input automation. A user-driven cold boot traversed the complete initialization family, `LOGO`/`LOGO.AGF`, diff --git a/docs/remake-architecture-and-roadmap.md b/docs/remake-architecture-and-roadmap.md index 63bdd18..0c2e964 100644 --- a/docs/remake-architecture-and-roadmap.md +++ b/docs/remake-architecture-and-roadmap.md @@ -468,7 +468,9 @@ JSON inspection/export, migrations, and namespaced mod state are additive extend replacement for compatibility-mode import/export. The recovered native contract lives in `docs/engine-re.md`. The common container codec/store and typed shared `SAVE.DAT` payload landed on 2026-07-24. Profile-owned selected integer/string cells now survive across scene VMs and are wired to -their four native opcodes while opaque catalog/version sections round-trip unchanged. Native `RT.DAT` +their four native opcodes. The base/append catalog sections are now decoded as encrypted resource-seen +markers: opcode `0x19d` queries them, successful VFS opens add them, and compatibility saves write +native-decodable tables. Native `RT.DAT` import/export and the packed-script/T1 ReadTextDB queue/commit/query lifecycle are also implemented, including `message:ReadTextSkip` ops `0x1ca`/`0x1cb` and state query `0x1cc`. Numbered active-frame state is now implemented in native layout 3: metadata query, paired `.DAT`/`.STH` lifecycle, exact native BMP diff --git a/docs/sys4-format-notes.md b/docs/sys4-format-notes.md index 9897cc9..eca7316 100644 --- a/docs/sys4-format-notes.md +++ b/docs/sys4-format-notes.md @@ -111,11 +111,11 @@ After decoding the common container, the shared payload is sequential: | Order | Field | |---:|---| -| 1 | `catalog_count:u32`, then `catalog_values[catalog_count]:u32` | +| 1 | `catalog_count:u32`, then the encrypted base-catalog unlock table | | 2 | `integer_count:u32`, then `integer_entries[integer_count]`, 16 bytes each | | 3 | `string_count:u32`, `string_blob_dwords:u32`, then the DWORD-padded string blob | -| 4 | for version 3.10 or later, `selector_counts[256]:u32` | -| 5 | for version 3.10 or later, `extended_count:u32`, then `extended_values[extended_count]:u32` | +| 4 | for version 3.10 or later, `selector_counts[256]:u32` for append-catalog unlock tables | +| 5 | for version 3.10 or later, `extended_count:u32`, then the flattened encrypted append tables | | 6 | `reserved_tail[9]:u32` | An integer entry is: @@ -135,14 +135,35 @@ type byte `0x05` plus the same eight-digit address and NUL; values are CP932. Th a DWORD count. AGE rounds with `(unpadded_bytes / 4) + 1`, deliberately adding a whole zero DWORD when the last value already ends on a DWORD boundary. -The nine-DWORD tail consists of one explicit terminator plus eight DWORDs produced by the native -allocation formula. Catalog, selector, extended, and tail sections are structurally bounded but not -fully assigned game-level meanings, so compatibility import/export preserves them rather than treating -them as mod storage. +The catalog sections are AGE's profile-wide **resource-seen/unlock database**. Each encrypted table begins +with two header DWORDs: -The installed Himegari profile is a complete oracle: version 3.10 decodes to 13,210 catalog values, -24,070 integer cells, 605 string cells, selector entry 1 equal to 81, 83 extended values, and nine tail -DWORDs. +| Table word | Meaning | +|---:|---| +| `0` | private exponent XOR `0x87912345` | +| `1` | modular-exponentiation modulus | +| `index + 2` | zero when locked/unseen; otherwise the encrypted resource marker | + +Loading computes `plain = cipher^private_exponent mod modulus`. A marker is available when its low word +equals `low16(index * 0x053d6f99 + 0xb0b0b0b0)`. Native `asset_open_indexed_entry` writes that marker +only after successfully opening the resource, and opcode `0x19d` queries it. Thus ordinary CG assets +unlock when their AGF is opened and H-scene entries unlock when their SP script is opened. + +The base table maps universal resource ids whose high byte is zero. For version 3.10 or later, +`selector_counts[selector]` gives the slot count of each append catalog; the extended array has its own +two-word crypto header followed by those selector tables concatenated in selector order. Packed resource +ids use the high byte as selector and the low 24 bits as the per-catalog index. Runtime lookup accepts +selectors 1 through 127. + +The nine-DWORD tail consists of one explicit terminator plus eight DWORDs produced by the native +allocation formula. It remains compatibility-owned rather than mod storage. + +The installed Himegari profile is a complete oracle: version 3.10 decodes to 13,210 base-table DWORDs +(13,208 resource slots), 24,070 integer cells, 605 string cells, selector entry 1 equal to 81, 83 +extended-table DWORDs (two header plus 81 resource slots), and nine tail DWORDs. The port copy contains +7,966 valid base markers and 75 valid append markers. Those include all 851 CGINIT gallery images and +all 118 SPINIT H-scene scripts; a different encryption key can change every nonzero stored DWORD without +changing the decoded unlock set. The integer/string maps are AGE's generic selected-cell service, not copies of whole global banks. Himegari scripts assign their meanings through opcodes `0x1a2`/`0x1a3` and `0x1a9`/`0x1aa`. Known @@ -345,7 +366,7 @@ The remaining uncertainty is deliberately narrower: - numbered layouts 1 and 2 are structurally identified but are not the implemented or installed Himegari compatibility target; -- the shared catalog/extended arrays and some selected cells are still opaque at the game-semantic level; +- some shared selected cells and the nine-DWORD tail remain opaque at the game-semantic level; - the `0x2d4` retained-graphics record is byte-bounded, but not every field is named. This distinction is important: AGE owns most numbered-save and ReadTextDB structure, while Himegari diff --git a/engine/Age.Engine.Tests/SharedProfileTests.cs b/engine/Age.Engine.Tests/SharedProfileTests.cs index 4775485..1ff83f3 100644 --- a/engine/Age.Engine.Tests/SharedProfileTests.cs +++ b/engine/Age.Engine.Tests/SharedProfileTests.cs @@ -190,4 +190,153 @@ public class SharedProfileTests Assert.Equal(42, jsonClone.Globals[0x900]); Assert.Equal(0, jsonClone.SharedProfile.LoadInteger(0x900)); } + + [Fact] + public void CatalogUnlockMarkersRoundTripBaseAndAppendResources() + { + var profile = new SharedProfile(); + profile.ConfigureCatalogUnlockSlots( + 6000, + new Dictionary { [1] = 32 }); + profile.MarkCatalogResourceOpened(5407); + profile.MarkCatalogResourceOpened(0x01000011); + + SharedProfilePayload encoded = profile.Snapshot(); + + Assert.Equal(6002, encoded.CatalogCompatibilityValues.Count); + Assert.Equal(0x8791233au, encoded.CatalogCompatibilityValues[0]); + Assert.Equal(0x6868a8e1u, encoded.CatalogCompatibilityValues[5407 + 2]); + Assert.Equal(32u, encoded.ExtendedSelectorCounts[1]); + Assert.Equal(34, encoded.ExtendedValues.Count); + Assert.Equal(0x42d5ee4eu, encoded.ExtendedValues[17 + 2]); + + var reloaded = new SharedProfile(); + reloaded.Replace(encoded); + + Assert.True(reloaded.IsCatalogResourceUnlocked(5407)); + Assert.True(reloaded.IsCatalogResourceUnlocked(0x01000011)); + Assert.False(reloaded.IsCatalogResourceUnlocked(5406)); + Assert.False(reloaded.IsCatalogResourceUnlocked(0x01000012)); + Assert.False(reloaded.IsCatalogResourceUnlocked(0x80000000)); + } + + [Fact] + public void CatalogUnlockOpcodeReturnsImportedProfilePredicate() + { + OpcodeTable table = OpcodeTableJson.Load(Paths.OpcodesJson); + Script script = ScriptAssembler.Assemble(table, "CATALOG_UNLOCK_QUERY", new List<(int, Operand[])> + { + (0x19d, [new Operand(GlobalInt, 0x900), new Operand(Immediate, 5407)]), + (0x19d, [new Operand(GlobalInt, 0x901), new Operand(Immediate, 5406)]), + (0x19d, [new Operand(GlobalInt, 0x902), new Operand(Immediate, 0x01000011)]), + (0x2, []), + }, []); + var profile = new SharedProfile(); + profile.ConfigureCatalogUnlockSlots(6000, new Dictionary { [1] = 32 }); + profile.MarkCatalogResourceOpened(5407); + profile.MarkCatalogResourceOpened(0x01000011); + var vm = new VirtualMachine( + script, table, new RecordingHost(), sharedProfile: profile); + + vm.Run(); + + Assert.Equal("exit", vm.HaltReason); + Assert.Equal(1, vm.Globals[0x900]); + Assert.Equal(0, vm.Globals[0x901]); + Assert.Equal(1, vm.Globals[0x902]); + } + + [Fact] + public void TrackingStoreMarksOnlySuccessfulCatalogOpens() + { + var profile = new SharedProfile(); + profile.ConfigureCatalogUnlockSlots(8); + var tracker = new CatalogTrackingAssetStore( + new SelectiveAssetStore(), + entry => profile.MarkCatalogResourceOpened(entry.PackedId)); + var available = new AssetEntry("A.BIN", "DATA1.ALF", 0, 1, RawIndex: 3); + var missing = new AssetEntry("MISSING.BIN", "DATA1.ALF", 0, 1, RawIndex: 4); + + Assert.Equal(new byte[] { 42 }, tracker.ReadAll(available)); + Assert.Throws(() => tracker.ReadAll(missing)); + + Assert.True(profile.IsCatalogResourceUnlocked(3)); + Assert.False(profile.IsCatalogResourceUnlocked(4)); + } + + [Fact] + public void InstalledSharedProfileUnlocksKnownCgAndHSceneResourcesWhenPresent() + { + string root = Path.Combine( + Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), + "Eushully", "姫狩りダンジョンマイスター", "SAVE"); + if (!File.Exists(Path.Combine(root, "SAVE.DAT"))) return; + + var store = new DirectoryNativeDatStore( + root, + new NativeSaveIdentity( + NativeSaveMagic.S4SD, 0x4a343234, "姫狩りダンジョンマイスター", 3, 10)); + var profile = new SharedProfile(); + + Assert.True(profile.Load(store)); + Assert.True(profile.IsCatalogResourceUnlocked(0x3324)); // ED.AGF + + using var cgDocument = System.Text.Json.JsonDocument.Parse( + File.ReadAllBytes(Path.Combine(Paths.Build, "data", "CGINIT.json"))); + uint[] cgIds = cgDocument.RootElement.GetProperty("records").EnumerateArray() + .Select(record => record.GetProperty("gallery_image_asset_id").GetUInt32()) + .ToArray(); + Assert.Equal(851, cgIds.Length); + Assert.All(cgIds, id => Assert.True( + profile.IsCatalogResourceUnlocked(id), $"CG resource 0x{id:x} should be unlocked")); + + using var hDocument = System.Text.Json.JsonDocument.Parse( + File.ReadAllBytes(Path.Combine(Paths.Build, "data", "SPINIT.json"))); + uint[] hSceneIds = hDocument.RootElement.GetProperty("records").EnumerateArray() + .SelectMany(page => page.GetProperty("script_resource_ids").EnumerateArray()) + .Select(id => id.GetUInt32()) + .Where(id => id != 0) + .ToArray(); + Assert.Equal(118, hSceneIds.Length); + Assert.All(hSceneIds, id => Assert.True( + profile.IsCatalogResourceUnlocked(id), $"H-scene resource 0x{id:x} should be unlocked")); + + int newlyOpenedId = Enumerable.Range(0, 13_208) + .First(id => !profile.IsCatalogResourceUnlocked(id)); + profile.MarkCatalogResourceOpened(newlyOpenedId); + string roundTripRoot = Path.Combine( + Path.GetTempPath(), "age-installed-profile-unlocks-" + Guid.NewGuid().ToString("N")); + try + { + var roundTripStore = new DirectoryNativeDatStore(roundTripRoot, store.Identity); + profile.Save(roundTripStore, Timestamp, profile.AccumulatedPlaySeconds); + var reloaded = new SharedProfile(); + + Assert.True(reloaded.Load(roundTripStore)); + Assert.True(reloaded.IsCatalogResourceUnlocked(newlyOpenedId)); + Assert.All(cgIds, id => Assert.True(reloaded.IsCatalogResourceUnlocked(id))); + Assert.All(hSceneIds, id => Assert.True(reloaded.IsCatalogResourceUnlocked(id))); + } + finally + { + if (Directory.Exists(roundTripRoot)) Directory.Delete(roundTripRoot, recursive: true); + } + } + + private sealed class SelectiveAssetStore : IAssetStore + { + public Stream Open(AssetEntry entry) + { + if (entry.Name == "MISSING.BIN") throw new FileNotFoundException(); + return new MemoryStream([42], writable: false); + } + + public byte[] ReadAll(AssetEntry entry) + { + using Stream stream = Open(entry); + var result = new byte[stream.Length]; + stream.ReadExactly(result); + return result; + } + } } diff --git a/engine/Age.Engine/Persistence/SharedProfile.cs b/engine/Age.Engine/Persistence/SharedProfile.cs index 0b1c182..efd22f2 100644 --- a/engine/Age.Engine/Persistence/SharedProfile.cs +++ b/engine/Age.Engine/Persistence/SharedProfile.cs @@ -6,9 +6,9 @@ using System.Text; namespace Age.Engine.Persistence; /// -/// Typed logical contents of AGE's shared SAVE.DAT payload. The catalog and extended arrays are -/// intentionally opaque: the selected-cell service owns only the integer and string maps, while -/// native import/export must preserve the other engine-owned sections losslessly. +/// Typed logical contents of AGE's shared SAVE.DAT payload. CatalogCompatibilityValues retains the +/// historical codec-facing name but carries AGE's encrypted base resource-unlock table; the selector +/// and extended arrays carry the parallel append-catalog tables. /// public sealed class SharedProfilePayload { @@ -337,17 +337,28 @@ public static class SharedProfilePayloadCodec } /// -/// Profile-lifetime selected cells plus the opaque SAVE.DAT sections and independent RT.DAT read -/// history. VM opcodes mutate this object; explicit Load/Save calls own both filesystem lifecycles. +/// Profile-lifetime selected cells, catalog resource-unlock markers, and independent RT.DAT read +/// history. VM opcodes and successful catalog opens mutate this object; explicit Load/Save calls own +/// both filesystem lifecycles. /// public sealed class SharedProfile { + private const uint CatalogPrivateExponentMask = 0x87912345; + private const uint CanonicalCatalogPrivateExponent = 127; + private const uint CanonicalCatalogPublicExponent = 14_478_031; + private const uint CanonicalCatalogModulus = 1_838_797_217; + private uint[] _catalogCompatibilityValues = Array.Empty(); private readonly Dictionary _integerCells = new(); private readonly Dictionary _stringCells = new(); private uint[] _extendedSelectorCounts = Array.Empty(); private uint[] _extendedValues = Array.Empty(); private uint[] _reservedTail = new uint[SharedProfilePayloadCodec.ReservedTailDwordCount]; + private readonly object _catalogUnlockLock = new(); + private readonly HashSet _unlockedCatalogResources = new(); + private int _baseCatalogSlotCount; + private readonly int[] _appendCatalogSlotCounts = new int[SharedProfilePayloadCodec.ExtendedSelectorCount]; + private bool _catalogUnlocksDirty; private uint _accumulatedPlaySeconds; public IReadOnlyDictionary IntegerCells => _integerCells; @@ -357,6 +368,71 @@ public sealed class SharedProfile /// The engine setting manipulated by opcodes 0x1ca/0x1cb. public bool ReadMessageSkipEnabled { get; set; } + /// + /// Set the mounted catalog geometry used when newly opened resources must be written back to + /// SAVE.DAT. Existing imported unlocks survive when catalogs grow; absent new slots begin locked. + /// + public void ConfigureCatalogUnlockSlots( + int baseSlotCount, + IEnumerable>? appendSlotCounts = null) + { + if (baseSlotCount < 0) + throw new ArgumentOutOfRangeException(nameof(baseSlotCount)); + + var configuredAppendCounts = new int[SharedProfilePayloadCodec.ExtendedSelectorCount]; + if (appendSlotCounts != null) + { + foreach ((int selector, int count) in appendSlotCounts) + { + if (selector is <= 0 or >= 0x80) + throw new ArgumentOutOfRangeException( + nameof(appendSlotCounts), "Append catalog selectors must be in 1..127."); + if (count < 0 || count > 0x1000000) + throw new ArgumentOutOfRangeException( + nameof(appendSlotCounts), "Append catalog slot counts must fit the packed 24-bit index."); + configuredAppendCounts[selector] = count; + } + } + + lock (_catalogUnlockLock) + { + bool changed = _baseCatalogSlotCount != baseSlotCount + || !_appendCatalogSlotCounts.SequenceEqual(configuredAppendCounts); + _baseCatalogSlotCount = baseSlotCount; + configuredAppendCounts.CopyTo(_appendCatalogSlotCounts, 0); + _catalogUnlocksDirty |= changed; + } + } + + /// AGE marks a resource unlocked only after its catalog entry opens successfully. + public void MarkCatalogResourceOpened(long packedId) + { + if (packedId < 0 || packedId > uint.MaxValue) return; + uint id = unchecked((uint)packedId); + int selector = (int)(id >> 24); + int index = (int)(id & 0x00ff_ffff); + if (selector >= 0x80) return; + + lock (_catalogUnlockLock) + { + if (selector == 0) + _baseCatalogSlotCount = Math.Max(_baseCatalogSlotCount, checked(index + 1)); + else + _appendCatalogSlotCounts[selector] = + Math.Max(_appendCatalogSlotCounts[selector], checked(index + 1)); + if (_unlockedCatalogResources.Add(id)) _catalogUnlocksDirty = true; + } + } + + /// Native opcode 0x19d's profile-wide resource-seen predicate. + public bool IsCatalogResourceUnlocked(long packedId) + { + if (packedId < 0 || packedId > uint.MaxValue) return false; + uint id = unchecked((uint)packedId); + if ((id >> 24) >= 0x80) return false; + lock (_catalogUnlockLock) return _unlockedCatalogResources.Contains(id); + } + public void StoreInteger(int address, long value) { ValidateAddress(address); @@ -408,31 +484,60 @@ public sealed class SharedProfile { ArgumentNullException.ThrowIfNull(store); NativeSaveMetadata metadata = store.Identity.CreateMetadata(timestamp, accumulatedPlaySeconds); - byte[] payload = SharedProfilePayloadCodec.Encode(Snapshot(), metadata); + SharedProfilePayload snapshot = Snapshot(); + byte[] payload = SharedProfilePayloadCodec.Encode(snapshot, metadata); store.SaveShared(payload, timestamp, accumulatedPlaySeconds); + lock (_catalogUnlockLock) + { + _catalogCompatibilityValues = snapshot.CatalogCompatibilityValues.ToArray(); + _extendedSelectorCounts = snapshot.ExtendedSelectorCounts.ToArray(); + _extendedValues = snapshot.ExtendedValues.ToArray(); + _catalogUnlocksDirty = false; + } _accumulatedPlaySeconds = accumulatedPlaySeconds; ReadText.Save(store); } public SharedProfilePayload Snapshot() - => new( - _catalogCompatibilityValues, + { + uint[] catalog; + uint[] selectors; + uint[] extended; + lock (_catalogUnlockLock) + { + if (_catalogUnlocksDirty) + (catalog, selectors, extended) = EncodeCatalogUnlockSections(); + else + { + catalog = _catalogCompatibilityValues.ToArray(); + selectors = _extendedSelectorCounts.ToArray(); + extended = _extendedValues.ToArray(); + } + } + return new SharedProfilePayload( + catalog, _integerCells, _stringCells, - _extendedSelectorCounts, - _extendedValues, + selectors, + extended, _reservedTail); + } public void Replace(SharedProfilePayload payload) { ArgumentNullException.ThrowIfNull(payload); - _catalogCompatibilityValues = payload.CatalogCompatibilityValues.ToArray(); + lock (_catalogUnlockLock) + { + _catalogCompatibilityValues = payload.CatalogCompatibilityValues.ToArray(); + _extendedSelectorCounts = payload.ExtendedSelectorCounts.ToArray(); + _extendedValues = payload.ExtendedValues.ToArray(); + DecodeCatalogUnlockSections(); + _catalogUnlocksDirty = false; + } _integerCells.Clear(); foreach (var entry in payload.IntegerCells) _integerCells.Add(entry.Key, entry.Value); _stringCells.Clear(); foreach (var entry in payload.StringCells) _stringCells.Add(entry.Key, entry.Value); - _extendedSelectorCounts = payload.ExtendedSelectorCounts.ToArray(); - _extendedValues = payload.ExtendedValues.ToArray(); _reservedTail = payload.ReservedTail.ToArray(); } @@ -446,14 +551,130 @@ public sealed class SharedProfile private void ClearSharedPayload() { - _catalogCompatibilityValues = Array.Empty(); + lock (_catalogUnlockLock) + { + _catalogCompatibilityValues = Array.Empty(); + _extendedSelectorCounts = Array.Empty(); + _extendedValues = Array.Empty(); + _unlockedCatalogResources.Clear(); + _baseCatalogSlotCount = 0; + Array.Clear(_appendCatalogSlotCounts); + _catalogUnlocksDirty = false; + } _integerCells.Clear(); _stringCells.Clear(); - _extendedSelectorCounts = Array.Empty(); - _extendedValues = Array.Empty(); _reservedTail = new uint[SharedProfilePayloadCodec.ReservedTailDwordCount]; } + private void DecodeCatalogUnlockSections() + { + _unlockedCatalogResources.Clear(); + _baseCatalogSlotCount = Math.Max(0, _catalogCompatibilityValues.Length - 2); + Array.Clear(_appendCatalogSlotCounts); + + DecodeCatalogTable(_catalogCompatibilityValues, _baseCatalogSlotCount, 0, 0); + + int encodedOffset = 0; + int availableExtendedSlots = Math.Max(0, _extendedValues.Length - 2); + for (int selector = 0; selector < _extendedSelectorCounts.Length + && selector < _appendCatalogSlotCounts.Length; selector++) + { + uint rawCount = _extendedSelectorCounts[selector]; + int count = rawCount > int.MaxValue ? 0 : unchecked((int)rawCount); + _appendCatalogSlotCounts[selector] = count; + int readable = Math.Min(count, Math.Max(0, availableExtendedSlots - encodedOffset)); + DecodeCatalogTable(_extendedValues, readable, encodedOffset, selector); + encodedOffset = checked(encodedOffset + readable); + if (readable != count) break; + } + } + + private void DecodeCatalogTable( + IReadOnlyList encoded, + int count, + int encodedOffset, + int selector) + { + if (encoded.Count < 2 || encoded[1] == 0) return; + uint exponent = encoded[0] ^ CatalogPrivateExponentMask; + uint modulus = encoded[1]; + for (int index = 0; index < count && encodedOffset + index + 2 < encoded.Count; index++) + { + uint cipher = encoded[encodedOffset + index + 2]; + if (cipher == 0) continue; + uint plain = ModularPow(cipher, exponent, modulus); + if (unchecked((ushort)plain) != CatalogUnlockStamp(index)) continue; + _unlockedCatalogResources.Add(unchecked(((uint)selector << 24) | (uint)index)); + } + } + + private (uint[] Catalog, uint[] Selectors, uint[] Extended) EncodeCatalogUnlockSections() + { + uint[] catalog = NewEncodedCatalogTable(_baseCatalogSlotCount); + uint[] selectors = new uint[SharedProfilePayloadCodec.ExtendedSelectorCount]; + int extendedSlotCount = 0; + for (int selector = 0; selector < selectors.Length; selector++) + { + int count = _appendCatalogSlotCounts[selector]; + selectors[selector] = checked((uint)count); + extendedSlotCount = checked(extendedSlotCount + count); + } + uint[] extended = NewEncodedCatalogTable(extendedSlotCount); + + int[] selectorOffsets = new int[selectors.Length]; + int offset = 0; + for (int selector = 0; selector < selectors.Length; selector++) + { + selectorOffsets[selector] = offset; + offset = checked(offset + _appendCatalogSlotCounts[selector]); + } + + foreach (uint id in _unlockedCatalogResources) + { + int selector = (int)(id >> 24); + int index = (int)(id & 0x00ff_ffff); + uint cipher = ModularPow( + CatalogUnlockStamp(index), + CanonicalCatalogPublicExponent, + CanonicalCatalogModulus); + if (selector == 0) + { + if (index < _baseCatalogSlotCount) catalog[index + 2] = cipher; + } + else if ((uint)selector < (uint)_appendCatalogSlotCounts.Length + && index < _appendCatalogSlotCounts[selector]) + { + extended[selectorOffsets[selector] + index + 2] = cipher; + } + } + return (catalog, selectors, extended); + } + + private static uint[] NewEncodedCatalogTable(int slotCount) + { + var result = new uint[checked(slotCount + 2)]; + result[0] = CanonicalCatalogPrivateExponent ^ CatalogPrivateExponentMask; + result[1] = CanonicalCatalogModulus; + return result; + } + + private static ushort CatalogUnlockStamp(int index) + => unchecked((ushort)((uint)index * 0x053d6f99u + 0xb0b0b0b0u)); + + private static uint ModularPow(uint value, uint exponent, uint modulus) + { + if (modulus == 0) return 0; + ulong result = 1; + ulong factor = value % modulus; + for (int bit = 0; bit < 32; bit++) + { + if ((exponent & (1u << bit)) != 0) + result = result * factor % modulus; + factor = factor * factor % modulus; + } + return unchecked((uint)result); + } + private static void ValidateAddress(int address) { if (address < 0) diff --git a/engine/Age.Engine/Sys4/Sys4AssetStore.cs b/engine/Age.Engine/Sys4/Sys4AssetStore.cs index 980517b..95f7254 100644 --- a/engine/Age.Engine/Sys4/Sys4AssetStore.cs +++ b/engine/Age.Engine/Sys4/Sys4AssetStore.cs @@ -7,6 +7,36 @@ public interface IAssetStore byte[] ReadAll(AssetEntry entry); } +/// Reports only successful catalog opens, matching AGE's profile unlock marker timing. +public sealed class CatalogTrackingAssetStore : IAssetStore +{ + private readonly IAssetStore _inner; + private readonly Action _opened; + + public CatalogTrackingAssetStore(IAssetStore inner, Action opened) + { + _inner = inner ?? throw new ArgumentNullException(nameof(inner)); + _opened = opened ?? throw new ArgumentNullException(nameof(opened)); + } + + public Stream Open(AssetEntry entry) + { + Stream stream = _inner.Open(entry); + _opened(entry); + return stream; + } + + public byte[] ReadAll(AssetEntry entry) + { + using Stream stream = Open(entry); + if (stream.Length > int.MaxValue) + throw new InvalidDataException($"{entry.Name}: payload is too large"); + var bytes = new byte[checked((int)stream.Length)]; + stream.ReadExactly(bytes); + return bytes; + } +} + /// Native base-game precedence: exact-basename loose roots first, indexed ALF range second. public sealed class Sys4AssetStore : IAssetStore { diff --git a/engine/Age.Engine/Vm/VirtualMachine.cs b/engine/Age.Engine/Vm/VirtualMachine.cs index 1d57a26..8e2519a 100644 --- a/engine/Age.Engine/Vm/VirtualMachine.cs +++ b/engine/Age.Engine/Vm/VirtualMachine.cs @@ -1191,6 +1191,9 @@ public sealed class VirtualMachine case "halve-strlen": // 0x1a6: strlen(native encoded bytes) >> 1 Write(a[0], NativeStringByteLength(ReadStr(a[1])) >> 1); return pc + 1; + case "is-catalog-resource-unlocked": // 0x19d + Write(a[0], _sharedProfile.IsCatalogResourceUnlocked(Read(a[1])) ? 1 : 0); + return pc + 1; case "save-numbered-slot": // 0x19e { if (_nativeDatStore == null) diff --git a/godot/Main.cs b/godot/Main.cs index c1bbe87..27e011e 100644 --- a/godot/Main.cs +++ b/godot/Main.cs @@ -231,13 +231,38 @@ public partial class Main : Godot.Control GD.Print($"[renderer] retained backend={(_useGpuBackend ? "gpu" : "software")}"); var table = OpcodeTableJson.Load(Paths.OpcodesJson); + // Persistence opcodes retain AGE's native filenames and binary formats, but the port owns the + // root interception point. Keep authored saves isolated from the original installation under + // Godot's per-application user directory. + var nativeSaveStore = new DirectoryNativeDatStore( + ProjectSettings.GlobalizePath("user://SAVE"), + new NativeSaveIdentity( + NativeSaveMagic.S4SD, 0x4a343234, "姫狩りダンジョンマイスター", + SaveVersion1: 3, SaveVersion2: 10, NumberedCompatibilityId: 0x42323234)); + var sharedProfile = new SharedProfile(); + if (!_selftest) sharedProfile.Load(nativeSaveStore); + // Full op handling everywhere: the provider lets call-script load & run subroutines. Selftest // runs a SYNTHESIZED scene (not a real scene in a crippled mode) so its output is deterministic. Script script; IScriptProvider provider; Sys4ScriptProvider? scripts = null; + IAssetStore? trackedAssetStore = null; if (_selftest) (script, provider) = BuildSelfTestScene(table); - else { scripts = Sys4ScriptProvider.Load(table); script = scripts.RequireByName(scene + ".BIN"); provider = scripts; } + else + { + var catalog = Sys4AssetCatalog.Load(Paths.Sys4Ini); + sharedProfile.ConfigureCatalogUnlockSlots( + catalog.RawSlots.Count, + catalog.AppendPacks.Select(pair => + new KeyValuePair(pair.Key, pair.Value.RawSlots.Count))); + trackedAssetStore = new CatalogTrackingAssetStore( + new Sys4AssetStore(catalog, Paths.GameDir, Paths.GameDir), + entry => sharedProfile.MarkCatalogResourceOpened(entry.PackedId)); + scripts = new Sys4ScriptProvider(table, catalog, trackedAssetStore); + script = scripts.RequireByName(scene + ".BIN"); + provider = scripts; + } _scripts = scripts; bool directSceneHarness = !_selftest && !scene.Equals("SYSTEM4", System.StringComparison.OrdinalIgnoreCase); @@ -246,7 +271,9 @@ public partial class Main : Godot.Control pageMapPath = System.IO.Path.Combine(Paths.Build, $"page-map-{scene.ToUpperInvariant()}.jsonl"); _locator = new PageLocatorState(scene, _selftest ? null : pageMapPath); _locatorHud.Visible = _locatorHudVisible; - var resources = scripts != null ? new ResourceMap(scripts.Catalog) : ResourceMap.Load(); + var resources = scripts != null + ? new ResourceMap(scripts.Catalog, trackedAssetStore) + : ResourceMap.Load(); _host = new GodotAdvHost( this, resources, scene, _clock, _locator, _timeline, synchronizeExplicitPresentation: !_selftest) @@ -263,16 +290,6 @@ public partial class Main : Godot.Control Age.Engine.Diagnostics.ITraceSink sink = _trace; if (histFile != null) { _hist = new Age.Engine.Diagnostics.HistogramTraceSink(); sink = new Age.Engine.Diagnostics.CompositeTraceSink(_trace, _hist); } - // Persistence opcodes retain AGE's native filenames and binary formats, but the port owns the - // root interception point. Keep authored saves isolated from the original installation under - // Godot's per-application user directory. - var nativeSaveStore = new DirectoryNativeDatStore( - ProjectSettings.GlobalizePath("user://SAVE"), - new NativeSaveIdentity( - NativeSaveMagic.S4SD, 0x4a343234, "姫狩りダンジョンマイスター", - SaveVersion1: 3, SaveVersion2: 10, NumberedCompatibilityId: 0x42323234)); - var sharedProfile = new SharedProfile(); - if (!_selftest) sharedProfile.Load(nativeSaveStore); _vm = new VirtualMachine(script, table, _host, new VmOptions( MaxSteps: 20_000_000, diff --git a/vm-map/opcodes.toml b/vm-map/opcodes.toml index d42ba09..a6453d7 100644 --- a/vm-map/opcodes.toml +++ b/vm-map/opcodes.toml @@ -3351,28 +3351,29 @@ evidence = "Ghidra /v2: op_0x19c_resume_adv_skip_service@0x4165a0 sets ctx+0x551 [[opcode]] op = 0x19d -label = "u0041C680" +label = "is-catalog-resource-unlocked" argc = 2 abi_source = "kelebek+decode-validated" [opcode.semantics] -name = "u0041C680" -category = "unknown" -summary = "" +name = "is-catalog-resource-unlocked" +category = "compute" +summary = "(out)(packed_resource_id) - return 1 when the shared SAVE.DAT catalog marker for the selected base or append resource decodes to AGE's deterministic per-index unlock stamp, otherwise 0." noop_headless = false -source = "kelebek" -confidence = "low" +source = "investigation" +confidence = "high" depends_on = [] -evidence = "" +details = "Base ids use the low 24-bit index in the primary catalog marker table. A nonzero high-byte selector addresses that append catalog's independent table; selectors require SaveVersion 3.10 or later, and AGE's signed selector lookup only supports 1..127. Successful native catalog opens write the stamp `low16(index * 0x053d6f99 + 0xb0b0b0b0)` and its encrypted SAVE.DAT form. CGMODE tests CGINIT image ids and HMODE tests SPINIT scene-script ids through this predicate. Port status (2026-07-28): shared-profile import decodes both catalog sections, successful VFS opens mark resources, shared SAVE.DAT writes native-decodable encrypted markers, and the opcode returns the live profile predicate." +evidence = "Ghidra /v2: op_0x19d_handler@0x427810 fetches operand 2, rejects packed append ids when `set:SaveVersion1/2` is older than 3.10, calls asset_catalog_is_resource_unlocked@0x415920 on ctx+0x9c24c, and writes the bool to operand 1. The helper selects base table +0x41c or append pointer table +0x3844 and compares the stored low word to `(short)index * 0x6f99 - 0x4f50`. asset_catalog_mark_resource_opened@0x44e410 writes the equivalent full stamp and encrypted profile marker after asset_open_indexed_entry succeeds. shared_profile_load decodes the catalog arrays with modular exponentiation before populating those tables. Corpus: four calls, in CGMODE, HMODE, MMODE, and ED. The installed port SAVE.DAT decodes to 7,966/13,208 base markers and 75/81 append markers; all 851 CGINIT gallery images and all 118 SPINIT H-scene scripts validate unlocked." [[opcode.semantics.args]] i = 1 -role = "" +role = "out" observed_types = ["l-int"] [[opcode.semantics.args]] i = 2 -role = "" +role = "packed_resource_id" observed_types = ["imm", "l-ptr"] [[opcode]]