From ac5437c5cfac55832efafe42c3947ea8692dd106 Mon Sep 17 00:00:00 2001 From: gamer147 Date: Sat, 18 Jul 2026 23:41:23 -0400 Subject: [PATCH] Implement ADV History data navigation --- docs/engine-re.md | 10 +- docs/opcode-reference.md | 14 +- docs/phase-a-slice-plan.md | 24 +++ docs/sys4-format-notes.md | 13 ++ .../Age.Engine.Tests/HistoryDataOpsTests.cs | 164 ++++++++++++++++++ engine/Age.Engine/Model/AdvTextHistory.cs | 97 ++++++++++- engine/Age.Engine/Model/Script.cs | 2 + engine/Age.Engine/Sys4/Sys4Loader.cs | 10 +- engine/Age.Engine/Vm/VirtualMachine.cs | 75 ++++++++ tools/age_opcodes_himegari.py | 2 - vm-map/opcodes.toml | 18 +- 11 files changed, 406 insertions(+), 23 deletions(-) create mode 100644 engine/Age.Engine.Tests/HistoryDataOpsTests.cs diff --git a/docs/engine-re.md b/docs/engine-re.md index 91a4606..6791b87 100644 --- a/docs/engine-re.md +++ b/docs/engine-re.md @@ -1325,8 +1325,14 @@ and `0x85` feed it directly; Godot remains only a presentation host and does not The model intentionally has no JSON or disk serialization. `GameSession.ToJson()` continues to snapshot only the pre-existing global banks, so landing the live backlog does not silently choose a save/profile backend. Native numbered-save restoration remains the explicit future integration seam described above. -The next History slice can now implement generic inline arrays/value dispatch and ops `0x1d0..0x1d4` over -real retained data before adding host rendering. +The second port slice implements generic inline-array copying (`0x64`), formatted value dispatch +(`0xa1/0xa2/0xa3`), cumulative navigation (`0x1d0`), and metadata/voice queries (`0x1d3/0x1d4`). The C# +loader retains the original body dwords so `0x64` copies the file's plain count-prefixed values; AGE's +rotate/XOR work belongs to its native in-memory representation, not the SYS4 file format. Navigation mirrors +manager `+0xd6c`: layout define/reset updates the latest-entry anchor, while each `0x1d0` delta is cumulative +and does not mutate it. Query scans stop at the next group-start record; their third operands are unused by +the native helpers. History's remaining work is therefore presentation and interaction rather than backlog +data access. The original dependency order was **Hide Window first** to establish reusable callback/coroutine input, then Read-message Skip, then History after both the input layer and message-completion seam exist. Hide Window is diff --git a/docs/opcode-reference.md b/docs/opcode-reference.md index 641472c..b725f5f 100644 --- a/docs/opcode-reference.md +++ b/docs/opcode-reference.md @@ -76,9 +76,9 @@ - **evidence:** Ghidra /v2: op_0x1bb_handler@0x41f650 writes 0x80000000 to ctx+0x55110 for operand 0 and zero for operand 1, rejecting other values. Text/layout/metadata/voice paths test or propagate that high bit before appending retained history. HISTORY.BIN disables recording at entry and reenables it at exit so the backlog UI does not record itself. ### 0x1d0 `step-text-history` (step-text-history, argc 3) -- **summary:** (out_layout_slot)(out_record_index)(delta) - move through retained ADV text-history groups and return the resolved layout slot and first record index, or -1 outputs at a boundary. +- **summary:** (out_layout_slot)(out_record_index)(delta) - resolve a cumulative delta from the latest retained ADV boundary and return its layout slot and first record index, or -1 outputs at a boundary. - **grounding:** source=investigation, confidence=high -- **evidence:** Ghidra /v2: op_0x1d0_step_text_history@0x427360 calls text_history_step_group@0x4537c0 with operand 3 and mode mask 2, then writes the index entry's first field (layout slot) and second field (record index). The helper walks the 8-byte logical index over 0x48-byte retained records, skips record-flag bit1 for mode mask 2, and returns -1 outputs at a boundary. HISTORY.BIN uses negative deltas to count/page backward and positive deltas to page forward. +- **evidence:** Ghidra /v2: op_0x1d0_step_text_history@0x427360 calls text_history_step_group@0x4537c0 with operand 3 and mode mask 2, then writes the index entry's first field (layout slot) and second field (record index). Layout define/reset set manager+0xd6c to the newest entry; the helper reads but does not mutate that anchor, so deltas are cumulative. It skips duplicate record offsets and record-flag bit1, and returns -1 outputs at a boundary. HISTORY.BIN uses negative cumulative deltas to count/page backward. ### 0x1d1 `render-text-history` (u0041BAE0, argc 5) - **summary:** (layout_slot)(record_index)(flags)(color_a)(color_b) - render retained ADV text records into a selected text layout/surface. @@ -90,15 +90,15 @@ - **grounding:** source=investigation, confidence=high - **evidence:** Ghidra /v2: op_0x1d2_handler@0x41f9c0 tests ctx+0x55110 and, when recording is enabled, calls text_history_append_typed_metadata@0x455f00. That appends a 0x48-byte record with flag 0x20000000, operand 1 at +0x14, and operand 2 at +0x18, consuming the pending group-start flag if set. The corpus has 17,323 uses; HISTORY.BIN later queries metadata types 1 and 2 through op 0x1d3. This opcode is effectful, not a statement marker. -### 0x1d3 `find-text-history-value` (u0041BB90, argc 5) +### 0x1d3 `find-text-history-value` (find-text-history-value, argc 5) - **summary:** (out_found)(out_value)(direction)(record_index)(value_type) - find typed metadata within one retained ADV message group. - **grounding:** source=investigation, confidence=high -- **evidence:** Ghidra /v2: op_0x1d3_find_text_history_value@0x4273c0 calls text_history_find_typed_value@0x450840 and writes its boolean result plus returned value. The helper scans the 0x48-byte text records within a logical group for flag 0x20000000 and matching type in record+0x18, returning record+0x14. HISTORY.BIN queries types 1 and 2 for line decoration/name metadata. +- **evidence:** Ghidra /v2: op_0x1d3_find_text_history_value@0x4273c0 calls text_history_find_typed_value@0x450840 and writes its boolean result plus returned value. The helper scans forward from operand 4 until the next group-start bit for flag 0x20000000 and matching type in record+0x18, returning the last match's record+0x14. Operand 3 reaches an unused helper parameter. HISTORY.BIN passes 1 and queries types 1 and 2 for line decoration/name metadata. -### 0x1d4 `find-text-history-pair` (u0041BC00, argc 4) +### 0x1d4 `find-text-history-pair` (find-text-history-pair, argc 4) - **summary:** (out_a)(out_b)(direction)(record_index) - find paired metadata within one retained ADV message group. - **grounding:** source=investigation, confidence=high -- **evidence:** Ghidra /v2: op_0x1d4_find_text_history_pair@0x427430 calls text_history_find_pair@0x4509f0 and writes two outputs. The helper scans the logical group for a record flagged 0x40000000 and returns its +0x14/+0x18 pair. HISTORY.BIN uses the pair to expose voice replay for a backlog entry. +- **evidence:** Ghidra /v2: op_0x1d4_find_text_history_pair@0x427430 calls text_history_find_pair@0x4509f0 and writes two outputs. The helper scans forward from operand 4 until the next group-start bit and returns the last 0x40000000 record's +0x14/+0x18 pair, defaulting both outputs to -1. Operand 3 reaches an unused helper parameter. HISTORY.BIN passes 1 and uses the pair to expose voice replay for a backlog entry. ### 0x204 `draw-string` (draw-string, argc 4) - **summary:** (surface_slot)(x)(y)(string) - rasterize a CP932 string immediately into a numbered graphics surface using current font/color/effect state. @@ -152,7 +152,7 @@ ### 0x64 `copy-inline-int-array` (copy-inline-int-array, argc 2) - **summary:** (destination)(inline_blob_offset) - decode the count-prefixed integer literal blob at codebase + offset*4 and copy its values to consecutive VM integer cells beginning at destination. - **grounding:** source=investigation, confidence=high -- **evidence:** Ghidra /v2: op_0x64_handler@0x426b00 resolves operand 1 as a writable VM address, reads a count dword followed by encoded values from frame_codebase + operand2*4, reverses the engine rotate/XOR encoding with anti_tamper_a, and writes consecutive dwords. HISTORY.BIN uses 15 blobs to initialize its rectangle, coordinate, and lookup arrays. +- **evidence:** Ghidra /v2: op_0x64_handler@0x426b00 resolves operand 1 as a writable VM address, reads a count dword followed by values from frame_codebase + operand2*4, reverses the native loader's rotate/XOR in-memory representation with anti_tamper_a, and writes consecutive dwords. The original SYS4 file footer stores the count and values plainly; HISTORY.BIN uses 15 blobs to initialize its rectangle, coordinate, and lookup arrays. ### 0x135 `bit-set` (bit-set, argc 2) - **summary:** (value)(bit_index) - set the indexed bit in the destination integer. diff --git a/docs/phase-a-slice-plan.md b/docs/phase-a-slice-plan.md index 89a2e6a..7a19dd6 100644 --- a/docs/phase-a-slice-plan.md +++ b/docs/phase-a-slice-plan.md @@ -1704,3 +1704,27 @@ safe-noop. **Next:** implement generic inline integer arrays (`0x64`) and value-switch dispatch (`0xa1/0xa2/0xa3`), then wire History navigation/query ops `0x1d0`, `0x1d3`, and `0x1d4` against this model. Rendering op `0x1d1` and its presentation operations can follow once those data-side script paths execute correctly. + +### ADV History data-side navigation implemented (2026-07-18) + +The second bounded slice is complete. `Script.BodyDwords` retains the original SYS4 body for runtime data +operands, and op `0x64` copies its plain count-prefixed footer values into consecutive global or local VM +cells. The native rotate/XOR sequence reverses AGE's loader-created in-memory representation; applying it +again to the file would corrupt HISTORY's visibly plain rectangle and coordinate arrays. Generic +`0xa1/0xa2/0xa3` value-switch sequences now use formatted keys and branch to their registered or default PC. + +The retained model now exposes native navigation and query behavior. Layout boundaries update a latest-entry +anchor; op `0x1d0` resolves cumulative deltas from that anchor without mutating it, skipping duplicate first- +record offsets and navigation-filtered records. Ops `0x1d3/0x1d4` scan from a supplied record through one +group and return typed metadata or the voice pair with native defaults. Ghidra confirms both opcodes' third +operands reach unused helper parameters despite HISTORY passing 1. + +Eight new focused cases cover signed inline values, real HISTORY footer retention, matched/default value +switches, cumulative group navigation, duplicate/filter skipping, metadata/voice queries, and missing-value +defaults. Validation: engine 190/190, opcode generator tests/lint clean, zero-warning Godot build, threaded +`SELFTEST OK`. HISTORY is now 63/78 distinct opcodes and 833/854 instructions handled or safe-noop; its 15 +remaining effectful gaps total 21 instructions. + +**Next:** implement History's display slice around `0x1d1`, layout origin `0x198`, message-window alpha +`0x131`, surface fill `0x20b`, and retained-object presentation `0x222`. After visible backlog rows render, +finish rectangle hover/interaction, voice replay, and the `0xd3/0xd4/0xd5` smooth-scroll scheduler details. diff --git a/docs/sys4-format-notes.md b/docs/sys4-format-notes.md index 262bb0b..84856f5 100644 --- a/docs/sys4-format-notes.md +++ b/docs/sys4-format-notes.md @@ -110,6 +110,19 @@ task. These tags give a head start on labeling the disassembly.* - Non-scene scripts (`MENU`, `ADDEXP`) contain only a handful of control strings — consistent with the inventory's subsystem/data-table categorization. +## Inline integer arrays — CONFIRMED + +Opcode `0x64` operand 2 points to a footer blob in the same body-dword coordinate space as strings. The blob +is `[count:u32, value0:u32, ... valueN:u32]`; these dwords are stored plainly in the original SYS4 file. +For example, `HISTORY.BIN@0x13d4` is `[4, 0, 5, 0, 538]`, and its fifteen blobs initialize the backlog UI's +rectangle, coordinate, and lookup arrays. An op-`0x64` reference lowers the executable-code boundary just +like a type-2 string reference. + +Native `AGE.EXE` applies a per-context rotate/XOR representation after loading script data and reverses it +inside the opcode handler. That transformation is not part of the on-disk format: file readers and the port +must copy the footer's plain dwords, including two's-complement negative values, without applying the native +memory-cookie transform. + ## Patch-override caveat (re-confirmed) 49 loose script `.BIN` files in the game root shadow DATA1 copies at runtime and differ slightly in size; diff --git a/engine/Age.Engine.Tests/HistoryDataOpsTests.cs b/engine/Age.Engine.Tests/HistoryDataOpsTests.cs new file mode 100644 index 0000000..cf26443 --- /dev/null +++ b/engine/Age.Engine.Tests/HistoryDataOpsTests.cs @@ -0,0 +1,164 @@ +using Age.Engine.Model; +using Age.Engine.Sys4; +using Age.Engine.Vm; + +public class HistoryDataOpsTests +{ + private const int T_IMM = 0, T_GINT = 3, T_LINT = 9; + private static readonly OpcodeTable Table = OpcodeTableJson.Load(Paths.OpcodesJson); + private static Operand I(long value) => new(T_IMM, value); + private static Operand G(int address) => new(T_GINT, address); + private static Operand L(int address) => new(T_LINT, address); + + [Fact] + public void CopyInlineIntArrayWritesConsecutiveSignedDwords() + { + const int arrayOffset = 20; + var body = new uint[arrayOffset + 5]; + body[arrayOffset] = 4; + body[arrayOffset + 1] = 0; + body[arrayOffset + 2] = 5; + body[arrayOffset + 3] = 0xffff_ff74; + body[arrayOffset + 4] = 538; + var instructions = new[] + { + new Instruction(0, 0x64, new[] { L(4), I(arrayOffset) }), + new Instruction(5, 0x55, new[] { G(0x100), L(4) }), + new Instruction(10, 0x55, new[] { G(0x101), L(5) }), + new Instruction(15, 0x55, new[] { G(0x102), L(6) }), + new Instruction(20, 0x55, new[] { G(0x103), L(7) }), + new Instruction(25, 0x2, Array.Empty()), + }; + var script = new Script + { + Name = "INLINE_ARRAY", + Header = new ScriptHeader(0, 0, 0, 0, 0, 0), + Instructions = instructions, + IndexByOffset = instructions.Select((ins, index) => (ins.Offset, index)) + .ToDictionary(pair => pair.Offset, pair => pair.index), + Strings = new Dictionary(), + BodyDwords = body, + }; + + var vm = new VirtualMachine(script, Table, new RecordingHost()); + vm.Run(); + + Assert.Equal("exit", vm.HaltReason); + Assert.Equal(new long[] { 0, 5, -140, 538 }, + new[] { vm.Globals[0x100], vm.Globals[0x101], vm.Globals[0x102], vm.Globals[0x103] }); + } + + [Fact] + public void LoaderRetainsRealHistoryFooterArraysForRuntimeCopy() + { + var history = Sys4Loader.Load(Paths.Scripts()["HISTORY.BIN"], Table); + + Assert.Equal(4u, history.BodyDwords[0x13d4]); + Assert.Equal(new uint[] { 0, 5, 0, 538 }, history.BodyDwords.Skip(0x13d5).Take(4)); + Assert.Equal(14u, history.BodyDwords[0x1424]); + Assert.Equal(0xffff_ff74u, history.BodyDwords[0x1424 + 10]); + } + + [Theory] + [InlineData(3, 30)] + [InlineData(5, 50)] + [InlineData(9, -1)] + public void ValueSwitchJumpsToMatchingCaseOrDefault(int selector, int expected) + { + var script = ScriptAssembler.Assemble(Table, "VALUE_SWITCH", + new List<(int, Operand[])> + { + (0xa1, Array.Empty()), // 0 + (0xa2, new[] { I(3), I(16) }), // 1 + (0xa2, new[] { I(5), I(24) }), // 6 + (0xa3, new[] { G(0x100), I(32) }), // 11 + (0x55, new[] { G(0x101), I(30) }), // 16 + (0x8c, new[] { I(37) }), // 21 + (0x55, new[] { G(0x101), I(50) }), // 24 + (0x8c, new[] { I(37) }), // 29 + (0x55, new[] { G(0x101), I(-1) }), // 32 + (0x2, Array.Empty()), // 37 + }, Array.Empty()); + var vm = new VirtualMachine(script, Table, new RecordingHost()); + vm.Globals[0x100] = selector; + + vm.Run(); + + Assert.Equal("exit", vm.HaltReason); + Assert.Equal(expected, unchecked((int)vm.Globals[0x101])); + } + + [Fact] + public void HistoryOpcodesStepFromLatestBoundaryAndQueryOneGroup() + { + var history = new AdvTextHistory(); + history.ResetLayout(1); + history.AppendMetadata(111, 1, AdvTextStyle.Default); + history.AppendText(0, 10, "older", AdvTextStyle.Default); + history.ResetLayout(2); + history.AppendMetadata(222, 2, AdvTextStyle.Default); + history.AppendVoice(77, 9, AdvTextStyle.Default); + history.AppendText(0, 11, "target", AdvTextStyle.Default); + history.ResetLayout(3); + history.AppendText(0, 12, "latest", AdvTextStyle.Default); + + var script = ScriptAssembler.Assemble(Table, "HISTORY_QUERY", + new List<(int, Operand[])> + { + (0x51, new[] { L(10), I(0), I(1) }), // delta = -1 + (0x1d0, new[] { L(0), L(1), L(10) }), + (0x55, new[] { G(0x100), L(0) }), + (0x55, new[] { G(0x101), L(1) }), + (0x1d3, new[] { L(2), L(3), I(1), L(1), I(2) }), + (0x55, new[] { G(0x102), L(2) }), + (0x55, new[] { G(0x103), L(3) }), + (0x1d4, new[] { L(4), L(5), I(1), L(1) }), + (0x55, new[] { G(0x104), L(4) }), + (0x55, new[] { G(0x105), L(5) }), + (0x2, Array.Empty()), + }, Array.Empty()); + var vm = new VirtualMachine(script, Table, new RecordingHost(), textHistory: history); + + vm.Run(); + + Assert.Equal("exit", vm.HaltReason); + Assert.Equal(2, vm.Globals[0x100]); + Assert.Equal(2, vm.Globals[0x101]); + Assert.Equal(1, vm.Globals[0x102]); + Assert.Equal(222, vm.Globals[0x103]); + Assert.Equal(77, vm.Globals[0x104]); + Assert.Equal(9, vm.Globals[0x105]); + } + + [Fact] + public void StepSkipsFilteredAndDuplicateLogicalEntries() + { + var history = new AdvTextHistory(); + history.ResetLayout(1); + history.AppendText(0, 1, "visible", AdvTextStyle.Default); + history.ResetLayout(2); + history.ResetLayout(2); // duplicate first-record index, as native structural calls may produce + history.AppendText(0, 2, "filtered", AdvTextStyle.Default, + AdvTextHistoryRecordFlags.NavigationFiltered); + history.ResetLayout(3); + history.AppendText(0, 3, "latest", AdvTextStyle.Default); + + Assert.True(history.TryStepGroup(-1, out var prior)); + Assert.Equal(new AdvTextHistoryEntry(1, 0), prior); + Assert.False(history.TryStepGroup(-2, out var boundary)); + Assert.Equal(new AdvTextHistoryEntry(-1, -1), boundary); + } + + [Fact] + public void MissingHistoryQueriesUseNativeOutputDefaults() + { + var history = new AdvTextHistory(); + history.ResetLayout(1); + history.AppendText(0, 1, "plain", AdvTextStyle.Default); + + Assert.False(history.TryFindMetadata(0, 7, out long metadata)); + Assert.Equal(0, metadata); + Assert.False(history.TryFindVoicePair(0, out long voiceId, out long voiceArgument)); + Assert.Equal((-1L, -1L), (voiceId, voiceArgument)); + } +} diff --git a/engine/Age.Engine/Model/AdvTextHistory.cs b/engine/Age.Engine/Model/AdvTextHistory.cs index 9053993..f5173bb 100644 --- a/engine/Age.Engine/Model/AdvTextHistory.cs +++ b/engine/Age.Engine/Model/AdvTextHistory.cs @@ -78,6 +78,7 @@ public sealed class AdvTextHistory private readonly List _entries = new(); private readonly Dictionary _layouts = new(); private readonly HashSet _pendingGroupStarts = new(); + private int _navigationAnchorIndex = -1; public IReadOnlyList Records => _records; public IReadOnlyList Entries => _entries; @@ -114,10 +115,11 @@ public sealed class AdvTextHistory layout.CursorY = y; } - public void AppendText(int requestedSlot, int sourceOffset, string text, AdvTextStyle style) + public void AppendText(int requestedSlot, int sourceOffset, string text, AdvTextStyle style, + AdvTextHistoryRecordFlags flags = AdvTextHistoryRecordFlags.None) { int slot = ResolveLayout(requestedSlot); - AppendRecord(slot, AdvTextHistoryRecordKind.Text, AdvTextHistoryRecordFlags.None, + AppendRecord(slot, AdvTextHistoryRecordKind.Text, flags, style, text, 0, 0, sourceOffset); } @@ -135,6 +137,81 @@ public sealed class AdvTextHistory _records.Clear(); _entries.Clear(); _pendingGroupStarts.Clear(); + _navigationAnchorIndex = -1; + } + + /// + /// Resolve a logical entry relative to AGE's latest-boundary navigation anchor. Repeated calls do not + /// mutate the anchor; HISTORY.BIN supplies cumulative deltas while counting and paging backward. + /// + public bool TryStepGroup(int delta, out AdvTextHistoryEntry entry) + { + entry = new AdvTextHistoryEntry(-1, -1); + if ((uint)_navigationAnchorIndex >= (uint)_entries.Count) return false; + + int index = _navigationAnchorIndex; + if (delta < 0) + { + for (int remaining = -delta; remaining > 0; remaining--) + { + int firstRecord = _entries[index].FirstRecordIndex; + do + { + if (firstRecord == 0 || index < 1) return false; + index--; + } + while (IsNavigationFiltered(_entries[index]) + || _entries[index].FirstRecordIndex == firstRecord); + } + } + else + { + for (int remaining = delta; remaining > 0; remaining--) + { + int firstRecord = _entries[index].FirstRecordIndex; + do + { + index++; + if (index >= _entries.Count) return false; + // Native treats the last entry's record offset as the forward sentinel. + if (_entries[index].FirstRecordIndex == _entries[^1].FirstRecordIndex) return false; + } + while (IsNavigationFiltered(_entries[index]) + || _entries[index].FirstRecordIndex == firstRecord); + } + } + + entry = _entries[index]; + return true; + } + + public bool TryFindMetadata(int firstRecordIndex, long metadataType, out long value) + { + value = 0; + bool found = false; + foreach (var record in EnumerateGroup(firstRecordIndex)) + { + if (!record.Flags.HasFlag(AdvTextHistoryRecordFlags.TypedMetadata) + || record.AuxValue != metadataType) continue; + value = record.Value; + found = true; + } + return found; + } + + public bool TryFindVoicePair(int firstRecordIndex, out long voiceId, out long voiceArgument) + { + voiceId = -1; + voiceArgument = -1; + bool found = false; + foreach (var record in EnumerateGroup(firstRecordIndex)) + { + if (!record.Flags.HasFlag(AdvTextHistoryRecordFlags.VoicePair)) continue; + voiceId = record.Value; + voiceArgument = record.AuxValue; + found = true; + } + return found; } private int SelectLayout(int requestedSlot) @@ -161,6 +238,22 @@ public sealed class AdvTextHistory if (RecordingSuppressed) return; _entries.Add(new AdvTextHistoryEntry(slot, _records.Count)); _pendingGroupStarts.Add(slot); + _navigationAnchorIndex = _entries.Count - 1; + } + + private bool IsNavigationFiltered(AdvTextHistoryEntry entry) + => (uint)entry.FirstRecordIndex < (uint)_records.Count + && _records[entry.FirstRecordIndex].Flags.HasFlag(AdvTextHistoryRecordFlags.NavigationFiltered); + + private IEnumerable EnumerateGroup(int firstRecordIndex) + { + if ((uint)firstRecordIndex >= (uint)_records.Count) yield break; + for (int i = firstRecordIndex; i < _records.Count; i++) + { + if (i > firstRecordIndex + && _records[i].Flags.HasFlag(AdvTextHistoryRecordFlags.GroupStart)) yield break; + yield return _records[i]; + } } private void AppendRecord(int slot, AdvTextHistoryRecordKind kind, AdvTextHistoryRecordFlags flags, diff --git a/engine/Age.Engine/Model/Script.cs b/engine/Age.Engine/Model/Script.cs index e002dfc..c1157bb 100644 --- a/engine/Age.Engine/Model/Script.cs +++ b/engine/Age.Engine/Model/Script.cs @@ -6,5 +6,7 @@ public sealed class Script public required IReadOnlyList Instructions { get; init; } public required IReadOnlyDictionary IndexByOffset { get; init; } public required IReadOnlyDictionary Strings { get; init; } + /// Unmodified SYS4 body dwords, retained for inline data operands such as opcode 0x64. + public IReadOnlyList BodyDwords { get; init; } = Array.Empty(); public string GetString(int offset) => Strings.TryGetValue(offset, out var s) ? s : ""; } diff --git a/engine/Age.Engine/Sys4/Sys4Loader.cs b/engine/Age.Engine/Sys4/Sys4Loader.cs index ab09af8..8b25632 100644 --- a/engine/Age.Engine/Sys4/Sys4Loader.cs +++ b/engine/Age.Engine/Sys4/Sys4Loader.cs @@ -22,7 +22,15 @@ public static class Sys4Loader var header = new ScriptHeader(fields[0], fields[1], fields[2], fields[3], fields[4], fields[5]); var (instrs, idxByOff, strings) = DecodeCode(dw, fields, nbody, table); - return new Script { Name = name, Header = header, Instructions = instrs, IndexByOffset = idxByOff, Strings = strings }; + return new Script + { + Name = name, + Header = header, + Instructions = instrs, + IndexByOffset = idxByOff, + Strings = strings, + BodyDwords = dw, + }; } private static (List, Dictionary, Dictionary) diff --git a/engine/Age.Engine/Vm/VirtualMachine.cs b/engine/Age.Engine/Vm/VirtualMachine.cs index fa28fe2..698c869 100644 --- a/engine/Age.Engine/Vm/VirtualMachine.cs +++ b/engine/Age.Engine/Vm/VirtualMachine.cs @@ -35,6 +35,7 @@ public sealed class VirtualMachine private bool _autoVoicePending; private volatile bool _messageSkipEnabled; private AdvTextStyle _advTextStyle = AdvTextStyle.Default; + private readonly Dictionary _valueSwitchTargets = new(StringComparer.Ordinal); public long CallScriptDispatches { get; private set; } public Dictionary Globals { get; } = new(); @@ -217,6 +218,24 @@ public sealed class VirtualMachine } } + private void WriteConsecutive(Operand destination, int index, long value) + { + int address = checked((int)destination.Value + index); + switch (destination.Type) + { + case T_GINT: case T_GFLOAT: Globals[address] = value; break; + case T_LINT: _cur.Locals.I[address] = value; break; + case T_LFLOAT: _cur.Locals.F[address] = value; break; + case T_GPTR: Globals[checked((int)Gi(Globals, (int)destination.Value) + index)] = value; break; + case T_LPTR: Globals[checked((int)Gi(_cur.Locals.P, (int)destination.Value) + index)] = value; break; + } + } + + private string FormatSwitchValue(Operand operand) + => IsStr(operand) + ? ReadStr(operand) + : unchecked((int)Read(operand)).ToString(System.Globalization.CultureInfo.InvariantCulture); + private enum FrameOutcome { Returned, Halted, RanOff } public void Run(int entryOffset = 0) @@ -333,6 +352,26 @@ public sealed class VirtualMachine LookupStore(a[0], BaseAddr(a[1]) + Read(a[2])); return pc + 1; case "lookup-array-2d": LookupStore(a[0], BaseAddr(a[1]) + Read(a[2]) * Read(a[3]) + Read(a[4])); return pc + 1; + case "copy-inline-int-array": // 0x64: count dword followed by plain file values + { + int offset = checked((int)Read(a[1])); + if ((uint)offset >= (uint)_cur.Script.BodyDwords.Count) + { + HaltReason ??= $"inline-array-offset@0x{offset:x}"; + return HALT; + } + uint rawCount = _cur.Script.BodyDwords[offset]; + int available = _cur.Script.BodyDwords.Count - offset - 1; + if (rawCount > (uint)available) + { + HaltReason ??= $"inline-array-length@0x{offset:x}:{rawCount}"; + return HALT; + } + int count = (int)rawCount; + for (int i = 0; i < count; i++) + WriteConsecutive(a[0], i, unchecked((int)_cur.Script.BodyDwords[offset + 1 + i])); + return pc + 1; + } case "bit-set": { long bit = Read(a[1]); @@ -357,6 +396,16 @@ public sealed class VirtualMachine long tgt = Read(a[0]) != 0 ? a[1].Value : a[2].Value; return tgt == NoJump ? pc + 1 : _cur.Script.IndexByOffset.GetValueOrDefault((int)tgt, pc + 1); } + case "begin-value-switch": + _valueSwitchTargets.Clear(); return pc + 1; + case "add-value-switch-case": + _valueSwitchTargets[FormatSwitchValue(a[0])] = checked((int)Read(a[1])); return pc + 1; + case "value-switch-jump": + { + int target = _valueSwitchTargets.TryGetValue(FormatSwitchValue(a[0]), out int matched) + ? matched : checked((int)Read(a[1])); + return _cur.Script.IndexByOffset.GetValueOrDefault(target, pc + 1); + } case "u0041ADB0": case "coroutine-save-yield-handlers": // 0x7b: retain native handler metadata _cur.CoroutineYieldHandlerA = (int)Read(a[0]); @@ -619,6 +668,32 @@ public sealed class VirtualMachine return pc + 1; case "append-text-history-metadata": // 0x1d2: typed value attached to the current group TextHistory.AppendMetadata(Read(a[0]), Read(a[1]), _advTextStyle); return pc + 1; + case "step-text-history": // 0x1d0: cumulative delta from the latest retained boundary + if (TextHistory.TryStepGroup((int)Read(a[2]), out var historyEntry)) + { + Write(a[0], historyEntry.LayoutSlot); + Write(a[1], historyEntry.FirstRecordIndex); + } + else + { + Write(a[0], -1); + Write(a[1], -1); + } + return pc + 1; + case "u0041BB90": + case "find-text-history-value": // 0x1d3: operand 3 is accepted but ignored natively + { + bool found = TextHistory.TryFindMetadata((int)Read(a[3]), Read(a[4]), out long value); + Write(a[0], found ? 1 : 0); + Write(a[1], value); + return pc + 1; + } + case "u0041BC00": + case "find-text-history-pair": // 0x1d4: operand 3 is accepted but ignored natively + TextHistory.TryFindVoicePair((int)Read(a[3]), out long voiceId, out long voiceArgument); + Write(a[0], voiceId); + Write(a[1], voiceArgument); + return pc + 1; case "clear-text-history": // 0x85: bound the backlog to the current ordinary ADV block TextHistory.Clear(); return pc + 1; case "set-font-size": diff --git a/tools/age_opcodes_himegari.py b/tools/age_opcodes_himegari.py index e070e4a..2deacb6 100644 --- a/tools/age_opcodes_himegari.py +++ b/tools/age_opcodes_himegari.py @@ -40,8 +40,6 @@ INFERRED: dict[int, dict] = { 0x1ca: dict(name='set-read-message-skip', category='input', noop=False, confidence='high', source='investigation', summary='(enabled) - set the engine setting `message:ReadTextSkip`, which skips only previously read text.'), 0x1cb: dict(name='get-read-message-skip', category='input', noop=False, confidence='high', source='investigation', summary='(out) - read the engine setting `message:ReadTextSkip`.'), 0x1d1: dict(name='render-text-history', category='adv', noop=False, confidence='high', source='investigation', summary='(layout_slot)(record_index)(flags)(color_a)(color_b) - render retained ADV text records into a selected text layout/surface.'), - 0x1d3: dict(name='find-text-history-value', category='adv', noop=False, confidence='high', source='investigation', summary='(out_found)(out_value)(direction)(record_index)(value_type) - find typed metadata within one retained ADV message group.'), - 0x1d4: dict(name='find-text-history-pair', category='adv', noop=False, confidence='high', source='investigation', summary='(out_a)(out_b)(direction)(record_index) - find paired metadata within one retained ADV message group.'), 0x1d5: dict(name='cond-block', category='marker', noop=True, confidence='high', source='inference', summary='zero-arg; ALWAYS follows jcc — marks conditional body entry'), 0x1f4: dict(name='stmt-begin', category='marker', noop=True, confidence='high', source='investigation', summary='zero-arg; opens scripts, pairs with stmt-end 0x1f5'), 0x1f5: dict(name='stmt-end', category='marker', noop=True, confidence='high', source='investigation', summary='zero-arg; precedes exit/next-stmt, pairs with 0x1f4'), diff --git a/vm-map/opcodes.toml b/vm-map/opcodes.toml index 8b64770..bb74f6f 100644 --- a/vm-map/opcodes.toml +++ b/vm-map/opcodes.toml @@ -859,7 +859,7 @@ noop_headless = false source = "investigation" confidence = "high" depends_on = [] -evidence = "Ghidra /v2: op_0x64_handler@0x426b00 resolves operand 1 as a writable VM address, reads a count dword followed by encoded values from frame_codebase + operand2*4, reverses the engine rotate/XOR encoding with anti_tamper_a, and writes consecutive dwords. HISTORY.BIN uses 15 blobs to initialize its rectangle, coordinate, and lookup arrays." +evidence = "Ghidra /v2: op_0x64_handler@0x426b00 resolves operand 1 as a writable VM address, reads a count dword followed by values from frame_codebase + operand2*4, reverses the native loader's rotate/XOR in-memory representation with anti_tamper_a, and writes consecutive dwords. The original SYS4 file footer stores the count and values plainly; HISTORY.BIN uses 15 blobs to initialize its rectangle, coordinate, and lookup arrays." [[opcode.semantics.args]] i = 1 @@ -4276,12 +4276,12 @@ abi_source = "kelebek+decode-validated" [opcode.semantics] name = "step-text-history" category = "adv" -summary = "(out_layout_slot)(out_record_index)(delta) - move through retained ADV text-history groups and return the resolved layout slot and first record index, or -1 outputs at a boundary." +summary = "(out_layout_slot)(out_record_index)(delta) - resolve a cumulative delta from the latest retained ADV boundary and return its layout slot and first record index, or -1 outputs at a boundary." noop_headless = false source = "investigation" confidence = "high" depends_on = [] -evidence = "Ghidra /v2: op_0x1d0_step_text_history@0x427360 calls text_history_step_group@0x4537c0 with operand 3 and mode mask 2, then writes the index entry's first field (layout slot) and second field (record index). The helper walks the 8-byte logical index over 0x48-byte retained records, skips record-flag bit1 for mode mask 2, and returns -1 outputs at a boundary. HISTORY.BIN uses negative deltas to count/page backward and positive deltas to page forward." +evidence = "Ghidra /v2: op_0x1d0_step_text_history@0x427360 calls text_history_step_group@0x4537c0 with operand 3 and mode mask 2, then writes the index entry's first field (layout slot) and second field (record index). Layout define/reset set manager+0xd6c to the newest entry; the helper reads but does not mutate that anchor, so deltas are cumulative. It skips duplicate record offsets and record-flag bit1, and returns -1 outputs at a boundary. HISTORY.BIN uses negative cumulative deltas to count/page backward." [[opcode.semantics.args]] i = 1 @@ -4367,7 +4367,7 @@ observed_types = ["imm"] [[opcode]] op = 0x1d3 -label = "u0041BB90" +label = "find-text-history-value" argc = 5 abi_source = "kelebek+decode-validated" @@ -4379,7 +4379,7 @@ noop_headless = false source = "investigation" confidence = "high" depends_on = [] -evidence = "Ghidra /v2: op_0x1d3_find_text_history_value@0x4273c0 calls text_history_find_typed_value@0x450840 and writes its boolean result plus returned value. The helper scans the 0x48-byte text records within a logical group for flag 0x20000000 and matching type in record+0x18, returning record+0x14. HISTORY.BIN queries types 1 and 2 for line decoration/name metadata." +evidence = "Ghidra /v2: op_0x1d3_find_text_history_value@0x4273c0 calls text_history_find_typed_value@0x450840 and writes its boolean result plus returned value. The helper scans forward from operand 4 until the next group-start bit for flag 0x20000000 and matching type in record+0x18, returning the last match's record+0x14. Operand 3 reaches an unused helper parameter. HISTORY.BIN passes 1 and queries types 1 and 2 for line decoration/name metadata." [[opcode.semantics.args]] i = 1 @@ -4393,7 +4393,7 @@ observed_types = ["l-int"] [[opcode.semantics.args]] i = 3 -role = "" +role = "reserved (ignored by native helper)" observed_types = ["imm"] [[opcode.semantics.args]] @@ -4408,7 +4408,7 @@ observed_types = ["imm"] [[opcode]] op = 0x1d4 -label = "u0041BC00" +label = "find-text-history-pair" argc = 4 abi_source = "kelebek+decode-validated" @@ -4420,7 +4420,7 @@ noop_headless = false source = "investigation" confidence = "high" depends_on = [] -evidence = "Ghidra /v2: op_0x1d4_find_text_history_pair@0x427430 calls text_history_find_pair@0x4509f0 and writes two outputs. The helper scans the logical group for a record flagged 0x40000000 and returns its +0x14/+0x18 pair. HISTORY.BIN uses the pair to expose voice replay for a backlog entry." +evidence = "Ghidra /v2: op_0x1d4_find_text_history_pair@0x427430 calls text_history_find_pair@0x4509f0 and writes two outputs. The helper scans forward from operand 4 until the next group-start bit and returns the last 0x40000000 record's +0x14/+0x18 pair, defaulting both outputs to -1. Operand 3 reaches an unused helper parameter. HISTORY.BIN passes 1 and uses the pair to expose voice replay for a backlog entry." [[opcode.semantics.args]] i = 1 @@ -4434,7 +4434,7 @@ observed_types = ["l-int"] [[opcode.semantics.args]] i = 3 -role = "" +role = "reserved (ignored by native helper)" observed_types = ["imm"] [[opcode.semantics.args]]