From 826391c5af65a6edcf35c30ef75df433072567c0 Mon Sep 17 00:00:00 2001 From: gamer147 Date: Wed, 29 Jul 2026 18:35:21 -0400 Subject: [PATCH] Match AGE RIFF decode boundaries --- docs/asset-resolution-re.md | 18 ++++++++- docs/engine-re.md | 17 +++++++++ docs/phase-b-framework.md | 18 +++++++++ .../RiffWaveSanitizerTests.cs | 37 +++++++++++++++++-- godot/Main.cs | 15 ++++++-- godot/RiffWaveSanitizer.cs | 16 ++++---- 6 files changed, 105 insertions(+), 16 deletions(-) diff --git a/docs/asset-resolution-re.md b/docs/asset-resolution-re.md index d5fa115..f438491 100644 --- a/docs/asset-resolution-re.md +++ b/docs/asset-resolution-re.md @@ -173,8 +173,11 @@ Godot's WAV loader adds one host-specific compatibility boundary. SC0000 `0xc29` the EV052DA glow. Its PCM is valid, but its trailing RIFF `LIST/INFO` fields contain Japanese CP932 text (`IPRD`, `IGNR`, and `ICMT`). Godot assumes INFO text is UTF-8 and formerly emitted one Unicode warning per invalid CP932 byte each time the sound was loaded; the duplicated SC0000 burst came from loading raw id -`0x28` on two channels. A complete extracted-corpus scan found 238 RIFF/WAVE files, 61 INFO chunks, no other -LIST type, and no invalid RIFF containers, explaining the same warnings around combat SFX. +`0x28` on two channels. A complete extracted-corpus scan found 238 RIFF/WAVE files, 61 INFO chunks, and no +other LIST type, explaining the same warnings around combat SFX. A later declared-extent audit corrected the +earlier claim that every container ended exactly at its RIFF boundary: seven entries have trailing bytes. +Six have small 74- or 234-byte tails; `A1215.WAV` is a 323,009-byte concatenation whose first RIFF ends at +157,940, followed by a multipart-upload header and a second complete RIFF. The correction is deliberately confined to the Godot frontend. `RiffWaveSanitizer` removes only `LIST` chunks whose form type is `INFO` from the transient byte array passed to `AudioStreamWav.LoadFromBuffer`, updates the @@ -184,6 +187,17 @@ byte-for-byte, including padding. The shared engine, original loose/archive payl bytes while retaining an identical PCM data chunk and loads headlessly without a Unicode warning. Synthetic chunk/padding tests and the installed E0808 regression cover the adapter. +`A1215.WAV` exposes a separate compatibility boundary. Native AGE's +`wav_decoder_open@0x488790` uses WinMM `mmioDescend` to enter the first `RIFF/WAVE`, finds its first `data` +child, and streams exactly that child's declared `cksize`; appended bytes are never decoded. Godot's +whole-buffer importer instead walks beyond the first RIFF and interprets the multipart header's `Cont` as a +chunk id, causing repeated out-of-range seeks. `RiffWaveSanitizer.PrepareForGodot` now validates the first +RIFF and all of its declared children, removes any INFO chunks, and returns a transient buffer ending exactly +at that RIFF's declared extent. It returns the original array without rewriting when the RIFF is invalid and +preserves the packed/original bytes in every case. The installed A1215 regression proves 323,009 source bytes +become the exact 157,940-byte first RIFF with the 157,896-byte PCM `data` chunk unchanged; the Godot selftest +loads that result through `AudioStreamWav.LoadFromBuffer` without seek errors. + ## Runtime asset-VFS track (VFS-A/B/C complete 2026-07-11) The pre-extracted tree and `build/textures/*.BMP` pipeline were a Phase-A bootstrap, not the desired final diff --git a/docs/engine-re.md b/docs/engine-re.md index 6a880bf..0954c9a 100644 --- a/docs/engine-re.md +++ b/docs/engine-re.md @@ -2987,6 +2987,23 @@ attenuation `-2377` to both channel loads, and the shared audio service later re five handlers. The bounded port does not yet import native audio preferences, so its extracted-WAV bootstrap uses unity gain and centered pan rather than hard-coding the captured user's setting. +**Native RIFF boundary contract (2026-07-29).** The RIFF branch constructs +`wav_decoder_ctor@0x4885b0`; `wav_decoder_open@0x488790` then opens the packed-entry handle with WinMM +`mmioOpenA`, descends into the first `RIFF/WAVE` with `MMIO_FINDRIFF`, and searches that RIFF's children for +`fmt ` and `data` with `MMIO_FINDCHUNK`. At `0x488976`, the returned `data` `MMCKINFO.cksize` is copied into +both remaining-byte and total-byte fields. `wav_decoder_fill_buffer@0x4889b0` reads +`min(remaining_bytes, 0x2000)`, decrements that counter, and closes the decoder when it reaches zero. +Native playback therefore cannot consume bytes after the first `data` chunk's declared extent. + +This normal RIFF behavior, rather than an asset-specific exception, makes malformed `A1215.WAV` playable. +Its packed entry is 323,009 bytes, but the first RIFF declares an extent of 157,940 bytes and its first +`data` chunk is 157,896 bytes. A multipart-upload header and a complete second RIFF begin after that declared +boundary; AGE ignores both. The whole-buffer Godot importer instead sees the appended `Cont` bytes as another +RIFF chunk and repeatedly seeks beyond the buffer. The port now reproduces the native boundary in its +transient Godot decoder copy, leaving the archive/VFS payload untouched. The saved `/v2` image names/comments +the WAV decoder constructor, open/close, buffer-fill, rewind, and query methods, and records this boundary +contract on the dispatcher and parser. + The native trace in `build/native-sfx-trace.jsonl` captures SC0000's first pair: `0xb4@0xc29` resolves raw catalog id `0x28` to `E0808.WAV`, loads channel 0, and `0xb5@0xc2e` starts it in the same millisecond. The next `0xb4@0xc31` preloads the same WAV into engine-owned secondary channel 4 for a later service start. The diff --git a/docs/phase-b-framework.md b/docs/phase-b-framework.md index 0f73018..07643be 100644 --- a/docs/phase-b-framework.md +++ b/docs/phase-b-framework.md @@ -910,6 +910,24 @@ again when FIELD resumes, while SC0600 retains its ordinary ADV marker and input parent-callback → child-dialogue regression preserves both halves of that lifetime. The engine suite passes 526/526, the Godot build has zero warnings, and the Himegari-targeted threaded selftest passes. +**Post-boss `A1215.WAV` lockup corrected (2026-07-29):** after the first boss, +SC0010 executes `play-sound-effect 0x125` at `0xea1`. The packed `DATA1/A1215.WAV` entry is 323,009 bytes: +a valid 157,940-byte RIFF, then a multipart-upload header, then a second valid RIFF. Godot's whole-buffer WAV +importer walks into the appended header, treats `Cont` as a chunk id with an impossible size, and loops through +`FileAccessMemory.seek` errors on the main thread. + +AGE does not special-case this resource. Its WinMM decoder (`wav_decoder_open@0x488790`) descends into the +first `RIFF/WAVE`, finds `fmt ` and `data` within that parent, saves the first `data` chunk's declared size, +and `wav_decoder_fill_buffer@0x4889b0` stops after exactly that many bytes. The appended upload material and +second RIFF are unreachable. + +`RiffWaveSanitizer.PrepareForGodot` now applies that first-RIFF boundary to the transient decoder buffer after +validating every declared child, then performs the existing INFO-metadata removal within that boundary. +Archive/VFS bytes remain pristine, and invalid RIFFs are returned unchanged. Synthetic concatenation and +installed-A1215 regressions prove the exact prefix and unchanged PCM payload. Validation passes 528/528 engine +tests, a zero-warning Godot build, and the Himegari-targeted threaded selftest, which loads the real A1215 +buffer through Godot and reports `first-riff-boundary=ok` without seek spam. + **Cyclic reset implemented (2026-07-29):** `0x230(handle)` now gets or creates the retained object, disables the four looping channels represented by the compositor, and clears the complete native start/period block—including the preserved raw state for the currently unmodeled second cyclic matrix. diff --git a/engine/Age.Engine.Tests/RiffWaveSanitizerTests.cs b/engine/Age.Engine.Tests/RiffWaveSanitizerTests.cs index 98cbe8a..487049c 100644 --- a/engine/Age.Engine.Tests/RiffWaveSanitizerTests.cs +++ b/engine/Age.Engine.Tests/RiffWaveSanitizerTests.cs @@ -15,7 +15,7 @@ public class RiffWaveSanitizerTests byte[] data = { 1, 2, 3 }; byte[] source = Wave(("fmt ", fmt), ("LIST", info), ("smpl", sampleLoop), ("data", data)); - byte[] sanitized = RiffWaveSanitizer.RemoveInfoMetadata(source); + byte[] sanitized = RiffWaveSanitizer.PrepareForGodot(source); Assert.NotSame(source, sanitized); Assert.Equal(sanitized.Length - 8, BinaryPrimitives.ReadInt32LittleEndian(sanitized.AsSpan(4, 4))); @@ -30,7 +30,21 @@ public class RiffWaveSanitizerTests { byte[] source = Wave(("fmt ", new byte[] { 1, 0 }), ("data", new byte[] { 1, 2 })); - Assert.Same(source, RiffWaveSanitizer.RemoveInfoMetadata(source)); + Assert.Same(source, RiffWaveSanitizer.PrepareForGodot(source)); + } + + [Fact] + public void DropsBytesAfterFirstDeclaredRiffExtent() + { + byte[] first = Wave(("fmt ", new byte[] { 1, 0 }), ("data", new byte[] { 1, 2, 3, 4 })); + byte[] second = Wave(("fmt ", new byte[] { 1, 0 }), ("data", new byte[] { 5, 6 })); + byte[] separator = Encoding.ASCII.GetBytes("Content-Disposition: form-data\r\n\r\n"); + byte[] source = [.. first, .. separator, .. second]; + + byte[] sanitized = RiffWaveSanitizer.PrepareForGodot(source); + + Assert.NotSame(source, sanitized); + Assert.Equal(first, sanitized); } [Fact] @@ -39,7 +53,7 @@ public class RiffWaveSanitizerTests var resources = ResourceMap.Load(); AudioPayload audio = resources.ReadAudio(resources.ResolveSoundEffect(0x28)!); - byte[] sanitized = RiffWaveSanitizer.RemoveInfoMetadata(audio.Bytes); + byte[] sanitized = RiffWaveSanitizer.PrepareForGodot(audio.Bytes); Assert.Equal("E0808.WAV", audio.Name); Assert.Equal(688_570, audio.Bytes.Length); @@ -48,6 +62,23 @@ public class RiffWaveSanitizerTests Assert.Equal(Chunk(audio.Bytes, "data"), Chunk(sanitized, "data")); } + [Fact] + public void InstalledFirstBossSoundStopsAtFirstDeclaredRiff() + { + var resources = ResourceMap.Load(); + AudioPayload audio = resources.ReadAudio(resources.ResolveSoundEffect(0x125)!); + + byte[] sanitized = RiffWaveSanitizer.PrepareForGodot(audio.Bytes); + + Assert.Equal("A1215.WAV", audio.Name); + Assert.Equal(323_009, audio.Bytes.Length); + Assert.Equal(157_940, sanitized.Length); + Assert.Equal(157_940, + 8 + BinaryPrimitives.ReadInt32LittleEndian(sanitized.AsSpan(4, 4))); + Assert.Equal(157_896, Chunk(sanitized, "data").Length); + Assert.Equal(audio.Bytes.AsSpan(0, sanitized.Length).ToArray(), sanitized); + } + private static byte[] Wave(params (string Id, byte[] Payload)[] chunks) { int length = 12 + chunks.Sum(chunk => 8 + chunk.Payload.Length + (chunk.Payload.Length & 1)); diff --git a/godot/Main.cs b/godot/Main.cs index b0d6196..2535338 100644 --- a/godot/Main.cs +++ b/godot/Main.cs @@ -2065,8 +2065,8 @@ public partial class Main : Godot.Control _sfx[channel].Stop(); _sfx[channel].Stream = null; // This is intentionally a Godot-only compatibility boundary. The VFS and engine retain the - // original WAV bytes; only Godot's UTF-8-assuming INFO parser sees the sanitized copy. - byte[] godotWav = RiffWaveSanitizer.RemoveInfoMetadata(wavBytes); + // original WAV bytes; Godot sees an AGE-compatible first-RIFF copy without CP932 INFO metadata. + byte[] godotWav = RiffWaveSanitizer.PrepareForGodot(wavBytes); var stream = AudioStreamWav.LoadFromBuffer(godotWav); if (stream == null) { GD.Print($"WAV load failed {assetName}"); return; } stream.LoopMode = AudioStreamWav.LoopModeEnum.Disabled; @@ -2325,9 +2325,15 @@ public partial class Main : Godot.Control new InputEventKey { PhysicalKeycode = Key.Ctrl }, out int ctrlVk) && ctrlVk == 0x11; var selftestResources = new ResourceMap(_catalog, _assetStore); AudioPayload glowSfx = selftestResources.ReadAudio(selftestResources.ResolveSoundEffect(0x28)!); - byte[] glowGodotWav = RiffWaveSanitizer.RemoveInfoMetadata(glowSfx.Bytes); + byte[] glowGodotWav = RiffWaveSanitizer.PrepareForGodot(glowSfx.Bytes); bool cp932WavMetadataOk = glowGodotWav.Length == 688_336 && AudioStreamWav.LoadFromBuffer(glowGodotWav) != null; + AudioPayload bossSfx = selftestResources.ReadAudio( + selftestResources.ResolveSoundEffect(0x125)!); + byte[] bossGodotWav = RiffWaveSanitizer.PrepareForGodot(bossSfx.Bytes); + bool firstRiffBoundaryOk = bossSfx.Bytes.Length == 323_009 + && bossGodotWav.Length == 157_940 + && AudioStreamWav.LoadFromBuffer(bossGodotWav) != null; AudioPayload bgm = selftestResources.ReadAudio(selftestResources.ResolveBgm(5)!); FadeBgm(0, 10.0); bool bgmFadeStarted = _bgmFadeTween?.IsValid() == true; @@ -2411,12 +2417,14 @@ public partial class Main : Godot.Control new Sys4LogicalCanvas(_screenWidth, _screenHeight)); textEffectSmoke.QueueFree(); ok &= launcherOk && sleepMinimumOk && inputTranslationOk && cp932WavMetadataOk + && firstRiffBoundaryOk && bgmReplacementCancelsFade && bgmOneShotModeOk && bgmLoopModeOk && bgmStopReleaseOk && textEffectModesOk && fontCalibrationOk && logicalCanvasOk; if (ok) GD.Print($"SELFTEST OK: threaded host matches headless ({actual.Count} lines, full handling); " + $"debug launcher catalog/UI smoke ({debugEntries.Count} packed scripts); " + $"sleep-min=1ms; native-key-translation=ok; cp932-wav-info=ok; " + + $"first-riff-boundary=ok; " + $"bgm-fade-replacement=ok; bgm-start-modes-stop=ok; " + $"text-effect-modes=ok; font-calibration=ok; " + $"logical-canvas={_screenWidth}x{_screenHeight}; " + @@ -2424,6 +2432,7 @@ public partial class Main : Godot.Control else GD.Print($"SELFTEST FAIL: threaded={actual.Count} vs headless={expected.Count}; " + $"debug-launcher={launcherOk}; sleep-min={sleepMinimumOk}; " + $"native-key-translation={inputTranslationOk}; cp932-wav-info={cp932WavMetadataOk}; " + + $"first-riff-boundary={firstRiffBoundaryOk}; " + $"bgm-fade-replacement={bgmReplacementCancelsFade}; " + $"bgm-one-shot={bgmOneShotModeOk}; bgm-loop={bgmLoopModeOk}; " + $"bgm-stop-release={bgmStopReleaseOk}; " + diff --git a/godot/RiffWaveSanitizer.cs b/godot/RiffWaveSanitizer.cs index 51c5b1a..64ddbba 100644 --- a/godot/RiffWaveSanitizer.cs +++ b/godot/RiffWaveSanitizer.cs @@ -1,12 +1,13 @@ using System; using System.Buffers.Binary; -/// Godot-specific WAV input adapter. Godot assumes RIFF INFO strings are UTF-8, while AGE's -/// Japanese assets commonly store them as CP932. Playback does not consume these tags, so remove only -/// INFO metadata while preserving every functional RIFF chunk byte-for-byte. +/// Godot-specific WAV input adapter. Native AGE decodes only the first declared RIFF/WAVE +/// extent, while Godot walks the entire supplied buffer. AGE's Japanese assets also commonly store +/// CP932 strings in INFO metadata that Godot assumes is UTF-8. Prepare a transient decoder copy that +/// follows AGE's first-RIFF boundary and removes only INFO metadata within it. internal static class RiffWaveSanitizer { - public static byte[] RemoveInfoMetadata(byte[] wavBytes) + public static byte[] PrepareForGodot(byte[] wavBytes) { ReadOnlySpan input = wavBytes; if (input.Length < 12 || !HasId(input, 0, "RIFF") || !HasId(input, 8, "WAVE")) @@ -26,9 +27,10 @@ internal static class RiffWaveSanitizer if (isInfoList) removedBytes = checked(removedBytes + chunkBytes); cursor += chunkBytes; } - if (removedBytes == 0) return wavBytes; + if (removedBytes == 0 && declaredEnd == wavBytes.Length) return wavBytes; - var output = new byte[checked(wavBytes.Length - removedBytes)]; + int newDeclaredEnd = checked(declaredEnd - removedBytes); + var output = new byte[newDeclaredEnd]; input[..12].CopyTo(output); cursor = 12; int destination = 12; @@ -43,8 +45,6 @@ internal static class RiffWaveSanitizer cursor += chunkBytes; } - int newDeclaredEnd = declaredEnd - removedBytes; - input[declaredEnd..].CopyTo(output.AsSpan(newDeclaredEnd)); BinaryPrimitives.WriteUInt32LittleEndian(output.AsSpan(4, 4), checked((uint)(newDeclaredEnd - 8))); return output;