Match AGE RIFF decode boundaries

This commit is contained in:
gamer147
2026-07-29 18:35:21 -04:00
parent da7e0e25ce
commit 32c5d1e901
6 changed files with 105 additions and 16 deletions

View File

@@ -173,8 +173,11 @@ Godot's WAV loader adds one host-specific compatibility boundary. SC0000 `0xc29`
the EV052DA glow. Its PCM is valid, but its trailing RIFF `LIST/INFO` fields contain Japanese CP932 text the EV052DA glow. Its PCM is valid, but its trailing RIFF `LIST/INFO` fields contain Japanese CP932 text
(`IPRD`, `IGNR`, and `ICMT`). Godot assumes INFO text is UTF-8 and formerly emitted one Unicode warning per (`IPRD`, `IGNR`, and `ICMT`). Godot assumes INFO text is UTF-8 and formerly emitted one Unicode warning per
invalid CP932 byte each time the sound was loaded; the duplicated SC0000 burst came from loading raw id invalid CP932 byte each time the sound was loaded; the duplicated SC0000 burst came from loading raw id
`0x28` on two channels. A complete extracted-corpus scan found 238 RIFF/WAVE files, 61 INFO chunks, no other `0x28` on two channels. A complete extracted-corpus scan found 238 RIFF/WAVE files, 61 INFO chunks, and no
LIST type, and no invalid RIFF containers, explaining the same warnings around combat SFX. other LIST type, explaining the same warnings around combat SFX. A later declared-extent audit corrected the
earlier claim that every container ended exactly at its RIFF boundary: seven entries have trailing bytes.
Six have small 74- or 234-byte tails; `A1215.WAV` is a 323,009-byte concatenation whose first RIFF ends at
157,940, followed by a multipart-upload header and a second complete RIFF.
The correction is deliberately confined to the Godot frontend. `RiffWaveSanitizer` removes only `LIST` chunks The correction is deliberately confined to the Godot frontend. `RiffWaveSanitizer` removes only `LIST` chunks
whose form type is `INFO` from the transient byte array passed to `AudioStreamWav.LoadFromBuffer`, updates the whose form type is `INFO` from the transient byte array passed to `AudioStreamWav.LoadFromBuffer`, updates the
@@ -184,6 +187,17 @@ byte-for-byte, including padding. The shared engine, original loose/archive payl
bytes while retaining an identical PCM data chunk and loads headlessly without a Unicode warning. Synthetic bytes while retaining an identical PCM data chunk and loads headlessly without a Unicode warning. Synthetic
chunk/padding tests and the installed E0808 regression cover the adapter. chunk/padding tests and the installed E0808 regression cover the adapter.
`A1215.WAV` exposes a separate compatibility boundary. Native AGE's
`wav_decoder_open@0x488790` uses WinMM `mmioDescend` to enter the first `RIFF/WAVE`, finds its first `data`
child, and streams exactly that child's declared `cksize`; appended bytes are never decoded. Godot's
whole-buffer importer instead walks beyond the first RIFF and interprets the multipart header's `Cont` as a
chunk id, causing repeated out-of-range seeks. `RiffWaveSanitizer.PrepareForGodot` now validates the first
RIFF and all of its declared children, removes any INFO chunks, and returns a transient buffer ending exactly
at that RIFF's declared extent. It returns the original array without rewriting when the RIFF is invalid and
preserves the packed/original bytes in every case. The installed A1215 regression proves 323,009 source bytes
become the exact 157,940-byte first RIFF with the 157,896-byte PCM `data` chunk unchanged; the Godot selftest
loads that result through `AudioStreamWav.LoadFromBuffer` without seek errors.
## Runtime asset-VFS track (VFS-A/B/C complete 2026-07-11) ## Runtime asset-VFS track (VFS-A/B/C complete 2026-07-11)
The pre-extracted tree and `build/textures/*.BMP` pipeline were a Phase-A bootstrap, not the desired final The pre-extracted tree and `build/textures/*.BMP` pipeline were a Phase-A bootstrap, not the desired final

View File

@@ -2987,6 +2987,23 @@ attenuation `-2377` to both channel loads, and the shared audio service later re
five handlers. The bounded port does not yet import native audio preferences, so its extracted-WAV bootstrap five handlers. The bounded port does not yet import native audio preferences, so its extracted-WAV bootstrap
uses unity gain and centered pan rather than hard-coding the captured user's setting. uses unity gain and centered pan rather than hard-coding the captured user's setting.
**Native RIFF boundary contract (2026-07-29).** The RIFF branch constructs
`wav_decoder_ctor@0x4885b0`; `wav_decoder_open@0x488790` then opens the packed-entry handle with WinMM
`mmioOpenA`, descends into the first `RIFF/WAVE` with `MMIO_FINDRIFF`, and searches that RIFF's children for
`fmt ` and `data` with `MMIO_FINDCHUNK`. At `0x488976`, the returned `data` `MMCKINFO.cksize` is copied into
both remaining-byte and total-byte fields. `wav_decoder_fill_buffer@0x4889b0` reads
`min(remaining_bytes, 0x2000)`, decrements that counter, and closes the decoder when it reaches zero.
Native playback therefore cannot consume bytes after the first `data` chunk's declared extent.
This normal RIFF behavior, rather than an asset-specific exception, makes malformed `A1215.WAV` playable.
Its packed entry is 323,009 bytes, but the first RIFF declares an extent of 157,940 bytes and its first
`data` chunk is 157,896 bytes. A multipart-upload header and a complete second RIFF begin after that declared
boundary; AGE ignores both. The whole-buffer Godot importer instead sees the appended `Cont` bytes as another
RIFF chunk and repeatedly seeks beyond the buffer. The port now reproduces the native boundary in its
transient Godot decoder copy, leaving the archive/VFS payload untouched. The saved `/v2` image names/comments
the WAV decoder constructor, open/close, buffer-fill, rewind, and query methods, and records this boundary
contract on the dispatcher and parser.
The native trace in `build/native-sfx-trace.jsonl` captures SC0000's first pair: `0xb4@0xc29` resolves The native trace in `build/native-sfx-trace.jsonl` captures SC0000's first pair: `0xb4@0xc29` resolves
raw catalog id `0x28` to `E0808.WAV`, loads channel 0, and `0xb5@0xc2e` starts it in the same millisecond. The next raw catalog id `0x28` to `E0808.WAV`, loads channel 0, and `0xb5@0xc2e` starts it in the same millisecond. The next
`0xb4@0xc31` preloads the same WAV into engine-owned secondary channel 4 for a later service start. The `0xb4@0xc31` preloads the same WAV into engine-owned secondary channel 4 for a later service start. The

View File

@@ -910,6 +910,24 @@ again when FIELD resumes, while SC0600 retains its ordinary ADV marker and input
parent-callback → child-dialogue regression preserves both halves of that lifetime. The engine suite passes parent-callback → child-dialogue regression preserves both halves of that lifetime. The engine suite passes
526/526, the Godot build has zero warnings, and the Himegari-targeted threaded selftest passes. 526/526, the Godot build has zero warnings, and the Himegari-targeted threaded selftest passes.
**Post-boss `A1215.WAV` lockup corrected (2026-07-29):** after the first boss,
SC0010 executes `play-sound-effect 0x125` at `0xea1`. The packed `DATA1/A1215.WAV` entry is 323,009 bytes:
a valid 157,940-byte RIFF, then a multipart-upload header, then a second valid RIFF. Godot's whole-buffer WAV
importer walks into the appended header, treats `Cont` as a chunk id with an impossible size, and loops through
`FileAccessMemory.seek` errors on the main thread.
AGE does not special-case this resource. Its WinMM decoder (`wav_decoder_open@0x488790`) descends into the
first `RIFF/WAVE`, finds `fmt ` and `data` within that parent, saves the first `data` chunk's declared size,
and `wav_decoder_fill_buffer@0x4889b0` stops after exactly that many bytes. The appended upload material and
second RIFF are unreachable.
`RiffWaveSanitizer.PrepareForGodot` now applies that first-RIFF boundary to the transient decoder buffer after
validating every declared child, then performs the existing INFO-metadata removal within that boundary.
Archive/VFS bytes remain pristine, and invalid RIFFs are returned unchanged. Synthetic concatenation and
installed-A1215 regressions prove the exact prefix and unchanged PCM payload. Validation passes 528/528 engine
tests, a zero-warning Godot build, and the Himegari-targeted threaded selftest, which loads the real A1215
buffer through Godot and reports `first-riff-boundary=ok` without seek spam.
**Cyclic reset implemented (2026-07-29):** `0x230(handle)` now gets or creates the retained object, **Cyclic reset implemented (2026-07-29):** `0x230(handle)` now gets or creates the retained object,
disables the four looping channels represented by the compositor, and clears the complete native disables the four looping channels represented by the compositor, and clears the complete native
start/period block—including the preserved raw state for the currently unmodeled second cyclic matrix. start/period block—including the preserved raw state for the currently unmodeled second cyclic matrix.

View File

@@ -15,7 +15,7 @@ public class RiffWaveSanitizerTests
byte[] data = { 1, 2, 3 }; byte[] data = { 1, 2, 3 };
byte[] source = Wave(("fmt ", fmt), ("LIST", info), ("smpl", sampleLoop), ("data", data)); byte[] source = Wave(("fmt ", fmt), ("LIST", info), ("smpl", sampleLoop), ("data", data));
byte[] sanitized = RiffWaveSanitizer.RemoveInfoMetadata(source); byte[] sanitized = RiffWaveSanitizer.PrepareForGodot(source);
Assert.NotSame(source, sanitized); Assert.NotSame(source, sanitized);
Assert.Equal(sanitized.Length - 8, BinaryPrimitives.ReadInt32LittleEndian(sanitized.AsSpan(4, 4))); Assert.Equal(sanitized.Length - 8, BinaryPrimitives.ReadInt32LittleEndian(sanitized.AsSpan(4, 4)));
@@ -30,7 +30,21 @@ public class RiffWaveSanitizerTests
{ {
byte[] source = Wave(("fmt ", new byte[] { 1, 0 }), ("data", new byte[] { 1, 2 })); byte[] source = Wave(("fmt ", new byte[] { 1, 0 }), ("data", new byte[] { 1, 2 }));
Assert.Same(source, RiffWaveSanitizer.RemoveInfoMetadata(source)); Assert.Same(source, RiffWaveSanitizer.PrepareForGodot(source));
}
[Fact]
public void DropsBytesAfterFirstDeclaredRiffExtent()
{
byte[] first = Wave(("fmt ", new byte[] { 1, 0 }), ("data", new byte[] { 1, 2, 3, 4 }));
byte[] second = Wave(("fmt ", new byte[] { 1, 0 }), ("data", new byte[] { 5, 6 }));
byte[] separator = Encoding.ASCII.GetBytes("Content-Disposition: form-data\r\n\r\n");
byte[] source = [.. first, .. separator, .. second];
byte[] sanitized = RiffWaveSanitizer.PrepareForGodot(source);
Assert.NotSame(source, sanitized);
Assert.Equal(first, sanitized);
} }
[Fact] [Fact]
@@ -39,7 +53,7 @@ public class RiffWaveSanitizerTests
var resources = ResourceMap.Load(); var resources = ResourceMap.Load();
AudioPayload audio = resources.ReadAudio(resources.ResolveSoundEffect(0x28)!); AudioPayload audio = resources.ReadAudio(resources.ResolveSoundEffect(0x28)!);
byte[] sanitized = RiffWaveSanitizer.RemoveInfoMetadata(audio.Bytes); byte[] sanitized = RiffWaveSanitizer.PrepareForGodot(audio.Bytes);
Assert.Equal("E0808.WAV", audio.Name); Assert.Equal("E0808.WAV", audio.Name);
Assert.Equal(688_570, audio.Bytes.Length); Assert.Equal(688_570, audio.Bytes.Length);
@@ -48,6 +62,23 @@ public class RiffWaveSanitizerTests
Assert.Equal(Chunk(audio.Bytes, "data"), Chunk(sanitized, "data")); Assert.Equal(Chunk(audio.Bytes, "data"), Chunk(sanitized, "data"));
} }
[Fact]
public void InstalledFirstBossSoundStopsAtFirstDeclaredRiff()
{
var resources = ResourceMap.Load();
AudioPayload audio = resources.ReadAudio(resources.ResolveSoundEffect(0x125)!);
byte[] sanitized = RiffWaveSanitizer.PrepareForGodot(audio.Bytes);
Assert.Equal("A1215.WAV", audio.Name);
Assert.Equal(323_009, audio.Bytes.Length);
Assert.Equal(157_940, sanitized.Length);
Assert.Equal(157_940,
8 + BinaryPrimitives.ReadInt32LittleEndian(sanitized.AsSpan(4, 4)));
Assert.Equal(157_896, Chunk(sanitized, "data").Length);
Assert.Equal(audio.Bytes.AsSpan(0, sanitized.Length).ToArray(), sanitized);
}
private static byte[] Wave(params (string Id, byte[] Payload)[] chunks) private static byte[] Wave(params (string Id, byte[] Payload)[] chunks)
{ {
int length = 12 + chunks.Sum(chunk => 8 + chunk.Payload.Length + (chunk.Payload.Length & 1)); int length = 12 + chunks.Sum(chunk => 8 + chunk.Payload.Length + (chunk.Payload.Length & 1));

View File

@@ -2065,8 +2065,8 @@ public partial class Main : Godot.Control
_sfx[channel].Stop(); _sfx[channel].Stop();
_sfx[channel].Stream = null; _sfx[channel].Stream = null;
// This is intentionally a Godot-only compatibility boundary. The VFS and engine retain the // This is intentionally a Godot-only compatibility boundary. The VFS and engine retain the
// original WAV bytes; only Godot's UTF-8-assuming INFO parser sees the sanitized copy. // original WAV bytes; Godot sees an AGE-compatible first-RIFF copy without CP932 INFO metadata.
byte[] godotWav = RiffWaveSanitizer.RemoveInfoMetadata(wavBytes); byte[] godotWav = RiffWaveSanitizer.PrepareForGodot(wavBytes);
var stream = AudioStreamWav.LoadFromBuffer(godotWav); var stream = AudioStreamWav.LoadFromBuffer(godotWav);
if (stream == null) { GD.Print($"WAV load failed {assetName}"); return; } if (stream == null) { GD.Print($"WAV load failed {assetName}"); return; }
stream.LoopMode = AudioStreamWav.LoopModeEnum.Disabled; stream.LoopMode = AudioStreamWav.LoopModeEnum.Disabled;
@@ -2325,9 +2325,15 @@ public partial class Main : Godot.Control
new InputEventKey { PhysicalKeycode = Key.Ctrl }, out int ctrlVk) && ctrlVk == 0x11; new InputEventKey { PhysicalKeycode = Key.Ctrl }, out int ctrlVk) && ctrlVk == 0x11;
var selftestResources = new ResourceMap(_catalog, _assetStore); var selftestResources = new ResourceMap(_catalog, _assetStore);
AudioPayload glowSfx = selftestResources.ReadAudio(selftestResources.ResolveSoundEffect(0x28)!); AudioPayload glowSfx = selftestResources.ReadAudio(selftestResources.ResolveSoundEffect(0x28)!);
byte[] glowGodotWav = RiffWaveSanitizer.RemoveInfoMetadata(glowSfx.Bytes); byte[] glowGodotWav = RiffWaveSanitizer.PrepareForGodot(glowSfx.Bytes);
bool cp932WavMetadataOk = glowGodotWav.Length == 688_336 bool cp932WavMetadataOk = glowGodotWav.Length == 688_336
&& AudioStreamWav.LoadFromBuffer(glowGodotWav) != null; && AudioStreamWav.LoadFromBuffer(glowGodotWav) != null;
AudioPayload bossSfx = selftestResources.ReadAudio(
selftestResources.ResolveSoundEffect(0x125)!);
byte[] bossGodotWav = RiffWaveSanitizer.PrepareForGodot(bossSfx.Bytes);
bool firstRiffBoundaryOk = bossSfx.Bytes.Length == 323_009
&& bossGodotWav.Length == 157_940
&& AudioStreamWav.LoadFromBuffer(bossGodotWav) != null;
AudioPayload bgm = selftestResources.ReadAudio(selftestResources.ResolveBgm(5)!); AudioPayload bgm = selftestResources.ReadAudio(selftestResources.ResolveBgm(5)!);
FadeBgm(0, 10.0); FadeBgm(0, 10.0);
bool bgmFadeStarted = _bgmFadeTween?.IsValid() == true; bool bgmFadeStarted = _bgmFadeTween?.IsValid() == true;
@@ -2411,12 +2417,14 @@ public partial class Main : Godot.Control
new Sys4LogicalCanvas(_screenWidth, _screenHeight)); new Sys4LogicalCanvas(_screenWidth, _screenHeight));
textEffectSmoke.QueueFree(); textEffectSmoke.QueueFree();
ok &= launcherOk && sleepMinimumOk && inputTranslationOk && cp932WavMetadataOk ok &= launcherOk && sleepMinimumOk && inputTranslationOk && cp932WavMetadataOk
&& firstRiffBoundaryOk
&& bgmReplacementCancelsFade && bgmOneShotModeOk && bgmLoopModeOk && bgmReplacementCancelsFade && bgmOneShotModeOk && bgmLoopModeOk
&& bgmStopReleaseOk && textEffectModesOk && fontCalibrationOk && bgmStopReleaseOk && textEffectModesOk && fontCalibrationOk
&& logicalCanvasOk; && logicalCanvasOk;
if (ok) GD.Print($"SELFTEST OK: threaded host matches headless ({actual.Count} lines, full handling); " + if (ok) GD.Print($"SELFTEST OK: threaded host matches headless ({actual.Count} lines, full handling); " +
$"debug launcher catalog/UI smoke ({debugEntries.Count} packed scripts); " + $"debug launcher catalog/UI smoke ({debugEntries.Count} packed scripts); " +
$"sleep-min=1ms; native-key-translation=ok; cp932-wav-info=ok; " + $"sleep-min=1ms; native-key-translation=ok; cp932-wav-info=ok; " +
$"first-riff-boundary=ok; " +
$"bgm-fade-replacement=ok; bgm-start-modes-stop=ok; " + $"bgm-fade-replacement=ok; bgm-start-modes-stop=ok; " +
$"text-effect-modes=ok; font-calibration=ok; " + $"text-effect-modes=ok; font-calibration=ok; " +
$"logical-canvas={_screenWidth}x{_screenHeight}; " + $"logical-canvas={_screenWidth}x{_screenHeight}; " +
@@ -2424,6 +2432,7 @@ public partial class Main : Godot.Control
else GD.Print($"SELFTEST FAIL: threaded={actual.Count} vs headless={expected.Count}; " + else GD.Print($"SELFTEST FAIL: threaded={actual.Count} vs headless={expected.Count}; " +
$"debug-launcher={launcherOk}; sleep-min={sleepMinimumOk}; " + $"debug-launcher={launcherOk}; sleep-min={sleepMinimumOk}; " +
$"native-key-translation={inputTranslationOk}; cp932-wav-info={cp932WavMetadataOk}; " + $"native-key-translation={inputTranslationOk}; cp932-wav-info={cp932WavMetadataOk}; " +
$"first-riff-boundary={firstRiffBoundaryOk}; " +
$"bgm-fade-replacement={bgmReplacementCancelsFade}; " + $"bgm-fade-replacement={bgmReplacementCancelsFade}; " +
$"bgm-one-shot={bgmOneShotModeOk}; bgm-loop={bgmLoopModeOk}; " + $"bgm-one-shot={bgmOneShotModeOk}; bgm-loop={bgmLoopModeOk}; " +
$"bgm-stop-release={bgmStopReleaseOk}; " + $"bgm-stop-release={bgmStopReleaseOk}; " +

View File

@@ -1,12 +1,13 @@
using System; using System;
using System.Buffers.Binary; using System.Buffers.Binary;
/// <summary>Godot-specific WAV input adapter. Godot assumes RIFF INFO strings are UTF-8, while AGE's /// <summary>Godot-specific WAV input adapter. Native AGE decodes only the first declared RIFF/WAVE
/// Japanese assets commonly store them as CP932. Playback does not consume these tags, so remove only /// extent, while Godot walks the entire supplied buffer. AGE's Japanese assets also commonly store
/// INFO metadata while preserving every functional RIFF chunk byte-for-byte.</summary> /// CP932 strings in INFO metadata that Godot assumes is UTF-8. Prepare a transient decoder copy that
/// follows AGE's first-RIFF boundary and removes only INFO metadata within it.</summary>
internal static class RiffWaveSanitizer internal static class RiffWaveSanitizer
{ {
public static byte[] RemoveInfoMetadata(byte[] wavBytes) public static byte[] PrepareForGodot(byte[] wavBytes)
{ {
ReadOnlySpan<byte> input = wavBytes; ReadOnlySpan<byte> input = wavBytes;
if (input.Length < 12 || !HasId(input, 0, "RIFF") || !HasId(input, 8, "WAVE")) if (input.Length < 12 || !HasId(input, 0, "RIFF") || !HasId(input, 8, "WAVE"))
@@ -26,9 +27,10 @@ internal static class RiffWaveSanitizer
if (isInfoList) removedBytes = checked(removedBytes + chunkBytes); if (isInfoList) removedBytes = checked(removedBytes + chunkBytes);
cursor += chunkBytes; cursor += chunkBytes;
} }
if (removedBytes == 0) return wavBytes; if (removedBytes == 0 && declaredEnd == wavBytes.Length) return wavBytes;
var output = new byte[checked(wavBytes.Length - removedBytes)]; int newDeclaredEnd = checked(declaredEnd - removedBytes);
var output = new byte[newDeclaredEnd];
input[..12].CopyTo(output); input[..12].CopyTo(output);
cursor = 12; cursor = 12;
int destination = 12; int destination = 12;
@@ -43,8 +45,6 @@ internal static class RiffWaveSanitizer
cursor += chunkBytes; cursor += chunkBytes;
} }
int newDeclaredEnd = declaredEnd - removedBytes;
input[declaredEnd..].CopyTo(output.AsSpan(newDeclaredEnd));
BinaryPrimitives.WriteUInt32LittleEndian(output.AsSpan(4, 4), BinaryPrimitives.WriteUInt32LittleEndian(output.AsSpan(4, 4),
checked((uint)(newDeclaredEnd - 8))); checked((uint)(newDeclaredEnd - 8)));
return output; return output;