From fdc7bb2bbce9d7b9b5e3ddb2d8b5980146de24de Mon Sep 17 00:00:00 2001 From: gamer147 Date: Tue, 28 Jul 2026 12:16:16 -0400 Subject: [PATCH] Fix MPEG movie startup cadence --- docs/asset-resolution-re.md | 4 + docs/engine-re.md | 51 ++++++++- docs/opcode-reference.md | 10 +- docs/platform-portability.md | 22 ++++ engine/Age.Engine.Tests/MovieOpcodeTests.cs | 111 +++++++++++++++++++- engine/Age.Engine/Vm/VirtualMachine.cs | 13 +-- godot/FfmpegMovieDecoder.cs | 65 ++++++++++-- godot/IMovieDecoder.cs | 1 + godot/Main.cs | 14 ++- vm-map/opcodes.toml | 12 +-- 10 files changed, 269 insertions(+), 34 deletions(-) diff --git a/docs/asset-resolution-re.md b/docs/asset-resolution-re.md index aeed339..d5fa115 100644 --- a/docs/asset-resolution-re.md +++ b/docs/asset-resolution-re.md @@ -434,6 +434,10 @@ depend on the movie service itself to park until EOF/input cancellation before t signature gate. `IHost.PlayModalMovieToSurface` is distinct from the non-modal call for lifecycle only: Godot reuses the asynchronous FFmpeg decoder and retained compositor but parks the VM thread until EOF or mouse/Accept/Cancel input. The wrapper's following release then tears down the completed/cancelled movie. +Native attaches the movie renderer to the already-created target without replacing that surface's resource +identity or color key. The port now does the same: `MovieSurfaceRegistry` resolves the live instance pixels while +the VM retains the mutable surface's `resourceId=0, colorKey=-1`. The former synthetic `colorKey=0` binding made +exact-black MPEG pixels transparent and was not part of the native asset-resolution contract. The FFmpeg backend now renders embedded audio through an engine-owned synchronized path rather than an unmanaged default-device side path. Timestamped stereo float PCM feeds a per-playback Godot `AudioStreamGenerator`; the sound-hardware position drives video presentation, and modal cancellation or diff --git a/docs/engine-re.md b/docs/engine-re.md index f18ce33..a571a0b 100644 --- a/docs/engine-re.md +++ b/docs/engine-re.md @@ -366,6 +366,38 @@ catalog opener as `0x20f`. Both paths now decode the MPEG audio pin through FFmp float PCM. Godot owns one `AudioStreamGenerator` per playback instance, applies the native flag-selected mute/music/SE/voice or default movie route, and uses the sound-hardware position as the presentation clock. +**Native first-frame preroll and stream origin (2026-07-28):** AGE does not seek or manually decode an +opening frame. `movie_to_texture_open_asset_graph@0x463e20` finishes with the DirectShow graph stopped. +`movie_start_modal_playback@0x463280` then calls `IMediaControl::Run` at vtable slot `+0x1c`; there is no +preceding `Pause` or `IMediaPosition` mutation in `0x20f`. DirectShow's stopped-to-running transition +necessarily passes through paused/preroll. For a file source, the graph manager waits for each renderer to +receive a sample; a video renderer displays that held first sample as a poster image before the graph enters +running state. See Microsoft's [Filter States](https://learn.microsoft.com/en-us/windows/win32/directshow/filter-states) +contract. + +The custom `movie_texture_renderer_receive_sample@0x4628d0` copies that preroll sample into the D3D texture +and sets renderer `+0x5b4`. `movie_consume_renderer_new_frame_flag@0x405120` clears the flag and makes the +outer engine tick render the retained surface. Audio is cued during the same graph transition but begins only +with the running reference clock. Native observation adds the decisive cadence constraint: OP's image advances +immediately with its audio; it does not hold the preroll image for roughly 600 ms. AGE contains no later +timestamp correction, so the DirectShow splitter/graph is necessarily presenting the video stream relative to +its first sample rather than exposing FFmpeg's shared program-stream timestamp origin. This last mapping is an +inference about DirectShow internals, but the visible native behavior is the parity oracle. + +The installed FFmpeg probe rules out dropped or transparent source frames. LOGO and OP first report video PTS +600/601 ms on the shim's shared mux timeline, then deliver every tested opening frame at consecutive 33/34 ms +steps; pixels change within the first 20 frames. Every decoded alpha byte in those frames is 255. MPEG-1 has no +alpha plane. Native `movie_texture_renderer_set_media_type@0x463750` accepts RGB24/RGB32, and the RGB24 copy +path promotes pixels to alpha `0xff` before comparing the complete packed ARGB value with its renderer key. +Opcode `0x20f` attaches that renderer to the existing D3D target and never reloads the retained surface or +assigns it an RGB color key. The former port did both `Gfx.SetSurface(..., colorKey: 0)` and RGB-only keying, +which incorrectly made exact-black movie texels transparent and explained grey-background MPEG speckles. + +The corrected port publishes frame zero before starting audio, then schedules all video frames from +`source_video_pts - first_video_pts`; audio retains its own complete stream from timestamp zero. No video or +audio samples are discarded. Signal-bearing PCM in OP's first 600 ms proves only that the audio data is real, +not that the MPEG mux's cross-stream start offset is an intended 600 ms visible lead. + An existing native operand trace identifies every observed heap codebase by a 100% match against its static instruction-offset set. The captured New Game route is: @@ -1293,18 +1325,22 @@ their later lifecycle/presentation slices. ### Movie-to-surface opcode `0x236` (2026-07-11) -The exact ABI is `play-movie-to-surface(resource_id, surface_slot, movie_flags, sync_mask)`. Handler +The exact ABI is `play-movie-to-surface(resource_id, surface_slot, movie_flags, start_delay_ms)`. Handler `op_0x236_play_movie_to_surface@0x423ee0` records a 9-dword instruction length and requires the destination texture to exist. It allocates/reuses a 0x478-byte `CMovieToTexture` object, binds the D3D device/backing texture, opens -operand 1 through `asset_open_indexed_entry`, constructs a DirectShow FilterGraph, and starts it. The graph +operand 1 through `asset_open_indexed_entry`, constructs a stopped DirectShow FilterGraph, and configures its +pending start. The graph queries `IGraphBuilder`, `IMediaControl`, `IMediaPosition`, `IMediaEvent`, and `IBasicAudio`; its custom `CMovieTextureRenderer` accepts RGB samples and copies the bottom-up frame into the retained texture. Operand 3 is retained movie mode plus sound-route policy. Bits `0x10000/0x20000/0x40000/0x80000` force mute/music/SE/voice routing; without an override, native setting `set:DependMovieSound` supplies the normal -movie route. The port applies those categories through Godot audio buses. Operand 4 is stored at movie object `+0x42c` as -the sync/device mask; it is not a duration or loop count. Replacing or releasing the owning surface stops -the graph and detaches the renderer. +movie route. The port applies those categories through Godot audio buses. Operand 4 is stored at movie object +`+0x42c` as a millisecond start delay: `movie_start_pending_after_sync_delay` records `timeGetTime()` on the +first service tick and starts the graph once `current - start >= operand4`. It is not a device mask or movie +duration. Replacing or releasing the owning surface stops the graph and detaches the renderer. The port still +threads this value through a legacy `syncMask`-named host parameter but does not yet delay FFmpeg start for +nonzero values. The corpus has 25 sites: 24 pass literal zero, while BTL supplies a dynamic local-pointer value. Graph construction/open is synchronous, but playback and sample delivery are asynchronous. The handler returns normally and the interpreter advances one instruction: at SC0000 `0x13c8`, the native operand @@ -1314,6 +1350,11 @@ not itself block the VM. SC0000 prepares additional static layers, then reaches service continues sampling the retained movie until DirectShow EOF, after which the following script cleanup releases it. The static preparation before `0x21c` is not a movie teardown boundary. +Non-modal start is deferred through `movie_play_configure@0x4625e0`: the outer tick's +`movie_start_pending_after_sync_delay@0x4633b0` records the first service time and calls the same shared +`IMediaControl::Run` worker once operand 4's sync delay has elapsed. A zero mask starts on the first service +tick. It therefore receives the same stopped-to-paused preroll and first-frame poster behavior as modal `0x20f`. + **Manual-test corrections (2026-07-11):** the initial port incorrectly treated pre-yield static loads as surface replacement, producing start/first-frame/stop all in render frame 0. The bounded host now retains the movie through `0x21c` until DirectShow completion. A follow-up compositor trace proved the movie object diff --git a/docs/opcode-reference.md b/docs/opcode-reference.md index 9f1afb7..e37ff94 100644 --- a/docs/opcode-reference.md +++ b/docs/opcode-reference.md @@ -744,9 +744,9 @@ This is a target-pixel operation, not retained-object teardown. It invokes IDire ### 0x20f `play-modal-movie-to-surface` (play-modal-movie-to-surface, argc 3) - **summary:** (packed_resource_id)(surface_slot)(movie_flags) - open a universal packed SYS4INI/AAI MPEG asset into an existing retained surface, start its native movie graph, and arm modal run-state bit 0x2000 so script execution remains parked until the movie completes or is skipped. This is the LOGO/OP/ED whole-movie path; op 0x236 uses the same resolver with non-modal lifecycle. - **grounding:** source=investigation, confidence=high -- **evidence:** Ghidra /v2: op_0x20f_play_modal_movie_to_surface@0x422e50 shares the movie-object allocation, DirectShow graph open, audio-route, and volume setup used by 0x236, then calls movie_start_modal_playback@0x463280, ORs EngineCtx+0xa0ce4 with 0x2000, and marks movie presentation dirty. The main loop and window procedure special-case run-state 0x2000. Corpus has exactly three sites: LOGO (0x335f,42,4), OP (0x3364,42,4), and ED (0x3324,42,dynamic flags). Those ids are universal raw SYS4INI indexes for MPEG-pack LOGO.AGF, OP.AGF, and ED.AGF; each script releases its surface only after 0x20f resumes. +- **evidence:** Ghidra /v2: op_0x20f_play_modal_movie_to_surface@0x422e50 requires an existing retained D3D target, shares the movie-object allocation, DirectShow graph open, audio-route, and volume setup used by 0x236, then calls movie_start_modal_playback@0x463280, ORs EngineCtx+0xa0ce4 with 0x2000, and marks movie presentation dirty. It does not reload/rebind the gfx surface record or assign an RGB color key. The start worker calls IMediaControl::Run at vtable +0x1c on the freshly stopped graph; AGE performs no explicit Pause, seek, or opening-frame decode. DirectShow's stopped-to-paused-to-running transition cues the first sample. movie_texture_renderer_receive_sample@0x4628d0 copies it to the D3D texture and sets renderer +0x5b4; movie_consume_renderer_new_frame_flag@0x405120 makes the outer tick render it. Native OP then advances video immediately with audio, so FFmpeg's roughly 600 ms shared-mux video origin is not an AGE-visible hold. Corpus has exactly three sites: LOGO (0x335f,42,4), OP (0x3364,42,4), and ED (0x3324,42,dynamic flags). -Implemented through IHost.PlayModalMovieToSurface. Its operand uses the same native universal packed-id catalog contract as 0x236; the separate host call exists for modal wait/cancel lifecycle, not a different resolver. ResourceMap.ResolveMovie selects through ResolvePacked and retains MPEG signature validation in ReadMovie. Godot reuses the asynchronous FFmpeg decoder/retained-surface compositor, parks only the VM thread until EOF, and treats mouse click or Accept/Cancel input as completion before wrapper cleanup releases both video and per-playback audio output. Audio-bearing movies deliver timestamped stereo float PCM through AudioStreamGenerator and use the sound-hardware position as the video/master clock; video-only movies retain monotonic-clock pacing. +Implemented through IHost.PlayModalMovieToSurface. Its operand uses the same native universal packed-id catalog contract as 0x236; the separate host call exists for modal wait/cancel lifecycle, not a different resolver. ResourceMap.ResolveMovie selects through ResolvePacked and retains MPEG signature validation in ReadMovie. Godot reuses the asynchronous FFmpeg decoder/retained-surface compositor, parks only the VM thread until EOF, and treats mouse click or Accept/Cancel input as completion before wrapper cleanup releases both video and per-playback audio output. Movie playback leaves the existing mutable target at resourceId 0 with no RGB color key; MovieSurfaceRegistry supplies its live pixels. Audio-bearing movies deliver timestamped stereo float PCM through AudioStreamGenerator and use the sound-hardware position as the master clock. The first decoded image is latched before audio starts, and all video deadlines are rebased to the first video PTS while the complete audio stream begins at its own timestamp zero. No opening samples are discarded. ### 0x212 `set-gfx-field64` (set-gfx-field64, argc 2) - **summary:** 0x212 (obj_idx)(val) — handler gfx_op_0x212_set_field64 @0x4230c0: obj=[ctx+0x14d54 + obj_idx*4]; if obj: *(obj+0x64)=val. The generic instruction length is 5 dwords. See docs/engine-re.md gfx op-contract table. @@ -872,11 +872,11 @@ Implemented through IHost.PlayModalMovieToSurface. Its operand uses the same nat - **evidence:** Ghidra handler 0x423da0 converts axis ints to floats -> worker 0x47f060. gfx_object_anim_interpolate@0x473ed0 consumes obj+0x228/+0x214/+0x244 on retained-gfx owner+0xb550 (EngineCtx+0x51b64) and matrix4_make_axis_angle@0x48b215. gfx_object_composite@0x47f650 calls one-shot transform first, cyclic animation second. ### 0x236 `play-movie-to-surface` (play-movie-to-surface, argc 4) -- **summary:** (packed_resource_id)(surface_slot)(movie_flags)(sync_mask) - synchronously open a universal packed SYS4INI/AAI movie and construct its DirectShow graph, then start asynchronous frame delivery into the retained destination surface. The opcode itself is non-blocking: the VM advances to the next instruction. SC0000's 0x33 and BTL's 0x2axx/0x2bxx MVB ids are already absolute base-catalog indexes. +- **summary:** (packed_resource_id)(surface_slot)(movie_flags)(start_delay_ms) - synchronously open a universal packed SYS4INI/AAI movie and construct its DirectShow graph, then start asynchronous frame delivery into the retained destination surface after the requested service-tick delay. The opcode itself is non-blocking: the VM advances to the next instruction. SC0000's 0x33 and BTL's 0x2axx/0x2bxx MVB ids are already absolute base-catalog indexes. - **grounding:** source=investigation, confidence=high -- **evidence:** Ghidra /v2 op_0x236_play_movie_to_surface@0x423ee0 fetches operand 1 and passes it unchanged to movie_to_texture_open_asset_graph@0x463e20, which passes it unchanged to asset_open_indexed_entry@0x44f390. The opener directly indexes the flat base table or selected AAI table and has no scene input. SC0000 native operand capture and exact 0x13c8->0x13d1 trace prove nonblocking behavior. BTL's live 0x2b21 site supplies 0x2af1/0x2af5/0x2bca/0x2bd8/0x2bde, the exact base entries MVB001/MVB004/MVB914/MVB958/MVB955. +- **evidence:** Ghidra /v2 op_0x236_play_movie_to_surface@0x423ee0 fetches operand 1 and passes it unchanged to movie_to_texture_open_asset_graph@0x463e20, which passes it unchanged to asset_open_indexed_entry@0x44f390. The opener directly indexes the flat base table or selected AAI table and has no scene input. The handler leaves playback pending through movie_play_configure@0x4625e0; movie_start_pending_after_sync_delay@0x4633b0 records the first service tick and calls the shared IMediaControl::Run worker after operand 4's delay, giving non-modal playback the same DirectShow first-sample preroll as 0x20f. SC0000 native operand capture and exact 0x13c8->0x13d1 trace prove nonblocking behavior. BTL's live 0x2b21 site supplies 0x2af1/0x2af5/0x2bca/0x2bd8/0x2bde, the exact base entries MVB001/MVB004/MVB914/MVB958/MVB955. -The handler requires an existing destination texture, allocates/reuses a 0x478-byte movie-to-texture object for the surface, opens operand 1 through the native universal packed-id reader, builds FilterGraph/IGraphBuilder/IMediaControl/IMediaPosition/IMediaEvent/IBasicAudio, and presents bottom-up RGB samples through the movie texture renderer. Operand 3 selects movie/sound routing policy: bits 0x10000/0x20000/0x40000/0x80000 force mute/music/SE/voice routes, otherwise set:DependMovieSound selects the normal movie route; SC0000's low value 2 is retained as native movie mode state. Godot maps those categories to audio buses and sends FFmpeg-decoded timestamped stereo float PCM through a per-playback AudioStreamGenerator. Audio-bearing movies use the sound-hardware position as the master presentation clock; video-only movies retain monotonic pacing. Operand 4 is the movie sync/device mask. Static layer preparation after 0x236 does not terminate the retained movie; 0x21c services it through EOF and subsequent surface cleanup stops/detaches both decoder and audio output. The port type-checks the selected ResolvePacked record as MPEG without adding a scene base or fallback. +The handler requires an existing destination texture, allocates/reuses a 0x478-byte movie-to-texture object for the surface without replacing its resource identity/color key, opens operand 1 through the native universal packed-id reader, builds FilterGraph/IGraphBuilder/IMediaControl/IMediaPosition/IMediaEvent/IBasicAudio, and presents bottom-up RGB samples through the movie texture renderer. Operand 3 selects movie/sound routing policy: bits 0x10000/0x20000/0x40000/0x80000 force mute/music/SE/voice routes, otherwise set:DependMovieSound selects the normal movie route; SC0000's low value 2 is retained as native movie mode state. Godot maps those categories to audio buses and sends FFmpeg-decoded timestamped stereo float PCM through a per-playback AudioStreamGenerator. The first image is latched before audio start; video cadence is rebased to its first decoded PTS for both audio-bearing and video-only streams, while audio retains every sample from its own timestamp zero. Operand 4 delays the pending graph start in native service ticks. Static layer preparation after 0x236 does not terminate the retained movie; 0x21c services it through EOF and subsequent surface cleanup stops/detaches both decoder and audio output. The port type-checks the selected ResolvePacked record as MPEG without adding a scene base or fallback. ### 0x238 `set-anim-clock` (set-anim-clock, argc 1) - **summary:** (duration) — configure the separate global finite-animation service window: native ctx+0x51b78=0 (start), +0x51b7c=duration. The generic instruction length is 3 dwords. NON-BLOCKING: gfx_animation_service_poll seeds the start from the shared frame timestamp and keeps redraw active through the duration. This is not the time source or cadence for cyclic object channels, which sample ctx+0x51b64/+0x51b68 with object-local starts and periods. SC0000 opening @0x123bd/@0x13858. Handler 0x4240e0; Kelebek VA 0x422390 is drift. diff --git a/docs/platform-portability.md b/docs/platform-portability.md index aeedd64..8dd0580 100644 --- a/docs/platform-portability.md +++ b/docs/platform-portability.md @@ -133,6 +133,28 @@ the first timestamp and material later gaps/overlaps. A 4,093-block OP regressio The clean audible recheck passed for LOGO/OP and CHAPTER. DirectShow and the managed Windows platform annotations were then deleted; missing, distorted, or unsynchronized audio on any of the 29 audio-bearing assets is now a bug. +Movie startup was corrected on 2026-07-28 after a first-frame latch exposed a second, visible 600 ms freeze. +`LOGO.AGF` and `OP.AGF` begin audio at 0 ms on the shim's shared mux timeline, while their first decoded video +frames carry PTS 600 ms and 601 ms. Signal-bearing PCM in OP's first 600 ms initially looked like an intentional +audio lead, but native OP advances video and audio immediately. A real-asset regression proves the decoder is not +losing opening data: the first 20 LOGO/OP video frames are consecutive 33/34 ms samples, change within that window, +and contain alpha 255 at every pixel. The mux offset is therefore not a presentation deadline. + +The paced decoder still latches frame zero and Godot still waits to start its `AudioStreamGenerator` until that +image has been published. It now schedules every later video frame from +`source_video_pts - first_video_pts`, including audio-bearing movies, while submitting the complete audio stream +from its own timestamp zero. This preserves every decoded sample and immediately advances frame cadence instead +of freezing frame zero until the audio clock reaches the program-stream offset. Startup logs and F6 snapshots +retain the raw first video PTS for diagnosis. + +A separate compositor error amplified the symptom. MPEG-1 has no transparency and FFmpeg's RGBA conversion +produces opaque pixels, but the VM had replaced the existing mutable movie target with an invented RGB color key +of zero. Exact black became transparent and exposed the Godot clear color, producing the previously reported grey +MPEG speckles. Both movie opcodes now retain the existing surface's no-key state; the host's instance binding +supplies changing movie pixels without changing surface resource identity. Native `0x20f` likewise attaches its +DirectShow renderer to the existing D3D target without assigning an RGB key. The annotated native details and the +DirectShow preroll/stream-origin inference are recorded in `docs/engine-re.md`. + Native deliverables are RID-specific and bundled with the Godot export; the runtime must not discover an arbitrary system FFmpeg. The first implementation gate is Windows x64 because that is the current runnable target, but the C ABI and loader paths must reserve Windows x64, Linux x64, macOS x64, and macOS arm64 from the diff --git a/engine/Age.Engine.Tests/MovieOpcodeTests.cs b/engine/Age.Engine.Tests/MovieOpcodeTests.cs index 84fda99..38cdaf9 100644 --- a/engine/Age.Engine.Tests/MovieOpcodeTests.cs +++ b/engine/Age.Engine.Tests/MovieOpcodeTests.cs @@ -46,11 +46,13 @@ public class MovieOpcodeTests private int _decodeCalls; public FfmpegMovieInfo Info { get; } = info; public int FailOnDecodeCall { get; init; } = -1; + public int DecodeCalls => Volatile.Read(ref _decodeCalls); public bool Disposed { get; private set; } public bool TryDecodeNextVideoFrame(out FfmpegVideoFrame frame) { - if (_decodeCalls++ == FailOnDecodeCall) + int call = Interlocked.Increment(ref _decodeCalls) - 1; + if (call == FailOnDecodeCall) throw new InvalidDataException("synthetic decode failure"); return _frames.TryDequeue(out frame!); } @@ -176,6 +178,65 @@ public class MovieOpcodeTests Assert.Null(decoder.Failure); } + [Fact] + public void FfmpegDecoderPublishesAndRetainsFirstDecodedFrameBeforePacing() + { + var source = new FakeFfmpegFrameSource( + new FfmpegMovieInfo(1, 1, 700, 30, 1, true, 1000, 2, 50), + SyntheticMovieFrame(1, 600), SyntheticMovieFrame(2, 634)); + using var clock = new ManualMoviePacingClock(); + clock.AdvanceTo(1000); + using var decoder = new FfmpegMovieDecoder(source, clock); + + Assert.True(SpinWait.SpinUntil(() => source.DecodeCalls == 1, 1000)); + Thread.Sleep(20); + Assert.Equal(1, source.DecodeCalls); + Assert.Equal(600, decoder.FirstFramePresentationTimeMs); + Assert.True(decoder.TryTakeFrame(out var first)); + Assert.Equal((byte)1, first.Pixels[0]); + + Assert.True(SpinWait.SpinUntil(() => source.DecodeCalls >= 2, 1000)); + Assert.True(SpinWait.SpinUntil(() => decoder.TryTakeFrame(out _), 1000)); + } + + [Fact] + public void FfmpegAudioBearingDecoderRebasesVideoPacingToFirstSourceTimestamp() + { + var source = new FakeFfmpegFrameSource( + new FfmpegMovieInfo(1, 1, 700, 30, 1, true, 1000, 2, 50), + SyntheticMovieFrame(1, 600), SyntheticMovieFrame(2, 634)); + using var clock = new ManualMoviePacingClock(); + using var decoder = new FfmpegMovieDecoder(source, clock); + + Assert.True(SpinWait.SpinUntil(() => decoder.TryTakeFrame(out _), 1000)); + Assert.True(clock.WaitForDeadline(34)); + clock.AdvanceTo(33); + Assert.False(decoder.TryTakeFrame(out _)); + clock.AdvanceTo(34); + Assert.True(SpinWait.SpinUntil(() => decoder.TryTakeFrame(out _), 1000)); + } + + [Fact] + public void FfmpegVideoOnlyDecoderRebasesPacingToFirstSourceTimestamp() + { + var source = new FakeFfmpegFrameSource( + new FfmpegMovieInfo(1, 1, 700, 30, 1, false), + SyntheticMovieFrame(1, 600), SyntheticMovieFrame(2, 634)); + using var clock = new ManualMoviePacingClock(); + using var decoder = new FfmpegMovieDecoder(source, clock); + + Assert.True(SpinWait.SpinUntil(() => decoder.TryTakeFrame(out _), 1000)); + Assert.True(clock.WaitForDeadline(34)); + clock.AdvanceTo(33); + Assert.False(decoder.TryTakeFrame(out _)); + clock.AdvanceTo(34); + Assert.True(SpinWait.SpinUntil(() => decoder.TryTakeFrame(out _), 1000)); + + Assert.True(clock.WaitForDeadline(100)); + clock.AdvanceTo(100); + Assert.True(SpinWait.SpinUntil(() => decoder.IsCompleted, 1000)); + } + [Fact] public void FfmpegDecoderDisposalInterruptsFutureFrameWait() { @@ -184,6 +245,7 @@ public class MovieOpcodeTests SyntheticMovieFrame(1, 0), SyntheticMovieFrame(2, 60000)); using var clock = new ManualMoviePacingClock(); var decoder = new FfmpegMovieDecoder(source, clock); + Assert.True(SpinWait.SpinUntil(() => decoder.TryTakeFrame(out _), 1000)); Assert.True(clock.WaitForDeadline(60000)); var elapsed = Stopwatch.StartNew(); @@ -204,6 +266,7 @@ public class MovieOpcodeTests using var clock = new ManualMoviePacingClock(); using var decoder = new FfmpegMovieDecoder(source, clock); + Assert.True(SpinWait.SpinUntil(() => decoder.TryTakeFrame(out _), 1000)); Assert.True(SpinWait.SpinUntil(() => decoder.IsCompleted, 1000)); Assert.Equal("synthetic decode failure", decoder.Failure); } @@ -229,6 +292,7 @@ public class MovieOpcodeTests Assert.Equal(new long[] { 0, 50 }, chunks.Select(chunk => chunk.PresentationTimeMs)); Assert.Equal(100, chunks.Sum(chunk => chunk.FrameCount)); + Assert.True(SpinWait.SpinUntil(() => decoder.TryTakeFrame(out _), 1000)); decoder.MarkAudioSubmitted(); clock.AdvanceTo(99); Assert.False(decoder.IsCompleted); @@ -323,6 +387,7 @@ public class MovieOpcodeTests var table = OpcodeTableJson.Load(Paths.OpcodesJson); var script = ScriptAssembler.Assemble(table, "MODAL", new List<(int, Operand[])> { + (0x1f8, new[] { new Operand(0, 42), new Operand(0, 800), new Operand(0, 600), new Operand(0, 0) }), (0x20f, new[] { new Operand(0, 0x335f), new Operand(0, 42), new Operand(0, 4) }), (0x55, new[] { new Operand(3, 0x1234), new Operand(0, 0x5678) }), (0x2, System.Array.Empty()), @@ -335,7 +400,9 @@ public class MovieOpcodeTests Assert.Equal(new[] { (0x335fL, 42, 4L) }, host.ModalMovies); Assert.Equal(0x5678, vm.Globals[0x1234]); vm.Gfx.BindDraw(1, 42, 0, 0, 1, 1, 0, 0); - Assert.Equal(0x335f, vm.Gfx.SnapshotVisibleObjects().Single().SurfaceResId); + var visible = vm.Gfx.SnapshotVisibleObjects().Single(); + Assert.Equal(0, visible.SurfaceResId); + Assert.Equal(-1, visible.ColorKey); } [Fact] @@ -382,7 +449,9 @@ public class MovieOpcodeTests Assert.Equal(0x5678, vm.Globals[0x1234]); Assert.Equal(new[] { (0x33L, 5, 2L, 0L) }, host.Movies); vm.Gfx.BindDraw(1, 5, 0, 0, 1, 1, 0, 0); - Assert.Equal(0x33, vm.Gfx.SnapshotVisibleObjects().Single().SurfaceResId); + var visible = vm.Gfx.SnapshotVisibleObjects().Single(); + Assert.Equal(0, visible.SurfaceResId); + Assert.Equal(-1, visible.ColorKey); } [Fact] @@ -409,7 +478,9 @@ public class MovieOpcodeTests vm.Run(); Assert.Equal(0, resourceDuringSetup); - Assert.Equal(0x33, vm.Gfx.SnapshotVisibleObjects().Single().SurfaceResId); + var visible = vm.Gfx.SnapshotVisibleObjects().Single(); + Assert.Equal(0, visible.SurfaceResId); + Assert.Equal(-1, visible.ColorKey); } [Fact] @@ -505,6 +576,38 @@ public class MovieOpcodeTests Assert.Equal(new byte[] { 0, 0, 1, 0xba }, movie.Bytes[..4]); } + [Theory] + [InlineData(0x335f, "LOGO.AGF")] + [InlineData(0x3364, "OP.AGF")] + public void ModalMovieOpeningFramesAreOpaqueAndHaveContinuousCadence( + int resourceId, string expectedName) + { + if (!OperatingSystem.IsWindows()) return; + ConfigureFfmpegNativeProbe(); + var catalog = Sys4AssetCatalog.Load(Paths.Sys4Ini); + var resources = new ResourceMap(catalog, new Sys4AssetStore(catalog, Paths.GameDir)); + var payload = resources.ReadMovie(resources.ResolveMovie(resourceId)!); + + using var movie = new FfmpegMovieSession(payload); + Assert.Equal(expectedName, payload.Name); + long priorTimestamp = -1; + byte[]? firstPixels = null; + bool openingChanged = false; + for (int frameIndex = 0; frameIndex < 20; frameIndex++) + { + Assert.True(movie.TryDecodeNextVideoFrame(out var frame), + $"{expectedName} ended before opening frame {frameIndex}"); + if (priorTimestamp >= 0) + Assert.InRange(frame.PresentationTimeMs - priorTimestamp, 33, 34); + priorTimestamp = frame.PresentationTimeMs; + firstPixels ??= frame.Image.Pixels; + openingChanged |= !firstPixels.AsSpan().SequenceEqual(frame.Image.Pixels); + for (int alpha = 3; alpha < frame.Image.Pixels.Length; alpha += 4) + Assert.Equal((byte)255, frame.Image.Pixels[alpha]); + } + Assert.True(openingChanged, $"{expectedName} should change within its first 20 decoded frames"); + } + [Theory] [InlineData(0x2be3, "MVB961.AGF", 280, 352, 500)] [InlineData(0x2b94, "MVB238.AGF", 280, 352, 866)] diff --git a/engine/Age.Engine/Vm/VirtualMachine.cs b/engine/Age.Engine/Vm/VirtualMachine.cs index 48f14d8..757be95 100644 --- a/engine/Age.Engine/Vm/VirtualMachine.cs +++ b/engine/Age.Engine/Vm/VirtualMachine.cs @@ -2416,13 +2416,15 @@ public sealed class VirtualMachine long resourceId = Read(a[0]); int surfaceSlot = (int)Read(a[1]); // Modal and non-modal paths share packed resolution and retained-surface composition. - // The distinct host entry point owns only 0x20f's blocking lifecycle. - Gfx.SetSurface(surfaceSlot, resourceId, 0); + // The distinct host entry point owns only 0x20f's blocking lifecycle. Native attaches its + // renderer to the existing mutable target; it does not reload that surface with resource + // id/movie bytes or apply color key 0. Keep the created surface's no-key state so MPEG + // black remains opaque. _host.PlayModalMovieToSurface(resourceId, surfaceSlot, Read(a[2])); return pc + 1; } case "u004221A0": // pre-reference compatibility - case "play-movie-to-surface": // 0x236 (resource)(surface)(movie flags)(sync mask) + case "play-movie-to-surface": // 0x236 (resource)(surface)(movie flags)(start delay ms) { long resourceId = Read(a[0]); int surfaceSlot = (int)Read(a[1]); @@ -2442,9 +2444,8 @@ public sealed class VirtualMachine // boundary; publishing the movie resource first would let a concurrent compositor mistake // the MPEG payload's .AGF name for a still image before the host registers/decodes it. long? stopTimeMs = _host.PlayMovieToSurface(resourceId, surfaceSlot, Read(a[2]), Read(a[3])); - // The native CMovieToTexture renderer replaces the pixels of the already-created surface. - // Retain the same resource binding so the compositor resolves live movie frames for its objects. - Gfx.SetSurface(surfaceSlot, resourceId, 0); + // Native replaces the pixels of the already-created surface without changing its resource + // identity or color key. The host's playback-to-surface binding resolves the live frame. // Native never encounters a missing system decoder for shipped assets. If a host backend // cannot initialize one, model the valid movie as completing immediately: BTL feeds this // value into its effect timeline, where zero is a safe duration and -1 is not meaningful. diff --git a/godot/FfmpegMovieDecoder.cs b/godot/FfmpegMovieDecoder.cs index c9bbcae..0f864bc 100644 --- a/godot/FfmpegMovieDecoder.cs +++ b/godot/FfmpegMovieDecoder.cs @@ -7,6 +7,7 @@ using Age.Engine.Sys4; internal interface IMoviePacingClock { + void StartPresentation() { } bool WaitUntil(long elapsedMilliseconds, WaitHandle cancellation); } @@ -17,14 +18,22 @@ internal interface IExternallyAdvancedMoviePacingClock : IMoviePacingClock internal sealed class StopwatchMoviePacingClock : IMoviePacingClock { - private readonly long _startedAt = Stopwatch.GetTimestamp(); + private long _startedAt; + + public void StartPresentation() + { + long startedAt = Stopwatch.GetTimestamp(); + Interlocked.CompareExchange(ref _startedAt, startedAt, 0); + } public bool WaitUntil(long elapsedMilliseconds, WaitHandle cancellation) { + StartPresentation(); while (true) { + long startedAt = Interlocked.Read(ref _startedAt); double remaining = elapsedMilliseconds - - Stopwatch.GetElapsedTime(_startedAt).TotalMilliseconds; + - Stopwatch.GetElapsedTime(startedAt).TotalMilliseconds; if (remaining <= 0) return true; int waitMilliseconds = (int)Math.Clamp(Math.Ceiling(remaining), 1, 1000); if (cancellation.WaitOne(waitMilliseconds)) return false; @@ -71,6 +80,7 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder private readonly IFfmpegFrameSource _source; private readonly IMoviePacingClock _clock; private readonly ManualResetEvent _cancel = new(false); + private readonly ManualResetEvent _firstFramePresented = new(false); private readonly Thread _thread; private readonly Thread? _audioThread; private readonly object _frameLock = new(); @@ -87,10 +97,20 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder private int _queuedAudioFrames; private int _activeWorkers; private int _disposed; + private int _firstFrameTaken; + private long _firstFramePresentationTimeMs = -1; public long? StopTimeMs => _source.Info.StopTimeMs; public bool IsCompleted => _completed; public string? Failure => Volatile.Read(ref _failure); + public long? FirstFramePresentationTimeMs + { + get + { + long value = Interlocked.Read(ref _firstFramePresentationTimeMs); + return value < 0 ? null : value; + } + } public MovieAudioInfo? AudioInfo { get; } public bool AudioDecodingCompleted => _audioDecodingCompleted; @@ -134,6 +154,7 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder if (_audioThread?.IsAlive == true) _audioThread.Join(); _source.Dispose(); _cancel.Dispose(); + _firstFramePresented.Dispose(); _audioSpace.Dispose(); if (_clock is IDisposable disposableClock) disposableClock.Dispose(); throw; @@ -151,8 +172,13 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder } frame = _latestFrame; _latestFrame = null; - return true; } + if (Interlocked.Exchange(ref _firstFrameTaken, 1) == 0) + { + _clock.StartPresentation(); + _firstFramePresented.Set(); + } + return true; } public bool TryTakeAudioChunk(out MovieAudioChunk chunk) @@ -189,14 +215,17 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder { long lastTimestamp = -1; long decodedFrames = 0; + long firstTimestamp = -1; while (!_cancel.WaitOne(0)) { if (!_source.TryDecodeNextVideoFrame(out var frame)) { if (decodedFrames == 0) throw new InvalidDataException("FFmpeg stream ended before producing a video frame"); - long completionTime = Math.Max(_source.Info.StopTimeMs, - lastTimestamp + FrameIntervalMilliseconds(_source.Info)); + long completionTime = PresentationDeadline( + Math.Max(_source.Info.StopTimeMs, + lastTimestamp + FrameIntervalMilliseconds(_source.Info)), + firstTimestamp); if (_clock.WaitUntil(completionTime, _cancel)) { _videoTimelineCompleted = true; @@ -207,7 +236,20 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder if (frame.PresentationTimeMs < 0 || frame.PresentationTimeMs < lastTimestamp) throw new InvalidDataException( $"FFmpeg returned non-monotonic video timestamp {frame.PresentationTimeMs} after {lastTimestamp}"); - if (!_clock.WaitUntil(frame.PresentationTimeMs, _cancel)) return; + if (decodedFrames == 0) + { + firstTimestamp = frame.PresentationTimeMs; + Interlocked.Exchange(ref _firstFramePresentationTimeMs, firstTimestamp); + lock (_frameLock) _latestFrame = frame.Image; + lastTimestamp = frame.PresentationTimeMs; + decodedFrames++; + int signalled = WaitHandle.WaitAny([_cancel, _firstFramePresented]); + if (signalled == 0) return; + continue; + } + if (!_clock.WaitUntil(PresentationDeadline(frame.PresentationTimeMs, firstTimestamp), + _cancel)) + return; lock (_frameLock) _latestFrame = frame.Image; lastTimestamp = frame.PresentationTimeMs; decodedFrames++; @@ -295,16 +337,27 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder return Math.Max(1, scaledDenominator / info.FrameRateNumerator); } + private long PresentationDeadline(long sourceTimestamp, long firstVideoTimestamp) + { + // MPEG program streams may put the first video sample hundreds of milliseconds after the audio + // stream's mux timestamp origin. DirectShow presents the video's first sample as video time zero; + // retaining the absolute mux offset here would freeze that sample until the audio clock caught up. + // Preserve every decoded frame and its cadence, but rebase the video stream to its first sample. + return Math.Max(0, sourceTimestamp - firstVideoTimestamp); + } + public void Dispose() { if (Interlocked.Exchange(ref _disposed, 1) != 0) return; _cancel.Set(); + _firstFramePresented.Set(); _audioSpace.Set(); if (_thread.IsAlive && Thread.CurrentThread != _thread) _thread.Join(); if (_audioThread?.IsAlive == true && Thread.CurrentThread != _audioThread) _audioThread.Join(); _cancel.Dispose(); + _firstFramePresented.Dispose(); _audioSpace.Dispose(); if (_clock is IDisposable disposableClock) disposableClock.Dispose(); } diff --git a/godot/IMovieDecoder.cs b/godot/IMovieDecoder.cs index aa3792b..65fe357 100644 --- a/godot/IMovieDecoder.cs +++ b/godot/IMovieDecoder.cs @@ -13,6 +13,7 @@ internal interface IMovieDecoder : IDisposable long? StopTimeMs { get; } bool IsCompleted { get; } string? Failure { get; } + long? FirstFramePresentationTimeMs => null; bool TryTakeFrame(out RgbaImage frame); MovieAudioInfo? AudioInfo => null; bool AudioDecodingCompleted => true; diff --git a/godot/Main.cs b/godot/Main.cs index 0162183..24a5acf 100644 --- a/godot/Main.cs +++ b/godot/Main.cs @@ -624,6 +624,7 @@ public partial class Main : Godot.Control stop_time_ms = pair.Value.Decoder.StopTimeMs, decoder_completed = pair.Value.Decoder.IsCompleted, decoder_failure = pair.Value.Decoder.Failure, + first_frame_source_pts_ms = pair.Value.Decoder.FirstFramePresentationTimeMs, frame_seen = _movieFrameSeen.Contains(pair.Key), completion_notified = _movieCompletionNotified.Contains(pair.Key), watchdog_ms = pair.Value.WatchdogMs, @@ -649,6 +650,7 @@ public partial class Main : Godot.Control stop_time_ms = pair.Value.Decoder.StopTimeMs, decoder_completed = pair.Value.Decoder.IsCompleted, decoder_failure = pair.Value.Decoder.Failure, + first_frame_source_pts_ms = pair.Value.Decoder.FirstFramePresentationTimeMs, }) .ToArray(); var snapshot = new @@ -1804,13 +1806,21 @@ public partial class Main : Godot.Control if (_host == null) return; foreach (var (playbackId, movie) in _movies) { - if (_movieAudio.TryGetValue(playbackId, out var audio)) audio.Update(); + bool frameWasAlreadySeen = _movieFrameSeen.Contains(playbackId); + _movieAudio.TryGetValue(playbackId, out var audio); + if (frameWasAlreadySeen) audio?.Update(); if (movie.Decoder.TryTakeFrame(out var frame)) { _host.PublishMovieFrame(playbackId, movie.Name, movie.AssetId, frame); if (_movieFrameSeen.Add(playbackId)) + { GD.Print($"movie first frame {movie.Name} playback={playbackId}: " + - $"{frame.Width}x{frame.Height} RGBA8 at render frame {_timelineFrame}"); + $"{frame.Width}x{frame.Height} RGBA8 at render frame {_timelineFrame} " + + $"(source PTS {movie.Decoder.FirstFramePresentationTimeMs?.ToString() ?? "unknown"} ms)"); + // Do not let decode startup consume the opening audio timeline. The first PCM push + // begins only after the first decoded image has reached the retained movie surface. + audio?.Update(); + } } bool watchdogExpired = Stopwatch.GetElapsedTime(movie.StartedAtTimestamp).TotalMilliseconds >= movie.WatchdogMs; diff --git a/vm-map/opcodes.toml b/vm-map/opcodes.toml index c0c7bc2..16a3879 100644 --- a/vm-map/opcodes.toml +++ b/vm-map/opcodes.toml @@ -5255,8 +5255,8 @@ noop_headless = false source = "investigation" confidence = "high" depends_on = [] -evidence = "Ghidra /v2: op_0x20f_play_modal_movie_to_surface@0x422e50 shares the movie-object allocation, DirectShow graph open, audio-route, and volume setup used by 0x236, then calls movie_start_modal_playback@0x463280, ORs EngineCtx+0xa0ce4 with 0x2000, and marks movie presentation dirty. The main loop and window procedure special-case run-state 0x2000. Corpus has exactly three sites: LOGO (0x335f,42,4), OP (0x3364,42,4), and ED (0x3324,42,dynamic flags). Those ids are universal raw SYS4INI indexes for MPEG-pack LOGO.AGF, OP.AGF, and ED.AGF; each script releases its surface only after 0x20f resumes." -details = "Implemented through IHost.PlayModalMovieToSurface. Its operand uses the same native universal packed-id catalog contract as 0x236; the separate host call exists for modal wait/cancel lifecycle, not a different resolver. ResourceMap.ResolveMovie selects through ResolvePacked and retains MPEG signature validation in ReadMovie. Godot reuses the asynchronous FFmpeg decoder/retained-surface compositor, parks only the VM thread until EOF, and treats mouse click or Accept/Cancel input as completion before wrapper cleanup releases both video and per-playback audio output. Audio-bearing movies deliver timestamped stereo float PCM through AudioStreamGenerator and use the sound-hardware position as the video/master clock; video-only movies retain monotonic-clock pacing." +evidence = "Ghidra /v2: op_0x20f_play_modal_movie_to_surface@0x422e50 requires an existing retained D3D target, shares the movie-object allocation, DirectShow graph open, audio-route, and volume setup used by 0x236, then calls movie_start_modal_playback@0x463280, ORs EngineCtx+0xa0ce4 with 0x2000, and marks movie presentation dirty. It does not reload/rebind the gfx surface record or assign an RGB color key. The start worker calls IMediaControl::Run at vtable +0x1c on the freshly stopped graph; AGE performs no explicit Pause, seek, or opening-frame decode. DirectShow's stopped-to-paused-to-running transition cues the first sample. movie_texture_renderer_receive_sample@0x4628d0 copies it to the D3D texture and sets renderer +0x5b4; movie_consume_renderer_new_frame_flag@0x405120 makes the outer tick render it. Native OP then advances video immediately with audio, so FFmpeg's roughly 600 ms shared-mux video origin is not an AGE-visible hold. Corpus has exactly three sites: LOGO (0x335f,42,4), OP (0x3364,42,4), and ED (0x3324,42,dynamic flags)." +details = "Implemented through IHost.PlayModalMovieToSurface. Its operand uses the same native universal packed-id catalog contract as 0x236; the separate host call exists for modal wait/cancel lifecycle, not a different resolver. ResourceMap.ResolveMovie selects through ResolvePacked and retains MPEG signature validation in ReadMovie. Godot reuses the asynchronous FFmpeg decoder/retained-surface compositor, parks only the VM thread until EOF, and treats mouse click or Accept/Cancel input as completion before wrapper cleanup releases both video and per-playback audio output. Movie playback leaves the existing mutable target at resourceId 0 with no RGB color key; MovieSurfaceRegistry supplies its live pixels. Audio-bearing movies deliver timestamped stereo float PCM through AudioStreamGenerator and use the sound-hardware position as the master clock. The first decoded image is latched before audio starts, and all video deadlines are rebased to the first video PTS while the complete audio stream begins at its own timestamp zero. No opening samples are discarded." [[opcode.semantics.args]] i = 1 @@ -6241,13 +6241,13 @@ abi_source = "kelebek+decode-validated" [opcode.semantics] name = "play-movie-to-surface" category = "draw" -summary = "(packed_resource_id)(surface_slot)(movie_flags)(sync_mask) - synchronously open a universal packed SYS4INI/AAI movie and construct its DirectShow graph, then start asynchronous frame delivery into the retained destination surface. The opcode itself is non-blocking: the VM advances to the next instruction. SC0000's 0x33 and BTL's 0x2axx/0x2bxx MVB ids are already absolute base-catalog indexes." +summary = "(packed_resource_id)(surface_slot)(movie_flags)(start_delay_ms) - synchronously open a universal packed SYS4INI/AAI movie and construct its DirectShow graph, then start asynchronous frame delivery into the retained destination surface after the requested service-tick delay. The opcode itself is non-blocking: the VM advances to the next instruction. SC0000's 0x33 and BTL's 0x2axx/0x2bxx MVB ids are already absolute base-catalog indexes." noop_headless = false source = "investigation" confidence = "high" depends_on = [] -evidence = "Ghidra /v2 op_0x236_play_movie_to_surface@0x423ee0 fetches operand 1 and passes it unchanged to movie_to_texture_open_asset_graph@0x463e20, which passes it unchanged to asset_open_indexed_entry@0x44f390. The opener directly indexes the flat base table or selected AAI table and has no scene input. SC0000 native operand capture and exact 0x13c8->0x13d1 trace prove nonblocking behavior. BTL's live 0x2b21 site supplies 0x2af1/0x2af5/0x2bca/0x2bd8/0x2bde, the exact base entries MVB001/MVB004/MVB914/MVB958/MVB955." -details = "The handler requires an existing destination texture, allocates/reuses a 0x478-byte movie-to-texture object for the surface, opens operand 1 through the native universal packed-id reader, builds FilterGraph/IGraphBuilder/IMediaControl/IMediaPosition/IMediaEvent/IBasicAudio, and presents bottom-up RGB samples through the movie texture renderer. Operand 3 selects movie/sound routing policy: bits 0x10000/0x20000/0x40000/0x80000 force mute/music/SE/voice routes, otherwise set:DependMovieSound selects the normal movie route; SC0000's low value 2 is retained as native movie mode state. Godot maps those categories to audio buses and sends FFmpeg-decoded timestamped stereo float PCM through a per-playback AudioStreamGenerator. Audio-bearing movies use the sound-hardware position as the master presentation clock; video-only movies retain monotonic pacing. Operand 4 is the movie sync/device mask. Static layer preparation after 0x236 does not terminate the retained movie; 0x21c services it through EOF and subsequent surface cleanup stops/detaches both decoder and audio output. The port type-checks the selected ResolvePacked record as MPEG without adding a scene base or fallback." +evidence = "Ghidra /v2 op_0x236_play_movie_to_surface@0x423ee0 fetches operand 1 and passes it unchanged to movie_to_texture_open_asset_graph@0x463e20, which passes it unchanged to asset_open_indexed_entry@0x44f390. The opener directly indexes the flat base table or selected AAI table and has no scene input. The handler leaves playback pending through movie_play_configure@0x4625e0; movie_start_pending_after_sync_delay@0x4633b0 records the first service tick and calls the shared IMediaControl::Run worker after operand 4's delay, giving non-modal playback the same DirectShow first-sample preroll as 0x20f. SC0000 native operand capture and exact 0x13c8->0x13d1 trace prove nonblocking behavior. BTL's live 0x2b21 site supplies 0x2af1/0x2af5/0x2bca/0x2bd8/0x2bde, the exact base entries MVB001/MVB004/MVB914/MVB958/MVB955." +details = "The handler requires an existing destination texture, allocates/reuses a 0x478-byte movie-to-texture object for the surface without replacing its resource identity/color key, opens operand 1 through the native universal packed-id reader, builds FilterGraph/IGraphBuilder/IMediaControl/IMediaPosition/IMediaEvent/IBasicAudio, and presents bottom-up RGB samples through the movie texture renderer. Operand 3 selects movie/sound routing policy: bits 0x10000/0x20000/0x40000/0x80000 force mute/music/SE/voice routes, otherwise set:DependMovieSound selects the normal movie route; SC0000's low value 2 is retained as native movie mode state. Godot maps those categories to audio buses and sends FFmpeg-decoded timestamped stereo float PCM through a per-playback AudioStreamGenerator. The first image is latched before audio start; video cadence is rebased to its first decoded PTS for both audio-bearing and video-only streams, while audio retains every sample from its own timestamp zero. Operand 4 delays the pending graph start in native service ticks. Static layer preparation after 0x236 does not terminate the retained movie; 0x21c services it through EOF and subsequent surface cleanup stops/detaches both decoder and audio output. The port type-checks the selected ResolvePacked record as MPEG without adding a scene base or fallback." [[opcode.semantics.args]] i = 1 @@ -6266,7 +6266,7 @@ observed_types = ["imm", "g-int", "l-ptr"] [[opcode.semantics.args]] i = 4 -role = "sync_mask" +role = "start_delay_ms" observed_types = ["imm", "l-ptr"] [[opcode]]