From 12af952b776a17c3a00151321d4b008daaf44623 Mon Sep 17 00:00:00 2001 From: gamer147 Date: Wed, 29 Jul 2026 15:41:01 -0400 Subject: [PATCH] Implement positioned movie playback opcode --- docs/engine-re.md | 14 +- docs/opcode-reference.md | 2 +- docs/phase-b-framework.md | 28 ++- docs/platform-portability.md | 6 +- docs/tools-reference.md | 4 + engine/Age.Engine.Tests/MovieOpcodeTests.cs | 184 +++++++++++++++++++- engine/Age.Engine.Tests/TestSupport.cs | 9 + engine/Age.Engine/Hosting/IHost.cs | 5 + engine/Age.Engine/Vm/VirtualMachine.cs | 11 ++ godot/FfmpegMovieDecoder.cs | 112 ++++++++++-- godot/FfmpegMovieNative.cs | 26 ++- godot/GodotAdvHost.cs | 20 ++- godot/IMovieDecoder.cs | 3 +- godot/Main.cs | 7 +- godot/MovieRuntime.cs | 18 +- native/age_movie_ffmpeg/age_movie.c | 110 ++++++++++-- native/age_movie_ffmpeg/age_movie.h | 8 + vm-map/opcodes.toml | 2 +- 18 files changed, 523 insertions(+), 46 deletions(-) diff --git a/docs/engine-re.md b/docs/engine-re.md index dac2d10..b89375d 100644 --- a/docs/engine-re.md +++ b/docs/engine-re.md @@ -1705,12 +1705,14 @@ The sole call, `CALLBACK_LOAD@0x189`, clarifies its purpose. Ordinary ADV movie after `0x236`; the load callback passes that value minus one to `0x241`. This is terminal-frame reconstruction after a numbered load, not restoration of a separately sampled live playback cursor. -The current FFmpeg backend has no initial-position seam. A faithful implementation should add a synchronous -pre-play seek for both video and audio, discard keyframe preroll until the requested timestamp, then rebase -the existing decoder pacing/completion timeline at that point. Simply decoding from zero until -`stop_time_ms-1` would make load latency proportional to the movie length; seeking video without matching -audio would replay stale audio during restoration. Once that bounded decoder primitive exists, the rest of -`0x241` can delegate to the existing `0x236` host/surface lifecycle. +The portable implementation delegates graph/surface ownership to the existing `0x236` lifecycle and adds +one synchronous initial-position boundary before decoder workers start. FFmpeg ABI v3 seeks the independent +video and audio demuxers to their preceding indexed packet. Managed video preroll retains the latest frame +at or before the requested position and queues the first later frame, which is essential for +`stop_time_ms-1` to restore the terminal image rather than run off EOF. Audio preroll drops complete earlier +blocks, trims a straddling block at sample precision, and rebases its remaining timestamps with video +pacing/completion to zero. A real archived `CHAPTER.AGF` regression proves both demuxers seek near 11 seconds; +synthetic regressions pin active-frame selection, terminal-frame retention, audio trimming, and pacing. ### Movie-mask transition `0x24d` and tiled-surface edge `0x248` (2026-07-29) diff --git a/docs/opcode-reference.md b/docs/opcode-reference.md index d0fc260..20b9b53 100644 --- a/docs/opcode-reference.md +++ b/docs/opcode-reference.md @@ -1109,7 +1109,7 @@ The surface object's +0x414 member is IMediaPosition. Its vtable +0x28 entry is - **depends on:** 0x236, 0x23f - **evidence:** Ghidra /v2: op_0x241_play_movie_to_surface_at_position@0x4247e0 uses the same 0x478-byte movie-to-texture object, destination render-target check, packed asset open, sound-route/volume path, and movie_play_configure worker as op 0x236. Between open and configure it calls movie+0x414 IMediaPosition vtable+0x20 put_CurrentPosition with operand5/1000.0; operand4 is then passed unchanged as the start delay. Corpus: sole site CALLBACK_LOAD@0x189 receives the layer resource, surface and flags from globals 0x3276/0x3239/0x328a, delay 0, and global 0x329e[layer]-1. The ordinary ADV creation path writes op 0x23f stop_time_ms into 0x329e immediately after op 0x236. -The seek is applied through IMediaPosition::put_CurrentPosition before movie_play_configure records the flags and start delay. CALLBACK_LOAD does not restore a separately sampled live playback cursor: the ADV setup path stores opcode 0x23f's stop time in global array 0x329e, and load passes stop_time_ms-1. The shipped use therefore reconstructs the movie layer at its terminal frame after a numbered load. The portable FFmpeg seam currently has no initial-position parameter; implementation requires a pre-play seek with keyframe-preroll discard and matching audio positioning, then can reuse the existing 0x236 surface binding, routing, completion, and delayed-start lifecycle. +The seek is applied through IMediaPosition::put_CurrentPosition before movie_play_configure records the flags and start delay. CALLBACK_LOAD does not restore a separately sampled live playback cursor: the ADV setup path stores opcode 0x23f's stop time in global array 0x329e, and load passes stop_time_ms-1. The shipped use therefore reconstructs the movie layer at its terminal frame after a numbered load. Port status (2026-07-29): implemented through the existing 0x236 surface binding, routing, and completion lifecycle. FFmpeg ABI v3 seeks its independent video and audio demuxers before worker start; managed preroll selects the video frame active at the requested position (retaining the last frame at stop_time_ms-1), trims audio to the same point, and rebases both remaining timelines to playback time zero. Negative positions clamp to zero and positions at or beyond the graph stop clamp to stop_time_ms-1. ### 0x242 `set-object-animation-detached` (set-object-animation-detached, argc 2) - **summary:** Replace the retained object's animation-control word at obj+0x2d0. Bit 0 detaches finite one-shot channels from blocking presentation and protects them from 0x243 forced completion until they finish naturally. diff --git a/docs/phase-b-framework.md b/docs/phase-b-framework.md index 18db46a..d32a012 100644 --- a/docs/phase-b-framework.md +++ b/docs/phase-b-framework.md @@ -754,13 +754,18 @@ gate on the one-player-attack acceptance path. ### MPEG movie audio implemented; audible acceptance in progress (2026-07-25) -The FFmpeg shim ABI is now version 2. Each in-memory MPEG payload feeds independent seekable video and audio +The FFmpeg shim ABI is now version 3. Each in-memory MPEG payload feeds independent seekable video and audio demuxers, preserving concurrent decode without temporary files, and exposes timestamped interleaved stereo float PCM resampled at the source rate. Video and audio timestamps share the same normalized media origin. The managed decoder owns bounded video and PCM queues plus separate workers; audio-bearing movies use the Godot audio-device clock as the presentation master, while video-only movies retain monotonic stopwatch pacing. Completion waits for the final video interval and for decoded PCM to be submitted. +ABI v3 adds a synchronous dual-demuxer initial seek used by opcode `0x241`. Native seek lands on preceding +indexed packets; managed preroll selects the video frame active at the requested position, sample-trims audio +to the same boundary, and rebases remaining pacing and completion. This preserves CALLBACK_LOAD's terminal +frame restoration without decode-from-zero latency or stale pre-seek audio. + Godot creates a per-playback `AudioStreamGenerator`, compensates for output latency, handles timestamp gaps and overlaps, and tears it down with the corresponding playback instance. Native movie flags route sound to the Movie bus by default, with exact overrides for mute (`0x10000`), Music (`0x20000`), SFX (`0x40000`), and Voice @@ -776,7 +781,7 @@ unselected but in-tree until that live gate. The first normal windowed LOGO/OP run confirmed that audio reaches the intended output, but it sounded crackly/warbled. Corpus endpoints showed continuous decoded PCM; the fault was presentation alignment. -Audio PTS crosses ABI v2 in whole milliseconds, losing up to 44 samples of precision at 44.1 kHz, while the +Audio PTS crosses the ABI in whole milliseconds, losing up to 44 samples of precision at 44.1 kHz, while the initial sink inserted or dropped that tiny discrepancy at every MPEG block boundary. Established timelines now tolerate 2 ms of timestamp quantization and remain sample-contiguous, while initial offsets and material later gaps/overlaps still insert silence or trim PCM. A regression simulates all 4,093 OP blocks without a @@ -1092,6 +1097,25 @@ Godot build, clean diff checking, and the Himegari-targeted threaded `SELFTEST O **NEXT:** implement positioned movie playback `0x241` with synchronized FFmpeg initial seek, then finish the DEBUG-only `0x24d` green-channel movie-mask compositor. +**POSITIONED MOVIE `0x241` IMPLEMENTED (2026-07-29):** the opcode now uses the existing nonmodal +movie-to-surface ownership and stop-time metadata path while passing its fifth operand through a dedicated +initial-position host seam. FFmpeg ABI v3 seeks the independent video and audio format/decoder pipelines +before worker start. Video keyframe preroll selects the frame active at the requested position and retains a +later decoded frame for normal pacing; EOF after preroll therefore publishes the terminal frame for +CALLBACK_LOAD's native `stop_time_ms-1` call. Audio preroll drops earlier blocks, sample-trims a block that +straddles the position, and rebases the remaining timestamps to the same playback-zero origin. + +Focused regressions cover exact opcode/host dispatch and retained graph stop time, runtime initial-position +and watchdog propagation, active-frame selection, subsequent pacing, terminal-frame retention, synchronized +audio trim/rebase, and a real VFS `CHAPTER.AGF` seek proving both native demuxers reach the requested +11-second neighborhood. Ordinary `0x236` and modal `0x20f` continue to open at position zero. The remaining +effectful inventory is one opcode / two DEBUG-only instructions: `0x24d`. +Validation passes 511/511 engine tests, the complete 213/213 installed-movie corpus gate, opcode/global/ +EngineCtx build and lint suites, the zero-warning Godot build, clean diff checking, and the +Himegari-targeted threaded `SELFTEST OK`. + +**NEXT:** implement the DEBUG-only `0x24d` green-channel movie-mask compositor. + ## Later Phase B breadth **INIT data-semantics side track started (2026-07-22).** Before naming more gameplay state, the static diff --git a/docs/platform-portability.md b/docs/platform-portability.md index 780b52e..bedae12 100644 --- a/docs/platform-portability.md +++ b/docs/platform-portability.md @@ -164,6 +164,10 @@ start. Builds use shared libraries, `$ORIGIN`/`@loader_path`-style local lookup hashes and configure arguments, and no committed original-game data. Packaging automation is part of completing the backend, not a prerequisite for the first native decode spike. +The current movie ABI is version 3. Its position-seek entry point resets both independent demux/decoder +pipelines before managed keyframe/audio preroll, so opcode `0x241` has the same bounded, synchronized behavior +on every future native target rather than relying on a Windows-only DirectShow cursor. + The Windows-x64 spike is now complete. `bootstrap-win64.ps1` verifies the immutable archive SHA before extraction, and `build-win64.ps1` builds the shim with MSVC and places the DLL, import artifacts, required LGPL shared libraries, and license under disposable `build/native/win-x64`. The managed resolver accepts @@ -184,7 +188,7 @@ display sizes (120x120 through 800x600), independently matched sequence-header d duration and frame-rate metadata, verified tightly packed RGBA size and nondecreasing timestamps, observed changing imagery in every asset, and closed every session. The set includes all 184 video-only and 29 audio-bearing streams; at that video-only milestone audio presence was detected but PCM remained intentionally undelivered. -The later ABI-v2 gate above supersedes that limitation. The 263-second `ED.AGF` was the +The later audio-delivery gate above supersedes that limitation. The 263-second `ED.AGF` was the slowest decode at 4.4 seconds in the deliberately unpaced gate, so its prior 30-second failure was test-harness whole-frame hashing overhead rather than a decoder incompatibility. The disposable machine report is `build/movie-corpus-ffmpeg.json`; invocation and report semantics are canonical in `docs/tools-reference.md`. diff --git a/docs/tools-reference.md b/docs/tools-reference.md index e3020ec..2560966 100644 --- a/docs/tools-reference.md +++ b/docs/tools-reference.md @@ -230,6 +230,10 @@ movie is copied into the repository or native output. When `build/native/win-x64 `dotnet build godot/Himegari.csproj` also stages the shim, its five DLL dependencies, and `FFmpeg-LICENSE.txt` beside `Himegari.dll` for development playback. +The current native movie ABI is version 3. In addition to sequential video/audio decode, it exposes a +synchronous position seek for both independent demuxers; exact video-frame selection and audio trimming are +performed by the managed decoder's preroll before normal paced delivery. + The corpus gate intentionally bypasses presentation waits: it validates video/audio decode compatibility and lifecycle, not wall-clock playback pacing or audible output. `--expected-count` makes additions, omissions, or profile changes explicit; changing the pinned FFmpeg dependency requires rerunning this gate. diff --git a/engine/Age.Engine.Tests/MovieOpcodeTests.cs b/engine/Age.Engine.Tests/MovieOpcodeTests.cs index 38cdaf9..ea460bb 100644 --- a/engine/Age.Engine.Tests/MovieOpcodeTests.cs +++ b/engine/Age.Engine.Tests/MovieOpcodeTests.cs @@ -12,6 +12,7 @@ public class MovieOpcodeTests private sealed class FakeMovieDecoder : IMovieDecoder { public long? StopTimeMs { get; init; } + public long InitialPositionMs { get; init; } public bool IsCompleted { get; set; } public string? Failure { get; set; } public bool Disposed { get; private set; } @@ -31,9 +32,11 @@ public class MovieOpcodeTests private sealed class FakeMovieDecoderFactory(FakeMovieDecoder decoder) : IMovieDecoderFactory { public MoviePayload? OpenedPayload { get; private set; } - public IMovieDecoder Open(MoviePayload movie) + public long OpenedInitialPositionMs { get; private set; } + public IMovieDecoder Open(MoviePayload movie, long initialPositionMs = 0) { OpenedPayload = movie; + OpenedInitialPositionMs = initialPositionMs; return decoder; } } @@ -47,8 +50,11 @@ public class MovieOpcodeTests public FfmpegMovieInfo Info { get; } = info; public int FailOnDecodeCall { get; init; } = -1; public int DecodeCalls => Volatile.Read(ref _decodeCalls); + public long? SeekPositionMs { get; private set; } public bool Disposed { get; private set; } + public void Seek(long positionMs) => SeekPositionMs = positionMs; + public bool TryDecodeNextVideoFrame(out FfmpegVideoFrame frame) { int call = Interlocked.Increment(ref _decodeCalls) - 1; @@ -112,8 +118,10 @@ public class MovieOpcodeTests var runtime = MovieRuntime.Open("TEST.AGF", 7, 0x123, payload, factory); Assert.Same(payload, factory.OpenedPayload); + Assert.Equal(0, factory.OpenedInitialPositionMs); Assert.Same(decoder, runtime.Decoder); Assert.Equal(0x123, runtime.ResourceId); + Assert.Equal(0, runtime.InitialPositionMs); Assert.Equal(1876, runtime.Decoder.StopTimeMs); Assert.Equal(5000, runtime.WatchdogMs); Assert.True(runtime.Decoder.TryTakeFrame(out var frame)); @@ -122,6 +130,22 @@ public class MovieOpcodeTests Assert.True(decoder.Disposed); } + [Fact] + public void MovieRuntimePassesInitialPositionAndBasesWatchdogOnRemainingDuration() + { + var decoder = new FakeMovieDecoder { StopTimeMs = 10000, InitialPositionMs = 9000 }; + var factory = new FakeMovieDecoderFactory(decoder); + + var runtime = MovieRuntime.Open("TEST.AGF", 7, 0x123, + new MoviePayload("TEST.AGF", new byte[] { 0, 0, 1, 0xba }), + factory, initialPositionMs: 9000); + + Assert.Equal(9000, factory.OpenedInitialPositionMs); + Assert.Equal(9000, runtime.InitialPositionMs); + Assert.Equal(5000, runtime.WatchdogMs); + runtime.Decoder.Dispose(); + } + [Theory] [InlineData(null, 30000)] [InlineData(-1L, 30000L)] @@ -237,6 +261,106 @@ public class MovieOpcodeTests Assert.True(SpinWait.SpinUntil(() => decoder.IsCompleted, 1000)); } + [Fact] + public void FfmpegPositionedDecoderPublishesFrameActiveAtSeekThenContinuesPacing() + { + var source = new FakeFfmpegFrameSource( + new FfmpegMovieInfo(1, 1, 120, 25, 1, false), + SyntheticMovieFrame(1, 0), + SyntheticMovieFrame(2, 40), + SyntheticMovieFrame(3, 80)); + using var clock = new ManualMoviePacingClock(); + using var decoder = new FfmpegMovieDecoder(source, clock, initialPositionMs: 60); + + Assert.Equal(60, source.SeekPositionMs); + Assert.Equal(60, decoder.InitialPositionMs); + Assert.True(SpinWait.SpinUntil(() => decoder.TryTakeFrame(out _), 1000)); + Assert.Equal(40, decoder.FirstFramePresentationTimeMs); + Assert.True(clock.WaitForDeadline(40)); + clock.AdvanceTo(39); + Assert.False(decoder.TryTakeFrame(out _)); + clock.AdvanceTo(40); + RgbaImage? next = null; + Assert.True(SpinWait.SpinUntil(() => + { + if (!decoder.TryTakeFrame(out var frame)) return false; + next = frame; + return true; + }, 1000)); + Assert.Equal((byte)3, next!.Pixels[0]); + } + + [Fact] + public void FfmpegPositionedDecoderRetainsTerminalFrameAtStopTimeMinusOne() + { + var source = new FakeFfmpegFrameSource( + new FfmpegMovieInfo(1, 1, 120, 25, 1, false), + SyntheticMovieFrame(1, 0), + SyntheticMovieFrame(2, 40), + SyntheticMovieFrame(3, 80)); + using var clock = new ManualMoviePacingClock(); + using var decoder = new FfmpegMovieDecoder(source, clock, initialPositionMs: 119); + + RgbaImage? terminal = null; + Assert.True(SpinWait.SpinUntil(() => + { + if (!decoder.TryTakeFrame(out var frame)) return false; + terminal = frame; + return true; + }, 1000)); + + Assert.Equal((byte)3, terminal!.Pixels[0]); + Assert.Equal(80, decoder.FirstFramePresentationTimeMs); + Assert.True(clock.WaitForDeadline(40)); + } + + [Fact] + public void FfmpegPositionedDecoderClampsInitialPositionToGraphBounds() + { + var belowStart = new FakeFfmpegFrameSource( + new FfmpegMovieInfo(1, 1, 120, 25, 1, false), + SyntheticMovieFrame(1, 0)); + using (var decoder = new FfmpegMovieDecoder(belowStart, null, initialPositionMs: -50)) + { + Assert.Equal(0, decoder.InitialPositionMs); + Assert.Null(belowStart.SeekPositionMs); + } + + var beyondStop = new FakeFfmpegFrameSource( + new FfmpegMovieInfo(1, 1, 120, 25, 1, false), + SyntheticMovieFrame(1, 80)); + using (var decoder = new FfmpegMovieDecoder(beyondStop, null, initialPositionMs: 500)) + { + Assert.Equal(119, decoder.InitialPositionMs); + Assert.Equal(119, beyondStop.SeekPositionMs); + } + } + + [Fact] + public void FfmpegPositionedDecoderTrimsAndRebasesAudioAtTheSamePosition() + { + var source = new FakeFfmpegFrameSource( + new FfmpegMovieInfo(1, 1, 150, 10, 1, true, 1000, 2, 50), + SyntheticMovieFrame(1, 0), SyntheticMovieFrame(2, 100)); + source.EnqueueAudio( + new FfmpegAudioChunk(Enumerable.Range(0, 100).Select(value => (float)value).ToArray(), + 50, 50), + new FfmpegAudioChunk(Enumerable.Repeat(0.25f, 100).ToArray(), 50, 100)); + using var clock = new ManualMoviePacingClock(); + using var decoder = new FfmpegMovieDecoder(source, clock, initialPositionMs: 75); + + var chunks = new List(); + Assert.True(SpinWait.SpinUntil(() => + { + while (decoder.TryTakeAudioChunk(out var chunk)) chunks.Add(chunk); + return decoder.AudioDecodingCompleted && chunks.Count == 2; + }, 1000)); + + Assert.Equal(new[] { 25, 50 }, chunks.Select(chunk => chunk.FrameCount)); + Assert.Equal(new long[] { 0, 25 }, chunks.Select(chunk => chunk.PresentationTimeMs)); + Assert.Equal(50f, chunks[0].InterleavedStereo[0]); + } + [Fact] public void FfmpegDecoderDisposalInterruptsFutureFrameWait() { @@ -454,6 +578,33 @@ public class MovieOpcodeTests Assert.Equal(-1, visible.ColorKey); } + [Fact] + public void PositionedMovieDispatchesExactOperandsAndRetainsGraphStopTime() + { + var table = OpcodeTableJson.Load(Paths.OpcodesJson); + var script = ScriptAssembler.Assemble(table, "POSITIONED-MOVIE", + [ + (0x241, + [ + new Operand(0, 0x33), new Operand(0, 5), new Operand(0, 2), + new Operand(0, 0), new Operand(0, 1875), + ]), + (0x23f, [new Operand(3, 0x1234), new Operand(0, 5)]), + (0x55, [new Operand(3, 0x1235), new Operand(0, 0x5678)]), + (0x2, Array.Empty()), + ], []); + var host = new RecordingHost { MovieStopTimeMs = 1876 }; + var vm = new VirtualMachine(script, table, host); + + vm.Run(); + + Assert.Equal("exit", vm.HaltReason); + Assert.Equal(new[] { (0x33L, 5, 2L, 0L, 1875L) }, host.PositionedMovies); + Assert.Empty(host.Movies); + Assert.Equal(1876, vm.Globals[0x1234]); + Assert.Equal(0x5678, vm.Globals[0x1235]); + } + [Fact] public void PlayMovieKeepsCreatedSurfaceBlankDuringSynchronousHostSetup() { @@ -683,6 +834,37 @@ public class MovieOpcodeTests Assert.True(heardSignal, $"{expectedName} should contain non-silent MPEG audio"); } + [Fact] + public void FfmpegShimSeeksBothVideoAndAudioNearRequestedPosition() + { + if (!OperatingSystem.IsWindows()) return; + ConfigureFfmpegNativeProbe(); + var catalog = Sys4AssetCatalog.Load(Paths.Sys4Ini); + var resources = new ResourceMap(catalog, new Sys4AssetStore(catalog, Paths.GameDir)); + var payload = resources.ReadMovie(resources.ResolveMovie(0x33)!); // CHAPTER.AGF + const long targetMs = 11000; + + using var movie = new FfmpegMovieSession(payload); + movie.Seek(targetMs); + + long videoTimestamp = -1; + for (int frame = 0; frame < 120 && videoTimestamp < targetMs; frame++) + { + Assert.True(movie.TryDecodeNextVideoFrame(out var decoded)); + videoTimestamp = decoded.PresentationTimeMs; + } + Assert.InRange(videoTimestamp, targetMs, movie.Info.StopTimeMs); + + long audioEndTimestamp = -1; + for (int chunk = 0; chunk < 100 && audioEndTimestamp < targetMs; chunk++) + { + Assert.True(movie.TryDecodeNextAudioChunk(out var decoded)); + audioEndTimestamp = decoded.PresentationTimeMs + + decoded.FrameCount * 1000L / movie.Info.AudioSampleRate; + } + Assert.InRange(audioEndTimestamp, targetMs, movie.Info.StopTimeMs + 100); + } + [Fact] public void FfmpegShimRejectsTruncatedMovieWithBoundedDiagnostic() { diff --git a/engine/Age.Engine.Tests/TestSupport.cs b/engine/Age.Engine.Tests/TestSupport.cs index 473ed3b..6f78f16 100644 --- a/engine/Age.Engine.Tests/TestSupport.cs +++ b/engine/Age.Engine.Tests/TestSupport.cs @@ -49,6 +49,8 @@ internal class RecordingHost : IHost public readonly List<(int Category, int BasisPoints)> AudioVolumeChanges = new(); public readonly List<(int Category, bool Enabled)> AudioRouteChanges = new(); public readonly List<(long Resource, int Surface, long Flags, long SyncMask)> Movies = new(); + public readonly List<(long Resource, int Surface, long Flags, long SyncMask, long PositionMs)> + PositionedMovies = new(); public System.Action? OnPlayMovie; public long? MovieStopTimeMs; public readonly HashSet ActiveMovieSurfaces = new(); @@ -216,6 +218,13 @@ internal class RecordingHost : IHost OnPlayMovie?.Invoke(); return MovieStopTimeMs; } + public long? PlayMovieToSurfaceAtPosition( + long resourceId, int surfaceSlot, long movieFlags, long syncMask, long positionMs) + { + PositionedMovies.Add((resourceId, surfaceSlot, movieFlags, syncMask, positionMs)); + OnPlayMovie?.Invoke(); + return MovieStopTimeMs; + } public bool IsMovieSurfaceActive(int surfaceSlot) => ActiveMovieSurfaces.Contains(surfaceSlot); public void PlayModalMovieToSurface(long resourceId, int surfaceSlot, long movieFlags) => ModalMovies.Add((resourceId, surfaceSlot, movieFlags)); diff --git a/engine/Age.Engine/Hosting/IHost.cs b/engine/Age.Engine/Hosting/IHost.cs index fa719bc..9f8c576 100644 --- a/engine/Age.Engine/Hosting/IHost.cs +++ b/engine/Age.Engine/Hosting/IHost.cs @@ -176,6 +176,11 @@ public interface IHost /// when the host could not obtain usable timing metadata. Native op 0x23f queries this state /// immediately after 0x236 returns. long? PlayMovieToSurface(long resourceId, int surfaceSlot, long movieFlags, long syncMask) => null; + // Native op 0x241 uses the same graph/surface lifecycle as 0x236, but seeks the graph before + // playback configuration. Hosts without positioned decoding may fall back to ordinary playback. + long? PlayMovieToSurfaceAtPosition( + long resourceId, int surfaceSlot, long movieFlags, long syncMask, long positionMs) + => PlayMovieToSurface(resourceId, surfaceSlot, movieFlags, syncMask); bool IsMovieSurfaceActive(int surfaceSlot) => false; // Native op 0x20f uses a universal packed id and parks script execution until the movie // reaches EOF or the player cancels it. The decoder remains asynchronous; the interactive host diff --git a/engine/Age.Engine/Vm/VirtualMachine.cs b/engine/Age.Engine/Vm/VirtualMachine.cs index 30c5d6f..97d99b5 100644 --- a/engine/Age.Engine/Vm/VirtualMachine.cs +++ b/engine/Age.Engine/Vm/VirtualMachine.cs @@ -2688,6 +2688,17 @@ public sealed class VirtualMachine Gfx.SetMovieStopTime(surfaceSlot, stopTimeMs ?? 0); return pc + 1; // native cmd size 9 resumes at the next instruction; playback is asynchronous } + case "u00422B80": // pre-reference compatibility + case "play-movie-to-surface-at-position": // 0x241 (+ initial position ms) + { + long resourceId = Read(a[0]); + int surfaceSlot = unchecked((int)Read(a[1])); + long? stopTimeMs = _host.PlayMovieToSurfaceAtPosition( + resourceId, surfaceSlot, Read(a[2]), Read(a[3]), Read(a[4])); + // As with 0x236, seeking changes decoder position but not the graph's stop metadata. + Gfx.SetMovieStopTime(surfaceSlot, stopTimeMs ?? 0); + return pc + 1; + } // ---- gfx command-buffer ops (VM-internal GfxState; docs/engine-re.md op-contract table) ---- case "query-gfx-object?": // 0x215 (out)(handle) -> slot | -1 if (_diagSetTexture) // reuse the flag: show what the slot query returns (grey-BG slot dig) diff --git a/godot/FfmpegMovieDecoder.cs b/godot/FfmpegMovieDecoder.cs index 0f864bc..2fb6b1b 100644 --- a/godot/FfmpegMovieDecoder.cs +++ b/godot/FfmpegMovieDecoder.cs @@ -88,6 +88,7 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder private readonly Queue _audioChunks = new(); private readonly AutoResetEvent _audioSpace = new(false); private readonly int _maximumQueuedAudioFrames; + private readonly long _initialPositionMs; private RgbaImage? _latestFrame; private volatile bool _completed; private volatile bool _videoTimelineCompleted; @@ -101,6 +102,7 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder private long _firstFramePresentationTimeMs = -1; public long? StopTimeMs => _source.Info.StopTimeMs; + public long InitialPositionMs => _initialPositionMs; public bool IsCompleted => _completed; public string? Failure => Volatile.Read(ref _failure); public long? FirstFramePresentationTimeMs @@ -114,12 +116,27 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder public MovieAudioInfo? AudioInfo { get; } public bool AudioDecodingCompleted => _audioDecodingCompleted; - public FfmpegMovieDecoder(MoviePayload movie) - : this(new FfmpegMovieSession(movie), null) { } + public FfmpegMovieDecoder(MoviePayload movie, long initialPositionMs = 0) + : this(new FfmpegMovieSession(movie), null, initialPositionMs) { } - internal FfmpegMovieDecoder(IFfmpegFrameSource source, IMoviePacingClock? clock) + internal FfmpegMovieDecoder(IFfmpegFrameSource source, IMoviePacingClock? clock, + long initialPositionMs = 0) { _source = source ?? throw new ArgumentNullException(nameof(source)); + _initialPositionMs = Math.Clamp( + initialPositionMs, + 0, + Math.Max(0, source.Info.StopTimeMs - 1)); + try + { + if (_initialPositionMs > 0) + source.Seek(_initialPositionMs); + } + catch + { + source.Dispose(); + throw; + } _clock = clock ?? (source.Info.HasAudio ? new ExternallyAdvancedMoviePacingClock() : new StopwatchMoviePacingClock()); @@ -216,9 +233,24 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder long lastTimestamp = -1; long decodedFrames = 0; long firstTimestamp = -1; + FfmpegVideoFrame? selectedSeekFrame = null; + FfmpegVideoFrame? pendingAfterSeek = null; + if (_initialPositionMs > 0) + selectedSeekFrame = SelectFrameAtInitialPosition(out pendingAfterSeek); while (!_cancel.WaitOne(0)) { - if (!_source.TryDecodeNextVideoFrame(out var frame)) + FfmpegVideoFrame frame; + if (selectedSeekFrame != null) + { + frame = selectedSeekFrame; + selectedSeekFrame = null; + } + else if (pendingAfterSeek != null) + { + frame = pendingAfterSeek; + pendingAfterSeek = null; + } + else if (!_source.TryDecodeNextVideoFrame(out frame)) { if (decodedFrames == 0) throw new InvalidDataException("FFmpeg stream ended before producing a video frame"); @@ -265,11 +297,37 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder } } + private FfmpegVideoFrame? SelectFrameAtInitialPosition(out FfmpegVideoFrame? pending) + { + pending = null; + FfmpegVideoFrame? candidate = null; + long priorTimestamp = -1; + while (!_cancel.WaitOne(0) && _source.TryDecodeNextVideoFrame(out FfmpegVideoFrame frame)) + { + if (frame.PresentationTimeMs < 0 || frame.PresentationTimeMs < priorTimestamp) + throw new InvalidDataException( + $"FFmpeg returned non-monotonic video timestamp {frame.PresentationTimeMs} " + + $"after {priorTimestamp} during seek preroll"); + priorTimestamp = frame.PresentationTimeMs; + if (frame.PresentationTimeMs <= _initialPositionMs) + { + candidate = frame; + continue; + } + if (candidate == null) + return frame; + pending = frame; + return candidate; + } + return candidate; + } + private void AudioDecodeThread() { try { - long priorTimestamp = -1; + long priorSourceTimestamp = -1; + long priorRebasedTimestamp = -1; while (!_cancel.WaitOne(0)) { while (Volatile.Read(ref _queuedAudioFrames) >= _maximumQueuedAudioFrames) @@ -286,18 +344,23 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder if (decoded.FrameCount <= 0 || decoded.InterleavedStereo.Length != checked(decoded.FrameCount * 2) || decoded.PresentationTimeMs < 0 - || decoded.PresentationTimeMs < priorTimestamp) + || decoded.PresentationTimeMs < priorSourceTimestamp) throw new InvalidDataException( $"FFmpeg returned invalid audio block {decoded.FrameCount}f " + - $"at {decoded.PresentationTimeMs} ms after {priorTimestamp} ms"); - var chunk = new MovieAudioChunk(decoded.InterleavedStereo, decoded.FrameCount, - decoded.PresentationTimeMs); + $"at {decoded.PresentationTimeMs} ms after {priorSourceTimestamp} ms"); + priorSourceTimestamp = decoded.PresentationTimeMs; + MovieAudioChunk? chunk = RebaseAudioChunk(decoded); + if (chunk == null) continue; + if (chunk.PresentationTimeMs < priorRebasedTimestamp) + throw new InvalidDataException( + $"FFmpeg seek produced non-monotonic rebased audio timestamp " + + $"{chunk.PresentationTimeMs} after {priorRebasedTimestamp} ms"); lock (_audioLock) { _audioChunks.Enqueue(chunk); _queuedAudioFrames += chunk.FrameCount; } - priorTimestamp = decoded.PresentationTimeMs; + priorRebasedTimestamp = chunk.PresentationTimeMs; } } catch (Exception error) @@ -310,6 +373,32 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder } } + private MovieAudioChunk? RebaseAudioChunk(FfmpegAudioChunk decoded) + { + int trimFrames = 0; + if (_initialPositionMs > decoded.PresentationTimeMs) + { + long deltaMs = _initialPositionMs - decoded.PresentationTimeMs; + long required = checked( + (deltaMs * (long)_source.Info.AudioSampleRate + 999) / 1000); + trimFrames = checked((int)Math.Min(decoded.FrameCount, required)); + } + if (trimFrames >= decoded.FrameCount) return null; + + float[] samples = decoded.InterleavedStereo; + int frameCount = decoded.FrameCount - trimFrames; + if (trimFrames > 0) + { + var trimmed = new float[checked(frameCount * 2)]; + Array.Copy(samples, checked(trimFrames * 2), trimmed, 0, trimmed.Length); + samples = trimmed; + } + long trimmedTimestamp = decoded.PresentationTimeMs + + trimFrames * 1000L / _source.Info.AudioSampleRate; + long rebasedTimestamp = Math.Max(0, trimmedTimestamp - _initialPositionMs); + return new MovieAudioChunk(samples, frameCount, rebasedTimestamp); + } + private void Fail(Exception error) { Interlocked.CompareExchange(ref _failure, error.Message, null); @@ -365,5 +454,6 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder internal sealed class FfmpegMovieDecoderFactory : IMovieDecoderFactory { - public IMovieDecoder Open(MoviePayload movie) => new FfmpegMovieDecoder(movie); + public IMovieDecoder Open(MoviePayload movie, long initialPositionMs = 0) + => new FfmpegMovieDecoder(movie, initialPositionMs); } diff --git a/godot/FfmpegMovieNative.cs b/godot/FfmpegMovieNative.cs index fdd92e3..f46e14f 100644 --- a/godot/FfmpegMovieNative.cs +++ b/godot/FfmpegMovieNative.cs @@ -21,6 +21,11 @@ internal sealed record FfmpegAudioChunk(float[] InterleavedStereo, int FrameCoun internal interface IFfmpegFrameSource : IDisposable { FfmpegMovieInfo Info { get; } + void Seek(long positionMs) + { + if (positionMs != 0) + throw new NotSupportedException("movie source does not support positioned playback"); + } bool TryDecodeNextVideoFrame(out FfmpegVideoFrame frame); bool TryDecodeNextAudioChunk(out FfmpegAudioChunk chunk) { @@ -39,7 +44,7 @@ internal sealed class FfmpegMovieSession : IFfmpegFrameSource public FfmpegMovieSession(MoviePayload movie) { ArgumentNullException.ThrowIfNull(movie); - if (FfmpegMovieNative.AbiVersion() != 2) + if (FfmpegMovieNative.AbiVersion() != 3) throw new InvalidOperationException("unsupported age_movie_ffmpeg ABI version"); byte[] error = new byte[1024]; @@ -70,6 +75,20 @@ internal sealed class FfmpegMovieSession : IFfmpegFrameSource } } + public void Seek(long positionMs) + { + ObjectDisposedException.ThrowIf(_handle.IsClosed, this); + int result; + string? error = null; + lock (_decodeLock) + { + result = FfmpegMovieNative.Seek(_handle, Math.Max(0, positionMs)); + if (result != 0) error = FfmpegMovieNative.LastError(_handle); + } + if (result != 0) + throw new InvalidDataException($"FFmpeg movie seek failed: {error}"); + } + public bool TryDecodeNextVideoFrame(out FfmpegVideoFrame frame) { ObjectDisposedException.ThrowIf(_handle.IsClosed, this); @@ -240,6 +259,11 @@ internal static class FfmpegMovieNative nuint rgbaSize, out long presentationTimeMs); + [DllImport(LibraryName, EntryPoint = "age_movie_seek", CallingConvention = CallingConvention.Cdecl)] + internal static extern int Seek( + FfmpegMovieHandle movie, + long positionMs); + [DllImport(LibraryName, EntryPoint = "age_movie_decode_audio", CallingConvention = CallingConvention.Cdecl)] internal static extern int DecodeAudio( FfmpegMovieHandle movie, diff --git a/godot/GodotAdvHost.cs b/godot/GodotAdvHost.cs index e991f56..432d46d 100644 --- a/godot/GodotAdvHost.cs +++ b/godot/GodotAdvHost.cs @@ -1273,7 +1273,18 @@ public sealed class GodotAdvHost : IHost var asset = _res.ResolveMovie(resourceId); if (asset == null) { Godot.GD.Print($"movie unresolved {scene}:0x{resourceId:x}"); return null; } StartMovie(asset, resourceId, surfaceSlot, movieFlags, syncMask, modal: false, - out long? stopTimeMs, out _); + initialPositionMs: 0, out long? stopTimeMs, out _); + return stopTimeMs ?? 0; + } + + public long? PlayMovieToSurfaceAtPosition( + long resourceId, int surfaceSlot, long movieFlags, long syncMask, long positionMs) + { + string scene = CurrentScene; + var asset = _res.ResolveMovie(resourceId); + if (asset == null) { Godot.GD.Print($"movie unresolved {scene}:0x{resourceId:x}"); return null; } + StartMovie(asset, resourceId, surfaceSlot, movieFlags, syncMask, modal: false, + initialPositionMs: positionMs, out long? stopTimeMs, out _); return stopTimeMs ?? 0; } @@ -1316,6 +1327,7 @@ public sealed class GodotAdvHost : IHost try { if (!StartMovie(asset, resourceId, surfaceSlot, movieFlags, 0, modal: true, + initialPositionMs: 0, out _, out long playbackId)) return; _timeline?.State("modal-movie-wait", new() { @@ -1349,7 +1361,8 @@ public sealed class GodotAdvHost : IHost } private bool StartMovie(AssetEntry asset, long resourceId, int surfaceSlot, long movieFlags, - long syncMask, bool modal, out long? stopTimeMs, out long playbackId) + long syncMask, bool modal, long initialPositionMs, + out long? stopTimeMs, out long playbackId) { stopTimeMs = null; // A playback is a surface-owned instance, not the shared resource id. BTL can schedule the same @@ -1372,10 +1385,11 @@ public sealed class GodotAdvHost : IHost ["resource"] = resourceId, ["playback"] = playbackId, ["surface"] = surfaceSlot, ["file"] = movie.Name, ["flags"] = movieFlags, ["sync_mask"] = syncMask, ["modal"] = modal, + ["initial_position_ms"] = Math.Max(0, initialPositionMs), }); bool started = _main.TryPlayMovie( movie.Bytes, movie.Name, playbackId, resourceId, asset.PackedId, movieFlags, - out stopTimeMs); + initialPositionMs, out stopTimeMs); if (!started) { stopTimeMs = 0; diff --git a/godot/IMovieDecoder.cs b/godot/IMovieDecoder.cs index 65fe357..0e6fcf4 100644 --- a/godot/IMovieDecoder.cs +++ b/godot/IMovieDecoder.cs @@ -11,6 +11,7 @@ internal sealed record MovieAudioChunk(float[] InterleavedStereo, int FrameCount internal interface IMovieDecoder : IDisposable { long? StopTimeMs { get; } + long InitialPositionMs => 0; bool IsCompleted { get; } string? Failure { get; } long? FirstFramePresentationTimeMs => null; @@ -28,5 +29,5 @@ internal interface IMovieDecoder : IDisposable internal interface IMovieDecoderFactory { - IMovieDecoder Open(MoviePayload movie); + IMovieDecoder Open(MoviePayload movie, long initialPositionMs = 0); } diff --git a/godot/Main.cs b/godot/Main.cs index 84b41f2..d40cfeb 100644 --- a/godot/Main.cs +++ b/godot/Main.cs @@ -738,6 +738,7 @@ public partial class Main : Godot.Control name = pair.Value.Name, asset_id = pair.Value.AssetId, stop_time_ms = pair.Value.Decoder.StopTimeMs, + initial_position_ms = pair.Value.InitialPositionMs, decoder_completed = pair.Value.Decoder.IsCompleted, decoder_failure = pair.Value.Decoder.Failure, first_frame_source_pts_ms = pair.Value.Decoder.FirstFramePresentationTimeMs, @@ -764,6 +765,7 @@ public partial class Main : Godot.Control name = pair.Value.Name, asset_id = pair.Value.AssetId, stop_time_ms = pair.Value.Decoder.StopTimeMs, + initial_position_ms = pair.Value.InitialPositionMs, decoder_completed = pair.Value.Decoder.IsCompleted, decoder_failure = pair.Value.Decoder.Failure, first_frame_source_pts_ms = pair.Value.Decoder.FirstFramePresentationTimeMs, @@ -2127,6 +2129,7 @@ public partial class Main : Godot.Control public bool TryPlayMovie(byte[] mpegBytes, string assetName, long playbackId, long resourceId, int assetId, long movieFlags, + long initialPositionMs, out long? stopTimeMs) { stopTimeMs = null; @@ -2134,7 +2137,8 @@ public partial class Main : Godot.Control { var payload = new Age.Engine.Sys4.MoviePayload(assetName, mpegBytes); var runtime = MovieRuntime.Open( - assetName, assetId, resourceId, payload, _movieDecoderFactory, movieFlags); + assetName, assetId, resourceId, payload, _movieDecoderFactory, movieFlags, + initialPositionMs); stopTimeMs = runtime.Decoder.StopTimeMs; while (!_pendingMovies.TryAdd(playbackId, runtime)) if (_pendingMovies.TryRemove(playbackId, out var prior)) prior.Decoder.Dispose(); @@ -2164,6 +2168,7 @@ public partial class Main : Godot.Control _movieCompletionNotified.Remove(playbackId); GD.Print($"movie started {movie.Name} playback={playbackId} " + $"({movie.Decoder.StopTimeMs?.ToString() ?? "unknown"} ms from VFS" + + (movie.InitialPositionMs > 0 ? $", start={movie.InitialPositionMs}ms" : "") + (movie.Decoder.AudioInfo is { } audio ? $", audio={audio.SampleRate}Hz stereo route={MovieAudioRouteFromFlags(movie.MovieFlags)}" : "") + ")"); diff --git a/godot/MovieRuntime.cs b/godot/MovieRuntime.cs index 80c5d66..c2ad3b9 100644 --- a/godot/MovieRuntime.cs +++ b/godot/MovieRuntime.cs @@ -4,16 +4,22 @@ using Age.Engine.Sys4; /// Presentation-side ownership for one decoder plus its fail-safe completion deadline. internal sealed record MovieRuntime(string Name, int AssetId, long ResourceId, IMovieDecoder Decoder, - long MovieFlags, long StartedAtTimestamp, long WatchdogMs) + long MovieFlags, long InitialPositionMs, + long StartedAtTimestamp, long WatchdogMs) { public static MovieRuntime Open(string name, int assetId, long resourceId, MoviePayload payload, - IMovieDecoderFactory factory, long movieFlags = 0) + IMovieDecoderFactory factory, long movieFlags = 0, + long initialPositionMs = 0) { ArgumentNullException.ThrowIfNull(factory); - IMovieDecoder decoder = factory.Open(payload); - return new MovieRuntime(name, assetId, resourceId, decoder, movieFlags, Stopwatch.GetTimestamp(), - decoder.StopTimeMs is >= 0 and var stopTime - ? Math.Clamp(stopTime + 2000, 5000, 300000) + IMovieDecoder decoder = factory.Open(payload, Math.Max(0, initialPositionMs)); + long remainingMs = decoder.StopTimeMs is >= 0 and var stopTime + ? Math.Max(0, stopTime - decoder.InitialPositionMs) + : -1; + return new MovieRuntime(name, assetId, resourceId, decoder, movieFlags, + decoder.InitialPositionMs, Stopwatch.GetTimestamp(), + remainingMs >= 0 + ? Math.Clamp(remainingMs + 2000, 5000, 300000) : 30000); } } diff --git a/native/age_movie_ffmpeg/age_movie.c b/native/age_movie_ffmpeg/age_movie.c index c684280..10a482e 100644 --- a/native/age_movie_ffmpeg/age_movie.c +++ b/native/age_movie_ffmpeg/age_movie.c @@ -16,7 +16,7 @@ #include #include -#define AGE_MOVIE_ABI_VERSION 2u +#define AGE_MOVIE_ABI_VERSION 3u #define AGE_MOVIE_IO_BUFFER_SIZE 32768 #define AGE_MOVIE_ERROR_SIZE 512 @@ -42,7 +42,9 @@ struct age_movie { int video_input_eof; int video_decoder_draining; int video_decoder_eof; + int video_seek_preroll; int64_t decoded_video_frame_count; + int64_t video_fallback_origin_ms; age_movie_io_state audio_io_state; AVIOContext *audio_io; @@ -57,10 +59,13 @@ struct age_movie { int audio_decoder_draining; int audio_decoder_eof; int audio_frame_pending; + int audio_seek_preroll; int64_t decoded_audio_sample_count; + int64_t audio_fallback_origin_ms; int audio_sample_rate; int64_t timeline_origin_ms; + int64_t stop_time_ms; int width; int height; char error[AGE_MOVIE_ERROR_SIZE]; @@ -285,6 +290,62 @@ static int rewind_audio_for_decode(age_movie *movie, AVStream *stream) { return result; } +static int64_t stream_timestamp_for_position( + int64_t timeline_origin_ms, int64_t position_ms, AVRational time_base) { + int64_t absolute_ms = timeline_origin_ms; + if (position_ms > 0 && absolute_ms <= INT64_MAX - position_ms) + absolute_ms += position_ms; + return av_rescale_q_rnd(absolute_ms, (AVRational){1, 1000}, time_base, + AV_ROUND_DOWN | AV_ROUND_PASS_MINMAX); +} + +static int seek_video_for_decode(age_movie *movie, int64_t position_ms) { + int64_t target = stream_timestamp_for_position( + movie->timeline_origin_ms, position_ms, movie->video_time_base); + int result = av_seek_frame( + movie->video_format, movie->video_stream_index, target, AVSEEK_FLAG_BACKWARD); + if (result < 0) + result = avformat_seek_file(movie->video_format, movie->video_stream_index, + INT64_MIN, target, INT64_MAX, AVSEEK_FLAG_BACKWARD); + if (result < 0) return result; + + avcodec_flush_buffers(movie->video_codec); + if (movie->video_packet != NULL) av_packet_unref(movie->video_packet); + if (movie->video_frame != NULL) av_frame_unref(movie->video_frame); + movie->video_input_eof = 0; + movie->video_decoder_draining = 0; + movie->video_decoder_eof = 0; + movie->video_seek_preroll = position_ms > 0; + movie->decoded_video_frame_count = 0; + movie->video_fallback_origin_ms = position_ms; + return 0; +} + +static int seek_audio_for_decode(age_movie *movie, int64_t position_ms) { + if (movie->audio_stream_index < 0) return 0; + int64_t target = stream_timestamp_for_position( + movie->timeline_origin_ms, position_ms, movie->audio_time_base); + int result = av_seek_frame( + movie->audio_format, movie->audio_stream_index, target, AVSEEK_FLAG_BACKWARD); + if (result < 0) + result = avformat_seek_file(movie->audio_format, movie->audio_stream_index, + INT64_MIN, target, INT64_MAX, AVSEEK_FLAG_BACKWARD); + if (result < 0) return result; + + avcodec_flush_buffers(movie->audio_codec); + if (movie->audio_packet != NULL) av_packet_unref(movie->audio_packet); + if (movie->audio_frame != NULL) av_frame_unref(movie->audio_frame); + movie->audio_input_eof = 0; + movie->audio_decoder_draining = 0; + movie->audio_decoder_eof = 0; + movie->audio_frame_pending = 0; + movie->audio_seek_preroll = position_ms > 0; + movie->decoded_audio_sample_count = 0; + movie->audio_fallback_origin_ms = position_ms; + swr_close(movie->swr); + return swr_init(movie->swr); +} + static void destroy_movie(age_movie *movie) { if (movie == NULL) return; swr_free(&movie->swr); @@ -361,6 +422,10 @@ int32_t AGE_MOVIE_CALL age_movie_open(const uint8_t *bytes, size_t length, movie->video_stream_index = result; AVStream *stream = movie->video_format->streams[movie->video_stream_index]; movie->video_time_base = stream->time_base; + movie->video_fallback_origin_ms = + stream->start_time == AV_NOPTS_VALUE ? 0 + : rescale_ms_down(stream->start_time, movie->video_time_base) + - movie->timeline_origin_ms; movie->video_codec = avcodec_alloc_context3(video_decoder); if (movie->video_codec == NULL) { @@ -399,6 +464,7 @@ int32_t AGE_MOVIE_CALL age_movie_open(const uint8_t *bytes, size_t length, set_error(movie, "determine video duration", result); goto failure; } + movie->stop_time_ms = duration_ms; result = rewind_for_decode(movie, stream); if (result < 0) { set_error(movie, "rewind MPEG stream", result); @@ -440,6 +506,10 @@ int32_t AGE_MOVIE_CALL age_movie_open(const uint8_t *bytes, size_t length, movie->audio_stream_index = result; AVStream *audio_stream = movie->audio_format->streams[movie->audio_stream_index]; movie->audio_time_base = audio_stream->time_base; + movie->audio_fallback_origin_ms = + audio_stream->start_time == AV_NOPTS_VALUE ? 0 + : rescale_ms_down(audio_stream->start_time, movie->audio_time_base) + - movie->timeline_origin_ms; movie->audio_codec = avcodec_alloc_context3(audio_decoder); if (movie->audio_codec == NULL) { snprintf(movie->error, sizeof(movie->error), "audio decoder allocation failed"); @@ -512,6 +582,24 @@ failure: return AGE_MOVIE_ERROR; } +int32_t AGE_MOVIE_CALL age_movie_seek(age_movie *movie, int64_t position_ms) { + if (movie == NULL || position_ms < 0) return AGE_MOVIE_INVALID_ARGUMENT; + if (movie->stop_time_ms > 0 && position_ms >= movie->stop_time_ms) + position_ms = movie->stop_time_ms - 1; + + int result = seek_video_for_decode(movie, position_ms); + if (result < 0) { + set_error(movie, "seek MPEG video stream", result); + return AGE_MOVIE_ERROR; + } + result = seek_audio_for_decode(movie, position_ms); + if (result < 0) { + set_error(movie, "seek MPEG audio stream", result); + return AGE_MOVIE_ERROR; + } + return 0; +} + int32_t AGE_MOVIE_CALL age_movie_decode_video(age_movie *movie, uint8_t *rgba, size_t rgba_size, int64_t *out_pts_ms) { if (movie == NULL || rgba == NULL || out_pts_ms == NULL) return AGE_MOVIE_INVALID_ARGUMENT; @@ -522,6 +610,7 @@ int32_t AGE_MOVIE_CALL age_movie_decode_video(age_movie *movie, for (;;) { int result = avcodec_receive_frame(movie->video_codec, movie->video_frame); if (result == 0) { + movie->video_seek_preroll = 0; movie->sws = sws_getCachedContext(movie->sws, movie->video_frame->width, movie->video_frame->height, (enum AVPixelFormat)movie->video_frame->format, @@ -561,10 +650,8 @@ int32_t AGE_MOVIE_CALL age_movie_decode_video(age_movie *movie, snprintf(movie->error, sizeof(movie->error), "decoded frame has no timestamp or frame rate"); return AGE_MOVIE_ERROR; } - int64_t stream_start = movie->video_format->streams[movie->video_stream_index]->start_time; - int64_t start_ms = stream_start == AV_NOPTS_VALUE ? 0 - : rescale_ms_down(stream_start, movie->video_time_base) - movie->timeline_origin_ms; - *out_pts_ms = start_ms + av_rescale_q_rnd(movie->decoded_video_frame_count, av_inv_q(rate), + *out_pts_ms = movie->video_fallback_origin_ms + + av_rescale_q_rnd(movie->decoded_video_frame_count, av_inv_q(rate), (AVRational){1, 1000}, AV_ROUND_DOWN | AV_ROUND_PASS_MINMAX); } else { *out_pts_ms = rescale_ms_down(timestamp, movie->video_time_base) @@ -618,6 +705,8 @@ int32_t AGE_MOVIE_CALL age_movie_decode_video(age_movie *movie, if (movie->video_input_eof) continue; result = avcodec_send_packet(movie->video_codec, movie->video_packet); av_packet_unref(movie->video_packet); + if (result == AVERROR_INVALIDDATA && movie->video_seek_preroll) + continue; if (result < 0) { set_error(movie, "send MPEG video packet", result); return AGE_MOVIE_ERROR; @@ -647,12 +736,8 @@ int32_t AGE_MOVIE_CALL age_movie_decode_audio(age_movie *movie, int64_t timestamp = movie->audio_frame->best_effort_timestamp; if (timestamp == AV_NOPTS_VALUE) timestamp = movie->audio_frame->pts; if (timestamp == AV_NOPTS_VALUE) { - AVStream *audio_stream = movie->audio_format->streams[movie->audio_stream_index]; - int64_t stream_start = audio_stream->start_time; - int64_t start_ms = stream_start == AV_NOPTS_VALUE ? 0 - : rescale_ms_down(stream_start, movie->audio_time_base) - - movie->timeline_origin_ms; - *out_pts_ms = start_ms + av_rescale_q_rnd(movie->decoded_audio_sample_count, + *out_pts_ms = movie->audio_fallback_origin_ms + + av_rescale_q_rnd(movie->decoded_audio_sample_count, (AVRational){1, movie->audio_sample_rate}, (AVRational){1, 1000}, AV_ROUND_DOWN | AV_ROUND_PASS_MINMAX); } else { @@ -678,6 +763,7 @@ int32_t AGE_MOVIE_CALL age_movie_decode_audio(age_movie *movie, int result = avcodec_receive_frame(movie->audio_codec, movie->audio_frame); if (result == 0) { + movie->audio_seek_preroll = 0; movie->audio_frame_pending = 1; continue; } @@ -724,6 +810,8 @@ int32_t AGE_MOVIE_CALL age_movie_decode_audio(age_movie *movie, if (movie->audio_input_eof) continue; result = avcodec_send_packet(movie->audio_codec, movie->audio_packet); av_packet_unref(movie->audio_packet); + if (result == AVERROR_INVALIDDATA && movie->audio_seek_preroll) + continue; if (result < 0) { set_error(movie, "send MPEG audio packet", result); return AGE_MOVIE_ERROR; diff --git a/native/age_movie_ffmpeg/age_movie.h b/native/age_movie_ffmpeg/age_movie.h index 25f5399..d26b184 100644 --- a/native/age_movie_ffmpeg/age_movie.h +++ b/native/age_movie_ffmpeg/age_movie.h @@ -49,6 +49,14 @@ AGE_MOVIE_API int32_t AGE_MOVIE_CALL age_movie_open( char *error_buffer, size_t error_buffer_size); +/* + * Seeks both the video and audio demux/decoder pipelines to the keyframe at or before + * position_ms. The caller performs decoded-sample preroll to the exact requested position. + */ +AGE_MOVIE_API int32_t AGE_MOVIE_CALL age_movie_seek( + age_movie *movie, + int64_t position_ms); + /* Writes one tightly packed top-down RGBA8 frame and its zero-based presentation timestamp. */ AGE_MOVIE_API int32_t AGE_MOVIE_CALL age_movie_decode_video( age_movie *movie, diff --git a/vm-map/opcodes.toml b/vm-map/opcodes.toml index 6781c84..4849519 100644 --- a/vm-map/opcodes.toml +++ b/vm-map/opcodes.toml @@ -6517,7 +6517,7 @@ abi_source = "kelebek+decode-validated" name = "play-movie-to-surface-at-position" category = "draw" summary = "(packed_resource_id)(surface_slot)(movie_flags)(start_delay_ms)(position_ms) - perform the same non-modal movie-to-retained-surface open as opcode 0x236, seek the graph to the requested millisecond position, then arm playback." -details = "The seek is applied through IMediaPosition::put_CurrentPosition before movie_play_configure records the flags and start delay. CALLBACK_LOAD does not restore a separately sampled live playback cursor: the ADV setup path stores opcode 0x23f's stop time in global array 0x329e, and load passes stop_time_ms-1. The shipped use therefore reconstructs the movie layer at its terminal frame after a numbered load. The portable FFmpeg seam currently has no initial-position parameter; implementation requires a pre-play seek with keyframe-preroll discard and matching audio positioning, then can reuse the existing 0x236 surface binding, routing, completion, and delayed-start lifecycle." +details = "The seek is applied through IMediaPosition::put_CurrentPosition before movie_play_configure records the flags and start delay. CALLBACK_LOAD does not restore a separately sampled live playback cursor: the ADV setup path stores opcode 0x23f's stop time in global array 0x329e, and load passes stop_time_ms-1. The shipped use therefore reconstructs the movie layer at its terminal frame after a numbered load. Port status (2026-07-29): implemented through the existing 0x236 surface binding, routing, and completion lifecycle. FFmpeg ABI v3 seeks its independent video and audio demuxers before worker start; managed preroll selects the video frame active at the requested position (retaining the last frame at stop_time_ms-1), trims audio to the same point, and rebases both remaining timelines to playback time zero. Negative positions clamp to zero and positions at or beyond the graph stop clamp to stop_time_ms-1." noop_headless = false source = "investigation" confidence = "high"