From 3378abdecaa481eeffb6f792a4dce58fe6c4511a Mon Sep 17 00:00:00 2001 From: gamer147 Date: Wed, 29 Jul 2026 16:01:00 -0400 Subject: [PATCH] Implement movie mask transition opcode --- docs/engine-re.md | 29 ++- docs/opcode-reference.md | 4 +- docs/phase-b-framework.md | 24 +++ docs/platform-portability.md | 9 +- .../ForegroundTransitionTests.cs | 19 ++ .../Age.Engine.Tests/MovieMaskSurfaceTests.cs | 55 ++++++ engine/Age.Engine.Tests/MovieOpcodeTests.cs | 79 +++++++- engine/Age.Engine.Tests/TestSupport.cs | 7 + engine/Age.Engine/Hosting/IHost.cs | 15 ++ engine/Age.Engine/Model/GfxState.cs | 61 +++++- engine/Age.Engine/Sys4/MovieMaskSurface.cs | 66 +++++++ engine/Age.Engine/Vm/VirtualMachine.cs | 8 + godot/FfmpegMovieDecoder.cs | 31 +++- godot/GodotAdvHost.cs | 173 +++++++++++++++++- godot/IMovieDecoder.cs | 3 +- godot/Main.cs | 11 +- godot/MovieRuntime.cs | 20 +- vm-map/opcodes.toml | 4 +- 18 files changed, 573 insertions(+), 45 deletions(-) create mode 100644 engine/Age.Engine.Tests/MovieMaskSurfaceTests.cs create mode 100644 engine/Age.Engine/Sys4/MovieMaskSurface.cs diff --git a/docs/engine-re.md b/docs/engine-re.md index b89375d..ddddc79 100644 --- a/docs/engine-re.md +++ b/docs/engine-re.md @@ -1761,12 +1761,29 @@ header identifies an MPEG program stream despite the extension. One call uses a post-exit developer menu, although the port deliberately exposes DEBUG through its F4 diagnostic route. The implementation boundary is consequently larger than ordinary movie playback or a scalar crossfade, -but no mask heuristic remains: the source is specifically the decoded green byte. The port needs a -movie-frame-to-mask publication path, old/new range capture, per-pixel alpha composition, and -movie-completion-driven blocking cleanup. The software compositor can serve as the correctness oracle; -the GPU path can fall back while a type-1 command is active, as it already does for whole-screen -transitions. The handlers, sample worker, compositor, and exact helpers are named and commented in the -saved `/v2` Ghidra image. +but no mask heuristic remains: the source is specifically the decoded green byte. It requires +movie-frame-to-mask publication, retained-range capture, per-pixel alpha composition, and +movie-completion-driven blocking cleanup. The handlers, sample worker, compositor, and exact helpers are +named and commented in the saved `/v2` Ghidra image. + +**Port implementation (2026-07-29):** `IHost.PlayMovieMaskTransition` carries the complete +twelve-operand request. At dispatch, Godot snapshots the old/source retained range through +`RetainedSurfaceRasterizer` into the script-created scratch surface; the immediately preceding +`draw-texture` command key therefore displays that captured range without a second compositor model. +Mask playback remains instance-keyed in the ordinary movie registry, but frame publication is redirected: +`MovieMaskSurface.ExtractGreen` copies logical top-down RGBA green bytes into the requested mask dimensions, +and `MovieMaskSurface.Apply` republishes an immutable dynamic-surface snapshot with the exact packed-ARGB +multiplication above. Signed rectangles clip mask and destination coordinates together, while pixels outside +the rectangle remain the captured source. + +The initial mask is published before decoder adoption. `MovieRuntime` defers its first-frame take for the +requested start delay, so decoder pacing cannot run ahead during that interval. Its optional presentation +duration scales frame deadlines by `requested_duration/native_stop_time` and holds the terminal frame until +the requested endpoint; this is the managed equivalent of `IMediaPosition::put_Rate`. Completion publishes +the mode-specific terminal fill, marks the type-1 command complete in `GfxState`, and only then releases +opcode `0x21c`'s blocking lifetime. The existing click/skip completion path deliberately affects only type 0. +Because the result is an ordinary dynamic RGBA scratch surface, both the software oracle and accepted Godot +GPU renderer consume the same pixels; no backend-specific mask shader or approximation is involved. **`0x248` port implementation (2026-07-29):** `GfxState` retains the complete signed dword with native zero initialization. The setter neither publishes a retained mutation nor rebuilds surfaces that already diff --git a/docs/opcode-reference.md b/docs/opcode-reference.md index 20b9b53..9f015cd 100644 --- a/docs/opcode-reference.md +++ b/docs/opcode-reference.md @@ -1138,12 +1138,12 @@ Port status (2026-07-29): implemented as retained signed-dword graphics configur - **evidence:** Ghidra /v2: op_0x249_load_raw_texture_surface@0x424b20 is instruction-length 7 and is contract-identical to gfx_op_0x1f9_load_surface through release, unchanged packed operand, asset_open_indexed_entry, RGB colorkey conversion, load failure, and cleanup. Its only relevant distinction is mode-1 gfx_surface_mode1_ctor, a tiled large-image wrapper: gfx_tiled_surface_create@0x432ff0 splits logical dimensions into ordinary mode-0 child textures; gfx_tiled_surface_upload_agf@0x431a10 decodes/uploads regions; gfx_tiled_surface_blit@0x4316b0 subdivides logical source rectangles. Corpus literals include FIELD 0x32da..0x32dd -> SO005/SO007/SO008A/SO007A. The former claim that only 0x249 bypasses scene normalization was wrong because native never performs scene normalization for 0x1f9 either. ### 0x24d `play-movie-mask-transition` (play-movie-mask-transition, argc 12) -- **summary:** Open a movie into a scratch surface, retime it to the requested duration, copy each decoded frame's green channel into a byte-per-pixel mask, and register a blocking retained-surface transition from an old object range to a newly drawn range within the supplied rectangle. +- **summary:** Capture an old retained-object range into the newly drawn scratch surface, retime a movie to the requested duration, and use each decoded frame's green channel as a byte-per-pixel alpha mask within the supplied rectangle. - **grounding:** source=investigation, confidence=high - **depends on:** 0x236, 0x223 - **evidence:** Ghidra /v2: op_0x24d_play_movie_mask_transition@0x424db0 opens operand 10 through movie_to_texture_open_asset_graph, configures delay operand 11, sets IMediaPosition::put_Rate to native_stop_time_ms / operand12_ms, allocates a width*height byte mask through movie_texture_allocate_transition_mask@0x415d90, and calls gfx_movie_mask_transition_register@0x47f560 with operands 1-9/11. movie_texture_renderer_receive_sample@0x4628d0's mask-active branch copies byte +1 of every bottom-up RGB24 sample pixel, i.e. its green channel, directly into that mask. The retained compositor's type-1 branch applies the movie-updated mask between the old and new object ranges and holds its blocking dirty state until playback completes. Both DEBUG.BIN sites use TEST.AGF (an MPEG program stream), delay 0, and duration 1000 ms. -The RGB24 sample callback copies the bottom-up green byte verbatim. In the native 32-bit mask compositor, captured RGB remains unchanged and the output alpha byte is the high byte of `((source_argb >> 8) * mask_byte)`; this records the exact packed-integer operation, including its low-color carry, rather than approximating it as a scalar luminance crossfade. Mode 1 initializes the mask to 0 and completes at 255; all other modes initialize at 255 and complete at 0. +The RGB24 sample callback copies the bottom-up green byte verbatim. In the native 32-bit mask compositor, captured RGB remains unchanged and the output alpha byte is the high byte of `((source_argb >> 8) * mask_byte)`; this records the exact packed-integer operation, including its low-color carry, rather than approximating it as a scalar luminance crossfade. Mode 1 initializes the mask to 0 and completes at 255; all other modes initialize at 255 and complete at 0. Port status (2026-07-29): implemented through the existing retained-range software rasterizer and dynamic scratch-surface path. The decoder honors start delay and native stop/duration retiming, publishes logical top-down green masks, and holds opcode 0x21c until movie completion; click completion remains limited to type-0 transitions. ### 0x24e `set-gfx-animation-service-flags` (set-gfx-animation-service-flags, argc 1) - **summary:** Replace the retained graphics animation-service flags with operand 1. BTL brackets combat presentation with values 1 and 0; GAMECLEAR uses 3 and 0. diff --git a/docs/phase-b-framework.md b/docs/phase-b-framework.md index d32a012..1823e81 100644 --- a/docs/phase-b-framework.md +++ b/docs/phase-b-framework.md @@ -1116,6 +1116,30 @@ Himegari-targeted threaded `SELFTEST OK`. **NEXT:** implement the DEBUG-only `0x24d` green-channel movie-mask compositor. +**MOVIE-MASK TRANSITION `0x24d` IMPLEMENTED (2026-07-29):** the final effectful opcode now dispatches +all twelve operands through a dedicated host request. The old/source retained range is captured into the +script-created scratch surface with the existing platform-neutral retained rasterizer. Each due TEST.AGF +frame publishes its logical green channel as a byte mask, and the scratch surface applies AGE's exact +packed-ARGB alpha multiplication inside the signed/clipped destination rectangle while preserving captured +RGB and every pixel outside the rectangle. Mode 1 runs from zero to 255; other modes run from 255 to zero. + +Movie playback retains ordinary instance ownership and VFS/FFmpeg decode, while the mask path redirects +frame publication away from visible movie RGB. Start delay parks first-frame consumption, stop/duration +retiming scales decoder presentation deadlines, and the terminal frame is held to the requested endpoint. +The type-1 command remains blocking until decoder completion and cannot be click-completed through the +type-0 transition shortcut. Completion publishes the exact terminal fill before opcode `0x21c` resumes. +The resulting scratch surface is an ordinary dynamic RGBA image, so software and GPU presentation share +one correctness path. + +Focused regressions cover exact VM dispatch, transition lifetime, click immunity, green extraction, +dimension padding, signed clipping, RGB/outside preservation, the native packed-color carry, requested +decoder retiming, and real VFS TEST.AGF metadata/green-mask decode. The effectful opcode inventory is now +empty. Validation passes 518/518 engine tests, the zero-warning Godot build, opcode/global/EngineCtx suites, +clean diff checking, and the Himegari-targeted threaded selftest. + +**NEXT:** return to playthrough-led polish and save compatibility testing; no decoded effectful opcode gap +remains in the shipped 481-script corpus. + ## Later Phase B breadth **INIT data-semantics side track started (2026-07-22).** Before naming more gameplay state, the static diff --git a/docs/platform-portability.md b/docs/platform-portability.md index bedae12..788462d 100644 --- a/docs/platform-portability.md +++ b/docs/platform-portability.md @@ -30,8 +30,8 @@ or replaced before claiming portable exports. |---|---|---|---| | SYS4INI per-game startup profile | `Sys4AssetCatalog` parses and retains the bounded ordered startup trailer; `Sys4LogicalCanvas` applies `SCREENX`/`SCREENY` with AGE's independent `640x480` fallbacks. Native SYS4INI also carries text, ADV input/skip, save ABI/path, audio, legacy renderer, and Windows registration settings | Godot uses the selected canvas for content scaling, backbuffers/compositor bounds, primary surfaces, layout/input fallbacks, and the default windowed client. `--window-width`/`--window-height` vary only the physical client while preserving the canvas and letterbox policy. The port still does not source `CancelMesSkipOnClick`, `CoexistMesSkip`, cursor/redraw policy, or wheel action ids from the trailer | Validate the existing presentation policy on Linux/macOS window managers. Apply further semantic keys explicitly, translate save roots through the host, and classify DirectDraw/fullscreen-bit and registration/key settings as native compatibility metadata. Canonical inventory and consumers: `sys4-format-notes.md` and `engine-re.md` | | Retained graphics presentation | Backend-neutral `GfxState`; accepted default Godot `Sprite2D` GPU stage plus the retained software pixel oracle, using runtime `ImageTexture`, canvas transforms/materials, and no native graphics API | GPU backend caches static/color-key variants, updates dynamic surfaces, handles retained range transitions, and falls back whole-frame for the legacy host screen-transition path | Godot owns D3D/Vulkan/Metal/OpenGL selection; validate shader/blend/filter behavior per target rather than adding a platform renderer | -| AGE movie decode (`0x236` scene movies; `0x20f` modal LOGO/OP/ED) | `FfmpegMovieDecoder` is the sole factory over the project-owned `native/age_movie_ffmpeg` ABI | Windows-x64 passes the complete 213-payload installed video/audio corpus gate plus audible LOGO/OP/CHAPTER playback | Add target-specific native builds and export packaging | -| Movie integration | Each surface owns a unique playback-instance id; `MovieRuntime` owns `IMovieDecoder` from an injected factory; video-only streams use monotonic pacing while audio-bearing streams use the Godot output clock | Concurrent/restarted uses of one asset have independent frame/audio/completion/teardown state; managed code is no longer Windows-annotated, while only the win-x64 native bundle exists today | Add Linux/macOS native builds and smoke gates | +| AGE movie decode (`0x236` scene movies; `0x20f` modal LOGO/OP/ED; `0x24d` movie masks) | `FfmpegMovieDecoder` is the sole factory over the project-owned `native/age_movie_ffmpeg` ABI | Windows-x64 passes the complete 213-payload installed video/audio corpus gate plus audible LOGO/OP/CHAPTER playback and real TEST.AGF green-mask decode | Add target-specific native builds and export packaging | +| Movie integration | Each surface owns a unique playback-instance id; `MovieRuntime` owns `IMovieDecoder` from an injected factory; video-only streams use monotonic pacing while audio-bearing streams use the Godot output clock. Opcode `0x24d` redirects decoded green bytes through the platform-neutral retained rasterizer and exact managed packed-alpha mask helper | Concurrent/restarted uses of one asset have independent frame/audio/completion/teardown state. Ordinary and mask movies share VFS/FFmpeg ownership; the mask result is a backend-neutral dynamic RGBA surface consumed by either Godot renderer. Managed code is no longer Windows-annotated, while only the win-x64 native bundle exists today | Add Linux/macOS native builds and smoke gates | | Movie audio | ABI v2 returns timestamped stereo float PCM; bounded managed buffering feeds a per-playback Godot `AudioStreamGenerator` and routes native movie flags to engine buses | All 29 installed audio-bearing streams decode with signal; synchronized LOGO/OP/CHAPTER playback is audibly accepted | Treat absent, distorted, or unsynchronized audio from an audio-bearing movie as a runtime bug | | ADV font discovery/raster fidelity | Opcode `0x1a5` reaches presentation; `godot/Main.cs` loads Windows `ＭＳ 明朝`/`ＭＳ ゴシック` from their known TTC files when available, otherwise uses the existing Japanese-font/default fallback. AGE actually uses a display information DC, weight-700 `LOGFONTA`, `GGO_GRAY4_BITMAP`, and its own integer glyph/outline compositor; the current `Label` backend substitutes FreeType embolden and a Godot outline | On the reference Windows install, regular advances and direct SC0000 placement/bounds agree, but the behaviorally different provisional bold path has 45% fewer bright pixels and much softer edge coverage. Other platforms normally lack the proprietary faces, so substitute metrics remain profile-dependent | Deferred until gameplay settles: implement the Phase-E decoded mask/metrics/compositor backlog in `docs/remake-architecture-and-roadmap.md`, with a shipped GDI reference backend on Windows and an explicitly defined portable rasterizer/substitution policy | | Filesystem semantics | Several filename and containment comparisons use `OrdinalIgnoreCase`; installed assets are conventionally uppercase | Needs validation on case-sensitive filesystems; may hide casing or containment mistakes | Add Linux/macOS tests with mixed-case synthetic roots and use filesystem-appropriate containment rules | @@ -51,13 +51,14 @@ dependency. The live connection is now backend-neutral: ``` -VM op 0x236 (non-modal) / op 0x20f (modal) - -> IHost.PlayMovieToSurface / PlayModalMovieToSurface +VM op 0x236 (non-modal) / op 0x20f (modal) / op 0x24d (green-mask transition) + -> IHost.PlayMovieToSurface / PlayModalMovieToSurface / PlayMovieMaskTransition -> VFS-owned MoviePayload bytes -> IMovieDecoderFactory -> FfmpegMovieDecoder -> FfmpegMovieSession -> age_movie C ABI -> newest due RGBA frame -> retained movie surface -> Godot compositor + or green byte -> exact captured-range alpha mask -> dynamic scratch surface -> timestamped stereo float PCM -> per-playback AudioStreamGenerator ``` diff --git a/engine/Age.Engine.Tests/ForegroundTransitionTests.cs b/engine/Age.Engine.Tests/ForegroundTransitionTests.cs index 8082311..c014fd0 100644 --- a/engine/Age.Engine.Tests/ForegroundTransitionTests.cs +++ b/engine/Age.Engine.Tests/ForegroundTransitionTests.cs @@ -1,6 +1,7 @@ using System.Collections.Generic; using System.Linq; using Age.Engine.Model; +using Age.Engine.Hosting; using Age.Engine.Sys4; using Age.Engine.Vm; using Xunit; @@ -74,4 +75,22 @@ public class ForegroundTransitionTests Assert.Equal(1, host.TransitionWaits); Assert.Equal(1.0, vm.Gfx.SnapshotForegroundTransitions(100).Single().Progress); } + + [Fact] + public void MovieMaskTransitionBlocksUntilMovieCompletionAndIgnoresClickCompletion() + { + var gfx = new GfxState(); + var request = new MovieMaskTransitionRequest( + 11, 45, 10, 1, -184, 0, 800, 600, 0, 0x325e, 0, 1000); + + gfx.QueueMovieMaskTransition(request); + + Assert.True(gfx.HasActiveForegroundTransitions(0)); + Assert.True(gfx.HasActiveTimedPresentation(0)); + Assert.Equal(0, gfx.CompleteForegroundTransitions(100)); + Assert.False(gfx.SnapshotMovieMaskTransitions().Single().Completed); + Assert.True(gfx.CompleteMovieMaskTransition(45)); + Assert.False(gfx.HasActiveForegroundTransitions(100)); + Assert.True(gfx.SnapshotMovieMaskTransitions().Single().Completed); + } } diff --git a/engine/Age.Engine.Tests/MovieMaskSurfaceTests.cs b/engine/Age.Engine.Tests/MovieMaskSurfaceTests.cs new file mode 100644 index 0000000..5fd2398 --- /dev/null +++ b/engine/Age.Engine.Tests/MovieMaskSurfaceTests.cs @@ -0,0 +1,55 @@ +using Age.Engine.Model; +using Age.Engine.Sys4; +using Xunit; + +public class MovieMaskSurfaceTests +{ + [Fact] + public void ExtractGreenUsesLogicalRowsAndRequestedMaskDimensions() + { + var frame = new RgbaImage(2, 2, + [ + 1, 10, 2, 255, 3, 20, 4, 255, + 5, 30, 6, 255, 7, 40, 8, 255, + ]); + + Assert.Equal(new byte[] { 10, 20, 99, 30, 40, 99 }, + MovieMaskSurface.ExtractGreen(frame, 3, 2, 99)); + } + + [Fact] + public void ApplyPreservesRgbAndUsesExactPackedCarryAlpha() + { + var captured = new RgbaImage(2, 1, + [ + 0, 0, 0, 128, + 255, 0, 0, 128, + ]); + + RgbaImage result = MovieMaskSurface.Apply(captured, [255, 255], 2, 1, 0, 0); + + Assert.Equal(new byte[] + { + 0, 0, 0, 127, // packed 0x80000000: native differs from alpha*255/255 + 255, 0, 0, 128, // packed red carry reaches the output alpha byte + }, result.Pixels); + Assert.Equal(new byte[] { 0, 0, 0, 128, 255, 0, 0, 128 }, captured.Pixels); + } + + [Fact] + public void ApplyClipsSignedRectangleAndLeavesOutsidePixelsUnchanged() + { + var captured = new RgbaImage(3, 1, + [ + 1, 2, 3, 255, + 4, 5, 6, 255, + 7, 8, 9, 255, + ]); + + RgbaImage result = MovieMaskSurface.Apply(captured, [0, 64, 128], 3, 1, -1, 0); + + Assert.Equal((byte)63, result.Pixels[3]); // mask column 1 + Assert.Equal((byte)127, result.Pixels[7]); // mask column 2 + Assert.Equal((byte)255, result.Pixels[11]); + } +} diff --git a/engine/Age.Engine.Tests/MovieOpcodeTests.cs b/engine/Age.Engine.Tests/MovieOpcodeTests.cs index ea460bb..8a260a9 100644 --- a/engine/Age.Engine.Tests/MovieOpcodeTests.cs +++ b/engine/Age.Engine.Tests/MovieOpcodeTests.cs @@ -33,10 +33,13 @@ public class MovieOpcodeTests { public MoviePayload? OpenedPayload { get; private set; } public long OpenedInitialPositionMs { get; private set; } - public IMovieDecoder Open(MoviePayload movie, long initialPositionMs = 0) + public long? OpenedPresentationDurationMs { get; private set; } + public IMovieDecoder Open( + MoviePayload movie, long initialPositionMs = 0, long? presentationDurationMs = null) { OpenedPayload = movie; OpenedInitialPositionMs = initialPositionMs; + OpenedPresentationDurationMs = presentationDurationMs; return decoder; } } @@ -202,6 +205,27 @@ public class MovieOpcodeTests Assert.Null(decoder.Failure); } + [Fact] + public void FfmpegDecoderRetimeUsesRequestedDurationAndHoldsTerminalFrameToEndpoint() + { + var source = new FakeFfmpegFrameSource( + new FfmpegMovieInfo(1, 1, 1000, 30, 1, false), + SyntheticMovieFrame(1, 600), SyntheticMovieFrame(2, 900)); + using var clock = new ManualMoviePacingClock(); + using var decoder = new FfmpegMovieDecoder( + source, clock, presentationDurationMs: 500); + + Assert.True(SpinWait.SpinUntil(() => decoder.TryTakeFrame(out _), 1000)); + Assert.True(clock.WaitForDeadline(150)); // (900 - 600) * 500 / 1000 + clock.AdvanceTo(150); + Assert.True(SpinWait.SpinUntil(() => decoder.TryTakeFrame(out _), 1000)); + Assert.True(clock.WaitForDeadline(500)); + clock.AdvanceTo(499); + Assert.False(decoder.IsCompleted); + clock.AdvanceTo(500); + Assert.True(SpinWait.SpinUntil(() => decoder.IsCompleted, 1000)); + } + [Fact] public void FfmpegDecoderPublishesAndRetainsFirstDecodedFrameBeforePacing() { @@ -605,6 +629,37 @@ public class MovieOpcodeTests Assert.Equal(0x5678, vm.Globals[0x1235]); } + [Fact] + public void MovieMaskTransitionDispatchesAllTwelveOperandsAndResumes() + { + var table = OpcodeTableJson.Load(Paths.OpcodesJson); + var script = ScriptAssembler.Assemble(table, "MOVIE-MASK", + [ + (0x24d, + [ + new Operand(0, 11), new Operand(0, 45), + new Operand(0, 10), new Operand(0, 1), + new Operand(0, unchecked((uint)-184)), new Operand(0, 0), + new Operand(0, 800), new Operand(0, 600), + new Operand(0, 0), new Operand(0, 0x325e), + new Operand(0, 0), new Operand(0, 1000), + ]), + (0x55, [new Operand(3, 0x1234), new Operand(0, 0x5678)]), + (0x2, Array.Empty()), + ], []); + var host = new RecordingHost(); + var vm = new VirtualMachine(script, table, host); + + vm.Run(); + + Assert.Equal("exit", vm.HaltReason); + Assert.Equal(0x5678, vm.Globals[0x1234]); + Assert.Equal(new MovieMaskTransitionRequest( + 11, 45, 10, 1, -184, 0, 800, 600, 0, 0x325e, 0, 1000), + host.MovieMaskTransitions.Single()); + Assert.True(vm.Gfx.SnapshotMovieMaskTransitions().Single().Completed); + } + [Fact] public void PlayMovieKeepsCreatedSurfaceBlankDuringSynchronousHostSetup() { @@ -727,6 +782,28 @@ public class MovieOpcodeTests Assert.Equal(new byte[] { 0, 0, 1, 0xba }, movie.Bytes[..4]); } + [Fact] + public void DebugTestMovieDecodesToExactGreenMaskDimensions() + { + if (!OperatingSystem.IsWindows()) return; + ConfigureFfmpegNativeProbe(); + var catalog = Sys4AssetCatalog.Load(Paths.Sys4Ini); + var resources = new ResourceMap(catalog, new Sys4AssetStore(catalog, Paths.GameDir)); + var payload = resources.ReadMovie(resources.ResolveMovie(0x325e)!); + + using var movie = new FfmpegMovieSession(payload); + Assert.Equal("TEST.AGF", payload.Name); + Assert.Equal(800, movie.Info.Width); + Assert.Equal(600, movie.Info.Height); + Assert.Equal(1000, movie.Info.StopTimeMs); + Assert.False(movie.Info.HasAudio); + Assert.True(movie.TryDecodeNextVideoFrame(out var frame)); + + byte[] mask = MovieMaskSurface.ExtractGreen(frame.Image, 800, 600, 255); + Assert.Equal(800 * 600, mask.Length); + Assert.True(mask.Distinct().Skip(1).Any(), "TEST.AGF should publish a nonuniform green mask"); + } + [Theory] [InlineData(0x335f, "LOGO.AGF")] [InlineData(0x3364, "OP.AGF")] diff --git a/engine/Age.Engine.Tests/TestSupport.cs b/engine/Age.Engine.Tests/TestSupport.cs index 6f78f16..5567491 100644 --- a/engine/Age.Engine.Tests/TestSupport.cs +++ b/engine/Age.Engine.Tests/TestSupport.cs @@ -51,6 +51,7 @@ internal class RecordingHost : IHost public readonly List<(long Resource, int Surface, long Flags, long SyncMask)> Movies = new(); public readonly List<(long Resource, int Surface, long Flags, long SyncMask, long PositionMs)> PositionedMovies = new(); + public readonly List MovieMaskTransitions = new(); public System.Action? OnPlayMovie; public long? MovieStopTimeMs; public readonly HashSet ActiveMovieSurfaces = new(); @@ -225,6 +226,12 @@ internal class RecordingHost : IHost OnPlayMovie?.Invoke(); return MovieStopTimeMs; } + public void PlayMovieMaskTransition(GfxState gfx, MovieMaskTransitionRequest request) + { + MovieMaskTransitions.Add(request); + gfx.QueueMovieMaskTransition(request); + gfx.CompleteMovieMaskTransition(request.SurfaceSlot); + } public bool IsMovieSurfaceActive(int surfaceSlot) => ActiveMovieSurfaces.Contains(surfaceSlot); public void PlayModalMovieToSurface(long resourceId, int surfaceSlot, long movieFlags) => ModalMovies.Add((resourceId, surfaceSlot, movieFlags)); diff --git a/engine/Age.Engine/Hosting/IHost.cs b/engine/Age.Engine/Hosting/IHost.cs index 9f8c576..0c7aca3 100644 --- a/engine/Age.Engine/Hosting/IHost.cs +++ b/engine/Age.Engine/Hosting/IHost.cs @@ -28,6 +28,13 @@ public readonly record struct SurfaceRectCopy( int SourceSurface, int DestinationSurface, int SourceX, int SourceY, int Width, int Height, int DestinationX, int DestinationY); +/// Opcode 0x24d's captured-range movie-mask transition. The movie's decoded green channel +/// becomes a byte-per-pixel alpha mask over SourceRange in the scratch surface. +public readonly record struct MovieMaskTransitionRequest( + long CommandKey, int SurfaceSlot, long SourceRangeStart, int SourceRangeCount, + int X, int Y, int Width, int Height, long Mode, long ResourceId, + long StartDelayMs, long DurationMs); + /// A synchronous AGE-owned diagnostic prompt after native body/context formatting. public readonly record struct DiagnosticMessage(string Caption, string Text); @@ -181,6 +188,14 @@ public interface IHost long? PlayMovieToSurfaceAtPosition( long resourceId, int surfaceSlot, long movieFlags, long syncMask, long positionMs) => PlayMovieToSurface(resourceId, surfaceSlot, movieFlags, syncMask); + // Native op 0x24d captures a retained range into the scratch surface, then publishes each + // decoded movie frame's green channel as an exact packed-alpha mask. A nonvisual host completes + // the lifecycle immediately so a following 0x21c can never strand script execution. + void PlayMovieMaskTransition(GfxState gfx, MovieMaskTransitionRequest request) + { + gfx.QueueMovieMaskTransition(request); + gfx.CompleteMovieMaskTransition(request.SurfaceSlot); + } bool IsMovieSurfaceActive(int surfaceSlot) => false; // Native op 0x20f uses a universal packed id and parks script execution until the movie // reaches EOF or the player cancels it. The decoder remains asynchronous; the interactive host diff --git a/engine/Age.Engine/Model/GfxState.cs b/engine/Age.Engine/Model/GfxState.cs index b8647ee..ecb726e 100644 --- a/engine/Age.Engine/Model/GfxState.cs +++ b/engine/Age.Engine/Model/GfxState.cs @@ -1,4 +1,5 @@ using System.Linq; +using Age.Engine.Hosting; namespace Age.Engine.Model; @@ -32,6 +33,9 @@ public readonly record struct SurfaceTransitionState(long CommandKey, int Target long RangeAStart, int RangeACount, long RangeBStart, int RangeBCount, long DelayMs, long DurationMs, long StartMs, double Progress, bool Forced); +public readonly record struct MovieMaskTransitionState( + MovieMaskTransitionRequest Request, bool Completed); + /// One synchronized sample of op 0x202's native one-shot packed-color channel. public readonly record struct ColorTransitionState(long Current, long Target, long DelayMs, long DurationMs, long StartMs, double Progress, bool Active); @@ -112,6 +116,11 @@ public sealed class GfxState public long StartMs = -1; public bool Forced; } + private sealed class MovieMaskTransition + { + public required MovieMaskTransitionRequest Request; + public bool Completed; + } public sealed class GfxObject { // The native numbered-save record contains several full transform matrices and reserved fields @@ -410,6 +419,7 @@ public sealed class GfxState _reloadableSurfaces.Clear(); _movieStopTimesMs.Clear(); _surfaceTransitions.Clear(); + _movieMaskTransitions.Clear(); CurrentObject = 0; CurrentRenderTargetSlot = -1; _rangeTransformFirst = 0; @@ -525,6 +535,7 @@ public sealed class GfxState // the movie object exists but its host decoder supplied no usable IMediaPosition stop time. private readonly Dictionary _movieStopTimesMs = new(); private readonly Dictionary _surfaceTransitions = new(); + private readonly Dictionary _movieMaskTransitions = new(); public void SetSurface(int slot, long resId, long colorKey) { lock (_lock) @@ -587,6 +598,7 @@ public sealed class GfxState _createdSurfaces.Remove(slot); _movieStopTimesMs.Remove(slot); _surfaceTransitions.Remove(slot); + _movieMaskTransitions.Remove(slot); } if (CurrentRenderTargetSlot >= firstSlot && CurrentRenderTargetSlot < end) CurrentRenderTargetSlot = -1; @@ -707,6 +719,7 @@ public sealed class GfxState _createdSurfaces.Remove(slot); _movieStopTimesMs.Remove(slot); _surfaceTransitions.Remove(slot); + _movieMaskTransitions.Remove(slot); MarkRetainedMutation(); } } @@ -729,6 +742,45 @@ public sealed class GfxState } } + public void QueueMovieMaskTransition(MovieMaskTransitionRequest request) + { + lock (_lock) + { + _movieMaskTransitions[request.SurfaceSlot] = new MovieMaskTransition + { + Request = request with + { + SourceRangeCount = System.Math.Max(0, request.SourceRangeCount), + Width = System.Math.Max(0, request.Width), + Height = System.Math.Max(0, request.Height), + StartDelayMs = System.Math.Max(0, request.StartDelayMs), + DurationMs = System.Math.Max(0, request.DurationMs), + }, + }; + MarkRetainedMutation(); + } + } + + public bool CompleteMovieMaskTransition(int surfaceSlot) + { + lock (_lock) + { + if (!_movieMaskTransitions.TryGetValue(surfaceSlot, out var transition) + || transition.Completed) + return false; + transition.Completed = true; + MarkRetainedMutation(); + return true; + } + } + + public IReadOnlyList SnapshotMovieMaskTransitions() + { + lock (_lock) + return _movieMaskTransitions.Values + .Select(t => new MovieMaskTransitionState(t.Request, t.Completed)).ToList(); + } + /// Start every pending foreground transition at the native present boundary. public int StartForegroundTransitions(long nowMs) { @@ -744,7 +796,8 @@ public sealed class GfxState public bool HasActiveForegroundTransitions(long nowMs) { lock (_lock) - return _surfaceTransitions.Values.Any(t => TransitionProgress(t, nowMs) < 1.0); + return _surfaceTransitions.Values.Any(t => TransitionProgress(t, nowMs) < 1.0) + || _movieMaskTransitions.Values.Any(t => !t.Completed); } /// Native op 0x21c keeps presenting until both queued surface commands and finite one-shot @@ -753,6 +806,7 @@ public sealed class GfxState { lock (_lock) return _surfaceTransitions.Values.Any(t => TransitionProgress(t, nowMs) < 1.0) || + _movieMaskTransitions.Values.Any(t => !t.Completed) || _rangeTransform.ScaleEnabled || _rangeTransform.RotationChannelEnabled || _rangeTransform.TranslationEnabled || _objects.Values.Any(o => o.Visible && @@ -780,10 +834,12 @@ public sealed class GfxState return new GfxDiagnosticSnapshot( nowMs, _surfaceTransitions.Values.Any(t => TransitionProgress(t, nowMs) < 1.0) + || _movieMaskTransitions.Values.Any(t => !t.Completed) || range != null || objects.Length != 0, _objects.Count, _objects.Values.Count(o => o.Visible), - _surfaceTransitions.Values.Count(t => TransitionProgress(t, nowMs) < 1.0), + _surfaceTransitions.Values.Count(t => TransitionProgress(t, nowMs) < 1.0) + + _movieMaskTransitions.Values.Count(t => !t.Completed), AnimationServiceFlags, AnimClockDurationTicks, AnimClockGeneration, @@ -866,6 +922,7 @@ public sealed class GfxState { lock (_lock) return _surfaceTransitions.Values.Any(t => TransitionProgress(t, nowMs) < 1.0) || + _movieMaskTransitions.Values.Any(t => !t.Completed) || _rangeTransform.ScaleEnabled || _rangeTransform.RotationChannelEnabled || _rangeTransform.TranslationEnabled || _objects.Values.Any(o => o.Visible && diff --git a/engine/Age.Engine/Sys4/MovieMaskSurface.cs b/engine/Age.Engine/Sys4/MovieMaskSurface.cs new file mode 100644 index 0000000..a639ed2 --- /dev/null +++ b/engine/Age.Engine/Sys4/MovieMaskSurface.cs @@ -0,0 +1,66 @@ +using Age.Engine.Model; + +namespace Age.Engine.Sys4; + +/// Platform-neutral pixel oracle for opcode 0x24d's type-1 movie transition. +public static class MovieMaskSurface +{ + /// Extract logical top-down green samples into the requested native mask dimensions. + /// Missing source pixels retain . + public static byte[] ExtractGreen(RgbaImage frame, int width, int height, byte fill) + { + int safeWidth = System.Math.Max(0, width); + int safeHeight = System.Math.Max(0, height); + var mask = new byte[checked(safeWidth * safeHeight)]; + if (fill != 0) System.Array.Fill(mask, fill); + int copyWidth = System.Math.Min(safeWidth, frame.Width); + int copyHeight = System.Math.Min(safeHeight, frame.Height); + for (int row = 0; row < copyHeight; row++) + { + int source = checked(row * frame.Width * 4 + 1); + int destination = checked(row * safeWidth); + for (int column = 0; column < copyWidth; column++, source += 4) + mask[destination + column] = frame.Pixels[source]; + } + return mask; + } + + /// Apply AGE's exact packed-ARGB multiplication to a captured RGBA surface. RGB and + /// pixels outside the requested rectangle are preserved. The multiplication deliberately includes + /// carry from the packed red/green bytes; it is not conventional alpha*mask/255. + public static RgbaImage Apply( + RgbaImage captured, ReadOnlySpan mask, int maskWidth, int maskHeight, int x, int y) + { + if (maskWidth < 0 || maskHeight < 0 + || mask.Length != checked(maskWidth * maskHeight)) + throw new System.ArgumentException("mask dimensions do not match its byte count", nameof(mask)); + var output = new RgbaImage( + captured.Width, captured.Height, (byte[])captured.Pixels.Clone()); + long left = System.Math.Max(0L, x); + long top = System.Math.Max(0L, y); + long right = System.Math.Min((long)captured.Width, (long)x + maskWidth); + long bottom = System.Math.Min((long)captured.Height, (long)y + maskHeight); + if (right <= left || bottom <= top) return output; + + for (int destinationY = (int)top; destinationY < (int)bottom; destinationY++) + { + int maskY = destinationY - y; + for (int destinationX = (int)left; destinationX < (int)right; destinationX++) + { + int pixel = checked((destinationY * captured.Width + destinationX) * 4); + byte red = captured.Pixels[pixel]; + byte green = captured.Pixels[pixel + 1]; + byte blue = captured.Pixels[pixel + 2]; + byte alpha = captured.Pixels[pixel + 3]; + uint argb = (uint)alpha << 24 + | (uint)red << 16 + | (uint)green << 8 + | blue; + uint product = unchecked((argb >> 8) + * mask[checked(maskY * maskWidth + destinationX - x)]); + output.Pixels[pixel + 3] = (byte)(product >> 24); + } + } + return output; + } +} diff --git a/engine/Age.Engine/Vm/VirtualMachine.cs b/engine/Age.Engine/Vm/VirtualMachine.cs index 97d99b5..4b70414 100644 --- a/engine/Age.Engine/Vm/VirtualMachine.cs +++ b/engine/Age.Engine/Vm/VirtualMachine.cs @@ -2846,6 +2846,14 @@ public sealed class VirtualMachine Gfx.ResetAnimClock(); return pc + 1; case "set-gfx-animation-service-flags": // 0x24e: bit 1 suppresses op 0x243 Gfx.SetAnimationServiceFlags(Read(a[0])); return pc + 1; + case "play-movie-mask-transition": // 0x24d: captured retained range + movie green-channel mask + _host.PlayMovieMaskTransition(Gfx, new MovieMaskTransitionRequest( + Read(a[0]), unchecked((int)Read(a[1])), + Read(a[2]), unchecked((int)Read(a[3])), + unchecked((int)Read(a[4])), unchecked((int)Read(a[5])), + unchecked((int)Read(a[6])), unchecked((int)Read(a[7])), + Read(a[8]), Read(a[9]), Read(a[10]), Read(a[11]))); + return pc + 1; case "queue-surface-alpha-transition": // 0x223: target surface crossfade over two object ranges Gfx.QueueSurfaceAlphaTransition(Read(a[0]), (int)Read(a[1]), Read(a[2]), (int)Read(a[3]), Read(a[4]), (int)Read(a[5]), Read(a[6]), Read(a[7])); return pc + 1; diff --git a/godot/FfmpegMovieDecoder.cs b/godot/FfmpegMovieDecoder.cs index 2fb6b1b..763c50e 100644 --- a/godot/FfmpegMovieDecoder.cs +++ b/godot/FfmpegMovieDecoder.cs @@ -89,6 +89,7 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder private readonly AutoResetEvent _audioSpace = new(false); private readonly int _maximumQueuedAudioFrames; private readonly long _initialPositionMs; + private readonly long? _presentationDurationMs; private RgbaImage? _latestFrame; private volatile bool _completed; private volatile bool _videoTimelineCompleted; @@ -116,17 +117,21 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder public MovieAudioInfo? AudioInfo { get; } public bool AudioDecodingCompleted => _audioDecodingCompleted; - public FfmpegMovieDecoder(MoviePayload movie, long initialPositionMs = 0) - : this(new FfmpegMovieSession(movie), null, initialPositionMs) { } + public FfmpegMovieDecoder( + MoviePayload movie, long initialPositionMs = 0, long? presentationDurationMs = null) + : this(new FfmpegMovieSession(movie), null, initialPositionMs, presentationDurationMs) { } internal FfmpegMovieDecoder(IFfmpegFrameSource source, IMoviePacingClock? clock, - long initialPositionMs = 0) + long initialPositionMs = 0, long? presentationDurationMs = null) { _source = source ?? throw new ArgumentNullException(nameof(source)); _initialPositionMs = Math.Clamp( initialPositionMs, 0, Math.Max(0, source.Info.StopTimeMs - 1)); + _presentationDurationMs = presentationDurationMs is >= 0 + ? Math.Max(0, presentationDurationMs.Value) + : null; try { if (_initialPositionMs > 0) @@ -254,10 +259,11 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder { if (decodedFrames == 0) throw new InvalidDataException("FFmpeg stream ended before producing a video frame"); - long completionTime = PresentationDeadline( - Math.Max(_source.Info.StopTimeMs, - lastTimestamp + FrameIntervalMilliseconds(_source.Info)), - firstTimestamp); + long completionTime = _presentationDurationMs + ?? PresentationDeadline( + Math.Max(_source.Info.StopTimeMs, + lastTimestamp + FrameIntervalMilliseconds(_source.Info)), + firstTimestamp); if (_clock.WaitUntil(completionTime, _cancel)) { _videoTimelineCompleted = true; @@ -432,7 +438,11 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder // stream's mux timestamp origin. DirectShow presents the video's first sample as video time zero; // retaining the absolute mux offset here would freeze that sample until the audio clock caught up. // Preserve every decoded frame and its cadence, but rebase the video stream to its first sample. - return Math.Max(0, sourceTimestamp - firstVideoTimestamp); + long sourceElapsed = Math.Max(0, sourceTimestamp - firstVideoTimestamp); + if (_presentationDurationMs is not { } duration + || _source.Info.StopTimeMs <= 0) + return sourceElapsed; + return checked(sourceElapsed * duration / _source.Info.StopTimeMs); } public void Dispose() @@ -454,6 +464,7 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder internal sealed class FfmpegMovieDecoderFactory : IMovieDecoderFactory { - public IMovieDecoder Open(MoviePayload movie, long initialPositionMs = 0) - => new FfmpegMovieDecoder(movie, initialPositionMs); + public IMovieDecoder Open( + MoviePayload movie, long initialPositionMs = 0, long? presentationDurationMs = null) + => new FfmpegMovieDecoder(movie, initialPositionMs, presentationDurationMs); } diff --git a/godot/GodotAdvHost.cs b/godot/GodotAdvHost.cs index 432d46d..688d488 100644 --- a/godot/GodotAdvHost.cs +++ b/godot/GodotAdvHost.cs @@ -41,6 +41,11 @@ public sealed class GodotAdvHost : IHost private readonly Dictionary _surfaceColorKeys = new(); private readonly Dictionary _surfaceResources = new(); // surface slot -> packed catalog id private readonly MovieSurfaceRegistry _movieSurfaces = new(); + private sealed record MovieMaskPlayback( + GfxState Gfx, MovieMaskTransitionRequest Request, RgbaImage Captured); + private readonly object _movieMaskLock = new(); + private readonly Dictionary _movieMasksByPlayback = new(); + private readonly Dictionary _movieMaskPlaybackBySurface = new(); private readonly string?[] _sfxNames = new string?[10]; // SC0000 native channel subset // slot -> dimensions of the currently allocated surface. Slot 0 begins as the selected game's // logical canvas, but op 0x1fa releases it like any other slot; subsequent queries return 0x0. @@ -1256,6 +1261,12 @@ public sealed class GodotAdvHost : IHost public (RgbaImage Image, string Name, int AssetId, bool IsDynamic)? ResolveSurfaceTexture( int surfaceSlot, long fallbackResourceId) { + lock (_movieMaskLock) + if (_movieMaskPlaybackBySurface.ContainsKey(surfaceSlot)) + lock (_imageLock) + if (_surfaceImages.TryGetValue(surfaceSlot, out var masked)) + return (masked, $"", + int.MinValue + surfaceSlot, true); if (_movieSurfaces.TryResolveSurface(surfaceSlot, out var movie) && movie != null) return (movie.Image, movie.Name, movie.AssetId, true); if (_movieSurfaces.IsBound(surfaceSlot)) return null; @@ -1273,7 +1284,8 @@ public sealed class GodotAdvHost : IHost var asset = _res.ResolveMovie(resourceId); if (asset == null) { Godot.GD.Print($"movie unresolved {scene}:0x{resourceId:x}"); return null; } StartMovie(asset, resourceId, surfaceSlot, movieFlags, syncMask, modal: false, - initialPositionMs: 0, out long? stopTimeMs, out _); + initialPositionMs: 0, startDelayMs: 0, presentationDurationMs: null, + movieMask: null, out long? stopTimeMs, out _); return stopTimeMs ?? 0; } @@ -1284,10 +1296,40 @@ public sealed class GodotAdvHost : IHost var asset = _res.ResolveMovie(resourceId); if (asset == null) { Godot.GD.Print($"movie unresolved {scene}:0x{resourceId:x}"); return null; } StartMovie(asset, resourceId, surfaceSlot, movieFlags, syncMask, modal: false, - initialPositionMs: positionMs, out long? stopTimeMs, out _); + initialPositionMs: positionMs, startDelayMs: 0, presentationDurationMs: null, + movieMask: null, out long? stopTimeMs, out _); return stopTimeMs ?? 0; } + public void PlayMovieMaskTransition(GfxState gfx, MovieMaskTransitionRequest request) + { + gfx.QueueMovieMaskTransition(request); + RgbaImage captured = CaptureRetainedRange( + gfx, request.SurfaceSlot, request.SourceRangeStart, request.SourceRangeCount); + byte initialFill = request.Mode == 1 ? (byte)0 : (byte)255; + PublishMaskedCapture( + request.SurfaceSlot, captured, + CreateFilledMask(request.Width, request.Height, initialFill), request); + + var asset = _res.ResolveMovie(request.ResourceId); + if (asset == null) + { + Godot.GD.Print($"movie mask unresolved {CurrentScene}:0x{request.ResourceId:x}"); + PublishMovieMaskTerminal(new MovieMaskPlayback(gfx, request, captured)); + return; + } + + var mask = new MovieMaskPlayback(gfx, request, captured); + bool started = StartMovie( + asset, request.ResourceId, request.SurfaceSlot, movieFlags: 6, syncMask: 0, + modal: false, initialPositionMs: 0, + startDelayMs: request.StartDelayMs, + presentationDurationMs: request.DurationMs, + movieMask: mask, out long? stopTimeMs, out _); + gfx.SetMovieStopTime(request.SurfaceSlot, stopTimeMs ?? 0); + if (!started) PublishMovieMaskTerminal(mask); + } + public bool IsMovieSurfaceActive(int surfaceSlot) => _movieSurfaces.IsActive(surfaceSlot); @@ -1327,7 +1369,8 @@ public sealed class GodotAdvHost : IHost try { if (!StartMovie(asset, resourceId, surfaceSlot, movieFlags, 0, modal: true, - initialPositionMs: 0, + initialPositionMs: 0, startDelayMs: 0, + presentationDurationMs: null, movieMask: null, out _, out long playbackId)) return; _timeline?.State("modal-movie-wait", new() { @@ -1361,7 +1404,8 @@ public sealed class GodotAdvHost : IHost } private bool StartMovie(AssetEntry asset, long resourceId, int surfaceSlot, long movieFlags, - long syncMask, bool modal, long initialPositionMs, + long syncMask, bool modal, long initialPositionMs, long startDelayMs, + long? presentationDurationMs, MovieMaskPlayback? movieMask, out long? stopTimeMs, out long playbackId) { stopTimeMs = null; @@ -1369,14 +1413,27 @@ public sealed class GodotAdvHost : IHost // asset on multiple surfaces; replacing one binding must not erase another binding's completion. MovieSurfaceBinding binding = _movieSurfaces.Begin(surfaceSlot, resourceId, out var replaced); playbackId = binding.PlaybackId; - if (replaced is { } prior) _main.CallDeferred("StopMovie", prior.PlaybackId); + if (replaced is { } prior) + { + AbandonMovieMaskPlayback(prior.PlaybackId); + ForgetMovieMaskSurface(prior.SurfaceSlot, prior.PlaybackId); + _main.CallDeferred("StopMovie", prior.PlaybackId); + } + if (movieMask != null) + lock (_movieMaskLock) + { + _movieMasksByPlayback[playbackId] = movieMask; + _movieMaskPlaybackBySurface[surfaceSlot] = playbackId; + } lock (_imageLock) { - _surfaceImages.Remove(surfaceSlot); + if (movieMask == null) _surfaceImages.Remove(surfaceSlot); _surfaceColorKeys.Remove(surfaceSlot); } // Movie surfaces inherit the selected game's primary size until a decoded frame supplies content. - _slotDims[surfaceSlot] = (_screenWidth, _screenHeight); + _slotDims[surfaceSlot] = movieMask == null + ? (_screenWidth, _screenHeight) + : (movieMask.Captured.Width, movieMask.Captured.Height); try { var movie = _res.ReadMovie(asset); @@ -1386,10 +1443,13 @@ public sealed class GodotAdvHost : IHost ["surface"] = surfaceSlot, ["file"] = movie.Name, ["flags"] = movieFlags, ["sync_mask"] = syncMask, ["modal"] = modal, ["initial_position_ms"] = Math.Max(0, initialPositionMs), + ["start_delay_ms"] = Math.Max(0, startDelayMs), + ["presentation_duration_ms"] = presentationDurationMs, + ["movie_mask"] = movieMask != null, }); bool started = _main.TryPlayMovie( movie.Bytes, movie.Name, playbackId, resourceId, asset.PackedId, movieFlags, - initialPositionMs, out stopTimeMs); + initialPositionMs, startDelayMs, presentationDurationMs, out stopTimeMs); if (!started) { stopTimeMs = 0; @@ -1399,6 +1459,7 @@ public sealed class GodotAdvHost : IHost } catch (System.Exception e) { + AbandonMovieMaskPlayback(playbackId); _movieSurfaces.Abandon(playbackId, out _); _slotDims.Remove(surfaceSlot); stopTimeMs = 0; @@ -1427,6 +1488,8 @@ public sealed class GodotAdvHost : IHost if (movieRelease.Kind == MovieSurfaceReleaseKind.Released) { var binding = movieRelease.Binding; + AbandonMovieMaskPlayback(binding.PlaybackId); + ForgetMovieMaskSurface(binding.SurfaceSlot, binding.PlaybackId); _timeline?.Event("movie-stop", new() { ["resource"] = binding.ResourceId, @@ -1577,6 +1640,8 @@ public sealed class GodotAdvHost : IHost } foreach (MovieSurfaceBinding binding in stoppedMovies) { + AbandonMovieMaskPlayback(binding.PlaybackId); + ForgetMovieMaskSurface(binding.SurfaceSlot, binding.PlaybackId); _timeline?.Event("movie-stop", new() { ["resource"] = binding.ResourceId, @@ -1592,12 +1657,25 @@ public sealed class GodotAdvHost : IHost // sample callback: the retained object keeps its surface binding while only the surface pixels change. public void PublishMovieFrame(long playbackId, string name, int assetId, RgbaImage frame) { + MovieMaskPlayback? maskPlayback; + lock (_movieMaskLock) _movieMasksByPlayback.TryGetValue(playbackId, out maskPlayback); + if (maskPlayback != null) + { + var request = maskPlayback.Request; + byte initialFill = request.Mode == 1 ? (byte)0 : (byte)255; + byte[] mask = MovieMaskSurface.ExtractGreen( + frame, request.Width, request.Height, initialFill); + PublishMaskedCapture(request.SurfaceSlot, maskPlayback.Captured, mask, request); + return; + } if (_movieSurfaces.PublishFrame(playbackId, frame, name, assetId)) System.Threading.Interlocked.Exchange(ref _presentRequested, 1); } public void NotifyMovieCompleted(long playbackId) { + MovieMaskPlayback? maskPlayback = RemoveMovieMaskPlayback(playbackId); + if (maskPlayback != null) PublishMovieMaskTerminal(maskPlayback); if (!_movieSurfaces.Complete(playbackId)) return; _timeline?.Event("movie-complete", new() { ["playback"] = playbackId }); _frameSignal.Set(); @@ -1606,6 +1684,85 @@ public sealed class GodotAdvHost : IHost private bool HasActiveMoviePresentation() => _movieSurfaces.HasActivePlayback; + private RgbaImage CaptureRetainedRange( + GfxState gfx, int targetSlot, long firstHandle, int count) + { + var dimensions = _slotDims.GetValueOrDefault( + targetSlot, (W: _screenWidth, H: _screenHeight)); + int width = System.Math.Max(0, dimensions.W); + int height = System.Math.Max(0, dimensions.H); + var captured = new RgbaImage(width, height, new byte[checked(width * height * 4)]); + IReadOnlyList visible = gfx.SnapshotVisibleObjects(_clock.NowMs); + RetainedSurfaceRasterizer.CompositeRange( + captured, visible, firstHandle, System.Math.Max(0, count), + item => + { + var raw = gfx.TryGet(item.Handle); + var resolved = raw != null + ? ResolveSurfaceTexture(raw.SourceSlot, item.SurfaceResId) + : ResolveResIdTexture(item.SurfaceResId); + return resolved == null + ? null + : RgbaSurfaceOps.WithColorKey(resolved.Value.Image, item.ColorKey); + }); + return captured; + } + + private static byte[] CreateFilledMask(int width, int height, byte fill) + { + var mask = new byte[checked(System.Math.Max(0, width) * System.Math.Max(0, height))]; + if (fill != 0) System.Array.Fill(mask, fill); + return mask; + } + + private void PublishMaskedCapture( + int surfaceSlot, RgbaImage captured, byte[] mask, MovieMaskTransitionRequest request) + { + RgbaImage image = MovieMaskSurface.Apply( + captured, mask, System.Math.Max(0, request.Width), + System.Math.Max(0, request.Height), request.X, request.Y); + lock (_imageLock) _surfaceImages[surfaceSlot] = image; + _slotDims[surfaceSlot] = (image.Width, image.Height); + System.Threading.Interlocked.Exchange(ref _presentRequested, 1); + _frameSignal.Set(); + } + + private void PublishMovieMaskTerminal(MovieMaskPlayback playback) + { + byte terminalFill = playback.Request.Mode == 1 ? (byte)255 : (byte)0; + PublishMaskedCapture( + playback.Request.SurfaceSlot, playback.Captured, + CreateFilledMask(playback.Request.Width, playback.Request.Height, terminalFill), + playback.Request); + playback.Gfx.CompleteMovieMaskTransition(playback.Request.SurfaceSlot); + } + + private MovieMaskPlayback? RemoveMovieMaskPlayback(long playbackId) + { + lock (_movieMaskLock) + { + if (!_movieMasksByPlayback.Remove(playbackId, out var playback)) return null; + return playback; + } + } + + private void AbandonMovieMaskPlayback(long playbackId) + { + MovieMaskPlayback? playback = RemoveMovieMaskPlayback(playbackId); + if (playback != null) + { + ForgetMovieMaskSurface(playback.Request.SurfaceSlot, playbackId); + playback.Gfx.CompleteMovieMaskTransition(playback.Request.SurfaceSlot); + } + } + + private void ForgetMovieMaskSurface(int surfaceSlot, long playbackId) + { + lock (_movieMaskLock) + if (_movieMaskPlaybackBySurface.GetValueOrDefault(surfaceSlot) == playbackId) + _movieMaskPlaybackBySurface.Remove(surfaceSlot); + } + private RgbaImage? Decode(AssetEntry asset) { lock (_imageLock) diff --git a/godot/IMovieDecoder.cs b/godot/IMovieDecoder.cs index 0e6fcf4..5a6fa1d 100644 --- a/godot/IMovieDecoder.cs +++ b/godot/IMovieDecoder.cs @@ -29,5 +29,6 @@ internal interface IMovieDecoder : IDisposable internal interface IMovieDecoderFactory { - IMovieDecoder Open(MoviePayload movie, long initialPositionMs = 0); + IMovieDecoder Open( + MoviePayload movie, long initialPositionMs = 0, long? presentationDurationMs = null); } diff --git a/godot/Main.cs b/godot/Main.cs index d40cfeb..5491799 100644 --- a/godot/Main.cs +++ b/godot/Main.cs @@ -2129,7 +2129,8 @@ public partial class Main : Godot.Control public bool TryPlayMovie(byte[] mpegBytes, string assetName, long playbackId, long resourceId, int assetId, long movieFlags, - long initialPositionMs, + long initialPositionMs, long startDelayMs, + long? presentationDurationMs, out long? stopTimeMs) { stopTimeMs = null; @@ -2138,7 +2139,7 @@ public partial class Main : Godot.Control var payload = new Age.Engine.Sys4.MoviePayload(assetName, mpegBytes); var runtime = MovieRuntime.Open( assetName, assetId, resourceId, payload, _movieDecoderFactory, movieFlags, - initialPositionMs); + initialPositionMs, startDelayMs, presentationDurationMs); stopTimeMs = runtime.Decoder.StopTimeMs; while (!_pendingMovies.TryAdd(playbackId, runtime)) if (_pendingMovies.TryRemove(playbackId, out var prior)) prior.Decoder.Dispose(); @@ -2180,6 +2181,9 @@ public partial class Main : Godot.Control if (_host == null) return; foreach (var (playbackId, movie) in _movies) { + long elapsedMs = (long)Stopwatch.GetElapsedTime( + movie.StartedAtTimestamp).TotalMilliseconds; + if (elapsedMs < movie.StartDelayMs) continue; bool frameWasAlreadySeen = _movieFrameSeen.Contains(playbackId); _movieAudio.TryGetValue(playbackId, out var audio); if (frameWasAlreadySeen) audio?.Update(); @@ -2196,8 +2200,7 @@ public partial class Main : Godot.Control audio?.Update(); } } - bool watchdogExpired = Stopwatch.GetElapsedTime(movie.StartedAtTimestamp).TotalMilliseconds - >= movie.WatchdogMs; + bool watchdogExpired = elapsedMs >= movie.WatchdogMs; if ((movie.Decoder.IsCompleted || watchdogExpired) && _movieCompletionNotified.Add(playbackId)) { if (movie.Decoder.Failure is { } failure) diff --git a/godot/MovieRuntime.cs b/godot/MovieRuntime.cs index c2ad3b9..5b8a650 100644 --- a/godot/MovieRuntime.cs +++ b/godot/MovieRuntime.cs @@ -5,21 +5,31 @@ using Age.Engine.Sys4; /// Presentation-side ownership for one decoder plus its fail-safe completion deadline. internal sealed record MovieRuntime(string Name, int AssetId, long ResourceId, IMovieDecoder Decoder, long MovieFlags, long InitialPositionMs, - long StartedAtTimestamp, long WatchdogMs) + long StartedAtTimestamp, long StartDelayMs, + long? PresentationDurationMs, long WatchdogMs) { public static MovieRuntime Open(string name, int assetId, long resourceId, MoviePayload payload, IMovieDecoderFactory factory, long movieFlags = 0, - long initialPositionMs = 0) + long initialPositionMs = 0, long startDelayMs = 0, + long? presentationDurationMs = null) { ArgumentNullException.ThrowIfNull(factory); - IMovieDecoder decoder = factory.Open(payload, Math.Max(0, initialPositionMs)); + long safeDelayMs = Math.Max(0, startDelayMs); + long? safeDurationMs = presentationDurationMs is >= 0 + ? Math.Max(0, presentationDurationMs.Value) + : null; + IMovieDecoder decoder = factory.Open( + payload, Math.Max(0, initialPositionMs), safeDurationMs); long remainingMs = decoder.StopTimeMs is >= 0 and var stopTime ? Math.Max(0, stopTime - decoder.InitialPositionMs) : -1; + long watchdogBasis = safeDurationMs ?? remainingMs; + if (watchdogBasis >= 0) watchdogBasis = checked(watchdogBasis + safeDelayMs); return new MovieRuntime(name, assetId, resourceId, decoder, movieFlags, decoder.InitialPositionMs, Stopwatch.GetTimestamp(), - remainingMs >= 0 - ? Math.Clamp(remainingMs + 2000, 5000, 300000) + safeDelayMs, safeDurationMs, + watchdogBasis >= 0 + ? Math.Clamp(watchdogBasis + 2000, 5000, 300000) : 30000); } } diff --git a/vm-map/opcodes.toml b/vm-map/opcodes.toml index 4849519..fe06114 100644 --- a/vm-map/opcodes.toml +++ b/vm-map/opcodes.toml @@ -6654,8 +6654,8 @@ abi_source = "kelebek+decode-validated" [opcode.semantics] name = "play-movie-mask-transition" category = "draw" -summary = "Open a movie into a scratch surface, retime it to the requested duration, copy each decoded frame's green channel into a byte-per-pixel mask, and register a blocking retained-surface transition from an old object range to a newly drawn range within the supplied rectangle." -details = "The RGB24 sample callback copies the bottom-up green byte verbatim. In the native 32-bit mask compositor, captured RGB remains unchanged and the output alpha byte is the high byte of `((source_argb >> 8) * mask_byte)`; this records the exact packed-integer operation, including its low-color carry, rather than approximating it as a scalar luminance crossfade. Mode 1 initializes the mask to 0 and completes at 255; all other modes initialize at 255 and complete at 0." +summary = "Capture an old retained-object range into the newly drawn scratch surface, retime a movie to the requested duration, and use each decoded frame's green channel as a byte-per-pixel alpha mask within the supplied rectangle." +details = "The RGB24 sample callback copies the bottom-up green byte verbatim. In the native 32-bit mask compositor, captured RGB remains unchanged and the output alpha byte is the high byte of `((source_argb >> 8) * mask_byte)`; this records the exact packed-integer operation, including its low-color carry, rather than approximating it as a scalar luminance crossfade. Mode 1 initializes the mask to 0 and completes at 255; all other modes initialize at 255 and complete at 0. Port status (2026-07-29): implemented through the existing retained-range software rasterizer and dynamic scratch-surface path. The decoder honors start delay and native stop/duration retiming, publishes logical top-down green masks, and holds opcode 0x21c until movie completion; click completion remains limited to type-0 transitions." noop_headless = false source = "investigation" confidence = "high"