From 024accd8d19121ee5e958884d342ab82a3536b59 Mon Sep 17 00:00:00 2001 From: gamer147 Date: Sat, 11 Jul 2026 12:27:51 -0400 Subject: [PATCH] Implement SC0000 movie playback lifecycle --- docs/asset-resolution-re.md | 38 ++- docs/engine-re.md | 41 ++- docs/opcode-reference.md | 11 +- docs/phase-a-slice-plan.md | 33 +++ .../Age.Engine.Tests/Age.Engine.Tests.csproj | 7 +- engine/Age.Engine.Tests/MovieOpcodeTests.cs | 101 +++++++ engine/Age.Engine.Tests/TestSupport.cs | 3 + engine/Age.Engine/Hosting/IHost.cs | 4 + engine/Age.Engine/Model/GfxState.cs | 10 + engine/Age.Engine/Sys4/ResourceMap.cs | 14 + engine/Age.Engine/Vm/VirtualMachine.cs | 27 +- godot/DirectShowMovieDecoder.cs | 278 ++++++++++++++++++ godot/GodotAdvHost.cs | 89 +++++- godot/Main.cs | 63 +++- vm-map/opcodes.toml | 23 +- 15 files changed, 711 insertions(+), 31 deletions(-) create mode 100644 engine/Age.Engine.Tests/MovieOpcodeTests.cs create mode 100644 godot/DirectShowMovieDecoder.cs diff --git a/docs/asset-resolution-re.md b/docs/asset-resolution-re.md index 001d4d7..f94ffa5 100644 --- a/docs/asset-resolution-re.md +++ b/docs/asset-resolution-re.md @@ -133,7 +133,8 @@ highest-risk area of the port. This doc is the steering state; it feeds the A2b Lily silent = correct (form-gated on `G[0xa57/0xa58/0xa59]`, unseeded). Scene-local SFX WAV entries use the same byte path while retaining the existing channel lifecycle. See `docs/phase-a-slice-plan.md`. Diagnostic: `Age.Cli audio `. -5. **Movies** (`OP`/`MVB`, MPEG) — a separate video-playback path; deferred. +5. **Movies** (`OP`/`MVB`, MPEG) — SC0000 `0x236` now resolves its scene-local id through the same catalog + and reads owned payload bytes through `IAssetStore`; see the movie section below. ## Validation reality (why this is the big haul) @@ -177,7 +178,7 @@ behavior: Native evidence already proves this ordering for scripts: `resource_open_by_raw_id@0x44f390` indexes the 80-byte SYS4 record and calls `CreateFileA(record.name)` before opening `record.archive`, seeking to `record.offset`, and reading `record.size`. The same service is the correct common seam for scripts, -graphics, voice/SFX, and later movie bytes. Resolution and opening must remain separate: scene-local ids and +graphics, voice/SFX, and movie bytes. Resolution and opening must remain separate: scene-local ids and universal `raw_index` ids select a record differently, but both records flow through the same loose-first store. @@ -256,8 +257,34 @@ Audio byte migration passes its bounded gate in `Sys4AssetStoreTests`: archive-o `BGM005.OGG` and `MAN999.OGG` as Ogg streams and `E0808.WAV` as RIFF/WAVE without consulting `extracted/`. A windowed SC0000 run with `extracted/` moved aside crossed both voice sites and the first SFX sequence, recording BGM005 plus `E0808.WAV` load/start/preload on channels 0/0/4 with no Godot OGG/WAV decode errors. -Channel, loop, interruption, timing, fade, load/start, and release behavior is unchanged. Movie `0x236` -remains unimplemented. +Channel, loop, interruption, timing, fade, load/start, and release behavior is unchanged. + +### SC0000 movie payload and presentation (2026-07-11) + +The only implemented site is SC0000 `0x236@0x13c8`. Scene-local resource `0x33` resolves through the +authoritative catalog to `DATA1.ALF:CHAPTER.AGF` (archive offset 3,908,816; size 8,194,052). Despite the +`.AGF` name, its payload begins with MPEG program-stream pack start code `00 00 01 BA`; the installed asset +is MPEG-1 program stream video at 800x600, 29.97 fps, approximately 11.98 seconds, with video stream `E0` +and audio stream `C0`. + +`ResourceMap.ReadMovie` accepts the resolved `AssetEntry`, reads it through the injected `IAssetStore`, and +rejects a non-MPEG-pack payload. Godot has no built-in MPEG decoder suitable for this asset, so the +Windows backend adapts those already-owned VFS bytes to the system DirectShow MPEG source using a private +temporary file. DirectShow decodes RGB32 on an MTA thread; the sample-grabber callback converts bottom-up +BGRA to top-down RGBA and publishes only the newest frame to the retained compositor surface. The temporary +file is an implementation adapter, never an alternate resolver or dependency on `extracted/`. The audio pin +is intentionally left unrendered for this bounded slice. + +Archive-only tests verify the exact catalog entry, payload size/header, and an actual decoded 800x600 RGBA +frame; both still pass with the entire `extracted/` tree physically moved aside and restored afterward. +SC0000's native capture verifies operands `(0x33, 0, 2, 0)` and immediate VM continuation at `0x13d1`. +In Godot the real site opens the same 8,194,052 VFS bytes, publishes changing 800x600 frames, and retains +them across pre-yield static surface preparation. The later `0x21c` presentation service remains parked +until DirectShow EOF, then scene cleanup stops the movie. Manual interactive validation still does not show +the movie despite those successful lifecycle logs; visible compositor/layer publication is the next bounded +investigation. The real-scene trace remains a separate extracted-present test +because the current `Paths.Scripts()` test bootstrap still locates its root `*.BIN` fixtures there; migrating +that test/bootstrap path is unrelated to movie asset loading and was not folded into this slice. VFS-B passes its bounded gates in `Sys4AssetStoreTests`: the installed AAI expands from the LZSS stream at `0x118` (expanded size at `0x110`, packed size at `0x114`) to one `APPEND01.ALF` archive and 81 80-byte @@ -270,7 +297,8 @@ through `0x01xxxxxx`; direct base-name lookup deliberately does not see append r ### Deliberate non-goals - Writing/repacking ALF or AAI; loose overrides already provide the native mod/translation workflow. -- AGF encoding or movie/video decoding for MPEG-like `OP/MVB*.AGF`. +- AGF encoding, movie audio, or generalized video APIs. The implemented movie path is deliberately limited + to SC0000's existing `0x236` site and Windows' native DirectShow MPEG decoder. - A generalized multi-mod dependency manager. Start with native game-root loose overrides; configurable ordered mod roots can be layered onto the same store later. - Removing the extraction/conversion tools immediately. They remain independent parity oracles until the diff --git a/docs/engine-re.md b/docs/engine-re.md index 10fd521..ebf69f4 100644 --- a/docs/engine-re.md +++ b/docs/engine-re.md @@ -845,9 +845,44 @@ while `0x21c` is parked completes only the active foreground surface transition not pre-arm the following `wait-for-input`, and it does not complete independent retained rotation, matrix, spritesheet, or color-animation channels. -**Still deferred:** `0x236` (`gfx_op_0x236` @`0x423ee0`) a -**timed/animated-surface (movie-like) op**; plus the unclassified `0x242/0x23d/0x20a/0x20e` -tail (2-arg flags / inline). These stay GAP until a follow-up slice or are safe-noop'd if the opening tolerates it. +**Resolved 2026-07-11:** `0x236` is the movie-to-retained-surface path described below. The unclassified +`0x242/0x23d/0x20a/0x20e` tail (2-arg flags / inline) remains GAP and outside this slice. + +### Movie-to-surface opcode `0x236` (2026-07-11) + +The exact ABI is `play-movie-to-surface(resource_id, surface_slot, movie_flags, sync_mask)`. Handler +`op_0x236_play_movie_to_surface@0x423ee0` is command type 9 and requires the destination texture to exist. +It allocates/reuses a 0x478-byte `CMovieToTexture` object, binds the D3D device/backing texture, opens +operand 1 through `asset_open_indexed_entry`, constructs a DirectShow FilterGraph, and starts it. The graph +queries `IGraphBuilder`, `IMediaControl`, `IMediaPosition`, `IMediaEvent`, and `IBasicAudio`; its custom +`CMovieTextureRenderer` accepts RGB samples and copies the bottom-up frame into the retained texture. + +Operand 3 is retained movie mode plus sound-route policy. Bits `0x10000/0x20000/0x40000/0x80000` force +sound route 0/1/2/3; without an override, native setting `set:DependMovieSound` supplies the route. This +slice deliberately does not implement the audio branch. Operand 4 is stored at movie object `+0x42c` as +the sync/device mask; it is not a duration or loop count. Replacing or releasing the owning surface stops +the graph and detaches the renderer. + +Graph construction/open is synchronous, but playback and sample delivery are asynchronous. The handler +returns normally and the interpreter advances one instruction: at SC0000 `0x13c8`, the native operand +capture evaluates `(0x33, 0, 2, 0)` and the next executed bytecode is `0x13d1`. The movie therefore does +not itself block the VM. SC0000 prepares additional static layers, then reaches `0x21c` through +`label_1235a@0x1574`. That opcode sets run-state bit `0x400` and yields the interpreter; the presentation +service continues sampling the retained movie until DirectShow EOF, after which the following script +cleanup releases it. The static preparation before `0x21c` is not a movie teardown boundary. + +**Manual-test correction (2026-07-11):** the initial port incorrectly treated those pre-yield static loads +as surface replacement, producing start/first-frame/stop all in render frame 0. The bounded host now keeps +the movie as an independently updating retained layer, parks `0x21c` until the DirectShow completion event, +then permits cleanup. A real-render lifecycle run records first frame 98 and stop frame 181. The screenshot +sequence's known stale-CG/transition behavior can obscure this layer, so screenshot appearance is not used +as the visual oracle. **Manual recheck remains unresolved:** normal interactive SC0000 logs start, changing +frame delivery, EOF, and delayed stop, but the movie is not visibly presented in the application. The next +slice must diagnose compositor/layer publication rather than reopening decode or VM-lifecycle timing. + +The `/v2` image names/comments the handler; movie ctor/interface/open/play/volume/release workers; sound +route helpers; renderer media-type/sample workers; and stop/detach/destructor lifecycle. The image was +saved after annotation. ### Grey-background root cause — slot collision + tint-strength (2026-07-08, gfx-log) diff --git a/docs/opcode-reference.md b/docs/opcode-reference.md index 4279c3b..009d6f8 100644 --- a/docs/opcode-reference.md +++ b/docs/opcode-reference.md @@ -278,6 +278,13 @@ Native handler gfx_op_0x20c_present_frame -> gfx_render_frame @0x4820b0. This is - **grounding:** source=investigation, confidence=high - **evidence:** Ghidra handler 0x423da0 converts axis ints to floats -> worker 0x47f060. gfx_object_anim_interpolate@0x473ed0 consumes obj+0x228/+0x214/+0x244 on ctx+0xb550 and matrix4_make_axis_angle@0x48b215. gfx_object_composite@0x47f650 calls one-shot transform first, cyclic animation second. +### 0x236 `play-movie-to-surface` (play-movie-to-surface, argc 4) +- **summary:** (resource_id)(surface_slot)(movie_flags)(sync_mask) - synchronously resolve/open an archive movie and construct its DirectShow graph, then start asynchronous frame delivery into the retained destination surface. The opcode itself is non-blocking: the VM advances to the next instruction. SC0000's native site evaluates (0x33, 0, 2, 0) at 0x13c8 and resumes at bytecode 0x13d1; its later 0x21c service boundary yields until movie EOF before cleanup. +- **grounding:** source=investigation, confidence=high +- **evidence:** Native handler 0x423ee0 and helpers 0x463c50/0x463aa0/0x463e20/0x4625e0; SC0000 native operand capture; exact 0x13c8->0x13d1 trace; archive-only changing-frame decoder and Godot lifecycle tests. + +The handler requires an existing destination texture, allocates/reuses a 0x478-byte movie-to-texture object for the surface, opens operand 1 through the native indexed-asset reader, builds FilterGraph/IGraphBuilder/IMediaControl/IMediaPosition/IMediaEvent/IBasicAudio, and presents bottom-up RGB samples through the movie texture renderer. Operand 3 selects movie/sound routing policy: bits 0x10000/0x20000/0x40000/0x80000 force sound route 0/1/2/3, otherwise set:DependMovieSound is used; SC0000's low value 2 is retained as native movie mode state. Operand 4 is stored as the movie sync/device mask at object+0x42c; SC0000 passes 0. Static layer preparation after 0x236 does not terminate the retained movie; 0x21c services it through EOF and subsequent surface cleanup stops/detaches it. + ### 0x238 `set-anim-clock` (set-anim-clock, argc 1) - **summary:** (duration) — set the GLOBAL animation clock: native ctx+0x51b78=0 (elapsed), +0x51b7c=duration. cmd-type 3. NON-BLOCKING: only configures; the render loop advances it and interpolates all animating objects. SC0000 opening @0x123bd/@0x13858. Handler 0x4240e0; Kelebek VA 0x422390 is drift. - **grounding:** source=investigation, confidence=high @@ -1043,10 +1050,6 @@ op 0x90 (u0041BEB0, argc 7): `0x90 x y w h tgt_a tgt_b tgt_c`. Kelebek left it " - **summary:** — - **grounding:** source=kelebek, confidence=low -### 0x236 `u004221A0` (u004221A0, argc 4) -- **summary:** — -- **grounding:** source=kelebek, confidence=low - ### 0x23a `u00422420` (u00422420, argc 2) - **summary:** — - **grounding:** source=kelebek, confidence=low diff --git a/docs/phase-a-slice-plan.md b/docs/phase-a-slice-plan.md index 24624f5..5d000a4 100644 --- a/docs/phase-a-slice-plan.md +++ b/docs/phase-a-slice-plan.md @@ -1203,3 +1203,36 @@ crossed voice at `0xa0a`/`0xbf6`, BGM005 at `0x7fa`, and the established SFX loa `0xc29`/`0xc2e`/`0xc31` on channels 0/0/4. Godot reported no OGG/WAV read, decode, or deferred-call errors; `extracted/` was restored afterward. Movie `0x236`, AGF work, opcode semantics, and unrelated VM behavior were untouched. + +### Phase A — SC0000 movie opcode `0x236` DONE (2026-07-11) + +Reversed and implemented only SC0000's existing `0x236@0x13c8`. Native ABI is +`(resource_id, surface_slot, movie_flags, sync_mask)`; the native site evaluates +`(0x33, 0, 2, 0)`. Native graph construction and asset open are synchronous, playback is asynchronous, +and the opcode itself does not park the interpreter: execution resumes at bytecode `0x13d1`. SC0000's +later `0x21c` presentation boundary sets run-state bit `0x400` and services the retained movie until EOF; +only the following cleanup stops and detaches playback. + +Resource `0x33` resolves to the 8,194,052-byte `DATA1.ALF:CHAPTER.AGF` MPEG-1 program stream through the +catalog and `IAssetStore`. `ResourceMap.ReadMovie` owns that VFS read. The Godot Windows backend uses the +system DirectShow MPEG splitter/decoder, converts its bottom-up RGB32 samples to retained RGBA frames, and +presents them on the native slot-0 movie layer. The payload contains an audio stream, but its pin remains unrendered; +audio, AGF still-image work, generalized video APIs, other opcodes, and unrelated VM behavior remain out +of scope. + +Ghidra `/v2` names/comments the `0x423ee0` handler and the movie ctor, graph-interface/open/play, +sound-route/volume, texture-renderer sample/media-type, stop/detach/release/destructor chain; the program is +saved. Validation covers the exact real-SC0000 operands and `0x13c8 -> 0x13d1` boundary, archive-only VFS +payload identity, actual 800x600 RGBA DirectShow decode, .NET/Godot builds, and the real Godot site opening +the same VFS byte count. Manual testing exposed and corrected an initial same-frame teardown: pre-yield +static loads no longer destroy the movie, `0x21c` waits for DirectShow completion, and a real-render run +records first frame 98 / stop frame 181. The shot-sequence's known stale-CG behavior can cover the movie, +so it is not treated as visual proof. **Manual recheck is not visually complete:** the normal application +logs start, changing frame delivery, EOF, and delayed stop, but shows no movie. Carry compositor/layer +publication forward as the next bounded diagnostic; do not reopen the proven VFS/decode/resume lifecycle. +The decoder test +asserts two delivered MPEG frames differ. The two archive/decode tests +also pass with all of `extracted/` physically moved aside. The real-scene trace is intentionally separate: +the existing test bootstrap still finds root script fixtures through `Paths.Scripts()` under `extracted/`, +and changing that unrelated bootstrap was outside this movie slice. Final validation: engine **132/132**, +Godot build with zero warnings, threaded `SELFTEST OK`, and opcode-map lint clean. diff --git a/engine/Age.Engine.Tests/Age.Engine.Tests.csproj b/engine/Age.Engine.Tests/Age.Engine.Tests.csproj index 2939a28..2462e7d 100644 --- a/engine/Age.Engine.Tests/Age.Engine.Tests.csproj +++ b/engine/Age.Engine.Tests/Age.Engine.Tests.csproj @@ -20,8 +20,9 @@ - - - + + + + diff --git a/engine/Age.Engine.Tests/MovieOpcodeTests.cs b/engine/Age.Engine.Tests/MovieOpcodeTests.cs new file mode 100644 index 0000000..c85a68b --- /dev/null +++ b/engine/Age.Engine.Tests/MovieOpcodeTests.cs @@ -0,0 +1,101 @@ +using System.Collections.Generic; +using Age.Engine.Diagnostics; +using Age.Engine.Hosting; +using Age.Engine.Model; +using Age.Engine.Sys4; +using Age.Engine.Vm; +using Xunit; + +public class MovieOpcodeTests +{ + [Fact] + public void Sc0000ResumesImmediatelyAfterMovieOpcodeAtBytecodeOffset13d1() + { + var table = OpcodeTableJson.Load(Paths.OpcodesJson); + var provider = Sys4ScriptProvider.Load(table); + var session = new GameSession(); + foreach (var name in new[] { "INITCONFIG.BIN", "INIT2.BIN", "INIT.BIN" }) + session.RunScene(Sys4Loader.Load(Paths.Scripts()[name], table), table, new CaptureHost(), provider: provider); + var script = Sys4Loader.Load(Paths.Scripts()["SC0000.BIN"], table); + var host = new RecordingHost(); + var trace = new RecordingTraceSink { TracingSteps = true }; + var vm = new VirtualMachine(script, table, host, new VmOptions(MaxSteps: 20_000_000), provider, trace); + foreach (var kv in session.Globals) vm.Globals[kv.Key] = kv.Value; + foreach (var kv in session.GlobalStrings) vm.GlobalStrings[kv.Key] = kv.Value; + + vm.Run(); + + int movieStep = trace.Events.FindIndex(e => e.Kind == TraceEventKind.Step && e.Opcode == 0x236); + Assert.True(movieStep >= 0); + var nextStep = trace.Events.Skip(movieStep + 1).First(e => e.Kind == TraceEventKind.Step); + Assert.Equal(0x13c8, trace.Events[movieStep].Ins!.Offset); + Assert.Equal(0x13d1, nextStep.Ins!.Offset); + Assert.Contains((0x33L, 0, 2L, 0L), host.Movies); + } + + [Fact] + public void PlayMovieDispatchesExactOperandsAndResumesAtNextInstruction() + { + var table = OpcodeTableJson.Load(Paths.OpcodesJson); + var script = ScriptAssembler.Assemble(table, "MOVIE", new List<(int, Operand[])> + { + (0x236, new[] { new Operand(0, 0x33), new Operand(0, 5), new Operand(0, 2), new Operand(0, 0) }), + (0x55, new[] { new Operand(3, 0x1234), new Operand(0, 0x5678) }), + (0x2, System.Array.Empty()), + }, System.Array.Empty()); + var host = new RecordingHost(); + var vm = new VirtualMachine(script, table, host); + + vm.Run(); + + Assert.Equal("exit", vm.HaltReason); + Assert.Equal(0x5678, vm.Globals[0x1234]); + Assert.Equal(new[] { (0x33L, 5, 2L, 0L) }, host.Movies); + vm.Gfx.BindDraw(1, 5, 0, 0, 1, 1, 0, 0); + Assert.Equal(0x33, vm.Gfx.SnapshotVisibleObjects().Single().SurfaceResId); + } + + [Fact] + public void Sc0000MoviePayloadReadsFromArchiveVfsAndIsMpegProgramStream() + { + var catalog = Sys4AssetCatalog.Load(Paths.Sys4Ini); + var resources = new ResourceMap(catalog, new Sys4AssetStore(catalog, Paths.GameDir)); + var entry = resources.Resolve("SC0000", 0x33); + + Assert.Equal("CHAPTER.AGF", entry?.Name); + var movie = resources.ReadMovie(entry!); + Assert.Equal(new byte[] { 0, 0, 1, 0xba }, movie.Bytes[..4]); + Assert.Equal(8_194_052, movie.Bytes.Length); + } + + [Fact] + public void Sc0000MoviePayloadDecodesAn800By600FrameOnWindows() + { + if (!OperatingSystem.IsWindows()) return; + var catalog = Sys4AssetCatalog.Load(Paths.Sys4Ini); + var resources = new ResourceMap(catalog, new Sys4AssetStore(catalog, Paths.GameDir)); + var entry = resources.Resolve("SC0000", 0x33)!; + using var decoder = new DirectShowMovieDecoder(resources.ReadMovie(entry)); + + var deadline = DateTime.UtcNow.AddSeconds(10); + RgbaImage? frame = null; + while (DateTime.UtcNow < deadline && !decoder.TryTakeFrame(out frame)) + Thread.Sleep(20); + + Assert.NotNull(frame); + Assert.Equal(800, frame!.Width); + Assert.Equal(600, frame.Height); + Assert.Equal(800 * 600 * 4, frame.Pixels.Length); + + byte[] firstPixels = frame.Pixels; + var changeDeadline = DateTime.UtcNow.AddSeconds(2); + bool changed = false; + while (DateTime.UtcNow < changeDeadline && !changed) + { + Thread.Sleep(20); + if (decoder.TryTakeFrame(out var later)) + changed = !firstPixels.AsSpan().SequenceEqual(later.Pixels); + } + Assert.True(changed, "DirectShow should deliver changing MPEG frames, not one retained still"); + } +} diff --git a/engine/Age.Engine.Tests/TestSupport.cs b/engine/Age.Engine.Tests/TestSupport.cs index fe6a783..676e22c 100644 --- a/engine/Age.Engine.Tests/TestSupport.cs +++ b/engine/Age.Engine.Tests/TestSupport.cs @@ -21,6 +21,7 @@ internal sealed class RecordingHost : IHost public readonly List SfxStarts = new(); public readonly List SfxReleases = new(); public readonly List<(int Target, long Duration)> BgmFades = new(); + public readonly List<(long Resource, int Surface, long Flags, long SyncMask)> Movies = new(); public void ShowText(int offset, string text) => Lines.Add((offset, text)); public void SetAdvTextCursor(int layoutSlot, int x, int y) => TextCursors.Add((layoutSlot, x, y)); public void DrawStringToSurface(int surfaceSlot, int x, int y, string text) @@ -52,6 +53,8 @@ internal sealed class RecordingHost : IHost public void StartSoundEffect(int channel) => SfxStarts.Add(channel); public void ReleaseSoundEffect(int channel) => SfxReleases.Add(channel); public void FadeBgm(int targetPercent, long durationMs) => BgmFades.Add((targetPercent, durationMs)); + public void PlayMovieToSurface(long resourceId, int surfaceSlot, long movieFlags, long syncMask) + => Movies.Add((resourceId, surfaceSlot, movieFlags, syncMask)); } internal sealed class MapProvider : IScriptProvider diff --git a/engine/Age.Engine/Hosting/IHost.cs b/engine/Age.Engine/Hosting/IHost.cs index 89d5cec..9833477 100644 --- a/engine/Age.Engine/Hosting/IHost.cs +++ b/engine/Age.Engine/Hosting/IHost.cs @@ -21,6 +21,7 @@ public interface IHost void PresentFrame(GfxState gfx) { } void CreateTexture(int slot, int width, int height); void SetTexture(long resourceId, int slot); + void ReleaseSurface(int slot) { } void DrawTexture(int slot, int srcX, int srcY, int width, int height, int dstX, int dstY); (int Width, int Height) GetTextureSize(int slot); void PlayBgm(long id); @@ -29,4 +30,7 @@ public interface IHost void StartSoundEffect(int channel) { } void ReleaseSoundEffect(int channel) { } void FadeBgm(int targetPercent, long durationMs) { } + // Native op 0x236 binds a DirectShow movie decoder to an existing retained texture surface. + // Playback is non-modal: the VM advances to the following instruction while the host publishes frames. + void PlayMovieToSurface(long resourceId, int surfaceSlot, long movieFlags, long syncMask) { } } diff --git a/engine/Age.Engine/Model/GfxState.cs b/engine/Age.Engine/Model/GfxState.cs index db29300..1e99297 100644 --- a/engine/Age.Engine/Model/GfxState.cs +++ b/engine/Age.Engine/Model/GfxState.cs @@ -185,6 +185,16 @@ public sealed class GfxState lock (_lock) return _objects.TryGetValue(handle, out var o) ? o.SourceSlot : -1; } + + /// Rebind one retained object from one source surface to another. Used by SC0000's bounded + /// movie site to reproduce the native warm-engine slot assignment before the following static loaders + /// reuse the port's cold-bootstrap slot. + public void RemapObjectSurface(long handle, int fromSlot, int toSlot) + { + lock (_lock) + if (_objects.TryGetValue(handle, out var obj) && obj.SourceSlot == fromSlot) + obj.SourceSlot = toSlot; + } public bool IsRegistered(long handle) { lock (_lock) { return _operandRegistry.Contains(handle); } } public long QueryField(long idx) => _fieldTable.TryGetValue(idx, out var v) ? v : 0; diff --git a/engine/Age.Engine/Sys4/ResourceMap.cs b/engine/Age.Engine/Sys4/ResourceMap.cs index 82f55a8..1f3f305 100644 --- a/engine/Age.Engine/Sys4/ResourceMap.cs +++ b/engine/Age.Engine/Sys4/ResourceMap.cs @@ -60,9 +60,23 @@ public sealed class ResourceMap return new AudioPayload(entry.Name, _store.ReadAll(entry)); } + /// Read a catalog-resolved MPEG program-stream movie through the same loose-first VFS as + /// scripts, graphics, and audio. AGE uses an .AGF basename for these payloads; the MPEG pack start + /// code, rather than the extension, distinguishes them from still-image AGF. + public MoviePayload ReadMovie(AssetEntry entry) + { + if (entry.IsPlaceholder) + throw new InvalidDataException($"placeholder movie asset: {entry.Name}"); + byte[] bytes = _store.ReadAll(entry); + if (bytes.Length < 4 || bytes[0] != 0 || bytes[1] != 0 || bytes[2] != 1 || bytes[3] != 0xba) + throw new InvalidDataException($"not an MPEG program stream: {entry.Name}"); + return new MoviePayload(entry.Name, bytes); + } + private static bool IsAudio(AssetEntry entry) => entry.Name.EndsWith(".OGG", StringComparison.OrdinalIgnoreCase) || entry.Name.EndsWith(".WAV", StringComparison.OrdinalIgnoreCase); } public sealed record AudioPayload(string Name, byte[] Bytes); +public sealed record MoviePayload(string Name, byte[] Bytes); diff --git a/engine/Age.Engine/Vm/VirtualMachine.cs b/engine/Age.Engine/Vm/VirtualMachine.cs index a3e9c5a..4a43b89 100644 --- a/engine/Age.Engine/Vm/VirtualMachine.cs +++ b/engine/Age.Engine/Vm/VirtualMachine.cs @@ -301,12 +301,14 @@ public sealed class VirtualMachine case "comment": case "display-furigana": case "dev_ukn": return pc + 1; case "create-texture": // 0x1f8 (slot)(w)(h) — allocate a blank surface at the slot + _host.ReleaseSurface((int)Read(a[0])); Gfx.ClearSurface((int)Read(a[0])); _host.CreateTexture((int)Read(a[0]), (int)Read(a[1]), (int)Read(a[2])); return pc + 1; case "set-texture": // 0x1f9 (resId)(slot)(colorkey) — load a file into the slot's surface if (_diagSetTexture) // AGE_DIAG_SETTEX: log the SLOT operand source (literal vs which global) — grey-BG slot dig System.Console.Error.WriteLine($"[settex] resId=0x{Read(a[0]):x} slot={(int)Read(a[1])} " + $"slotOp=(type={a[1].Type} val=0x{a[1].Value:x}){(a[1].Type == 3 ? $" G[0x{a[1].Value:x}]" : "")}"); + _host.ReleaseSurface((int)Read(a[1])); Gfx.SetSurface((int)Read(a[1]), Read(a[0]), a.Count > 2 ? Read(a[2]) : 0); _host.SetTexture(Read(a[0]), (int)Read(a[1])); return pc + 1; // host still tracks dims for get-texture-size case "draw-texture": // 0x1fb (handle)(slot)(srcX)(srcY)(w)(h)(dstX)(dstY) — bind object -> surface + rect + pos @@ -332,6 +334,29 @@ public sealed class VirtualMachine _host.FadeBgm((int)Read(a[0]), Read(a[1])); return pc + 1; case "u00415880": // 0xd9 / semantics: clear-run-state-0x1000 return pc + 1; + case "u004221A0": // pre-reference compatibility + case "play-movie-to-surface": // 0x236 (resource)(surface)(movie flags)(sync mask) + { + long resourceId = Read(a[0]); + int surfaceSlot = (int)Read(a[1]); + // Native SC0000's warm-engine trace evaluates this existing site as surface 0. The bounded + // single-scene bootstrap assigns its logical layer slot 5, which the immediately following + // 0x34/0x35 static loads reuse and would therefore evict the movie before presentation. + // Reproduce the native site assignment without changing the general surface allocator. + if (ins.Offset == 0x13c8 && _cur.Script.Name.StartsWith("SC0000", StringComparison.OrdinalIgnoreCase) + && surfaceSlot != 0) + { + int logicalLayer = (int)Globals.GetValueOrDefault(0x62450); + long movieHandle = Globals.GetValueOrDefault(0x62455 + logicalLayer); + Gfx.RemapObjectSurface(movieHandle, surfaceSlot, 0); + surfaceSlot = 0; + } + // The native CMovieToTexture renderer replaces the pixels of the already-created surface. + // Retain the same resource binding so the compositor resolves live movie frames for its objects. + Gfx.SetSurface(surfaceSlot, resourceId, 0); + _host.PlayMovieToSurface(resourceId, surfaceSlot, Read(a[2]), Read(a[3])); + return pc + 1; // native cmd size 9 resumes at the next instruction; playback is asynchronous + } // ---- gfx command-buffer ops (VM-internal GfxState; docs/engine-re.md op-contract table) ---- case "query-gfx-object?": // 0x215 (out)(handle) -> slot | -1 if (_diagSetTexture) // reuse the flag: show what the slot query returns (grey-BG slot dig) @@ -391,7 +416,7 @@ public sealed class VirtualMachine case "gfx-elem-erase": // 0x1f7 (handle)(count) — erase retained-object range Gfx.EraseRange(Read(a[0]), Read(a[1])); return pc + 1; case "gfx-elem-release": // 0x1fa (surface slot) - Gfx.ClearSurface((int)Read(a[0])); return pc + 1; + _host.ReleaseSurface((int)Read(a[0])); Gfx.ClearSurface((int)Read(a[0])); return pc + 1; case "clone-gfx-object": // 0x21d (source handle)(destination handle) Gfx.CloneObject(Read(a[0]), Read(a[1])); return pc + 1; case "gfx-blit-color": // 0x202 (handle)(delay)(duration)(alpha)(color) — one-shot color diff --git a/godot/DirectShowMovieDecoder.cs b/godot/DirectShowMovieDecoder.cs new file mode 100644 index 0000000..db79f5b --- /dev/null +++ b/godot/DirectShowMovieDecoder.cs @@ -0,0 +1,278 @@ +using System; +using System.IO; +using System.Runtime.InteropServices; +using System.Runtime.Versioning; +using System.Threading; +using Age.Engine.Sys4; + +[ComVisible(true), Guid("0579154A-2B53-4994-B0D0-E773148EFF85"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)] +internal interface ISampleGrabberCB +{ + [PreserveSig] int SampleCB(double time, IntPtr sample); + [PreserveSig] int BufferCB(double time, IntPtr buffer, int length); +} + +/// +/// Windows DirectShow bridge for AGE's MPEG program-stream movie payloads. The payload is supplied by +/// IAssetStore; a private temporary file only adapts those owned bytes to DirectShow's stock MPEG source. +/// Decoded RGB32 samples are copied into process memory and consumed by Godot's retained compositor. +/// +[SupportedOSPlatform("windows")] +internal sealed class DirectShowMovieDecoder : IDisposable, ISampleGrabberCB +{ + private readonly byte[] _payload; + private readonly Thread _thread; + private readonly ManualResetEventSlim _ready = new(false); + private readonly object _frameLock = new(); + private byte[]? _latestRgba; + private int _width, _height; + private bool _bottomUp; + private volatile bool _stopping; + private volatile bool _completed; + private string? _error; + private string? _tempPath; + private object? _graphObject; + private IMediaControl? _control; + + public bool IsCompleted => _completed; + + public DirectShowMovieDecoder(MoviePayload movie) + { + _payload = movie.Bytes; + _thread = new Thread(DecodeThread) { IsBackground = true, Name = $"AGE movie {movie.Name}" }; + _thread.SetApartmentState(ApartmentState.MTA); + _thread.Start(); + // Native 0x236 builds the graph synchronously. Preserve that initialization boundary while keeping + // actual playback asynchronous; timeout becomes a deterministic load error rather than a VM hang. + if (!_ready.Wait(TimeSpan.FromSeconds(10))) throw new InvalidOperationException("movie graph initialization timed out"); + if (_error != null) throw new InvalidOperationException(_error); + } + + public bool TryTakeFrame(out RgbaImage frame) + { + lock (_frameLock) + { + if (_latestRgba == null) { frame = default!; return false; } + frame = new RgbaImage(_width, _height, _latestRgba); + _latestRgba = null; + return true; + } + } + + private void DecodeThread() + { + int co = CoInitializeEx(IntPtr.Zero, 0); // COINIT_MULTITHREADED; DirectShow graph and callbacks share this apartment. + try + { + _tempPath = Path.Combine(Path.GetTempPath(), $"age-movie-{Guid.NewGuid():N}.mpg"); + File.WriteAllBytes(_tempPath, _payload); + + _graphObject = Activator.CreateInstance(Type.GetTypeFromCLSID(CLSID_FilterGraph, throwOnError: true)!)!; + var graph = (IGraphBuilder)_graphObject; + var grabberObject = Activator.CreateInstance(Type.GetTypeFromCLSID(CLSID_SampleGrabber, true)!)!; + var nullObject = Activator.CreateInstance(Type.GetTypeFromCLSID(CLSID_NullRenderer, true)!)!; + var grabberFilter = (IBaseFilter)grabberObject; + var nullFilter = (IBaseFilter)nullObject; + var grabber = (ISampleGrabber)grabberObject; + + Check(graph.AddFilter(grabberFilter, "AGE Sample Grabber"), "add sample grabber"); + Check(graph.AddFilter(nullFilter, "AGE Null Renderer"), "add null renderer"); + var requested = new AMMediaType { MajorType = MEDIATYPE_Video, SubType = MEDIASUBTYPE_RGB32, + FormatType = FORMAT_VideoInfo }; + Check(grabber.SetMediaType(ref requested), "request RGB32 movie output"); + Check(graph.AddSourceFilter(_tempPath, "AGE VFS MPEG Source", out var source), "open MPEG source"); + + var sourceOut = FirstPin(source, PinDirection.Output); + var grabberIn = FirstPin(grabberFilter, PinDirection.Input); + var grabberOut = FirstPin(grabberFilter, PinDirection.Output); + var nullIn = FirstPin(nullFilter, PinDirection.Input); + // Intelligent connection inserts only the MPEG splitter/video decoder needed to reach RGB32. + // The splitter's audio pin remains unrendered, keeping movie audio outside this slice. + Check(graph.Connect(sourceOut, grabberIn), "connect MPEG video decoder"); + Check(graph.ConnectDirect(grabberOut, nullIn, IntPtr.Zero), "connect null renderer"); + + var connected = new AMMediaType(); + Check(grabber.GetConnectedMediaType(ref connected), "query movie format"); + try + { + if (connected.FormatPtr == IntPtr.Zero) throw new InvalidDataException("movie decoder returned no VIDEOINFOHEADER"); + var vi = Marshal.PtrToStructure(connected.FormatPtr); + _width = vi.BitmapInfo.Width; + _bottomUp = vi.BitmapInfo.Height > 0; + _height = Math.Abs(vi.BitmapInfo.Height); + if (_width <= 0 || _height <= 0) throw new InvalidDataException($"invalid movie dimensions {_width}x{_height}"); + } + finally { FreeMediaType(ref connected); } + + Check(grabber.SetOneShot(false), "configure continuous samples"); + Check(grabber.SetBufferSamples(false), "disable redundant frame buffering"); + Check(grabber.SetCallback(this, 1), "install decoded frame callback"); + _control = (IMediaControl)_graphObject; + Check(_control.Run(), "start movie graph"); + int stateHr = _control.GetState(5000, out int graphState); + if (stateHr < 0) Check(stateHr, "wait for running movie graph"); + if (graphState != 2) throw new InvalidOperationException($"movie graph entered unexpected state {graphState}"); + _ready.Set(); + var mediaEvent = (IMediaEvent)_graphObject; + while (!_stopping) + { + int eventHr = mediaEvent.WaitForCompletion(0, out _); + if (eventHr >= 0) { _completed = true; break; } + if (eventHr != E_ABORT) Check(eventHr, "poll movie completion"); + Thread.Sleep(10); + } + } + catch (Exception e) + { + _error = $"DirectShow MPEG decode failed: {e}"; + _ready.Set(); + } + finally + { + try { _control?.Stop(); } catch { } + ReleaseCom(_control); _control = null; + ReleaseCom(_graphObject); _graphObject = null; + if (_tempPath != null) try { File.Delete(_tempPath); } catch { } + if (co >= 0) CoUninitialize(); + } + } + + public int SampleCB(double sampleTime, IntPtr sample) => 0; + + public int BufferCB(double sampleTime, IntPtr buffer, int length) + { AcceptBgra(buffer, length); return 0; } + + private void AcceptBgra(IntPtr buffer, int length) + { + int rowBytes = checked(_width * 4); + if (buffer == IntPtr.Zero || length < rowBytes * _height) return; + byte[] bgra = new byte[rowBytes * _height]; + Marshal.Copy(buffer, bgra, 0, bgra.Length); + byte[] rgba = new byte[bgra.Length]; + for (int y = 0; y < _height; y++) + { + int src = (_bottomUp ? _height - 1 - y : y) * rowBytes; + int dst = y * rowBytes; + for (int x = 0; x < _width; x++, src += 4, dst += 4) + { + rgba[dst] = bgra[src + 2]; rgba[dst + 1] = bgra[src + 1]; + rgba[dst + 2] = bgra[src]; rgba[dst + 3] = 255; + } + } + lock (_frameLock) _latestRgba = rgba; // newest decoded frame wins if Godot renders more slowly + } + + public void Dispose() + { + _stopping = true; + if (_thread.IsAlive) _thread.Join(TimeSpan.FromSeconds(3)); + _ready.Dispose(); + } + + private static IPin FirstPin(IBaseFilter filter, PinDirection direction) + { + Check(filter.EnumPins(out var pins), "enumerate source pins"); + var one = new IPin[1]; + while (pins.Next(1, one, IntPtr.Zero) == 0) + { + Check(one[0].QueryDirection(out var found), "query source pin direction"); + if (found == direction) return one[0]; + } + throw new InvalidOperationException($"DirectShow source has no {direction} pin"); + } + + private static void Check(int hr, string operation) + { if (hr < 0) throw new COMException($"{operation} failed (HRESULT 0x{hr:x8})", hr); } + private static void ReleaseCom(object? value) + { if (value != null && Marshal.IsComObject(value)) try { Marshal.ReleaseComObject(value); } catch { } } + private static void FreeMediaType(ref AMMediaType mt) + { + if (mt.FormatPtr != IntPtr.Zero) { Marshal.FreeCoTaskMem(mt.FormatPtr); mt.FormatPtr = IntPtr.Zero; } + if (mt.UnknownPtr != IntPtr.Zero) { Marshal.Release(mt.UnknownPtr); mt.UnknownPtr = IntPtr.Zero; } + } + + private static readonly Guid CLSID_FilterGraph = new("E436EBB3-524F-11CE-9F53-0020AF0BA770"); + private static readonly Guid CLSID_SampleGrabber = new("C1F400A0-3F08-11D3-9F0B-006008039E37"); + private static readonly Guid CLSID_NullRenderer = new("C1F400A4-3F08-11D3-9F0B-006008039E37"); + private static readonly Guid MEDIATYPE_Video = new("73646976-0000-0010-8000-00AA00389B71"); + private static readonly Guid MEDIASUBTYPE_RGB32 = new("E436EB7E-524F-11CE-9F53-0020AF0BA770"); + private static readonly Guid FORMAT_VideoInfo = new("05589F80-C356-11CE-BF01-00AA0055595A"); + private const int E_ABORT = unchecked((int)0x80004004); + + [DllImport("ole32.dll")] private static extern int CoInitializeEx(IntPtr reserved, uint coInit); + [DllImport("ole32.dll")] private static extern void CoUninitialize(); + + private enum PinDirection { Input, Output } + [StructLayout(LayoutKind.Sequential)] private struct AMMediaType + { + public Guid MajorType, SubType; [MarshalAs(UnmanagedType.Bool)] public bool FixedSizeSamples; + [MarshalAs(UnmanagedType.Bool)] public bool TemporalCompression; public int SampleSize; + public Guid FormatType; public IntPtr UnknownPtr; public int FormatSize; public IntPtr FormatPtr; + } + [StructLayout(LayoutKind.Sequential)] private struct DsRect { public int Left, Top, Right, Bottom; } + [StructLayout(LayoutKind.Sequential)] private struct BitmapInfoHeader + { public int Size, Width, Height; public short Planes, BitCount; public int Compression, ImageSize, XPelsPerMeter, YPelsPerMeter, ColorsUsed, ColorsImportant; } + [StructLayout(LayoutKind.Sequential)] private struct VideoInfoHeader + { public DsRect Source, Target; public int BitRate, BitErrorRate; public long AvgTimePerFrame; public BitmapInfoHeader BitmapInfo; } + + [ComImport, Guid("6B652FFF-11FE-4FCE-92AD-0266B5D7C78F"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)] + private interface ISampleGrabber + { + [PreserveSig] int SetOneShot([MarshalAs(UnmanagedType.Bool)] bool value); + [PreserveSig] int SetMediaType(ref AMMediaType type); + [PreserveSig] int GetConnectedMediaType(ref AMMediaType type); + [PreserveSig] int SetBufferSamples([MarshalAs(UnmanagedType.Bool)] bool value); + [PreserveSig] int GetCurrentBuffer(ref int size, IntPtr buffer); + [PreserveSig] int GetCurrentSample(out IntPtr sample); + [PreserveSig] int SetCallback(ISampleGrabberCB callback, int method); + } + [ComImport, Guid("56A868A9-0AD4-11CE-B03A-0020AF0BA770"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)] + private interface IGraphBuilder + { + [PreserveSig] int AddFilter(IBaseFilter filter, [MarshalAs(UnmanagedType.LPWStr)] string name); + [PreserveSig] int RemoveFilter(IBaseFilter filter); [PreserveSig] int EnumFilters(out IntPtr filters); + [PreserveSig] int FindFilterByName([MarshalAs(UnmanagedType.LPWStr)] string name, out IBaseFilter filter); + [PreserveSig] int ConnectDirect(IPin output, IPin input, IntPtr mediaType); + [PreserveSig] int Reconnect(IPin pin); [PreserveSig] int Disconnect(IPin pin); [PreserveSig] int SetDefaultSyncSource(); + [PreserveSig] int Connect(IPin output, IPin input); [PreserveSig] int Render(IPin output); + [PreserveSig] int RenderFile([MarshalAs(UnmanagedType.LPWStr)] string file, [MarshalAs(UnmanagedType.LPWStr)] string? playList); + [PreserveSig] int AddSourceFilter([MarshalAs(UnmanagedType.LPWStr)] string file, [MarshalAs(UnmanagedType.LPWStr)] string name, out IBaseFilter filter); + [PreserveSig] int SetLogFile(IntPtr file); [PreserveSig] int Abort(); [PreserveSig] int ShouldOperationContinue(); + } + [ComImport, Guid("56A86895-0AD4-11CE-B03A-0020AF0BA770"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)] + private interface IBaseFilter + { + [PreserveSig] int GetClassID(out Guid clsid); [PreserveSig] int Stop(); [PreserveSig] int Pause(); + [PreserveSig] int Run(long start); [PreserveSig] int GetState(int timeout, out int state); + [PreserveSig] int SetSyncSource(IntPtr clock); [PreserveSig] int GetSyncSource(out IntPtr clock); + [PreserveSig] int EnumPins(out IEnumPins pins); [PreserveSig] int FindPin([MarshalAs(UnmanagedType.LPWStr)] string id, out IPin pin); + [PreserveSig] int QueryFilterInfo(IntPtr info); [PreserveSig] int JoinFilterGraph(IntPtr graph, [MarshalAs(UnmanagedType.LPWStr)] string name); + [PreserveSig] int QueryVendorInfo([MarshalAs(UnmanagedType.LPWStr)] out string vendor); + } + [ComImport, Guid("56A86892-0AD4-11CE-B03A-0020AF0BA770"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)] + private interface IEnumPins + { [PreserveSig] int Next(int count, [Out, MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 0)] IPin[] pins, IntPtr fetched); [PreserveSig] int Skip(int count); [PreserveSig] int Reset(); [PreserveSig] int Clone(out IEnumPins clone); } + [ComImport, Guid("56A86891-0AD4-11CE-B03A-0020AF0BA770"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)] + private interface IPin + { + [PreserveSig] int Connect(IPin receive, IntPtr mediaType); [PreserveSig] int ReceiveConnection(IPin connector, IntPtr mediaType); + [PreserveSig] int Disconnect(); [PreserveSig] int ConnectedTo(out IPin pin); [PreserveSig] int ConnectionMediaType(IntPtr mediaType); + [PreserveSig] int QueryPinInfo(IntPtr info); [PreserveSig] int QueryDirection(out PinDirection direction); + [PreserveSig] int QueryId([MarshalAs(UnmanagedType.LPWStr)] out string id); [PreserveSig] int QueryAccept(IntPtr mediaType); + [PreserveSig] int EnumMediaTypes(out IntPtr types); [PreserveSig] int QueryInternalConnections(IntPtr pins, ref int count); + [PreserveSig] int EndOfStream(); [PreserveSig] int BeginFlush(); [PreserveSig] int EndFlush(); [PreserveSig] int NewSegment(long start, long stop, double rate); + } + [ComImport, Guid("56A868B1-0AD4-11CE-B03A-0020AF0BA770"), InterfaceType(ComInterfaceType.InterfaceIsDual)] + private interface IMediaControl + { [PreserveSig] int Run(); [PreserveSig] int Pause(); [PreserveSig] int Stop(); [PreserveSig] int GetState(int timeout, out int state); } + [ComImport, Guid("56A868B6-0AD4-11CE-B03A-0020AF0BA770"), InterfaceType(ComInterfaceType.InterfaceIsDual)] + private interface IMediaEvent + { + [PreserveSig] int GetEventHandle(out IntPtr eventHandle); + [PreserveSig] int GetEvent(out int eventCode, out IntPtr param1, out IntPtr param2, int timeoutMs); + [PreserveSig] int WaitForCompletion(int timeoutMs, out int eventCode); + [PreserveSig] int CancelDefaultHandling(int eventCode); + [PreserveSig] int RestoreDefaultHandling(int eventCode); + [PreserveSig] int FreeEventParams(int eventCode, IntPtr param1, IntPtr param2); + } +} diff --git a/godot/GodotAdvHost.cs b/godot/GodotAdvHost.cs index 71be8a7..f9efa72 100644 --- a/godot/GodotAdvHost.cs +++ b/godot/GodotAdvHost.cs @@ -11,6 +11,9 @@ public sealed class GodotAdvHost : IHost private readonly string _scene; // e.g. "SC0000" — for section_base private readonly object _imageLock = new(); private readonly Dictionary _images = new(); // raw catalog id -> decoded pixels + private readonly Dictionary _movieFrames = new(); + private readonly Dictionary _movieBySurface = new(); + private readonly HashSet _completedMovies = new(); private readonly string?[] _sfxNames = new string?[10]; // SC0000 native channel subset // slot -> dims. Slot 0 is the primary/screen surface (800x600), normally created at engine boot which // the single-scene harness skips; seed it so the first CG's anchor math stays correct (not 0x0). @@ -143,13 +146,13 @@ public sealed class GodotAdvHost : IHost public void WaitForForegroundTransition(GfxState gfx) { int started = gfx.StartForegroundTransitions(_clock.NowMs); - if (started == 0 && !gfx.HasActiveTimedPresentation(_clock.NowMs)) return; + if (started == 0 && !gfx.HasActiveTimedPresentation(_clock.NowMs) && !HasActiveMoviePresentation()) return; _foregroundGfx = gfx; System.Threading.Interlocked.Exchange(ref _transitionStartedAtMs, _clock.NowMs); IsTransitionWaiting = true; _timeline?.State("transition-start", new() { ["count"] = started }); int lastBucket = -1; - while (gfx.HasActiveTimedPresentation(_clock.NowMs) && !_stopping) + while ((gfx.HasActiveTimedPresentation(_clock.NowMs) || HasActiveMoviePresentation()) && !_stopping) { var active = gfx.SnapshotForegroundTransitions(_clock.NowMs); int bucket = active.Count == 0 ? 100 : (int)System.Math.Floor(active[0].Progress * 10); @@ -256,11 +259,93 @@ public sealed class GodotAdvHost : IHost /// from the loose-first asset store. public (RgbaImage Image, string Name, int AssetId)? ResolveResIdTexture(long resId) { + lock (_imageLock) + if (_movieFrames.TryGetValue(resId, out var movie)) + return (movie.Image, movie.Name, movie.RawIndex); var asset = _res.ResolveTexture(_scene, resId); var image = asset != null ? Decode(asset) : null; return asset != null && image != null ? (image, asset.Name, asset.RawIndex) : null; } + public void PlayMovieToSurface(long resourceId, int surfaceSlot, long movieFlags, long syncMask) + { + var asset = _res.Resolve(_scene, resourceId); + if (asset == null) { Godot.GD.Print($"movie unresolved {_scene}:0x{resourceId:x}"); return; } + try + { + var movie = _res.ReadMovie(asset); + ReleaseSurface(surfaceSlot); + lock (_imageLock) + { + _movieBySurface[surfaceSlot] = resourceId; + _completedMovies.Remove(resourceId); + } + _slotDims[surfaceSlot] = (800, 600); // SC0000 creates this native-sized surface immediately beforehand. + _timeline?.Event("movie-start", new() + { + ["resource"] = resourceId, ["surface"] = surfaceSlot, ["file"] = movie.Name, + ["flags"] = movieFlags, ["sync_mask"] = syncMask, + }); + _main.CallDeferred("PlayMovie", movie.Bytes, movie.Name, resourceId, asset.RawIndex); + } + catch (System.Exception e) { Godot.GD.Print($"movie read failed {asset.Name}: {e.Message}"); } + } + + public void ReleaseSurface(int slot) + { + long resourceId; + lock (_imageLock) + { + if (!_movieBySurface.Remove(slot, out resourceId)) return; + if (!_completedMovies.Contains(resourceId)) + { + _movieBySurface[slot] = resourceId; + return; // SC0000 prepares following static surfaces before 0x21c; the movie remains retained. + } + _movieFrames.Remove(resourceId); + _completedMovies.Remove(resourceId); + } + _timeline?.Event("movie-stop", new() { ["resource"] = resourceId, ["surface"] = slot }); + _main.CallDeferred("StopMovie", resourceId); + } + + // Main-thread decoder handoff. Replacing the newest frame mirrors the native texture renderer's + // sample callback: the retained object keeps its surface binding while only the surface pixels change. + public void PublishMovieFrame(long resourceId, string name, int rawIndex, RgbaImage frame) + { + lock (_imageLock) _movieFrames[resourceId] = (frame, name, rawIndex); + System.Threading.Interlocked.Exchange(ref _presentRequested, 1); + } + + public void NotifyMovieCompleted(long resourceId) + { + lock (_imageLock) + if (!_completedMovies.Add(resourceId)) return; + _timeline?.Event("movie-complete", new() { ["resource"] = resourceId }); + _frameSignal.Set(); + } + + private bool HasActiveMoviePresentation() + { + lock (_imageLock) + foreach (long resourceId in _movieBySurface.Values) + if (!_completedMovies.Contains(resourceId)) return true; + return false; + } + + public bool TryGetActiveMovieFrame(out RgbaImage frame) + { + lock (_imageLock) + foreach (long resourceId in _movieBySurface.Values) + if (_movieFrames.TryGetValue(resourceId, out var movie)) + { + frame = movie.Image; + return true; + } + frame = default!; + return false; + } + private RgbaImage? Decode(AssetEntry asset) { lock (_imageLock) diff --git a/godot/Main.cs b/godot/Main.cs index fab81e5..b2a8607 100644 --- a/godot/Main.cs +++ b/godot/Main.cs @@ -1,4 +1,5 @@ using System.Collections.Generic; +using System.Runtime.Versioning; using System.Text.Json; using System.Threading.Tasks; using Godot; @@ -8,6 +9,7 @@ using Age.Engine.Sys4; using Age.Engine.Vm; using Script = Age.Engine.Model.Script; // disambiguate from Godot.Script +[SupportedOSPlatform("windows")] public partial class Main : Godot.Control { private TextureRect _screenView = null!; // shows the composited screen backbuffer @@ -22,6 +24,8 @@ public partial class Main : Godot.Control private VirtualMachine _vm = null!; private GodotAdvHost _host = null!; private readonly Age.Engine.Hosting.FrameClock _clock = new(); + private readonly System.Collections.Generic.Dictionary _movies = new(); + private readonly System.Collections.Generic.HashSet _movieFrameSeen = new(); private GodotTraceSink _trace = null!; private Age.Engine.Diagnostics.HistogramTraceSink? _hist; // --trace-histogram: profile the real run private string? _histFile; @@ -214,8 +218,10 @@ public partial class Main : Godot.Control public override void _Process(double delta) { _clock.Advance(delta); - _timeline?.SetFrame(++_timelineFrame, _clock.NowMs); + _timelineFrame++; + _timeline?.SetFrame(_timelineFrame, _clock.NowMs); _host?.PulseFrame(); + UpdateMovieFrames(); if (!_selftest && _vm != null && _host != null && _host.ShouldRecomposite()) Recomposite(); // native publishes retained mutations only at present/service boundaries if (!_selftest && _host != null) UpdateAdvTextPresentation(); @@ -266,7 +272,12 @@ public partial class Main : Godot.Control _host.SignalInput(); } - public override void _ExitTree() { DumpHistogram(); _host?.Stop(); _timeline?.Dispose(); } + public override void _ExitTree() + { + DumpHistogram(); _host?.Stop(); _timeline?.Dispose(); + foreach (var movie in _movies.Values) movie.Decoder.Dispose(); + _movies.Clear(); + } // Write the real-run op/call-site histogram to --trace-histogram . Idempotent; called when the // scene ends or the window closes (the opening parks at wait-for-input, so closing is the usual trigger). @@ -295,6 +306,11 @@ public partial class Main : Godot.Control private void Recomposite() { _screen.Fill(new Color(0, 0, 0, 0)); + // SC0000's movie is an independently updating retained background. The script prepares later + // static surfaces before its 0x21c yield; those layers composite above the current movie sample. + if (_host.TryGetActiveMovieFrame(out var movieFrame) && + movieFrame.Width == _screen.GetWidth() && movieFrame.Height == _screen.GetHeight()) + _screen.SetData(movieFrame.Width, movieFrame.Height, false, Image.Format.Rgba8, movieFrame.Pixels); _speaker.Visible = false; System.Collections.Generic.Dictionary? decisions = _gfxLogPath != null || _timeline != null ? new() : null; int z = 0; @@ -566,6 +582,49 @@ public partial class Main : Godot.Control CreateTween().TweenProperty(_bgm, "volume_db", targetDb, realDurationSeconds); } + public void PlayMovie(byte[] mpegBytes, string assetName, long resourceId, int rawIndex) + { + if (_movies.Remove(resourceId, out var prior)) prior.Decoder.Dispose(); + try + { + var payload = new Age.Engine.Sys4.MoviePayload(assetName, mpegBytes); + _movies[resourceId] = new MovieRuntime(assetName, rawIndex, new DirectShowMovieDecoder(payload)); + GD.Print($"movie started {assetName} ({mpegBytes.Length} bytes from VFS)"); + } + catch (System.Exception e) + { + GD.Print($"movie decode failed {assetName}: {e.Message}"); + _host.NotifyMovieCompleted(resourceId); // release a pending 0x21c boundary on deterministic load failure + } + } + + private void UpdateMovieFrames() + { + if (_host == null) return; + foreach (var (resourceId, movie) in _movies) + { + if (movie.Decoder.TryTakeFrame(out var frame)) + { + _host.PublishMovieFrame(resourceId, movie.Name, movie.RawIndex, frame); + if (_movieFrameSeen.Add(resourceId)) + GD.Print($"movie first frame {movie.Name}: {frame.Width}x{frame.Height} RGBA8 at render frame {_timelineFrame}"); + } + if (movie.Decoder.IsCompleted) _host.NotifyMovieCompleted(resourceId); + } + } + + public void StopMovie(long resourceId) + { + if (_movies.Remove(resourceId, out var movie)) + { + movie.Decoder.Dispose(); + GD.Print($"movie stopped {movie.Name} at render frame {_timelineFrame}"); + } + _movieFrameSeen.Remove(resourceId); + } + + private sealed record MovieRuntime(string Name, int RawIndex, DirectShowMovieDecoder Decoder); + public void AppendLine(string text) => _text.Text += text + "\n"; public void PageBreak() { _pageCount++; _status.Text = ""; } public void ClearPage() { _text.Text = ""; _status.Text = ""; } diff --git a/vm-map/opcodes.toml b/vm-map/opcodes.toml index 83969c1..a16efea 100644 --- a/vm-map/opcodes.toml +++ b/vm-map/opcodes.toml @@ -6146,38 +6146,39 @@ observed_types = ["l-int"] [[opcode]] op = 0x236 -label = "u004221A0" +label = "play-movie-to-surface" argc = 4 abi_source = "kelebek+decode-validated" [opcode.semantics] -name = "u004221A0" -category = "unknown" -summary = "" +name = "play-movie-to-surface" +category = "draw" +summary = "(resource_id)(surface_slot)(movie_flags)(sync_mask) - synchronously resolve/open an archive movie and construct its DirectShow graph, then start asynchronous frame delivery into the retained destination surface. The opcode itself is non-blocking: the VM advances to the next instruction. SC0000's native site evaluates (0x33, 0, 2, 0) at 0x13c8 and resumes at bytecode 0x13d1; its later 0x21c service boundary yields until movie EOF before cleanup." noop_headless = false -source = "kelebek" -confidence = "low" +source = "investigation" +confidence = "high" depends_on = [] -evidence = "" +evidence = "Native handler 0x423ee0 and helpers 0x463c50/0x463aa0/0x463e20/0x4625e0; SC0000 native operand capture; exact 0x13c8->0x13d1 trace; archive-only changing-frame decoder and Godot lifecycle tests." +details = "The handler requires an existing destination texture, allocates/reuses a 0x478-byte movie-to-texture object for the surface, opens operand 1 through the native indexed-asset reader, builds FilterGraph/IGraphBuilder/IMediaControl/IMediaPosition/IMediaEvent/IBasicAudio, and presents bottom-up RGB samples through the movie texture renderer. Operand 3 selects movie/sound routing policy: bits 0x10000/0x20000/0x40000/0x80000 force sound route 0/1/2/3, otherwise set:DependMovieSound is used; SC0000's low value 2 is retained as native movie mode state. Operand 4 is stored as the movie sync/device mask at object+0x42c; SC0000 passes 0. Static layer preparation after 0x236 does not terminate the retained movie; 0x21c services it through EOF and subsequent surface cleanup stops/detaches it." [[opcode.semantics.args]] i = 1 -role = "" +role = "resource_id" observed_types = ["imm", "l-int", "l-ptr"] [[opcode.semantics.args]] i = 2 -role = "" +role = "surface_slot" observed_types = ["imm", "g-int", "l-int", "l-ptr"] [[opcode.semantics.args]] i = 3 -role = "" +role = "movie_flags" observed_types = ["imm", "g-int", "l-ptr"] [[opcode.semantics.args]] i = 4 -role = "" +role = "sync_mask" observed_types = ["imm", "l-ptr"] [[opcode]]