diff --git a/docs/PROJECT-STRUCTURE.md b/docs/PROJECT-STRUCTURE.md index 138af0d..6f8341e 100644 --- a/docs/PROJECT-STRUCTURE.md +++ b/docs/PROJECT-STRUCTURE.md @@ -100,6 +100,9 @@ S:\Game Hacking\Eushully\Himegari\ ← workspace root (three siblings) ├── engine/ DELIVERABLE — the .NET VM core (AgeEngine.sln: Age.Engine / Age.Cli / tests) │ └── Age.Engine/Sys4/ runtime catalog parser, loose-first bounded ALF asset store, │ script provider, AGF/LZSS and Windows CUR decoders, and resource facade + ├── native/ authored native runtime boundaries + │ └── age_movie_ffmpeg/ project-owned FFmpeg C ABI, immutable Windows dependency manifest, + │ and bootstrap/build scripts (outputs stay under disposable build/) ├── tools/frida/ runtime-capture + engine-dump scripts (see tools/frida/README.md) └── godot/ DELIVERABLE — the Godot/C# ADV front-end (references Age.Engine), including the TITLE-only F4 debug scene launcher diff --git a/docs/phase-b-framework.md b/docs/phase-b-framework.md index 5f072ad..50122ed 100644 --- a/docs/phase-b-framework.md +++ b/docs/phase-b-framework.md @@ -582,9 +582,55 @@ an infinite callback count: the total horizon starts at `base_time + 1000`, and loop polls `0x23a` until surfaces 7..10 are inactive. MVB908's stop log occurs after that loop. As a bounded safety correction, decoder failure now becomes an explicitly completed zero-duration effect, while started movies have a stop-time-based completion watchdog so a missing EOF cannot hold `0x21c` forever. **NEXT:** -manually recheck the same exchange. If it still stalls, capture the VM/service coordinate after BTL cleanup -as a separate timed-presentation issue; otherwise proceed to the software MPEG backend for real 280x352 -duration/frames. Preserve created destination dimensions and failed-movie identity through that backend change. +replace DirectShow through the bounded FFmpeg plan below. The user confirmed that combat can still stall, but +that symptom remains tabled until the replacement removes DirectShow as a variable. Preserve created destination +dimensions and failed-movie identity through the backend change. + +### Portable FFmpeg movie backend selected; bounded implementation plan (2026-07-21) + +FFmpeg is selected as the single movie backend for stock content and future profiles. The exact native ABI, +dependency pin, corpus codec inventory, timing contract, and packaging rules live in +`docs/platform-portability.md`; this section owns execution order and acceptance. + +1. **Completed — land the seam without changing behavior.** Add `IMovieDecoder` plus an injected factory, type + `MovieRuntime` against the interface, and adapt DirectShow temporarily. Cover synchronous metadata, frame + handoff, EOF, failure-as-completed, replacement, cancellation, and disposal with fakes. This commit must retain + current live behavior and gives the FFmpeg work a testable boundary. +2. **Completed — prove the native ABI in isolation on Windows x64.** Add the C shim and pinned shared FFmpeg dependency, + then open VFS-owned bytes without a temporary file. The initial gate decodes changing RGBA frames and positive + stop times from one formerly failing 280x352 effect (`MVB961` or `MVB238`), aligned `MVB908`, and 800x600 + `CHAPTER`. It must also reject a truncated payload with a bounded diagnostic and survive repeated open/close. + Do not switch the live runtime at this step. +3. **Next — implement managed pacing and switch the factory.** Decode on a cancellable worker using FFmpeg timestamps + and a monotonic clock, publish only due frames, delay EOF completion through the final presentation interval, + and retain the existing watchdog. Make FFmpeg the default with no dimension/effect dispatch and exercise both + non-modal `0x236` and modal/cancellable `0x20f` paths. +4. **Run the installed-corpus gate.** Every one of the 213 MPEG payloads must open, report its expected display + dimensions and a positive stop time, produce a correctly sized RGBA frame, maintain nondecreasing timestamps, + reach EOF, and dispose within a bound. Normal tests use small project-authored 280-wide and aligned MPEG + fixtures so decoder behavior is never disabled when the original install is absent; the full local sweep is + an additional release/acceptance gate. Spot comparisons against DirectShow-compatible assets establish timing + tolerance before DirectShow is removed. +5. **Validate the live sequences, then delete DirectShow.** Recheck LOGO/OP cancellation, SC0000 CHAPTER, + aligned and 280-wide battle effects, combat cleanup, and the previously observed post-effect stall. Once those + pass, delete the COM declarations/temp-file path and Windows platform annotation rather than retaining a + fallback. If combat still stalls with FFmpeg, capture the VM/service coordinate after BTL cleanup and treat it + as a separate battle-timeline defect. +6. **Complete distributable packaging.** Bundle pinned dynamically linked libraries and exact license/source + provenance per target, validate loader isolation from machine-installed codecs, then add Linux/macOS build and + smoke gates. Timestamped PCM delivery and Godot `AudioStreamGenerator` integration remain a later movie-audio + slice; the video replacement must merely avoid closing that path. + +Steps 1 and 2 landed on 2026-07-21 without changing live playback: `Main` owns an +`IMovieDecoderFactory`, `MovieRuntime` is backend-neutral, and the default factory still constructs DirectShow. +The isolated Windows-x64 C ABI opens VFS bytes through seekable custom AVIO and decodes changing RGBA frames from +both formerly failing 280x352 effects (`MVB961` = 500 ms, `MVB238` = 866 ms), aligned `MVB908` (333 ms), and +800x600 `CHAPTER` (12016 ms). Truncated input is rejected with a bounded error and ten repeated open/decode/close +cycles pass. During the full-suite gate, direct swscale output into a managed array exposed native heap corruption; +conversion now targets an FFmpeg-owned aligned frame and copies only the exact visible RGBA rows across the ABI. +The complete 313-test suite, three additional 312-test concurrency-sensitive repeats, zero-warning Godot build, +and threaded selftest pass. Step 3 is the next bounded slice; do not infer that combat is fixed until the paced +backend is selected live and revalidated. **Mutable-surface fill/blend regression corrected.** The first visual recheck exposed BUNKI's menu interior as transparent. SYSTEM4 creates 800x600 surface 3 and fills it opaque white through `0x20b`; the metadata-only diff --git a/docs/platform-portability.md b/docs/platform-portability.md index 9c9fa76..a55a516 100644 --- a/docs/platform-portability.md +++ b/docs/platform-portability.md @@ -23,8 +23,8 @@ also softer Windows assumptions that should be tested or replaced before claimin | Area | Current dependency | Runtime impact | Portability status / future action | |---|---|---|---| -| AGE movie decode (`0x236` scene movies; `0x20f` modal LOGO/OP/ED) | `godot/DirectShowMovieDecoder.cs`: DirectShow COM objects plus `ole32.dll` `CoInitializeEx` / `CoUninitialize` | Movie playback cannot run outside Windows; the current Windows filter stack also rejects the installed MPEG effects whose display width is 8 mod 16 | Introduce a decoder interface and backend factory; use a software MPEG backend for portable coverage and as the Windows fallback for unsupported effect dimensions | -| Movie integration | `godot/Main.cs` directly constructs and stores `DirectShowMovieDecoder`; `Main` is marked `SupportedOSPlatform("windows")` | The frontend has no runtime fallback or OS-specific source selection | Type `MovieRuntime` against the decoder interface, select by platform/build, and move the Windows annotation to the DirectShow backend | +| AGE movie decode (`0x236` scene movies; `0x20f` modal LOGO/OP/ED) | Live playback still uses `godot/DirectShowMovieDecoder.cs`; the isolated `native/age_movie_ffmpeg` C ABI and `FfmpegMovieSession` probe path now decode the representative VFS movies | Live movies remain Windows-only and the selected live backend still rejects installed MPEG effects whose display width is 8 mod 16 | Add timestamp pacing to the proven FFmpeg path, select it as the sole live factory, run the corpus/live gates, then delete DirectShow | +| Movie integration | `MovieRuntime` now owns `IMovieDecoder` from an injected `IMovieDecoderFactory`; `Main` deliberately selects `DirectShowMovieDecoderFactory` and remains marked `SupportedOSPlatform("windows")` | Backend ownership is portable, but the selected live implementation is not yet | Implement the FFmpeg `IMovieDecoder`, switch the factory, and remove the Windows annotation when DirectShow is deleted | | Movie audio | DirectShow connects only the video pin to the sample grabber/null renderer | The MPEG audio stream is intentionally silent on every platform | Design a PCM/audio-clock contract or let a future backend own synchronized A/V; separate feature slice | | ADV font discovery | `godot/Main.cs` probes `C:/Windows/Fonts` for Japanese fonts | Harmless fallback today, but appearance depends on host fonts | Bundle/configure a redistributable font or add platform-specific discovery | | Filesystem semantics | Several filename and containment comparisons use `OrdinalIgnoreCase`; installed assets are conventionally uppercase | Needs validation on case-sensitive filesystems; may hide casing or containment mistakes | Add Linux/macOS tests with mixed-case synthetic roots and use filesystem-appropriate containment rules | @@ -45,27 +45,57 @@ The existing connection is localized but one abstraction short of being replacea VM op 0x236 (non-modal) / op 0x20f (modal) -> IHost.PlayMovieToSurface / PlayModalMovieToSurface -> VFS-owned MoviePayload bytes - -> DirectShowMovieDecoder + -> IMovieDecoderFactory + -> DirectShowMovieDecoder (current live selection) + -> FfmpegMovieSession -> age_movie C ABI (isolated gate only) -> newest RGBA frame -> retained movie surface -> Godot compositor ``` -Everything before and after `DirectShowMovieDecoder` is portable. The backend currently exposes the right -conceptual operations (`StopTimeMs`, `TryTakeFrame`, `IsCompleted`, and `Dispose`) but they are not formalized as an -interface. A future cleanup should: +Everything before and after the selected decoder is portable. The replacement decision is an in-process +FFmpeg backend behind a project-owned C ABI, not raw FFmpeg structs in Godot/C# and not a subprocess. FFmpeg +`n8.1.2-29-g703dcc25b9` is pinned by immutable release URL and SHA-256 in +`native/age_movie_ffmpeg/dependency-win64.json`; changing that pin requires rerunning the full installed-movie +gate. The shim dynamically links an LGPL build made without GPL or nonfree components and +uses only `libavformat`, `libavcodec`, `libavutil`, and `libswscale` for the video slice. `libswresample` and the +MPEG audio decoders may be packaged now, but PCM delivery remains a separate slice. Release artifacts must carry +the matching FFmpeg source/configuration and notices required by FFmpeg's +[license checklist](https://ffmpeg.org/legal.html). -1. Add an `IMovieDecoder` contract for synchronous stop-time metadata, frame delivery, completion, failure, - and disposal. Stop time must be available before `0x236` returns so an immediately following `0x23f` - remains deterministic. -2. Add an injected factory that accepts `MoviePayload` and selects an available backend. -3. Keep DirectShow in a Windows-specific source set or assembly, with its platform annotation local to it. -4. Implement a portable MPEG program-stream backend that produces the same top-down RGBA8 frames. -5. Preserve newest-frame-wins delivery, asynchronous decode, synchronous open failure, EOF notification, - and the retained-surface lifecycle already validated for opcode `0x236`; expose modal completion/cancel - separately for `0x20f` rather than blocking inside the decoder backend. -6. Treat synchronized movie audio as a separate extension of the contract rather than coupling it to the - compositor. +The boundary has two layers: + +1. A small native `age_movie` ABI owns all FFmpeg objects and version-sensitive calls. It accepts a borrowed + byte span only for the duration of `open`, copies it into native-owned memory, creates a seekable custom + `AVIOContext`, probes the MPEG program stream, and returns immutable video metadata. Sequential decode returns + one top-down tightly packed RGBA8 frame plus its normalized presentation timestamp. Conversion occurs in an + FFmpeg-owned aligned frame; only exact visible row bytes are copied into the caller's tightly packed buffer. + Return statuses distinguish frame, EOF, invalid arguments, undersized output, and decoder failure; the open + call returns a bounded UTF-8 diagnostic and an opened handle retains its last decode error. Close accepts a + null handle, and managed `SafeHandle` ownership guarantees one close for each successfully opened handle. No + FFmpeg pointer crosses the ABI. +2. Managed `IMovieDecoder` owns the native handle and the paced worker. Synchronous construction provides + `StopTimeMs` before `0x236` returns. The worker uses a monotonic playback origin, decodes ahead by at most one + frame, publishes frames when their timestamps become due, and retains newest-frame-wins behavior if Godot is + late. EOF becomes completion only after the final frame's presentation interval/stop time, so surface cleanup + cannot erase the last frame immediately. Disposal interrupts waits and joins without depending on Godot's main + thread. + +`StopTimeMs` performs a bounded packet scan over the seekable in-memory payload and takes the longest usable +FFmpeg format duration, video-stream duration, timestamp span, or constant-frame-rate packet-count duration. This +accounts for the final MPEG presentation interval that FFmpeg's stream duration can omit (`MVB908` is ten 30 fps +frames: 333 ms, not the raw stream field's 316 ms). The result is converted to integer milliseconds with the same +positive truncation used by native op `0x23f`, then the demuxer rewinds before playback. Streams with no usable +duration evidence fail initialization for the stock-content path rather than inventing timing. The existing host +safety rule still converts any backend initialization failure into an +explicitly completed zero-duration movie identity, and the presentation watchdog remains a last-resort guard for +an initialized backend that never reaches EOF. + +The factory is injection for tests and future decoder replacement, not runtime codec roulette. Once the FFmpeg +corpus gate passes it becomes the only shipped movie backend on every platform and DirectShow is deleted. Modal +completion/cancel remains owned above the decoder by the existing `0x20f` host path. A future audio implementation +will consume timestamped PCM and may become the presentation clock; it must not change the VM-facing stop-time, +surface, or cancellation contracts. This replacement is now also required for Windows gameplay parity. Archive-backed probes on 2026-07-21 show that the current DirectShow graph accepts tested MPEG widths divisible by 16 (208, 288, 304, 400, and @@ -73,10 +103,29 @@ show that the current DirectShow graph accepts tested MPEG widths divisible by 1 The dominant combat-effect family is 280x352 (125 installed `MVB` assets), so retaining DirectShow as the only Windows decoder is insufficient even though SC0000's 800x600 `CHAPTER.AGF` works. -An OS-specific build is also viable: include DirectShow only in Windows exports and a different decoder in -other exports. The current plain `net8.0` project has no conditional backend selection. COM declarations may -compile on another OS, but `[SupportedOSPlatform]` is analyzer metadata rather than a runtime guard, and -loading `ole32.dll` or DirectShow CLSIDs will fail there. +An archive scan on the same date classified all 213 installed movie payloads as MPEG-1 program streams with +MPEG-1 video and no MPEG-2 sequence extensions. Of those, 184 are video-only and 29 contain MPEG audio. The +large LOGO/OP/ED family uses MPEG Audio Layer II, while 25 small 120x120 `MVS` movies use Layer I. That inventory +is why the narrower `pl_mpeg` library was rejected as the primary backend: it fits every installed video stream +but only decodes Layer II audio and explicitly ignores program-stream PTS in its high-level synchronization. +FFmpeg covers the complete installed codec set and leaves the mod/profile boundary open without selecting a +different decoder per effect. + +Native deliverables are RID-specific and bundled with the Godot export; the runtime must not discover an +arbitrary system FFmpeg. The first implementation gate is Windows x64 because that is the current runnable +target, but the C ABI and loader paths must reserve Windows x64, Linux x64, macOS x64, and macOS arm64 from the +start. Builds use shared libraries, `$ORIGIN`/`@loader_path`-style local lookup on Unix targets, recorded source +hashes and configure arguments, and no committed original-game data. Packaging automation is part of completing +the backend, not a prerequisite for the first native decode spike. + +The Windows-x64 spike is now complete. `bootstrap-win64.ps1` verifies the immutable archive SHA before +extraction, and `build-win64.ps1` builds the shim with MSVC and places the DLL, import artifacts, required LGPL +shared libraries, and license under disposable `build/native/win-x64`. The managed resolver accepts +`AGE_FFMPEG_NATIVE_DIR` for the isolated gate and otherwise reserves application-local and +`runtimes//native` lookup. Representative VFS results are `MVB961` 280x352/500 ms, `MVB238` +280x352/866 ms, `MVB908` 400x400/333 ms, and `CHAPTER` 800x600/12016 ms, all with changing frames and +nondecreasing timestamps. Malformed input and repeated teardown are covered. This is not yet distributable +packaging and does not change the current DirectShow live selection. ## Cross-platform validation gates diff --git a/docs/tools-reference.md b/docs/tools-reference.md index 0cf2889..948f491 100644 --- a/docs/tools-reference.md +++ b/docs/tools-reference.md @@ -72,6 +72,20 @@ All opcode knowledge (ABI, semantics, provenance, `depends_on`) is hand-edited * |---|---|---|---| | `locate_page.py` | Resolve a run-relative ADV page number to its canonical wait script/offset, last show-text instruction, call stack, and nearby disassembly. Pure selection/window logic is tested by `test_locate_page.py`. | `py -3.11 -X utf8 tools/locate_page.py SC0000 14 [--map ] [--context N]` | `build/page-map-.jsonl` + script corpus → stdout | +## Native FFmpeg movie shim (Windows x64) + +These PowerShell tools build the isolated native movie gate; they do not switch live Godot playback away from +DirectShow. The dependency manifest pins an immutable LGPL shared FFmpeg archive and SHA-256. + +| Tool | Purpose | Run | Reads → Writes | +|---|---|---|---| +| `native/age_movie_ffmpeg/bootstrap-win64.ps1` | Download, hash-check, extract, and version-check the pinned FFmpeg SDK. Returns the resolved SDK root. | `.\\native\\age_movie_ffmpeg\\bootstrap-win64.ps1 [-Destination ]` | `dependency-win64.json`, network/archive cache → disposable `build/downloads/`, `build/ffmpeg-sdk/` | +| `native/age_movie_ffmpeg/build-win64.ps1` | Discover the MSVC x64 toolchain, build `age_movie_ffmpeg.dll`, and stage its exact shared-library/license dependencies. | `.\\native\\age_movie_ffmpeg\\build-win64.ps1 -SdkRoot [-OutputDirectory ]` | C ABI source + FFmpeg SDK → disposable `build/native/win-x64/` by default | + +The managed isolated probes load from `AGE_FFMPEG_NATIVE_DIR` when set, then application-local and +`runtimes/win-x64/native` locations. Build the shim before running the `FfmpegShim*` tests; no original-game +movie is copied into the repository or native output. + ## Engine (C#) — VM core, CLI, Godot frontend The `engine/` .NET solution (`AgeEngine.sln`) is the runtime VM; `godot/` is the ADV frontend. Not diff --git a/engine/Age.Engine.Tests/Age.Engine.Tests.csproj b/engine/Age.Engine.Tests/Age.Engine.Tests.csproj index 2462e7d..afc9d5c 100644 --- a/engine/Age.Engine.Tests/Age.Engine.Tests.csproj +++ b/engine/Age.Engine.Tests/Age.Engine.Tests.csproj @@ -22,6 +22,9 @@ + + + diff --git a/engine/Age.Engine.Tests/MovieOpcodeTests.cs b/engine/Age.Engine.Tests/MovieOpcodeTests.cs index a2a6463..1d4a9df 100644 --- a/engine/Age.Engine.Tests/MovieOpcodeTests.cs +++ b/engine/Age.Engine.Tests/MovieOpcodeTests.cs @@ -8,6 +8,73 @@ using Xunit; public class MovieOpcodeTests { + private sealed class FakeMovieDecoder : IMovieDecoder + { + public long? StopTimeMs { get; init; } + public bool IsCompleted { get; set; } + public bool Disposed { get; private set; } + public RgbaImage? Frame { get; set; } + + public bool TryTakeFrame(out RgbaImage frame) + { + if (Frame == null) { frame = default!; return false; } + frame = Frame; + Frame = null; + return true; + } + + public void Dispose() => Disposed = true; + } + + private sealed class FakeMovieDecoderFactory(FakeMovieDecoder decoder) : IMovieDecoderFactory + { + public MoviePayload? OpenedPayload { get; private set; } + public IMovieDecoder Open(MoviePayload movie) + { + OpenedPayload = movie; + return decoder; + } + } + + [Fact] + public void MovieRuntimeUsesInjectedDecoderAndRetainsSynchronousMetadata() + { + var decoder = new FakeMovieDecoder + { + StopTimeMs = 1876, + Frame = new RgbaImage(1, 1, new byte[] { 1, 2, 3, 4 }), + }; + var factory = new FakeMovieDecoderFactory(decoder); + var payload = new MoviePayload("TEST.AGF", new byte[] { 0, 0, 1, 0xba }); + + var runtime = MovieRuntime.Open("TEST.AGF", 7, payload, factory); + + Assert.Same(payload, factory.OpenedPayload); + Assert.Same(decoder, runtime.Decoder); + Assert.Equal(1876, runtime.Decoder.StopTimeMs); + Assert.Equal(5000, runtime.WatchdogMs); + Assert.True(runtime.Decoder.TryTakeFrame(out var frame)); + Assert.Equal(new byte[] { 1, 2, 3, 4 }, frame.Pixels); + runtime.Decoder.Dispose(); + Assert.True(decoder.Disposed); + } + + [Theory] + [InlineData(null, 30000)] + [InlineData(-1L, 30000L)] + [InlineData(10000L, 12000L)] + [InlineData(500000L, 300000L)] + public void MovieRuntimeComputesBoundedWatchdogFromDecoderMetadata(long? stopTimeMs, long expected) + { + var decoder = new FakeMovieDecoder { StopTimeMs = stopTimeMs }; + var runtime = MovieRuntime.Open("TEST.AGF", 7, + new MoviePayload("TEST.AGF", new byte[] { 0, 0, 1, 0xba }), + new FakeMovieDecoderFactory(decoder)); + + Assert.Equal(expected, runtime.WatchdogMs); + runtime.Decoder.Dispose(); + } + [Fact] public void InitialRootFlagStartsSetAndClearsWhenExitScriptRuns() { @@ -252,4 +319,81 @@ public class MovieOpcodeTests } Assert.True(changed, "DirectShow should deliver changing MPEG frames, not one retained still"); } + + [Theory] + [InlineData(0x2be3, "MVB961.AGF", 280, 352, 500)] + [InlineData(0x2b94, "MVB238.AGF", 280, 352, 866)] + [InlineData(0x2bc2, "MVB908.AGF", 400, 400, 333)] + [InlineData(0x33, "CHAPTER.AGF", 800, 600, 12016)] + public void FfmpegShimDecodesRepresentativeVfsMovie(int resourceId, string expectedName, + int expectedWidth, int expectedHeight, + long expectedStopTimeMs) + { + if (!OperatingSystem.IsWindows()) return; + ConfigureFfmpegNativeProbe(); + var catalog = Sys4AssetCatalog.Load(Paths.Sys4Ini); + var resources = new ResourceMap(catalog, new Sys4AssetStore(catalog, Paths.GameDir)); + var payload = resources.ReadMovie(resources.ResolveMovie(resourceId)!); + + using var movie = new FfmpegMovieSession(payload); + + Assert.Equal(expectedName, payload.Name); + Assert.Equal(expectedWidth, movie.Info.Width); + Assert.Equal(expectedHeight, movie.Info.Height); + Assert.Equal(expectedStopTimeMs, movie.Info.StopTimeMs); + Assert.True(movie.TryDecodeNextVideoFrame(out var first)); + Assert.Equal(expectedWidth * expectedHeight * 4, first.Image.Pixels.Length); + Assert.True(first.PresentationTimeMs >= 0); + bool changed = false; + long priorTimestamp = first.PresentationTimeMs; + for (int frameIndex = 0; frameIndex < 30 && movie.TryDecodeNextVideoFrame(out var later); frameIndex++) + { + Assert.True(later.PresentationTimeMs >= priorTimestamp); + priorTimestamp = later.PresentationTimeMs; + if (!first.Image.Pixels.AsSpan().SequenceEqual(later.Image.Pixels)) + { + changed = true; + break; + } + } + Assert.True(changed, $"{expectedName} should deliver changing decoded frames"); + } + + [Fact] + public void FfmpegShimRejectsTruncatedMovieWithBoundedDiagnostic() + { + if (!OperatingSystem.IsWindows()) return; + ConfigureFfmpegNativeProbe(); + var payload = new MoviePayload("TRUNCATED.AGF", new byte[] { 0, 0, 1, 0xba, 0x21, 0, 1, 0 }); + + var error = Assert.Throws(() => new FfmpegMovieSession(payload)); + + Assert.Contains("TRUNCATED.AGF", error.Message); + Assert.Contains("FFmpeg", error.Message); + } + + [Fact] + public void FfmpegShimSupportsRepeatedOpenAndClose() + { + if (!OperatingSystem.IsWindows()) return; + ConfigureFfmpegNativeProbe(); + var catalog = Sys4AssetCatalog.Load(Paths.Sys4Ini); + var resources = new ResourceMap(catalog, new Sys4AssetStore(catalog, Paths.GameDir)); + var payload = resources.ReadMovie(resources.ResolveMovie(0x2bc2)!); + + for (int iteration = 0; iteration < 10; iteration++) + { + using var movie = new FfmpegMovieSession(payload); + Assert.True(movie.TryDecodeNextVideoFrame(out _)); + } + } + + private static void ConfigureFfmpegNativeProbe() + { + string nativeDirectory = Environment.GetEnvironmentVariable("AGE_FFMPEG_NATIVE_DIR") + ?? Path.Combine(Paths.Build, "native", "win-x64"); + Assert.True(File.Exists(Path.Combine(nativeDirectory, "age_movie_ffmpeg.dll")), + $"Build the FFmpeg shim first: native/age_movie_ffmpeg/build-win64.ps1 (expected {nativeDirectory})"); + Environment.SetEnvironmentVariable("AGE_FFMPEG_NATIVE_DIR", nativeDirectory); + } } diff --git a/godot/DirectShowMovieDecoder.cs b/godot/DirectShowMovieDecoder.cs index 898e2b2..0d97aeb 100644 --- a/godot/DirectShowMovieDecoder.cs +++ b/godot/DirectShowMovieDecoder.cs @@ -18,7 +18,7 @@ internal interface ISampleGrabberCB /// Decoded RGB32 samples are copied into process memory and consumed by Godot's retained compositor. /// [SupportedOSPlatform("windows")] -internal sealed class DirectShowMovieDecoder : IDisposable, ISampleGrabberCB +internal sealed class DirectShowMovieDecoder : IMovieDecoder, ISampleGrabberCB { private readonly byte[] _payload; private readonly Thread _thread; diff --git a/godot/FfmpegMovieNative.cs b/godot/FfmpegMovieNative.cs new file mode 100644 index 0000000..a8010f4 --- /dev/null +++ b/godot/FfmpegMovieNative.cs @@ -0,0 +1,162 @@ +using System; +using System.IO; +using System.Runtime.InteropServices; +using Age.Engine.Sys4; +using Microsoft.Win32.SafeHandles; + +internal readonly record struct FfmpegMovieInfo( + int Width, + int Height, + long StopTimeMs, + int FrameRateNumerator, + int FrameRateDenominator, + bool HasAudio); + +internal sealed record FfmpegVideoFrame(RgbaImage Image, long PresentationTimeMs); + +/// Sequential, unpaced access to the project-owned FFmpeg C ABI for isolated probes and playback. +internal sealed class FfmpegMovieSession : IDisposable +{ + private FfmpegMovieHandle _handle; + public FfmpegMovieInfo Info { get; } + + public FfmpegMovieSession(MoviePayload movie) + { + ArgumentNullException.ThrowIfNull(movie); + if (FfmpegMovieNative.AbiVersion() != 1) + throw new InvalidOperationException("unsupported age_movie_ffmpeg ABI version"); + + byte[] error = new byte[1024]; + int result = FfmpegMovieNative.Open(movie.Bytes, (nuint)movie.Bytes.Length, + out _handle, out var nativeInfo, error, (nuint)error.Length); + if (result != 0 || _handle.IsInvalid) + { + _handle?.Dispose(); + throw new InvalidDataException( + $"FFmpeg movie open failed for {movie.Name}: {FfmpegMovieNative.DecodeUtf8(error)}"); + } + Info = new FfmpegMovieInfo(nativeInfo.Width, nativeInfo.Height, nativeInfo.StopTimeMs, + nativeInfo.FrameRateNumerator, nativeInfo.FrameRateDenominator, nativeInfo.HasAudio != 0); + if (Info.Width <= 0 || Info.Height <= 0 || Info.StopTimeMs <= 0) + { + _handle.Dispose(); + throw new InvalidDataException( + $"FFmpeg returned invalid metadata for {movie.Name}: {Info.Width}x{Info.Height}, {Info.StopTimeMs} ms"); + } + } + + public bool TryDecodeNextVideoFrame(out FfmpegVideoFrame frame) + { + ObjectDisposedException.ThrowIf(_handle.IsClosed, this); + byte[] pixels = new byte[checked(Info.Width * Info.Height * 4)]; + int result = FfmpegMovieNative.DecodeVideo(_handle, pixels, (nuint)pixels.Length, out long ptsMs); + if (result == FfmpegMovieNative.EndOfFile) + { + frame = default!; + return false; + } + if (result != FfmpegMovieNative.Frame) + throw new InvalidDataException($"FFmpeg movie decode failed: {FfmpegMovieNative.LastError(_handle)}"); + frame = new FfmpegVideoFrame(new RgbaImage(Info.Width, Info.Height, pixels), ptsMs); + return true; + } + + public void Dispose() => _handle.Dispose(); +} + +internal sealed class FfmpegMovieHandle : SafeHandleZeroOrMinusOneIsInvalid +{ + private FfmpegMovieHandle() : base(ownsHandle: true) { } + protected override bool ReleaseHandle() + { + FfmpegMovieNative.Close(handle); + return true; + } +} + +internal static class FfmpegMovieNative +{ + internal const int EndOfFile = 0; + internal const int Frame = 1; + private const string LibraryName = "age_movie_ffmpeg"; + + [StructLayout(LayoutKind.Sequential)] + internal struct NativeInfo + { + public int Width; + public int Height; + public long StopTimeMs; + public int FrameRateNumerator; + public int FrameRateDenominator; + public int HasAudio; + } + + static FfmpegMovieNative() + { + NativeLibrary.SetDllImportResolver(typeof(FfmpegMovieNative).Assembly, ResolveLibrary); + } + + private static IntPtr ResolveLibrary(string libraryName, System.Reflection.Assembly assembly, + DllImportSearchPath? searchPath) + { + if (!string.Equals(libraryName, LibraryName, StringComparison.Ordinal)) return IntPtr.Zero; + string fileName = OperatingSystem.IsWindows() + ? "age_movie_ffmpeg.dll" + : OperatingSystem.IsMacOS() ? "libage_movie_ffmpeg.dylib" : "libage_movie_ffmpeg.so"; + string? configured = Environment.GetEnvironmentVariable("AGE_FFMPEG_NATIVE_DIR"); + string rid = OperatingSystem.IsWindows() ? "win-x64" + : OperatingSystem.IsMacOS() ? (RuntimeInformation.ProcessArchitecture == Architecture.Arm64 ? "osx-arm64" : "osx-x64") + : "linux-x64"; + string[] candidates = + { + configured == null ? "" : Path.Combine(configured, fileName), + Path.Combine(AppContext.BaseDirectory, fileName), + Path.Combine(AppContext.BaseDirectory, "runtimes", rid, "native", fileName), + }; + foreach (string candidate in candidates) + if (!string.IsNullOrEmpty(candidate) && File.Exists(candidate)) + return NativeLibrary.Load(candidate); + throw new DllNotFoundException( + $"{fileName} was not found; set AGE_FFMPEG_NATIVE_DIR or package runtimes/{rid}/native"); + } + + internal static string DecodeUtf8(byte[] buffer) + { + int length = Array.IndexOf(buffer, (byte)0); + if (length < 0) length = buffer.Length; + return System.Text.Encoding.UTF8.GetString(buffer, 0, length); + } + + internal static string LastError(FfmpegMovieHandle handle) + { + IntPtr text = LastErrorNative(handle); + return text == IntPtr.Zero ? "unknown decoder error" : Marshal.PtrToStringUTF8(text) ?? "unknown decoder error"; + } + + internal static uint AbiVersion() => AbiVersionNative(); + + [DllImport(LibraryName, EntryPoint = "age_movie_abi_version", CallingConvention = CallingConvention.Cdecl)] + private static extern uint AbiVersionNative(); + + [DllImport(LibraryName, EntryPoint = "age_movie_open", CallingConvention = CallingConvention.Cdecl)] + internal static extern int Open( + [In] byte[] bytes, + nuint length, + out FfmpegMovieHandle movie, + out NativeInfo info, + [Out] byte[] errorBuffer, + nuint errorBufferSize); + + [DllImport(LibraryName, EntryPoint = "age_movie_decode_video", CallingConvention = CallingConvention.Cdecl)] + internal static extern int DecodeVideo( + FfmpegMovieHandle movie, + [Out] byte[] rgba, + nuint rgbaSize, + out long presentationTimeMs); + + [DllImport(LibraryName, EntryPoint = "age_movie_last_error", CallingConvention = CallingConvention.Cdecl)] + private static extern IntPtr LastErrorNative(FfmpegMovieHandle movie); + + [DllImport(LibraryName, EntryPoint = "age_movie_close", CallingConvention = CallingConvention.Cdecl)] + internal static extern void Close(IntPtr movie); +} diff --git a/godot/IMovieDecoder.cs b/godot/IMovieDecoder.cs new file mode 100644 index 0000000..da0a0fb --- /dev/null +++ b/godot/IMovieDecoder.cs @@ -0,0 +1,25 @@ +using System; +using System.Runtime.Versioning; +using Age.Engine.Sys4; + +/// +/// Platform-neutral movie playback boundary. Construction is synchronous so metadata needed by the VM is +/// available before op 0x236 returns; frame delivery and completion remain asynchronous. +/// +internal interface IMovieDecoder : IDisposable +{ + long? StopTimeMs { get; } + bool IsCompleted { get; } + bool TryTakeFrame(out RgbaImage frame); +} + +internal interface IMovieDecoderFactory +{ + IMovieDecoder Open(MoviePayload movie); +} + +[SupportedOSPlatform("windows")] +internal sealed class DirectShowMovieDecoderFactory : IMovieDecoderFactory +{ + public IMovieDecoder Open(MoviePayload movie) => new DirectShowMovieDecoder(movie); +} diff --git a/godot/Main.cs b/godot/Main.cs index 926f9d8..a75ddea 100644 --- a/godot/Main.cs +++ b/godot/Main.cs @@ -56,6 +56,7 @@ public partial class Main : Godot.Control // 0x236 creates its graph synchronously on the VM thread so 0x23f can query timing immediately. // Presentation ownership transfers here; _Process adopts staged decoders before sampling frames. private readonly System.Collections.Concurrent.ConcurrentDictionary _pendingMovies = new(); + private IMovieDecoderFactory _movieDecoderFactory = new DirectShowMovieDecoderFactory(); private readonly System.Collections.Generic.HashSet _movieFrameSeen = new(); private readonly System.Collections.Generic.HashSet _movieCompletionNotified = new(); private GodotTraceSink _trace = null!; @@ -1183,7 +1184,7 @@ public partial class Main : Godot.Control try { var payload = new Age.Engine.Sys4.MoviePayload(assetName, mpegBytes); - var runtime = new MovieRuntime(assetName, assetId, new DirectShowMovieDecoder(payload)); + var runtime = MovieRuntime.Open(assetName, assetId, payload, _movieDecoderFactory); stopTimeMs = runtime.Decoder.StopTimeMs; while (!_pendingMovies.TryAdd(resourceId, runtime)) if (_pendingMovies.TryRemove(resourceId, out var prior)) prior.Decoder.Dispose(); @@ -1242,16 +1243,6 @@ public partial class Main : Godot.Control _movieCompletionNotified.Remove(resourceId); } - private sealed record MovieRuntime(string Name, int AssetId, DirectShowMovieDecoder Decoder, - long StartedAtTimestamp, long WatchdogMs) - { - public MovieRuntime(string name, int assetId, DirectShowMovieDecoder decoder) - : this(name, assetId, decoder, Stopwatch.GetTimestamp(), - decoder.StopTimeMs is >= 0 and var stopTime - ? System.Math.Clamp(stopTime + 2000, 5000, 300000) - : 30000) { } - } - public void AppendLine(string text) => _text.Text += text + "\n"; public void PageBreak() { diff --git a/godot/MovieRuntime.cs b/godot/MovieRuntime.cs new file mode 100644 index 0000000..c6c7f12 --- /dev/null +++ b/godot/MovieRuntime.cs @@ -0,0 +1,19 @@ +using System; +using System.Diagnostics; +using Age.Engine.Sys4; + +/// Presentation-side ownership for one decoder plus its fail-safe completion deadline. +internal sealed record MovieRuntime(string Name, int AssetId, IMovieDecoder Decoder, + long StartedAtTimestamp, long WatchdogMs) +{ + public static MovieRuntime Open(string name, int assetId, MoviePayload payload, + IMovieDecoderFactory factory) + { + ArgumentNullException.ThrowIfNull(factory); + IMovieDecoder decoder = factory.Open(payload); + return new MovieRuntime(name, assetId, decoder, Stopwatch.GetTimestamp(), + decoder.StopTimeMs is >= 0 and var stopTime + ? Math.Clamp(stopTime + 2000, 5000, 300000) + : 30000); + } +} diff --git a/native/age_movie_ffmpeg/age_movie.c b/native/age_movie_ffmpeg/age_movie.c new file mode 100644 index 0000000..8fc79fc --- /dev/null +++ b/native/age_movie_ffmpeg/age_movie.c @@ -0,0 +1,475 @@ +#include "age_movie.h" + +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include + +#define AGE_MOVIE_ABI_VERSION 1u +#define AGE_MOVIE_IO_BUFFER_SIZE 32768 +#define AGE_MOVIE_ERROR_SIZE 512 + +struct age_movie { + uint8_t *payload; + size_t payload_size; + size_t payload_position; + AVIOContext *io; + AVFormatContext *format; + AVCodecContext *video_codec; + AVPacket *packet; + AVFrame *frame; + AVFrame *rgba_frame; + struct SwsContext *sws; + int video_stream_index; + AVRational video_time_base; + int64_t timestamp_origin; + int64_t first_frame_timestamp; + int input_eof; + int decoder_draining; + int decoder_eof; + int64_t decoded_frame_count; + int width; + int height; + char error[AGE_MOVIE_ERROR_SIZE]; +}; + +static void copy_error(char *destination, size_t destination_size, const char *message) { + if (destination == NULL || destination_size == 0) { + return; + } + snprintf(destination, destination_size, "%s", message != NULL ? message : "unknown FFmpeg error"); +} + +static void set_error(age_movie *movie, const char *operation, int ffmpeg_error) { + char detail[AV_ERROR_MAX_STRING_SIZE] = {0}; + if (av_strerror(ffmpeg_error, detail, sizeof(detail)) < 0) { + snprintf(detail, sizeof(detail), "FFmpeg error %d", ffmpeg_error); + } + snprintf(movie->error, sizeof(movie->error), "%s: %s", operation, detail); +} + +static int read_packet(void *opaque, uint8_t *buffer, int buffer_size) { + age_movie *movie = (age_movie *)opaque; + if (movie == NULL || buffer == NULL || buffer_size <= 0) { + return AVERROR(EINVAL); + } + if (movie->payload_position >= movie->payload_size) { + return AVERROR_EOF; + } + size_t remaining = movie->payload_size - movie->payload_position; + size_t requested = (size_t)buffer_size; + size_t count = remaining < requested ? remaining : requested; + memcpy(buffer, movie->payload + movie->payload_position, count); + movie->payload_position += count; + return (int)count; +} + +static int64_t seek_payload(void *opaque, int64_t offset, int whence) { + age_movie *movie = (age_movie *)opaque; + if (movie == NULL) { + return AVERROR(EINVAL); + } + if ((whence & AVSEEK_SIZE) != 0) { + return movie->payload_size <= INT64_MAX ? (int64_t)movie->payload_size : AVERROR(EOVERFLOW); + } + + int base_whence = whence & ~AVSEEK_FORCE; + int64_t base; + switch (base_whence) { + case SEEK_SET: base = 0; break; + case SEEK_CUR: + if (movie->payload_position > INT64_MAX) return AVERROR(EOVERFLOW); + base = (int64_t)movie->payload_position; + break; + case SEEK_END: + if (movie->payload_size > INT64_MAX) return AVERROR(EOVERFLOW); + base = (int64_t)movie->payload_size; + break; + default: return AVERROR(EINVAL); + } + if (offset == INT64_MIN || (offset > 0 && base > INT64_MAX - offset) || + (offset < 0 && base < -offset)) { + return AVERROR(EOVERFLOW); + } + int64_t target = base + offset; + if (target < 0 || (uint64_t)target > movie->payload_size) { + return AVERROR(EINVAL); + } + movie->payload_position = (size_t)target; + return target; +} + +static int64_t rescale_ms_down(int64_t value, AVRational time_base) { + return av_rescale_q_rnd(value, time_base, (AVRational){1, 1000}, + AV_ROUND_DOWN | AV_ROUND_PASS_MINMAX); +} + +static int determine_duration(age_movie *movie, AVStream *stream, int64_t *out_duration_ms) { + int64_t best_duration_ms = 0; + if (stream->duration != AV_NOPTS_VALUE && stream->duration > 0) { + int64_t candidate = rescale_ms_down(stream->duration, stream->time_base); + if (candidate > best_duration_ms) best_duration_ms = candidate; + } + if (movie->format->duration != AV_NOPTS_VALUE && movie->format->duration > 0) { + int64_t candidate = av_rescale_q_rnd(movie->format->duration, + (AVRational){1, AV_TIME_BASE}, (AVRational){1, 1000}, + AV_ROUND_DOWN | AV_ROUND_PASS_MINMAX); + if (candidate > best_duration_ms) best_duration_ms = candidate; + } + + int64_t first = AV_NOPTS_VALUE; + int64_t end = AV_NOPTS_VALUE; + int64_t video_packet_count = 0; + AVPacket *probe = av_packet_alloc(); + if (probe == NULL) return AVERROR(ENOMEM); + int result; + while ((result = av_read_frame(movie->format, probe)) >= 0) { + if (probe->stream_index == movie->video_stream_index) { + video_packet_count++; + int64_t timestamp = probe->pts != AV_NOPTS_VALUE ? probe->pts : probe->dts; + if (timestamp != AV_NOPTS_VALUE) { + if (first == AV_NOPTS_VALUE || timestamp < first) first = timestamp; + int64_t packet_end = timestamp; + if (probe->duration > 0 && timestamp <= INT64_MAX - probe->duration) { + packet_end += probe->duration; + } + if (end == AV_NOPTS_VALUE || packet_end > end) end = packet_end; + } + } + av_packet_unref(probe); + } + av_packet_free(&probe); + if (result != AVERROR_EOF) return result; + if (first != AV_NOPTS_VALUE && end != AV_NOPTS_VALUE && end > first) { + int64_t candidate = rescale_ms_down(end - first, stream->time_base); + if (candidate > best_duration_ms) best_duration_ms = candidate; + } + AVRational average_rate = stream->avg_frame_rate; + if (video_packet_count > 0 && average_rate.num > 0 && average_rate.den > 0) { + int64_t candidate = av_rescale_q_rnd(video_packet_count, av_inv_q(average_rate), + (AVRational){1, 1000}, AV_ROUND_DOWN | AV_ROUND_PASS_MINMAX); + if (candidate > best_duration_ms) best_duration_ms = candidate; + } + *out_duration_ms = best_duration_ms; + return best_duration_ms > 0 ? 0 : AVERROR_INVALIDDATA; +} + +static int rewind_for_decode(age_movie *movie, AVStream *stream) { + int64_t target = stream->start_time != AV_NOPTS_VALUE ? stream->start_time : 0; + int result = av_seek_frame(movie->format, movie->video_stream_index, target, AVSEEK_FLAG_BACKWARD); + if (result < 0) { + movie->payload_position = 0; + if (movie->io != NULL) { + avio_flush(movie->io); + movie->io->eof_reached = 0; + movie->io->error = 0; + } + result = avformat_seek_file(movie->format, movie->video_stream_index, + INT64_MIN, target, INT64_MAX, 0); + } + if (result >= 0) avcodec_flush_buffers(movie->video_codec); + return result; +} + +static void destroy_movie(age_movie *movie) { + if (movie == NULL) return; + sws_freeContext(movie->sws); + av_frame_free(&movie->rgba_frame); + av_frame_free(&movie->frame); + av_packet_free(&movie->packet); + avcodec_free_context(&movie->video_codec); + avformat_close_input(&movie->format); + if (movie->io != NULL) { + av_freep(&movie->io->buffer); + avio_context_free(&movie->io); + } + av_free(movie->payload); + free(movie); +} + +uint32_t AGE_MOVIE_CALL age_movie_abi_version(void) { + return AGE_MOVIE_ABI_VERSION; +} + +int32_t AGE_MOVIE_CALL age_movie_open(const uint8_t *bytes, size_t length, + age_movie **out_movie, age_movie_info *out_info, + char *error_buffer, size_t error_buffer_size) { + if (out_movie != NULL) *out_movie = NULL; + if (out_info != NULL) memset(out_info, 0, sizeof(*out_info)); + if (bytes == NULL || length == 0 || out_movie == NULL || out_info == NULL) { + copy_error(error_buffer, error_buffer_size, "invalid movie open arguments"); + return AGE_MOVIE_INVALID_ARGUMENT; + } + + age_movie *movie = (age_movie *)calloc(1, sizeof(*movie)); + if (movie == NULL) { + copy_error(error_buffer, error_buffer_size, "movie allocation failed"); + return AGE_MOVIE_ERROR; + } + movie->video_stream_index = -1; + movie->timestamp_origin = AV_NOPTS_VALUE; + movie->first_frame_timestamp = AV_NOPTS_VALUE; + + if (length > SIZE_MAX - AV_INPUT_BUFFER_PADDING_SIZE) { + snprintf(movie->error, sizeof(movie->error), "movie payload is too large"); + goto failure; + } + movie->payload = (uint8_t *)av_mallocz(length + AV_INPUT_BUFFER_PADDING_SIZE); + if (movie->payload == NULL) { + snprintf(movie->error, sizeof(movie->error), "movie payload allocation failed"); + goto failure; + } + memcpy(movie->payload, bytes, length); + movie->payload_size = length; + + uint8_t *io_buffer = (uint8_t *)av_malloc(AGE_MOVIE_IO_BUFFER_SIZE); + if (io_buffer == NULL) { + snprintf(movie->error, sizeof(movie->error), "movie IO buffer allocation failed"); + goto failure; + } + movie->io = avio_alloc_context(io_buffer, AGE_MOVIE_IO_BUFFER_SIZE, 0, + movie, read_packet, NULL, seek_payload); + if (movie->io == NULL) { + av_free(io_buffer); + snprintf(movie->error, sizeof(movie->error), "movie AVIO context allocation failed"); + goto failure; + } + + movie->format = avformat_alloc_context(); + if (movie->format == NULL) { + snprintf(movie->error, sizeof(movie->error), "movie format context allocation failed"); + goto failure; + } + movie->format->pb = movie->io; + movie->format->flags |= AVFMT_FLAG_CUSTOM_IO; + int result = avformat_open_input(&movie->format, NULL, NULL, NULL); + if (result < 0) { + set_error(movie, "open MPEG program stream", result); + goto failure; + } + result = avformat_find_stream_info(movie->format, NULL); + if (result < 0) { + set_error(movie, "probe MPEG streams", result); + goto failure; + } + + const AVCodec *video_decoder = NULL; + result = av_find_best_stream(movie->format, AVMEDIA_TYPE_VIDEO, -1, -1, &video_decoder, 0); + if (result < 0 || video_decoder == NULL) { + set_error(movie, "find MPEG video stream", result < 0 ? result : AVERROR_DECODER_NOT_FOUND); + goto failure; + } + movie->video_stream_index = result; + AVStream *stream = movie->format->streams[movie->video_stream_index]; + movie->video_time_base = stream->time_base; + movie->timestamp_origin = stream->start_time; + + movie->video_codec = avcodec_alloc_context3(video_decoder); + if (movie->video_codec == NULL) { + snprintf(movie->error, sizeof(movie->error), "video decoder allocation failed"); + goto failure; + } + result = avcodec_parameters_to_context(movie->video_codec, stream->codecpar); + if (result < 0) { + set_error(movie, "copy video parameters", result); + goto failure; + } + result = avcodec_open2(movie->video_codec, video_decoder, NULL); + if (result < 0) { + set_error(movie, "open MPEG-1 video decoder", result); + goto failure; + } + movie->width = movie->video_codec->width; + movie->height = movie->video_codec->height; + if (movie->width <= 0 || movie->height <= 0) { + snprintf(movie->error, sizeof(movie->error), "invalid video dimensions %dx%d", movie->width, movie->height); + goto failure; + } + if ((size_t)movie->width > SIZE_MAX / 4u / (size_t)movie->height) { + snprintf(movie->error, sizeof(movie->error), "video dimensions overflow RGBA storage"); + goto failure; + } + + int64_t duration_ms = 0; + result = rewind_for_decode(movie, stream); + if (result < 0) { + set_error(movie, "rewind MPEG stream for duration scan", result); + goto failure; + } + result = determine_duration(movie, stream, &duration_ms); + if (result < 0) { + set_error(movie, "determine video duration", result); + goto failure; + } + result = rewind_for_decode(movie, stream); + if (result < 0) { + set_error(movie, "rewind MPEG stream", result); + goto failure; + } + + movie->packet = av_packet_alloc(); + movie->frame = av_frame_alloc(); + movie->rgba_frame = av_frame_alloc(); + if (movie->packet == NULL || movie->frame == NULL || movie->rgba_frame == NULL) { + snprintf(movie->error, sizeof(movie->error), "video packet/frame allocation failed"); + goto failure; + } + movie->rgba_frame->format = AV_PIX_FMT_RGBA; + movie->rgba_frame->width = movie->width; + movie->rgba_frame->height = movie->height; + result = av_frame_get_buffer(movie->rgba_frame, 32); + if (result < 0) { + set_error(movie, "allocate aligned RGBA frame", result); + goto failure; + } + + AVRational frame_rate = av_guess_frame_rate(movie->format, stream, NULL); + out_info->width = movie->width; + out_info->height = movie->height; + out_info->stop_time_ms = duration_ms; + out_info->frame_rate_num = frame_rate.num; + out_info->frame_rate_den = frame_rate.den; + for (unsigned int index = 0; index < movie->format->nb_streams; index++) { + if (movie->format->streams[index]->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) { + out_info->has_audio = 1; + break; + } + } + *out_movie = movie; + copy_error(error_buffer, error_buffer_size, ""); + return 0; + +failure: + copy_error(error_buffer, error_buffer_size, movie->error); + destroy_movie(movie); + return AGE_MOVIE_ERROR; +} + +int32_t AGE_MOVIE_CALL age_movie_decode_video(age_movie *movie, + uint8_t *rgba, size_t rgba_size, int64_t *out_pts_ms) { + if (movie == NULL || rgba == NULL || out_pts_ms == NULL) return AGE_MOVIE_INVALID_ARGUMENT; + size_t required = (size_t)movie->width * (size_t)movie->height * 4u; + if (rgba_size < required) return AGE_MOVIE_BUFFER_TOO_SMALL; + if (movie->decoder_eof) return AGE_MOVIE_EOF; + + for (;;) { + int result = avcodec_receive_frame(movie->video_codec, movie->frame); + if (result == 0) { + movie->sws = sws_getCachedContext(movie->sws, + movie->frame->width, movie->frame->height, (enum AVPixelFormat)movie->frame->format, + movie->width, movie->height, AV_PIX_FMT_RGBA, + SWS_BILINEAR, NULL, NULL, NULL); + if (movie->sws == NULL) { + snprintf(movie->error, sizeof(movie->error), "create RGBA conversion context failed"); + return AGE_MOVIE_ERROR; + } + result = av_frame_make_writable(movie->rgba_frame); + if (result < 0) { + set_error(movie, "make aligned RGBA frame writable", result); + return AGE_MOVIE_ERROR; + } + result = sws_scale(movie->sws, + (const uint8_t * const *)movie->frame->data, movie->frame->linesize, + 0, movie->frame->height, movie->rgba_frame->data, movie->rgba_frame->linesize); + if (result != movie->height) { + snprintf(movie->error, sizeof(movie->error), "RGBA conversion returned %d of %d rows", result, movie->height); + return AGE_MOVIE_ERROR; + } + size_t row_size = (size_t)movie->width * 4u; + for (int row = 0; row < movie->height; row++) { + memcpy(rgba + (size_t)row * row_size, + movie->rgba_frame->data[0] + (ptrdiff_t)row * movie->rgba_frame->linesize[0], + row_size); + } + + int64_t timestamp = movie->frame->best_effort_timestamp; + if (timestamp == AV_NOPTS_VALUE) { + timestamp = movie->frame->pts; + } + if (timestamp == AV_NOPTS_VALUE) { + AVRational rate = av_guess_frame_rate(movie->format, + movie->format->streams[movie->video_stream_index], movie->frame); + if (rate.num <= 0 || rate.den <= 0) { + snprintf(movie->error, sizeof(movie->error), "decoded frame has no timestamp or frame rate"); + return AGE_MOVIE_ERROR; + } + *out_pts_ms = av_rescale_q_rnd(movie->decoded_frame_count, av_inv_q(rate), + (AVRational){1, 1000}, AV_ROUND_DOWN | AV_ROUND_PASS_MINMAX); + } else { + if (movie->timestamp_origin == AV_NOPTS_VALUE) { + if (movie->first_frame_timestamp == AV_NOPTS_VALUE) movie->first_frame_timestamp = timestamp; + movie->timestamp_origin = movie->first_frame_timestamp; + } + int64_t relative = timestamp >= movie->timestamp_origin ? timestamp - movie->timestamp_origin : 0; + *out_pts_ms = rescale_ms_down(relative, movie->video_time_base); + } + movie->decoded_frame_count++; + av_frame_unref(movie->frame); + return AGE_MOVIE_FRAME; + } + if (result == AVERROR_EOF) { + movie->decoder_eof = 1; + return AGE_MOVIE_EOF; + } + if (result != AVERROR(EAGAIN)) { + set_error(movie, "receive decoded video frame", result); + return AGE_MOVIE_ERROR; + } + + if (movie->input_eof) { + if (!movie->decoder_draining) { + result = avcodec_send_packet(movie->video_codec, NULL); + if (result < 0 && result != AVERROR_EOF) { + set_error(movie, "flush MPEG-1 video decoder", result); + return AGE_MOVIE_ERROR; + } + movie->decoder_draining = 1; + continue; + } + snprintf(movie->error, sizeof(movie->error), "video decoder requested input after drain"); + return AGE_MOVIE_ERROR; + } + + for (;;) { + result = av_read_frame(movie->format, movie->packet); + if (result == AVERROR_EOF) { + movie->input_eof = 1; + break; + } + if (result < 0) { + set_error(movie, "read MPEG video packet", result); + return AGE_MOVIE_ERROR; + } + if (movie->packet->stream_index != movie->video_stream_index) { + av_packet_unref(movie->packet); + continue; + } + break; + } + + if (movie->input_eof) continue; + result = avcodec_send_packet(movie->video_codec, movie->packet); + av_packet_unref(movie->packet); + if (result < 0) { + set_error(movie, "send MPEG video packet", result); + return AGE_MOVIE_ERROR; + } + } +} + +const char *AGE_MOVIE_CALL age_movie_last_error(const age_movie *movie) { + return movie != NULL ? movie->error : "movie handle is null"; +} + +void AGE_MOVIE_CALL age_movie_close(age_movie *movie) { + destroy_movie(movie); +} diff --git a/native/age_movie_ffmpeg/age_movie.h b/native/age_movie_ffmpeg/age_movie.h new file mode 100644 index 0000000..4b530c2 --- /dev/null +++ b/native/age_movie_ffmpeg/age_movie.h @@ -0,0 +1,63 @@ +#ifndef AGE_MOVIE_H +#define AGE_MOVIE_H + +#include +#include + +#if defined(_WIN32) +#define AGE_MOVIE_API __declspec(dllexport) +#define AGE_MOVIE_CALL __cdecl +#else +#define AGE_MOVIE_API __attribute__((visibility("default"))) +#define AGE_MOVIE_CALL +#endif + +#ifdef __cplusplus +extern "C" { +#endif + +typedef struct age_movie age_movie; + +typedef struct age_movie_info { + int32_t width; + int32_t height; + int64_t stop_time_ms; + int32_t frame_rate_num; + int32_t frame_rate_den; + int32_t has_audio; +} age_movie_info; + +enum age_movie_result { + AGE_MOVIE_EOF = 0, + AGE_MOVIE_FRAME = 1, + AGE_MOVIE_ERROR = -1, + AGE_MOVIE_INVALID_ARGUMENT = -2, + AGE_MOVIE_BUFFER_TOO_SMALL = -3 +}; + +AGE_MOVIE_API uint32_t AGE_MOVIE_CALL age_movie_abi_version(void); + +/* Copies bytes before returning; the caller's buffer is borrowed only during this call. */ +AGE_MOVIE_API int32_t AGE_MOVIE_CALL age_movie_open( + const uint8_t *bytes, + size_t length, + age_movie **out_movie, + age_movie_info *out_info, + char *error_buffer, + size_t error_buffer_size); + +/* Writes one tightly packed top-down RGBA8 frame and its zero-based presentation timestamp. */ +AGE_MOVIE_API int32_t AGE_MOVIE_CALL age_movie_decode_video( + age_movie *movie, + uint8_t *rgba, + size_t rgba_size, + int64_t *out_pts_ms); + +AGE_MOVIE_API const char *AGE_MOVIE_CALL age_movie_last_error(const age_movie *movie); +AGE_MOVIE_API void AGE_MOVIE_CALL age_movie_close(age_movie *movie); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/native/age_movie_ffmpeg/bootstrap-win64.ps1 b/native/age_movie_ffmpeg/bootstrap-win64.ps1 new file mode 100644 index 0000000..9a2b6f2 --- /dev/null +++ b/native/age_movie_ffmpeg/bootstrap-win64.ps1 @@ -0,0 +1,35 @@ +param( + [string]$Destination +) + +$ErrorActionPreference = 'Stop' +$repoRoot = (Resolve-Path (Join-Path $PSScriptRoot '..\..')).Path +if ([string]::IsNullOrWhiteSpace($Destination)) { + $Destination = Join-Path $repoRoot 'build\ffmpeg-sdk' +} +$manifest = Get-Content -LiteralPath (Join-Path $PSScriptRoot 'dependency-win64.json') -Raw | ConvertFrom-Json +$downloadDir = Join-Path $repoRoot 'build\downloads' +$archivePath = Join-Path $downloadDir $manifest.archive +$extractRoot = Join-Path $Destination ([IO.Path]::GetFileNameWithoutExtension($manifest.archive)) +New-Item -ItemType Directory -Force -Path $downloadDir | Out-Null + +if (-not (Test-Path -LiteralPath $archivePath)) { + Invoke-WebRequest -Uri $manifest.url -OutFile $archivePath +} +$actualHash = (Get-FileHash -Algorithm SHA256 -LiteralPath $archivePath).Hash.ToLowerInvariant() +if ($actualHash -ne $manifest.sha256) { + throw "FFmpeg archive SHA-256 mismatch: expected $($manifest.sha256), got $actualHash" +} +if (-not (Test-Path -LiteralPath $extractRoot)) { + Expand-Archive -LiteralPath $archivePath -DestinationPath $Destination +} +$sdkRoot = Get-ChildItem -LiteralPath $Destination -Directory | + Where-Object { Test-Path -LiteralPath (Join-Path $_.FullName 'include\libavformat\avformat.h') } | + Select-Object -First 1 -ExpandProperty FullName +if (-not $sdkRoot) { throw "FFmpeg SDK was not found under $Destination" } +$reported = & (Join-Path $sdkRoot 'bin\ffmpeg.exe') -version | Select-Object -First 1 +$expectedVersion = $manifest.ffmpeg_version -replace '-20260721$', '' +if ($reported -notlike "*$expectedVersion*" ) { + throw "Unexpected FFmpeg build: $reported" +} +Write-Output $sdkRoot diff --git a/native/age_movie_ffmpeg/build-win64.ps1 b/native/age_movie_ffmpeg/build-win64.ps1 new file mode 100644 index 0000000..a31920d --- /dev/null +++ b/native/age_movie_ffmpeg/build-win64.ps1 @@ -0,0 +1,43 @@ +param( + [Parameter(Mandatory = $true)] + [string]$SdkRoot, + [string]$OutputDirectory +) + +$ErrorActionPreference = 'Stop' +$repoRoot = (Resolve-Path (Join-Path $PSScriptRoot '..\..')).Path +$SdkRoot = (Resolve-Path $SdkRoot).Path +if ([string]::IsNullOrWhiteSpace($OutputDirectory)) { + $OutputDirectory = Join-Path $repoRoot 'build\native\win-x64' +} +$include = Join-Path $SdkRoot 'include' +$lib = Join-Path $SdkRoot 'lib' +$bin = Join-Path $SdkRoot 'bin' +foreach ($required in @( + (Join-Path $include 'libavformat\avformat.h'), + (Join-Path $lib 'avformat.lib'), + (Join-Path $bin 'avformat-62.dll') +)) { + if (-not (Test-Path -LiteralPath $required)) { throw "Missing FFmpeg SDK file: $required" } +} + +$vswhere = Join-Path ${env:ProgramFiles(x86)} 'Microsoft Visual Studio\Installer\vswhere.exe' +if (-not (Test-Path -LiteralPath $vswhere)) { throw "Visual Studio Installer vswhere.exe was not found" } +$vsInstall = & $vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath +if (-not $vsInstall) { throw "A Visual C++ x64 build environment was not found" } +$vcvars = Join-Path $vsInstall 'VC\Auxiliary\Build\vcvars64.bat' +if (-not (Test-Path -LiteralPath $vcvars)) { throw "Visual C++ x64 environment not found: $vcvars" } +New-Item -ItemType Directory -Force -Path $OutputDirectory | Out-Null +$source = Join-Path $PSScriptRoot 'age_movie.c' +$object = Join-Path $OutputDirectory 'age_movie.obj' +$dll = Join-Path $OutputDirectory 'age_movie_ffmpeg.dll' +$importLibrary = Join-Path $OutputDirectory 'age_movie_ffmpeg.lib' +$compile = 'call "{0}" && cl.exe /nologo /std:c11 /utf-8 /O2 /MD /W4 /external:I"{1}" /external:W0 /LD /Fo"{2}" "{3}" /link /OUT:"{4}" /IMPLIB:"{5}" /LIBPATH:"{6}" avformat.lib avcodec.lib avutil.lib swscale.lib' -f + $vcvars, $include, $object, $source, $dll, $importLibrary, $lib +& cmd.exe /d /s /c $compile +if ($LASTEXITCODE -ne 0) { throw "age_movie_ffmpeg compilation failed with exit code $LASTEXITCODE" } +foreach ($runtime in @('avformat-62.dll', 'avcodec-62.dll', 'avutil-60.dll', 'swscale-9.dll', 'swresample-6.dll')) { + Copy-Item -LiteralPath (Join-Path $bin $runtime) -Destination $OutputDirectory -Force +} +Copy-Item -LiteralPath (Join-Path $SdkRoot 'LICENSE.txt') -Destination (Join-Path $OutputDirectory 'FFmpeg-LICENSE.txt') -Force +Write-Output $dll diff --git a/native/age_movie_ffmpeg/dependency-win64.json b/native/age_movie_ffmpeg/dependency-win64.json new file mode 100644 index 0000000..fb27197 --- /dev/null +++ b/native/age_movie_ffmpeg/dependency-win64.json @@ -0,0 +1,10 @@ +{ + "provider": "BtbN/FFmpeg-Builds", + "release_tag": "autobuild-2026-07-21-13-38", + "archive": "ffmpeg-n8.1.2-29-g703dcc25b9-win64-lgpl-shared-8.1.zip", + "url": "https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2026-07-21-13-38/ffmpeg-n8.1.2-29-g703dcc25b9-win64-lgpl-shared-8.1.zip", + "sha256": "50764b52d38cb0baf8af938c0e4ca886d2c6753b520b2c319b532ed5c17e7cbf", + "ffmpeg_version": "n8.1.2-29-g703dcc25b9-20260721", + "ffmpeg_commit": "703dcc25b9", + "variant": "win64 LGPL shared 8.1" +}