Enable paced FFmpeg movie playback

This commit is contained in:
gamer147
2026-07-22 08:39:27 -04:00
parent 760f84e218
commit 816673cf6a
12 changed files with 365 additions and 22 deletions

View File

@@ -601,11 +601,11 @@ dependency pin, corpus codec inventory, timing contract, and packaging rules liv
stop times from one formerly failing 280x352 effect (`MVB961` or `MVB238`), aligned `MVB908`, and 800x600
`CHAPTER`. It must also reject a truncated payload with a bounded diagnostic and survive repeated open/close.
Do not switch the live runtime at this step.
3. **Next — implement managed pacing and switch the factory.** Decode on a cancellable worker using FFmpeg timestamps
3. **Completed — implement managed pacing and switch the factory.** Decode on a cancellable worker using FFmpeg timestamps
and a monotonic clock, publish only due frames, delay EOF completion through the final presentation interval,
and retain the existing watchdog. Make FFmpeg the default with no dimension/effect dispatch and exercise both
non-modal `0x236` and modal/cancellable `0x20f` paths.
4. **Run the installed-corpus gate.** Every one of the 213 MPEG payloads must open, report its expected display
4. **Next — run the installed-corpus gate.** Every one of the 213 MPEG payloads must open, report its expected display
dimensions and a positive stop time, produce a correctly sized RGBA frame, maintain nondecreasing timestamps,
reach EOF, and dispose within a bound. Normal tests use small project-authored 280-wide and aligned MPEG
fixtures so decoder behavior is never disabled when the original install is absent; the full local sweep is
@@ -632,6 +632,21 @@ The complete 313-test suite, three additional 312-test concurrency-sensitive rep
and threaded selftest pass. Step 3 is the next bounded slice; do not infer that combat is fixed until the paced
backend is selected live and revalidated.
Step 3 landed on 2026-07-22. `FfmpegMovieDecoder` owns the native session on a cancellable background thread,
decodes at most one frame ahead, waits against a monotonic `Stopwatch` clock for each normalized PTS, preserves
newest-frame-wins handoff, and reports EOF only after the larger of declared stop time or the final frame interval.
Disposal interrupts a far-future frame wait and joins the worker; asynchronous decode failure records a diagnostic
and completes the decoder so AGE cannot remain blocked. `Main` now selects `FfmpegMovieDecoderFactory`, while the
Godot build stages the shim and its five local shared-library dependencies beside `Himegari.dll`. DirectShow stays
in-tree but is no longer selected; delete it only after the corpus and live acceptance gates. Deterministic fake-
clock tests cover due-frame publication, final completion, cancellation, and failure. Real paced probes cover
formerly failing 280x352 `MVB961` and aligned `MVB908`. A natural headless SYSTEM4 smoke played and released
`LOGO.AGF` at its reported 7288 ms, then opened 106919 ms `OP.AGF` and published its first frame before bounded
shutdown. The user then confirmed the opening movies play correctly in a normal windowed run. The complete suite
is 318 tests, the Godot build is warning-free, and threaded selftest passes. **NEXT:** run the 213-payload
installed-corpus gate, then validate SC0000 CHAPTER and the original battle exchange before removing DirectShow
or judging the tabled combat stall.
**Mutable-surface fill/blend regression corrected.** The first visual recheck exposed BUNKI's menu interior
as transparent. SYSTEM4 creates 800x600 surface 3 and fills it opaque white through `0x20b`; the metadata-only
host fill left the new pixel buffer transparent. Implementing the fill alone made the panel solid gray and

View File

@@ -16,16 +16,18 @@ The VM and content pipeline are already mostly platform-neutral:
- Movie payloads arrive from `IAssetStore` as owned bytes and decoded frames enter the compositor as the
platform-neutral `RgbaImage` type.
The current runtime's only direct native Windows API use is the movie decoder described below. There are
also softer Windows assumptions that should be tested or replaced before claiming portable exports.
The selected movie path now uses the project-owned FFmpeg C ABI rather than a Windows multimedia API, but only a
Windows-x64 native bundle is built and staged today. The retired-live DirectShow implementation remains in-tree
until the corpus and manual gates pass. There are also softer Windows assumptions that should be tested or
replaced before claiming portable exports.
## Dependency inventory
| Area | Current dependency | Runtime impact | Portability status / future action |
|---|---|---|---|
| AGE movie decode (`0x236` scene movies; `0x20f` modal LOGO/OP/ED) | Live playback still uses `godot/DirectShowMovieDecoder.cs`; the isolated `native/age_movie_ffmpeg` C ABI and `FfmpegMovieSession` probe path now decode the representative VFS movies | Live movies remain Windows-only and the selected live backend still rejects installed MPEG effects whose display width is 8 mod 16 | Add timestamp pacing to the proven FFmpeg path, select it as the sole live factory, run the corpus/live gates, then delete DirectShow |
| Movie integration | `MovieRuntime` now owns `IMovieDecoder` from an injected `IMovieDecoderFactory`; `Main` deliberately selects `DirectShowMovieDecoderFactory` and remains marked `SupportedOSPlatform("windows")` | Backend ownership is portable, but the selected live implementation is not yet | Implement the FFmpeg `IMovieDecoder`, switch the factory, and remove the Windows annotation when DirectShow is deleted |
| Movie audio | DirectShow connects only the video pin to the sample grabber/null renderer | The MPEG audio stream is intentionally silent on every platform | Design a PCM/audio-clock contract or let a future backend own synchronized A/V; separate feature slice |
| AGE movie decode (`0x236` scene movies; `0x20f` modal LOGO/OP/ED) | `FfmpegMovieDecoder` is the selected live factory over the project-owned `native/age_movie_ffmpeg` ABI; `DirectShowMovieDecoder` remains unselected pending deletion | Windows-x64 live playback now covers formerly rejected 280-wide effects, but other native targets and the full 213-payload gate remain | Run the corpus and windowed live gates, then delete DirectShow and add target-specific native builds |
| Movie integration | `MovieRuntime` owns `IMovieDecoder` from an injected factory; the FFmpeg worker paces PTS against a monotonic clock and supports cancellation/failure completion | Backend ownership is portable, while `Main` remains annotated Windows because only the win-x64 bundle is available | Add Linux/macOS builds and remove the Windows annotation after DirectShow is deleted |
| Movie audio | FFmpeg detects the audio stream but the current ABI returns video frames only | MPEG movie audio remains intentionally silent | Extend the ABI with timestamped PCM and select an audio/presentation clock; separate feature slice |
| ADV font discovery | `godot/Main.cs` probes `C:/Windows/Fonts` for Japanese fonts | Harmless fallback today, but appearance depends on host fonts | Bundle/configure a redistributable font or add platform-specific discovery |
| Filesystem semantics | Several filename and containment comparisons use `OrdinalIgnoreCase`; installed assets are conventionally uppercase | Needs validation on case-sensitive filesystems; may hide casing or containment mistakes | Add Linux/macOS tests with mixed-case synthetic roots and use filesystem-appropriate containment rules |
| Install/repository discovery | `engine/Age.Engine/Sys4/Paths.cs` finds `age-reimpl` above `AppContext.BaseDirectory` and assumes the current workspace sibling layout | Suitable for development, not packaged exports on any OS | Replace runtime discovery with a user-selected game root/profile; retain repository paths only for developer tools/tests |
@@ -39,15 +41,16 @@ Godot abstractions.
## Movie backend replacement seam
The existing connection is localized but one abstraction short of being replaceable without edits:
The live connection is now backend-neutral:
```
VM op 0x236 (non-modal) / op 0x20f (modal)
-> IHost.PlayMovieToSurface / PlayModalMovieToSurface
-> VFS-owned MoviePayload bytes
-> IMovieDecoderFactory
-> DirectShowMovieDecoder (current live selection)
-> FfmpegMovieSession -> age_movie C ABI (isolated gate only)
-> FfmpegMovieDecoder (current live selection)
-> FfmpegMovieSession -> age_movie C ABI
-> DirectShowMovieDecoder (unselected; retained only through acceptance)
-> newest RGBA frame
-> retained movie surface
-> Godot compositor
@@ -91,8 +94,9 @@ safety rule still converts any backend initialization failure into an
explicitly completed zero-duration movie identity, and the presentation watchdog remains a last-resort guard for
an initialized backend that never reaches EOF.
The factory is injection for tests and future decoder replacement, not runtime codec roulette. Once the FFmpeg
corpus gate passes it becomes the only shipped movie backend on every platform and DirectShow is deleted. Modal
The factory is injection for tests and future decoder replacement, not runtime codec roulette. FFmpeg is now the
selected Windows-x64 live backend; once its corpus and live gates pass, DirectShow is deleted rather than shipped
as a fallback. Modal
completion/cancel remains owned above the decoder by the existing `0x20f` host path. A future audio implementation
will consume timestamped PCM and may become the presentation clock; it must not change the VM-facing stop-time,
surface, or cancellation contracts.
@@ -125,7 +129,11 @@ shared libraries, and license under disposable `build/native/win-x64`. The manag
`runtimes/<rid>/native` lookup. Representative VFS results are `MVB961` 280x352/500 ms, `MVB238`
280x352/866 ms, `MVB908` 400x400/333 ms, and `CHAPTER` 800x600/12016 ms, all with changing frames and
nondecreasing timestamps. Malformed input and repeated teardown are covered. This is not yet distributable
packaging and does not change the current DirectShow live selection.
packaging. `FfmpegMovieDecoder` now adds cancellable timestamp pacing and is the live selection. The Godot build
copies the shim, five required shared libraries, and FFmpeg license beside `Himegari.dll`; a natural SYSTEM4
smoke completed 7288 ms `LOGO.AGF`, opened 106919 ms `OP.AGF`, and published frames from both without loading
DirectShow; the user subsequently confirmed both opening movies work in normal windowed playback. Full
export/source-offer packaging and non-Windows builds remain outstanding.
## Cross-platform validation gates

View File

@@ -74,8 +74,8 @@ All opcode knowledge (ABI, semantics, provenance, `depends_on`) is hand-edited *
## Native FFmpeg movie shim (Windows x64)
These PowerShell tools build the isolated native movie gate; they do not switch live Godot playback away from
DirectShow. The dependency manifest pins an immutable LGPL shared FFmpeg archive and SHA-256.
These PowerShell tools build the selected Windows-x64 live movie backend. The dependency manifest pins an
immutable LGPL shared FFmpeg archive and SHA-256.
| Tool | Purpose | Run | Reads → Writes |
|---|---|---|---|
@@ -84,7 +84,9 @@ DirectShow. The dependency manifest pins an immutable LGPL shared FFmpeg archive
The managed isolated probes load from `AGE_FFMPEG_NATIVE_DIR` when set, then application-local and
`runtimes/win-x64/native` locations. Build the shim before running the `FfmpegShim*` tests; no original-game
movie is copied into the repository or native output.
movie is copied into the repository or native output. When `build/native/win-x64/age_movie_ffmpeg.dll` exists,
`dotnet build godot/Himegari.csproj` also stages the shim, its five DLL dependencies, and `FFmpeg-LICENSE.txt`
beside `Himegari.dll` for development playback.
## Engine (C#) — VM core, CLI, Godot frontend