Add isolated FFmpeg movie decoder shim

This commit is contained in:
gamer147
2026-07-22 00:17:27 -04:00
parent 633e3b3085
commit 5f115e694f
16 changed files with 1118 additions and 36 deletions

View File

@@ -100,6 +100,9 @@ S:\Game Hacking\Eushully\Himegari\ ← workspace root (three siblings)
├── engine/ DELIVERABLE — the .NET VM core (AgeEngine.sln: Age.Engine / Age.Cli / tests)
│ └── Age.Engine/Sys4/ runtime catalog parser, loose-first bounded ALF asset store,
│ script provider, AGF/LZSS and Windows CUR decoders, and resource facade
├── native/ authored native runtime boundaries
│ └── age_movie_ffmpeg/ project-owned FFmpeg C ABI, immutable Windows dependency manifest,
│ and bootstrap/build scripts (outputs stay under disposable build/)
├── tools/frida/ runtime-capture + engine-dump scripts (see tools/frida/README.md)
└── godot/ DELIVERABLE — the Godot/C# ADV front-end (references Age.Engine),
including the TITLE-only F4 debug scene launcher

View File

@@ -582,9 +582,55 @@ an infinite callback count: the total horizon starts at `base_time + 1000`, and
loop polls `0x23a` until surfaces 7..10 are inactive. MVB908's stop log occurs after that loop. As a bounded
safety correction, decoder failure now becomes an explicitly completed zero-duration effect, while started
movies have a stop-time-based completion watchdog so a missing EOF cannot hold `0x21c` forever. **NEXT:**
manually recheck the same exchange. If it still stalls, capture the VM/service coordinate after BTL cleanup
as a separate timed-presentation issue; otherwise proceed to the software MPEG backend for real 280x352
duration/frames. Preserve created destination dimensions and failed-movie identity through that backend change.
replace DirectShow through the bounded FFmpeg plan below. The user confirmed that combat can still stall, but
that symptom remains tabled until the replacement removes DirectShow as a variable. Preserve created destination
dimensions and failed-movie identity through the backend change.
### Portable FFmpeg movie backend selected; bounded implementation plan (2026-07-21)
FFmpeg is selected as the single movie backend for stock content and future profiles. The exact native ABI,
dependency pin, corpus codec inventory, timing contract, and packaging rules live in
`docs/platform-portability.md`; this section owns execution order and acceptance.
1. **Completed — land the seam without changing behavior.** Add `IMovieDecoder` plus an injected factory, type
`MovieRuntime` against the interface, and adapt DirectShow temporarily. Cover synchronous metadata, frame
handoff, EOF, failure-as-completed, replacement, cancellation, and disposal with fakes. This commit must retain
current live behavior and gives the FFmpeg work a testable boundary.
2. **Completed — prove the native ABI in isolation on Windows x64.** Add the C shim and pinned shared FFmpeg dependency,
then open VFS-owned bytes without a temporary file. The initial gate decodes changing RGBA frames and positive
stop times from one formerly failing 280x352 effect (`MVB961` or `MVB238`), aligned `MVB908`, and 800x600
`CHAPTER`. It must also reject a truncated payload with a bounded diagnostic and survive repeated open/close.
Do not switch the live runtime at this step.
3. **Next — implement managed pacing and switch the factory.** Decode on a cancellable worker using FFmpeg timestamps
and a monotonic clock, publish only due frames, delay EOF completion through the final presentation interval,
and retain the existing watchdog. Make FFmpeg the default with no dimension/effect dispatch and exercise both
non-modal `0x236` and modal/cancellable `0x20f` paths.
4. **Run the installed-corpus gate.** Every one of the 213 MPEG payloads must open, report its expected display
dimensions and a positive stop time, produce a correctly sized RGBA frame, maintain nondecreasing timestamps,
reach EOF, and dispose within a bound. Normal tests use small project-authored 280-wide and aligned MPEG
fixtures so decoder behavior is never disabled when the original install is absent; the full local sweep is
an additional release/acceptance gate. Spot comparisons against DirectShow-compatible assets establish timing
tolerance before DirectShow is removed.
5. **Validate the live sequences, then delete DirectShow.** Recheck LOGO/OP cancellation, SC0000 CHAPTER,
aligned and 280-wide battle effects, combat cleanup, and the previously observed post-effect stall. Once those
pass, delete the COM declarations/temp-file path and Windows platform annotation rather than retaining a
fallback. If combat still stalls with FFmpeg, capture the VM/service coordinate after BTL cleanup and treat it
as a separate battle-timeline defect.
6. **Complete distributable packaging.** Bundle pinned dynamically linked libraries and exact license/source
provenance per target, validate loader isolation from machine-installed codecs, then add Linux/macOS build and
smoke gates. Timestamped PCM delivery and Godot `AudioStreamGenerator` integration remain a later movie-audio
slice; the video replacement must merely avoid closing that path.
Steps 1 and 2 landed on 2026-07-21 without changing live playback: `Main` owns an
`IMovieDecoderFactory`, `MovieRuntime` is backend-neutral, and the default factory still constructs DirectShow.
The isolated Windows-x64 C ABI opens VFS bytes through seekable custom AVIO and decodes changing RGBA frames from
both formerly failing 280x352 effects (`MVB961` = 500 ms, `MVB238` = 866 ms), aligned `MVB908` (333 ms), and
800x600 `CHAPTER` (12016 ms). Truncated input is rejected with a bounded error and ten repeated open/decode/close
cycles pass. During the full-suite gate, direct swscale output into a managed array exposed native heap corruption;
conversion now targets an FFmpeg-owned aligned frame and copies only the exact visible RGBA rows across the ABI.
The complete 313-test suite, three additional 312-test concurrency-sensitive repeats, zero-warning Godot build,
and threaded selftest pass. Step 3 is the next bounded slice; do not infer that combat is fixed until the paced
backend is selected live and revalidated.
**Mutable-surface fill/blend regression corrected.** The first visual recheck exposed BUNKI's menu interior
as transparent. SYSTEM4 creates 800x600 surface 3 and fills it opaque white through `0x20b`; the metadata-only

View File

@@ -23,8 +23,8 @@ also softer Windows assumptions that should be tested or replaced before claimin
| Area | Current dependency | Runtime impact | Portability status / future action |
|---|---|---|---|
| AGE movie decode (`0x236` scene movies; `0x20f` modal LOGO/OP/ED) | `godot/DirectShowMovieDecoder.cs`: DirectShow COM objects plus `ole32.dll` `CoInitializeEx` / `CoUninitialize` | Movie playback cannot run outside Windows; the current Windows filter stack also rejects the installed MPEG effects whose display width is 8 mod 16 | Introduce a decoder interface and backend factory; use a software MPEG backend for portable coverage and as the Windows fallback for unsupported effect dimensions |
| Movie integration | `godot/Main.cs` directly constructs and stores `DirectShowMovieDecoder`; `Main` is marked `SupportedOSPlatform("windows")` | The frontend has no runtime fallback or OS-specific source selection | Type `MovieRuntime` against the decoder interface, select by platform/build, and move the Windows annotation to the DirectShow backend |
| AGE movie decode (`0x236` scene movies; `0x20f` modal LOGO/OP/ED) | Live playback still uses `godot/DirectShowMovieDecoder.cs`; the isolated `native/age_movie_ffmpeg` C ABI and `FfmpegMovieSession` probe path now decode the representative VFS movies | Live movies remain Windows-only and the selected live backend still rejects installed MPEG effects whose display width is 8 mod 16 | Add timestamp pacing to the proven FFmpeg path, select it as the sole live factory, run the corpus/live gates, then delete DirectShow |
| Movie integration | `MovieRuntime` now owns `IMovieDecoder` from an injected `IMovieDecoderFactory`; `Main` deliberately selects `DirectShowMovieDecoderFactory` and remains marked `SupportedOSPlatform("windows")` | Backend ownership is portable, but the selected live implementation is not yet | Implement the FFmpeg `IMovieDecoder`, switch the factory, and remove the Windows annotation when DirectShow is deleted |
| Movie audio | DirectShow connects only the video pin to the sample grabber/null renderer | The MPEG audio stream is intentionally silent on every platform | Design a PCM/audio-clock contract or let a future backend own synchronized A/V; separate feature slice |
| ADV font discovery | `godot/Main.cs` probes `C:/Windows/Fonts` for Japanese fonts | Harmless fallback today, but appearance depends on host fonts | Bundle/configure a redistributable font or add platform-specific discovery |
| Filesystem semantics | Several filename and containment comparisons use `OrdinalIgnoreCase`; installed assets are conventionally uppercase | Needs validation on case-sensitive filesystems; may hide casing or containment mistakes | Add Linux/macOS tests with mixed-case synthetic roots and use filesystem-appropriate containment rules |
@@ -45,27 +45,57 @@ The existing connection is localized but one abstraction short of being replacea
VM op 0x236 (non-modal) / op 0x20f (modal)
-> IHost.PlayMovieToSurface / PlayModalMovieToSurface
-> VFS-owned MoviePayload bytes
-> DirectShowMovieDecoder
-> IMovieDecoderFactory
-> DirectShowMovieDecoder (current live selection)
-> FfmpegMovieSession -> age_movie C ABI (isolated gate only)
-> newest RGBA frame
-> retained movie surface
-> Godot compositor
```
Everything before and after `DirectShowMovieDecoder` is portable. The backend currently exposes the right
conceptual operations (`StopTimeMs`, `TryTakeFrame`, `IsCompleted`, and `Dispose`) but they are not formalized as an
interface. A future cleanup should:
Everything before and after the selected decoder is portable. The replacement decision is an in-process
FFmpeg backend behind a project-owned C ABI, not raw FFmpeg structs in Godot/C# and not a subprocess. FFmpeg
`n8.1.2-29-g703dcc25b9` is pinned by immutable release URL and SHA-256 in
`native/age_movie_ffmpeg/dependency-win64.json`; changing that pin requires rerunning the full installed-movie
gate. The shim dynamically links an LGPL build made without GPL or nonfree components and
uses only `libavformat`, `libavcodec`, `libavutil`, and `libswscale` for the video slice. `libswresample` and the
MPEG audio decoders may be packaged now, but PCM delivery remains a separate slice. Release artifacts must carry
the matching FFmpeg source/configuration and notices required by FFmpeg's
[license checklist](https://ffmpeg.org/legal.html).
1. Add an `IMovieDecoder` contract for synchronous stop-time metadata, frame delivery, completion, failure,
and disposal. Stop time must be available before `0x236` returns so an immediately following `0x23f`
remains deterministic.
2. Add an injected factory that accepts `MoviePayload` and selects an available backend.
3. Keep DirectShow in a Windows-specific source set or assembly, with its platform annotation local to it.
4. Implement a portable MPEG program-stream backend that produces the same top-down RGBA8 frames.
5. Preserve newest-frame-wins delivery, asynchronous decode, synchronous open failure, EOF notification,
and the retained-surface lifecycle already validated for opcode `0x236`; expose modal completion/cancel
separately for `0x20f` rather than blocking inside the decoder backend.
6. Treat synchronized movie audio as a separate extension of the contract rather than coupling it to the
compositor.
The boundary has two layers:
1. A small native `age_movie` ABI owns all FFmpeg objects and version-sensitive calls. It accepts a borrowed
byte span only for the duration of `open`, copies it into native-owned memory, creates a seekable custom
`AVIOContext`, probes the MPEG program stream, and returns immutable video metadata. Sequential decode returns
one top-down tightly packed RGBA8 frame plus its normalized presentation timestamp. Conversion occurs in an
FFmpeg-owned aligned frame; only exact visible row bytes are copied into the caller's tightly packed buffer.
Return statuses distinguish frame, EOF, invalid arguments, undersized output, and decoder failure; the open
call returns a bounded UTF-8 diagnostic and an opened handle retains its last decode error. Close accepts a
null handle, and managed `SafeHandle` ownership guarantees one close for each successfully opened handle. No
FFmpeg pointer crosses the ABI.
2. Managed `IMovieDecoder` owns the native handle and the paced worker. Synchronous construction provides
`StopTimeMs` before `0x236` returns. The worker uses a monotonic playback origin, decodes ahead by at most one
frame, publishes frames when their timestamps become due, and retains newest-frame-wins behavior if Godot is
late. EOF becomes completion only after the final frame's presentation interval/stop time, so surface cleanup
cannot erase the last frame immediately. Disposal interrupts waits and joins without depending on Godot's main
thread.
`StopTimeMs` performs a bounded packet scan over the seekable in-memory payload and takes the longest usable
FFmpeg format duration, video-stream duration, timestamp span, or constant-frame-rate packet-count duration. This
accounts for the final MPEG presentation interval that FFmpeg's stream duration can omit (`MVB908` is ten 30 fps
frames: 333 ms, not the raw stream field's 316 ms). The result is converted to integer milliseconds with the same
positive truncation used by native op `0x23f`, then the demuxer rewinds before playback. Streams with no usable
duration evidence fail initialization for the stock-content path rather than inventing timing. The existing host
safety rule still converts any backend initialization failure into an
explicitly completed zero-duration movie identity, and the presentation watchdog remains a last-resort guard for
an initialized backend that never reaches EOF.
The factory is injection for tests and future decoder replacement, not runtime codec roulette. Once the FFmpeg
corpus gate passes it becomes the only shipped movie backend on every platform and DirectShow is deleted. Modal
completion/cancel remains owned above the decoder by the existing `0x20f` host path. A future audio implementation
will consume timestamped PCM and may become the presentation clock; it must not change the VM-facing stop-time,
surface, or cancellation contracts.
This replacement is now also required for Windows gameplay parity. Archive-backed probes on 2026-07-21
show that the current DirectShow graph accepts tested MPEG widths divisible by 16 (208, 288, 304, 400, and
@@ -73,10 +103,29 @@ show that the current DirectShow graph accepts tested MPEG widths divisible by 1
The dominant combat-effect family is 280x352 (125 installed `MVB` assets), so retaining DirectShow as the
only Windows decoder is insufficient even though SC0000's 800x600 `CHAPTER.AGF` works.
An OS-specific build is also viable: include DirectShow only in Windows exports and a different decoder in
other exports. The current plain `net8.0` project has no conditional backend selection. COM declarations may
compile on another OS, but `[SupportedOSPlatform]` is analyzer metadata rather than a runtime guard, and
loading `ole32.dll` or DirectShow CLSIDs will fail there.
An archive scan on the same date classified all 213 installed movie payloads as MPEG-1 program streams with
MPEG-1 video and no MPEG-2 sequence extensions. Of those, 184 are video-only and 29 contain MPEG audio. The
large LOGO/OP/ED family uses MPEG Audio Layer II, while 25 small 120x120 `MVS` movies use Layer I. That inventory
is why the narrower `pl_mpeg` library was rejected as the primary backend: it fits every installed video stream
but only decodes Layer II audio and explicitly ignores program-stream PTS in its high-level synchronization.
FFmpeg covers the complete installed codec set and leaves the mod/profile boundary open without selecting a
different decoder per effect.
Native deliverables are RID-specific and bundled with the Godot export; the runtime must not discover an
arbitrary system FFmpeg. The first implementation gate is Windows x64 because that is the current runnable
target, but the C ABI and loader paths must reserve Windows x64, Linux x64, macOS x64, and macOS arm64 from the
start. Builds use shared libraries, `$ORIGIN`/`@loader_path`-style local lookup on Unix targets, recorded source
hashes and configure arguments, and no committed original-game data. Packaging automation is part of completing
the backend, not a prerequisite for the first native decode spike.
The Windows-x64 spike is now complete. `bootstrap-win64.ps1` verifies the immutable archive SHA before
extraction, and `build-win64.ps1` builds the shim with MSVC and places the DLL, import artifacts, required LGPL
shared libraries, and license under disposable `build/native/win-x64`. The managed resolver accepts
`AGE_FFMPEG_NATIVE_DIR` for the isolated gate and otherwise reserves application-local and
`runtimes/<rid>/native` lookup. Representative VFS results are `MVB961` 280x352/500 ms, `MVB238`
280x352/866 ms, `MVB908` 400x400/333 ms, and `CHAPTER` 800x600/12016 ms, all with changing frames and
nondecreasing timestamps. Malformed input and repeated teardown are covered. This is not yet distributable
packaging and does not change the current DirectShow live selection.
## Cross-platform validation gates

View File

@@ -72,6 +72,20 @@ All opcode knowledge (ABI, semantics, provenance, `depends_on`) is hand-edited *
|---|---|---|---|
| `locate_page.py` | Resolve a run-relative ADV page number to its canonical wait script/offset, last show-text instruction, call stack, and nearby disassembly. Pure selection/window logic is tested by `test_locate_page.py`. | `py -3.11 -X utf8 tools/locate_page.py SC0000 14 [--map <jsonl>] [--context N]` | `build/page-map-<SCENE>.jsonl` + script corpus → stdout |
## Native FFmpeg movie shim (Windows x64)
These PowerShell tools build the isolated native movie gate; they do not switch live Godot playback away from
DirectShow. The dependency manifest pins an immutable LGPL shared FFmpeg archive and SHA-256.
| Tool | Purpose | Run | Reads → Writes |
|---|---|---|---|
| `native/age_movie_ffmpeg/bootstrap-win64.ps1` | Download, hash-check, extract, and version-check the pinned FFmpeg SDK. Returns the resolved SDK root. | `.\\native\\age_movie_ffmpeg\\bootstrap-win64.ps1 [-Destination <dir>]` | `dependency-win64.json`, network/archive cache → disposable `build/downloads/`, `build/ffmpeg-sdk/` |
| `native/age_movie_ffmpeg/build-win64.ps1` | Discover the MSVC x64 toolchain, build `age_movie_ffmpeg.dll`, and stage its exact shared-library/license dependencies. | `.\\native\\age_movie_ffmpeg\\build-win64.ps1 -SdkRoot <bootstrap-output> [-OutputDirectory <dir>]` | C ABI source + FFmpeg SDK → disposable `build/native/win-x64/` by default |
The managed isolated probes load from `AGE_FFMPEG_NATIVE_DIR` when set, then application-local and
`runtimes/win-x64/native` locations. Build the shim before running the `FfmpegShim*` tests; no original-game
movie is copied into the repository or native output.
## Engine (C#) — VM core, CLI, Godot frontend
The `engine/` .NET solution (`AgeEngine.sln`) is the runtime VM; `godot/` is the ADV frontend. Not