diff --git a/docs/engine-re.md b/docs/engine-re.md index cc61649..8d8eb13 100644 --- a/docs/engine-re.md +++ b/docs/engine-re.md @@ -455,25 +455,39 @@ contract. The custom `movie_texture_renderer_receive_sample@0x4628d0` copies that preroll sample into the D3D texture and sets renderer `+0x5b4`. `movie_consume_renderer_new_frame_flag@0x405120` clears the flag and makes the outer engine tick render the retained surface. Audio is cued during the same graph transition but begins only -with the running reference clock. Native observation adds the decisive cadence constraint: OP's image advances -immediately with its audio; it does not hold the preroll image for roughly 600 ms. AGE contains no later -timestamp correction, so the DirectShow splitter/graph is necessarily presenting the video stream relative to -its first sample rather than exposing FFmpeg's shared program-stream timestamp origin. This last mapping is an -inference about DirectShow internals, but the visible native behavior is the parity oracle. +with the running reference clock. A read-only `IMediaSample::GetTime` trace on 2026-08-11 replaced the earlier +visual inference with direct evidence: native LOGO's video samples reach the inherited DirectShow base renderer +at exactly 0, 33.3667, 66.7333 ms and continue at that cadence. A follow-up hook installed while `quartz.dll` +loaded, before graph `Run`, proved that native audio also begins at graph time 0 with contiguous 26.1224 ms +samples; its first non-silent PCM block begins at 2,351 ms. Native video-content fingerprints show real changing +frames throughout 0--600 ms and match FFmpeg-decoded fingerprints at the same timestamps from 1,501 ms onward. +AGE therefore preserves the common program timeline; it does not drop the opening video interval. -The installed FFmpeg probe rules out dropped or transparent source frames. LOGO and OP first report video PTS -600/601 ms on the shim's shared mux timeline, then deliver every tested opening frame at consecutive 33/34 ms -steps; pixels change within the first 20 frames. Every decoded alpha byte in those frames is 255. MPEG-1 has no -alpha plane. Native `movie_texture_renderer_set_media_type@0x463750` accepts RGB24/RGB32, and the RGB24 copy +The earlier installed FFmpeg probe proved that its returned frames were consecutive and opaque, but not that +decode began at the first packet: its explicit seek had already skipped to PTS 600/601 ms. After the corrected +common-origin rewind, LOGO and OP begin at PTS 0; their first 20 frames remain consecutive 33/34 ms samples and +change within that window. Every decoded alpha byte is 255. MPEG-1 has no alpha plane. Native +`movie_texture_renderer_set_media_type@0x463750` accepts RGB24/RGB32, and the RGB24 copy path promotes pixels to alpha `0xff` before comparing the complete packed ARGB value with its renderer key. Opcode `0x20f` attaches that renderer to the existing D3D target and never reloads the retained surface or assigns it an RGB color key. The former port did both `Gfx.SetSurface(..., colorKey: 0)` and RGB-only keying, which incorrectly made exact-black movie texels transparent and explained grey-background MPEG speckles. -The corrected port publishes frame zero before starting audio, then schedules all video frames from -`source_video_pts - first_video_pts`; audio retains its own complete stream from timestamp zero. No video or -audio samples are discarded. Signal-bearing PCM in OP's first 600 ms proves only that the audio data is real, -not that the MPEG mux's cross-stream start offset is an intended 600 ms visible lead. +The port's error was below managed pacing. `rewind_for_decode` sought the video demuxer to +`AVStream.start_time` (600/601 ms for LOGO/OP), skipping valid earlier packets/decoded frames. Managed pacing then +treated the first surviving frame as presentation time zero while audio correctly retained the common origin. +For LOGO this decoded only 200 frames and moved picture about 600 ms ahead of sound. The native shim now rewinds +video to the format's common `timeline_origin_ms`; LOGO produces 218 consecutive frames beginning at PTS 0. The +asset-level cue confirms the correction: audio signal begins around 2,380 ms and the first `E` appears at video +PTS 2,502 ms, preserving the authored roughly 122 ms sound lead instead of the old roughly 478 ms lag. + +**Audio-bearing graph completion (2026-08-11):** DirectShow remains active through the common graph endpoint. +The port had completed once final PCM was *submitted* to Godot, not when it became audible, and LOGO could still +have roughly 375 ms queued. Audio-bearing playback holds the terminal image until the sound-hardware clock reaches +the full source endpoint; audio decode EOF and PCM submission remain additional requirements. With the recovered +opening video frames, LOGO's video and audio now naturally share the approximately 7,288 ms graph lifetime. A live +Windows F6 series showed zero generator underruns and a stable output-latency-compensated clock, ruling out decode +starvation and startup buffer loss. An existing native operand trace identifies every observed heap codebase by a 100% match against its static instruction-offset set. The captured New Game route is: diff --git a/docs/opcode-reference.md b/docs/opcode-reference.md index 657db5f..e7d765e 100644 --- a/docs/opcode-reference.md +++ b/docs/opcode-reference.md @@ -927,9 +927,9 @@ This is a target-pixel operation, not retained-object teardown. It invokes IDire ### 0x20f `play-modal-movie-to-surface` (play-modal-movie-to-surface, argc 3) - **summary:** (packed_resource_id)(surface_slot)(movie_flags) - open a universal packed SYS4INI/AAI MPEG asset into an existing retained surface, start its native movie graph, and arm modal run-state bit 0x2000 so script execution remains parked until the movie completes or is skipped. This is the LOGO/OP/ED whole-movie path; op 0x236 uses the same resolver with non-modal lifecycle. - **grounding:** source=investigation, confidence=high -- **evidence:** Ghidra /v2: op_0x20f_play_modal_movie_to_surface@0x422e50 requires an existing retained D3D target, shares the movie-object allocation, DirectShow graph open, audio-route, and volume setup used by 0x236, then calls movie_start_modal_playback@0x463280, ORs EngineCtx+0xa0ce4 with 0x2000, and marks movie presentation dirty. It does not reload/rebind the gfx surface record or assign an RGB color key. The start worker calls IMediaControl::Run at vtable +0x1c on the freshly stopped graph; AGE performs no explicit Pause, seek, or opening-frame decode. DirectShow's stopped-to-paused-to-running transition cues the first sample. movie_texture_renderer_receive_sample@0x4628d0 copies it to the D3D texture and sets renderer +0x5b4; movie_consume_renderer_new_frame_flag@0x405120 makes the outer tick render it. Native OP then advances video immediately with audio, so FFmpeg's roughly 600 ms shared-mux video origin is not an AGE-visible hold. Corpus has exactly three sites: LOGO (0x335f,42,4), OP (0x3364,42,4), and ED (0x3324,42,dynamic flags). +- **evidence:** Ghidra /v2: op_0x20f_play_modal_movie_to_surface@0x422e50 requires an existing retained D3D target, shares the movie-object allocation, DirectShow graph open, audio-route, and volume setup used by 0x236, then calls movie_start_modal_playback@0x463280, ORs EngineCtx+0xa0ce4 with 0x2000, and marks movie presentation dirty. It does not reload/rebind the gfx surface record or assign an RGB color key. The start worker calls IMediaControl::Run at vtable +0x1c on the freshly stopped graph; AGE performs no explicit Pause, seek, or opening-frame decode. DirectShow's stopped-to-paused-to-running transition cues the first sample. movie_texture_renderer_receive_sample@0x4628d0 copies it to the D3D texture and sets renderer +0x5b4; movie_consume_renderer_new_frame_flag@0x405120 makes the outer tick render it. Read-only 2026-08-11 hooks installed before graph Run proved native LOGO video starts at 0 and advances by 33.3667 ms while audio starts at 0 in contiguous 26.1224 ms blocks; native video fingerprints expose changing 0--600 ms frames and match FFmpeg at equal later timestamps. The graph stays alive through its common stop endpoint. Corpus has exactly three sites: LOGO (0x335f,42,4), OP (0x3364,42,4), and ED (0x3324,42,dynamic flags). -Implemented through IHost.PlayModalMovieToSurface. Its operand uses the same native universal packed-id catalog contract as 0x236; the separate host call exists for modal wait/cancel lifecycle, not a different resolver. ResourceMap.ResolveMovie selects through ResolvePacked and retains MPEG signature validation in ReadMovie. Godot reuses the asynchronous FFmpeg decoder/retained-surface compositor, parks only the VM thread until EOF, and treats mouse click or Accept/Cancel input as completion before wrapper cleanup releases both video and per-playback audio output. Movie playback leaves the existing mutable target at resourceId 0 with no RGB color key; MovieSurfaceRegistry supplies its live pixels. Audio-bearing movies deliver timestamped stereo float PCM through AudioStreamGenerator and use the sound-hardware position as the master clock. The first decoded image is latched before audio starts, and all video deadlines are rebased to the first video PTS while the complete audio stream begins at its own timestamp zero. No opening samples are discarded. +Implemented through IHost.PlayModalMovieToSurface. Its operand uses the same native universal packed-id catalog contract as 0x236; the separate host call exists for modal wait/cancel lifecycle, not a different resolver. ResourceMap.ResolveMovie selects through ResolvePacked and retains MPEG signature validation in ReadMovie. Godot reuses the asynchronous FFmpeg decoder/retained-surface compositor, parks only the VM thread until EOF, and treats mouse click or Accept/Cancel input as completion before wrapper cleanup releases both video and per-playback audio output. Movie playback leaves the existing mutable target at resourceId 0 with no RGB color key; MovieSurfaceRegistry supplies its live pixels. Audio-bearing movies deliver timestamped stereo float PCM through AudioStreamGenerator and use the sound-hardware position as the master clock. The native shim rewinds both demuxers to the common media origin; this preserves LOGO's 218-frame PTS-0 opening rather than skipping to the video stream's declared 600 ms start and moving picture ahead of audio. The first decoded image is latched before audio starts, later frames retain their common-timeline cadence, and the terminal image remains through the full audio graph endpoint. ### 0x215 `query-gfx-object?` (query-gfx-object?, argc 2) - **summary:** 0x215 (out_slot)(handle) — query the retained gfx-object map. Handler gfx_op_0x215_query_source_slot@0x42a0b0 calls gfx_object_query_source_slot@0x47f280 with owner ctx+0x46614. The worker searches owner+0x408, the same map populated by geometry/draw workers, and returns obj+4: the live source-surface slot written by draw-texture, or -1 if absent. gfx_object_init_default zeroes obj+4, so a geometry/animation-created but draw-unbound object returns slot 0, not -1. SC0000 uses a successful result for query-guarded teardown: op 0x1f7 erases the object group and op 0x1fa releases this slot. Matching the zero default prevents stale transforms from surviving into the EV050EA CG at SC0000 page 58. @@ -1064,7 +1064,7 @@ Port status (2026-07-28): implemented as an independent retained-object scale cy - **depended on by:** 0x241, 0x24d - **evidence:** Ghidra /v2 op_0x236_play_movie_to_surface@0x423ee0 fetches operand 1 and passes it unchanged to movie_to_texture_open_asset_graph@0x463e20, which passes it unchanged to asset_open_indexed_entry@0x44f390. The opener directly indexes the flat base table or selected AAI table and has no scene input. The handler leaves playback pending through movie_play_configure@0x4625e0; movie_start_pending_after_sync_delay@0x4633b0 records the first service tick and calls the shared IMediaControl::Run worker after operand 4's delay, giving non-modal playback the same DirectShow first-sample preroll as 0x20f. SC0000 native operand capture and exact 0x13c8->0x13d1 trace prove nonblocking behavior. BTL's live 0x2b21 site supplies 0x2af1/0x2af5/0x2bca/0x2bd8/0x2bde, the exact base entries MVB001/MVB004/MVB914/MVB958/MVB955. -The handler requires an existing destination texture, allocates/reuses a 0x478-byte movie-to-texture object for the surface without replacing its resource identity/color key, opens operand 1 through the native universal packed-id reader, builds FilterGraph/IGraphBuilder/IMediaControl/IMediaPosition/IMediaEvent/IBasicAudio, and presents bottom-up RGB samples through the movie texture renderer. Operand 3 selects movie/sound routing policy: bits 0x10000/0x20000/0x40000/0x80000 force mute/music/SE/voice routes, otherwise set:DependMovieSound selects the normal movie route; SC0000's low value 2 is retained as native movie mode state. Godot maps those categories to audio buses and sends FFmpeg-decoded timestamped stereo float PCM through a per-playback AudioStreamGenerator. The first image is latched before audio start; video cadence is rebased to its first decoded PTS for both audio-bearing and video-only streams, while audio retains every sample from its own timestamp zero. Operand 4 delays the pending graph start in native service ticks. Static layer preparation after 0x236 does not terminate the retained movie; 0x21c services it through EOF and subsequent surface cleanup stops/detaches both decoder and audio output. The port type-checks the selected ResolvePacked record as MPEG without adding a scene base or fallback. +The handler requires an existing destination texture, allocates/reuses a 0x478-byte movie-to-texture object for the surface without replacing its resource identity/color key, opens operand 1 through the native universal packed-id reader, builds FilterGraph/IGraphBuilder/IMediaControl/IMediaPosition/IMediaEvent/IBasicAudio, and presents bottom-up RGB samples through the movie texture renderer. Operand 3 selects movie/sound routing policy: bits 0x10000/0x20000/0x40000/0x80000 force mute/music/SE/voice routes, otherwise set:DependMovieSound selects the normal movie route; SC0000's low value 2 is retained as native movie mode state. Godot maps those categories to audio buses and sends FFmpeg-decoded timestamped stereo float PCM through a per-playback AudioStreamGenerator. Both native demuxers begin at the common timeline origin; the first image is latched before audio start and later video/audio retain their source timing. This recovers valid MPEG video preroll before AVStream.start_time instead of advancing picture by roughly 600 ms. Audio-bearing completion retains the terminal frame through the full graph endpoint. Operand 4 delays the pending graph start in native service ticks. Static layer preparation after 0x236 does not terminate the retained movie; 0x21c services it through EOF and subsequent surface cleanup stops/detaches both decoder and audio output. The port type-checks the selected ResolvePacked record as MPEG without adding a scene base or fallback. ### 0x238 `set-anim-clock` (set-anim-clock, argc 1) - **summary:** (duration) — configure the separate global finite-animation service window: native ctx+0x51b78=0 (start), +0x51b7c=duration. The generic instruction length is 3 dwords. NON-BLOCKING: gfx_animation_service_poll seeds the start from the shared frame timestamp and keeps redraw active through the duration. This is not the time source or cadence for cyclic object channels, which sample ctx+0x51b64/+0x51b68 with object-local starts and periods. SC0000 opening @0x123bd/@0x13858. Handler 0x4240e0; Kelebek VA 0x422390 is drift. diff --git a/docs/phase-b-framework.md b/docs/phase-b-framework.md index 95bf006..62e8c4c 100644 --- a/docs/phase-b-framework.md +++ b/docs/phase-b-framework.md @@ -759,7 +759,8 @@ demuxers, preserving concurrent decode without temporary files, and exposes time float PCM resampled at the source rate. Video and audio timestamps share the same normalized media origin. The managed decoder owns bounded video and PCM queues plus separate workers; audio-bearing movies use the Godot audio-device clock as the presentation master, while video-only movies retain monotonic stopwatch pacing. -Completion waits for the final video interval and for decoded PCM to be submitted. +Completion waits for the final video interval and for decoded PCM to be submitted through the shared zero-based +presentation endpoint. ABI v3 adds a synchronous dual-demuxer initial seek used by opcode `0x241`. Native seek lands on preceding indexed packets; managed preroll selects the video frame active at the requested position, sample-trims audio @@ -793,6 +794,16 @@ Linux/macOS FFmpeg builds and packaging smoke gates when work returns to cross-p Post-cleanup validation passes all 405 engine tests, the warning-free Godot build, threaded selftest, opcode lint/tooling, and the 213/213 installed video/audio corpus gate. +The shared Windows/Linux residual A/V report was investigated on 2026-08-11. A read-only native LOGO trace +proved DirectShow supplies AGE zero-based 33.3667 ms video samples and contiguous 26.1224 ms audio samples; an +early hook measured the first non-silent native PCM block at 2,351 ms. Sparse native video fingerprints match +FFmpeg frames at equal timestamps and expose changing native frames during 0--600 ms. The shim had skipped those +frames by rewinding to the video stream's declared 600 ms `start_time`, then managed pacing displayed frame 600 at +time zero while audio retained the common origin. Rewinding video to the format timeline origin recovers 218 LOGO +frames beginning at PTS 0 instead of 200 beginning at 600. Its audio onset around 2,380 ms again precedes the +first `E` at PTS 2,502 ms. Completion separately retains the terminal image until queued audio drains through the +full graph endpoint. Focused and real-asset regressions pin both corrections; manual LOGO/OP confirmation remains. + **CORPUS-WIDE OPCODE HARDENING PASS STARTED (2026-07-28).** With numbered-save implementation complete and further live save testing explicitly deferred, the active breadth pass now ranks every Himegari-observed VM fallthrough across all 481 decoded scripts. The starting diff --git a/docs/platform-portability.md b/docs/platform-portability.md index 07ec0ad..ae2d7fe 100644 --- a/docs/platform-portability.md +++ b/docs/platform-portability.md @@ -33,8 +33,9 @@ guarded exporter produce a self-contained artifact whose headless self-test pass portable TextServer backend and installed Noto CJK fonts. A subsequent WSLg interactive run created the Linux window, accepted title/menu input, rendered both opening movies with sound, and progressed naturally into SC0000. This establishes the packaged loader, archive, font, window/input, movie audio/presentation, managed host, and -teardown paths. Slight A/V desynchronization also occurs on Windows and remains a shared runtime follow-up rather -than a Linux-only portability blocker. A complete combined game/export installation copied to WSL ext4 passed +teardown paths. The shared Windows/Linux A/V follow-up identified skipped opening video preroll in the FFmpeg shim +rather than a Linux-only clock defect; the fix now awaits a manual LOGO/OP recheck. A complete combined game/export +installation copied to WSL ext4 passed hash verification plus headless and windowed executable-directory runs; mixed-case fixtures and fresh interactive profile/save behavior remain to validate. The accepted DirectShow fallback was deleted after the corpus and audible LOGO/OP/CHAPTER gates passed. There are also softer Windows assumptions that should be tested or replaced before @@ -186,7 +187,7 @@ entry. The system-font-only policy remains supported until the bundled-font opti | Retained graphics presentation | Backend-neutral `GfxState`; accepted default Godot `Sprite2D` GPU stage plus the retained software pixel oracle, using runtime `ImageTexture`, canvas transforms/materials, and no native graphics API | GPU backend caches static/color-key variants, updates dynamic surfaces, handles retained range transitions, and falls back whole-frame for the legacy host screen-transition path | Godot owns D3D/Vulkan/Metal/OpenGL selection; validate shader/blend/filter behavior per target rather than adding a platform renderer | | AGE movie decode (`0x236` scene movies; `0x20f` modal LOGO/OP/ED; `0x24d` movie masks) | `FfmpegMovieDecoder` is the sole factory over the project-owned `native/age_movie_ffmpeg` ABI | Windows-x64 and Linux-x64 both pass the complete 213-payload installed video/audio corpus gate. The Linux library bundle is present in a validated Godot export; Windows also passes audible LOGO/OP/CHAPTER playback and real TEST.AGF green-mask decode | Add macOS native builds, distributable source-offer packaging, and real Linux/macOS window/audio movie gates | | Movie integration | Each surface owns a unique playback-instance id; `MovieRuntime` owns `IMovieDecoder` from an injected factory; video-only streams use monotonic pacing while audio-bearing streams use the Godot output clock. Opcode `0x24d` redirects decoded green bytes through the platform-neutral retained rasterizer and exact managed packed-alpha mask helper | Concurrent/restarted uses of one asset have independent frame/audio/completion/teardown state. Ordinary and mask movies share VFS/FFmpeg ownership; the mask result is a backend-neutral dynamic RGBA surface consumed by either Godot renderer. Managed code is no longer Windows-annotated, and target-aware builds stage either the win-x64 or Linux-x64 native bundle | Add macOS native builds and Linux/macOS in-engine smoke gates | -| Movie audio | ABI v2 returns timestamped stereo float PCM; bounded managed buffering feeds a per-playback Godot `AudioStreamGenerator` and routes native movie flags to engine buses | All 29 installed audio-bearing streams decode with signal; synchronized LOGO/OP/CHAPTER playback is audibly accepted | Treat absent, distorted, or unsynchronized audio from an audio-bearing movie as a runtime bug | +| Movie audio | ABI v3 returns timestamped stereo float PCM; bounded managed buffering feeds a per-playback Godot `AudioStreamGenerator` and routes native movie flags to engine buses | All 29 installed audio-bearing streams decode with signal. Video follows the output-latency-compensated hardware clock; video and audio both retain the common zero-based media timeline, and completion holds the terminal image through the audibly drained graph endpoint | Manually recheck the 2026-08-11 LOGO/OP synchronization fix on Windows and Linux; treat any remaining absent, distorted, or unsynchronized audio as a runtime bug | | ADV font discovery/raster fidelity | `Age.Engine.Text` remains OS-neutral. On compatible ACP-932 Windows, `auto` selects the separate exact-to-request `Age.Engine.Text.Windows` GDI gray-4 backend. Otherwise Godot TextServer supplies Unicode masks and metrics from ordered system-font substitutions in `godot/config/himegari-text-rendering.json`; this policy is explicitly not GDI pixel-exact. Immediate `0x204`/`0x205`, live ADV, retained History, and the wait atlas all enter ordinary RGBA surfaces/retained handles | Three Mincho/Gothic regular/bold samples byte-match an independent Unicode GDI oracle. A live `SC0000@0x869` trace proves that the `AGE Patch.exe`/`jprun.dll` wrapper overrides authored Mincho to MS Gothic: 13 captured masks match Gothic byte-for-byte and Mincho 0/13. This is wrapper behavior, not the true-native default, and reproducing it is outside the compatibility target. Forced exact and forced portable self-tests cover immediate, live, and History rasterization, bounded caching, complete-line materialization, retained ordering, alpha/tint/fade, transforms, clipping, capture, HIDEWIN suspension, partial erase/republication, reset, and teardown. Gameplay `Label` pools and overlay projection are gone | Select and smoke-test a redistributable Linux font-bundling policy or retain the explicit system-font prerequisite; add macOS runtime coverage and keep the GDI project out of non-Windows deliverables | | Filesystem semantics | Several filename and containment comparisons use `OrdinalIgnoreCase`; installed assets are conventionally uppercase | Needs validation on case-sensitive filesystems; may hide casing or containment mistakes | Add Linux/macOS tests with mixed-case synthetic roots and use filesystem-appropriate containment rules | | Save/profile/settings storage | `Sys4PersistencePaths` models AGE's independent `SAVEPATH` and `REGFILEPATH` resolutions. Godot replaces Himegari's related profile directory with `user://`, yielding `user://SAVE` for native S3SD/S4SD/S3RT files and thumbnails plus `user://SYS4REG.INI` for the BOM-less CP932 options file; the preserving writer changes only its nine `[sound]` keys | Save payloads and engine options are isolated together without changing either native format. Native/drop-in resolution remains available through `USEAPPDATAFOLDER` plus both SYS4INI paths. A single-root override is rejected when `SAVEPATH` is not beneath `REGFILEPATH`, preventing cross-profile guesses | Expose explicit profile/native selection through the future launcher and allow independent overrides for profiles whose two native paths are unrelated. Validate CP932 availability, replace/flush, case, permissions, and interrupted-write behavior on each export target | @@ -245,8 +246,9 @@ The boundary has two layers: Video-only movies use the monotonic playback origin. Audio-bearing movies feed a per-instance `AudioStreamGenerator`; the monotonic-clamped Godot sound-hardware position (`playback + time-since-mix - cached output latency`) advances due video frames. Completion requires video - stop time, audio decode EOF, and PCM submission. Disposal interrupts waits, joins both workers, and stops the - generator without depending on native default-device playback. + endpoint, audio decode EOF, PCM submission, and audio-clock arrival at the common presentation endpoint. Disposal + interrupts waits, joins both workers, and stops the generator without depending on native default-device + playback. `StopTimeMs` performs a bounded packet scan over the seekable in-memory payload and takes the longest usable FFmpeg format duration, video-stream duration, timestamp span, or constant-frame-rate packet-count duration. This @@ -290,19 +292,21 @@ the first timestamp and material later gaps/overlaps. A 4,093-block OP regressio The clean audible recheck passed for LOGO/OP and CHAPTER. DirectShow and the managed Windows platform annotations were then deleted; missing, distorted, or unsynchronized audio on any of the 29 audio-bearing assets is now a bug. -Movie startup was corrected on 2026-07-28 after a first-frame latch exposed a second, visible 600 ms freeze. -`LOGO.AGF` and `OP.AGF` begin audio at 0 ms on the shim's shared mux timeline, while their first decoded video -frames carry PTS 600 ms and 601 ms. Signal-bearing PCM in OP's first 600 ms initially looked like an intentional -audio lead, but native OP advances video and audio immediately. A real-asset regression proves the decoder is not -losing opening data: the first 20 LOGO/OP video frames are consecutive 33/34 ms samples, change within that window, -and contain alpha 255 at every pixel. The mux offset is therefore not a presentation deadline. +Movie startup was first corrected on 2026-07-28 after a first-frame latch exposed a visible 600 ms freeze. That +pass correctly stopped pacing the first surviving frame against its raw PTS, but it misclassified the 600/601 ms +LOGO/OP video start as a harmless mux offset. The shim's initial seek had already skipped the actual PTS-0 opening +frames. The opacity portion of the old real-asset regression remains valid—returned frames are consecutive, +changing, and alpha 255—but completeness now has its own PTS-0 assertion and native-content fingerprint evidence. The paced decoder still latches frame zero and Godot still waits to start its `AudioStreamGenerator` until that -image has been published. It now schedules every later video frame from -`source_video_pts - first_video_pts`, including audio-bearing movies, while submitting the complete audio stream -from its own timestamp zero. This preserves every decoded sample and immediately advances frame cadence instead -of freezing frame zero until the audio clock reaches the program-stream offset. Startup logs and F6 snapshots -retain the raw first video PTS for diagnosis. +image has been published. The defect was the native shim's initial video rewind: it sought to the video stream's +declared 600/601 ms `start_time`, skipping valid opening packets while audio began at the common zero origin. +LOGO consequently decoded 200 frames from PTS 600 and moved picture roughly 600 ms ahead. Rewinding video to the +format's common media origin recovers 218 consecutive LOGO frames beginning at PTS 0. Native DirectShow traces +show both output pins start at zero, native video fingerprints match FFmpeg at equal timestamps, and native PCM +first becomes non-silent around 2,351 ms. The corrected asset relation places FFmpeg audio onset around 2,380 ms +before the first `E` at video PTS 2,502 ms, rather than 478 ms after the prematurely displayed cue. Completion +retains the full common endpoint. Startup logs and F6 snapshots retain the raw first video PTS for diagnosis. A separate compositor error amplified the symptom. MPEG-1 has no transparency and FFmpeg's RGBA conversion produces opaque pixels, but the VM had replaced the existing mutable movie target with an invented RGB color key @@ -356,8 +360,9 @@ against an explicit installed-game path using the portable text backend, with a leaked-object warning. Minimal Ubuntu required `fonts-noto-cjk`; before installation the exact same artifact failed explicitly on Japanese glyph U+59EB. After WSLg was enabled, the same artifact also opened windowed, accepted normal title/menu input, displayed both opening movies, and reached SC0000 through the natural startup -path. Both movies produced audible output; the slight A/V desynchronization also reproduces on Windows and is -tracked as a shared runtime follow-up rather than a Linux-only defect. This interactive acceptance establishes +path. Both movies produced audible output; the slight A/V desynchronization also reproduced on Windows and was +traced to skipped opening video preroll corrected on 2026-08-11. Manual confirmation of that correction +remains a cross-platform follow-up. This interactive acceptance establishes WSLg window/input/movie/audio presentation, but not native-desktop compositor/driver behavior or case-sensitive install/profile behavior. diff --git a/docs/remake-architecture-and-roadmap.md b/docs/remake-architecture-and-roadmap.md index 8072e05..740c868 100644 --- a/docs/remake-architecture-and-roadmap.md +++ b/docs/remake-architecture-and-roadmap.md @@ -1459,5 +1459,6 @@ opcode gap remains in the shipped 481-script corpus. Use the next reproducible n to choose the concrete gameplay slice instead of returning to opcode-coverage work. The open portability gates remain separate follow-ups: deliberately mixed-case synthetic roots, fresh-profile -save/settings writes, native Linux desktop coverage, shared movie A/V synchronization, and a deliberate Linux +save/settings writes, native Linux desktop coverage, a manual Windows/Linux recheck of the 2026-08-11 movie +A/V opening-preroll fix, and a deliberate Linux font-bundling policy. Do not change remote or release policy without the user's explicit direction. diff --git a/engine/Age.Engine.Frontend/FfmpegMovieDecoder.cs b/engine/Age.Engine.Frontend/FfmpegMovieDecoder.cs index c41b324..ff671e6 100644 --- a/engine/Age.Engine.Frontend/FfmpegMovieDecoder.cs +++ b/engine/Age.Engine.Frontend/FfmpegMovieDecoder.cs @@ -260,11 +260,10 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder { if (decodedFrames == 0) throw new InvalidDataException("FFmpeg stream ended before producing a video frame"); - long completionTime = _presentationDurationMs - ?? PresentationDeadline( - Math.Max(_source.Info.StopTimeMs, - lastTimestamp + FrameIntervalMilliseconds(_source.Info)), - firstTimestamp); + long sourceEndpoint = Math.Max( + _source.Info.StopTimeMs, + lastTimestamp + FrameIntervalMilliseconds(_source.Info)); + long completionTime = CompletionDeadline(sourceEndpoint, firstTimestamp); if (_clock.WaitUntil(completionTime, _cancel)) { _videoTimelineCompleted = true; @@ -446,6 +445,19 @@ internal sealed class FfmpegMovieDecoder : IMovieDecoder return checked(sourceElapsed * duration / _source.Info.StopTimeMs); } + private long CompletionDeadline(long sourceEndpoint, long firstVideoTimestamp) + { + if (_presentationDurationMs is { } duration) return duration; + if (AudioInfo != null) + { + // The audio pin begins at the common mux origin even when the video stream declares + // a later start. Keep the terminal image while the audio-master clock drains through + // the full graph endpoint rather than completing at the last video sample. + return Math.Max(0, sourceEndpoint - _initialPositionMs); + } + return PresentationDeadline(sourceEndpoint, firstVideoTimestamp); + } + public void Dispose() { if (Interlocked.Exchange(ref _disposed, 1) != 0) return; diff --git a/engine/Age.Engine.Tests/MovieOpcodeTests.cs b/engine/Age.Engine.Tests/MovieOpcodeTests.cs index 381f806..2106d13 100644 --- a/engine/Age.Engine.Tests/MovieOpcodeTests.cs +++ b/engine/Age.Engine.Tests/MovieOpcodeTests.cs @@ -449,6 +449,28 @@ public class MovieOpcodeTests Assert.Null(decoder.Failure); } + [Fact] + public void FfmpegAudioBearingDecoderHoldsTerminalFrameUntilGraphAudioEndpoint() + { + var source = new FakeFfmpegFrameSource( + new FfmpegMovieInfo(1, 1, 700, 30, 1, true, 1000, 2, 700), + SyntheticMovieFrame(1, 600), SyntheticMovieFrame(2, 634)); + source.EnqueueAudio(new FfmpegAudioChunk(new float[1400], 700, 0)); + using var clock = new ManualMoviePacingClock(); + using var decoder = new FfmpegMovieDecoder(source, clock); + + Assert.True(SpinWait.SpinUntil(() => decoder.TryTakeFrame(out _), 1000)); + Assert.True(SpinWait.SpinUntil(() => decoder.TryTakeAudioChunk(out _), 1000)); + Assert.True(SpinWait.SpinUntil(() => decoder.AudioDecodingCompleted, 1000)); + decoder.MarkAudioSubmitted(); + + clock.AdvanceTo(699); + Assert.True(clock.WaitForDeadline(700)); + Assert.False(decoder.IsCompleted); + clock.AdvanceTo(700); + Assert.True(SpinWait.SpinUntil(() => decoder.IsCompleted, 1000)); + } + [Fact] public void MovieAudioTimelineDoesNotSpliceContinuousMpegBlocksAtRoundedMillisecondTimestamps() { @@ -801,11 +823,15 @@ public class MovieOpcodeTests Assert.Equal(600, movie.Info.Height); Assert.Equal(1000, movie.Info.StopTimeMs); Assert.False(movie.Info.HasAudio); - Assert.True(movie.TryDecodeNextVideoFrame(out var frame)); - - byte[] mask = MovieMaskSurface.ExtractGreen(frame.Image, 800, 600, 255); - Assert.Equal(800 * 600, mask.Length); - Assert.True(mask.Distinct().Skip(1).Any(), "TEST.AGF should publish a nonuniform green mask"); + byte[]? mask = null; + for (int frameIndex = 0; frameIndex < 60 && movie.TryDecodeNextVideoFrame(out var frame); frameIndex++) + { + mask = MovieMaskSurface.ExtractGreen(frame.Image, 800, 600, 255); + Assert.Equal(800 * 600, mask.Length); + if (mask.Distinct().Skip(1).Any()) break; + } + Assert.NotNull(mask); + Assert.True(mask!.Distinct().Skip(1).Any(), "TEST.AGF should publish a nonuniform green mask"); } [Theory] @@ -832,6 +858,8 @@ public class MovieOpcodeTests $"{expectedName} ended before opening frame {frameIndex}"); if (priorTimestamp >= 0) Assert.InRange(frame.PresentationTimeMs - priorTimestamp, 33, 34); + else + Assert.Equal(0, frame.PresentationTimeMs); priorTimestamp = frame.PresentationTimeMs; firstPixels ??= frame.Image.Pixels; openingChanged |= !firstPixels.AsSpan().SequenceEqual(frame.Image.Pixels); @@ -868,7 +896,7 @@ public class MovieOpcodeTests Assert.True(first.PresentationTimeMs >= 0); bool changed = false; long priorTimestamp = first.PresentationTimeMs; - for (int frameIndex = 0; frameIndex < 30 && movie.TryDecodeNextVideoFrame(out var later); frameIndex++) + for (int frameIndex = 0; frameIndex < 120 && movie.TryDecodeNextVideoFrame(out var later); frameIndex++) { Assert.True(later.PresentationTimeMs >= priorTimestamp); priorTimestamp = later.PresentationTimeMs; diff --git a/native/age_movie_ffmpeg/age_movie.c b/native/age_movie_ffmpeg/age_movie.c index 10a482e..daa77c7 100644 --- a/native/age_movie_ffmpeg/age_movie.c +++ b/native/age_movie_ffmpeg/age_movie.c @@ -257,7 +257,12 @@ static int determine_duration(age_movie *movie, AVStream *stream, int64_t *out_d } static int rewind_for_decode(age_movie *movie, AVStream *stream) { - int64_t target = stream->start_time != AV_NOPTS_VALUE ? stream->start_time : 0; + /* MPEG video can contain decodable preroll before AVStream.start_time. DirectShow + presents those opening frames from graph time zero; seeking to the video pin's + later declared start skips them and advances picture relative to audio. */ + int64_t target = av_rescale_q_rnd(movie->timeline_origin_ms, + (AVRational){1, 1000}, stream->time_base, + AV_ROUND_DOWN | AV_ROUND_PASS_MINMAX); int result = av_seek_frame(movie->video_format, movie->video_stream_index, target, AVSEEK_FLAG_BACKWARD); if (result < 0) { movie->video_io_state.position = 0; @@ -422,10 +427,7 @@ int32_t AGE_MOVIE_CALL age_movie_open(const uint8_t *bytes, size_t length, movie->video_stream_index = result; AVStream *stream = movie->video_format->streams[movie->video_stream_index]; movie->video_time_base = stream->time_base; - movie->video_fallback_origin_ms = - stream->start_time == AV_NOPTS_VALUE ? 0 - : rescale_ms_down(stream->start_time, movie->video_time_base) - - movie->timeline_origin_ms; + movie->video_fallback_origin_ms = 0; movie->video_codec = avcodec_alloc_context3(video_decoder); if (movie->video_codec == NULL) { diff --git a/vm-map/opcodes.toml b/vm-map/opcodes.toml index 39be22c..8b4860c 100644 --- a/vm-map/opcodes.toml +++ b/vm-map/opcodes.toml @@ -5303,8 +5303,8 @@ noop_headless = false source = "investigation" confidence = "high" depends_on = [] -evidence = "Ghidra /v2: op_0x20f_play_modal_movie_to_surface@0x422e50 requires an existing retained D3D target, shares the movie-object allocation, DirectShow graph open, audio-route, and volume setup used by 0x236, then calls movie_start_modal_playback@0x463280, ORs EngineCtx+0xa0ce4 with 0x2000, and marks movie presentation dirty. It does not reload/rebind the gfx surface record or assign an RGB color key. The start worker calls IMediaControl::Run at vtable +0x1c on the freshly stopped graph; AGE performs no explicit Pause, seek, or opening-frame decode. DirectShow's stopped-to-paused-to-running transition cues the first sample. movie_texture_renderer_receive_sample@0x4628d0 copies it to the D3D texture and sets renderer +0x5b4; movie_consume_renderer_new_frame_flag@0x405120 makes the outer tick render it. Native OP then advances video immediately with audio, so FFmpeg's roughly 600 ms shared-mux video origin is not an AGE-visible hold. Corpus has exactly three sites: LOGO (0x335f,42,4), OP (0x3364,42,4), and ED (0x3324,42,dynamic flags)." -details = "Implemented through IHost.PlayModalMovieToSurface. Its operand uses the same native universal packed-id catalog contract as 0x236; the separate host call exists for modal wait/cancel lifecycle, not a different resolver. ResourceMap.ResolveMovie selects through ResolvePacked and retains MPEG signature validation in ReadMovie. Godot reuses the asynchronous FFmpeg decoder/retained-surface compositor, parks only the VM thread until EOF, and treats mouse click or Accept/Cancel input as completion before wrapper cleanup releases both video and per-playback audio output. Movie playback leaves the existing mutable target at resourceId 0 with no RGB color key; MovieSurfaceRegistry supplies its live pixels. Audio-bearing movies deliver timestamped stereo float PCM through AudioStreamGenerator and use the sound-hardware position as the master clock. The first decoded image is latched before audio starts, and all video deadlines are rebased to the first video PTS while the complete audio stream begins at its own timestamp zero. No opening samples are discarded." +evidence = "Ghidra /v2: op_0x20f_play_modal_movie_to_surface@0x422e50 requires an existing retained D3D target, shares the movie-object allocation, DirectShow graph open, audio-route, and volume setup used by 0x236, then calls movie_start_modal_playback@0x463280, ORs EngineCtx+0xa0ce4 with 0x2000, and marks movie presentation dirty. It does not reload/rebind the gfx surface record or assign an RGB color key. The start worker calls IMediaControl::Run at vtable +0x1c on the freshly stopped graph; AGE performs no explicit Pause, seek, or opening-frame decode. DirectShow's stopped-to-paused-to-running transition cues the first sample. movie_texture_renderer_receive_sample@0x4628d0 copies it to the D3D texture and sets renderer +0x5b4; movie_consume_renderer_new_frame_flag@0x405120 makes the outer tick render it. Read-only 2026-08-11 hooks installed before graph Run proved native LOGO video starts at 0 and advances by 33.3667 ms while audio starts at 0 in contiguous 26.1224 ms blocks; native video fingerprints expose changing 0--600 ms frames and match FFmpeg at equal later timestamps. The graph stays alive through its common stop endpoint. Corpus has exactly three sites: LOGO (0x335f,42,4), OP (0x3364,42,4), and ED (0x3324,42,dynamic flags)." +details = "Implemented through IHost.PlayModalMovieToSurface. Its operand uses the same native universal packed-id catalog contract as 0x236; the separate host call exists for modal wait/cancel lifecycle, not a different resolver. ResourceMap.ResolveMovie selects through ResolvePacked and retains MPEG signature validation in ReadMovie. Godot reuses the asynchronous FFmpeg decoder/retained-surface compositor, parks only the VM thread until EOF, and treats mouse click or Accept/Cancel input as completion before wrapper cleanup releases both video and per-playback audio output. Movie playback leaves the existing mutable target at resourceId 0 with no RGB color key; MovieSurfaceRegistry supplies its live pixels. Audio-bearing movies deliver timestamped stereo float PCM through AudioStreamGenerator and use the sound-hardware position as the master clock. The native shim rewinds both demuxers to the common media origin; this preserves LOGO's 218-frame PTS-0 opening rather than skipping to the video stream's declared 600 ms start and moving picture ahead of audio. The first decoded image is latched before audio starts, later frames retain their common-timeline cadence, and the terminal image remains through the full audio graph endpoint." [[opcode.semantics.args]] i = 1 @@ -6297,7 +6297,7 @@ source = "investigation" confidence = "high" depends_on = [] evidence = "Ghidra /v2 op_0x236_play_movie_to_surface@0x423ee0 fetches operand 1 and passes it unchanged to movie_to_texture_open_asset_graph@0x463e20, which passes it unchanged to asset_open_indexed_entry@0x44f390. The opener directly indexes the flat base table or selected AAI table and has no scene input. The handler leaves playback pending through movie_play_configure@0x4625e0; movie_start_pending_after_sync_delay@0x4633b0 records the first service tick and calls the shared IMediaControl::Run worker after operand 4's delay, giving non-modal playback the same DirectShow first-sample preroll as 0x20f. SC0000 native operand capture and exact 0x13c8->0x13d1 trace prove nonblocking behavior. BTL's live 0x2b21 site supplies 0x2af1/0x2af5/0x2bca/0x2bd8/0x2bde, the exact base entries MVB001/MVB004/MVB914/MVB958/MVB955." -details = "The handler requires an existing destination texture, allocates/reuses a 0x478-byte movie-to-texture object for the surface without replacing its resource identity/color key, opens operand 1 through the native universal packed-id reader, builds FilterGraph/IGraphBuilder/IMediaControl/IMediaPosition/IMediaEvent/IBasicAudio, and presents bottom-up RGB samples through the movie texture renderer. Operand 3 selects movie/sound routing policy: bits 0x10000/0x20000/0x40000/0x80000 force mute/music/SE/voice routes, otherwise set:DependMovieSound selects the normal movie route; SC0000's low value 2 is retained as native movie mode state. Godot maps those categories to audio buses and sends FFmpeg-decoded timestamped stereo float PCM through a per-playback AudioStreamGenerator. The first image is latched before audio start; video cadence is rebased to its first decoded PTS for both audio-bearing and video-only streams, while audio retains every sample from its own timestamp zero. Operand 4 delays the pending graph start in native service ticks. Static layer preparation after 0x236 does not terminate the retained movie; 0x21c services it through EOF and subsequent surface cleanup stops/detaches both decoder and audio output. The port type-checks the selected ResolvePacked record as MPEG without adding a scene base or fallback." +details = "The handler requires an existing destination texture, allocates/reuses a 0x478-byte movie-to-texture object for the surface without replacing its resource identity/color key, opens operand 1 through the native universal packed-id reader, builds FilterGraph/IGraphBuilder/IMediaControl/IMediaPosition/IMediaEvent/IBasicAudio, and presents bottom-up RGB samples through the movie texture renderer. Operand 3 selects movie/sound routing policy: bits 0x10000/0x20000/0x40000/0x80000 force mute/music/SE/voice routes, otherwise set:DependMovieSound selects the normal movie route; SC0000's low value 2 is retained as native movie mode state. Godot maps those categories to audio buses and sends FFmpeg-decoded timestamped stereo float PCM through a per-playback AudioStreamGenerator. Both native demuxers begin at the common timeline origin; the first image is latched before audio start and later video/audio retain their source timing. This recovers valid MPEG video preroll before AVStream.start_time instead of advancing picture by roughly 600 ms. Audio-bearing completion retains the terminal frame through the full graph endpoint. Operand 4 delays the pending graph start in native service ticks. Static layer preparation after 0x236 does not terminate the retained movie; 0x21c services it through EOF and subsequent surface cleanup stops/detaches both decoder and audio output. The port type-checks the selected ResolvePacked record as MPEG without adding a scene base or fallback." [[opcode.semantics.args]] i = 1