diff --git a/docs/platform-portability.md b/docs/platform-portability.md index 0a395e7..0e717bd 100644 --- a/docs/platform-portability.md +++ b/docs/platform-portability.md @@ -221,8 +221,10 @@ VM op 0x236 (non-modal) / op 0x20f (modal) / op 0x24d (green-mask transition) Everything before and after the selected decoder is portable. The replacement decision is an in-process FFmpeg backend behind a project-owned C ABI, not raw FFmpeg structs in Godot/C# and not a subprocess. FFmpeg -`n8.1.2-29-g703dcc25b9` is pinned by immutable release URL and SHA-256 in each target manifest under -`native/age_movie_ffmpeg/`; changing either pin requires rerunning the full installed-movie gate. The shim +`n8.1.2-44-g7c533d0f86` is pinned by dated release URL, exact byte size, and SHA-256 in each target manifest under +`native/age_movie_ffmpeg/`. BtbN prunes older dated autobuild releases, so upstream availability is not permanent: +the current paired pin restores hosted builds, while a durable project-owned mirror remains necessary to prevent +future 404-driven repins. Changing either pin requires rerunning the full installed-movie gate. The shim dynamically links an LGPL build made without GPL or nonfree components and uses `libavformat`, `libavcodec`, `libavutil`, `libswscale`, and `libswresample`. Release artifacts must carry the matching FFmpeg source/configuration and notices required by FFmpeg's diff --git a/docs/tools-reference.md b/docs/tools-reference.md index 7a45945..5f6b10d 100644 --- a/docs/tools-reference.md +++ b/docs/tools-reference.md @@ -291,18 +291,22 @@ send either the `step-limit` console block or the generated JSON. ## Native FFmpeg movie shim (Windows and Linux x64) -The target-specific dependency manifests pin immutable LGPL shared FFmpeg archives and SHA-256 hashes. Windows -development can use PowerShell/MSVC. Linux-hosted builds use Bash with Python 3, curl, MinGW-w64/binutils for the -Windows target, a native C compiler/binutils for Linux, and the ordinary archive/core utilities available. +The target-specific dependency manifests pin dated LGPL shared FFmpeg archives by exact byte size and SHA-256. +BtbN prunes older dated autobuild releases, so the bytes are immutable while retained but their upstream URLs +are not permanent; a failed 404 requires a paired manifest repin and full native/movie validation, while a +project-owned durable mirror remains the long-term fix. Bootstrap downloads remove partial files on HTTP failure +before validating the complete archive. Windows development can use PowerShell/MSVC. Linux-hosted builds use Bash +with Python 3, curl, MinGW-w64/binutils for the Windows target, a native C compiler/binutils for Linux, and the +ordinary archive/core utilities available. | Tool | Purpose | Run | Reads → Writes | |---|---|---|---| -| `native/age_movie_ffmpeg/bootstrap-win64.ps1` | Download, hash-check, extract, and version-check the pinned FFmpeg SDK. Returns the resolved SDK root. | `.\\native\\age_movie_ffmpeg\\bootstrap-win64.ps1 [-Destination ]` | `dependency-win64.json`, network/archive cache → disposable `build/downloads/`, `build/ffmpeg-sdk/` | +| `native/age_movie_ffmpeg/bootstrap-win64.ps1` | Download, size/hash-check, extract, and version-check the pinned FFmpeg SDK. Returns the resolved SDK root. | `.\\native\\age_movie_ffmpeg\\bootstrap-win64.ps1 [-Destination ]` | `dependency-win64.json`, network/archive cache → disposable `build/downloads/`, `build/ffmpeg-sdk/` | | `native/age_movie_ffmpeg/build-win64.ps1` | Discover the MSVC x64 toolchain, build `age_movie_ffmpeg.dll`, and stage its exact shared-library/license dependencies. | `.\\native\\age_movie_ffmpeg\\build-win64.ps1 -SdkRoot [-OutputDirectory ]` | C ABI source + FFmpeg SDK → disposable `build/native/win-x64/` by default | -| `native/age_movie_ffmpeg/bootstrap-win64.sh` | Download, SHA-256-check, and extract the exact pinned Windows FFmpeg SDK, including its MinGW import libraries. Prints the resolved SDK root. | `native/age_movie_ffmpeg/bootstrap-win64.sh [destination]` | `dependency-win64.json`, network/archive cache → disposable `build/downloads/`, `build/ffmpeg-sdk/` | +| `native/age_movie_ffmpeg/bootstrap-win64.sh` | Download, size/SHA-256-check, and extract the exact pinned Windows FFmpeg SDK, including its MinGW import libraries. Prints the resolved SDK root. | `native/age_movie_ffmpeg/bootstrap-win64.sh [destination]` | `dependency-win64.json`, network/archive cache → disposable `build/downloads/`, `build/ffmpeg-sdk/` | | `native/age_movie_ffmpeg/build-win64.sh` | Cross-build the AMD64 PE shim with MinGW, stage exactly five FFmpeg DLLs and the license, and run the static native-bundle contract gate. `MINGW_CC` and `MINGW_OBJDUMP` may override the standard tool names. | `sdk_root="$(native/age_movie_ffmpeg/bootstrap-win64.sh)"` then `native/age_movie_ffmpeg/build-win64.sh "$sdk_root" [output-directory]` | C ABI source + Windows FFmpeg SDK → disposable `build/native/win-x64/` by default, including `verification.json` | | `tools/verify_windows_native.py` | Without executing Windows code, require the exact native bundle, AMD64 PE architecture, all seven ABI exports and five FFmpeg imports, and no Cygwin/MSYS compatibility runtime. Optionally writes the machine-readable verification report used by later packaging. | `python3 -X utf8 tools/verify_windows_native.py [--objdump ] [--report ]` | disposable Windows native bundle → stdout and optional JSON report | -| `native/age_movie_ffmpeg/bootstrap-linux-x64.sh` | Download, SHA-256-check, extract, and version-check the pinned Linux x64 FFmpeg SDK. Prints the resolved SDK root. | `native/age_movie_ffmpeg/bootstrap-linux-x64.sh [destination]` | `dependency-linux-x64.json`, network/archive cache → disposable `build/downloads/`, `build/ffmpeg-sdk/` | +| `native/age_movie_ffmpeg/bootstrap-linux-x64.sh` | Download, size/SHA-256-check, extract, and version-check the pinned Linux x64 FFmpeg SDK. Prints the resolved SDK root. | `native/age_movie_ffmpeg/bootstrap-linux-x64.sh [destination]` | `dependency-linux-x64.json`, network/archive cache → disposable `build/downloads/`, `build/ffmpeg-sdk/` | | `native/age_movie_ffmpeg/build-linux-x64.sh` | Build the ELF64 x86-64 shim; stage the five exact FFmpeg SONAME libraries and license; reject stale `.so` files, unresolved/nonlocal FFmpeg dependencies, a missing `$ORIGIN` runpath, or a glibc requirement newer than the pinned 2.28 baseline. | `sdk_root="$(native/age_movie_ffmpeg/bootstrap-linux-x64.sh)"` then `native/age_movie_ffmpeg/build-linux-x64.sh "$sdk_root" [output-directory]` | C ABI source + FFmpeg SDK → disposable `build/native/linux-x64/` by default | | `tools/movie-corpus-gate` | Discover every MPEG program stream stored under an `.AGF` catalog entry; decode every video frame and, when present, every audio block through the unpaced FFmpeg session; validate independent sequence dimensions, metadata, RGBA size, stereo finite PCM, monotonic video/audio timestamps, EOF, timeout, and teardown; then emit a per-asset JSON report. Audio fields include sample rate/channels, block and PCM-frame counts, first/last PTS, signal presence, and decode time. Exit 0 means the expected corpus count and every asset passed; exit 1 is a gate failure; exit 2 means the native shim is absent. | `dotnet run --project tools/movie-corpus-gate -- --output build/movie-corpus-ffmpeg.json --expected-count 213 --max-item-ms 30000` · optional `--native-dir ` | `SYS4INI.BIN` + loose/ALF VFS assets + staged FFmpeg shim → stdout progress + disposable `build/movie-corpus-ffmpeg.json` | diff --git a/native/age_movie_ffmpeg/bootstrap-linux-x64.sh b/native/age_movie_ffmpeg/bootstrap-linux-x64.sh index 4e3d527..fbe025f 100755 --- a/native/age_movie_ffmpeg/bootstrap-linux-x64.sh +++ b/native/age_movie_ffmpeg/bootstrap-linux-x64.sh @@ -11,7 +11,7 @@ manifest_value() { "$manifest" "$1" } -for command in python3 curl sha256sum tar awk; do +for command in python3 curl sha256sum tar awk wc; do command -v "$command" >/dev/null 2>&1 || { echo "required command was not found: $command" >&2 exit 1 @@ -21,6 +21,7 @@ done archive="$(manifest_value archive)" url="$(manifest_value url)" expected_hash="$(manifest_value sha256)" +expected_size="$(manifest_value size)" expected_version="$(manifest_value ffmpeg_version)" archive_root="${archive%.tar.xz}" download_dir="$repo_root/build/downloads" @@ -29,9 +30,14 @@ sdk_root="$destination/$archive_root" mkdir -p -- "$download_dir" "$destination" if [[ ! -f "$archive_path" ]]; then - curl --fail --location --retry 3 --output "$archive_path" "$url" + curl --fail --location --retry 3 --remove-on-error --output "$archive_path" "$url" fi +actual_size="$(wc -c < "$archive_path")" +if [[ "$actual_size" != "$expected_size" ]]; then + echo "FFmpeg archive size mismatch: expected $expected_size, got $actual_size" >&2 + exit 1 +fi actual_hash="$(sha256sum "$archive_path" | awk '{ print $1 }')" if [[ "$actual_hash" != "$expected_hash" ]]; then echo "FFmpeg archive SHA-256 mismatch: expected $expected_hash, got $actual_hash" >&2 diff --git a/native/age_movie_ffmpeg/bootstrap-win64.ps1 b/native/age_movie_ffmpeg/bootstrap-win64.ps1 index 78625fb..e58abbe 100644 --- a/native/age_movie_ffmpeg/bootstrap-win64.ps1 +++ b/native/age_movie_ffmpeg/bootstrap-win64.ps1 @@ -10,11 +10,24 @@ if ([string]::IsNullOrWhiteSpace($Destination)) { $manifest = Get-Content -LiteralPath (Join-Path $PSScriptRoot 'dependency-win64.json') -Raw | ConvertFrom-Json $downloadDir = Join-Path $repoRoot 'build\downloads' $archivePath = Join-Path $downloadDir $manifest.archive +$partialArchivePath = "$archivePath.part" $extractRoot = Join-Path $Destination ([IO.Path]::GetFileNameWithoutExtension($manifest.archive)) New-Item -ItemType Directory -Force -Path $downloadDir | Out-Null if (-not (Test-Path -LiteralPath $archivePath)) { - Invoke-WebRequest -Uri $manifest.url -OutFile $archivePath + try { + Invoke-WebRequest -Uri $manifest.url -OutFile $partialArchivePath + Move-Item -LiteralPath $partialArchivePath -Destination $archivePath + } + finally { + if (Test-Path -LiteralPath $partialArchivePath) { + Remove-Item -LiteralPath $partialArchivePath -Force + } + } +} +$actualSize = (Get-Item -LiteralPath $archivePath).Length +if ($actualSize -ne $manifest.size) { + throw "FFmpeg archive size mismatch: expected $($manifest.size), got $actualSize" } $actualHash = (Get-FileHash -Algorithm SHA256 -LiteralPath $archivePath).Hash.ToLowerInvariant() if ($actualHash -ne $manifest.sha256) { @@ -28,7 +41,7 @@ if (-not (Test-Path -LiteralPath (Join-Path $sdkRoot 'include\libavformat\avform throw "Pinned FFmpeg SDK was not found at $sdkRoot" } $reported = & (Join-Path $sdkRoot 'bin\ffmpeg.exe') -version | Select-Object -First 1 -$expectedVersion = $manifest.ffmpeg_version -replace '-20260721$', '' +$expectedVersion = $manifest.ffmpeg_version -replace '-\d{8}$', '' if ($reported -notlike "*$expectedVersion*" ) { throw "Unexpected FFmpeg build: $reported" } diff --git a/native/age_movie_ffmpeg/bootstrap-win64.sh b/native/age_movie_ffmpeg/bootstrap-win64.sh index c29252f..ca3556c 100755 --- a/native/age_movie_ffmpeg/bootstrap-win64.sh +++ b/native/age_movie_ffmpeg/bootstrap-win64.sh @@ -11,7 +11,7 @@ manifest_value() { "$manifest" "$1" } -for command in python3 curl sha256sum awk; do +for command in python3 curl sha256sum awk wc; do command -v "$command" >/dev/null 2>&1 || { echo "required command was not found: $command" >&2 exit 1 @@ -21,6 +21,7 @@ done archive="$(manifest_value archive)" url="$(manifest_value url)" expected_hash="$(manifest_value sha256)" +expected_size="$(manifest_value size)" archive_root="${archive%.zip}" download_directory="$repo_root/build/downloads" archive_path="$download_directory/$archive" @@ -28,9 +29,14 @@ sdk_root="$destination/$archive_root" mkdir -p -- "$download_directory" "$destination" if [[ ! -f "$archive_path" ]]; then - curl --fail --location --retry 3 --output "$archive_path" "$url" + curl --fail --location --retry 3 --remove-on-error --output "$archive_path" "$url" fi +actual_size="$(wc -c < "$archive_path")" +if [[ "$actual_size" != "$expected_size" ]]; then + echo "FFmpeg archive size mismatch: expected $expected_size, got $actual_size" >&2 + exit 1 +fi actual_hash="$(sha256sum "$archive_path" | awk '{ print $1 }')" if [[ "$actual_hash" != "$expected_hash" ]]; then echo "FFmpeg archive SHA-256 mismatch: expected $expected_hash, got $actual_hash" >&2 diff --git a/native/age_movie_ffmpeg/dependency-linux-x64.json b/native/age_movie_ffmpeg/dependency-linux-x64.json index ac80f0c..6d13fb4 100644 --- a/native/age_movie_ffmpeg/dependency-linux-x64.json +++ b/native/age_movie_ffmpeg/dependency-linux-x64.json @@ -1,11 +1,12 @@ { "provider": "BtbN/FFmpeg-Builds", - "release_tag": "autobuild-2026-07-21-13-38", - "archive": "ffmpeg-n8.1.2-29-g703dcc25b9-linux64-lgpl-shared-8.1.tar.xz", - "url": "https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2026-07-21-13-38/ffmpeg-n8.1.2-29-g703dcc25b9-linux64-lgpl-shared-8.1.tar.xz", - "sha256": "d0026954f10d303d1fc6517ed6b2290c42323b3958f11baafb1b330dfc564b36", - "ffmpeg_version": "n8.1.2-29-g703dcc25b9-20260721", - "ffmpeg_commit": "703dcc25b9", + "release_tag": "autobuild-2026-08-17-13-05", + "archive": "ffmpeg-n8.1.2-44-g7c533d0f86-linux64-lgpl-shared-8.1.tar.xz", + "url": "https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2026-08-17-13-05/ffmpeg-n8.1.2-44-g7c533d0f86-linux64-lgpl-shared-8.1.tar.xz", + "size": 56931952, + "sha256": "194dc8d525ac8cea5a5ff2e6fd44754747e69af5a766e7a6727cce363bd65de6", + "ffmpeg_version": "n8.1.2-44-g7c533d0f86-20260817", + "ffmpeg_commit": "7c533d0f86", "variant": "linux64 LGPL shared 8.1", "minimum_glibc": "2.28", "minimum_linux_kernel": "4.18" diff --git a/native/age_movie_ffmpeg/dependency-win64.json b/native/age_movie_ffmpeg/dependency-win64.json index fb27197..0d4dbba 100644 --- a/native/age_movie_ffmpeg/dependency-win64.json +++ b/native/age_movie_ffmpeg/dependency-win64.json @@ -1,10 +1,11 @@ { "provider": "BtbN/FFmpeg-Builds", - "release_tag": "autobuild-2026-07-21-13-38", - "archive": "ffmpeg-n8.1.2-29-g703dcc25b9-win64-lgpl-shared-8.1.zip", - "url": "https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2026-07-21-13-38/ffmpeg-n8.1.2-29-g703dcc25b9-win64-lgpl-shared-8.1.zip", - "sha256": "50764b52d38cb0baf8af938c0e4ca886d2c6753b520b2c319b532ed5c17e7cbf", - "ffmpeg_version": "n8.1.2-29-g703dcc25b9-20260721", - "ffmpeg_commit": "703dcc25b9", + "release_tag": "autobuild-2026-08-17-13-05", + "archive": "ffmpeg-n8.1.2-44-g7c533d0f86-win64-lgpl-shared-8.1.zip", + "url": "https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2026-08-17-13-05/ffmpeg-n8.1.2-44-g7c533d0f86-win64-lgpl-shared-8.1.zip", + "size": 70837934, + "sha256": "681b9ca6d8f9be1e01d8873ad16f8a632f8a22b9653f1044837de6d5979b0fd6", + "ffmpeg_version": "n8.1.2-44-g7c533d0f86-20260817", + "ffmpeg_commit": "7c533d0f86", "variant": "win64 LGPL shared 8.1" } diff --git a/tools/test_release_workflow.py b/tools/test_release_workflow.py index aadd686..cdc6946 100755 --- a/tools/test_release_workflow.py +++ b/tools/test_release_workflow.py @@ -1,6 +1,7 @@ #!/usr/bin/env python3 from __future__ import annotations +import json import re import unittest from pathlib import Path @@ -15,6 +16,10 @@ MANAGED_SOLUTION = REPO / "godot/OME.sln" LINUX_BUILD = REPO / "tools/build-linux-x64.sh" WINDOWS_BUILD = REPO / "tools/build-windows-x64.sh" WINDOWS_HOSTED_EXPORT = REPO / "tools/export-linux-x64.ps1" +LINUX_FFMPEG_MANIFEST = REPO / "native/age_movie_ffmpeg/dependency-linux-x64.json" +WINDOWS_FFMPEG_MANIFEST = REPO / "native/age_movie_ffmpeg/dependency-win64.json" +LINUX_FFMPEG_BOOTSTRAP = REPO / "native/age_movie_ffmpeg/bootstrap-linux-x64.sh" +WINDOWS_FFMPEG_BOOTSTRAP = REPO / "native/age_movie_ffmpeg/bootstrap-win64.sh" def job(text: str, name: str, next_name: str | None) -> str: @@ -38,6 +43,25 @@ class ReleaseWorkflowTests(unittest.TestCase): cls.windows_build = WINDOWS_BUILD.read_text(encoding="utf-8") cls.windows_hosted_export = WINDOWS_HOSTED_EXPORT.read_text(encoding="utf-8") + def test_ffmpeg_inputs_are_dated_size_and_hash_pinned(self) -> None: + manifests = [ + json.loads(LINUX_FFMPEG_MANIFEST.read_text(encoding="utf-8")), + json.loads(WINDOWS_FFMPEG_MANIFEST.read_text(encoding="utf-8")), + ] + self.assertEqual(1, len({manifest["release_tag"] for manifest in manifests})) + for manifest in manifests: + self.assertRegex(manifest["release_tag"], r"^autobuild-\d{4}-\d{2}-\d{2}-\d{2}-\d{2}$") + self.assertNotEqual("latest", manifest["release_tag"]) + self.assertIn(f"/{manifest['release_tag']}/", manifest["url"]) + self.assertTrue(manifest["url"].endswith("/" + manifest["archive"])) + self.assertIn(manifest["ffmpeg_commit"], manifest["archive"]) + self.assertGreater(manifest["size"], 50_000_000) + self.assertRegex(manifest["sha256"], r"^[0-9a-f]{64}$") + for bootstrap in (LINUX_FFMPEG_BOOTSTRAP, WINDOWS_FFMPEG_BOOTSTRAP): + text = bootstrap.read_text(encoding="utf-8") + self.assertIn("manifest_value size", text) + self.assertIn("--remove-on-error", text) + def test_public_build_outputs_use_ome_branding(self) -> None: self.assertIn('export_path="../build/export/linux-x64/OME"', self.export_presets) self.assertIn('export_path="../build/export/windows-x64/OME.exe"', self.export_presets)