Require both platform artifacts for releases
All checks were successful
All checks were successful
This commit is contained in:
@@ -185,9 +185,11 @@ jobs:
|
||||
publish-release:
|
||||
name: Publish tagged Gitea release
|
||||
if: startsWith(gitea.ref, 'refs/tags/v')
|
||||
needs: linux-release
|
||||
needs:
|
||||
- linux-release
|
||||
- windows-release
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
timeout-minutes: 20
|
||||
permissions:
|
||||
contents: read
|
||||
# Gitea 1.25 granular scope; GitHub-oriented actionlint does not model it.
|
||||
@@ -204,11 +206,17 @@ jobs:
|
||||
python-version: "3.11"
|
||||
architecture: x64
|
||||
|
||||
- name: Download verified workflow artifact
|
||||
- name: Download verified Linux artifact
|
||||
uses: christopherhx/gitea-download-artifact@v4
|
||||
with:
|
||||
name: OpenMaidEngine-Himegari-linux-x64-${{ gitea.sha }}
|
||||
path: build/release-assets
|
||||
path: build/release-assets/linux
|
||||
|
||||
- name: Download verified Windows artifact
|
||||
uses: christopherhx/gitea-download-artifact@v4
|
||||
with:
|
||||
name: OpenMaidEngine-Himegari-windows-x64-${{ gitea.sha }}
|
||||
path: build/release-assets/windows
|
||||
|
||||
- name: Create or resume Gitea release
|
||||
env:
|
||||
@@ -219,8 +227,6 @@ jobs:
|
||||
--repository "${{ gitea.repository }}" \
|
||||
--tag "${{ gitea.ref_name }}" \
|
||||
--target "${{ gitea.sha }}" \
|
||||
--asset build/release-assets/OpenMaidEngine-Himegari-linux-x64.tar.gz \
|
||||
--asset build/release-assets/OpenMaidEngine-Himegari-linux-x64.tar.gz.sha256 \
|
||||
--asset build/release-assets/BUILD-INFO.json \
|
||||
--asset build/release-assets/SHA256SUMS \
|
||||
--asset build/release-assets/package-smoke.log
|
||||
--linux-artifact-directory build/release-assets/linux \
|
||||
--windows-artifact-directory build/release-assets/windows \
|
||||
--output-directory build/release-assets/prepared
|
||||
|
||||
@@ -142,13 +142,19 @@ release authority. Source-only workflow regressions pin those rules and preserve
|
||||
platform artifact jobs completed and the tag-only promotion job was skipped. The Windows hosted artifact path is
|
||||
therefore accepted; dual-platform promotion is the remaining CI/CD slice.
|
||||
|
||||
The next promotion slice will require both jobs, download both already-built archives, verify their commit/build
|
||||
evidence, and publish two distributable release assets:
|
||||
Dual-platform promotion was implemented on 2026-08-03. The tag-only job now requires both successful build jobs
|
||||
and downloads their retained artifacts into separate directories without rebuilding. The release helper rechecks
|
||||
both external archive hashes, exact clean tag-commit metadata and target RID, matching build metadata/payload
|
||||
ledgers inside each archive, Linux's accepted dynamic package smoke, and Windows's complete static AMD64/ABI/
|
||||
import report. It creates or resumes only a matching release, rejects unexpected or colliding assets, and exposes
|
||||
exactly two distributable archives:
|
||||
`OpenMaidEngine-Himegari-linux-x64.tar.gz` and `OpenMaidEngine-Himegari-windows-x64.zip`. A third small
|
||||
`RELEASE-SHA256SUMS` attachment will authenticate both archives while keeping the release below Gitea's observed
|
||||
five-attachment boundary; detailed build metadata, payload ledgers, and logs remain inside each archive and in
|
||||
the retained workflow artifacts. The release notes will state that Linux received the dynamic package smoke and
|
||||
Windows received structural cross-target verification only.
|
||||
Windows received structural cross-target verification only. Source-only paired creation/resume/refusal tests
|
||||
pass. The first hosted develop run must still prove both build jobs green and promotion skipped before the user
|
||||
selects a future tag; `v0.1.0` remains unchanged.
|
||||
|
||||
Code signing, an installer, Windows-on-Linux execution, and retroactively modifying the accepted Linux-only
|
||||
`v0.1.0` release are outside this effort. A future signing slice can use `osslsigncode` and protected credentials
|
||||
|
||||
@@ -1085,6 +1085,16 @@ do not mix mechanical moves with semantic changes.
|
||||
completed both platform artifact jobs and skipped tag-only promotion, accepting the runner/toolchain/cache/
|
||||
build/upload path and the non-tag authority boundary. Slice 4 is now active.
|
||||
|
||||
**Slice 4 implementation completed 2026-08-03; hosted acceptance pending:** tag promotion now requires both
|
||||
successful platform jobs and downloads their already-built artifacts into separate directories. The helper
|
||||
binds both archives to the same clean tag commit/RID, verifies their external hashes, compares external build
|
||||
metadata and payload ledgers with the copies inside each archive, requires Linux's dynamic smoke and Windows's
|
||||
AMD64/ABI/import report, and generates combined release checksums. Creation and retry expose exactly three
|
||||
public assets—Linux archive, Windows archive, and `RELEASE-SHA256SUMS`—while unexpected assets, release
|
||||
mismatches, and same-name/different-size collisions fail without mutation. Synthetic paired-artifact,
|
||||
creation/resume/refusal, and workflow-dependency tests pass. The next gate is a hosted develop run with both
|
||||
builds green and promotion skipped; only after that proof may the user deliberately select a future tag.
|
||||
|
||||
**Completion gate:** one deliberately selected future tag produces a single Gitea release containing the
|
||||
Linux `.tar.gz`, Windows `.zip`, and combined archive checksums; both archives bind to the tag commit, Linux
|
||||
retains its dynamic packaged smoke, Windows passes all structural PE/payload gates, and neither platform job
|
||||
|
||||
@@ -60,20 +60,24 @@ repository permission and have no secrets, private game corpus, or release/packa
|
||||
first hosted dual-job develop run at `9d3ab30` succeeded on 2026-08-03: both platform artifacts completed and
|
||||
the tag-only promotion job was skipped. This accepts the Windows artifact job and its non-tag authority boundary.
|
||||
|
||||
Only a successful `v*` tag run enables the dependent promotion job. It downloads that same workflow artifact
|
||||
rather than rebuilding, checks out only the release tool, and receives job-local `releases: write` plus
|
||||
`contents: read`; branch and manual runs skip the job and never receive release authority. The built-in
|
||||
`secrets.GITEA_TOKEN` calls the native Gitea API, so no personal token or third-party release action is required.
|
||||
`publish_gitea_release.py` validates the version-like tag, exact clean-build commit metadata, archive checksum,
|
||||
accepted package-smoke result, fixed five-file asset set, and any
|
||||
existing release/asset metadata. A retry resumes a matching partial release and uploads only missing files; it
|
||||
refuses mismatched releases or same-name/different-size assets instead of editing, deleting, or overwriting them.
|
||||
Only a successful `v*` tag run after both platform jobs enables the dependent promotion job. It downloads the
|
||||
two verified workflow artifacts into separate directories rather than rebuilding, checks out only the release
|
||||
tool, and receives job-local `releases: write` plus `contents: read`; branch and manual runs skip the job and
|
||||
never receive release authority. The built-in `secrets.GITEA_TOKEN` calls the native Gitea API, so no personal
|
||||
token or third-party release action is required. `publish_gitea_release.py` validates the version-like tag, both
|
||||
exact clean-build commit/target records, both external archive hashes, matching metadata/ledgers inside each
|
||||
archive, the accepted Linux package smoke, and the Windows AMD64/ABI/import verification. It then generates one
|
||||
`RELEASE-SHA256SUMS` and publishes exactly three assets: the Linux `.tar.gz`, Windows `.zip`, and combined
|
||||
checksums. Detailed evidence remains inside each archive and in retained workflow artifacts. A retry resumes a
|
||||
matching partial release and uploads only missing files; it refuses mismatched releases, unexpected assets, or
|
||||
same-name/different-size collisions instead of editing, deleting, or overwriting them. The first hosted develop
|
||||
run of this paired promotion contract remains pending; no new version tag is created as part of that proof.
|
||||
|
||||
| Tool | Purpose | Run | Reads → Writes |
|
||||
|---|---|---|---|
|
||||
| `publish_gitea_release.py` | Tag-only, retry-safe promotion of one verified Linux workflow artifact through Gitea's native release API. Requires `GITEA_TOKEN`; the token is passed to upload `curl` through standard input rather than its argument list. | `publish_gitea_release.py --server <url> --repository <owner/name> --tag <v...> --target <sha> --asset <path>` (exactly the five expected `--asset` names) | verified downloaded workflow artifact + tag context → matching Gitea release and five attachments |
|
||||
| `test_publish_gitea_release.py` | Pure synthetic creation, retry/resume, collision, and mismatch regressions. | `test_publish_gitea_release.py` | temporary files only |
|
||||
| `test_release_workflow.py` | Source-only policy regression for the independent read-only Linux/Windows jobs, target-specific MinGW/cache/artifact boundary, no-Wine/no-Windows-smoke rule, and deliberately unchanged Linux-only tag promotion dependency during Windows CI/CD slice 3. | `test_release_workflow.py` | `.gitea/workflows/linux-release-build.yml` → assertions only |
|
||||
| `publish_gitea_release.py` | Tag-only, retry-safe promotion of paired verified Linux/Windows workflow artifacts through Gitea's native release API. Revalidates both archives/evidence sets, writes combined release checksums, and exposes exactly three public assets. Requires `GITEA_TOKEN`; the token is passed to upload `curl` through standard input rather than its argument list. | `publish_gitea_release.py --server <url> --repository <owner/name> --tag <v...> --target <sha> --linux-artifact-directory <dir> --windows-artifact-directory <dir> --output-directory <dir>` | two verified downloaded workflow artifacts + tag context → matching Gitea release with Linux archive, Windows archive, and `RELEASE-SHA256SUMS` |
|
||||
| `test_publish_gitea_release.py` | Pure synthetic paired-artifact/archive validation, three-asset creation, retry/resume, collision, unexpected-asset, and mismatch regressions. | `test_publish_gitea_release.py` | temporary files only |
|
||||
| `test_release_workflow.py` | Source-only policy regression for the independent read-only Linux/Windows jobs, target-specific MinGW/cache/artifact boundary, no-Wine/no-Windows-smoke rule, and promotion's dependency/download of both successful artifacts. | `test_release_workflow.py` | `.gitea/workflows/linux-release-build.yml` → assertions only |
|
||||
|
||||
## Optional local binary tools
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Promote one verified Linux workflow artifact to a matching Gitea release."""
|
||||
"""Validate paired platform artifacts and promote three assets to Gitea."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
@@ -9,23 +9,42 @@ import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import tarfile
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable, Protocol
|
||||
|
||||
import verify_windows_native
|
||||
|
||||
EXPECTED_ASSETS = frozenset({
|
||||
"OpenMaidEngine-Himegari-linux-x64.tar.gz",
|
||||
"OpenMaidEngine-Himegari-linux-x64.tar.gz.sha256",
|
||||
|
||||
LINUX_ARCHIVE = "OpenMaidEngine-Himegari-linux-x64.tar.gz"
|
||||
WINDOWS_ARCHIVE = "OpenMaidEngine-Himegari-windows-x64.zip"
|
||||
RELEASE_CHECKSUMS = "RELEASE-SHA256SUMS"
|
||||
LINUX_ARTIFACT_FILES = frozenset({
|
||||
LINUX_ARCHIVE,
|
||||
f"{LINUX_ARCHIVE}.sha256",
|
||||
"BUILD-INFO.json",
|
||||
"SHA256SUMS",
|
||||
"package-smoke.log",
|
||||
})
|
||||
WINDOWS_ARTIFACT_FILES = frozenset({
|
||||
WINDOWS_ARCHIVE,
|
||||
f"{WINDOWS_ARCHIVE}.sha256",
|
||||
"BUILD-INFO.json",
|
||||
"SHA256SUMS",
|
||||
"WINDOWS-VERIFICATION.json",
|
||||
})
|
||||
EXPECTED_RELEASE_ASSETS = frozenset({LINUX_ARCHIVE, WINDOWS_ARCHIVE, RELEASE_CHECKSUMS})
|
||||
LINUX_PACKAGE_ROOT = "OpenMaidEngine-Himegari-linux-x64"
|
||||
WINDOWS_PACKAGE_ROOT = "OpenMaidEngine-Himegari-windows-x64"
|
||||
SMOKE_MARKER = "PACKAGE SMOKE OK: opcodes=548 ffmpeg-abi=3"
|
||||
TAG_PATTERN = re.compile(r"v[0-9][0-9A-Za-z.+-]*\Z")
|
||||
COMMIT_PATTERN = re.compile(r"[0-9a-f]{40}\Z")
|
||||
TOKEN_PATTERN = re.compile(r"[A-Za-z0-9._-]+\Z")
|
||||
MAX_EVIDENCE_SIZE = 1024 * 1024
|
||||
|
||||
|
||||
class ReleaseApi(Protocol):
|
||||
@@ -113,14 +132,8 @@ class GiteaApi:
|
||||
)
|
||||
process = subprocess.run(
|
||||
[
|
||||
"curl",
|
||||
"--config", "-",
|
||||
"--fail-with-body",
|
||||
"--silent",
|
||||
"--show-error",
|
||||
"--request", "POST",
|
||||
"--form", f"attachment=@{asset}",
|
||||
url,
|
||||
"curl", "--config", "-", "--fail-with-body", "--silent", "--show-error",
|
||||
"--request", "POST", "--form", f"attachment=@{asset}", url,
|
||||
],
|
||||
input=curl_config,
|
||||
text=True,
|
||||
@@ -143,62 +156,223 @@ class GiteaApi:
|
||||
return result
|
||||
|
||||
|
||||
def validate_inputs(tag: str, target: str, assets: list[Path]) -> None:
|
||||
def _validate_tag_target(tag: str, target: str) -> None:
|
||||
if not TAG_PATTERN.fullmatch(tag):
|
||||
raise ValueError(f"release tag must be v-prefixed and version-like: {tag}")
|
||||
if not COMMIT_PATTERN.fullmatch(target):
|
||||
raise ValueError(f"release target must be a lowercase SHA-1 commit: {target}")
|
||||
names = [asset.name for asset in assets]
|
||||
if len(names) != len(set(names)):
|
||||
raise ValueError("release asset names must be unique")
|
||||
if set(names) != EXPECTED_ASSETS:
|
||||
missing = sorted(EXPECTED_ASSETS - set(names))
|
||||
extra = sorted(set(names) - EXPECTED_ASSETS)
|
||||
raise ValueError(f"unexpected release asset set; missing={missing}, extra={extra}")
|
||||
for asset in assets:
|
||||
if not asset.is_file():
|
||||
raise ValueError(f"release asset was not found: {asset}")
|
||||
if asset.stat().st_size <= 0:
|
||||
raise ValueError(f"release asset is empty: {asset}")
|
||||
|
||||
by_name = {asset.name: asset for asset in assets}
|
||||
|
||||
def _artifact_files(directory: Path, expected: frozenset[str], label: str) -> dict[str, Path]:
|
||||
directory = directory.resolve()
|
||||
if not directory.is_dir():
|
||||
raise ValueError(f"{label} artifact directory was not found: {directory}")
|
||||
entries = list(directory.iterdir())
|
||||
invalid = [entry.name for entry in entries if not entry.is_file() or entry.is_symlink()]
|
||||
if invalid:
|
||||
raise ValueError(f"{label} artifact has invalid entries: {', '.join(sorted(invalid))}")
|
||||
by_name = {entry.name: entry for entry in entries}
|
||||
if set(by_name) != expected:
|
||||
missing = sorted(expected - set(by_name))
|
||||
extra = sorted(set(by_name) - expected)
|
||||
raise ValueError(f"unexpected {label} artifact set; missing={missing}, extra={extra}")
|
||||
empty = sorted(name for name, path in by_name.items() if path.stat().st_size <= 0)
|
||||
if empty:
|
||||
raise ValueError(f"{label} artifact has empty files: {', '.join(empty)}")
|
||||
return by_name
|
||||
|
||||
|
||||
def _hash(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as stream:
|
||||
for chunk in iter(lambda: stream.read(1024 * 1024), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def _validate_external_checksum(archive: Path, checksum: Path, label: str) -> str:
|
||||
try:
|
||||
build_info = json.loads(by_name["BUILD-INFO.json"].read_text(encoding="utf-8"))
|
||||
except (json.JSONDecodeError, UnicodeDecodeError) as error:
|
||||
raise ValueError("BUILD-INFO.json is not valid UTF-8 JSON") from error
|
||||
expected_build_info = {
|
||||
line = checksum.read_text(encoding="ascii").strip()
|
||||
except UnicodeDecodeError as error:
|
||||
raise ValueError(f"{label} archive checksum is not ASCII") from error
|
||||
match = re.fullmatch(rf"([0-9a-f]{{64}}) {re.escape(archive.name)}", line)
|
||||
if match is None:
|
||||
raise ValueError(f"{label} archive checksum file has an unexpected format: {checksum}")
|
||||
actual = _hash(archive)
|
||||
if actual != match.group(1):
|
||||
raise ValueError(f"{label} archive does not match its external SHA-256 checksum")
|
||||
return actual
|
||||
|
||||
|
||||
def _json_bytes(data: bytes, label: str) -> dict[str, Any]:
|
||||
try:
|
||||
parsed = json.loads(data.decode("utf-8"))
|
||||
except (UnicodeDecodeError, json.JSONDecodeError) as error:
|
||||
raise ValueError(f"{label} is not valid UTF-8 JSON") from error
|
||||
if not isinstance(parsed, dict):
|
||||
raise ValueError(f"{label} JSON root is not an object")
|
||||
return parsed
|
||||
|
||||
|
||||
def _validate_build_info(data: bytes, target: str, rid: str, label: str) -> None:
|
||||
build_info = _json_bytes(data, label)
|
||||
expected = {
|
||||
"schema_version": 1,
|
||||
"source_commit": target,
|
||||
"source_dirty": False,
|
||||
"target": "linux-x64",
|
||||
"target": rid,
|
||||
}
|
||||
build_mismatches = {
|
||||
mismatches = {
|
||||
key: (build_info.get(key), value)
|
||||
for key, value in expected_build_info.items()
|
||||
for key, value in expected.items()
|
||||
if build_info.get(key) != value
|
||||
}
|
||||
if build_mismatches:
|
||||
raise ValueError(f"BUILD-INFO.json does not match this promotion: {build_mismatches}")
|
||||
if mismatches:
|
||||
raise ValueError(f"{label} does not match this promotion: {mismatches}")
|
||||
|
||||
archive = by_name["OpenMaidEngine-Himegari-linux-x64.tar.gz"]
|
||||
checksum_path = by_name["OpenMaidEngine-Himegari-linux-x64.tar.gz.sha256"]
|
||||
checksum_line = checksum_path.read_text(encoding="ascii").strip()
|
||||
checksum_match = re.fullmatch(
|
||||
r"([0-9a-f]{64}) OpenMaidEngine-Himegari-linux-x64\.tar\.gz",
|
||||
checksum_line,
|
||||
|
||||
def _tar_member(archive_path: Path, name: str) -> bytes:
|
||||
with tarfile.open(archive_path, mode="r:gz") as archive:
|
||||
try:
|
||||
member = archive.getmember(name)
|
||||
except KeyError as error:
|
||||
raise ValueError(f"Linux archive evidence was not found: {name}") from error
|
||||
if not member.isfile() or member.size <= 0 or member.size > MAX_EVIDENCE_SIZE:
|
||||
raise ValueError(f"Linux archive evidence has an invalid size or type: {name}")
|
||||
stream = archive.extractfile(member)
|
||||
if stream is None:
|
||||
raise ValueError(f"Linux archive evidence could not be read: {name}")
|
||||
return stream.read()
|
||||
|
||||
|
||||
def _zip_member(archive_path: Path, name: str) -> bytes:
|
||||
with zipfile.ZipFile(archive_path) as archive:
|
||||
try:
|
||||
info = archive.getinfo(name)
|
||||
except KeyError as error:
|
||||
raise ValueError(f"Windows archive evidence was not found: {name}") from error
|
||||
if info.is_dir() or info.file_size <= 0 or info.file_size > MAX_EVIDENCE_SIZE:
|
||||
raise ValueError(f"Windows archive evidence has an invalid size or type: {name}")
|
||||
with archive.open(info) as stream:
|
||||
return stream.read()
|
||||
|
||||
|
||||
def _validate_windows_verification(data: bytes) -> None:
|
||||
report = _json_bytes(data, "WINDOWS-VERIFICATION.json")
|
||||
native = report.get("native")
|
||||
if not isinstance(native, dict):
|
||||
raise ValueError("WINDOWS-VERIFICATION.json has no native report")
|
||||
machines = native.get("machines")
|
||||
expected_machines = {
|
||||
name: "AMD64"
|
||||
for name in (verify_windows_native.SHIM, *verify_windows_native.RUNTIME_DLLS)
|
||||
}
|
||||
expected = {
|
||||
"schema_version": 1,
|
||||
"target": "win-x64",
|
||||
"executable_machine": "AMD64",
|
||||
}
|
||||
mismatches = {
|
||||
key: (report.get(key), value)
|
||||
for key, value in expected.items()
|
||||
if report.get(key) != value
|
||||
}
|
||||
if native.get("schema_version") != 1 or native.get("target") != "win-x64":
|
||||
mismatches["native"] = ((native.get("schema_version"), native.get("target")), (1, "win-x64"))
|
||||
if native.get("bundle") != "data_Himegari_windows_x86_64":
|
||||
mismatches["bundle"] = (native.get("bundle"), "data_Himegari_windows_x86_64")
|
||||
if machines != expected_machines:
|
||||
mismatches["machines"] = (machines, expected_machines)
|
||||
exports_value = native.get("shim_exports")
|
||||
exports = (
|
||||
{name for name in exports_value if isinstance(name, str)}
|
||||
if isinstance(exports_value, list)
|
||||
else set()
|
||||
)
|
||||
if checksum_match is None:
|
||||
raise ValueError(f"archive checksum file has an unexpected format: {checksum_path}")
|
||||
digest = hashlib.sha256()
|
||||
with archive.open("rb") as stream:
|
||||
for chunk in iter(lambda: stream.read(1024 * 1024), b""):
|
||||
digest.update(chunk)
|
||||
if digest.hexdigest() != checksum_match.group(1):
|
||||
raise ValueError("Linux archive does not match its external SHA-256 checksum")
|
||||
if exports != verify_windows_native.REQUIRED_EXPORTS:
|
||||
mismatches["shim_exports"] = (exports_value, "required AGE ABI exports")
|
||||
imports_value = native.get("shim_imports")
|
||||
imports = (
|
||||
{name.lower() for name in imports_value if isinstance(name, str)}
|
||||
if isinstance(imports_value, list)
|
||||
else set()
|
||||
)
|
||||
required_imports = {name.lower() for name in verify_windows_native.RUNTIME_DLLS}
|
||||
if not required_imports.issubset(imports):
|
||||
mismatches["shim_imports"] = (sorted(imports), sorted(required_imports))
|
||||
forbidden_imports = imports & verify_windows_native.FORBIDDEN_RUNTIME_IMPORTS
|
||||
if forbidden_imports:
|
||||
mismatches["forbidden_imports"] = (sorted(forbidden_imports), [])
|
||||
if mismatches:
|
||||
raise ValueError(f"WINDOWS-VERIFICATION.json is not accepted: {mismatches}")
|
||||
|
||||
smoke_log = by_name["package-smoke.log"].read_text(encoding="utf-8")
|
||||
if "PACKAGE SMOKE OK: opcodes=548 ffmpeg-abi=3" not in smoke_log:
|
||||
raise ValueError("package-smoke.log does not contain the accepted package smoke result")
|
||||
|
||||
def prepare_release_assets(
|
||||
tag: str,
|
||||
target: str,
|
||||
linux_directory: Path,
|
||||
windows_directory: Path,
|
||||
output_directory: Path,
|
||||
) -> list[Path]:
|
||||
_validate_tag_target(tag, target)
|
||||
linux = _artifact_files(linux_directory, LINUX_ARTIFACT_FILES, "Linux")
|
||||
windows = _artifact_files(windows_directory, WINDOWS_ARTIFACT_FILES, "Windows")
|
||||
linux_hash = _validate_external_checksum(
|
||||
linux[LINUX_ARCHIVE], linux[f"{LINUX_ARCHIVE}.sha256"], "Linux"
|
||||
)
|
||||
windows_hash = _validate_external_checksum(
|
||||
windows[WINDOWS_ARCHIVE], windows[f"{WINDOWS_ARCHIVE}.sha256"], "Windows"
|
||||
)
|
||||
|
||||
linux_build_info = linux["BUILD-INFO.json"].read_bytes()
|
||||
windows_build_info = windows["BUILD-INFO.json"].read_bytes()
|
||||
linux_ledger = linux["SHA256SUMS"].read_bytes()
|
||||
windows_ledger = windows["SHA256SUMS"].read_bytes()
|
||||
windows_verification = windows["WINDOWS-VERIFICATION.json"].read_bytes()
|
||||
_validate_build_info(linux_build_info, target, "linux-x64", "Linux BUILD-INFO.json")
|
||||
_validate_build_info(windows_build_info, target, "win-x64", "Windows BUILD-INFO.json")
|
||||
if SMOKE_MARKER not in linux["package-smoke.log"].read_text(encoding="utf-8"):
|
||||
raise ValueError("package-smoke.log does not contain the accepted Linux package smoke result")
|
||||
_validate_windows_verification(windows_verification)
|
||||
|
||||
internal_linux_build = _tar_member(
|
||||
linux[LINUX_ARCHIVE], f"{LINUX_PACKAGE_ROOT}/BUILD-INFO.json"
|
||||
)
|
||||
internal_linux_ledger = _tar_member(
|
||||
linux[LINUX_ARCHIVE], f"{LINUX_PACKAGE_ROOT}/SHA256SUMS"
|
||||
)
|
||||
internal_windows_build = _zip_member(
|
||||
windows[WINDOWS_ARCHIVE], f"{WINDOWS_PACKAGE_ROOT}/BUILD-INFO.json"
|
||||
)
|
||||
internal_windows_ledger = _zip_member(
|
||||
windows[WINDOWS_ARCHIVE], f"{WINDOWS_PACKAGE_ROOT}/SHA256SUMS"
|
||||
)
|
||||
internal_windows_verification = _zip_member(
|
||||
windows[WINDOWS_ARCHIVE], f"{WINDOWS_PACKAGE_ROOT}/WINDOWS-VERIFICATION.json"
|
||||
)
|
||||
comparisons = (
|
||||
(linux_build_info, internal_linux_build, "Linux BUILD-INFO.json"),
|
||||
(linux_ledger, internal_linux_ledger, "Linux SHA256SUMS"),
|
||||
(windows_build_info, internal_windows_build, "Windows BUILD-INFO.json"),
|
||||
(windows_ledger, internal_windows_ledger, "Windows SHA256SUMS"),
|
||||
(windows_verification, internal_windows_verification, "WINDOWS-VERIFICATION.json"),
|
||||
)
|
||||
for external, internal, label in comparisons:
|
||||
if external != internal:
|
||||
raise ValueError(f"external and packaged {label} do not match")
|
||||
|
||||
if b" Himegari.x86_64\n" not in linux_ledger:
|
||||
raise ValueError("Linux SHA256SUMS does not contain the packaged executable")
|
||||
if b" Himegari.exe\n" not in windows_ledger:
|
||||
raise ValueError("Windows SHA256SUMS does not contain the packaged executable")
|
||||
|
||||
output_directory = output_directory.resolve()
|
||||
output_directory.mkdir(parents=True, exist_ok=True)
|
||||
checksum_path = output_directory / RELEASE_CHECKSUMS
|
||||
content = f"{linux_hash} {LINUX_ARCHIVE}\n{windows_hash} {WINDOWS_ARCHIVE}\n"
|
||||
with checksum_path.open("w", encoding="ascii", newline="\n") as stream:
|
||||
stream.write(content)
|
||||
return [linux[LINUX_ARCHIVE], windows[WINDOWS_ARCHIVE], checksum_path]
|
||||
|
||||
|
||||
def _validate_release(release: dict[str, Any], tag: str, target: str, title: str) -> int:
|
||||
@@ -227,9 +401,15 @@ def promote_release(
|
||||
upload: Callable[[int, Path], dict[str, Any]],
|
||||
tag: str,
|
||||
target: str,
|
||||
assets: list[Path],
|
||||
linux_directory: Path,
|
||||
windows_directory: Path,
|
||||
output_directory: Path,
|
||||
) -> dict[str, Any]:
|
||||
validate_inputs(tag, target, assets)
|
||||
assets = prepare_release_assets(
|
||||
tag, target, linux_directory, windows_directory, output_directory
|
||||
)
|
||||
if {asset.name for asset in assets} != EXPECTED_RELEASE_ASSETS:
|
||||
raise RuntimeError("prepared release asset set is invalid")
|
||||
title = f"OpenMaidEngine Himegari {tag}"
|
||||
release = api.get_release(tag)
|
||||
if release is None:
|
||||
@@ -238,9 +418,10 @@ def promote_release(
|
||||
"target_commitish": target,
|
||||
"name": title,
|
||||
"body": (
|
||||
"Automated Linux x64 release built from `" + target + "`.\n\n"
|
||||
"The attached archive passed the packaged opcode-metadata and FFmpeg ABI smoke gate. "
|
||||
"BUILD-INFO.json, SHA256SUMS, and package-smoke.log provide the external build evidence."
|
||||
"Automated Linux and Windows x64 release built from `" + target + "`.\n\n"
|
||||
"The Linux archive passed the dynamic opcode/FFmpeg package smoke. The Windows archive "
|
||||
"passed structural AMD64 PE, payload, ABI export, and FFmpeg import verification without "
|
||||
"executing the EXE. Each archive contains its detailed build metadata and payload ledger."
|
||||
),
|
||||
"draft": False,
|
||||
"prerelease": False,
|
||||
@@ -254,6 +435,9 @@ def promote_release(
|
||||
if name in existing_assets:
|
||||
raise ValueError(f"Gitea release has duplicate asset names: {name}")
|
||||
existing_assets[name] = existing
|
||||
unexpected = sorted(set(existing_assets) - EXPECTED_RELEASE_ASSETS)
|
||||
if unexpected:
|
||||
raise ValueError("Gitea release has unexpected assets: " + ", ".join(unexpected))
|
||||
|
||||
for asset in assets:
|
||||
existing = existing_assets.get(asset.name)
|
||||
@@ -268,9 +452,10 @@ def promote_release(
|
||||
raise RuntimeError(f"Gitea reported an unexpected uploaded asset: {asset.name}")
|
||||
|
||||
final_assets = {asset.get("name"): asset for asset in api.list_assets(release_id)}
|
||||
if set(final_assets) != EXPECTED_RELEASE_ASSETS:
|
||||
raise RuntimeError(f"release asset verification failed: {sorted(final_assets)}")
|
||||
for asset in assets:
|
||||
published = final_assets.get(asset.name)
|
||||
if published is None or published.get("size") != asset.stat().st_size:
|
||||
if final_assets[asset.name].get("size") != asset.stat().st_size:
|
||||
raise RuntimeError(f"release asset verification failed: {asset.name}")
|
||||
return release
|
||||
|
||||
@@ -281,12 +466,22 @@ def main(arguments: list[str] | None = None) -> int:
|
||||
parser.add_argument("--repository", required=True)
|
||||
parser.add_argument("--tag", required=True)
|
||||
parser.add_argument("--target", required=True)
|
||||
parser.add_argument("--asset", action="append", required=True, type=Path)
|
||||
parser.add_argument("--linux-artifact-directory", required=True, type=Path)
|
||||
parser.add_argument("--windows-artifact-directory", required=True, type=Path)
|
||||
parser.add_argument("--output-directory", required=True, type=Path)
|
||||
args = parser.parse_args(arguments)
|
||||
|
||||
token = os.environ.get("GITEA_TOKEN", "")
|
||||
api = GiteaApi(args.server, args.repository, token)
|
||||
release = promote_release(api, api.upload_asset, args.tag, args.target, args.asset)
|
||||
release = promote_release(
|
||||
api,
|
||||
api.upload_asset,
|
||||
args.tag,
|
||||
args.target,
|
||||
args.linux_artifact_directory,
|
||||
args.windows_artifact_directory,
|
||||
args.output_directory,
|
||||
)
|
||||
print(f"Gitea release ready: {release.get('html_url', args.tag)}")
|
||||
return 0
|
||||
|
||||
@@ -294,6 +489,6 @@ def main(arguments: list[str] | None = None) -> int:
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
raise SystemExit(main())
|
||||
except (OSError, RuntimeError, ValueError) as error:
|
||||
except (OSError, RuntimeError, ValueError, tarfile.TarError, zipfile.BadZipFile) as error:
|
||||
print(f"release promotion failed: {error}", file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
|
||||
@@ -2,17 +2,21 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import tarfile
|
||||
import tempfile
|
||||
import unittest
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import publish_gitea_release
|
||||
import verify_windows_native
|
||||
|
||||
|
||||
TARGET = "0123456789abcdef0123456789abcdef01234567"
|
||||
TAG = "v0.1.0"
|
||||
TAG = "v0.2.0"
|
||||
|
||||
|
||||
class FakeApi:
|
||||
@@ -48,46 +52,127 @@ def matching_release() -> dict[str, Any]:
|
||||
"name": f"OpenMaidEngine Himegari {TAG}",
|
||||
"draft": False,
|
||||
"prerelease": False,
|
||||
"html_url": "https://gitea.invalid/releases/tag/v0.1.0",
|
||||
"html_url": "https://gitea.invalid/releases/tag/v0.2.0",
|
||||
}
|
||||
|
||||
|
||||
def create_assets(root: Path) -> list[Path]:
|
||||
archive = root / "OpenMaidEngine-Himegari-linux-x64.tar.gz"
|
||||
archive.write_bytes(b"synthetic archive")
|
||||
checksum = root / "OpenMaidEngine-Himegari-linux-x64.tar.gz.sha256"
|
||||
checksum.write_text(
|
||||
f"{hashlib.sha256(archive.read_bytes()).hexdigest()} {archive.name}\n",
|
||||
encoding="ascii",
|
||||
)
|
||||
build_info = root / "BUILD-INFO.json"
|
||||
build_info.write_text(json.dumps({
|
||||
def json_bytes(value: dict[str, Any]) -> bytes:
|
||||
return (json.dumps(value, indent=2, sort_keys=True) + "\n").encode()
|
||||
|
||||
|
||||
def build_info(rid: str) -> bytes:
|
||||
return json_bytes({
|
||||
"schema_version": 1,
|
||||
"source_commit": TARGET,
|
||||
"source_dirty": False,
|
||||
"target": "linux-x64",
|
||||
}), encoding="utf-8")
|
||||
ledger = root / "SHA256SUMS"
|
||||
ledger.write_text(f"{'0' * 64} Himegari.x86_64\n", encoding="ascii")
|
||||
smoke = root / "package-smoke.log"
|
||||
smoke.write_text("PACKAGE SMOKE OK: opcodes=548 ffmpeg-abi=3\n", encoding="utf-8")
|
||||
return [archive, checksum, build_info, ledger, smoke]
|
||||
"target": rid,
|
||||
})
|
||||
|
||||
|
||||
def windows_verification() -> bytes:
|
||||
return json_bytes({
|
||||
"schema_version": 1,
|
||||
"target": "win-x64",
|
||||
"executable_machine": "AMD64",
|
||||
"native": {
|
||||
"schema_version": 1,
|
||||
"target": "win-x64",
|
||||
"bundle": "data_Himegari_windows_x86_64",
|
||||
"machines": {
|
||||
name: "AMD64"
|
||||
for name in (verify_windows_native.SHIM, *verify_windows_native.RUNTIME_DLLS)
|
||||
},
|
||||
"shim_exports": sorted(verify_windows_native.REQUIRED_EXPORTS),
|
||||
"shim_imports": [*verify_windows_native.RUNTIME_DLLS, "KERNEL32.dll"],
|
||||
},
|
||||
})
|
||||
|
||||
|
||||
def add_tar_bytes(archive: tarfile.TarFile, name: str, data: bytes) -> None:
|
||||
info = tarfile.TarInfo(name)
|
||||
info.size = len(data)
|
||||
archive.addfile(info, io.BytesIO(data))
|
||||
|
||||
|
||||
def write_checksum(directory: Path, archive: Path) -> None:
|
||||
digest = hashlib.sha256(archive.read_bytes()).hexdigest()
|
||||
(directory / f"{archive.name}.sha256").write_text(
|
||||
f"{digest} {archive.name}\n", encoding="ascii"
|
||||
)
|
||||
|
||||
|
||||
def create_artifacts(root: Path) -> tuple[Path, Path, Path]:
|
||||
linux = root / "linux"
|
||||
windows = root / "windows"
|
||||
prepared = root / "prepared"
|
||||
linux.mkdir()
|
||||
windows.mkdir()
|
||||
|
||||
linux_info = build_info("linux-x64")
|
||||
linux_ledger = f"{'0' * 64} Himegari.x86_64\n".encode()
|
||||
linux_archive = linux / publish_gitea_release.LINUX_ARCHIVE
|
||||
with tarfile.open(linux_archive, "w:gz") as archive:
|
||||
add_tar_bytes(
|
||||
archive,
|
||||
f"{publish_gitea_release.LINUX_PACKAGE_ROOT}/BUILD-INFO.json",
|
||||
linux_info,
|
||||
)
|
||||
add_tar_bytes(
|
||||
archive,
|
||||
f"{publish_gitea_release.LINUX_PACKAGE_ROOT}/SHA256SUMS",
|
||||
linux_ledger,
|
||||
)
|
||||
(linux / "BUILD-INFO.json").write_bytes(linux_info)
|
||||
(linux / "SHA256SUMS").write_bytes(linux_ledger)
|
||||
(linux / "package-smoke.log").write_text(
|
||||
publish_gitea_release.SMOKE_MARKER + "\n", encoding="utf-8"
|
||||
)
|
||||
write_checksum(linux, linux_archive)
|
||||
|
||||
windows_info = build_info("win-x64")
|
||||
windows_ledger = f"{'1' * 64} Himegari.exe\n".encode()
|
||||
verification = windows_verification()
|
||||
windows_archive = windows / publish_gitea_release.WINDOWS_ARCHIVE
|
||||
with zipfile.ZipFile(windows_archive, "w", compression=zipfile.ZIP_DEFLATED) as archive:
|
||||
root_name = publish_gitea_release.WINDOWS_PACKAGE_ROOT
|
||||
archive.writestr(f"{root_name}/BUILD-INFO.json", windows_info)
|
||||
archive.writestr(f"{root_name}/SHA256SUMS", windows_ledger)
|
||||
archive.writestr(f"{root_name}/WINDOWS-VERIFICATION.json", verification)
|
||||
(windows / "BUILD-INFO.json").write_bytes(windows_info)
|
||||
(windows / "SHA256SUMS").write_bytes(windows_ledger)
|
||||
(windows / "WINDOWS-VERIFICATION.json").write_bytes(verification)
|
||||
write_checksum(windows, windows_archive)
|
||||
return linux, windows, prepared
|
||||
|
||||
|
||||
class PublishGiteaReleaseTests(unittest.TestCase):
|
||||
def test_creates_release_and_uploads_verified_asset_set(self) -> None:
|
||||
def test_creates_release_and_uploads_three_verified_assets(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
assets = create_assets(Path(temporary))
|
||||
linux, windows, prepared = create_artifacts(Path(temporary))
|
||||
api = FakeApi()
|
||||
result = publish_gitea_release.promote_release(api, api.upload, TAG, TARGET, assets)
|
||||
result = publish_gitea_release.promote_release(
|
||||
api, api.upload, TAG, TARGET, linux, windows, prepared
|
||||
)
|
||||
self.assertEqual(17, result["id"])
|
||||
self.assertEqual(TAG, api.created_payload["tag_name"])
|
||||
self.assertEqual(TARGET, api.created_payload["target_commitish"])
|
||||
self.assertEqual({asset.name for asset in assets}, {asset["name"] for asset in api.assets})
|
||||
self.assertIn("Windows archive", api.created_payload["body"])
|
||||
self.assertEqual(
|
||||
publish_gitea_release.EXPECTED_RELEASE_ASSETS,
|
||||
{asset["name"] for asset in api.assets},
|
||||
)
|
||||
checksums = (prepared / publish_gitea_release.RELEASE_CHECKSUMS).read_text(
|
||||
encoding="ascii"
|
||||
)
|
||||
self.assertIn(publish_gitea_release.LINUX_ARCHIVE, checksums)
|
||||
self.assertIn(publish_gitea_release.WINDOWS_ARCHIVE, checksums)
|
||||
|
||||
def test_retry_keeps_matching_assets_and_uploads_only_missing(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
assets = create_assets(Path(temporary))
|
||||
linux, windows, prepared = create_artifacts(Path(temporary))
|
||||
assets = publish_gitea_release.prepare_release_assets(
|
||||
TAG, TARGET, linux, windows, prepared
|
||||
)
|
||||
api = FakeApi(matching_release())
|
||||
api.assets.append({"name": assets[0].name, "size": assets[0].stat().st_size})
|
||||
uploaded: list[str] = []
|
||||
@@ -96,40 +181,68 @@ class PublishGiteaReleaseTests(unittest.TestCase):
|
||||
uploaded.append(asset.name)
|
||||
return api.upload(release_id, asset)
|
||||
|
||||
publish_gitea_release.promote_release(api, upload, TAG, TARGET, assets)
|
||||
publish_gitea_release.promote_release(
|
||||
api, upload, TAG, TARGET, linux, windows, prepared
|
||||
)
|
||||
self.assertNotIn(assets[0].name, uploaded)
|
||||
self.assertEqual(len(assets) - 1, len(uploaded))
|
||||
self.assertEqual(2, len(uploaded))
|
||||
self.assertIsNone(api.created_payload)
|
||||
|
||||
def test_refuses_mismatched_release_and_asset_collision(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
assets = create_assets(Path(temporary))
|
||||
linux, windows, prepared = create_artifacts(Path(temporary))
|
||||
wrong_release = matching_release()
|
||||
wrong_release["target_commitish"] = "f" * 40
|
||||
api = FakeApi(wrong_release)
|
||||
with self.assertRaisesRegex(ValueError, "does not match"):
|
||||
publish_gitea_release.promote_release(api, api.upload, TAG, TARGET, assets)
|
||||
publish_gitea_release.promote_release(
|
||||
api, api.upload, TAG, TARGET, linux, windows, prepared
|
||||
)
|
||||
|
||||
api = FakeApi(matching_release())
|
||||
api.assets.append({"name": assets[0].name, "size": 999})
|
||||
api.assets.append({"name": publish_gitea_release.LINUX_ARCHIVE, "size": 999})
|
||||
with self.assertRaisesRegex(ValueError, "will not be overwritten"):
|
||||
publish_gitea_release.promote_release(api, api.upload, TAG, TARGET, assets)
|
||||
publish_gitea_release.promote_release(
|
||||
api, api.upload, TAG, TARGET, linux, windows, prepared
|
||||
)
|
||||
|
||||
def test_refuses_mismatched_build_evidence(self) -> None:
|
||||
api = FakeApi(matching_release())
|
||||
api.assets.append({"name": "stale.txt", "size": 1})
|
||||
with self.assertRaisesRegex(ValueError, "unexpected assets"):
|
||||
publish_gitea_release.promote_release(
|
||||
api, api.upload, TAG, TARGET, linux, windows, prepared
|
||||
)
|
||||
|
||||
def test_refuses_mismatched_archive_commit_and_structural_evidence(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
assets = create_assets(Path(temporary))
|
||||
by_name = {asset.name: asset for asset in assets}
|
||||
by_name["OpenMaidEngine-Himegari-linux-x64.tar.gz"].write_bytes(b"changed")
|
||||
linux, windows, prepared = create_artifacts(Path(temporary))
|
||||
(linux / publish_gitea_release.LINUX_ARCHIVE).write_bytes(b"changed")
|
||||
with self.assertRaisesRegex(ValueError, "external SHA-256"):
|
||||
publish_gitea_release.validate_inputs(TAG, TARGET, assets)
|
||||
publish_gitea_release.prepare_release_assets(
|
||||
TAG, TARGET, linux, windows, prepared
|
||||
)
|
||||
|
||||
assets = create_assets(Path(temporary))
|
||||
by_name = {asset.name: asset for asset in assets}
|
||||
build_info = json.loads(by_name["BUILD-INFO.json"].read_text(encoding="utf-8"))
|
||||
build_info["source_commit"] = "f" * 40
|
||||
by_name["BUILD-INFO.json"].write_text(json.dumps(build_info), encoding="utf-8")
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
linux, windows, prepared = create_artifacts(Path(temporary))
|
||||
info = json.loads((windows / "BUILD-INFO.json").read_text(encoding="utf-8"))
|
||||
info["source_commit"] = "f" * 40
|
||||
(windows / "BUILD-INFO.json").write_bytes(json_bytes(info))
|
||||
with self.assertRaisesRegex(ValueError, "does not match"):
|
||||
publish_gitea_release.validate_inputs(TAG, TARGET, assets)
|
||||
publish_gitea_release.prepare_release_assets(
|
||||
TAG, TARGET, linux, windows, prepared
|
||||
)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
linux, windows, prepared = create_artifacts(Path(temporary))
|
||||
report = json.loads(
|
||||
(windows / "WINDOWS-VERIFICATION.json").read_text(encoding="utf-8")
|
||||
)
|
||||
report["executable_machine"] = "I386"
|
||||
(windows / "WINDOWS-VERIFICATION.json").write_bytes(json_bytes(report))
|
||||
with self.assertRaisesRegex(ValueError, "not accepted"):
|
||||
publish_gitea_release.prepare_release_assets(
|
||||
TAG, TARGET, linux, windows, prepared
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -59,11 +59,21 @@ class ReleaseWorkflowTests(unittest.TestCase):
|
||||
self.assertIn("if-no-files-found: error", self.windows)
|
||||
self.assertIn("retention-days: 30", self.windows)
|
||||
|
||||
def test_slice_three_keeps_linux_only_tag_promotion_boundary(self) -> None:
|
||||
def test_tag_promotion_requires_and_downloads_both_platform_artifacts(self) -> None:
|
||||
self.assertIn("if: startsWith(gitea.ref, 'refs/tags/v')", self.publish)
|
||||
self.assertRegex(self.publish, r"(?m)^ needs: linux-release$")
|
||||
self.assertRegex(
|
||||
self.publish,
|
||||
r"(?m)^ needs:\n - linux-release\n - windows-release$",
|
||||
)
|
||||
self.assertIn("releases: write", self.publish)
|
||||
self.assertNotIn("windows-release", self.publish)
|
||||
self.assertIn("OpenMaidEngine-Himegari-linux-x64-${{ gitea.sha }}", self.publish)
|
||||
self.assertIn("OpenMaidEngine-Himegari-windows-x64-${{ gitea.sha }}", self.publish)
|
||||
self.assertIn("path: build/release-assets/linux", self.publish)
|
||||
self.assertIn("path: build/release-assets/windows", self.publish)
|
||||
self.assertIn("--linux-artifact-directory build/release-assets/linux", self.publish)
|
||||
self.assertIn("--windows-artifact-directory build/release-assets/windows", self.publish)
|
||||
self.assertIn("--output-directory build/release-assets/prepared", self.publish)
|
||||
self.assertNotIn("--asset", self.publish)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
Reference in New Issue
Block a user