Commit Graph

32 Commits

Author SHA1 Message Date
gamer147
b94a20b9ec feat(tools): per-scene opcode completeness tracker + 0x259 marker fix
Add tools/scene_opcode_coverage.py: histograms a scene's static opcodes and
classifies each vs the C# VM as impl / safe-noop / GAP (effectful op the VM
silently stubs). Implemented set is parsed live from VirtualMachine.cs case arms
(no drift); metadata from build/opcodes.json. Makes a half-rendered scene legible
("N ops still stubbed") instead of implying everything runs.

SC0000 baseline: 129 distinct ops, ~94.8% instruction-weighted handled, 68 GAP.
The tracker cross-checks opcodes.toml vs VM behavior and surfaced 0x259
(script-entry marker) missing its noop_headless flag -> reconciled in opcodes.toml
and rebuilt (regen: age_opcodes_himegari.py, opcode-reference.md).

Docs: tools-reference.md (tool row), phase-a-slice-plan.md (completeness gauge).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 22:57:25 -04:00
gamer147
5c21074454 chore(gfx): set dispatch labels for the gfx command-buffer ops
VM dispatches on the opcodes.toml label field (OpcodeTableJson), so the 14 gfx
ops' labels are set to their dispatch strings. Ops still stubbed until the VM
cases land (Task 3.3+); full suite green (34), lint clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 17:44:32 -04:00
gamer147
2a2377badb docs(gfx): reverse the full gfx command-buffer op contract (Phase 1)
Ghidra-read all 13 still-stubbed gfx-family handlers via the dispatch table
(0x1a2/0x1f7/0x1fa/0x1ff/0x202/0x203/0x212/0x213/0x216/0x217/0x218/0x219/0x21a);
recovered 0x219 (was unanalyzed). Every op writes a cmd-type into the current
object record then SETs fields (native worker) or QUERYs fields (returns to
operands). Recorded the op-contract table in engine-re.md, set opcodes.toml
semantics (source=investigation, confidence=high), renamed handlers in Ghidra.

Key finding: the drift has TWO stubbed drivers, not one — 0x215 (slot-select)
AND 0x218/0x21a (per-object geometry vectors feeding the anchor-preserve math).
The subsystem is a per-object record model (slot + position/anchor 3-vectors +
color) queried by the ops; native DirectDraw workers need not be modelled. This
is the spec for Phase 2 (host-side model design).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 17:01:14 -04:00
gamer147
5b07f4ac2b docs(gfx): settle render drift as native cmd-buffer op 0x215 (Ghidra), align docs
Read op 0x215's real handler FUN_0042a0b0 (resolved via the dispatch table
ctx[0x26c93+op]; Kelebek's 0x421160 is VA-drift). It writes cmd-type 5 into the
current gfx-object record and returns a std::map::find over an engine-internal
registry populated by sibling gfx ops (0x1a2 hash insert). The return is native
command-buffer state, not the VM global bank -> seeding story-state cannot fix
the drift. Verdict: (b) a genuine native op, NOT (a) state-divergence.

Reconcile the previously contradictory drift accounts onto one canonical home
(engine-re.md op 0x215), with opcodes.toml carrying the opcode-level semantics
and phase-a-slice-plan / tools-reference / frida README corrected to point at it
instead of repeating the disproven state-divergence conclusion.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 16:40:59 -04:00
gamer147
c302dda9db Resolve call-script dispatch: id = raw SYS4INI file index
Native-RE (Ghidra) cracked call-script <id> (opcode 0x03): its handler
FUN_0041bc90 -> loader FUN_0040e980 -> resolver FUN_0044f390 indexes an
80-byte record table at base + id*0x50 == the SYS4INI record layout. So
`call-script <id>` is a direct RAW index into the SYS4INI global file
table (the asset index we already parse) -- there is no separate on-disk
id->code registry. This resolves name-resolution.md #1, statically, no
Frida.

Confirmed: all 297 distinct corpus call-script ids resolve to a .BIN
script with a semantically-exact name (0x1ab->ADDITEM, 0x2ae7->MES,
0x143->BUNKI, 0x329d->CALCREVISE), 0 out-of-range, 0 alternate-pack.
Companion op 0x8f `call` is an intra-script JSR (FUN_0041fba0), not
cross-script.

- parse_sys4ini.py: preserve `raw_index` per entry (= the engine file id;
  index the RAW records incl. '@' placeholders) + emit
  build/callscript-names.json (id->name).
- sys4load.py: annotate `call-script 0x1ab =ADDITEM.BIN`.
- opcodes.toml 0x03/0x8f refined (source=investigation, confidence high,
  handler VAs) + rebuilt opcode-reference.md.
- docs: engine-re.md (op 0x03 section + backlog re-aimed),
  name-resolution.md #1 (SOLVED), script-inventory.md (call graph +
  living-reference decision), tools-reference.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 12:44:51 -04:00
gamer147
ed9da5b38d docs: propagate u00428010/op-0x1a2 correction into opcodes.toml + name-resolution
op 0x1a2 resolve-handle? -> gfx-cmd-register (verified handler FUN_0042d360, gfx
command-buffer op). name-resolution: decision->scene hop is native+unidentified, not
u00428010 (disproven). Regenerated opcode-reference + shim.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 11:47:35 -04:00
gamer147
fe23accfa2 feat: --verify execution-driven VM cross-check for SCJUMP decode (Task 4)
Reworked from static-witness-only to execution-driven: ~99% of SCJUMP decisions
are gated by a native computed value (op 0x60), so witness-synthesis alone can't
cover them. Value-local tracking resolves load-then-compare guards to real globals;
native terms honestly marked opaque. Verify: static 3/3 exact + 279/279 execution-
driven consistent over 2000 seeds, 0 failures.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 09:30:10 -04:00
gamer147
9862e29af0 feat: emit SCJUMP decision table (json+md, registry-named) (Task 3)
build/scjump-decisions.* are generated (build/ gitignored).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 09:20:33 -04:00
gamer147
110994ec77 feat: SCJUMP guarded-DFS decode -> decision list (Task 2)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 09:18:29 -04:00
gamer147
f2480bac1e feat: SCJUMP decoder scaffold — CFG acyclic check + chapter dispatch (Task 1)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 09:17:30 -04:00
gamer147
e9406e201a feat: curate story-flags into registry + docs (Task 6)
Bootstrap 56 high-signal skeletons; name game_mode/route_branch/scjump_decision_out2,
recategorize CONFIG-written globals as non-story. Docs: name-resolution.md registry
section, CLAUDE.md canonical+SoT tables + trigger (root, untracked), tools-reference rows.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 08:33:51 -04:00
gamer147
431707a0b9 feat: story_flags --bootstrap seeds skeletons into globals.toml (Task 5)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 08:27:31 -04:00
gamer147
03de0bf0da feat: static story-flag miner -> candidates JSON (Task 4)
Mines 1261 branch-read globals (205 story-flag candidates); build/story-flags-
candidates.json is generated (build/ gitignored). Known anchors 0x3234/0xa57 verified.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 08:26:03 -04:00
gamer147
3ea632df12 feat: sys4load labels globals from merged build/globals.json (Task 3)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 08:24:36 -04:00
gamer147
44a326bb7e feat: globals_build --build merge + generated JSON/MD (Task 2)
build/globals.json is generated (build/ is gitignored, regenerable via
globals_build.py --build); docs/global-reference.md is the tracked human view.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 08:23:17 -04:00
gamer147
ae5f2cca7b feat: globals.toml registry source + loader/lint (Task 1)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 08:21:59 -04:00
gamer147
896bea977b feat(frida): capture_gfx_objects + finding — drift is state-divergence
Live capture (esi=engine ctx via operand-fetch, poll object-record array
[esi+0x53d64] stride 120). KEY FINDING: through the full real opening, the record
array holds only 3 persistent UI objects — NO CG objects. The real game does NOT
draw opening CGs via the 0x212-0x21a positioned-object path our headless VM uses;
with state it takes a different (direct) branch. So the bg/sprite drift is a
STATE-DIVERGENCE artifact of the unseeded headless VM, not a missing native op —
the fix is the Phase B state/choices flow (makes label_12649 take the if-branch).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 00:04:49 -04:00
gamer147
eee070722a feat(frida): probe_handlers — confirms interpreter runs from module 0x400000
operand-fetch (call 0x41b940) fires ~8500/sec => the VM interpreter executes
from the in-place unpacked module at 0x400000 (NOT the heap copy) => handlers are
hookable by dump address. gfx-family(0x212-0x215)=0 at the title (no CG commands
until a scene runs).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 23:51:18 -04:00
gamer147
d85a6003cc fix(frida): dump_engine — use ptr.readByteArray (frida 17), skip system DLLs
Dump validated: AGE.EXE is unpacked in-place at 0x400000 in memory (AGF-decoder
landmark @0x474f1f reads real code with the 'BM' 0x4D42 check). Kelebek handler
VAs map directly (VA-0x400000 = file offset). Handler ABI: thiscall (esi=engine
context), operands fetched via call 0x41b940, per-object command-type table at
[esi+idx*120+0x53d88]. 0x215 is part of a native gfx command-buffer manager.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 23:43:54 -04:00
gamer147
fdd5bd0613 feat(frida): dump_engine.py — dump unpacked engine code from live process
Both on-disk engine images are the same packed binary, so native handler code
exists only unpacked in memory. dump_engine attaches, enumerates ranges, and
dumps the AGE.EXE module + large r-x heap regions (chunked) to build/engine-dump/
for offline disassembly (locate 0x215 @ VA 0x421160 via the dispatch table).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 23:33:02 -04:00
gamer147
6df1db0839 docs(opcodes): 0x215 = native graphics-object query (root cause of gfx drift)
Investigation upgrade: 0x215 queries the native object manager by element
handle-id (0x62455[idx]) and returns the object's slot/status; sign-tested to
drive label_12649's slot-select. Stubbing it collapses all draws onto slot 0 →
the anchor-preserve geometry reads foreign textures → cumulative bg/sprite
drift. Keystone for the graphics object-manager; exact return via unicorn
(handler @0x421160).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 23:21:08 -04:00
gamer147
f0d75fa0dc feat(a2b): first-pass texture render — full-screen event-CG layer
Wire asset resolution into a live Godot render. VM executes set-texture ->
ResourceMap resolves (scene,resId) -> files[section_base+resId] across all
archives -> pre-converted BMP -> composite behind the dialogue. The full-screen
event-CG layer (EV052*) renders end-to-end from the executed bytecode.

- Age.Engine/Sys4/ResourceMap.cs: Resolve + BMP TexturePath; Paths: asset JSONs
- GodotAdvHost: create/set/draw-texture -> TextureRect in a _stage layer
- IHost.DrawTexture + VM dispatch extended with dst x/y (draw-texture args 7/8)
- project.godot 800x600; convert_agf.py all-archive + --scene batch
- engine 8/8, C# --selftest still byte-matches vm0 trace (VM behaviour unchanged)

Known limitations (next chunk = graphics geometry/blend subsystem):
- sprites + BG* via the CG-load subroutine get garbage dst/size — native ops
  stubbed (0x208 get-texture-size + sprite position/anim chain)
- AE* fades draw opaque/instant (no alpha); no chromakey
- slot model approximates the game's immediate-mode blit-onto-slot-0 canvas
- AGF pre-converted to BMP offline (runtime decoder deferred)
See docs/phase-a-slice-plan.md (A2b section).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 21:22:42 -04:00
gamer147
102fe1d021 feat(assets): solve asset resolution (SYS4INI per-scene section manifest)
resId -> files[section_base(scene) + resId]. SYS4INI's file list is
sectioned, one per scene (SCxxxx.BIN + its cross-archive asset manifest);
file_number is the index within the section. Unified for set-texture,
play-bgm, play-voice. Fully static/general -> no per-scene capture.

- tools/parse_sys4ini.py: SYS4INI (S4IC422, LZSS) -> build/asset-index.json
- tools/resolve_asset.py: sections + (scene,resId) resolver -> build/asset-sections.json
- validated: 97% structural, SC0000 17/17 vs Frida, 586/595 captured loads
- opcodes.toml: set-texture/create/draw-texture, play-bgm/voice enriched (frida-grounded)
- Frida tooling (capture_load_order all-archive, correlate_scope, ...) + vm0 --settex
- docs: asset-resolution-re (step2 SOLVED), global-memory-re (shelved), tools-reference

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 20:48:22 -04:00
gamer147
30d2734df4 feat(a2b): asset-RE tooling + findings; render blocked on asset resolution
- tools/convert_agf.py (AGF->BMP stills), tools/frida/ (capture harness + README; Frida 17.15.3)
- docs/asset-resolution-re.md: foundational RE steering doc (resId->file; graphics+audio; not
  machine-verifiable -> Frida ground truth + human oracle). RE plan: parse SYS4INI, crack resId->name,
  backend render (A2b plan Tasks 3-5), audio, movies.
- Finding: SC0000 bg = slot-0 slideshow (res 0x23 first); resolution opaque (CGINIT not a name map,
  SYS4INI S4IC needs RE, Frida file-I/O noisy/memory-mapped, opening mixes MPEG movies).
- slice plan updated; texture ops already engine-driven (Task 1, prior commit).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 16:36:17 -04:00
gamer147
6feba77b8e feat(a1): CLI (run/trace) + vm0.py --trace + per-scene differential test (A1 green)
C# VM is byte-identical to vm0.py across all 297 SC/SP scenes (offsets+halt+steps);
RECOVER passes; SC0000 = 27994 steps / 186 lines matching the Python prototype.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 13:55:11 -04:00
gamer147
4f0b802d18 feat(opcodes): migrate to opcodes.toml as single source of truth; regenerate artifacts
- 248 opcodes seeded from Kelebek + corpus arg-types; 23 inferences migrated
- source = provenance (frida/unicorn methods = inference + confirm_by, not grounded)
- 0x90/0x97 enriched with depends_on + details
- differential-verified: SC0830/MENU disasm byte-identical, vm0 --test PASS, sweep 282/294

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 13:11:33 -04:00
gamer147
71a43ab48a feat(opcodes): emit opcodes.json, opcode-reference.md, coverage
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 13:04:35 -04:00
gamer147
0364207dff feat(opcodes): emit drop-in age_opcodes_himegari.py shim; wire --build/--lint
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 13:03:33 -04:00
gamer147
1f2a5f2bf1 feat(opcodes): bootstrap seeds 248 skeletons from Kelebek + corpus arg-types
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 13:02:28 -04:00
gamer147
06d1e286b9 feat(opcodes): linter (dangling-ref, confidence-ceiling, vocabulary)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 13:01:20 -04:00
gamer147
b33e1740cb feat(opcodes): data model + loader for opcodes.toml
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 13:00:31 -04:00
gamer147
aa3da00bfa chore: initialize age-reimpl repo
Reverse-engineering + open reimplementation workspace for Eushully's AGE/SYS4
engine (first target: Himegari). The repo root is age-reimpl/; the original game
install and the extracted ALF data are siblings outside the repo and are never
tracked. build/ (derived corpora) is gitignored and regenerated by the tools.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 12:58:30 -04:00