# SYS4 Script Format — Reverse-Engineering Notes (hex-first)

Derived purely from byte-pattern analysis of the 481 DATA1 `.BIN` scripts, before
any Ghidra work on `AGE.EXE`. Confidence levels flagged per finding. Probe scripts
live in `tools/probe_*.py`.

> **UPDATE 2026-07-05 — opcode set solved via Kelebek1's table.** Everything below
> under "Instruction stream — PARTIAL" is now resolved: code = instructions of
> `<opcode:u32> + argc*(<argtype:u32><value:u32>)`, length `1+2*argc`; inline strings
> live after code inside `[0,F8)`, so stop decoding at the first type-2/`0x64` arg
> offset. 476/476 scripts decode clean (0 unknown opcodes). Header fields F0–F5 are
> **local-variable counts** (F0=local_integer_1, F1=local_floats, F2=local_strings_1,
> F3=local_integer_2, F4=unknown, F5=local_strings_2). See `vm-mapping-plan.md` and
> `vm-map/opcodes.toml`. The tag values below (0x71/0x03/0x8F etc.) are the
> *opcodes at table targets*: 0x8F=`call`, 0x03=`call-script`, 0x71=`u0041A7B0`.

## Header — CONFIRMED

Fixed 60-byte (0x3C) header: 8-byte magic + thirteen little-endian u32 fields.
Verified across all 481 files.

```
off   field  meaning                                    evidence
0x00  magic  "SYS4422 " (0x53 59 53 34 34 32 32 20)     481/481 identical
0x08  F0     scenario/script id or flags                98 distinct; 0x5 dominant (252×)
0x0C  F1     = 1 always                                 481/481 == 1
0x10  F2     = 1 (457×) or 2 (20×); 4 outliers          format/feature flag
0x14  F3     0x06 (268×),0x04,0x01,0x05...              15 distinct — minor version?
0x18  F4     = 1 always                                 481/481 == 1
0x1C  F5     = 2 (319×) or 1 (144×); rarely 3/5         flag
0x20  F6     = 0x1C always                              481/481 == 0x1C  (header-size marker)
0x24  F7     table-1 entry count                        see below
0x28  F8     table-1 offset  == code-section length     ordering F8<=F10<=F12<=EOF
0x2C  F9     table-2 entry count
0x30  F10    table-2 offset
0x34  F11    table-3 entry count
0x38  F12    table-3 offset
0x3C  body   dword stream (code + 3 tables + strings)
```

**All offsets/counts are in DWORDS (×4 bytes), relative to body start (0x3C).**
The `0x1C` in F6 is the only *byte* count — it's the offset from F6's own position
(0x20) back-referenced, i.e. a self-describing "28 bytes of descriptor follow" marker
consistent with the SYS4/SYS5 family.

### Section layout (CONFIRMED — 0 ordering violations, 481/481)

```
body[0        .. F8)    CODE          bytecode instruction stream
body[F8       .. F10)   TABLE-1  (F7 entries, 1 dword each)  -> targets of type 0x71
body[F10      .. F12)   TABLE-2  (F9 entries, 1 dword each)  -> targets of type 0x03
body[F12      .. EOF)   TABLE-3  (F11 entries, 1 dword each) -> targets of type 0x8F
```

Every table entry is exactly **1 dword** — a pointer (dword index into body).
Solved algebraically across the whole corpus: `(F10-F8)/F7 == (F12-F10)/F9 ==
(EOF-F12)/F11 == 1` with zero non-integer results.

### The three tables are typed pointer indexes (CONFIRMED)

Each table points at body locations, and the dword *at* every target is a constant
tag identifying the pointed-to construct:

| Table | count/off | Target dword tag | Hits | Meaning (inferred) |
|---|---|---|---|---|
| T1 | F7 / F8  | **0x71** | 26,445/26,445 | labels / call targets (operand at +2 is small: mostly 1) |
| T2 | F9 / F10 | **0x03** | 3,018/3,018   | data/variable entries (operand at +2 large, e.g. addresses) |
| T3 | F11 / F12| **0x8F** | 72,941/72,941 | instruction/line entries (largest table; operand at +2 huge) |

100% type purity — not a single target had a different tag. T3 is the big one
(~73k entries corpus-wide), consistent with it being a per-instruction or
per-source-line index (a debug/line table). T1 ≈ labels, T2 ≈ a smaller symbol set.

## Instruction stream — PARTIAL

The code section is a flat dword stream. Recurring "type/opcode" dwords observed:
`0x03, 0x55, 0x6E, 0x6F, 0x71, 0x72, 0x8F`. These read as **operand-type tags**
in a tagged-operand VM rather than raw opcodes, e.g. the repeating shape:

```
... <TAG> <value> ...        tag 0x02 => string pointer (see below)
... 0x71 0x00 0x01 0x55 ...  label marker + following instruction
```

- **First body dword is 0x259 (601) in 301/481 files** — likely a standard
  "script entry" / prologue opcode. Second-most-common openers are small ints.
- `0x55` appears pervasively as an instruction lead — probably the most common
  opcode (statement / expression separator).

*Full opcode semantics need the VM dispatch loop in `AGE.EXE` — that's the Ghidra
task. These tags give a head start on labeling the disassembly.*

## Strings — CONFIRMED

- Stored inline in the body as **byte-complement (XOR 0xFF) cp932 / Shift-JIS**,
  packed 4 bytes per dword, NUL-terminated (a `\0` byte, i.e. `0xFF` after XOR,
  ends the string), then padded to the next dword.
- **Referenced by a tagged operand: the dword `0x02` immediately followed by the
  dword-offset of the string.** Confirmed directly by xref:
  - `MENU.BIN`: `...2 30b...` @0x2A9 → offset 0x30B = `"ＭＳ 明朝"`; `...2 30e...` → `"loadmesskip menu"`
  - `SC0030.BIN`: `...2 ef80...` → `"▼G0030　２章マップ０２１クリア"`; `...2 ef89...` → `"「よし、素晴らしい成果だな」"`
- Decoder (validated — pulls clean Japanese dialogue):
  ```python
  raw = bytes(b ^ 0xFF for b in body[off*4:])   # until a 0x00 appears
  text = raw.split(b"\0")[0].decode("cp932")
  ```
- Scene scripts hold the full dialogue; e.g. `SC0030.BIN` decodes to readable
  story text, choice-branch labels ("本来の分岐", "チェック用"), font names
  ("ＭＳ 明朝"), and engine directives ("loadmesskip advset", "ＡＤＶパート").
- Non-scene scripts (`MENU`, `ADDEXP`) contain only a handful of control strings —
  consistent with the inventory's subsystem/data-table categorization.

## Patch-override caveat (re-confirmed)

52 loose `.BIN` in the game root shadow their DATA1 copies at runtime and differ
slightly in size. The disassembler should target the **root** copies where present.
The header format is identical (same magic/layout) so tooling is copy-agnostic.

## What's solid vs. what needs Ghidra

**Solid (byte-verified, build a loader now):**
- 60-byte header, all 13 fields, dword units, section boundaries
- 3 typed pointer tables (0x71 / 0x03 / 0x8F), 1 dword each, 100% pure
- String encoding (XOR-0xFF cp932) + reference mechanism (tag 0x02 + offset)

**Needs the VM (Ghidra on `AGE.EXE`):**
- Opcode dispatch — confirm tagged-operand model, enumerate opcodes
- Meaning of F0/F2/F3/F5 flag fields
- Exact operand grammar per instruction (how many dwords each opcode consumes)
- Semantics of T1/T2/T3 beyond "label/data/line" guesses

## Loader — DONE

`tools/sys4load.py` parses the header, splits the 4 sections, resolves the 3
tables, decodes inline strings, and emits an assembly-ish listing with strings
inlined at their `2 <off>` refs (opcodes not yet named — code is chunked by the
T3 line-index). Importable API (`load()` → `Sys4Script`) plus CLI:

```
sys4load.py <file.BIN>              full listing
sys4load.py <file.BIN> --summary    header + section sizes + table/string counts
sys4load.py <file.BIN> --strings    decoded string pool
sys4load.py <file.BIN> --json       machine-readable structure
sys4load.py <dir> --validate        re-check invariants across a folder
```

`--validate` over all 481 DATA1 scripts: **481 parsed clean, 0 failures, 0 impure
table tags** — the format spec above is fully machine-verified. This listing is the
artifact to diff against Ghidra output once the VM dispatch loop is mapped.

### Observations surfaced by the listing (leads for the VM work)
- String-display sites look like `<opcode> 0x02 <str-off>` — e.g. opcodes `0x1A7`
  and `0x1A5` immediately precede string refs in `MENU.BIN`. Candidate text/message ops.
- T3 entries are 3-dword records `[0x8F, 0x00, value]`; T1 labels are `[0x71, 0x00,
  value]`. T3 acts as a per-statement line index (editor metadata) — note scene
  scripts even carry editor annotation strings like `"LABEL"`, `"ループ開始"` (loop start).
- `0x55` is the most frequent code lead (likely statement/expr separator); `0x09`
  recurs as an operand-type prefix (register/var reference?).