feat(diag): aggregating + filtered trace sinks (histogram, op-filter, per-script)

The existing trace framework only had a flat text formatter, so every question
became 'dump millions of lines, then grep'. This session that cost a long wrong
detour. Add, all observe-only (parity preserved):

- HistogramTraceSink: execution counts per opcode AND per call-site (script:pc)
  with a sample operand. Dumped sorted after the run. This is what instantly
  showed the 493k headless 'sleep's are INPUTNAME.BIN:0x1c3 (a name-entry poll
  loop), not the opening.
- TraceSinkBase: tracks the frame stack -> attributes each step to its REAL
  script (nested call-script frames included) = the 'which script is this pc in?'
  answer a bare step trace can't give.
- TextTraceSink: op-filter (--trace-ops sleep,draw-texture,...) + script:pc tags.
- CompositeTraceSink: fan-out (text + histogram + Godot's call-script queue).
- OpcodeTable.ByLabel: mnemonic -> opcode for --trace-ops.
- CLI: --trace-histogram, --trace-ops, robust --trace-file (mkdir -p).
- Godot: --trace-histogram <file> profiles the REAL run (headless flow diverges:
  real run to page 1 is 562 steps / 0 sleeps vs headless 2M steps / 493k sleeps).
- Also: --sleep-scale <f> debug knob to slow the paced opening for inspection.

Engine 56/56 (4 new); sweep parity 284/13; Godot builds + dogfooded end-to-end.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
gamer147
2026-07-08 09:50:07 -04:00
parent 35d3e97b27
commit 9ccfc43959
10 changed files with 354 additions and 29 deletions

View File

@@ -0,0 +1,17 @@
namespace Age.Engine.Diagnostics;
/// <summary>Fans one event stream out to several sinks, so e.g. a live text trace and an aggregating
/// histogram (or Godot's call-script queue) can run together. TracingSteps is true if ANY child needs
/// steps, so the VM's cheap gate stays correct.</summary>
public sealed class CompositeTraceSink : ITraceSink
{
private readonly ITraceSink[] _sinks;
public CompositeTraceSink(params ITraceSink[] sinks) => _sinks = sinks;
public bool TracingSteps
{
get { foreach (var s in _sinks) if (s.TracingSteps) return true; return false; }
}
public void Emit(in TraceEvent e) { foreach (var s in _sinks) s.Emit(in e); }
}

View File

@@ -0,0 +1,72 @@
using System.Collections.Generic;
using System.IO;
using System.Linq;
using Age.Engine.Model;
namespace Age.Engine.Diagnostics;
/// <summary>Aggregating diagnostic sink: counts opcode executions overall and per call-site
/// (script, pc), keeping a sample first operand. Answers "how many times did op X run, and from
/// where?" directly — the question that, this session, took a 2.5M-line text dump + grep/awk to
/// answer (sleep ran 493,175× from one call-site). Observe-only → trace parity preserved.
///
/// Thread-safe: Emit runs on the VM thread; WriteReport/OpCount may be read from another thread
/// (e.g. the Godot main thread at scene end), so both take a lock.</summary>
public sealed class HistogramTraceSink : TraceSinkBase
{
public override bool TracingSteps => true; // needs every Step
private readonly object _lock = new();
private readonly Dictionary<int, long> _opCounts = new();
private readonly Dictionary<(string Script, int Pc, int Op), Site> _sites = new();
private long _totalSteps;
private sealed class Site { public long Count; public long SampleArg0; public bool HasArg; }
protected override void OnEvent(in TraceEvent e)
{
if (e.Kind != TraceEventKind.Step) return;
var args = e.Ins?.Args;
bool hasArg = args is { Count: > 0 };
long arg0 = hasArg ? args![0].Value : 0;
var key = (CurrentScript, e.Pc, e.Opcode);
lock (_lock)
{
_totalSteps++;
_opCounts[e.Opcode] = _opCounts.GetValueOrDefault(e.Opcode) + 1;
if (!_sites.TryGetValue(key, out var s)) { s = new Site(); _sites[key] = s; }
s.Count++;
if (hasArg) { s.SampleArg0 = arg0; s.HasArg = true; }
}
}
/// <summary>How many times the given opcode executed.</summary>
public long OpCount(int op) { lock (_lock) return _opCounts.GetValueOrDefault(op); }
/// <summary>Total instructions executed (all frames).</summary>
public long TotalSteps { get { lock (_lock) return _totalSteps; } }
/// <summary>Write a sorted report: the opcode histogram, then the hottest call-sites (each tagged
/// with the script it ran in and a sample first operand).</summary>
public void WriteReport(TextWriter w, OpcodeTable? table = null, int topSites = 50)
{
string Mn(int op) => table?.Label(op) is { Length: > 0 } l ? l : $"0x{op:x}";
lock (_lock)
{
w.WriteLine($"=== opcode execution histogram — {_totalSteps} steps, {_opCounts.Count} distinct ops ===");
w.WriteLine($"{"count",12} op mnemonic");
foreach (var kv in _opCounts.OrderByDescending(k => k.Value).ThenBy(k => k.Key))
w.WriteLine($"{kv.Value,12} 0x{kv.Key:x3} {Mn(kv.Key)}");
w.WriteLine($"=== hottest call-sites (top {topSites} of {_sites.Count}) ===");
w.WriteLine($"{"count",12} script:pc mnemonic sample-arg0");
foreach (var kv in _sites.OrderByDescending(k => k.Value.Count).ThenBy(k => k.Key.Script).Take(topSites))
{
var (script, pc, op) = kv.Key;
var s = kv.Value;
string arg = s.HasArg ? $" arg0={s.SampleArg0}" : "";
w.WriteLine($"{s.Count,12} {script}:0x{pc:x} {Mn(op)}{arg}");
}
w.Flush();
}
}
}

View File

@@ -1,23 +1,29 @@
using System.Collections.Generic;
using System.Linq;
using Age.Engine.Model;
namespace Age.Engine.Diagnostics;
/// <summary>The one built-in formatter: writes each event as a deterministic text line to a
/// TextWriter (Console.Out or a file). Indents by frame depth. If an OpcodeTable is supplied, Step
/// lines show the mnemonic; otherwise the raw opcode. Step lines only appear when includeSteps is set.</summary>
public sealed class TextTraceSink : ITraceSink
/// <summary>The built-in text formatter: writes each event as a deterministic line to a TextWriter
/// (Console.Out or a file), indented by frame depth. Step lines are tagged with the current script
/// (the "which script is this op in" answer) and, when <paramref name="opFilter"/> is set, only ops in
/// the filter are printed — so you can watch just sleep/draw-texture/wait-for-input in execution order
/// instead of a multi-million-line flood. If an OpcodeTable is supplied, Step lines show the mnemonic.</summary>
public sealed class TextTraceSink : TraceSinkBase
{
private readonly TextWriter _w;
private readonly OpcodeTable? _table;
private readonly bool _steps;
private readonly HashSet<int>? _opFilter; // null = all ops; else only these ops' Step/Stub lines
public TextTraceSink(TextWriter writer, OpcodeTable? table = null, bool includeSteps = false)
{ _w = writer; _table = table; _steps = includeSteps; }
public TextTraceSink(TextWriter writer, OpcodeTable? table = null, bool includeSteps = false,
HashSet<int>? opFilter = null)
{ _w = writer; _table = table; _opFilter = opFilter; _steps = includeSteps || opFilter != null; }
public bool TracingSteps => _steps;
public override bool TracingSteps => _steps;
public void Emit(in TraceEvent e)
protected override void OnEvent(in TraceEvent e)
{
string indent = new string(' ', Math.Max(0, e.Depth - 1) * 2);
string indent = new string(' ', System.Math.Max(0, e.Depth - 1) * 2);
switch (e.Kind)
{
case TraceEventKind.FrameEnter:
@@ -25,12 +31,14 @@ public sealed class TextTraceSink : ITraceSink
case TraceEventKind.FrameExit:
_w.WriteLine($"{indent}« {e.Name} ({e.Text})"); break;
case TraceEventKind.Step:
_w.WriteLine($"{indent} {e.Pc:x4} {Mnemonic(e.Opcode)} {Args(e.Ins)}"); break;
if (_opFilter != null && !_opFilter.Contains(e.Opcode)) break;
_w.WriteLine($"{indent} {CurrentScript}:{e.Pc:x4} {Mnemonic(e.Opcode)} {Args(e.Ins)}"); break;
case TraceEventKind.CallScript:
_w.WriteLine($"{indent} call-script 0x{e.Id:x} ={e.Name ?? "?"} " +
$"({(e.Name != null ? "resolved" : "stub/unresolved")})"); break;
case TraceEventKind.Stub:
_w.WriteLine($"{indent} {e.Pc:x4} STUB op=0x{e.Opcode:x}"); break;
if (_opFilter != null && !_opFilter.Contains(e.Opcode)) break;
_w.WriteLine($"{indent} {CurrentScript}:{e.Pc:x4} STUB op=0x{e.Opcode:x}"); break;
case TraceEventKind.Halt:
_w.WriteLine($"halt: {e.Text} @ {e.Steps} steps"); break;
}

View File

@@ -0,0 +1,28 @@
using System.Collections.Generic;
namespace Age.Engine.Diagnostics;
/// <summary>Base for sinks that need to know the currently-executing script. Tracks the frame stack
/// from FrameEnter/FrameExit (which emit regardless of the TracingSteps gate) so a subclass can key
/// per-instruction facts by the REAL running script — including inside nested call-script frames.
/// This is the "which script is offset 0x1c3 actually in?" fix: a bare Step event only carries pc +
/// depth, not the script name.</summary>
public abstract class TraceSinkBase : ITraceSink
{
private readonly Stack<string> _frames = new();
/// <summary>Name of the innermost frame currently executing ("?" before the first FrameEnter).</summary>
protected string CurrentScript => _frames.Count > 0 ? _frames.Peek() : "?";
public abstract bool TracingSteps { get; }
public void Emit(in TraceEvent e)
{
// Push before dispatch so the FrameEnter itself is attributed to the entered frame; pop after
// dispatch so the FrameExit is still attributed to the exiting frame.
if (e.Kind == TraceEventKind.FrameEnter) _frames.Push(e.Name ?? "?");
OnEvent(e);
if (e.Kind == TraceEventKind.FrameExit && _frames.Count > 0) _frames.Pop();
}
protected abstract void OnEvent(in TraceEvent e);
}

View File

@@ -11,4 +11,12 @@ public sealed class OpcodeTable
}
public string Label(int op) => _t.TryGetValue(op, out var e) ? e.Label : "";
public int Argc(int op) => _t.TryGetValue(op, out var e) ? e.Argc : -1;
/// <summary>Reverse lookup: opcode by mnemonic label (e.g. "sleep" -> 0xc8), or null if none.
/// Used by --trace-ops to let a diagnostic filter name ops by mnemonic instead of raw hex.</summary>
public int? ByLabel(string label)
{
foreach (var kv in _t) if (kv.Value.Label == label) return kv.Key;
return null;
}
}