Expectrail: Interactive Demo and Audit Vault

Select Seeded Defect or Rejection Exploit Verified Accept

Examine how SAM 3.1 detected the defect, how Muse Code patched it, and how the Verifier guarded gameplay.

Illustration, not a capture: the animation, its overlay values and the case texts are drawn and written for this page from the recorded results. Only physics-fall-through / v1 shows Muse Code's recorded fix and reasoning. Measured numbers, each with its run, are in the Benchmark Matrix tab. Real frames: a recorded live review of Muse Code's fix, before and after, and the review that rejected the fix drawing the robot inside the ground.

Animated illustration (not a capture)

Simulation, not a capture

Clean Level Floor (y=640)

Player

blue robot (0.94)

REPLAY COORDINATES

x: 90.0, y: 636.6

Status: Normal

Before Fix (Defect Active)

Ready

Test Specification specs/player-stays-above-ground.json

"The player never falls through the ground."

ORACLE TYPE

never-crosses

SAM 3.1 PROMPT OBJECT

"blue robot"

Verifier Decision ACCEPT

1. regression (Visual Oracle) PASS

2. gameplay (the robot moves and stays alive) PASS

3. clean-control (No Regressions) PASS

4. held-out (Unseen Routes) PASS

all 4 checks passed; candidate verified under clean level reference.

candidate.patch: Muse Code's recorded fix policy: clean-v1

--- a/player/player.tscn

+++ b/player/player.tscn

@@ -187,7 +187,7 @@

[node name="Player" type="CharacterBody2D"]

-collision_mask = 14

+collision_mask = 30

floor_constant_speed = true

Muse Code's reasoning, from its recorded log

Root cause: the ground lives on physics layer 5 (value 16 in tileset.tres), but the player's mask was 14 (layers 2+3+4: enemies, coins, platforms), so the ground was invisible to it and it fell straight through from the first frames. Fix: in player.tscn, collision_mask = 14 → 30, adding the ground layer while preserving all existing bits.

Full Gate 2 Benchmark Matrix

Expectrail (Deterministic Verifier + SAM 3.1) vs. Muse Code with Raw Screenshots Baseline.

Sources: fixes and minutes from the development fix matrix (bench/fix-matrix, runs 40916077549, 40927304209, 40927316337); caught columns from the detection bench (bench/detection, run 40841977642); frame counts from docs/sam.md. Held-out set, first run: Expectrail fixed 8 of 8, screenshots 7 of 8 (run 41099054641).

Expectrail: 8 of 8 Fixed

SAM 3.1: 8 of 8 Caught (vs 3/8 without)

False Alarms: 0

Case ID	Defect Mechanism	Caught (No SAM)	Caught (With SAM 3.1)	Expectrail Fix	Screenshots Fix
physics-fall-through / v1	Player collision mask drops ground (14)	YES (position y)	YES	FIXED (12m)	FIXED (10m)
physics-fall-through / v2	Level tileset drops collision layer (0)	YES (position y)	YES	FIXED (13m)	FIXED (15m)
missing-texture / v1	Coin texture file UID stripped	NO (coords valid)	YES (no coins seen)	FIXED (17m)	FIXED (9m)
missing-texture / v2	Coin sprite texture property cleared	NO (coords valid)	YES (no coins seen)	FIXED (20m)	FIXED (9m)
layering / v1	Player z_index drops behind level (-3)	NO (coords valid)	YES (robot seen in 17 of 31)	FIXED (16m)	FIXED (18m)
layering / v2	Player Sprite2D z_index set to -6	NO (coords valid)	YES (occluded)	FIXED (10m)	FIXED (23m)
sinking / v1	Collision shape sits 26px too high	YES (ground line)	YES	FIXED (36m)	FIXED (22m)
sinking / v2	Sprite drawn 26px too low (physics ok)	NO (physics intact)	YES (hybrid check)	FIXED (23m)	FIXED (23m)

Why Visual Grounding (SAM 3.1) Beats Traditional QA

Without computer vision, 5 of the 8 bug variants pass conventional build checks completely undetected: missing textures, player layering behind tiles, and visual sprite offsets leave the physics coordinates valid. Meta's vision models (SAM 3.1, or the open-weight Muse Glimmer checked against the engine, also 8 of 8) see what the player actually sees on screen.

Select Seeded Defect or Rejection Exploit Verified Accept

Full Gate 2 Benchmark Matrix

Trust Architecture: "Deny-by-Absence" & Canary Probes

Mount Boundary Topology

Honeytoken Canary Attempts

7:30 AM Morning Report (Human Hand-Off)

# Expectrail morning report

## Checks

## Gameplay Metrics

## Verified Candidate Diff

## Unresolved for a Human