The experiment, and the ways it is allowed to fail.

What has been measured

Breadth sweep — 13 task families × 5 seeds × 2 episodes, on MiniWoB++ — a public benchmark of small synthetic web tasks. One world model per task-and-seed pair, shared across its two episodes only, so any second-episode improvement must come from the first episode's verified evidence.

MINIWOB-RESULTS.md @ track-b-v0.3-eval-freeze

• Both failures are one task family at one seed, in both episodes — the planner refusing to plan, not a wrong answer.
• The run summary under-counted actions at 212; 218 is the journal recount, and the gap is logged as a defect.
• In two families, part of episode one reuses evidence between similar fields. Logged as a defect, not counted as learning.
• These are small synthetic tasks. They show the loop runs end to end — not that it works in production software.

The question

Given the same model, observations, budgets and prior experience — does an agent with an actively learned, inspectable transition model complete unseen, stateful workflows more reliably and safely, with fewer actions and faster drift recovery, than reactive, retrieval and memory baselines?

The pre-registration pins seven hypotheses with a conjunctive pass rule: every applicable criterion must pass, so no single result can rescue another. The harness refuses to run a comparison whose resource contracts differ.

How this idea can die

The protocol names, in advance, the observations that would mean the hypothesis failed. Any one is sufficient.

1. 01

   Documentation or trajectory memory comes within five success points of the causal condition.

2. 02

   Gains disappear under equal compute and interaction budgets.

3. 03

   Multi-step prediction errors compound enough to make planning ineffective.

4. 04

   Performance requires privileged backend information.

5. 05

   Transfer does not reduce discovery interactions on unseen rule graphs.

6. 06

   Results hold only on cosmetic navigation tasks.

Not claimed yet

• That the hypothesis is true — the pre-registered confirmatory experiment has not run.
• Superiority over any frontier computer-use agent, or reliability in arbitrary production software.
• A causal graph. The learner holds an interventional transition model, and the implementation refuses the stronger word.
• That the confirmatory suite could run today — it is bound to an earlier freeze and is not authorised against the current candidate.
• Any positive verdict without independent audit. Self-attested all-pass evidence is defined as inconclusive.

These are internal development results in controlled environments. They do not establish superiority over frontier computer-use agents or reliability in arbitrary production software. External evaluations and design-partner pilots are next.

Provenance

Atila Vahedian is the sole author of every commit reachable fromv0.1.1-rc3. The protocol is the first commit; the implementation follows it. This is a one-person research project — no team, funding, customer or deployment is implied.

• MINIWOB-RESULTS.md

  The measured breadth sweep: every figure in the results table above.

• REVIEW-WORLD-MODEL.md

  Adversarial review of the world model: the unverified-outcome invariant, drift detection and recovery.

• PROTOCOL.md

  Pre-registration: hypotheses, conditions, resource equality, kill criteria, statistics.

• RESULTS.md

  Current evidence state and the “not yet run” verdict.

• artifacts/manifests/confirmatory-suite-preservation-v0.1.1-rc3.json

  That confirmatory execution against the current candidate is unauthorised until an explicit rebind.

• Git commit metadata

  Authorship and dates — the only claim sourced to history rather than file contents.