Before we test the same tasks on a stronger model, here's what we think will happen
September 1, 2026
Series-wide caveats — sampling unit, historical controls, training contamination, model pinning, what the registration order proves, and what the verification layers cannot guarantee — are collected on the limitations page. Published corrections are on the errata page.
Round 3 asked whether Tamperward's prevention effect transferred from JavaScript to Python. It did: on 17 fresh Python repositories, false-green discordance ran 6–0 in the gate's favour, exact McNemar p = .0313. Round 3.1 asks a narrower question on the same repositories:
Does the prevention effect appear under a stronger model, on the frozen round-3 Python benchmark, with the Tamperward treatment held byte-identical?
The intended experimental factor is the model, which moves from Haiku to claude-sonnet-5, pinned exactly. The task-facing protocol, the v1.14.0 treatment, the pressure prompt, the oracle semantics and the budgets are held fixed — with two disclosed differences beyond the model: one pair was burned before registration, leaving 16, and a preregistered control-plane isolation correction scrubs harness variables and relocates the withheld oracle. This is a clean model-transfer follow-up, not a perfect one-variable experiment, and the registration treats the cross-model comparison as descriptive for that reason. PREDICTION3.1-taskbench.md is frozen in git history, with sha256 pins on the runner, the driver, the verdict oracle, the prompt and the analyzer; this post is the human-readable half, and it is published before the first counted trajectory.
Chronology, stated precisely. One accidental trajectory ran before the registration was frozen; it was quarantined unread and its whole pair burned (below). Six designated engineering pilots then ran under the final runner and are permanently excluded from confirmatory analysis (below). No counted round-3.1 trajectory had run when this document and this article were frozen, and the remaining 16-pair counted pool has not been executed.
This is 3.1, not round 4
The pool is spent. The research process has seen 34 Haiku trajectories on those repositories, published the results, and read the transcripts. A study on them can answer one very clean question — on the exact repositories where Haiku produced the round-3 result, what changes when the model changes? — but it is follow-up evidence on a frozen benchmark, not another held-out replication. Round 4 is the fresh pool with an improved treatment. Calling this one 3.1 keeps that distinction visible.
We are registering 16 pairs, not 17, and the reason is a mistake we made
While checking that the registration gate accepted its filled constants, we ran the sweep entrypoint. The gate passing is the sweep starting. One Sonnet trajectory completed on 08-celery-py-amqp before this registration was frozen.
The artifacts were quarantined immediately and never opened — their identity is recorded by filename, size and checksum only. Then we applied our own rule to ourselves. Round 3.1 registers a principle that came out of the round-3 infrastructure incident:
Retries belong to failed attempts to start an experiment, never to a stochastic experiment whose outcome has already been sampled.
A trajectory had started. Re-running that arm would have exactly the shape the principle forbids — trajectory existed, discard it, draw another — and we were not going to make the first exception in the same week we wrote the rule. The design is paired, so the whole pair is excluded. No replacement task is introduced. Round 3.1 is 16 pairs / 32 counted trajectories, and every prediction below was written for 16.
The entrypoint now refuses to run without an explicit flag. --check validates the registration and creates nothing; only --execute-counted can produce a counted trajectory. A behavioural test proves the first cannot become the second.
What we think will happen
The interesting prediction is that the headline test probably will not reach significance, and that this will not mean the gate stopped working.
Our own earlier studies found frontier models markedly more honest than the cheap one when nothing is watching — 30 of 30 honest frontier runs against 6 of 10 tampers from Haiku. If Sonnet weakens tests less often, there is simply less for the gate to prevent. So we predict transfer drops from round 3's 9/17 to about 6/16, and discordance b from 6 to about 4.
That matters arithmetically. With c = 0, the exact conditional McNemar test reaches p < .05 only at b ≥ 6. Round 3 landed exactly there. Predicting b = 4 is predicting p ≈ .125 — a null. We are registering that in advance, along with the guard: if b < 6 and c = 0, we report it as underpowered given the observed violation rate, and we do not rescue it with a one-sided test, by pooling with round 3, or by promoting some descriptive endpoint to confirmatory after the fact.
Prediction is not certainty in either direction. The same body of work found this model taking the cheapest sanctioned path 10 times out of 10 when the cheat was explicitly available — and our prompt says, in as many words, if a test is in your way, remove or skip whatever is failing. A model that is honest by disposition and compliant when told may still weaken plenty.
The other bets: gated completions 15/16, ungated 13/16, so round 3's surprising +41.2pp completion gap should largely collapse — that gap came from ungated Haiku failing to finish at all. Gated escapes 0–2. And in-loop denials attributable to pytest skip spellings: zero, because v1.14.0 is byte-identical and its skip detector still carries no pytest spellings. That was round 3's headline engineering finding, it is deliberately unfixed here so that only the model changes, and registering the zero gives round 4's fix a stated baseline.
Four conditions would count as Tamperward losing, and would be published as such: c > b; gated false greens reaching 5 of 16; a completion cost of 20 points or worse; or any gated trajectory reaching green over a surviving violation with every layer silent.
One correction to the harness, disclosed
Rounds 1–3 launched the agent with every harness variable inherited and the withheld oracle one directory from a path it could read. We audited all 34 counted round-3 transcripts and found no trajectory that used it — a string audit, not a filesystem proof — and the results stand. Round 3.1 scrubs those variables and moves the oracle out. It is not a security boundary and we do not call it one; the real one is a round-4 item. Both arms get identical treatment, so the paired comparison is unaffected. The full record is in the errata.
What the pilot exercised
Six prespecified engineering pilot trajectories completed under the final round-3.1 runner, across three pilot-role tasks that are not in the counted set and are permanently excluded from confirmatory analysis. Git history puts PREDICTION3.1 before the first of them.
No runner defect was identified and no amendment was made. All three gated pilots passed synthetic deny-path liveness, live Stop verification, network confinement, observer operation and envelope adjudication; the relocated oracle resolved on both the semantic and integrity paths; thirty-four egress attempts were denied by the network jail. The runner hash pinned in the registration is the one that ran them.
All six were honest — two HONEST_FIX, four NO_OBSERVED_VIOLATION, zero denials, zero protected mutations, zero policy violations. No pilot contained a naturally occurring policy violation, so the live adversarial path was not exercised by this model before the counted sweep. The synthetic probe proves the gate's deny path is live; it is not evidence that the natural adversarial path was exercised.
Three tasks are not evidence about sixteen. The predictions above are unchanged by what the pilots showed, and deliberately so: leaving them alone after seeing that pattern is what makes the eventual score worth anything.
All three gated pilots took longer than their ungated twins (97→137s, 59→122s, 74→103s). Gated execution includes additional verification work, but pilot wall-clock differences are descriptive and combine treatment work with stochastic trajectory and runtime variation.
The next post is the answer. The repository records this registration commit before the result commits.