Test the checkpoint you would actually deploy

AI Bomber retained an iteration-65 champion while training had reached iteration 110. The training canary measured the latest network. That meant a canary result could describe a different artifact from the one available for deployment.

A champion is a selected checkpoint kept until another candidate meets the replacement rule. Latest is simply the most recent training output. Both are useful identities, but they answer different questions. The bounded audit asked what the saved champion could do on the same tactical diagnostics used to inspect the latest network, without restarting training or spending a new strength holdout.

The audit used six constructed tactical scenarios. Each isolates a behavior such as escaping after planting a bomb, clearing a corridor, chasing, timing movement around flames, breaking a stall, or trapping an opponent. A gate records whether the agent completes that task within its step budget.

Both checkpoints ran against the same native executable and the same diagnostic seeds. The search mode used 96 simulations. Raw-policy and search outcomes were recorded separately, because a search procedure can change what a network's preferences produce in play.

The champion passed four of the six gates and the latest checkpoint passed three, in both raw and search modes. Stall-breaking was the difference. Both passed bomb-and-escape, corridor clearing, and trapping. Both failed chase and flame timing. The extra champion pass is useful local evidence, but it is not a full-game win rate.

It would be tempting to turn the result into a story that more training made the agent worse. The artifacts do not support that causal conclusion. Their recorded training mixtures differ, including floor and mirror fractions. They are checkpoints from a changing process, not a controlled experiment that varied training duration alone.

The diagnostic seeds also differ from the historical training canary's seeds. This audit did not reproduce that old canary exactly. It placed both saved artifacts under a common current diagnostic, which is a narrower and still useful operation.

The practical finding is about attribution. Reporting only the latest network's gate score would have understated the retained champion's performance on this battery. Reporting only the champion's score would have hidden a weakness in the latest candidate. Keeping both identities makes the divergence visible without inventing an explanation for it.

The packaged champion replay lasts 121 frames and ends in an arena-crush win, with no bomb kill. The opponent is a heuristic agent that was part of the training setup. The replay is a real example of the saved champion playing, but it is not evidence of combat mastery against an unseen opponent.

That terminal cause matters. A binary win label combines different ways a match can end. If the research question concerns active combat skill, a win produced by the shrinking arena needs a different interpretation from a successful bombing sequence. The outcome should stay connected to the mechanism visible in the artifact.

The native viewer loaded this replay and a retained screenshot shows a specific frame. Separate deterministic resimulation was not performed for the Bomber bundle. Playback evidence and simulation-verification evidence remain distinct, just as a tactical gate and a match outcome remain distinct.

The evidence bundle preserves checkpoint hashes, native binary identity, search settings, seeds, and scenario outcomes. Those fields make it possible to tell whether a later score belongs to the same model under the same diagnostic conditions. A filename such as latest is insufficient because its contents can change while its label remains familiar.

The audit did not overturn the decision to stop the training campaign, and it did not establish a new strength result. It closed an instrumentation gap: the deployable champion now has its own gate evidence beside the latest candidate. That is enough to improve the next decision without pretending the diagnostics justify a new campaign.

I would apply the same rule to a production model registry. Evaluation should name the artifact and its role at the time of the run. A candidate improving during training, a champion selected by promotion, and the model serving users may all differ. Before asking whether the score went up, establish whose score it is.