AI Bomber

AI Bomber began as a deterministic Bomberman-style environment and grew into an AlphaZero self-play project. Its most consequential result was a retraction. A checkpoint passed an agent-ladder gate, but an outcome audit found that almost all of its wins came from the closing arena crushing the opponent. The score established survival under those rules, while the published claim implied broader combat ability.

That distinction changed the evaluation. Later runs used outcome attribution, tactical probes, and explicit continue-or-stop criteria. A staged rebuild made progress through an opponent ladder, then stopped when self-play failed its consistency requirement. The latest audit asks a further question: do those tactical measurements describe the saved champion that will actually be used, or only the latest network produced by training?

Bomb-based play depends on delayed consequences. A tile that is safe now may be lethal when the agent arrives, and a policy needs to reason about movement, blast propagation, escape routes, and the opponent together. If simulation state leaks between agents or danger checks use the wrong time, training can optimize against behavior the evaluator did not intend.

I built the original environment around bounded state, seeded randomness, explicit rules, and inspectable rewards. The base implementation supported rule-based agents before a neural trainer existed. That ordering made it possible to test game behavior on its own and later attach search and learning through the same environment boundary.

The C17 core owns configuration, seeded state, replay storage, and episode metrics. Environment modules implement map generation, movement, bombs and blasts, danger predictions, observations, rewards, and termination. Agents have a shared function-pointer interface with their own storage. Headless runners, benchmarks, evaluators, and the raylib viewer consume the same observation-action-step loop.

Fixed-size state avoids heap pointers in the saved representation and makes copying and serialization straightforward. It also imposes real limits, including a 31x31 maximum grid and fixed bounds on agents and bombs. The visualizer is optional at build time, so headless rollouts do not need graphics dependencies. Native C++/LibTorch training was added above this core rather than replacing the game rules.

The early hardening pass repaired a concrete isolation bug: heuristic and greedy agents used file-static implementation state. Two instances could share random state and interfere with each other. Moving that storage into each agent made independent instances possible and turned the isolation requirement into a tested contract.

Danger checking similarly needed the time of arrival, rather than only the current tile state. A named arrival-time check uses the predicted time to a blast to judge a move. Reward accounting keeps survival, crate destruction, damage, elimination, escape, and penalties separately visible before summing them. That decomposition is useful when an agent’s preferred behavior and the intended task diverge.

The base simulator has 12 CTest suites covering rules, determinism, replay, rewards, agents, and danger, with GCC, Clang, and MSVC in the configured CI matrix. Its reported headless throughput of about 234,000 steps per second comes from a historical commit-message benchmark, not a new measurement in this case study. The original results figure below describes that foundation, not the later learner’s strength.

The first native AlphaZero run completed 100 iterations using a 128-channel, 10-block residual network with 3,250,839 parameters. The retained checkpoint came from iteration 70. Validation improved early, regressed, and recovered to a modestly higher plateau. The run did not show the sharp late transition its grokking hypothesis called for.

Early evaluation was also contaminated by the bootstrap heuristic used in search. Scores from iterations 5 through 20 were heuristic-aided and cannot be read as clean neural-policy results. The later holdout evaluation used the retained checkpoint on seeds excluded from self-play and selection. Keeping that distinction matters because a learning curve can mix different sources of decision quality even when its label stays the same.

On the original untouched holdout, the iteration-70 checkpoint recorded 128 wins against random, 41 wins and 87 draws against the native heuristic, and 2 wins and 14 draws against native MCTS, with no losses in those sets. The corresponding heuristic and MCTS scores were 66.02% and 56.25%, counting a draw as half a win. The strong draw component is part of the result, not something a percentage should hide.

The opponent budget also bounds the conclusion. A later diagnostic against a strengthened MCTS opponent with 512 simulations and depth 16 produced 16 draws and no wins. That used a different diagnostic set, so it is not a paired estimate of the budget’s effect. It does establish that the earlier holdout score was insufficient evidence of superiority to the stronger search baseline. A comparison with an earlier NumPy model likewise used different seeds and search budgets and cannot support a clean improvement claim.

Continuation runs eventually produced a v5 checkpoint described as superhuman on the project’s agent ladder. The next audit found that 95 to 97% of its wins were arena-crush deaths. With sudden death disabled, a control collapsed to near-all draws. The reward assigned the same full value to a crush win and a bomb-kill, while a win-draw-loss gate could not distinguish the two. The claim was retracted and the original document retains that correction.

A bounded replay-reweighting intervention then produced 1 bomb-kill win in 64 games against the control’s 0 in 64, below its frozen success bar, and was stopped. The v7 restart used search and target repairs, teacher imitation, a gated opponent ladder, and guarded self-play. The ladder reached MCTS at 256 simulations, but the following 40-iteration self-play tranche failed tactical consistency: only 3 of 8 probes met the required gate count, against 6 of 8 required.

The smoothed top-rung win-rate criterion held at all 8 probes, which did not mean every raw probe was a winning result. No champion was promoted during that tranche. The July stop was honored, leaving the iteration-65 ladder champion in place. The experiments below retain the intervention and staged-training details without presenting the repeatedly observed ladder as an unseen-opponent strength test.

The training canary had been measuring the latest network, while the deployable artifact remained the saved champion. The September audit ran both through the same six standalone tactical scenarios, serially, with one native binary and 96-simulation search. It bound every result to checkpoint identity. The standalone seeds differ from the historical canary, so this is a new diagnostic comparison rather than a rerun of July.

The iteration-65 champion passed four scenarios and the iteration-110 checkpoint passed three. Both passed bomb escape, corridor clearing, and trapping, and both failed chase and flame timing. Stall-breaking separated them: the champion passed while the latest exhausted the step budget. Their training histories differ, so this is not a causal estimate of self-play’s effect or a full-game win rate.

The accompanying replay is equally specific. On the first predetermined diagnostic seed, the champion moved and placed bombs, then won at step 121 when the arena crushed the heuristic opponent. There was no bomb-kill. Native saved-state playback and the screenshot were checked, but a separate deterministic resimulation was not performed. The artifact makes the behavior inspectable without turning one illustrative win into combat mastery.

The project now has a reproducible simulation, a native learning pipeline, retained negative interventions, and checkpoint-specific tactical evidence. Those are useful systems results alongside an unfinished strength claim. The saved champion is not interchangeable with the last training output, a diagnostic gate is not a held-out win rate, and winning by the environment’s closing wall does not demonstrate every skill the game requires.

No training or reserved holdout was consumed for the September audit, and the July stop remains in effect. A further strength claim would need its own declared evaluation, opponent identities, win-cause accounting, and the required controls. Checkpoint-backed human play and full-simulator PPO are separate unfinished work. The immediate achievement is that the project can now attach a narrower, checkable meaning to each reported result.