AI Bomber

AI Bomber is a Bomberman-style simulation environment built to be trustworthy before anything gets trained on it. The first six sections cover the repository as of commit c50f423 (2026-07-06); the AlphaZero sections after them extend the story through the self-play campaign of July 7 to 14, 2026. At the base commit there is no neural network anywhere in the repository, and the README said so directly: "a local research sandbox, not a trained neural agent." What exists instead is a deterministic C17 environment with fixed-size state, per-instance agent RNG, arrival-time danger checking, a binary replay format, 12 CTest suites, CI on GCC, Clang, and MSVC with -Werror, and 8 design docs covering every subsystem.

The headline numbers are throughput and correctness, not a win rate. The headless simulator runs at roughly 234,000 steps per second and produces bit-identical episodes given the same seed and action sequence. Build and verify the environment first, that is the ordering principle: a fast, deterministic, well-tested simulator is what makes any later training result trustworthy, and skipping that step is why RL projects end up debugging their environment and their agent at the same time.

Reinforcement learning demos usually start from the model side: pick an algorithm, wire it to whatever environment is fastest to stand up, and start training. The environment itself rarely gets the same engineering attention as the model, which means bugs like shared static state between agent instances, or a "safe" check that ignores a bomb about to detonate by the time the agent arrives, silently corrupt every experiment run on top of them.

AI Bomber treats the environment as the deliverable. It asks a narrower question first: can this simulator reproduce the same result twice, run fast enough to be useful for training, and expose its internals cleanly enough for a Python or C++ policy to be dropped in without touching the core.

The project is a simulator-first C17 implementation of a four-agent Bomberman variant: movement, bomb placement, blast propagation, crate destruction, and powerups, with a raylib visualizer layered on top for debugging and a headless CLI for training-scale rollouts. Five modules split the work cleanly: core/ (a SplitMix64 PRNG with no global state, config bounds, a ring buffer for action history, binary replay save/load, and episode metrics), env/ (map generation, bomb and blast logic, the danger map, observation and reward builders, and the movement/placement/terminal rules), agents/ (a function-pointer interface with per-instance storage and four built-in policies: random, scripted, heuristic, greedy-crate), sim/ (episode runner, benchmark, and an agent-vs-agent evaluator), and viz/ (the raylib renderer, dashboard, and session manager).

The build separates concerns explicitly: -DAI_BOMBER_BUILD_VIZ=OFF compiles the environment, agents, and CLIs without linking raylib at all, so a training pipeline never pays for graphics dependencies it does not use. Correctness is enforced with CTest from the start rather than bolted on afterward, and CI runs the full matrix (GCC and Clang with -Werror on Linux, MSVC on Windows) on every push and pull request. The data flow is a straight loop: env_init and env_reset set up state from the config and seed, then each tick calls env_observe, hands the observation to whichever agent is attached, calls env_step with the returned action, updates metrics, and optionally records the tick to a replay buffer. The headless CLI, the benchmark CLI, and the visualizer are three different consumers of that same loop, not three different implementations of it.

A few choices are deliberate, not incidental. Fixed-size state with no heap pointers (bounded arrays capped at a 31x31 tile grid, with limits on agent and bomb counts) means saving and loading a replay is a straightforward binary serialization of the struct, with no allocator-related nondeterminism to chase down.

Per-instance agent state was a real bug, not a hypothetical one. Commit c50f423 ("Harden simulator") replaced file-static implementation structs in the heuristic and greedy agents with per-instance storage. Before that fix, two heuristic agents running side by side shared the same static RNG state and interfered with each other. ARCHITECTURE.md now calls this out explicitly as a determinism rule, not just a style preference.

Arrival-time danger checking replaces the naive "is this tile safe right now" question with the one an agent moving toward a bomb actually needs answered: danger_is_action_safe_at_arrival checks whether the blast reaches a tile after the agent would arrive there, using a per-tile time-to-blast field computed from every active bomb. Reward is a struct of eight-plus named components (survival, crate destruction, powerup pickup, enemy damage and elimination, win and death, escape from danger, trap opportunity, and penalty terms) rather than one scalar, which is slower to read but far faster to debug when an agent's behavior does not match expectations.

What is deliberately not here: FUTURE_MODELS.md lays out five planned model-integration paths (behavior cloning, DQN, PPO, a genetic algorithm over network weights, and evolution strategies) through the same C ABI, none of them implemented at this stage. The README states plainly that the project is "a local research sandbox, not a trained neural agent," a scope boundary stated up front rather than a gap discovered later.

Verified against a local checkout at commit c50f423 ("Harden simulator"), where the documentation, test suite, and CI configuration all match exactly. Headless simulator throughput: about 234,000 steps per second, single core, no visualizer linked, quoted directly from a later commit message reporting a passing test run at that throughput.

The test suite has 12 CTest suites in tests/, covering rules, determinism, replay round-trips, reward accounting, agent behavior, danger-map correctness, and the c50f423 hardening fixes specifically. An earlier assessment cited 14 test files. This write-up reports the verified count of 12 instead. Documentation totals 8 design docs in docs/, and the source is 59 C source and header files, 4,296 lines and 146 KB, across src/. CI runs two jobs on every push and pull request: GCC and Clang on Ubuntu with -Werror, and MSVC on Windows, both running the full CTest suite.

The day after the hardening commit, the project got a native C++/LibTorch AlphaZero trainer, and the first full run (alphazero-native-grokking-v1) completed all 100 iterations overnight on July 8. The trained model is a 128-channel, 10-block residual tower with 3,250,839 parameters. Checkpoint selection promoted at iterations 5, 20, and 70, and iteration 70 became the final checkpoint. A full multi-agent code audit of the run found no result-invalidating defect, with one honest asterisk: early-iteration evaluation search leaked the bootstrap heuristic, so the iteration 5 through 20 scores are heuristic-aided rather than clean.

The validation trajectory is reported the way the run's own documentation reports it: validation improved early, regressed, and recovered to a modestly higher plateau, with no sharp late transition. By the documented standard this is not evidence of grokking. It is noisy ordinary learning with a persistent late improvement, and the write-up says so flatly.

On an untouched holdout (seed base 1,500,001, never used for self-play or checkpoint selection), the iteration-70 checkpoint went 128-0-0 against the random agent, 41-87-0 against the native heuristic (66.02 percent score), and 2-14-0 against native MCTS (56.25 percent), with zero losses across all three opponents. Against the same native MCTS opponent, the earlier dependency-light NumPy reference model scored 37.5 percent, though the two runs used different seed ranges and search budgets, so that comparison is directional rather than a paired statistical test. The run's documentation states that caveat itself.

Between those clean numbers and the current state sits the most instructive week of the project. Continuation runs (v2 through v6) led to a v5 checkpoint that passed a statistically careful-looking agent-ladder gate, and the result was published in the repo as a superhuman agent-ladder claim on July 9. An overnight audit on July 10 took it apart: 95 to 97 percent of the checkpoint's wins were arena-crush deaths, the closing arena wall killing the loser, not bomb kills, and a control run with sudden death disabled collapsed to near-all draws. The root cause was a reward function that paid the same +1.0 for a crush win as for a bomb kill. The claim was formally retracted in the same document that made it, and the retraction header is still there.

Two designed interventions followed and were reported just as flatly. Capped cause-balanced replay reweighting (KL-105) ran to iteration 154 against frozen criteria and was verdicted flat-to-negative and killed. An opponent-model ablation (KL-110) came back mixed: the mechanism is real, but bomb-prior starvation gates it off. A from-scratch v7 redesign followed, grounded in a Pommerman and AlphaZero-family literature pass, and executed as a staged pipeline with pre-registered gates: search and target repairs, a teacher imitation bootstrap that passed its exit gate, an algorithmic opponent ladder cleared rung by rung (the ladder scores and the WAIT-collapse numbers behind this whole week are in the two experiment write-ups below), then guarded self-play. The final tranche held top-rung strength on 8 of 8 probes but failed its canary criterion (3 of 8 against a required 6 of 8), and the pre-registered rule said stop. The run stopped on July 14, the write-up records zero champion promotions across the final tranche, and three proposed follow-ups sit in the doc, none launched. That is where the repository stands.

Fixed-size, bounded state buys determinism and cheap serialization, but it caps the board at 31x31 tiles and a fixed agent and bomb count. Scaling up means touching the constants and re-verifying every test that assumes them. Writing the core in C17 instead of Python buys raw throughput and a clean C ABI, at the cost of the conveniences the Python ML ecosystem takes for granted. Making the visualizer optional at build time keeps the headless path fast and CI simple, but the compiled visualizer was not exercised in this write-up, no display was available, so nothing about bomber_viz beyond its documented behavior was directly checked. At the base commit, shipping the environment without a trained model was the whole point, and the AlphaZero sections above show what that discipline bought: when a trained checkpoint did arrive, its numbers came from an untouched holdout on a simulator that had already been hardened, and the one inflated claim the campaign produced was caught by an audit and retracted within a day.

The clearest lesson is about sequencing: the project spent its early commits entirely on the environment (state representation, rules, danger computation, replay, CI, docs) before writing anything resembling a trained agent, and the payoff shows up directly in the hardening commit. The per-instance-state bug would have been invisible, or worse, silently biased, if it had first surfaced three weeks into a training run instead of being caught by a test written against the simulator itself. "Safe" is a time-dependent property in a bomb-based game, and building danger_is_action_safe_at_arrival as its own named, tested function made that distinction explicit and checkable, rather than leaving it as an approximation inlined into the heuristic agent.