PacMan AI

PacMan AI began as a small game and learning project in May 2023. I built the board, entities and pygame loop, then explored ways to make an agent play it. The repository contains a NEAT path, a deep Q-network with prioritized replay, and earlier CoDeepNEAT and HyperNEAT experiments. The game provides a concrete place to connect observations, actions and rewards, but putting those pieces together does not by itself produce a trustworthy comparison.

The most useful story emerged from reviewing the environment. Reset could add a second set of ghosts and pellets. Removing a pellet could leave the network's observation unchanged. The replay buffer could report its full capacity while empty. Each problem affects a different assumption a training loop makes about its input. Changing a network architecture would leave those assumptions broken.

This page follows the corrections in a public, unmerged branch and checks them against the earlier main revision. The September 2026 audit uses tiny deterministic fixtures so each difference is visible. The original project remains archived, and the checked branch has not replaced main.

The original loader appended ghosts and pellets to collections already held by GameState. Calling reset loaded the maze again without clearing those collections. A three-row fixture with one pellet and one ghost makes the effect easy to see: their counts start at one each, become two after the first reset, and become three after the second. The displayed maze text has not changed, but the state used by the game has.

The repair clears the entity collections and reconstructs the player before reading the maze. Repeating the same resets now leaves one pellet and one ghost. A separate check on the first repository maze also removes a pellet and changes the score before resetting. The repaired state restores its 256 pellets, one ghost, zero score and three lives.

These are useful checks because they target the episode boundary directly. A successful reset should not depend on how much of the previous episode happened. The fixture catches accumulated state without needing to train an agent or infer the problem from a noisy reward curve.

The original Q-learning encoding read a static board containing the initial pellets. The game removed a consumed pellet from its entity list, but that removal did not change the encoded cell. In the small fixture, the pellet cell stays at one after removal on main. The repaired encoding composes the current board from walls, remaining pellets and current entity positions. The same cell then changes from one to minus one, the value used for an empty space.

A second issue came from assuming every maze contained 1,024 cells. The flat encoding reshaped the board to that fixed length. The first repository maze actually has 31 rows and 28 columns, so its 868 cells cannot be reshaped that way. The branch derives the flat length from the board and produces a Q-learning array with shape 31 by 28 by one. Maze two is 32 by 32 and happens to fit the older assumption.

That repairs the state encoder's shape contract. It does not establish that every agent configuration accepts every maze. The NEAT configuration and network input count still need to agree with their chosen board, and the branch's broader agent behavior was not exercised here. The terminal-state check is similarly narrow: clearing all pellets now ends an episode, alongside the existing loss condition of running out of lives.

The DQN code uses a SumTree to store replay entries with priorities. Its length is used to decide whether enough experience is available for a training batch. Main returns the capacity of the backing array. An empty tree with room for three entries therefore reports a length of three, and still reports three after only two inserts. Capacity describes available storage, which is different from the number of recorded transitions.

The branch adds an occupancy counter capped at capacity. In the checked three-entry tree, length starts at zero and becomes two after two inserts. A separate two-entry fixture confirms that a third insert overwrites the oldest entry while the total priority remains two. That overwrite behavior already worked on main. The branch also rejects sampling an empty tree instead of returning an unused slot with zero priority.

A further edge case still fails in both revisions. Adding the only entry to a capacity-one tree sends priority propagation into recursion and raises RecursionError. The audit therefore supports the specific occupancy and empty-sample repairs, with a clear remaining defect. It does not establish correct behavior for every capacity, priority value or sampling boundary.

On main, importing the DQN training module also constructs its environment and agent, creates output directories and enters a training loop. That makes importing a helper or checking a command risky and expensive. It also makes a simple test depend on the entire machine-learning stack before the test has asked to train anything.

The branch moves training into train_dqn and calls it through a guarded command-line entry point. Keras and environment imports occur inside that training function, and output directories are created there too. The CLI exposes episode and step limits, a seed and an output directory. Those source changes make the intended lifecycle much clearer: importing definitions and starting a run are separate operations.

This audit inspected that structure without importing either training module. The available Python environment lacked tqdm and pytest, so runtime CLI import and help behavior were not verified. A source guard is visible evidence of the refactor, while dependency compatibility and a successful training invocation remain separate work.

The repository also records an earlier attempt to use CoDeepNEAT and HyperNEAT. Commit messages describe the CoDeepNEAT experiment working on CIFAR and failing on Pac-Man, followed by a return to plain NEAT. That is a record of the direction taken during development. It contains no controlled comparison, fitness results or diagnosis that would explain the difference between those tasks.

The DQN history likewise includes architecture changes and saved replay filenames, but a filename containing a score cannot establish policy quality. There is no matched random-policy baseline or held-out comparison in the evidence used here. Historical pickle files were not opened for this audit. The separately authored, vendored pytorch_neat material also remains distinct from my game and integration work.

I keep those experiments visible because they explain the repository's shape. They should not be read as a ranking of learning algorithms. The clearer outcome is a smaller set of environment and training-lifecycle problems that can be reproduced and checked individually.

The downloadable evidence pins both source revisions and records the inspected file hashes, Python 3.12.14 and NumPy 2.3.5. Seven selected checks exercise reset replacement, encoding dimensions, pellet removal, terminal state, replay occupancy, overwrite and empty sampling. All seven pass on the repair branch. Main passes the overwrite check and fails the other six. These checks deliberately target the changed behavior, so their pass counts are not an overall project-quality score.

The branch's four existing GameState tests were also invoked directly and passed. They overlap four of the seven checks, rather than adding four independent results. This was not a full pytest run. The capacity-one failure was reproduced separately, and a second execution of the source probe matched the recorded observations.

The figures on this page visualize those software observations. They are not captured gameplay, a trained-agent demonstration or a learning curve. The probe and its reading guide make it possible to inspect how the claims were derived without presenting the unmerged branch as a finished release.

The next step is to resolve the replay-tree edge case and verify the explicit training entry point in a documented environment. A deterministic headless runner would then make it practical to record episode length, score and termination reason without relying on the renderer. Those measurements need to come from the same maze and reward rules for every controller being compared.

Only after that would a random-policy baseline and repeated, separately evaluated NEAT and DQN runs answer the original agent question. The training seed, evaluation courses and checkpoint-selection rule would need to be recorded before interpreting a better score. None of that evaluation has been performed for this case study.

The lasting value of PacMan AI is the ability to trace an apparently small implementation choice into the assumptions of a learning experiment. Reset, observation and buffer length all look routine until the experiment depends on them. Making those contracts visible gives future results a firmer starting point.