AI Frogger

The learned Frogger policy stayed alive for an entire episode by moving left and right. It never moved toward the goal. That behavior earned a better average reward than random movement or a greedy policy that walked straight into traffic, so reward alone could have made the run look like progress. Completed crossings told a different story.

I built this project to put policies through the same seeded road and river hazards, then inspect what their scores mean. The first bounded genetic-training run produced a usable experiment and an unsuccessful crossing policy. The useful result is the explanation the saved evidence makes possible: which actions it took, how episodes ended, and which part of the reward deserves testing next.

The goal is simple: cross traffic, ride moving logs across the river, and reach the far side. A legal move can still put the frog in danger after the hazards advance. A policy therefore needs information about what will happen next, rather than just the direction of the goal. The observation supplies local tiles, valid actions, and directional danger predictions for the next tick.

That distinction gives the baselines a purpose. Greedy moves upward whenever it can. The danger-aware policies check the next-tick hazard before advancing or sidestepping. The labels scripted and heuristic currently use equivalent action-selection logic, despite different decision text, so their matching results provide one reference behavior under two names.

FroggerEnv owns the configuration, seeded state, and diagnostics. A step applies the selected movement, advances traffic and log riding, resolves terminal conditions, and records a reward breakdown. Headless evaluation, genetic training, benchmarking, and the optional raylib viewer all use that core. A policy change can therefore be compared without also changing the game underneath it.

The agent interface separates observation from action selection, and saved weights can be loaded into the evaluator after training ends. Metrics exports retain episode rewards, lengths, terminal counts, and action distributions. The viewer reads the same state and decision information, but graphics are optional. Experiments and tests can run without a window, while replay and rendering still have their own verification requirements.

The learned policy is a linear scoring function. It flattens the observation, scores each action with a weight array, and chooses the highest-scoring valid action. There are no hidden layers or recurrent memory. The trainer evaluates candidate arrays over episodes, ranks their mean rewards, retains elites, and fills the population with mutated descendants. The selected weights are saved for later evaluation.

This makes fitness the decisive interface between game design and learning. The trainer has no separate understanding of crossing, progress, or an unhelpful sidestep unless those distinctions affect its reward. The September 18 run also had a comparison weakness: training maps depended on population index. Candidates played different maps, so their fitness combined policy behavior with the difficulty of the maps assigned to them. That historical run cannot isolate genetic selection quality from this difference.

The September 20 repair assigns every candidate the same episode seeds within a generation, then advances the batch for the next generation. There is a second comparison to fix: the retained champion’s old score came from old maps. The trainer now re-evaluates that incumbent on the current batch before deciding whether to replace it. An easier earlier batch can no longer win that decision through a stale score.

The checks use fixed policies and small synthetic fixtures. They verify that results follow a policy when its population position changes and that the incumbent uses the same batch as its challengers. Deliberately restoring candidate-specific seeds, skipping re-evaluation, or using a neighboring generation’s batch makes the regression fail. No new training campaign or held-out policy evaluation was run. This repairs the comparison mechanism, while the failed policy and the reward hypothesis below remain unchanged.

Before relying on an optimized test run, I repaired the assertion contract. Release builds define NDEBUG, and some tests had put persistence operations inside C assertions. Those builds could skip both the checks and the operations the tests appeared to exercise. The test targets now keep assertions active and have a compile-time guard against disabling them. The recorded local Release and Debug runs each passed all 12 tests.

The console summary had a second trap: one timed-out episode printed as two timeouts. The terminal-reason JSON counted that episode correctly, and the historical comparison below uses those counts. A September 20 repair gives episode completion sole ownership of timeout accounting. Tiny runner and CLI checks now reconcile timeout, goal and death outcomes in both Release and Debug, including an external step limit. Their JSON and CSV outputs are unchanged for the same environment-horizon fixtures, and the saved policy's 0/200 result is unchanged. This fixes reporting, not the policy.

The declared training plan used 12 candidates for 10 generations, 8 episodes per candidate, 4 elites, and a mutation probability of 0.05. Episodes used the default crossing layout and a 500-step limit. This was a bounded run, with no tuning permitted against its later held-out result.

After training, the saved policy and all four baseline labels played the same 200 seeds, disjoint from the declared training range. The automatic evaluation printed by the trainer overlaps training seeds and is excluded here. The downloadable plan, configuration, weights, raw summaries, and per-episode exports identify the exact comparison. Holding maps and episode limits fixed at evaluation makes the behavioral differences readable, even though candidate maps differed during that historical training run.

The danger-aware labels each completed 159 of 200 crossings. Random and greedy completed none, and neither did the learned policy. Every learned-policy episode reached the 500-step limit. Across its 100,000 actions, the policy chose only left and right, with no upward moves, downward moves, or waits.

Its average reward was -0.1000, compared with -0.8318 for random, -0.6821 for greedy, and 5.3792 for the danger-aware behavior. It had learned a better-scoring way to fail than the first two baselines. The goal counts and action distribution prevent that narrow improvement from becoming a claim that it learned Frogger. This result concerns one saved policy on one fixed evaluation set, and does not establish a limit of linear policies or genetic algorithms.

The reward code penalizes WAIT for stalling, but valid horizontal movement does not receive that penalty merely for making no progress. A timeout has a separate terminal penalty. This is consistent with the observed policy: move sideways, avoid an early death, and accept the timeout. The code provides a plausible incentive for the behavior, and the action counts show that the behavior occurred.

That is still a hypothesis about why training selected this policy. No reward ablation isolated the effect, and candidate-specific maps remain an alternative explanation for the historical result. With the comparison repair checked in software, the next experiment can test a reward intervention under a declared protocol. Increasing the training budget alone would not isolate the reward’s effect.

The local environment, trainer, saved-policy loading, and evaluation exports work, and the evidence package preserves the failed policy alongside its baselines. A separate heuristic replay reaches the goal in 16 actions. The browser viewer below displays 17 native states reconstructed from that existing recording, starting before the first action. Repeated reconstruction matches the retained terminal outcome, and every displayed frame comes from the unchanged simulation core. This illustrates successful baseline behavior, not a learned-policy success.

The original raylib viewer builds, but its rendered playback and controls remain unaccepted. This browser view is a separate rendering of exported state, with no policy or physics running in JavaScript. Configuration loading still supports a limited layout rather than a general validated JSON contract. The research question remains whether a controlled reward change produces crossing rather than another higher-scoring failure, under a new evaluation plan.