The frog that learned to go sideways

The saved Frogger policy took 100,000 actions across its held-out evaluation. It moved left 49,400 times and right 50,600 times. It never moved up, down, or waited. Every episode lasted the full 500-step allowance and ended in a timeout. The goal count was zero.

That result is more informative than saying the agent performed poorly. It describes a specific behavior: remaining on the starting row while moving sideways. A policy that repeatedly enters traffic and dies has a different problem. These action counts point toward the question that matters for the next experiment: why was surviving without crossing competitive during training?

The agent is a linear policy. It turns the observation into an action score for each choice and selects the highest-scoring valid action. A genetic algorithm searches over those weights by evaluating candidate policies, ranking their mean episode rewards, retaining elites, and mutating descendants.

The bounded run used a population of 12 for 10 generations, with eight episodes per candidate. That is a small search, and candidate maps depended on population index. Candidates therefore did not face identical environments. The reported evaluation uses a separate, disjoint set of 200 environment seeds shared by all the evaluated policy labels, with the same configuration and horizon.

This design supports a statement about the saved checkpoint on those maps. It cannot establish that linear policies or genetic algorithms generally fail at Frogger. It also cannot isolate the training reward as the reason this particular run selected sideways behavior. Search budget and unequal candidate maps remain alternative contributors.

The neuro policy's mean reward was -0.1000. Random play averaged -0.8318 and the greedy policy averaged -0.6821. All three completed no crossings. If I reported only their rewards, the evolved policy could look like progress toward the task. The terminal reasons and actions show the narrower achievement: it avoided dying until time ran out.

The danger-aware scripted and heuristic labels each reached 159 goals in 200 episodes, with a mean reward of 5.3792. Those labels use equivalent decision logic and produced matching records, so they are not two independent demonstrations. They do show that many of the evaluated maps are crossable under the same horizon.

Putting reward beside goals makes the interpretation concrete. The neuro policy beat two unsuccessful baselines on mean evaluation reward, but it did not approach the successful baseline's task completion. The result calls for a diagnosis of what training selected, rather than a success story about longer survival.

In the reward code, the stall penalty applies to WAIT. Valid horizontal movement does not receive it merely because the frog makes no progress toward the goal. Timeout has a separate terminal penalty. The distinction leaves room for a policy that stays out of danger while continuing to issue movement actions.

This is a plausible mechanism for the observed behavior. The policy can avoid the cost of early death without paying the explicit WAIT penalty, then receive the smaller timeout penalty. But source inspection establishes what incentives are available, not why an optimizer settled on one of them. The existing evidence does not include a matched reward ablation.

A useful follow-up would hold candidate maps constant within each comparison and vary the treatment of non-progress while preserving a fresh evaluation boundary. That would test the hypothesis. Changing the reward and running until something works would be useful development, but it would not by itself identify which change explained the original plateau.

September 20 update: the comparison prerequisite now passes software checks. Candidates share a batch, and the retained champion is re-evaluated on that same batch before replacement. The checks expose candidate-index and stale-score errors, including use of the wrong generation’s maps. No new training or held-out policy evaluation was run. The reward hypothesis remains untested, and the original crossing results are unchanged.

The action distribution supplied the decisive evidence without a gameplay screenshot. It is a behavior trace derived from the evaluation, not a visual observation of a trained replay. The packaged replay shows the heuristic baseline, so it would be misleading to present it as the learned policy's behavior.

The durable change in how I read this experiment is to keep reward, terminal outcomes, and action counts together. Each answers a different question: what did the optimizer value, did the task finish, and what did the agent actually do? When those disagree, the discrepancy is the research result. Here it narrowed the next question from improving a vague low score to testing a specific route by which activity can substitute for progress.