// the take
A from-scratch PyTorch implementation of Proximal Policy Optimization trained to play Super Mario Bros (NES, via the gym-super-mario-bros env). It's aimed at people who want a readable, self-contained PPO reference rather than a plug-in RL library — you can train an agent per level and watch it clear 31 of 32 stages.
Results are concrete and reproducible, not just claimed: per-level trained checkpoints are included for most stages, and the README is upfront about which level (8-4, the branching-path maze) PPO never solved. The code is a compact, single-file-per-concern implementation (env wrapper, model, training loop) that's actually readable end to end, useful if you want to learn PPO internals rather than import a black box. Docker support with GPU passthrough is provided for people who don't want to fight CUDA/gym version hell locally.
Last commit is from mid-2021 and it depends on the old OpenAI Gym API plus nes-py/gym-super-mario-bros, both of which have had breaking releases since — expect to pin old package versions to get this running at all. Training is single-environment, not vectorized across workers, so wall-clock training time is much slower than a modern PPO setup (SB3, CleanRL) using the same algorithm. Hyperparameters are brittle and tuned per level by hand: the README admits one stage took 70 failed runs before finding a working learning rate, which means this isn't a general-purpose Mario solver so much as 32 separately overfit training runs. No evaluation harness or metrics logging beyond the final demo video, so adapting it to a new environment means re-deriving reward shaping and LR schedules from scratch.
We read the README, the directory tree and a key source file in October 2026. Nobody pays to be listed. How we judge →