// the find
dennybritz/reinforcement-learning
Implementation of Reinforcement Learning Algorithms. Python, OpenAI Gym, Tensorflow. Exercises and Solutions to accompany Sutton's Book and David Silver's course.
A Jupyter notebook collection implementing RL algorithms from Sutton & Barto's textbook and David Silver's course — Dynamic Programming through A3C, with exercises and worked solutions side by side. Built for people working through the theory who want runnable code to test their understanding, not a production RL library. The pedagogy is the point.
The parallel exercise/solution structure is genuinely useful for self-study — you can attempt a notebook, get stuck, and compare against a reference implementation without hunting across the internet. Coverage of classical methods (DP, MC, TD) is thorough and maps cleanly to specific book chapters, so you always know what you're implementing and why. The custom environments (Cliff Walking, Windy Gridworld, Blackjack) match the textbook examples exactly, which matters when you're trying to reproduce specific results from the text. A3C is implemented in plain Python files rather than notebooks, which is the right call for anything involving threads.
Abandoned in 2023 and effectively frozen since 2019 — OpenAI Gym is deprecated, the TensorFlow 1.x code in DQN and Policy Gradient sections won't run without significant patching, and several algorithms (DDPG, prioritized replay) are permanently marked WIP. There's no requirements.txt that actually works today; getting the environment set up is its own debugging project. Anyone who wants modern RL (PPO, SAC, TD3) needs to look elsewhere — this stops at A3C, which is circa 2016. The notebook-first format also means there's no way to run the code headlessly or integrate it into anything larger.