// the find
Sentdex/pygta5
Explorations of Using Python to play Grand Theft Auto 5.
A research project by Sentdex exploring imitation learning for autonomous driving in GTA5, using a distributed system where NPC-controlled cars collect training data that feeds a continuously-updating neural network. The reboot (2022) is architecturally more interesting than the original — live training streamed to Twitch while a separate player model improves in real time. This is primarily a documented research journey, not a usable library.
The distributed architecture is genuinely clever: data collectors, a central server, and a trainer are decoupled so training is continuous and the player always runs the latest checkpoint. The dual-camera system (hood cam as model input, third-person for human observation) solves a real problem — you can watch what the AI sees without that view being the training input. The regression approach to steering output is more principled than the 2017 classification version, and the progression through Xception → InceptionResNetV2 → recurrent architectures with stacked frames is well-documented with TensorBoard logs for every run.
The repo hasn't been touched since March 2023 and model_0012_regnet is listed as TBA with no content — the project stalled. There's no runnable code in the reboot; the actual training and data collection scripts appear to live on private machines or the game mod side, so you can't reproduce any of this. The dependency on a GTA5 mod and specific Windows screen-capture setup means the barrier to entry is high and underdocumented. After 750k batches the model showed no further improvement and was abandoned, which is the honest outcome but also means none of the architectures actually solved highway driving reliably.