// the find
andyzeng/visual-pushing-grasping
Train robotic agents to learn to plan pushing and grasping actions for manipulation with deep reinforcement learning.
Reference implementation of the IROS 2018 paper on learning push-then-grasp synergies through self-supervised RL. Two fully convolutional networks (one for pushing, one for grasping) are trained jointly via Q-learning using only grasp success as reward, with pushes rewarded indirectly when they lead to future grasps. Aimed at robotics researchers who want to reproduce the paper or adapt the method to their own manipulation setups.
The sim-to-real pipeline is genuinely complete — V-REP simulation, real UR5 communication over TCP, RealSense depth streaming via a compiled C++ server, and a checkerboard calibration script that solves for camera extrinsics and a depth scale factor. The fully convolutional architecture is a good design choice: it produces a dense pixel-wise action map, so the robot doesn't need a separate region proposal step. The benchmark suite of 11 adversarial clutter scenarios is reproducible and exclusive from training, which is more rigorous than most academic repos. The logging is thorough — RGB-D images, heightmaps, action choices, rewards, and model snapshots all saved per session, with a plot script that tracks grasp success vs push-then-grasp success over training steps.
Dead since 2021. The pretrained weights were saved under PyTorch 0.3 and the README says they won't load on 1.0+, so the quick-start demo is broken without retraining from scratch. V-REP/CoppeliaSim has moved on significantly; the bundled `remoteApi.so` is a compiled binary for Ubuntu 16.04 and won't run anywhere modern without rebuilding or finding a matching CoppeliaSim version. The object set is nine colored blocks — no shape or texture variety — so generalization claims from the paper are hard to verify with this code as-is. Real robot support is Python 2.7 only, explicitly noted as not fully tested with Python 3, which means anyone running it today needs a compatibility shim or a patch pass through `robot.py`.