// the find
RobotecAI/rai
RAI is a vendor agnostic agentic framework for Physical AI robotics, utilizing ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more.
RAI is a Python framework that puts LLM-based agents on top of ROS 2, letting you write robot behaviors in terms of tool calls rather than state machines. It targets teams who already know ROS 2 and want to add natural-language task execution, voice interaction, or multimodal perception without rolling everything from scratch. The arxiv paper and ROSCon talk suggest this is research-grade software with production ambitions.
The vendor-agnostic LLM layer is the right call — swapping models is a config change, not a rewrite. The rai_whoami subsystem (synthesizing robot embodiment from URDFs and docs into a system prompt) solves a real problem that every team hitting this space has to solve independently. The benchmarking suite (rai_bench) with predefined manipulation scenarios and a tool-calling agent evaluator is unusually mature for a sub-600-star repo. CI is solid: pre-commit, license checks, codecov, uv lock verification — they're treating it like production infrastructure.
Hard Ubuntu-only (22.04/24.04) with no Windows or macOS path, so your dev machine must run Linux or you're in Docker for everything. The LangChain dependency is load-bearing throughout the agent layer — you're not just adding RAI, you're also committing to LangChain's churn and abstractions. The simulation integration requires O3DE, which has a notoriously painful setup; the demos sound impressive but the on-ramp is steep. At 532 stars the community is small, which means when you hit an obscure ROS 2 middleware issue combined with an agent timing bug, you're probably debugging it alone.