// the find
comet-ml/opik
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
Opik is an open-source LLM observability and evaluation platform from Comet — trace your LLM calls, run automated evaluations, and monitor production apps. It covers the full cycle from development tracing to production dashboards, and can be self-hosted via Docker Compose or Kubernetes, or used as a managed cloud service. It's aimed at teams building RAG pipelines, chatbots, or multi-agent systems who need more than just logging.
Integration breadth is genuinely impressive — 50+ supported frameworks (LangChain, LlamaIndex, CrewAI, Autogen, OpenAI Agents, Google ADK, and more) means you can drop it into almost any Python LLM stack with minimal code changes. The @opik.track decorator approach for arbitrary function tracing is clean — no framework lock-in required. Self-hosted path is well-thought-out: Docker Compose for local, Helm chart for Kubernetes, and the service profile flags (--infra, --backend, --guardrails) show someone actually uses the local setup. The LLM-as-a-judge metrics (hallucination, moderation, RAG answer relevance) are first-class, not bolted on — with a scoring API that's simple enough to add to a CI pipeline via the PyTest integration.
The backend is Java (Dropwizard) with ClickHouse and MySQL under the hood — that's a non-trivial self-hosting surface area that the Docker Compose setup hides until something breaks. The 'Opik Agent Optimizer' and 'Opik Guardrails' features get name-dropped prominently but have thin documentation compared to the tracing/eval core — feels like they're still catching up to the marketing copy. TypeScript SDK exists but the integration coverage is much thinner than Python, which matters if your team isn't Python-first. The cloud offering is the path of least resistance, which means the open-source version risks becoming a funnel rather than a genuinely maintained alternative — the changelog shows active development but the Comet branding is everywhere.