// the find
antoinezambelli/forge
A Python framework for self-hosted LLM tool-calling and multi-step agentic workflows
Forge is a reliability layer that wraps self-hosted LLM tool-calling — rescue-parsing malformed outputs, retrying on validation failures, and enforcing step ordering. It works as a drop-in proxy (OpenAI and Anthropic API shapes) or as a Python framework you build workflows on top of. Aimed at developers running 8B-class models locally who are tired of flaky tool calls breaking their pipelines.
The proxy mode is the smartest design choice here: you point Claude Code or opencode at localhost:8081 and forge silently fixes your local model's tool-call failures without touching the client. The synthetic `respond` tool injection is a clever hack that stops small models from hedging between text and tool calls, and the ADR explains exactly why it was needed. 865 deterministic unit tests that run without a backend are a real differentiator — most projects in this space have no meaningful test coverage. The eval harness with 26 reproducible scenarios and JSONL resumable batch runs means the performance numbers (8B from single digits to 84%) are actually verifiable, not marketing.
Context compaction is entirely the client's problem in proxy mode — if you're running a long session through the proxy, you'll hit the context limit and forge just passes the failure back. The `SlotWorker` for multi-agent GPU sharing sounds useful but the docs are thin on what happens when preemption races with a half-executed tool call. Python 3.12+ only and no Windows native support for managed backend mode (llama-server subprocess management) means anyone not on Linux or macOS is on their own. The eval suite measures tool-call reliability but nothing about latency or throughput, so you can't easily judge whether forge's retry overhead is acceptable for interactive use cases.