// the find
bricks-cloud/BricksLLM
🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.
BricksLLM is a self-hosted API gateway for LLM providers, sitting in front of OpenAI/Anthropic/Azure calls to enforce per-key rate limits, cost caps, and usage tracking. It's aimed at teams distributing API access internally — think giving each developer or tenant their own rate-limited key without handing out your actual provider credentials. YC S22.
Per-key cost and rate limiting backed by Redis is the right architecture — fast path stays in-memory with a 1s polling sync from Postgres, so limit checks don't add meaningful latency. PII detection via AWS Comprehend is a real feature, not a regex toy, and the masking/redaction policy system is configurable per route. Failover and retry logic across providers (e.g. OpenAI → Azure OpenAI fallback) is genuinely useful and not something you want to build yourself. The internal message bus for async token counting keeps the hot path non-blocking.
Last commit was January 2025 and activity has been sparse for over a year — the repo looks effectively unmaintained. The admin server uses a single shared password (`ADMIN_PASS`) with no per-user auth or RBAC, which is a real problem for any multi-team setup. Model support lags: no Gemini, no Bedrock native routing beyond Anthropic, no support for newer OpenAI endpoints like the Responses API. The caching layer is a simple exact-match cache on request hashes — there's no semantic or fuzzy caching, so cache hit rates in practice will be near zero for most LLM workloads.