// the find
tashfeenahmed/freellmapi
OpenAI-compatible proxy that stacks the free tiers of 16 LLM providers (~1.7B tokens/month) behind one /v1 endpoint — plus any custom OpenAI-compatible endpoint. Smart routing, automatic failover, encrypted keys. Personal experimentation only.
A self-hosted OpenAI-compatible proxy that aggregates free tiers from 18+ LLM providers behind one endpoint, with automatic failover, per-key rate tracking, and a React dashboard. It's for developers who want to run LLM-heavy tools locally without paying for API access — think side projects, local Claude Code, or testing pipelines.
The rate-limit tracking is genuinely useful: per-(platform, model, key) RPM/RPD/TPD counters that learn from error responses mean the router rarely hits a hard 429 rather than discovering it at request time. The embeddings failover design is correct — it refuses to switch models mid-collection, which is exactly the bug naive proxies introduce. The Thompson-sampling bandit for routing strategy is a reasonable fit for the explore/exploit problem of unknown per-provider latency. The test suite coverage is serious — provider adapters, fallback loop, tool-call rescue, streaming integrity all have dedicated test files.
SQLite is a real constraint that will bite anyone trying to run this on multiple machines or with any concurrent write load — the README buries this. The 'personal experimentation only' disclaimer exists because several of these free tiers explicitly prohibit commercial use or proxying, and the ToS review section in the readme is just a link, not actual guidance. The premium upsell (30-day model lag on the free tier, $19/yr to get day-one access) is woven into a self-hosted open-source tool in a way that will frustrate people who expect the catalog to just work; the free install is shipping 79 fewer models than premium right now. Context handoff on model switch is opt-in and injects a system message, which works fine for simple tasks but will silently break anything relying on exact system prompt structure.