FreeLLMAPI
A self-hosted, OpenAI-compatible proxy that stacks the free tiers of 16 LLM providers (~1.7B tokens/month) behind one
/v1endpoint, with smart routing, automatic failover, and encrypted keys. Current public release:v0.4.1; ~14K stars as of 2026-06-30. For personal experimentation only. Source: tashfeenahmed/freellmapi README, https://github.com/tashfeenahmed/freellmapi, 2026-06-30
Each provider's free tier is a toy on its own; stacked, they add up to roughly 1.7 billion tokens per month across 100+ models. The pain is that stacking sixteen SDKs and sixteen rate limits by hand is brittle, so FreeLLMAPI collapses them behind a single /v1/chat/completions endpoint. Point any OpenAI client at the local server and it routes transparently across whichever providers have budget. Source: tashfeenahmed/freellmapi README, GitHub
Providers and Features
- 16 providers: Google, Groq, Cerebras, SambaNova, NVIDIA, Mistral, OpenRouter, GitHub Models, Cohere, Cloudflare, HuggingFace, Z.ai (Zhipu), Ollama Cloud, Kilo, Pollinations, LLM7, plus any custom OpenAI-compatible endpoint (llama.cpp, LM Studio, vLLM). Mistral alone contributes ~1B tokens/month.
- Responses API (
/v1/responses) implemented as a shim, the wire format current Codex CLI versions require. - Automatic failover: on 429/5xx/timeout it cools the key down and retries the next model in the chain (up to 20 attempts).
- Per-key rate tracking (RPM/RPD/TPM/TPD), sticky sessions (30-min model affinity to avoid mid-conversation hallucination spikes), and AES-256-GCM encrypted key storage.
Source: tashfeenahmed/freellmapi README, GitHub
The reviewed image captures the product's fallback-chain UI: a monthly token-budget bar, sortable model chain, per-provider token limits, and toggles for each model/provider. That visual matters because it proves the tool is not only a proxy endpoint; it also exposes budget accounting and failover ordering as an operator surface. Source: local X image artifact, 2026-06-30
Current Source Snapshot
Current source snapshot as of 2026-06-30: tashfeenahmed/freellmapi, default branch main at db7b64ddc88bd7b69841d8511cd6276d6d358a1d, latest GitHub release v0.4.1 published 2026-06-20, MIT license, 14,165 stars, and 2,095 forks. Brin returned 65 caution for the repo and 59 caution for the skill, which reinforces the existing route: useful for local experiments and architecture study, not a production dependency. Source: GitHub API, git ls-remote, Brin API, 2026-06-30
Caveats
The README is explicit that this is for personal experimentation only and includes a Terms-of-Service review section, because aggregating provider free tiers can violate individual provider ToS. Treat it as a sandbox/learning tool, not production infrastructure. Contrast with Vercel AI Gateway, which is the production-grade, paid, single-endpoint router with provider sorting. The author noted the post's modest bookmark count with a wry "703 bookmarks for this??? ok." Source: X/@david_attisaas thread, 2026-05-19
Stack fit (Kevin)
- Throwaway local experiments and cheap evals — pointing a script or an The Eval Loop (Slop Is an Output Problem) test suite at free tokens is fine for prototyping; ~1.7B tokens/month covers a lot of low-stakes iteration without burning paid budget.
- Never for Dedalus or any shipped product — aggregating provider free tiers can violate individual ToS and the failover chain has no SLA; production routing belongs on a paid single-endpoint gateway (e.g. Vercel AI Gateway), the ranked choice in Kevin's stack.
- OpenAI-compatible means zero migration cost — the Codex CLI (via the
/v1/responsesshim) and any OpenAI client point at it unchanged, so it is a drop-in for local agent tinkering. - Read it as a design study — the real value for Kevin is the routing / failover / sticky-session / encrypted-key patterns, not depending on the service.
Timeline
- 2026-05-18 | @david_attisaas shared FreeLLMAPI, 1,938 likes, 3,109 bookmarks. Repo at ~6.9K stars: 16 stacked free-tier providers, ~1.7B tokens/month, OpenAI-compatible with failover and encrypted keys. Source: X/@david_attisaas, 2026-05-18
- 2026-06-13 | Quality remediation: confirmed repo live at ~10K stars (9,984), MIT-licensed; added stack-fit guidance (local experiments / cheap evals only, never production). Source: https://github.com/tashfeenahmed/freellmapi
- 2026-06-30 | Deep-reviewed the local image artifact and current source state. Recorded release
v0.4.1, source HEAD, 14,165 stars, and the budget/fallback-chain UI as the durable operator surface. Source: X bookmark artifact audit, 2026-06-30