TL;DR
A proxy framework that consolidates free tiers from 16 LLM providers into a single API, offering over 1.7 billion tokens per month.
Key features
Unified Endpoint: Consolidates 16 providers like Google, Groq, and Mistral into a single /v1/chat/completions endpoint.
Smart Routing: Automatically selects the best available model for each request.
Automatic Failover: Seamlessly switches to another provider upon rate limiting or errors.
Usage Tracking: Manages per-key usage to stay within each provider's free tier limits.
Security: Stores API keys encrypted.
When to use it
When you want to efficiently utilize the free quotas of multiple LLM providers.
When you need high monthly token volume without relying on a single LLM.
When you want to seamlessly test various models using an OpenAI-compatible client.