Providers¶
The proxy has two categories of providers:
-
OpenAI-compatible providers — Any provider whose API follows the OpenAI chat completions format. These are all handled by a single generic adapter. Most providers fall here, including Gemini (via Google's OpenAI-compat endpoint). Adding a new one is two lines of config.
-
Native SDK providers — Providers with genuinely different wire formats: Anthropic, Azure OpenAI, AWS Bedrock.
Built-in OpenAI-compatible providers¶
| Prefix | Provider | Auth | Docs |
|---|---|---|---|
openai/ |
OpenAI | OPENAI_API_KEY |
details |
gemini/ |
Google Gemini | GEMINI_API_KEY |
details |
xai/ |
xAI / Grok | XAI_API_KEY |
details |
groq/ |
Groq | GROQ_API_KEY |
details |
mistral/ |
Mistral AI | MISTRAL_API_KEY |
details |
deepseek/ |
DeepSeek | DEEPSEEK_API_KEY |
details |
together/ |
Together AI | TOGETHER_API_KEY |
details |
perplexity/ |
Perplexity | PERPLEXITY_API_KEY |
details |
kimi/ |
Kimi / Moonshot | KIMI_API_KEY |
details |
Built-in native-SDK providers¶
| Prefix | Provider | Auth | Docs |
|---|---|---|---|
anthropic/ |
Anthropic / Claude | ANTHROPIC_API_KEY |
details |
azure/ |
Azure OpenAI | AZURE_OPENAI_* vars |
details |
bedrock/ |
AWS Bedrock | AWS credential chain | details |
Self-hosted servers¶
Any OpenAI-compatible REST server works through the same generic adapter: vLLM, Ray Serve, LM Studio, LocalAI, Llamafile, Hugging Face TGI, Jan, even a real Ollama instance. See Self-hosted servers.
Adding a new provider¶
Adding a new provider covers:
- Adding an OpenAI-compatible provider via env vars (no code changes)
- Adding via YAML config
- Implementing a custom provider when the wire format is different