Providers

Learn how to configure agentgateway for a particular LLM provider Provider A service that provides LLM capabilities, such as OpenAI, Anthropic, or Azure. Agentgateway supports multiple LLM providers and can route to different providers based on configuration. .

Native providers

OpenAI Anthropic Google Gemini Google Vertex AI Amazon Bedrock Azure OpenAI

OpenAI-compatible providers

Popular OpenAI-compatible providers include xAI (Grok), Cohere, Together AI, Groq, DeepSeek, Mistral, Perplexity, and Fireworks AI.

Self-hosted solutions

Ollama

Additional self-hosted solutions like vLLM and LM Studio are also supported through the OpenAI-compatible configuration.

Advanced configurations

Multiple LLM providers

Path prefix

When using the advanced binds/listeners/routes configuration, you can set pathPrefix on an AI provider to prepend a custom path to all API requests. Use pathPrefix when routing through a proxy or custom API endpoint that requires a different base path.

backends:
- ai:
    name: openai
    pathPrefix: /custom/v1
    provider:
      openAI:
        model: gpt-4o-mini
  policies:
    backendAuth:
      key: "$OPENAI_API_KEY"