Skip to main content

Supported LLM Providers

The current status of different LLM Providers and limitations

There is always an up to date list in the application: https://app.llmasaservice.io/modelpricing

OpenAI​

We support streaming for the chat completions API. Limitations: We only support text prompts and completions.

Azure OpenAI​

We support streaming for chat completions API. Limitations: We only support text prompts and completions. We estimate token usage (the Microsoft API doesn't yet send token usage in streaming responses)

Anthropic Claude​

We support the Anthropic streaming API. We only support text prompts using the messages api. We have tested the endpoint https://api.anthropic.com/v1/messages. Anthropic support token usage in their streaming api.

Google Gemini​

We support the Gemini streaming api. We have tested on the https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-latest:streamGenerateContent?alt=sse&key={{API-KEY}}. Gemini supports token usage in their API.

Amazon Bedrock​

We support the Amazon Bedrock text completion API giving you access to Titan and Anthropic models.

Mistral​

We support all of the Mistral text completion models. Mistral supports token usage in streaming.