AI News
AI News AgentToolingPerplexity4 min read

Perplexity launches Orchestrator to route AI models

Perplexity introduces Orchestrator, an API that connects proprietary and third-party models through a single integration and billing system. It can switch providers when failures occur and direct each task to the most suitable model through automatic routing, which is available in preview.

Perplexity has launched Orchestrator, an API that brings together proprietary models, open-source models, and commercial models from different providers through a single integration and a single bill.

The tool aims to solve a practical problem: when an application depends on a single provider, an outage, capacity limit, or increase in latency can leave it without a response. Orchestrator can send the same request to other available infrastructure without changing the model you selected.

One connection for multiple providers

Orchestrator lets you use models hosted on Perplexity's inference infrastructure alongside proprietary models from other providers. The platform offers a catalog of available options and accepts the following formats:

  • OpenAI Chat Completions
  • OpenAI Responses
  • Anthropic Messages

This means you can connect using the OpenAI or Anthropic SDKs you already use by changing the API key, base URL, and selected model.

The difference becomes clear when something goes wrong. If a request for Claude cannot be handled through Anthropic because of an error, rate limit, or timeout, Orchestrator can try another implementation of the same model, such as AWS Bedrock. The model does not change, but the provider running it does.

The platform decides where to send each request using observed data on the availability and capacity of each deployment. It also sets limits for the time it takes to receive the first token and for the total duration of each attempt.

More availability, according to Perplexity's measurements

Perplexity compared the behavior of Claude and GPT requests sent through Orchestrator with requests directed to individual providers. The measurement covered October 4 to November 2, 2026.

According to the company, Claude and GPT requests through Perplexity had a higher success rate than any individual provider included in the comparison. Their daily rates also varied less during incidents affecting providers.

The calculation counted a request as failed when Orchestrator exhausted all available implementations or the layer serving the model returned an error. It did not include client errors, such as invalid requests or usage limits reached by the client itself.

Speed can also change depending on who runs the model. During the 30 days analyzed, Anthropic was the fastest provider for Claude Opus 5.5 on 15 days, while AWS Bedrock was fastest on the other 15. For GPT-5.6 Sol, Azure's median generation speed was 27% higher than OpenAI's during that period.

Perplexity is working on adaptive routing that accounts for these speed changes and can automatically choose the fastest available provider.

One bill for all models

Orchestrator charges for each request based on the token usage of the model that responds. Tokens are the units of text the model processes and generates.

The price per token remains the published price for that model, regardless of which provider handled the request. If Orchestrator switches providers after a failure, it does not apply a different fee for that reason.

Usage is deducted from the same prepaid credits in the Perplexity API project. In the console, you can view spending by model and filter usage by API key. You can also purchase credits through AWS Marketplace.

Automatic routing for each task

The Automatic routing feature is available as a preview. Instead of choosing a fixed model for an entire application, you can describe what type of task each model should handle.

For example, a support assistant could use:

  • GLM-5.3 Flash for routine questions, such as changing an account email address.
  • Claude Opus 5.5 for billing disputes or refund requests.
  • GPT-6.1 Sol for technical errors and integration problems.

Automatic requests use perplexity/auto. A decision model reads the conversation and compares it with the route descriptions, but it does not see the model names. It then sends the request to the model associated with the selected route. If none match, the decision model fails, or it takes too long, the system uses the default model.

This system can also switch models in the middle of a conversation. A question about an account can start with a fast model, move to a more careful one to resolve a billing issue, and return to the fast model when the user only says thank you.

Each request includes the complete message history, although each model maintains its own cache. Orchestrator tries to keep the same implementation whenever possible so it can reuse previously processed parts. A stable key such as prompt_cache_key can help maintain that affinity and reduce repeated work.

What changes for you

For anyone developing an AI application, Orchestrator reduces the work involved in integrating and monitoring multiple providers. You can maintain a single connection, one billing method, and a recovery policy for when a service fails.

It also lets you adjust cost and quality for each task. A simple question does not have to use the most expensive model, while a billing dispute, technical diagnosis, or research task can be directed to more capable models.

The tradeoff is that automatic routing is in preview and adds a decision layer before generating the response. You will need to monitor its latency, costs, and accuracy when interpreting the descriptions for each route. The next step will be to see whether this layer can maintain reliability without making day-to-day operations more complex.