xAI launches Grok Voice Agent API to build voice agents

xAI has launched Grok Voice Agent API, a tool that lets developers create voice assistants capable of holding conversations in dozens of languages, retrieving up-to-date information and taking actions through external tools.

The API brings to the wider market technology xAI already uses in Grok for its mobile apps and in millions of Tesla vehicles. In practice, a company could integrate it into a customer service platform, a travel app or an assistant that can query databases and act on your behalf.

Audio responses in under one second

Speed is one of the main selling points. According to xAI, Grok takes less than one second on average to start responding with audio. The company says this makes it almost five times faster than the nearest competitor in the Big Bench Audio benchmark, which measures voice agents' ability to solve complex problems.

The result was independently verified by Artificial Analysis. xAI also says that Grok Voice Agent API ranks first in that evaluation, although these figures depend on the specific conditions of each test and do not mean that every conversation will respond equally quickly.

The company says it developed much of the audio technology in-house, including the voice activity detector, the system that divides speech into processable units and the audio models. This control makes it possible to adjust how the agent listens, understands and responds separately.

It speaks multiple languages and can switch between them

Grok can automatically respond in the language a person is using and switch languages halfway through a conversation. Developers can also instruct it to always respond in a specific language through a system instruction.

xAI says the model maintains a natural level of pronunciation, accent and intonation. In blind human evaluations against OpenAI Realtime API, participants consistently preferred Grok in these areas, according to the company.

Developers will be able to choose from several voices, including Ara, Eve and Leo. They can also request expressive effects with prompts such as [whisper], [sigh] and [laugh], creating conversations with whispers, sighs or laughter.

It can search for data and use tools

The API does more than talk. It can retrieve real-time information from X and the web, as well as connect to tools built by individual developers.

The Tesla example shows how it works: when asked to organize a trip, Grok can search X for recommendations, calculate a route, add stops and prepare an itinerary. In the company's vehicles, it can also check the car's status and control navigation features.

This kind of integration is what separates a voice chatbot from an agent. The first responds; the second can query systems and complete tasks, as long as it has permission to use the connected tools.

Pricing and compatibility

The service has a flat rate of $0.05 per minute of audio. For an application handling many conversations, the cost will depend mainly on call duration and user volume.

Grok Voice Agent API is compatible with the OpenAI Realtime API specification, which could make it easier to migrate existing projects. It is also available through the official LiveKit plugin, a platform used to build real-time audio and video applications.

xAI has also enabled a browser-based testing environment for listening to the different voices. The company promises to share more updates over the coming weeks.

For you, the important change is that voice agents are starting to combine natural conversation, up-to-date search and concrete actions in a single interface. The question to watch now is not just whether they sound good, but how much control applications will have over the data, tools and decisions these agents can make.