OpenAI launches GPT-Realtime for voice agents
OpenAI has launched `gpt-realtime` and made its Realtime API available to everyone, giving developers a platform for building voice agents that speak more naturally, use tools, interpret images and receive phone calls. The model improves in reasoning, instruction following and function use, with prices 20% lower than its previous version.
OpenAI has launched gpt-realtime, its new voice model for real-time conversations, alongside the general release of its Realtime API. The platform lets you build agents that can talk over the phone, interpret images, use external tools and hold more natural conversations.
The main difference is how it processes audio. Instead of first converting speech to text, passing it through a model and then generating audio, the system receives and produces sound directly through a single model. That reduces latency and helps preserve details such as tone, pacing, pauses and laughter.
More accurate speech and instruction following
According to OpenAI’s internal evaluations, gpt-realtime improves on the previous model, released in December 2024:
- In audio reasoning, it achieved 82.8% accuracy, compared with 65.6% for the previous model.
- In instruction following, it reached 30.5%, compared with 20.6%.
- In function use, it scored 66.5%, compared with 49.7% previously.
In practice, this means an agent can follow more specific instructions, such as reading a legal notice word for word, repeating a phone number correctly or switching languages in the middle of a sentence.
It can also adapt the way it speaks. An application can ask it to respond quickly and professionally, or use an empathetic tone and a French accent. OpenAI says the model is better at identifying nonverbal cues, such as laughter, and recognizes alphanumeric sequences in Spanish, Chinese, Japanese and French.
The company has added two new voices, Cedar and Marin, and updated the eight voices that were already available. All of them use preset voices, a measure intended to reduce the risk of impersonating real people.
An agent that can use tools and see images
The API now supports remote MCP servers. MCP is a standard that connects a model to external tools, such as databases, calendars or customer service systems. Developers only need to provide the server address, and the API manages calls to those tools.
It also supports image inputs. You can send a photo, screenshot or document along with audio and ask the agent to explain what it sees or read text. It does not work as a continuous video stream: the application decides which images to share and when.
Another new feature is support for SIP, the protocol used by many telephone networks, PBX systems and office phones. This lets you connect a voice agent to the public telephone network without building the entire integration from scratch.
Long-running tool calls do not have to interrupt the conversation either. If an action takes time to complete, the agent can keep talking while it waits for the result. The feature is built into gpt-realtime, so it does not require additional code changes.
What changes for businesses and users
The API is now available to all developers, not just as a preview. It can support customer service, personal assistants, education or any service that needs to communicate by voice and access information in real time.
OpenAI is also cutting the price of gpt-realtime by 20% compared with gpt-4o-realtime-preview: it costs $32 per million input audio tokens and $64 per million output audio tokens. Cached input costs $0.40 per million tokens.
The platform includes controls for limiting conversation context and discarding several old turns at once. This can reduce the cost of long sessions. It also lets you save and reuse instructions, tools, variables and examples across different sessions.
OpenAI continues to use automatic filters to stop conversations that violate its policies and requires users to know when they are talking to an AI, unless that is obvious from the context. The API also supports data residency in the European Union for applications based there.
The next step will be seeing how these agents perform outside demonstrations: on real calls, with noise, interruptions, accents and ambiguous requests. The technology is ready to connect to phones and other tools. What matters now is which companies integrate it and what safeguards they use to protect the people who rely on it.