Agora and OpenAI’s Realtime API Power Seamless Interaction with Multimodal AI Agents

Agora’s Conversational AI Engine offers key enhancements to the Realtime API for more natural communication and interaction.

Agora, the leading platform for real-time engagement and conversational AI, announced expanded support for OpenAI’s Realtime API, now generally available. Agora’s integration with the new Realtime API now supports automated greetings, mixed-modality interaction, selective attention locking and more advanced functionality designed to power more natural interaction between users and AI agents.

This milestone builds on Agora’s partnership with OpenAI, as the Realtime API is the first multimodal large language model (MLLM) built into the Agora platform. The combined solution empowers developers to create more natural, responsive, and human-like AI agents by reducing development complexity while unlocking advanced capabilities in real-time interaction.

“Real-time multimodal interaction is the missing piece for AI agents to feel truly human,” said Tony Zhao, CEO of Agora. “By integrating OpenAI’s Realtime API into our Conversational AI Engine, we’re giving developers the tools to build experiences that are faster, smarter, and more natural than ever before.”

Agora’s Conversational AI Engine now offers more advanced features to enable natural interaction with AI agents:

• Automated Greetings: Ensures instant session awareness and a natural, welcoming onboarding experience.
• Mixed-Modality Interaction: Enables seamless switching between voice and text inputs within a single interactive session.
• Flexible Turn-Detection Options: Gives developers fine-grained control over conversational flow and turn-taking behavior.
• Uninterrupted Input: Agora’s proprietary Selective Attention Locking technology filters out ambient noise and interfering voices for uninterrupted engagement.

Through Agora’s Conversational AI Engine, developers gain access to a powerful set of tools that not only streamline adoption of the Realtime API but also unlock new features and use cases for multimodal AI agents. By combining OpenAI’s real-time language model with Agora’s global real-time network infrastructure (SDRTN^®) and purpose-built developer toolkit, teams can accelerate time to market, simplify application development, and deliver superior real-time conversational AI experiences.

Robotics startup Carbon Origins is already leveraging Agora’s technology integrated with OpenAI’s Realtime API to enable hands free operation of heavy equipment and enhance operator efficiency.

Marketing Technology News: Before Jumping on the AI Bandwagon, Focus on the Data

“The combination of OpenAI’s Realtime API and Agora’s conversational AI technology enable hands-free control of our autonomous robot fleet,” said Amogha Krishna Srirangarajan, CEO and Founder of Carbon Origins. “The technology powers the automation of complex checklists and system operations in our Constellation AI solution, allowing operators to focus on strategic tasks and orchestration instead of manual execution.”

The integration further strengthens Agora’s position as the leading platform for conversational AI, real-time engagement, and multimodal agent development, with applications spanning customer support, education, gaming, fan engagement, and beyond.