Thor Schaeff from Google DeepMind introduces the capabilities of the Gemini API, focusing on live voice agents and multimodal conversational AI. He begins by showcasing Gemini’s text-to-speech (TTS) model, highlighting its expressive speech synthesis with low latency and built-in audio tags. Thor emphasizes the versatility of the Gemini API, which supports stateful interactions allowing developers to maintain context across multiple requests, demonstrated through a creative storytelling session combined with image generation using the newly released Nano Banana Lights model. This model significantly reduces latency in image creation and editing, enhancing real-time interactive experiences.
Thor then discusses the recent release of Gemini’s live translation model, which offers near real-time multilingual translation. Attendees are invited to try a live translation session via a QR code, illustrating how the model automatically detects spoken language and translates it into the listener’s preferred language with minimal delay. This feature is particularly valuable for international audiences, enabling seamless communication across languages during live events. The live translation model supports around 90 languages and is accessible through the Gemini API, encouraging developers to integrate it into their applications.
The presentation moves on to Gemini 3.1 Flash Live, the latest model designed for real-time conversational agents. Thor demonstrates a live jukebox application where the agent acts as a radio DJ, generating music on demand using the Lyra 3 music generation model. This example showcases the model’s ability to handle multilingual conversations, tool invocation, and real-time audio responses. The Live API, which underpins this functionality, is a low-latency WebSocket interface that supports streaming text, audio, and video, enabling developers to build sophisticated multimodal conversational experiences with native audio understanding.
Thor highlights the advantages of Gemini 3.1 Flash Live, including its native audio processing that bypasses traditional text-based pipelines, allowing for natural speech comprehension and response in multiple languages. The model supports interruptions, tool integrations, and audio transcription, making it highly adaptable for real-time applications. He also mentions partnerships with companies like LifeKit and PipeKit that facilitate scaling real-time media applications. Developers are encouraged to explore these tools to accelerate their projects using the Gemini Live API.
To conclude, Thor invites attendees to experiment with Gemini models via Google AI Studio at ai.studio/live and ai.dev/live, where they can interact with live translation and conversational models, customize voices, and enable Google Search integration for real-time information retrieval. He demonstrates a multilingual conversation with the agent, showcasing its ability to understand and respond in different languages, including Chinese and German. Thor wraps up by promoting a follow-up workshop focused on live conversational agents and multimodal AI, encouraging developers to engage with the Gemini ecosystem and explore its potential for building the future of AI-driven communication.
