In this session, Florina Muntenescu and Oli Gaymond from Google DeepMind discuss the integration and development of AI capabilities on Android devices. They introduce the concept of building intelligent experiences using on-device models, hybrid approaches, or fully cloud-based inference. On-device models, such as Gemini Nano accessed via the ML Kit GenAI APIs, allow for local processing of sensitive data without sending it to servers, enabling offline functionality and reducing inference costs. Gemini Nano is optimized for Android hardware and shared across apps through the AI Core system service, ensuring efficient use of resources and privacy by isolating app requests and not storing input or output data on the device.
The ML Kit GenAI APIs provide specialized functionalities like summarization, proofreading, and rewriting, with the most powerful being the prompt API that supports text and image inputs and text outputs. While Gemini Nano models are currently available on newer flagship devices like Pixel 9 and 10, developers can extend AI capabilities to other devices using Firebase AI Logic’s hybrid inference, which falls back to cloud models when on-device models are unavailable. This approach offers a consistent developer experience across device capabilities and access to more powerful cloud-based models like Gemini Flash and Gemini Pro Flash.
Addressing concerns about resource usage, the speakers acknowledge that running AI models impacts battery life and RAM, but emphasize that AI Core optimizes performance and manages system resources efficiently. They explain that typical user interactions with AI features are intermittent and not continuous, minimizing battery drain. For developers using custom models, profiling tools are available to assess resource consumption. The centralized AI Core system also manages multiple apps accessing the same model by queuing requests and prioritizing foreground apps, ensuring smooth performance and scalability without developers needing to handle complex scheduling.
The discussion also covers user-facing AI applications, clarifying that default Google Assistant queries likely run on cloud servers rather than locally. Developers interested in building local AI features can use the ML Kit GenAI APIs but would need to create their own prompt-based skills rather than modifying existing assistant capabilities. The speakers highlight that the APIs are designed as foundational tools for developers to build customized intelligent experiences, including retrieval-augmented generation (RAG) solutions, with plans to introduce embedding APIs soon to support tasks like text similarity and vectorization.
Finally, the presenters address the diversity of Android devices and AI model compatibility. While classical ML Kit models for vision and natural language run on a broad range of devices, the newer GenAI models require more recent flagship hardware. AI Core ensures that if a device supports these models, they will run efficiently, but developers targeting wider device compatibility may use Light RT LM for custom models, which requires additional testing and optimization. Overall, Google aims to provide a comprehensive, scalable AI platform on Android that balances performance, privacy, and developer ease of use.
