The recent OpenAI Dev Day unveiled a new Realtime API for advanced speech-to-speech experiences, along with features for fine-tuning vision models and prompt caching to enhance AI development efficiency. Additionally, the departure of CTO Mira Moradi and other executives raised concerns about OpenAI’s leadership, while California’s Governor vetoed a proposed AI bill, and P Labs released Pika 1.5, showcasing significant advancements in animation.
The recent OpenAI Dev Day brought a series of exciting announcements, including the introduction of a new Realtime API that enables developers to create advanced speech-to-speech experiences. This API supports natural conversations using six preset voices and offers audio input and output capabilities in the chat completions API. The pricing structure for this service is designed to be affordable, with audio input costing approximately 6 cents per minute and output around 24 cents per minute, making it an attractive option for developers looking to integrate voice features into their applications.
Another significant update is the ability for developers to fine-tune vision models through the API, enhancing image understanding capabilities for various applications such as visual search, object detection, and medical image analysis. OpenAI is offering one million training tokens per day for free until August 2024, after which the cost will be $25 per million tokens. This feature aims to provide developers with more customization options to improve their AI applications.
OpenAI also introduced prompt caching, allowing developers to reuse previously seen input tokens to reduce costs and latency. This feature is particularly beneficial for applications that frequently use the same prompts, as it can lead to a 50% discount on processing times. Additionally, OpenAI announced model distillation, which enables the training of smaller, cost-efficient models using outputs from larger models, further enhancing the flexibility and efficiency of AI development.
In other news, OpenAI’s CTO, Mira Moradi, and several other high-level executives have departed from the company, raising questions about the future leadership and direction of OpenAI. Despite this talent drain, the company remains strong under Sam Altman’s leadership. Meanwhile, California’s Governor Gavin Newsom vetoed a proposed AI bill, citing concerns from industry experts about its premature and broad nature, which could hinder innovation in the AI space.
Lastly, P Labs released Pika 1.5, showcasing impressive advancements in animation that could rival Pixar-quality visuals. Additionally, there was controversy surrounding the open-source AI community, particularly involving a company called Pair AI, which faced backlash for mishandling licenses when forking an existing product. The founder acknowledged the mistakes and committed to rectifying them, highlighting the importance of transparency and adherence to open-source principles. New models, such as Liquid AI’s foundation models, were also introduced, demonstrating strong performance across various benchmarks, further enriching the landscape of AI development.