[ML News] Chips, Robots, and Models

The video transcription covers various industry news, including Meta’s new chip for meta training, Google Deep Mind’s low-cost robots, Apple’s AI training data deal, and Justin Trudeau’s AI investment in Canada. It also discusses new models, data sets, and tools introduced by companies like Google Deep Mind, OpenAI, Apple, and community projects like ML Commons, highlighting advancements in AI research and applications.

In the video transcription, various industry news, models, data sets, and tools were discussed. Meta announced the release of a new chip designed for meta training and inference acceleration, featuring high teraflops per second and large memory capacities. Google Deep Mind presented Aloa Unleashed, a video showcasing low-cost robots trained to perform complex tasks with perception, coordination, and flexibility. Apple signed a deal with Shutterstock for AI training data, paying up to $2 per image, showing the value placed on high-quality training data. Additionally, Justin Trudeau announced a $2.4 billion investment in AI-related ventures in Canada to enhance computing capabilities and technical infrastructure.

Several new models and data sets were highlighted, such as Google Deep Mind’s release of Recurrent Gemma and Code Gemma models, as well as OpenAI’s announcement of an 8x22B model and tokenizers for structured conversation processing. The introduction of Wizard LM2, a fine-tuned version of the Wizard LM language model, and the release of iFiX 2, claimed to be the strongest vision language model below 10 billion parameters, were also discussed. Furthermore, advancements in models like RWKV6 and T5 and new variants of Zephyr and FeT UI by Apple were mentioned, showcasing a continuous evolution in NLP and vision language models.

Community projects and benchmarks like AI Safety Benchmark by ML Commons and Ruler for assessing real context sizes of long-context language models were introduced. Notably, the challenges with SWE Bench for software engineering models were highlighted, indicating potential data contamination issues. Additionally, the introduction of real-world QA by XAI for understanding physical spaces through questions and images and the launch of Whisper Kit V0.60 for localized model inference on M chips were noted. Lastly, the availability of TorchTune library for fine-tuning LLMs using PyTorch was discussed, offering a native approach for PyTorch users.

The inclusion of new tools like Reader for AI wearable, Whisper Kit for localized model inference, and TorchTune for PyTorch fine-tuning showcased the expanding ecosystem surrounding AI development. The emphasis on bringing models to M chips for faster local inference and the introduction of TorchTune as a PyTorch-based fine-tuning library reflected the trend towards more accessible and efficient AI development tools. The video highlighted the importance of diverse datasets, hardware advancements, and community-driven benchmarks in advancing AI research and applications, with a focus on facilitating improved understanding, training, and deployment of AI models. The overall content of the video provided a comprehensive overview of recent developments in the AI industry, spanning hardware innovations, model releases, data set introductions, and tool advancements.