This week in AI showcases major breakthroughs in video generation and enhancement, including real-time video creation, transparent layer editing, and advanced lip-syncing, alongside innovations in AI-piloted drones and improved forecasting models. Additionally, new tools like GPT-4 Turbo, Weather Lab, and PartC highlight rapid progress in AI across diverse fields such as robotics, weather prediction, and 3D modeling, with many open-source releases promoting wider accessibility.
This week in AI has seen remarkable advancements, particularly in video generation and enhancement technologies. Seed VR2, a new AI video restoration tool, can sharpen and add detail to low-quality videos up to 1080p in a single step, outperforming previous models. Alongside this, Any2 Bouquet offers a professional blur effect to videos, allowing users to simulate cinematic depth of field by selectively blurring backgrounds or foregrounds with customizable focus and blur strength. Both tools are open source and provide powerful, free options for video post-processing.
Another exciting development is Omnisync, a lip-sync AI that can match any input audio to the mouth movements in existing videos, enhancing realism for real people, cartoons, or AI characters. Meanwhile, LayerFlow introduces the ability to generate videos with transparent layers and even separate existing videos into foreground and background transparent layers, enabling seamless compositing and background generation. These tools open new creative possibilities for video editing and production.
A groundbreaking real-time video generator has been unveiled, capable of producing minute-long videos at 24 frames per second on a single high-end GPU. This model allows interactive control over the video content and camera movements, representing a significant leap from previous slow video generation methods. Complementing this, Player One Egocentric World Simulator creates first-person perspective videos that respond to a user’s physical movements, offering immersive applications for gaming and virtual reality.
In robotics, an AI-piloted drone has outperformed top human pilots in an international racing competition, demonstrating AI’s growing prowess in physical tasks. On the AI model front, OpenAI quietly released GPT-4 Turbo (03 Pro), which offers marginal improvements in reasoning and STEM tasks over its predecessor but at a higher cost and slower response time. Google DeepMind also launched Weather Lab, an AI tool that predicts tropical cyclone paths up to 15 days in advance with higher accuracy than existing models, providing valuable insights for disaster preparedness.
Finally, PartC is a novel AI that generates segmented 3D models from single images, even reconstructing hidden parts of scenes, which could revolutionize fields like interior design and 3D modeling. Many of these tools are open source or have released code, signaling a trend toward accessible, cutting-edge AI technologies. Overall, this week highlights the rapid pace of AI innovation across video, robotics, weather forecasting, and 3D modeling, promising exciting applications in the near future.