This week in AI has seen significant advancements, including the launch of High 3D Gen for creating detailed 3D models from images, and HSMR for reconstructing human skeletons from videos. Additionally, innovative tools like Anime Gamer for interactive anime games, Skyreels A2 for video creation, and ByteDance’s Dream Actor M1 for animating characters highlight the rapid evolution of AI technologies, with more updates from OpenAI on the horizon.
This week in AI has seen remarkable advancements, including the introduction of a highly detailed 3D model generator called High 3D Gen. This AI can create intricate 3D models from a single image, capturing fine details and even estimating unseen aspects of the object. Comparisons with existing 3D generators like Hunyan 3D and Trellis show that High 3D Gen significantly outperforms them in terms of detail and accuracy. The tool uses a normal map and a diffusion model to generate 3D geometry, and a demo is available for users to try online.
Another innovative AI tool, HSMR (Human Skeleton Mesh Recovery), can create a 3D model of a person’s body, including their skeleton, from images or videos. This tool accurately estimates a person’s pose and movement, allowing for a 3D view from different angles. It utilizes a transformer model to analyze the input and reconstruct the skeleton, providing a more comprehensive understanding of body movements compared to previous tools. A demo and code are available for users to experiment with this technology.
Anime Gamer is an exciting new AI that can generate an interactive anime game based on text prompts. Users can control characters and environments in real-time, creating scenes and actions that affect character states like stamina and social energy. Although the video quality is currently limited, this concept hints at the future of AI-driven role-playing games where users can create dynamic narratives without predefined storylines or designs. The models and instructions for running this AI locally are also available.
Skywork AI’s Skyreels A2 can create videos by merging different reference images, allowing users to generate coherent scenes without filming. This tool can combine characters, objects, and backgrounds seamlessly, making it useful for creating commercials or music videos. Despite some noticeable flaws in the generated videos, the potential for creating content without the need for actors or extensive filming is significant. The models are open-source, enabling users to experiment with this technology.
Lastly, the video discusses the release of Dream Actor M1 by ByteDance, which can transfer the acting and movements of one person onto another image. This technology allows for the animation of both real and fictional characters, including the ability to animate historical figures. While this tool could disrupt the film industry by enabling anyone to create films from their home, it is currently only available as a technical paper with no open-source release. Additionally, updates on OpenAI’s upcoming models indicate a rapid acceleration in AI capabilities, with new releases expected soon.