James Russo, co-creator and tech lead of Hyperframes at HeyGen, presents the idea that HTML is all agents need to create great videos. He begins by highlighting the challenge many face in launching products or features through compelling videos, despite the ease of building with coding agents. HeyGen’s mission is to solve communication through video, starting with creating high-quality AI avatars as the main subject or A-roll of videos. However, great videos require more than just avatars—they need B-roll, animations, captions, and music, all layers that contribute to effective storytelling.
Russo argues that HTML, along with CSS and JavaScript, is the native language of large language models (LLMs) because most of their training data comes from web pages built with these technologies. Teaching models new domain-specific languages or custom JSON formats is less effective, akin to asking Shakespeare to write poetry in a foreign language. By allowing LLMs to generate video content in their native “tongue,” developers can achieve better, more natural outputs. This approach aligns with recent industry thoughts from experts like Tarik and Andrej Karpathy, who see HTML as a new form of markdown for visual representation.
HeyGen developed Hyperframes, an open-source framework that converts HTML-based agent outputs into deterministic MP4 videos. Unlike traditional video editing tools like After Effects or Premiere Pro, which are not agent-friendly, Hyperframes leverages the browser’s rendering capabilities to produce videos frame-by-frame by freezing the clock and capturing each frame once fully loaded. This method ensures that everything visible in the browser is accurately represented in the final video, enabling the inclusion of complex web technologies like 3.js, WebGL, and Lottie animations.
To improve video quality, HeyGen focuses on teaching agents video-specific skills rather than coding languages, emphasizing taste and design principles. While single-shot prompts can generate decent videos, achieving professional results requires iterative refinement, storyboarding, and motion design—processes supported by Hyperframes’ studio tools, including recently added keyframe editing. This approach keeps humans in the loop, allowing creators to manually tweak videos as needed, blending AI efficiency with human creativity.
Since its release, Hyperframes has seen significant adoption, with over 1.3 million videos rendered by 267,000 creators in 90 days and 32,000 GitHub stars. The framework works with any coding agent capable of writing HTML, CSS, and JavaScript, enabling the same agents that build products to also create launch videos. Russo acknowledges that while models still struggle with creativity, HeyGen is collaborating with the community to raise video quality standards through initiatives like a code-to-video benchmark. He invites interested developers to engage with the project and explore the open-source tools available.
