Timothy Garenbat’s Open Computer project creates a secure, user-friendly platform that runs AI agents locally within isolated virtual machines, overcoming the limitations and risks of traditional agent harnesses and container solutions. By leveraging accessibility trees for efficient app interaction and supporting collaborative human-agent workflows, Open Computer enables practical, safe AI usage on personal machines and aims to evolve into a full operating system for isolated AI agents.
In this video, Timothy Garenbat, founder of Anything LLM, introduces his project called Open Computer, which aims to create a safe, user-friendly environment for AI agents to operate locally on personal machines. He begins by discussing the limitations of current agent harnesses—tools that enable AI agents to perform tasks by interacting with APIs or executing commands. These harnesses often require powerful cloud models and large context windows, making them unsuitable for local models with smaller capacities. Moreover, granting agents unrestricted access to a user’s computer poses significant security risks, as agents might accidentally or maliciously delete data or execute harmful commands.
To address these security concerns, Timothy explains why traditional container solutions like Docker are insufficient because containers share the host machine’s kernel, allowing potential escape by malicious software. Instead, he advocates for using virtual machines (VMs) that provide a fully isolated kernel environment, containing any damage within the VM itself. Although VMs offer better security, they are typically complex for average users to set up and manage. Open Computer aims to bridge this gap by wrapping VMs in an accessible user interface that allows non-technical users to interact with AI agents safely and intuitively, without needing to understand the underlying technical details.
A key innovation in Open Computer is its approach to agent interaction with software and the environment. Unlike traditional “computer use” methods that rely on inefficient screenshot-based navigation, Open Computer leverages accessibility trees and native app integration to interact with applications more efficiently and token-effectively. This enables agents to perform tasks such as filling out forms or using native apps without the overhead and unreliability of image-based clicking. The system also supports multiple lightweight VMs running simultaneously, with agents able to request human assistance through notifications when they encounter obstacles, fostering a collaborative human-agent workflow.
Timothy demonstrates Open Computer’s capabilities through examples, including generating a GIF of Nvidia’s stock price over ten years using a local large language model running on his DGX Spark machine. The agent efficiently executes commands, manages deliverables, and even produces verification reports. He also showcases the agent’s ability to browse the web, parse content efficiently, and generate detailed HTML and PDF reports. Furthermore, the system can install and manipulate native Linux applications, such as a BMI calculator, using the accessibility tree method, highlighting the platform’s versatility and token efficiency.
In conclusion, Open Computer represents a promising step toward making AI agents more accessible, secure, and practical for everyday users by combining the safety of virtual machines with a user-friendly interface and efficient interaction methods. Timothy envisions this project evolving into a full operating system that allows each AI agent to have its own isolated “computer,” eliminating the need for agents to control the user’s actual machine. The project is open source and part of the Anything LLM ecosystem, aiming to empower users with local AI capabilities that are both powerful and safe.