Roey Ben Chaim from Zenity discusses the challenges of understanding agent skills’ true behavior due to hidden payloads and deceptive code, highlighting the limitations of static analysis. To combat this, his team developed a sandboxed detonation chamber that dynamically executes and monitors skills, uncovering numerous malicious tactics and emphasizing the need for rigorous inspection, least privilege access, and continuous monitoring in the AI agent ecosystem.
In this talk, Roey Ben Chaim, a staff engineer at Zenity, explores the challenges and insights gained from detonating thousands of agent skills within a controlled environment called the agent detonation chamber. Agent skills are essentially capabilities that can include instructions, scripts, and code, often written by third parties, making it difficult to discern their true intent. Roey highlights that simply reading the skill’s code or documentation is insufficient for understanding its behavior, as many skills load additional payloads from external sources, and language model analyzers can be easily deceived by cleverly crafted code or comments.
To address these challenges, Roey and his team developed a sandbox environment where skills could be executed safely and their behavior observed in real-time. They enhanced this environment with a “world builder” to simulate the necessary services and dependencies each skill might require, and created an operator to explore different execution paths within a skill. Despite these improvements, they encountered obstacles such as agents refusing to execute certain code due to built-in safeguards, which required adversarial techniques to bypass. They also implemented comprehensive monitoring to capture process creation, file system changes, and network activity to detect malicious behavior that static analysis would miss.
The team uncovered numerous malicious skills through this dynamic analysis approach. One example involved a skill that installed another skill which fetched hidden instructions from GitHub, demonstrating how static scanners fail to detect such multi-stage payloads. Another case involved a skill that used a non-official npm registry to install an info stealer. They also observed various malicious tactics including persistence mechanisms, silent reinstallation, remote prompt execution, and hook abuse, all designed to evade detection and maintain control over the agent environment.
A particularly alarming campaign involved an attacker impersonating the Paperclip open-source project by creating a fake organization and injecting malicious log actions that harvested credentials from sensitive paths like SSH and cloud service configurations. This payload was executed and self-destructed quickly to avoid detection. Despite rapid takedown efforts by platforms like Vercel and Microsoft, the malicious skills had already been downloaded over a million times, highlighting the scale and severity of the threat. Roey emphasizes that skills are just one part of the broader agent supply chain, which includes websites, databases, packages, and even user-generated content, all of which can introduce risks.
In conclusion, Roey stresses the importance of treating agent skills like package dependencies by thoroughly inspecting everything they load, limiting their access, applying least privilege principles, and continuously monitoring their behavior. The detonation chamber developed by Zenity provides a powerful tool for uncovering hidden malicious activity in agent skills. The research team’s work is ongoing and collaborative, and Roey invites viewers to test their own skills using the detonation chamber available at neoto.io, underscoring the critical need for vigilance and robust security practices in the evolving AI agent ecosystem.