From 36% to 100%: How Self-Improving Agents Write Their Own Skills — Rafal Wilinski, Runlayer

Rafal Wilinski discusses how self-improving AI agents, supported by structured “skills” and a unified Model-Client Protocol (MCP), can autonomously learn from experiences to continuously enhance their performance and collectively improve organizational knowledge sharing. This approach preserves procedural knowledge as a competitive advantage, enabling companies to achieve higher efficiency, reduce costs, and create a sustainable cycle of AI-driven innovation and expertise.

Rafal Wilinski, a founding engineer at Runlayer, presents the concept of self-improving agents that not only enhance their own skills but also collectively improve an entire organization through network effects. He begins by illustrating the ideal scenario where every complex problem solved by an expert is documented and shared automatically across the company, preventing redundant efforts. However, in reality, this knowledge often disappears after the task is completed. Wilinski emphasizes that with current technology, particularly through the use of “skills”—structured instructions that agents can access progressively—this vision is achievable. Skills help agents navigate complex tasks efficiently by providing relevant procedural knowledge, which is crucial as AI models become more capable but also prone to getting stuck in unproductive loops without proper guidance.

Wilinski identifies three main challenges with skills today: they are often developer-centric and difficult for non-technical users to create or distribute; creating effective skills requires significant effort and is often neglected; and not all AI clients support skills uniformly, limiting their accessibility across different departments. To address these issues, he highlights the potential of the Model-Client Protocol (MCP) as a unified distribution layer for skills. MCP allows centralized management and dissemination of skills across an organization, making it easier for both technical and non-technical users to access and contribute to the shared knowledge base. This approach also enhances security and policy enforcement, ensuring that the right skills reach the right agents.

A key part of Wilinski’s talk focuses on how agents can autonomously improve themselves by learning from both successes and failures. Drawing inspiration from the Voyager project in Minecraft, where agents explored and documented new procedures as reusable code, Runlayer has implemented a similar system. Their agents perform tasks, and when they succeed or encounter interesting outcomes, these experiences are distilled into reusable skills by a specialized LLM. This continuous learning loop allows the agents to refine their skills over time, increasing success rates dramatically—as demonstrated by improving a Chromium fork deployment success from 36% to 100%. This process not only enhances efficiency but also reduces costs by narrowing the search space for solutions.

Wilinski stresses the importance of preserving procedural knowledge as a form of organizational memory, especially since AI models themselves are transient and can degrade or become obsolete. By capturing and systematizing how work is done through skills, companies create a collective subconscious that is resilient to changes in model intelligence. This knowledge base acts as a competitive advantage or “moat,” as it is unique to each organization’s culture and operations and cannot be easily replicated by competitors. It also helps reduce reliance on constantly upgrading AI models by improving the effectiveness of existing ones through better guidance and context.

In conclusion, Wilinski envisions a future where companies leverage self-improving agents and a centralized skills ecosystem to create an organizational flywheel of continuous improvement. This system not only automates knowledge sharing and skill refinement but also empowers every agent and employee with the best available expertise. Runlayer is actively developing this “golden path” for AI implementation, combining skills, MCP, and autonomous learning to help organizations harness AI’s full potential while maintaining control and fostering innovation.

Useful Links