Johan Lajili from Poolside AI discusses the varied experiences people have when using AI for coding, highlighting the divide between those who find AI incredibly helpful and those who encounter poor results, especially in legacy or “brownfield” applications. He explains that the key difference lies in the feedback loop available to the AI agent. In greenfield projects, where the environment is new and predictable, AI intuition tends to be accurate. However, in brownfield projects, unexpected complexities and legacy code can confuse the AI, leading to less reliable outcomes.
Johan emphasizes that the AI agent’s confidence in its work is based on the information and verification it can perform. If the agent cannot fully verify its changes, it may claim success even if the code does not function correctly. This discrepancy leads to skepticism among users who see AI-generated code failing in real-world scenarios. He suggests that the difference in user experiences stems from whether the AI can effectively test and validate its work, which is often lacking in complex legacy systems.
To address this, Johan introduces a CLI tool called Spoolside, developed at Poolside, designed to help AI agents better test and interact with applications. Spoolside enables the AI to take screenshots, capture snapshots of application states, extract logs, restart services, and perform high-level commands to navigate and interact with the app. This enhanced interaction allows the AI to reproduce bugs and verify fixes autonomously, increasing trust in its outputs and reducing the need for constant human oversight.
Johan argues that the future role of engineers will shift from solely building products to enabling AI to work effectively on those products. This involves creating tools, improving codebases, and building knowledge bases that facilitate AI understanding and testing. He envisions engineers becoming “AIX engineers,” focusing on making AI reliable and self-sufficient in software development workflows. This approach, although it may slow down initial development, will pay off by preventing compounded errors and enabling scalable AI-driven coding.
In conclusion, Johan stresses the importance of designing AI feedback loops and testing mechanisms that mimic human testing behavior rather than rigid automated tests. He encourages engineers to think creatively about how AI interacts with their specific products, whether games, permission-heavy apps, or others, to help AI detect issues independently. By investing in these AI-centric engineering practices, teams can harness AI’s potential more effectively and build trust in AI-assisted development processes.
