Beware: Real risks of using AI as a software dev team

The speaker, an engineering leader, shares his experience of integrating AI into a software development team, highlighting significant risks such as loss of critical understanding, feature bloat, security vulnerabilities, and increased costs that ultimately led him to stop using AI tools. He urges leaders to adopt AI thoughtfully, maintain core engineering practices, prioritize human collaboration, and avoid relying on AI as a cure-all, emphasizing that software can and should be built responsibly without overdependence on AI.

In this candid and thoughtful video, the speaker, an engineering leader, shares his personal journey and challenges integrating AI tools into a small software development team. Initially skeptical, he embraced AI with optimism but soon encountered numerous issues that led him to ultimately stop using AI in his workflow. His message is directed primarily at managers, engineering leaders, CTOs, and developers who may be grappling with the realities of AI adoption, highlighting that while AI can seem magical, it carries significant risks and downsides that are often overlooked.

One major concern he raises is the loss of understanding and critical thinking within teams as AI-generated code becomes prevalent. When developers rely heavily on AI to write features they do not fully comprehend, debugging and maintaining software becomes difficult, especially when AI tools fail or produce incorrect outputs. He also warns against the misuse of AI-generated communication, such as overly verbose or irrelevant pull request descriptions and messages, which can frustrate human collaborators and degrade team communication.

The speaker emphasizes the dangers of feature bloat, where AI’s ability to rapidly generate large amounts of code leads to software that is overloaded with unnecessary features, reducing usability and stability. He also highlights the risk of junior engineers becoming overly dependent on AI chatbots instead of engaging with human teammates, which undermines their learning and growth. Additionally, rising costs of AI services and the instability of subscription models pose financial risks, especially when teams lose core coding skills and productivity gains do not translate into increased revenue.

Other practical issues include AI’s poor code reuse habits, leading to duplicated and inconsistent code, and the phenomenon of automation blindness, where developers trust AI-generated code without sufficient scrutiny, increasing the risk of bugs and security vulnerabilities. The speaker also cautions about the security risks of AI tools accessing sensitive environment variables and running commands autonomously, which can cause serious damage if not carefully controlled. Furthermore, he critiques the quality of AI-generated tests and the pitfalls of measuring productivity by superficial metrics like lines of code or token usage rather than meaningful business outcomes.

Ultimately, the speaker calls for thoughtful, measured leadership in adopting AI, urging teams to maintain core engineering principles, foster human collaboration, and set clear policies around responsible AI use. He stresses that AI is not a silver bullet and that software can and should still be built without it. His experience serves as a cautionary tale about the real harms AI can bring to teams, products, businesses, and society, encouraging leaders to carefully weigh the benefits and risks rather than blindly following hype or pressure.

Useful Links