This Is the Most Dangerous AI Coding Feature Yet

The video criticizes a new Cursor AI feature that lets developers approve code changes based on video demonstrations rather than reviewing the actual code, warning that this undermines software safety and accountability. The creator argues that shipping unreviewed, AI-generated code is unethical and dangerous, especially for applications where security and trust are critical.

The video discusses a new feature launched by Cursor, an AI coding tool, which allows developers to see a demonstration of what a feature looks like instead of reviewing the actual code changes (diffs). The creator expresses concern that Cursor has essentially admitted to not reviewing the code it ships, and that this new approach encourages developers to focus on the end result rather than the underlying code. The feature records a video of the AI agent using the mouse and keyboard to demonstrate the implemented feature, allowing developers to quickly evaluate if the feature works without ever looking at the code itself.

The creator presents a thought experiment to highlight the ethical dilemma: imagine developing a serious application, such as a cryptocurrency wallet, entirely through AI-generated code without ever reviewing it. If a customer asks whether the app is trustworthy, the developer cannot honestly answer, since neither they nor anyone else has reviewed the code. The only ethical response would be to admit ignorance about the app’s safety, which the creator argues is inherently immoral and dishonest if the code is shipped without human oversight.

The video critiques the trend of moving away from code review, emphasizing that code review is essential for ensuring the safety and reliability of software. The creator acknowledges the appeal of skipping code review for the sake of speed and automation, especially as AI agents can generate code much faster than humans can review it. However, this speed comes at the cost of transparency and accountability, as developers may lose track of what the code actually does.

The creator points out that while some may argue that the necessity of code review depends on the type of application—such as banking, finance, or even photo editing—every application has sensitive components, like authentication or data storage, where security is critical. If developers cannot confidently answer questions about the safety of user data because they have not reviewed the code, it undermines trust in the software. The creator also notes that large language models (LLMs) cannot be relied upon to verify code safety, as they are not capable of guaranteeing truthfulness.

In conclusion, the video warns that the push towards unreviewed, AI-generated code is dangerous, unethical, and potentially harmful. The creator urges viewers to consider whether society should allow unreviewed code to be deployed, especially in applications where security and trust are paramount. While open to counterarguments, the creator maintains that shipping code that no human has ever reviewed is irresponsible and poses significant risks to users and the broader software ecosystem.