Anthropic has launched two new models, Claude 3.5 Sonet and Claude 3.5 ha coup, along with a feature called “computer use,” which allows the AI to control a user’s computer to automate tasks like filling out forms. Demonstrations showcase Claude’s improved coding capabilities and its potential to streamline workflows, although challenges remain in accuracy and reliability for critical tasks.
Anthropic has recently launched two new models, Claude 3.5 Sonet and Claude 3.5 ha coup, along with an innovative feature called “computer use.” The Claude 3.5 Sonet model is an upgrade over its predecessor, Claude 3.5 Sonet, boasting significant improvements in coding capabilities, which is a key area where Claude has already excelled. The new model has shown better performance in various benchmarks, particularly in graduate-level reasoning and math problem-solving, although it still trails behind Gemini 1.5 Pro in some areas. The Claude 3.5 ha coup model is a smaller version that reportedly outperforms the larger Claude 3 Opus model.
The “computer use” feature allows Claude to control a user’s computer through prompts, enabling it to perform tasks such as filling out forms or managing data across applications. This feature is currently available via API for developers, marking a unique offering in the AI landscape. While similar projects have been attempted in the past, Anthropics emphasizes that their implementation is experimental and may not be reliable for critical tasks. The potential for this feature to automate mundane tasks is significant, as it could streamline workflows and reduce the need for manual input.
In a demonstration, Claude was tasked with filling out a vendor request form by gathering data from a spreadsheet and a CRM. The AI successfully navigated through the applications, taking screenshots and transferring the necessary information without human intervention. This showcases the potential for AI to handle repetitive tasks that typically consume a lot of time, hinting at a future where the interface between humans and computers becomes less pronounced as AI takes on more responsibilities.
The video also highlights the challenges of the computer use feature, particularly regarding the accuracy of the coordinate system used for mouse movements. The AI relies on pixel counting to navigate the screen, which can be imprecise. Anthropics acknowledges these limitations and suggests that the current implementation is a stepping stone toward more sophisticated AI interactions with computer systems. The ultimate goal is to create an operating system designed specifically for AI, allowing for more seamless integration and functionality.
In addition to the computer use feature, the video showcases Claude’s capabilities in coding tasks. In a separate demonstration, Claude was able to open a web browser, create a personal homepage, and troubleshoot coding errors by interacting with the VS Code environment. This illustrates the potential for AI to not only assist in coding but also to autonomously manage the entire coding process, from writing to debugging. Overall, the advancements in Claude’s capabilities signal a significant step forward in AI technology, with the promise of more intuitive and efficient human-computer interactions in the future.