Stanford University has introduced the Octopus v2, an on-device language model for super agents, which outperforms the GPT-4 in terms of accuracy and latency. The model is small in size and designed to run on various devices like phones, computers, and more. This development aligns with a trend towards creating on-device AI models to address concerns over privacy and cost associated with cloud-based models. The Octopus v2 demonstrates superior performance in function calling, a crucial aspect of AI agents, by decreasing context length by 95% and surpassing GPT-4 in both accuracy and latency.
The research presented by Stanford focuses on empowering on-device models with two billion parameters, which is relatively small compared to larger models like GPT-4. These on-device models have the potential to be deployed across a variety of edge devices such as smartphones, cars, and more. The Octopus v2 showcases its capabilities in tasks like creating calendar reminders, retrieving weather information, and sending text messages. The AI agent’s growing presence is highlighted, with advancements in agents like Multion and Adept AI, showcasing the use of language models to develop dependable software for users.
Stanford’s research methodology involved training the model using Google’s Gemma 2 billion model and comparing its performance with GPT-4, GPT-3.5, and other techniques like rag retrieval. The Octopus v2 model, along with Octopus v1 and Octopus 0, outperformed other models, including GPT-4, in terms of accuracy. Through techniques like low-rank adaptation and training on varying data set sizes, the Octopus models maintained high accuracy levels suitable for product deployment.
The study revealed that smaller models like Octopus can be highly effective in specific tasks, challenging the notion that progress in AI requires larger models with more parameters. Both Stanford’s Octopus v2 and Apple’s on-device model demonstrate that tiny agents can be fast, cost-effective, and accurate, even outperforming much larger models like GPT-4. This finding suggests that AI advancements can come from both increasing and decreasing model sizes, offering flexibility and efficiency in deploying AI agents across various devices and applications.
In conclusion, the emergence of on-device AI models like Octopus v2 represents a significant step towards addressing privacy and cost concerns associated with cloud-based models. These tiny agents showcase the potential for high performance in function calling and other tasks while being fast, accurate, and cost-effective. By demonstrating that smaller models can compete with and even outperform larger models, the research from Stanford and Apple highlights the importance of exploring diverse approaches to AI development, emphasizing efficiency and effectiveness in deploying AI agents across a range of devices and scenarios.
