The video discusses a new function calling feature in the Olama product, enhancing tool use with structured prompts for model integration, while clarifying misconceptions from social media about its capabilities. Despite its potential benefits, the feature currently shows limitations and inconsistencies across various models, with the presenter recommending older methods for more reliable results while expressing hope for future improvements.
In the video, the presenter discusses a newly introduced feature in the Olama product that enables function calling and tool use, which was demoed on July 18th. This feature has generated excitement among users, particularly since similar functionality was previously available in OpenAI’s offerings. The presenter, who attended Olama’s one-year anniversary celebration, clarifies misconceptions arising from social media interactions regarding the current capabilities of the tool. Despite having been available since late 2022, the recent update enhances the process of function calling in a more structured way.
The new feature allows for the integration of functions and their outputs during the fine-tuning of specific models. By providing a well-defined prompt format, the model can recognize which tools to use and the associated parameters. This change aims to eliminate ambiguity in prompting and makes it easier to supply function definitions separately. However, the presenter notes that this updated method works best with a limited selection of models, and its reliability can vary.
When using Olama in debug mode, the presenter demonstrates how the API reformats tool definitions as part of the prompt for the model. The key difference with the new approach lies in its ability to specify JSON formatting more easily, although it still requires a suitable model to function correctly. Some models have shown inconsistent performance, leading the presenter to recommend sticking with older, more reliable methods for those needing consistent results.
The video also provides a practical demonstration of how to implement the new tool use feature by defining functions that interact with online APIs to fetch weather data based on location. The presenter illustrates how to set up the model and send messages to retrieve weather information, detailing the parameters required for each function. They highlight the importance of correctly identifying models that support this new functionality, which can be found on Olama’s website.
In conclusion, while the new function calling feature in Olama has potential benefits, it currently exhibits limitations and inconsistencies across various models. The presenter expresses optimism about future improvements to the product and urges users to provide feedback on any issues encountered. They also share links to previous videos that demonstrate older, more reliable methods of function calling, encouraging viewers to explore both approaches as the technology continues to evolve.