In this video, Legare introduces how large language models (LLMs) can be extended beyond simple conversation to perform real actions in the digital world. He illustrates this with an example where a user might type a command like “Summarize this PDF and store the results in an S3 bucket,” and the system automatically coordinates multiple tools—such as extraction, summarization, and storage—behind the scenes to fulfill the request. This demonstrates how LLMs can be integrated with external services to accomplish complex tasks seamlessly.
Legare explains that while LLMs are powerful at understanding and generating language based on learned patterns, they are not inherently capable of precise computations or actions. For example, if asked to calculate “233 divided by 7,” an LLM would only guess based on patterns rather than performing actual math. To overcome this limitation, the system needs to call external APIs, such as a calculator API, to perform accurate computations. This concept can be scaled so that the LLM assistant can interact with any microservice—whether it’s a database, cloud storage, or a document summarizer—by recognizing when a tool is required.
The video then breaks down the architecture of a tool orchestrator, which enables an LLM to safely and reliably call external APIs. The first step is detecting when a tool call is necessary. This is achieved by fine-tuning the model on synthetic examples that include semantic cues like “calculate,” “translate,” “fetch,” or “upload,” signaling that an external tool should be used. Techniques such as few-shot prompting or taxonomy-based data generation help reinforce this understanding in the model.
Once the need for a tool is detected, the LLM generates a structured function call by referencing a function registry. This registry acts like a phone book, containing metadata about available tools, including their endpoint URLs, authentication methods, input/output schemas, and execution contexts. The registry can be implemented using YAML or JSON manifest files, microservice catalogs, or Kubernetes custom resources. The LLM uses this registry to create a schema that matches the requirements of the chosen tool.
Finally, the function call is executed in an isolated runtime environment, such as a Docker container or Kubernetes job, ensuring safety, scalability, and error handling without exposing the model directly to the internet. The tool’s response is then serialized and injected back into the LLM’s context, a process called return injection. This allows the model to incorporate the tool’s output into the conversation naturally, enabling it to provide accurate answers or confirm actions without breaking the flow. This orchestration transforms the LLM from a word predictor into an action executor capable of interacting with the digital ecosystem.
