The tutorial demonstrates how to build intelligent AI agents that combine internal documents and real-time web search—filtered by allowed domains—to provide accurate, well-cited responses using tools like Dockling and OpenAI’s APIs. It covers extracting and summarizing single web pages, performing domain-restricted searches, integrating internal knowledge bases, and unifying these components into a dynamic, interactive agent capable of reasoning and maintaining conversation context.
In this tutorial, the presenter demonstrates a common AI development pattern that combines internal knowledge, such as a company handbook, with real-time web search capabilities. This approach allows AI agents to access both predefined internal documents and dynamic external web content, either from specific URLs or broader web searches. The goal is to build intelligent agents that can decide whether to consult internal resources or perform web searches, filtered by allowed domains, to provide accurate and well-cited responses to user queries. The tutorial assumes familiarity with OpenAI, structured output, tools, and retrieval-augmented generation (RAG) pipelines.
The first part of the tutorial focuses on extracting content from a single web page. Using the Python library Dockling, the presenter shows how to convert HTML content from a URL into markdown format, which is easier for large language models (LLMs) to process. This markdown content is then summarized by an LLM, such as GPT-4.1 nano, to provide a concise overview of the page. This step is foundational for integrating web content into AI agents, enabling them to interpret and summarize specific web pages on demand.
Next, the tutorial covers broader web search functionality using OpenAI’s web search API. The presenter explains how to configure the agent to perform searches restricted to specific allowed domains, which is particularly useful for clients like government agencies that require focused and trustworthy sources. By using reasoning models like GPT-5 nano, the agent can dynamically explore search results, synthesize information, and provide answers with citations. This approach ensures transparency and reliability by including source URLs alongside the generated responses.
The third component involves integrating internal knowledge from documents like company handbooks. The presenter demonstrates loading a markdown handbook and creating a tool that the AI agent can call when it needs to reference internal policies or instructions. The agent intelligently decides when to use this tool based on the query, retrieves the relevant information, and generates answers with citations. This method complements the web search capabilities by ensuring that internal, authoritative knowledge is prioritized when appropriate.
Finally, the tutorial shows how to combine all these elements—single page extraction, web search, and internal knowledge—into a unified, dynamic AI agent. By organizing the code into modular tools and using clever prompting strategies, the agent can flexibly decide which resources to consult and how to synthesize information from multiple sources. The presenter also provides an example of an interactive terminal-based agent that maintains conversation history, allowing for more natural and continuous user interactions. This comprehensive approach equips developers with reusable patterns to build powerful AI assistants that leverage both internal and external knowledge effectively.