In the video, Hanna Lichtenberg and Amir from Mixedbread AI discuss the challenge of bridging the “knowledge gap” between the rapidly improving reasoning capabilities of large language models (LLMs) and the relatively slow progress in search and retrieval technologies. They highlight that while LLMs like GPT-5.5 have shown exponential growth in reasoning, traditional search methods have not kept pace, limiting the models’ ability to access the right knowledge efficiently. This gap is evident in real-world benchmarks such as BrowseComp Plus and Office QA Pro, where models perform significantly worse when retrieving information from noisy or large corpora compared to an ideal “Oracle” scenario where the correct documents are provided.
The speakers explain that current LLMs often generate poor search queries because they are primarily trained for coding tasks or mimic human web search behavior, which tends to rely on keyword-heavy queries. These queries are not well-suited for advanced semantic search tools, leading to suboptimal retrieval results. Additionally, many benchmarks favor simple keyword matching approaches, reinforcing this ineffective querying style. To address this, Mixedbread AI developed a specialized search agent designed to use powerful search tools more effectively by generating precise, semantically rich queries tailored to different search needs.
Their search agent operates within a harness built on the Mixedbread platform, utilizing four main search tools: overview semantic search, main semantic search, metadata-based filtering, and a grep tool for exact keyword matches. The agent is designed to conduct up to four rounds of parallel searches, intelligently splitting the user query into distinct aspects and selecting the most appropriate tool for each. This approach allows the agent to explore the corpus efficiently, avoid redundant information, and produce a ranked list of relevant documents as evidence for answering user queries.
Training the agent involved supervised fine-tuning with a larger teacher LLM followed by on-policy reinforcement learning guided by a custom search reward. This reward combines retrieval quality metrics (like NDCG) and LLM-based judgments on relevance, ranking plausibility, query naturalness, and exploration sufficiency. The training encourages the agent to write concise, natural language queries and balance exploration with efficiency. Early results show significant improvements on benchmarks, with the agent outperforming previous models in retrieval quality and efficiency, demonstrating the potential of this approach to close the knowledge gap.
Finally, Hanna and Amir share that while the fully trained agent is not yet publicly released, their beta version is already leading on benchmarks such as Snowflake’s Match QA, achieving high accuracy with less computational effort. They emphasize that there is still considerable room for advancement in integrating LLMs with retrieval tools and invite interested researchers and engineers to join their efforts in pushing the boundaries of knowledge retrieval and agentic search.
