In this episode of the Stanford AI and Healthcare podcast series, hosts Matt Lungren and Justin Nordon are joined by Seth Hayne, Senior Vice President of R&D at Epic, to discuss the latest developments in AI models, particularly GPT-5, and their applications in healthcare. They begin by reflecting on the mixed reactions to GPT-5’s release, noting that while it shows improvements on standard benchmarks, the leap forward is more incremental than revolutionary. Seth shares a lighthearted anecdote about testing GPT-5’s capabilities with his son’s Magic: The Gathering deck, highlighting that despite advances, some complex tasks remain challenging for AI. The conversation emphasizes the importance of model selection and the emergence of specialized models tailored to healthcare needs.
The discussion then shifts to the limitations of current benchmarks used to evaluate AI in healthcare, particularly multiple-choice questions that may not fully capture the complexity of clinical reasoning or real-world application. Seth and Justin stress that while these benchmarks provide useful signals, they do not reflect the nuanced workflows and checks required in clinical settings. They highlight the need for collaboration between academia and industry to develop evaluation methods that better represent how AI tools will be used in practice, including the integration of citations and contextual data to ensure reliability and safety.
A significant portion of the conversation focuses on the challenges of integrating AI models into healthcare workflows. Seth explains that building effective AI-powered software requires more than just the model’s output; it demands domain expertise, rigorous software development practices, and thoughtful user experience design. They discuss the balance between leveraging AI’s capabilities and maintaining deterministic, high-quality outputs, cautioning against over-reliance on AI where it may introduce noise or errors. The hosts also explore the evolving role of AI in assisting clinicians, emphasizing that AI should augment rather than replace human judgment, and that the right tool must be used for the right task.
The podcast highlights Epic’s innovative work on the Cosmos project, which involves training large-scale models on de-identified patient data to better understand the language and temporal nature of healthcare events. Seth describes how this approach treats patient histories as narratives, using transformer architectures to predict future clinical events over varying time horizons. This research aims to create generalist models that can improve predictions for chronic disease management and healthcare system capacity planning. The conversation underscores the potential of combining domain expertise with advanced AI techniques to create more effective and scalable healthcare solutions.
Finally, the hosts discuss the broader implications of AI adoption in healthcare, including concerns about deskilling clinicians who rely heavily on AI assistance. They acknowledge that while AI can improve diagnostic accuracy and efficiency, there is a risk that clinicians may lose proficiency if they become too dependent on these tools. The episode concludes with reflections on the future of healthcare, envisioning a more personalized, data-driven care experience where AI facilitates better communication between patients and providers. Seth emphasizes the importance of incremental progress, user-centered design, and ongoing collaboration to realize this vision, while also addressing systemic challenges such as physician shortages and access to care.
