Two CIOs from healthcare and financial sectors discuss the complexities of measuring AI’s true ROI, emphasizing that productivity gains often lack direct economic quantification and highlighting the differing value definitions between clinical improvements and enterprise efficiencies. They also explore cost management strategies, including in-house versus commercial AI solutions, token-based usage monitoring, data sovereignty concerns, and the need for balancing specialized vendor partnerships with internal capabilities to optimize AI adoption.
The video features a discussion between two CIOs from healthcare and financial services sectors about measuring the true return on investment (ROI) of AI technologies in enterprises. They highlight the challenge of quantifying productivity gains from AI, noting that while employees may become 20-40% more efficient, this improvement often does not translate directly into measurable economic value unless there is a structured approach to capturing and reinvesting those gains. The default scenario tends to be an implicit reinvestment of freed-up time, which is difficult to track and may not always add tangible value to the organization’s bottom line.
One CIO emphasizes that in healthcare, the definition of ROI differs significantly from traditional financial sectors. AI’s value is seen in clinical improvements such as early disease detection, enhanced radiology diagnostics, and freeing up doctors’ time through technologies like ambient transcription. These benefits are tangible and highly sought after by clinicians. However, on the enterprise side, the productivity gains are less clear-cut, likened to the adoption of email where efficiencies are realized over time without explicit measurement. The conversation also touches on the risk of AI-generated communication becoming overly verbose, complicating rather than simplifying workflows.
Cost considerations are a major theme, with the CIOs discussing the high expense of commercial AI tools versus building in-house solutions. One CIO notes that their internally developed tools cost about 10% of commercial equivalents, making the business case for AI adoption much stronger. They also discuss the concept of “tokenomics,” where AI usage is measured in tokens that represent computational resources, and the importance of managing token consumption carefully to control costs. Open-source models are presented as a cost-effective alternative for less critical workflows, while frontier models are preferred for high-stakes applications requiring greater accuracy and data integrity.
Data sovereignty and intellectual property concerns are also raised, particularly in healthcare, where sensitive biomedical research data must be protected. Running AI models in-house or on private infrastructure helps maintain control over proprietary information. However, managing such infrastructure is complex and resource-intensive, leading some organizations to prefer renting access to frontier models from cloud providers despite the challenges of scaling and fluctuating demand. The CIOs acknowledge that cloud providers face difficulties handling the highly variable and trend-driven usage patterns of AI models.
Finally, the discussion touches on the strategic decision between building AI capabilities internally versus buying commercial solutions. The healthcare CIO suggests leveraging specialized vendors for clinical AI products while maintaining internal environments for research and IP-sensitive work. Both agree that as AI adoption grows globally, the cost of models will need to decrease to sustain widespread use. Overall, the conversation underscores the nuanced and evolving nature of AI ROI measurement, cost management, and strategic deployment in enterprise settings.
Useful Links
