Startup dMatrix has partnered with NVIDIA to integrate its innovative Raptor XPU, featuring a unique 3D stacked DRAM architecture, into NVIDIA’s AI platforms to deliver ultra-low latency and efficient AI inference processing. This collaboration leverages NVIDIA’s extensive ecosystem to accelerate dMatrix’s market entry, targeting diverse customers across sectors needing rapid, real-time AI inference solutions.
Startup dMatrix has partnered with NVIDIA to integrate its specialized inference XPUs into NVIDIA’s platforms, targeting ultra-low latency AI workloads. This collaboration leverages NVIDIA’s extensive AI infrastructure, which is widely deployed across global cloud services and data centers. dMatrix’s technology works alongside NVIDIA GPUs using NVLink Fusion, enabling faster and more efficient AI inference processing.
Sid Sheth, CEO of dMatrix, highlighted that their memory-centric inference computing technology has been in development for over seven years, focusing on ultra-low latency inference. The demand for such technology has surged recently due to advancements in AI models like GPT codecs and agentic coding, which require rapid, low-latency compute to enable real-time interaction. The partnership with NVIDIA is expected to accelerate dMatrix’s go-to-market strategy by tapping into NVIDIA’s vast ecosystem and customer base.
Unlike traditional approaches that use High Bandwidth Memory (HBM), dMatrix’s Raptor XPU employs a novel three-dimensional stacked DRAM architecture directly integrated with compute units. This design addresses the critical memory bottleneck in AI workloads by improving latency, energy efficiency, and cost-effectiveness. The Raptor XPU is poised to be the first product of its kind in the market, with significant customer interest due to its unique capabilities.
The collaboration with NVIDIA not only provides engineering support but also enables dMatrix to leverage NVIDIA’s existing infrastructure, allowing for faster deployment of their technology. This strategic partnership is expected to bring the Raptor XPU to market within approximately twelve months, significantly quicker than if dMatrix were to go it alone. The integration with NVIDIA’s ecosystem is seen as a key factor in meeting the growing demand for ultra-low latency AI inference solutions.
Regarding customers, dMatrix plans to announce a diverse range of clients soon, including hyperscalers, frontier research labs, sovereign wealth funds involved in national AI projects, inference cloud providers, and high-frequency trading firms. This broad customer base reflects the wide applicability of dMatrix’s technology across various sectors that require cutting-edge AI inference capabilities.