Etched, a new AI chip company backed by investors like Jane Street, has developed innovative rack-scale inference systems featuring low voltage chips and cluster scale memory to optimize AI model deployment with reduced power consumption and latency. With $800 million in funding and a team including former NVIDIA engineers, Etched aims to challenge NVIDIA’s dominance by delivering cost-effective, high-performance AI inference solutions ready for large-scale data center deployment.
Etched, a new player in the AI chip industry, is emerging from stealth mode with ambitions to challenge established giants like NVIDIA. The company focuses on building rack-scale inference systems designed specifically for running AI models, rather than training them. Etched has raised an impressive $800 million in initial funding, attracting notable backers such as Jane Street and Venture Tech Alliance, which is linked to TSMC. CEO Gavin Uberti introduced the company’s core technologies, including low voltage inference and cluster scale memory, which underpin their innovative approach to AI inference.
The company’s rack-scale system features 32 custom chips interconnected through cluster scale memory technology. This design enables ultra-low latency communication between chips, allowing them to share and utilize each other’s high bandwidth memory (HBM) and SRAM. This architecture is aimed at optimizing inference workloads by improving memory access and reducing bottlenecks, which is critical for efficient AI model deployment at scale. The system is preassembled in racks, making it ready for large-scale deployment in data centers.
Etched’s ability to secure substantial funding from highly technical investors like Jane Street is attributed to the founders’ deep understanding of the technology and the market potential. The company has been quietly developing its technology for over two years, reaching a stage where the rack-scale inference systems are operational and validated. Despite the high valuation of around $5 billion, Uberti emphasizes that the company’s proven technology justifies the investment and positions Etched well for future growth.
A key innovation highlighted by Uberti is the company’s low voltage inference technique, which allows their chips to operate at less than half the voltage of typical NVIDIA GPUs. This significantly reduces power consumption and thermal throttling, enabling more compute to be packed onto each chip. The result is a more cost-effective solution that can handle more users and tokens per chip, improving the overall economics for customers by lowering the cost per token processed. This approach leverages existing HBM and SRAM more efficiently, providing a competitive edge in inference workloads.
Looking ahead, Etched is already running benchmarks that demonstrate best-in-class performance and has engaged customers testing the system in production environments. The company’s team, which includes many former NVIDIA engineers and leaders, is a critical asset in driving the platform’s development and scaling. With a world-class platform in production and significant capital raised, Etched aims to scale rapidly and carve out a meaningful presence in the AI inference market, challenging NVIDIA’s current dominance.