After five years of silence, IBM revealed a groundbreaking AI chip called North Pole that operates without any external memory like DRAM. Unlike conventional AI chips that constantly load and unload model weights from RAM for each token generated, North Pole stores the entire model within the chip itself. This unique architecture allows it to achieve remarkable speed and efficiency, outperforming even newer chips built on advanced 4nm nodes despite being manufactured on an older 12nm process. The chip was developed over 19 years with funding from the US Air Force, and IBM openly shared all its details, influencing startups and major companies like Nvidia.
North Pole’s design consists of 256 cores arranged in a grid, each with its own memory and math units, interconnected by thousands of wires. Interestingly, the chip lacks any form of conditional branching or dynamic decision-making during execution. Instead, all operations and data movements are pre-mapped by a compiler before the chip runs, making it incapable of training models or adapting based on intermediate results. While this might seem like a limitation, it aligns perfectly with the industry’s focus on inference rather than training, as inference consumes the majority of AI’s energy and resources.
Performance comparisons show North Pole excels in efficiency, delivering roughly 22 times more work per joule than tested GPUs on image recognition tasks. Despite its impressive capabilities, scaling the chip to handle larger models is challenging due to its lack of DRAM and reliance on SRAM, which is faster but much more space-consuming. For example, running an 8 billion parameter model requires 84 North Pole chips, and a 120 billion parameter model demands 440 chips across two racks. This high hardware requirement explains why the chip is not commercially available for purchase.
In November 2025, IBM demonstrated a full rack of 288 North Pole cards delivering exceptional throughput and bandwidth while consuming only about 30 kW of power. The rack uses standard PCIe Gen 3 interconnects instead of expensive alternatives, thanks to the chip’s architecture where weights never move, minimizing data transfer needs. However, preparing models for North Pole requires retraining them at low precision using GPUs, a process that can take weeks. Despite this, the retrained models maintain or slightly improve accuracy, validating the chip’s practical potential for inference workloads.
Ultimately, North Pole remains a research prototype rather than a commercial product. IBM continues to develop more conventional chips for near-term markets while viewing North Pole as a revolutionary technology for the future. Although you cannot buy the chip, it is accessible via IBM’s cloud services like Watson. Meanwhile, other companies push the boundaries of inference hardware, and some are even experimenting with using living human brains as computational devices, highlighting the diverse and rapidly evolving landscape of AI hardware innovation.
