The video investigates the claim that the VKORE 325M language model was trained entirely on specialized inference chips, revealing that while the laptop’s NPU is real and effective for certain image models, it cannot run the VKORE model due to software limitations. Additionally, despite similar loss scores between the NPU-trained and GPU-trained models, their outputs differ significantly, highlighting that matching loss does not ensure identical performance or accuracy.
The video begins by examining a claim from a lab that trained a language model called VKORE 325M, with 325.6 million parameters, entirely on a chip designed only for running models (inference), not training them. The training reportedly used 32 specialized inference chips over about four days and processed 5 billion tokens. The resulting model’s performance on unseen text was nearly identical to a version trained on a B200 GPU. However, the training method is patented and unreleased, making it impossible for others to reproduce the results. Importantly, the chips used were specialized inference hardware, not typical laptop NPUs.
Next, the presenter investigates the NPU in their own laptop, confirming its presence and functionality through Windows device status and driver updates. AMD’s software supports integer eight image models on this first-generation NPU but not language models. Running AMD’s benchmark test showed that the NPU processed 398 operators successfully, outperforming the CPU significantly in speed and latency on a small image model task. This confirms the laptop’s NPU is real and faster than the CPU for certain workloads, though limited to specific model types.
The presenter then attempts to run the VKORE model on their laptop’s NPU. While the model runs correctly on an RTX 4060 GPU with matching results to the official model card, exporting VKORE to ONNX and running it through AMD’s provider results in all operations falling back silently to the CPU, with none executed on the NPU. Performance benchmarks show the GPU processes about 18 tokens per second, while the CPU manages roughly 7.5 tokens per second. This demonstrates that the laptop’s NPU cannot currently run the VKORE language model with AMD’s existing software.
Further analysis compares the outputs of the NPU-trained model and the GPU-trained model on identical prompts. Despite having nearly identical loss scores, the two models produce different answers, such as the NPU-trained model incorrectly naming Montpellier as the capital of France, while the GPU-trained model correctly says Paris. This highlights that similar average loss values do not guarantee identical reasoning, factual accuracy, or behavior, as the models make different mistakes on a case-by-case basis.
In conclusion, the presenter confirms that while their laptop’s NPU is real and effective for certain image models, it cannot run the VKORE language model with current AMD software. The claim that the two VKORE checkpoints match in loss holds true on average, but the patented training method remains unreleased and unverified. The video encourages viewers to subscribe for more local AI hardware tests and invites suggestions for future chip evaluations.