# NVIDIA's 4B Model That Re-Runs Its Own Layers **URL:** \ **Category:** Content Creators\ **Tags:** loop-engineering, kai, machine-learning, nvidia, tech\ **Created:** [10 November 2026 15:30 UTC](https://www.artofsm.art/t/nvidias-4b-model-that-re-runs-its-own-layers/28059 "2026-11-10T15:30:06Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![artesia](https://www.artofsm.art/user_avatar/www.artofsm.art/artesia/32/36_2.png) [@artesia](https://www.artofsm.art/u/artesia)\ **Post date:** [10 November 2026 15:30 UTC](https://www.artofsm.art/t/nvidias-4b-model-that-re-runs-its-own-layers/28059/1 "2026-11-10T15:30:06Z")
[![](https://www.artofsm.art/uploads/default/original/3X/c/f/cf79bab15d4ffa32c69617f4a37fb53df6f0973a.jpeg "NVIDIA's 4B Model That Re-Runs Its Own Layers") ](https://www.youtube.com/watch?v=XK_hz11tmuc) NVIDIA’s 4 billion parameter model, Dory, improves reasoning and complex task performance by rerunning its middle layers multiple times per token, trading increased compute time for enhanced accuracy, especially in coding and multi-step reasoning tasks. While this looping approach boosts reasoning depth and efficiency on consumer GPUs, it does not enhance factual recall compared to larger models, requiring careful configuration of loops and caching to balance memory use and speed. ---
**Author:** ![artesia](https://www.artofsm.art/user_avatar/www.artofsm.art/artesia/32/36_2.png) [@artesia](https://www.artofsm.art/u/artesia)\ **Post date:** [10 November 2026 15:50 UTC](https://www.artofsm.art/t/nvidias-4b-model-that-re-runs-its-own-layers/28059/3 "2026-11-10T15:50:37Z")
NVIDIA recently released a 4 billion parameter AI model called Dory, which has garnered significant attention due to its impressive performance despite its relatively small size of 3.7 GB. Unlike traditional language models that process each token through all layers once, Dory employs a unique approach where its middle layers can be rerun multiple times—up to four loops—before producing each token. This looping mechanism allows the model to deepen its reasoning by revisiting the same parameters repeatedly, effectively trading off compute time for improved performance on complex tasks like code debugging and multi-step reasoning. The model was trained with this looping capability in mind, including reinforcement learning that directs harder tasks to more loops, making it adaptable and efficient for various workloads. Benchmark results show that Dory’s looping strategy yields substantial gains in reasoning-heavy tasks. For example, its score on the SWEBench verified coding benchmark improves from 50.3% at one loop to 61.8% at four loops, surpassing larger models like Google’s 26 billion parameter Gemma in certain areas. However, Dory does not improve on factual recall benchmarks regardless of loop count, highlighting a key limitation: looping enhances reasoning depth but cannot compensate for missing knowledge that larger models inherently possess. This distinction is crucial, as models like Gemma use a mixture of experts architecture to access a broader knowledge base, whereas Dory focuses on more deliberate, iterative reasoning with a fixed parameter set. Performance-wise, the looping mechanism increases memory reads per token, resulting in slower token generation rates as loops increase. On GPUs like the RTX 4090, four loops run about 2.4 times slower than a single loop, with similar slowdowns observed across different hardware. However, in agent-based tasks involving multi-step workflows, the increased reasoning per token reduces the total number of tokens generated, balancing out the slower speed and maintaining comparable overall runtime with improved output quality. Nvidia recommends using two loops as a practical compromise, capturing most of the accuracy benefits while keeping inference times reasonable for typical use cases. Memory usage is another important consideration, especially regarding the KV cache that stores attention keys and values. Each loop traditionally maintains its own cache, causing memory demands to multiply with more loops and longer context windows. Nvidia provides a “last loop” flag that enables cache sharing across loops, significantly reducing memory requirements and enabling larger context windows on GPUs with limited VRAM. This optimization is essential for users with 8 to 12 GB cards to run Dory effectively, as the default settings in some software packages can lead to excessive memory consumption and poor performance if not adjusted properly. In summary, Dory represents a novel approach to balancing model size, reasoning depth, and hardware constraints. It excels in tasks requiring iterative reasoning and multi-turn interactions, making it particularly suitable for coding agents and complex workflows on consumer-grade GPUs. However, it is not a universal replacement for larger models when factual knowledge and recall are paramount. Users must carefully configure loop counts, caching strategies, and context sizes to optimize performance. With ongoing software improvements like wavefront decoding, the speed penalty of looping is expected to decrease, further enhancing Dory’s appeal as a powerful, accessible AI model for those with limited GPU memory. ## Useful Links - [Dory model card (BF16)](https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16) — Central to understanding the model’s specifications, training, and performance claims. - [Dory GGUF (Q5KM) model file](https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-GGUF) — Essential for running the model locally and understanding quantization effects on performance. - [Paper: Dory1.0-4B: A Looped Normalized-GPT](https://arxiv.org/abs/2611.03998) — Primary source explaining the novel looping mechanism and training curriculum. - [Gemma 4 26B-A4B model card](https://huggingface.co/google/gemma-4-26B-A4B-it) — Provides context for performance and architectural comparisons with Dory. - [GitHub: chrisbennight/dory-vllm-experiments](https://github.com/chrisbennight/dory-vllm-experiments) — Demonstrates practical software optimizations reducing the speed penalty of looping.