Last week, Google made a bold move by releasing Gemma 4, a large language model that is truly free and open source under the Apache 2.0 license. Unlike other models that come with restrictive licenses or require massive infrastructure to run, Gemma 4 is remarkably small and efficient. The model is compact enough to run on consumer GPUs, with an even smaller Edge version capable of running on devices like phones or Raspberry Pis, all while maintaining intelligence comparable to much larger models that typically demand data center-level hardware.
Compared to other open-source models like Meta’s Llama or OpenAI’s GPT OSS, Gemma 4 stands out due to its combination of being American-made, Apache 2.0 licensed, and exceptionally tiny. For instance, the 31 billion parameter version of Gemma 4 performs similarly to models like Kimmy K2.5 but requires only a 20 GB download and can run on a single RTX 4090 GPU. In contrast, Kimmy K2.5 demands over 600 GB of storage, at least 256 GB of RAM, and multiple high-end GPUs, making local deployment impractical for most users.
Google’s breakthrough with Gemma 4 is not just about shrinking the model size but addressing the real bottleneck in AI: memory bandwidth. Running large language models locally is limited by the speed and efficiency of reading model weights from GPU memory rather than raw CPU power. To tackle this, Google introduced Turbo Quant, a novel quantization technique that compresses model weights more effectively by transforming data into polar coordinates and applying the Johnson-Lindenstrauss transform. This approach reduces memory overhead while preserving the model’s performance, although the exact mathematical details are complex.
However, Turbo Quant is not the sole secret behind Gemma 4’s efficiency. Some versions of the model, indicated by an “E” in their names, use “effective parameters” through per-layer embeddings. This technique provides each neural network layer with its own mini embedding for each token, allowing information to be introduced precisely when needed rather than all at once. This innovation leads to a smaller, smarter, and more efficient model architecture, making Gemma 4 a solid all-around model suitable for local use and fine-tuning with personal data.
While Gemma 4 is impressive, it is not yet advanced enough to replace high-end coding tools. The video also highlights Code Rabbit, a sponsor, which recently launched a CLI update that enables automated code review and bug fixing for AI agents. This tool simplifies setup, removes rate limits, and integrates directly with agents to improve code quality before pull requests. Overall, Google’s release of Gemma 4 marks a significant step forward in accessible, efficient, and truly open-source AI models, potentially reshaping how developers and hobbyists interact with large language models.
