The DeepSeek V3.1 local AI model, tested on a high-end quad 3090 GPU rig, demonstrates strong coding abilities and improved complex reasoning but suffers from slow token generation and precision limitations. While not yet a “ChatGPT 5 killer” or true AGI, it represents a significant advancement for local AI users, especially in programming tasks, with recommendations to use smaller quantization models for better performance.
The video provides an in-depth review of running DeepSeek V3.1 locally using a quad 3090 GPU rig with 512GB of RAM, focusing on the Q4KM quantization model. The reviewer emphasizes consistency in testing by using the same questions repeatedly and recommends considering smaller quants like Q3 for better token processing speeds. They also highlight the importance of stable network connections when downloading large models, suggesting the use of Hugging Face’s downloader over the built-in Llama C++ downloader to avoid timeouts. The setup involves offloading layers to GPUs and managing memory constraints, with a context size capped at 4096 tokens to maintain stability.
Performance-wise, DeepSeek V3.1 shows mixed results. Token generation speeds are relatively slow, averaging around 3 to 4 tokens per second, which is consistent with other large models but may hinder practical use. The model demonstrates thoughtful reasoning, especially in puzzle-like questions, but struggles with precision tasks such as counting words accurately, indicating it is not yet at the level of “Garage AGI.” Despite some shortcomings, the model handles complex queries and reasoning better than many previous iterations, showing promise in its cognitive capabilities.
One of the standout features of DeepSeek V3.1 is its coding ability. The model successfully generates a highly detailed Flappy Bird clone in Python with impressive graphical elements like animated clouds, which the reviewer praises as the best code output seen so far in zero-shot scenarios. This coding proficiency marks a significant improvement and suggests that DeepSeek is particularly strong in programming tasks, making it a valuable tool for developers running local AI models. Other tasks, such as parsing and generating SVG animations, show mixed quality, with some outputs being impressive but occasionally flawed in layout or precision.
The review also touches on ethical and logical reasoning tests, where DeepSeek provides thoughtful, if somewhat cautious, responses. It handles ethical dilemmas by refusing to endorse harmful actions, aligning with expected AI safety protocols. The model performs well on straightforward math and logic problems, though some questions reveal limitations in complexity handling. Overall, DeepSeek V3.1 is seen as a solid and powerful local AI model, especially for users with high-end hardware, but it still faces challenges in speed and precision that prevent it from being a complete AGI solution.
In conclusion, DeepSeek V3.1 represents a significant step forward in local AI model development, particularly excelling in code generation and complex reasoning. While it is not yet the “ChatGPT 5 killer” or a true AGI, it offers a robust option for users with capable hardware setups. The reviewer encourages experimentation with smaller quants for better performance and highlights the availability of detailed guides for installation and use. DeepSeek’s improvements make it a noteworthy contender in the open-source AI landscape, though further enhancements in speed, precision, and multi-modality are needed to reach the next level.