Deepseek R1 671b Running LOCAL AI LLM is a ChatGPT Killer!

The video discusses the Deepseek R1 671b model running locally, highlighting its strengths and weaknesses, as well as the challenges faced during setup, particularly regarding RAM requirements and parallel processing performance. The presenter emphasizes the model’s potential for AI development while acknowledging that it is still a work in progress and encourages community engagement for troubleshooting and improvement.

In the video, the presenter discusses running the Deepseek R1 671b model locally, emphasizing that this may not be the most efficient method. They outline the strengths and weaknesses of the model while sharing their personal experiences and challenges encountered during the setup process. The presenter encourages viewers to engage in the comments, particularly those with technical expertise, to share insights or solutions to the issues faced. They also touch on the broader implications of Deepseek’s release, suggesting that the excitement in the market may be overstated, and highlight the open-source nature of the project as a potential pathway toward achieving artificial general intelligence (AGI).

The presenter explains the hardware setup, detailing the use of an R930 server with a significant amount of RAM, which is crucial for running such large models. They note that while the R930 is not the ideal choice for most users, it has unique capabilities that allow for extensive RAM configurations. The discussion includes the cost-effectiveness of building a system capable of handling the Deepseek model, emphasizing the importance of RAM size and configuration in optimizing performance. The presenter also reflects on their previous recommendations for building AI home servers, noting the advantages of specific motherboards that support higher RAM capacities.

As the video progresses, the presenter shares their troubleshooting efforts, particularly regarding the parallel processing settings that have led to slower performance than expected. They express frustration over the inability to adjust the parallel processing to improve token generation speeds, which significantly impacts the model’s efficiency. The presenter highlights the importance of understanding the system’s RAM demands and how it interacts with the model’s performance, indicating that they have spent considerable time attempting to resolve these issues.

The presenter conducts several tests with the Deepseek model, including a complex scenario involving ethical decision-making in a high-stakes situation. They report on the model’s performance, noting the time taken to generate responses and the quality of the answers provided. While some responses were satisfactory, others fell short of expectations, leading the presenter to conclude that while the Deepseek model shows promise, it does not yet represent a breakthrough in AGI capabilities. They emphasize that the model’s performance is still a work in progress and that further improvements are needed.

In conclusion, the presenter reflects on the implications of the Deepseek R1 671b model for the future of AI, acknowledging its potential while recognizing the challenges that remain. They reiterate the importance of community engagement in troubleshooting and improving the model’s performance. The video wraps up with a mention of upcoming AI developments and models, indicating that the landscape is rapidly evolving. The presenter encourages viewers to stay tuned for further updates and insights as they continue to explore the capabilities of local AI models.

How much ram did he have?

The presenter had a total of 1.5 terabytes (1500 gigabytes) of RAM in their system. They mentioned that they filled all 96 DIMM slots in the R930 server to achieve this capacity.

Did he quantize? And was he using CPU or GPU?

The presenter was running the Deepseek R1 671b model on CPUs, as they encountered challenges with GPU setups. They did not specifically mention whether they quantized the model, but they were focused on troubleshooting the CPU performance and parallel processing issues.

Does he mention the wattage when running the model?

The presenter did not mention the wattage consumed while running the Deepseek R1 671b model in the video.