The video discusses the upcoming release of the new AI model Llama 3-405B, which is a dense model being trained alongside larger versions like 8 billion and 70 billion. An unexpected update on WhatsApp suggests that some users have been prompted to try out Llama 3-405B, indicating that the model may soon be released and sparking discussions on the hardware requirements and cost implications of running such a massive model locally.
The video discusses the upcoming release of a new AI model called Llama 3-405B, which is a dense model being trained alongside versions 8 billion and 70 billion. While expectations were high for Llama 3-400B or 45B, the hype around Llama 3-70B has faded. However, recent developments indicate that Llama 3-405B may soon be released. An unexpected update came from WhatsApp, where some users were prompted to try out Llama 3-405B as a preview instead of the default Llama 3-70B for AI chat interactions. This suggests that Llama 3-405B is currently usable and may be close to completion.
The video highlights the confirmation of Llama 3-405B’s existence through various sources, including Android users and a Twitter account that tracks WhatsApp beta releases. The update was tagged by Meta, the company behind WhatsApp, indicating that it will allow users to choose the Meta AI Llama model in future app updates. This development suggests that Llama 3-405B is a real and tangible model that users may soon interact with. Speculations arise about the imminent release of the model, with some experts predicting an announcement within the week.
The video delves into the hardware requirements needed to run the large-scale Llama 3-405B model locally. Recommendations include a 12-channel AMD server board, high-end CPUs, memory, and multiple GPUs like RTX 490s or 309s. However, even with a costly setup, the expected token output is limited, emphasizing the challenges of running such a massive model on local hardware. The video also touches upon the potential cost implications of setting up a system capable of running Llama 3-405B locally, which could amount to tens of thousands of dollars.
Further discussions in the video mention the excitement surrounding the possibility of running Llama 3-405B locally, despite the significant financial investment required. The speaker expresses interest in exploring an ideal local system setup and hints at a potential collaboration with a sponsor to build the hardware. The video concludes by prompting viewers to share their thoughts on the release of Llama 3-405B, their local AI system setups, and their interest in running such advanced models locally. Overall, the video provides insights into the developments surrounding Llama 3-405B and the implications for AI enthusiasts and researchers.