Meta AI has introduced two models in the Llama 3 series - an 8 billion parameter model and a 70 billion parameter model, with a 405 billion parameter model in development. These models have shown advancements over previous versions, with competitive performance in benchmarks and potential for future multimodal versions, though with certain licensing restrictions.
Meta AI has released two models of the Llama 3 series - an 8 billion parameter model and a 70 billion parameter model, with a 405 billion parameter model on the horizon. The smaller model is reported to outperform the larger model from the previous Llama 2 series, showcasing significant advancements. These models are available in both base and instruction-tuned formats, with a focus on text-only inputs for now, hinting at potential future multimodal versions.
The models have been trained on over 15 trillion tokens, a substantial amount compared to previous models in the field. Despite a lack of detailed technical reports, the models boast competitive performance in benchmarks compared to proprietary models like Gemini Pro 1.5 and Claude Sonnet. The benchmarks show promising results, especially in tasks like GSM A Marks, where the Llama 3 models excel.
However, the licensing terms for Llama 3 models have raised concerns, including restrictions on using the model to improve other large language models and requirements to include “Llama 3” in the name of fine-tuned models. Despite these restrictions, commercial use is allowed under certain conditions. The upcoming 405 billion parameter model is expected to rival GPT-4 in performance, with early checkpoint results showing promising outcomes.
Researchers and developers can access and experiment with Llama 3 models through platforms like Hugging Face, AMA, or by deploying their own instances. The models demonstrate proficiency in various tasks like role-playing, reasoning, creative writing, and code generation. While the models show promise in certain areas, there is room for further exploration and fine-tuning to maximize their potential across a wider range of applications.