Phi-3: Tiny Open-Source Model BEATS Mixtral AND Fits On Your Phone!

Microsoft has released Phi-3, a small language model focusing on high-quality data training, with mini, small, and medium versions available. Phi-3 excels in running on a phone, achieving good performance on various tasks like answering questions and natural language processing, but has limitations in storing vast factual knowledge and is restricted to the English language.

Microsoft has released Phi-3, the third iteration of their small language model that focuses on high quality data training. The Phi-3 models come in mini, small, and medium versions, with the mini model being able to fit on a phone and achieve acceptable speeds in terms of tokens per second. These models are compared to other large models like Gemma 7B, Mistal 7B, and Mial 8*7B, showcasing their performance and size on a quality versus size chart.

The Phi-3 models are highly capable, with the mini version achieving a 69% score on MMLU and an 8.38 score on MTP Bench despite its small size. The innovation lies in the data set used for training, which is a scaled-up version of the one used for F2, composed of heavily filtered web data and synthetic data. The model is built upon a similar block structure as Llama 2 and uses the same tokenizer, allowing for easy adaptation of packages developed for the Llama family of models.

One of the key features of Phi-3 is its ability to run locally on a phone, making it suitable for various tasks that an AI assistant would typically perform. While the model may not have the capacity to store a vast amount of factual knowledge, it can excel when combined with tools, multi-agents, and access to the internet. The Phi-3 models are open-source, but the availability of weights for download is not clear.

In testing Phi-3 using LM Studio, the model demonstrated strengths in answering questions, logic and reasoning problems, and natural language to code tasks. It performed well in math problems, prediction tasks, and natural language processing challenges. However, there were some weaknesses identified, such as the model’s inability to store extensive factual knowledge and its restriction to the English language.

Overall, Phi-3 showcases impressive performance for its size and has the potential to enhance AI assistant capabilities on mobile devices. While it may not excel in all areas, the model’s ability to run locally and perform well on various tasks makes it a valuable addition to the field of language models.