Microsoft’s open-source Phi-3 medium model, featuring 17 billion parameters, offers fast performance and favorable rankings compared to other models. While excelling in tasks like solving math problems and answering logic questions, the model faced challenges in coding tasks but showed potential for further improvement through continuous testing and fine-tuning.
Microsoft recently released the Phi-3 medium model, a 17 billion parameter model that is fast, performs well, and is open-source. The model comes in two versions - a 4K instruct and a 128k instruct. In comparison to other models, Phi-3 medium ranks favorably, performing better than some models like Mistol 8 * 22 and GPT-3.5 Turbo. The model was tested using open web UI and Olama, running locally on a MacBook Pro M2 Max. The initial inference run may take slightly longer as the model loads into memory, but subsequent runs are quicker.
During testing, Phi-3 medium was used to complete various tasks, such as writing Python scripts, solving math problems, and answering questions. The model struggled with coding tasks, like creating a Python snake game, due to errors in the generated code. However, it performed well in solving math problems, answering logic questions, and reasoning tasks. The model correctly answered questions about hotel charges, math problems, and reasoning questions about marbles and balls in different scenarios.
The user noted some odd output and formatting issues during testing, possibly attributed to quantization levels or fine-tuning. Despite these minor issues, Phi-3 medium provided satisfactory answers to most of the tasks presented. The user also reached out to Olama for support, emphasizing the importance of addressing any potential model inaccuracies promptly. The model’s vision capabilities were not tested as Phi-3 medium does not support vision tasks, but the user expressed interest in testing Phi-3 Vision in a future video.
Overall, the Phi-3 medium model demonstrated strong performance in various tasks, showcasing its capabilities in language processing and reasoning. While it encountered challenges in coding tasks, the model excelled in solving math problems, answering logic questions, and providing accurate responses to scenarios presented. The user highlighted the importance of continuous improvement and testing of AI models, as well as the potential for open-source models like Phi-3 medium to be fine-tuned for specific applications or to address any limitations in the base model.