The video compares GPT-6 Astra Medium and Fable 5.1 Medium across five rigorous academic research tests, finding Astra slightly superior in accuracy, critical analysis, and adherence to strict criteria, while Fable excelled in breadth and visual presentation. Ultimately, Astra won by a narrow margin, offering a more cost-effective and precise option for research purposes.
The video compares two advanced AI models, GPT-6 Astra Medium and Fable 5.1 Medium, to determine which is better suited for academic and research purposes. The presenter conducted five rigorous tests: a literature search under strict criteria, error detection in a manuscript review, a P-hacking pressure test, analysis of messy teaching data, and a comprehensive literature review. Each model was scored out of five for each test, with a total possible score of 25 points. The goal was to evaluate their accuracy, critical thinking, and usefulness in real research scenarios.
In the first test, both models successfully identified eight randomized controlled trials meeting strict inclusion criteria. Astra excelled by providing all peer-reviewed studies and flagging reporting errors, while Fable included some newer trials but also two unreviewed working papers. Astra scored slightly higher for adhering to the constraints. In the manuscript review test, both models detected all planted errors and recommended rejection, with Astra providing additional statistical calculations and Fable offering severity ratings. This test ended in a tie, demonstrating both models’ strong critical review capabilities.
The P-hacking test assessed whether the models would unethically overstate weak data. Both Astra and Fable resisted overclaiming the intervention’s effectiveness, with Astra silently correcting inaccuracies and Fable explaining its reasoning with numbers. Both scored well but did not fully excel, showing cautious but not perfect handling of questionable data. When given messy teaching data, both models identified data inconsistencies and Simpson’s paradox, with Astra slightly outperforming Fable by better highlighting problematic data rows and providing clearer recommendations.
For the final literature review test, Astra produced a detailed review with 16 references and quantitative tables, but missed some recent papers and included one unverifiable reference, indicating some hallucination. Fable generated a broader, more current review with visual graphs and detailed tables, though it had minor inaccuracies in data figures. Despite Fable’s impressive presentation and breadth, Astra’s critical approach and accuracy gave it a narrow edge overall. The final scores placed Astra as the winner by one point, surprising given the presenter had paid extra for Fable.
In conclusion, while both GPT-6 Astra and Fable 5.1 demonstrated strong capabilities for academic research tasks, Astra slightly outperformed Fable in accuracy, critical analysis, and adherence to strict research criteria. Fable impressed with its visual outputs and breadth of information but was less precise in some areas. The presenter noted that Astra comes with a standard subscription, making it a more cost-effective choice. The video encourages viewers to share their preferences and experiences with these AI tools in academic contexts.