The video discusses Jev, a novel AI model developed by Typesafe that has garnered significant attention despite not being a large language model (LLM). Unlike the generative focus of AI in recent years, Jev is a fast, typed classification model designed for ultra-fast decision-making with calibrated probabilities, meaning its confidence scores reliably indicate the likelihood of correctness. This calibration allows developers to integrate Jev into production systems with measurable trust, addressing a key limitation of LLMs whose probability outputs are often uncalibrated. Jev operates via an API, taking structured inputs like game state and questions, and outputs decisions rapidly, making it suitable for tasks requiring quick, reliable classification rather than natural language generation.
Jev is positioned as a “system one” model, drawing inspiration from Daniel Kahneman’s “Thinking Fast and Slow,” where system one represents fast, intuitive thinking. It complements rather than replaces LLMs, which remain essential for tasks involving text generation, chat, and complex reasoning (“system two”). Jev excels in scenarios like guardrails and tool selection, often matching or surpassing LLMs in benchmarks, especially in latency and cost efficiency. However, Jev has limitations, including a cap on parallel decisions (up to 255) and a token limit of about 32,000, beyond which performance degrades. Its closed-source, proprietary nature contrasts with the growing ecosystem of open-source alternatives.
The video highlights several open-source models, particularly the Gliner family and its variants like Gliner 2.5 and Gligard, which are smaller, faster, and runnable on consumer hardware. These models perform competitively with Jev on many benchmarks, sometimes outperforming it, especially in guardrail tasks. Unlike Jev, these open-source models lack guaranteed calibrated probabilities, which is a significant advantage of Jev. They use a different input-output schema involving labeled tokens and are bi-directional encoder models rather than auto-regressive. Despite some drawbacks, their speed and accessibility make them compelling alternatives for many applications.
A key insight from the video is the synergy between fast classification models like Jev or Gliner and slower, reasoning-capable LLMs. In practical tests with the video game Doom, fast models alone tend to behave tactically, focusing on immediate actions. When paired with an LLM acting as a strategic coach, the system can execute more complex plans, such as navigating to collect weapons or health packs, significantly improving performance. This mirrors the system one/system two framework and is already being explored in robotics. The local deployment of open-source models like Gliner offers much lower latency than Jev’s API-based approach, resulting in faster decision-making and better real-time responsiveness.
In conclusion, Jev represents an important step toward more interpretable, fast, and reliable AI decision-making models that complement LLMs rather than replace them. However, its proprietary nature and performance constraints open the door for open-source alternatives like Gliner to gain traction, especially given their speed and accessibility. The calibrated probability feature remains a standout advantage of Jev, but this may be replicated in open-source models soon. Overall, integrating these system one models into AI stacks, particularly for guardrails and tool selection, promises a less dystopian and more democratized AI future where intelligence is not monopolized by a few large companies but accessible and usable by many.
Useful Links
