Alistair Pullen of Cosine AI discusses the challenges and strategies involved in developing the UK’s first sovereign large language model, emphasizing the importance of architecture, data quality, and reinforcement learning to compete with well-funded US AI giants. He highlights innovative approaches like agentic engineering and government-backed access to domestic compute resources, underscoring the strategic necessity of sovereign AI amid geopolitical and supply chain constraints.
In this insightful discussion, Alistair Pullen of Cosine AI shares the journey and challenges of building the UK’s first sovereign large language model (LLM) amidst the global AI race. Cosine AI, originally focused on coding agents for highly regulated sectors like finance and defense, recently received government backing and access to the Isambard supercomputer cluster, enabling them to ambitiously pursue sovereign AI development. This support is crucial, as the compute resources required for training frontier AI models are immense and typically dominated by US-based companies with multi-billion dollar budgets. Despite limited resources compared to these giants, Cosine leverages a licensing model where customers deploy models on their own infrastructure, reducing inference costs and enabling a more feasible approach to building competitive AI.
Alistair emphasizes three critical factors for creating competitive models: architecture and raw model size, active parameter count, and data quality. He explains that many open-source or smaller-scale models fall short because they do not match the scale or architectural sophistication of leading models like Anthropic’s Opus or Mistral’s offerings. Moreover, data plays a pivotal role, especially post-training data and reinforcement learning (RL) from real user interactions, which help models better understand and respond to the messy, real-world prompts users provide. Cosine’s approach includes synthesizing RL training data from real engineering tasks, enabling models to learn from diverse and practical coding challenges beyond simple bug fixes.
The conversation also delves into the complexities of reinforcement learning, particularly the challenge of “slop” — where models produce technically correct but inefficient or overly complex code. Cosine is innovating by improving credit attribution within RL trajectories, allowing models to learn which specific parts of their output contribute positively or negatively to the final result. This targeted feedback enhances learning efficiency and helps models develop more elegant and maintainable solutions. However, Alistair acknowledges that fully removing humans from the loop in complex tasks like software engineering is still a distant goal, requiring advances in model capabilities, RL environments, and harness engineering.
Agentic engineering and the use of multiple specialized sub-agents or “swarms” are highlighted as powerful strategies to tackle complex problems by decomposing them into manageable subtasks. Cosine’s swarm approach orchestrates hundreds of agents working hierarchically, enabling the completion of sophisticated projects that single models cannot handle alone. While this introduces challenges such as agent interference and increased “understanding debt,” Cosine mitigates these through mechanisms like file locks and inter-agent communication. Memory management remains a difficult area, with current solutions often feeling like hacks; the team is exploring more elegant, latent-space-based memory approaches to improve agent performance and reliability.
Finally, Alistair reflects on the geopolitical and supply chain implications of sovereign AI development. The US export controls and hardware restrictions have accelerated the UK’s urgency to build independent AI capabilities. Cosine’s access to domestic compute infrastructure and government support positions them well, though challenges remain in hardware supply and scaling. Despite feeling the pressure of competing against better-funded US labs, Alistair expresses determination and optimism about making sovereign AI a reality. The conversation closes with a call to action for the UK AI community to rise to the challenge, underscoring the strategic importance of sovereign AI in the global landscape.