Lin Qiao, CEO of Firework, emphasizes the critical role of post-training in embedding a company’s unique knowledge and judgment into AI models to maintain competitive advantage and tailor AI performance to specific domains. She outlines the stages of post-training, practical challenges, and its growing importance for sustainable scaling, cost reduction, and creating specialized AI solutions beyond generic off-the-shelf models.
In this talk, Lin Qiao, CEO and co-founder of Firework, delves into the importance of post-training in building durable AI-powered businesses. She highlights the recent shift in software development where individuals can now build sophisticated applications quickly without deep coding expertise, leading to increased competition and a need for companies to embed their unique judgment and taste into AI models. Lin emphasizes that relying solely on off-the-shelf APIs risks losing a company’s distinctiveness, and post-training offers a way to bake in proprietary knowledge and customer understanding into AI systems.
Lin explains the concept of “owning intelligence, not renting,” which starts with curating high-quality, domain-specific data and progresses through various stages of model refinement. These stages include prompt engineering, retrieval-augmented generation (RAG), supervised fine-tuning, preference tuning, and reinforcement learning, each addressing different challenges such as dynamic data, personalized tastes, or domain expertise. She draws parallels between this process and human learning, where knowledge is acquired gradually and refined through experience and judgment.
The talk also covers practical challenges in post-training, such as ensuring data quality, building reliable evaluation metrics, avoiding pitfalls like reward hacking, and maintaining consistency between training and serving environments. Lin stresses the importance of involving product teams in data curation and evaluation to align AI outputs with business goals. She shares examples of companies like Cursor, Doximity, and Factory that have successfully implemented post-training to achieve superior model performance tailored to their unique needs, often surpassing frontier models in specialized domains.
Lin discusses the diverse types of organizations engaging in post-training, from frontier agent builders creating custom AI tools to large incumbents seeking cost-effective AI deployment. She notes that post-training is becoming essential for companies to maintain competitive advantage and manage operational costs, especially as AI feature adoption scales. The future, she predicts, will see millions of specialized models tailored to specific applications, underscoring the need for early experimentation and iterative development in post-training practices.
In the Q&A, Lin addresses when companies typically start post-training, explaining that it usually follows achieving product-market fit when meaningful, high-quality data becomes available. At this stage, companies focus on scaling sustainably by embedding their unique value into AI models and reducing costs through post-training. She highlights that post-training not only preserves a company’s competitive moat by encoding proprietary knowledge but also significantly lowers operational expenses, enabling support for higher traffic and healthier unit economics.