Joshua Snyder from PostHog presents a system that automates software development by transforming product signals—such as errors and user feedback—into actionable pull requests through the use of large language models and intelligent agents. This approach streamlines issue detection, diagnosis, and code fixing, aiming to deliver ready-to-merge PRs with minimal human intervention and ultimately enabling products to self-build and evolve autonomously.
Joshua Snyder from PostHog presents an innovative approach to software development where products essentially build themselves by automating the process from product signals to pull requests. PostHog collects vast amounts of product data through various tools like session replay, error tracking, and experiments. Traditionally, engineers analyze this data via dashboards to identify issues, create tickets, and eventually write code fixes—a process that can take hours or days. PostHog aims to streamline this by automatically detecting product signals, diagnosing problems, and generating pull requests (PRs) for fixes, reducing the need for manual intervention.
The pipeline developed by PostHog begins with ingesting a massive volume of diverse signals, such as errors, logs, and user feedback, while filtering out malicious or irrelevant data using a large language model (LLM) classifier. These signals are normalized into a consistent format and assigned importance weights. A key challenge is grouping these noisy signals into coherent problem reports. Initial attempts to cluster signals using embeddings failed due to structural differences across data types. The team overcame this by generating queries from signals via LLMs and then matching those queries in embedding space, enabling more accurate grouping of related issues.
Once grouped, the reports are analyzed by a research agent running in a sandbox environment, equipped with tools like PostHog’s MCP server, codebase context, and integrations with external platforms such as Linear and Notion. This agent synthesizes information to summarize the problem, prioritize it, and identify the appropriate code reviewers using Git blame. Reports are then assessed for actionability: some require more data, some need human input, and the best cases are immediately actionable, allowing the agent to write fixes and create PRs autonomously.
The execution phase involves cloning the user’s repository into a sandbox where the agent iteratively develops and tests code changes, pushing PRs that ideally pass continuous integration (CI) checks without human intervention. This approach aims to deliver ready-to-merge PRs by the time developers start their day, minimizing manual debugging and review. Throughout development, the team learned important lessons, including the necessity of testing on real production data, the importance of embedding the right data representations, the risk of noisy PRs from vague problem descriptions, and the value of investing in agent-driven iterations despite initial cost concerns.
Looking ahead, PostHog envisions a future where products self-build by automatically shipping experiments, measuring their impact, and deploying changes behind feature flags with minimal human oversight. The system will continuously learn from outcomes—whether PRs are accepted, rejected, or cause issues—to improve future automation. Joshua encourages developers with data-rich products to experiment with agents, highlighting their potential to transform how software is maintained and evolved by automating routine tasks and enabling engineers to focus on more exciting, high-impact work.