Kenny Workman of LatchBio discusses the creation of verifiable AI agents designed to handle complex biological data, particularly in spatial biology, by developing specialized benchmarks like Spatial Bench to ensure accuracy, interaction with real data, and safety. LatchBio combines research, product development, and collaboration to advance AI-driven biological discovery, expanding into various omics fields and emphasizing verifiability, robustness, and biosecurity in their tools.
Kenny Workman, co-founder and CTO of LatchBio, presents on the development of verifiable AI environments tailored for biology, particularly focusing on spatial biology. He begins by highlighting the exponential growth of biological data generated from experimental techniques such as single-cell biology, spatial biology, and proteomics. These experiments produce massive datasets, often exceeding what a typical scientist can manage on personal computing devices. This data explosion motivates the need for AI agents capable of handling complex biological data analysis, akin to how code serves as a verifiable substrate in software engineering.
LatchBio initially started as a data tool vendor for biotech and pharma, focusing on storing and transforming large experimental datasets. Over time, they shifted towards building AI agents that assist scientists by interacting with complex biological data, especially spatial biology data. These agents use chat interfaces to help scientists explore questions such as gene expression differences in tissue samples. Although early prototypes were imperfect, they demonstrated the potential for AI agents to augment biological research by orchestrating complex workflows and collaborating like teams of scientists.
To evaluate and improve these AI agents, LatchBio developed a specialized benchmark called Spatial Bench, consisting of 146 problems spanning various spatial biology techniques and tasks. Existing benchmarks were insufficient as they did not capture the nuanced, multi-step nature of biological data analysis. The benchmark emphasizes verifiability, durability, and interaction with real data rather than relying on memorized knowledge. Human verification became crucial due to the inherent ambiguity and complexity of biological tasks, revealing challenges such as ambiguous problem statements and arbitrary thresholds commonly used in bioinformatics.
Expanding beyond spatial biology, LatchBio has extended their benchmarking efforts to other omics fields like single-cell and epigenomics, as well as complex areas such as drug discovery and preclinical pharmacology. They have also incorporated biosecurity considerations by collaborating with specialized companies to address potential misuse of biological knowledge. Their benchmarks include routine scientific tasks and more challenging “red team” tasks designed to test AI safety and robustness. These efforts have gained recognition in the AI and biology communities, with benchmarks being adopted by leading organizations like Anthropic.
In conclusion, LatchBio operates as a research and deployment lab focused on building and refining AI agents for biological research. Their approach integrates product development, benchmarking, and collaboration with experimental kit manufacturers to drive progress. The company is experiencing growth and actively hiring across engineering and scientific disciplines. Workman invites interested individuals to engage with their work, emphasizing the exciting potential of AI to transform biological discovery through verifiable, agent-driven environments.