[BidClub_]
Latent Space · · 81 分钟

🔬超越 AlphaFold:Boltz 如何将药物发现的未来开源

Gabriele CorsoJeremy WohlwendBrandon

播客
TL;DR
  • AlphaFold 2 的里程碑式成果极具冲击力,但远没有达到“蛋白质折叠已被解决”:在具备进化线索时,它对单链蛋白结构的预测已好到惊人。 它仍无法解释折叠路径、中间状态、蛋白质运动或状态概率;Jeremy Wohlwend 的谨慎区分是,模型学会了“直奔答案”(go “straight to the answer”),而不是重建生物学如何抵达答案。AlphaFold 也让 Jeremy 从理论和方法型 ML 转向应用型 ML。

  • 结构生物学模型遵循与 LLM 不同的扩展规律:参数不到 10 亿,但推理成本异常高。 AlphaFold 2 只有约7000万至1亿个参数,但成对表示会带来立方级运算;专用架构仍“远胜”简单 transformer;把答案回收3至4次、某些时候几十次,则像是推理能力的早期形态。

  • AlphaFold 3 的闭源发布为 Boltz-1、最终也为这家公司打开了窗口。 DeepMind 曾开源 AlphaFold 2,据称已有超过100万名科学家使用,但随着 Isomorphic Labs 追逐制药业务,AlphaFold 3 被保留为闭源。Boltz 随后做出了完全开源、精度接近 AlphaFold 3 的模型;它只训练了一次,训练过程中一度进行实时“手术”,最终“靠某种奇迹……跑通了”。

  • Boltz 的商业判断是,开放权重并不会让完整产品商品化:价值会沉淀在科学工作流、加速基础设施、交互界面和验证环节。 Jeremy Wohlwend 表示,对真正工作的化学家和生物学家来说,“把模型放上 GitHub 绝对不够”;Boltz 计划继续开放基础模型,同时销售更易用、速度更快的服务,并从社区反馈中学习。

  • 技术引擎正从相信单次预测,转向生成、打分、搜索。 BoltzGen 联合设计蛋白质的原子结构和氨基酸身份,再用结构一致性、置信度以及日益成熟的直接亲和力预测筛选候选。创始人的商业化归纳很关键:只要足够多的样本里有一个好答案,“那本质上就只是一个排序问题”。

  • BoltzGen 最有力的证据来自有意覆盖广泛场景的湿实验验证,包括在已知训练交互之外的泛化能力。 针对9个没有已知蛋白质相互作用的 PDB 靶点,团队为每个靶点测试了15个 mini-proteins 和15个 nanobodies,在三分之二的靶点上获得纳摩尔级结合物——按他们的说法,这大致是治疗所需的结合强度;整个项目覆盖约25家外部实验室。

  • Boltz Lab 将这些模型转化为大规模并行设计活动,但创始人明确拒绝“AI 制药”的捷径。 一次活动可能筛选10万个候选,平台可以调动数千张 GPU;其小分子筛选管线据称比开源版本快10倍。但“这些不是药物”:成药性、ADME、毒性、细胞环境和实验室迭代仍不可或缺,真正能说服怀疑者的是命中结果,而不是看起来漂亮的预测分子。

摘要 · 为研究而整理的核心内容

1. AlphaFold 2 让职业轨迹转向,并攻克一个边界清晰的科学问题

  • Gabriele Corso 将结构生物学定义为:在不必总是解析晶体并进行 X 射线照射的情况下,以原子分辨率观察蛋白质及其他细胞分子。AlphaFold 2 证明,应用机器学习可以在一个长期悬而未决的科学问题上取得突破。Jeremy Wohlwend 表示,这让他从理论和方法型 ML 转向应用型 ML。

  • Gabriele 表示,他们的团队原本已在研究小分子,AlphaFold 出现后转向生物大分子。它回答了一个问题,却打开了更多问题:“除了蛋白质,我们能不能对其他东西也这么做?”这意味着要处理蛋白质—小分子相互作用、核酸、多链蛋白以及越来越复杂的分子系统。

  • CASP 14 提供了决定性测试:组织方特意汇集了与既有知识不同的结构,AlphaFold 2“彻底把所有东西都甩在了身后”。随后 CASP 15 和 CASP 16 扩展到其他模态,但这些方向的表现仍明显弱于单体蛋白。

2. 结构预测远比折叠或动力学前进得更远

  • Jeremy 不愿说这个问题已经“解决”。最准确的说法是,模型在预测单链蛋白结构方面已经非常出色,尤其是在拥有丰富进化信息时;如果缺乏这种协同进化景观,表现可能大幅下滑。

  • 结构预测和折叠是两项不同任务。如今的系统往往能从序列直接跳到一个合理的最终结构,但折叠研究的是蛋白质如何从无序状态进入该结构,包括中间状态;Jeremy 表示,“我不认为我们在后者上取得了多大进展”。

  • 蛋白质也不是静止物体:它们会在对应不同能量状态的多种形状之间运动。尚未解决的问题不仅是列出这些状态,还包括估计每种状态“以什么频率、什么概率”出现。

  • Gabriele 解释了这对疾病研究的重要性:错误折叠参与了某些疾病,而如果不了解这一过程,“我们就不知道该如何干预”。最终结构就像看到一辆组装完成的汽车;折叠路径则是汽车如何被制造出来,以及装配可能在哪里失败。

3. 进化突变将模型引入正确的结构能谷

  • 协同进化提供了强大的几何线索。如果一个氨基酸发生突变并扰乱附近的三维邻居,进化可能会在其他位置选择补偿性变化;因此,相关突变暗示两个序列位置在折叠结构中彼此接近。

  • Jeremy 将其比作一个巨大的三维能量谷,以及压倒性的搜索空间。进化线索能把模型带到“离答案相当近”的位置;此后,模型可能利用部分已学到的物理规律,向低能量状态下降——即使它无法独立走完整个景观。

  • 多序列比对(MSA)提供最初的接触线索。成对架构随后传播约束:如果两个残基必须接近,而其中一个又与另一个相连,更多距离关系就会受到约束;最后,系统将粗略的距离矩阵解码为坐标。

  • 这重新定义了一个老的理论难题:蛋白质折叠过去被视为经典的 NP-hard 组合问题,因为可能构象会随序列长度爆炸式增长。机器学习则从序列和进化中发现了人类无法显式提取的可用信号。

4. AlphaFold 3 通过生成式建模统一分子相互作用

  • AlphaFold 2 之后,显而易见的前沿是相互作用:蛋白质通过其他蛋白链、小分子、RNA 和 DNA 发挥功能,而治疗手段则通过改变这些关系进行干预。此前不同团队往往分别建模不同类型的相互作用;AlphaFold 3 则在一个大型系统中对它们进行统一训练。

  • 关键的概念转变,是从回归转向生成式建模。一个回归模型如果在多个结构之间犹豫,往往会把它们平均成一个可能很差的答案;生成式模型则从后验分布中采样多个备选方案,再交给独立的打分模型排序。

  • 这种分布式视角也为包含多个有效结构的动态系统提供了路径。不过,Gabriele 仍将其视为起点,而不是对分子运动或不确定性的完整处理。

  • 讨论中将更接近 transformer 的最终坐标生成组件,与专门的等变架构进行了对比,但 Jeremy 不接受纯粹的“苦涩教训”式解读。这个领域的工作共识是,专用架构仍远未被简单 transformer 追上,并能带来“远胜”的表现。

5. 生物学模型每个参数消耗的计算远超 LLM

  • AlphaFold 2 大约有7000万至1亿个参数,AlphaFold 3 也仍不到10亿个参数,按当代 LLM 标准看都很小。但运行这些模型可能更昂贵,因为其架构在成对表示上运算,把二次级工作量转化为立方级交互。

  • Jeremy 对比称,语言模型存储海量信息,而结构模型的任务是“解码这个输入”。模板可以充当数据库查询,但大量计算都用于解决空间约束,而不是记忆事实。

  • Recycling 增加了测试时深度:模型先建立关于成对相互作用的初步描述,再把结果反馈回输入并重新解码。标准运行通常回收3至4次;研究人员曾尝试几十轮,Gabriele 称其是“非常、非常早期的推理版本”。

6. AlphaFold 3 闭源催化了 Boltz-1

  • AlphaFold 2 是开源的,据称已覆盖超过100万名科学家。AlphaFold 3 则以一篇异常出色的论文出现,却没有发布模型;创始人认为,这与 DeepMind 分拆出 Isomorphic Labs、转向商业制药开发有关。

  • 这拿走了学术界和产业界原本期待继续搭建的基础。Gabriele 和 Jeremy 结合 AlphaFold 3 论文、此前的 diffusion 工作以及重建的数据管线,做出了 Boltz-1——他们称其为首个精度接近 AlphaFold 3 的完全开源模型。

  • 工作始于5月下旬,模型在11月问世。主要瓶颈不是想法,而是算力:他们的 Department of Energy 集群每次训练要跑2天,随后却要排队等1周,直到 Genesis 帮助他们完成训练。

  • 他们只训练了这个大模型一次。Gabriele 回忆说,他们曾停止模型、修复 bug,再重启训练,却没有回到起点;这个无法复现的模型经历了如此奇怪的训练课程,以至于“学到了一些奇怪的东西”,但“靠某种奇迹,居然跑通了”。

7. 基准测试防止亮眼均值掩盖脆弱泛化

  • Boltz-1 相比此前的开源模型实现了大幅跃升,但 Jeremy 承认,AlphaFold 3 在某些场景仍占优势,尤其是许多抗体—抗原预测。即使某个模型的总体表现更好,不同模型也可能在不同样本上胜出。

  • 结构预测相对容易进行回溯评估:只使用截止日期前发布的 Protein Data Bank 结构训练,再在更新、且刻意不同于训练集的结构上测试。问题在于,模型之间的比较必须建立在对各自训练数据有可信了解的基础上。

  • Gabriele 早期参与 DiffDock 的经历确立了这套工作循环。它曾在当时的标准基准上击败传统物理方法,但生物学家在陌生蛋白质上暴露出其失败;团队于是先建立更难的基准,再围绕泛化能力重新设计。

  • Gabriele 欢迎领域内的“健康怀疑”,因为许多失败仍然显而易见。他的乐观来自改进速度:如果近期这种速度能够延续,“未来某个时候,事情会变得相当酷”。

8. 开源同时构建研究飞轮和产品机会

  • Boltz 以民主化访问为目标,采用公益公司的组织形式,但 Jeremy 表示,学术 GitHub 仓库无法把模型真正送进治疗项目。科学家需要准备好的靶点、面向具体领域的工作流、可靠算力,以及无需配置 GPU 的交互界面。

  • 商业分工十分明确:继续发布基础模型,让研究人员改进模型并暴露失败模式;同时把最好的科学体验产品化。Jeremy 将其类比为:即使是 ML 专家,也会选择 ChatGPT 或 Claude Code,而不是亲自托管一个开源 LLM。

  • 一个拥有数千人的 Slack 社区逐渐实现了自我运转,研究人员也在 Boltz 之上发布扩展。Jeremy 认为,易用性是采用率的一部分原因:代码质量“远非完美”,但比许多替代方案更容易运行。

  • 贡献内容从一个可追溯至 AlphaFold 2 的架构组件所需的复杂 GPU kernel,到针对环肽的各种 hack。套件也已超越 Boltz-1:Boltz-2 增加了亲和力预测,估计相互作用强度,而不只是预测结构。

9. 推理时搜索正把预测变成排序问题

  • 社区成员 Tim O’Donnell 暴露了抗体—抗原预测中的一个有用失败模式。他每隔一段距离指定可能的结合残基——先是第1个、第11个、第21个,以此类推——让 Boltz 在这些条件下生成结构,再按置信度对结果排序;这种粗略扫描显著改善了结果。

  • 创始人的反应揭示了研究价值:“模型为什么会这么蠢?”但暴力搜索的成功提出了更好的问题:如何让搜索变得系统且高效,而不是期待一次无条件预测就找到正确结合位点。

  • 创始人将这一机制推广开来:只要采样足够多的候选结构,并假设其中一部分包含好答案,瓶颈就变成识别它。于是,“本质上就只是一个排序问题”,更好的打分模型可能成为下一轮重大提升的路径。

10. BoltzGen 联合设计蛋白质序列和原子结构

  • BoltzGen 将结构预测器视为面向特定领域的基础模型。用户提供一个靶点——蛋白质、DNA 或 RNA——以及抗体框架或肽等高层规格;空白 token 则代表待设计的蛋白质。

  • Diffusion 模型同时生成新蛋白质的结构和氨基酸身份。由于不同氨基酸具有不同的原子组成,对原子位置进行监督,就能隐式揭示每个位置对应的氨基酸,从而无需分别优化离散序列目标和连续结构目标。

  • 评估从一致性开始:Boltz-2 独立预测生成序列应如何折叠,再将该结果与 BoltzGen 设计的结构进行比较。两者一致,会提高对该序列能够实现目标设计的信心。

  • 但置信度本身“并不是一个很好的亲和力预测器”。Gabriele 表示,后续工作通过直接预测结合亲和力,而不是将结构置信度作为代理指标,提升了命中率;新的结果预计将公布。

11. 湿实验命中率,而非模型外观,才是平台的锚点

  • Boltz 既不是生物实验室,也不是治疗公司,因此 Hannes Stärk 协调约25家学术和产业实验室,在肽、无序蛋白、小分子结合蛋白、nanobodies 以及具有治疗意义的靶点上测试设计。BoltzGen 论文已经包含约8至10家实验室的结果。

  • 最干净的泛化测试选取了9个 PDB 靶点,这些靶点在数据库中没有已知相互作用,限制了模型模仿熟悉结合物的能力。每个靶点测试15个 mini-proteins 和15个 nanobodies,三分之二的靶点产生了纳摩尔级结合物;创始人称,纳摩尔级结合大致相当于治疗所需的亲和力。

  • Boltz Lab 将靶点准备、蛋白质和小分子“agents”、基础设施、API 访问以及协作界面打包在一起。一次活动可能生成10万个候选;并行运行可以替代数周的串行推理,托管版小分子筛选据称比开源管线快10倍。

  • 服务面向学术机构、初创公司、生物科技公司和大型企业,早期用户可获得免费额度,大型组织则可以采用安全部署。但 Jeremy 的边界是绝对的:“这些不是药物。”ADME、成药性、毒性、细胞通路和反复实验结果,仍然决定一个结合物能否成为获批疗法。

  • 内部药物化学家最初高度怀疑,并提出大量修改要求,后来却成了团队算力使用最重的人:他在数百张 GPU 上并行运行不同假设,并运用专业判断,而不是盲选排名前15的结果。Gabriele 的采用标准是实证性的:“要让人相信,你必须拿出他们原本认为不可能的东西。”

  • 最终说服人的,是湿实验结果回传之后的命中。Jeremy 和 Brandon 强调,科学家会被命中结果说服,而不只是被外观漂亮的预测分子吸引。Jeremy 还表示,广泛且重复的 CRO 测试有助于追踪平台逐月进步,同时避免对单一治疗项目过拟合。

Gabriele Corso

Actually, we only trained the big model once. That’s how much compute we had. We could only train it once, and while the model was training, we were finding bugs left and right.

A lot of them were bugs that I wrote. I remember us doing surgery in the middle: stopping the run, making the fix, and relaunching. We never actually went back to the start. We just kept training it with the bug fixes along the way.

Jeremy Wohlwend

Yeah.

Gabriele Corso
Jeremy Wohlwend

Right.

Gabriele Corso

Yeah.

Jeremy Wohlwend

It was impossible—

Gabriele Corso

Yeah.

Jeremy Wohlwend

—to reproduce now.

Gabriele Corso

Yeah, no, that model has gone through such a curriculum that it’s learned some weird stuff. But somehow, by some miracle, it worked out.

Brandon

It’s a pleasure to have with us today Gabriele Corso and Jeremy Wohlwend. They recently founded Boltz, a company trying to democratize and bring structure prediction in biology to the masses. They’re both recent PhD graduates from MIT and have been working on all sorts of foundational papers in generative biology. Anyway, it’s a pleasure to have you here. Thanks for coming.

Jeremy Wohlwend

Thank you.

Gabriele Corso

Thank you.

Brandon

I guess we’re maybe 6 years post-AlphaFold 2 right now, which was kind of a big moment. Is that right?

Jeremy Wohlwend

I think it was 2021.

Brandon

Yeah.

Jeremy Wohlwend

So, yeah, going on 5 years.

Brandon

5 years.

Jeremy Wohlwend

Yeah.

Brandon

5 years, yeah.

Jeremy Wohlwend

Yeah.

Brandon

So maybe for the audience, let’s go back to that moment in time and explain what this big moment was and why it was interesting. Why was everyone so excited? I think you two were probably quite excited, so why were you personally excited?

1. Why AlphaFold Changed Biology

Jeremy Wohlwend

I’d start with why that was interesting from a scientific standpoint. Maybe first, as an introduction for those in the audience who are not structural biologists: the idea of structural biology is that we want to try to understand how proteins and other molecules take shape inside our cells and how they interact.

Structural biology is this beautiful discipline where we’re somehow able to understand these minuscule structures at atomic detail using incredibly complex methods like X-ray crystallography. The dream in computational biology has always been: can we understand the structures without having to resolve this crystal, shoot X-rays, and so on?

AlphaFold was a real breakthrough in this problem of protein folding, which is trying to understand the structure of a single protein. To me, it was exciting across many dimensions. First, I was a computer scientist working a lot on machine learning, and I saw the impact that work somewhat similar to what I was doing could have on a longstanding scientific problem.

Second, from a more personal perspective, seeing the structures coming out of these models—seeing this beautiful creation of life—was very inspiring to me. That was one of the things that led me to start working on structural biology, and in particular with machine learning.

Brandon

Were you a structural biologist before AlphaFold came out? I mean, you did machine learning, but it wasn’t in structural biology, so that actually shifted your career quite dramatically.

Jeremy Wohlwend

Yeah, very dramatically. I was working on some pretty theoretical, methodological things, and I was starting to see some of the challenges in doing somewhat theoretical or methodological work and seeing the potential impact of applied machine learning.

AlphaFold was really a machine-learning breakthrough, but it was also applied machine learning, and so that led me to want to start working in applied ML.

Gabriele Corso

Our group at the time was working a lot on small molecules already, and I think AlphaFold is what triggered this shift to working on biologics. At the time, I think it opened as many questions as it answered.

The immediate follow-ups were, “Okay, can we do this on things other than proteins? Can we model interactions of small molecules with proteins, nucleic acids with proteins? Can we model more complex protein systems?”

Very rapidly after AlphaFold, people realized that machine learning could really target this problem very differently than previous methodologies.

Brandon

Going back to the AlphaFold 2 moment, I remember this very well. I was at NeurIPS when, I guess, the results of this famous competition came out. So I wanted you to talk about CASP, what it is, and why it was so interesting and exciting.

2. The CASP 14 Breakthrough

Gabriele Corso

Every couple of years, the goal has always been to find protein structures that are a little bit different from what’s known. CASP, over the years, has put in a lot of effort to gather structures from academic groups and even industry groups to try to create a test set that would be difficult for different methods.

CASP 14 was when AlphaFold 2 really blew everything out of the water. The improvement was so large over the previous method and also over the previous competitions.

Now CASP continues. We’ve had CASP 15, and we have CASP 16. What’s happened now is that it’s really expanding to all these other modalities, as I was mentioning: proteins, small molecules, nucleic acids, and so on. But the goal remains to really challenge the models: how well do these models generalize? We’ve seen in some of the latest CASP competitions that, while we’re becoming really, really good at proteins, especially monomeric proteins, other modalities still remain pretty difficult. And that’s really essential in the field: that there are these efforts to gather benchmarks that are challenging, so it keeps us in line about what the models can do or not.

Brandon

Yeah.

Gabriele Corso

Yeah.

Brandon

It’s interesting you say that. In some sense, at CASP 14, a problem was solved—pretty comprehensively, right? But at the same time, it was really only the beginning. Can you explain what the specific problem was that was arguably solved and what remains, which is probably quite open?

3. Structure Prediction Is Not Folding

Gabriele Corso

I think we’ll steer away from the term “solved” because—

Brandon

Solved.

Gabriele Corso

—we have many friends in the community who get pretty upset at that word, and I think fairly so. But the problem on which a lot of progress was made was the ability to predict the structure of single-chain proteins.

Proteins can be composed of many chains, and single-chain proteins are just a single sequence of amino acids. One reason we’ve been able to make such progress is that we take a lot of hints from evolution. The way the models work is that they decode a lot of hints that come from evolutionary landscapes.

If you have some protein in an animal and you go find the similar protein across different organisms, you might find different mutations in them. As it turns out, if you take a lot of these sequences together and analyze them, you see that some positions in the sequence tend to evolve at the same time as other positions in the sequence—a correlation between different positions.

It turns out that this is typically a hint that those 2 positions are close in 3D. Part of the breakthrough has been our ability to decode that very effectively.

But what it implies is that, in the absence of that co-evolutionary landscape, the models don’t perform as well. When that information is available, maybe one could say the problem is somewhat solved from the perspective of structure prediction. When it isn’t, it’s much more challenging.

It’s also worth differentiating structure prediction and folding, because sometimes we conflate them a little bit. Folding is the more complex process of actually understanding how it goes from this disordered state into a structured-like state, and I don’t think we’ve made that much progress on that.

But the idea of going straight to the answer is something we’ve become pretty good at.

Brandon

So there’s this protein that’s just a long chain, and it folds up.

Gabriele Corso

Yeah.

Brandon

And so we’re good at getting from that long chain, in whatever form it was originally, to the thing, but we don’t know how it necessarily gets to that state.

Gabriele Corso

Yeah.

Brandon

And there might be intermediate states that it’s in sometimes that we’re not aware of.

Gabriele Corso

That’s right, and that also relates to our general ability to model the different…

Jeremy Wohlwend

Proteins are not static. They move. They take different—

Brandon

Yeah.

Jeremy Wohlwend

…shapes based on their energy states, and I think we're also not that good at understanding the different states that the protein can be in, and at what frequency or probability.

Brandon

Yeah.

Jeremy Wohlwend

So I think the 2 problems are quite related in some ways. There's still a lot to solve, but I think it was very surprising at the time that, even with these evolutionary hints, we were able to make such dramatic progress.

Brandon

So I want to ask why the intermediate states matter. But first, I kind of want to understand why we care what proteins are shaped like.

4. Why Protein Shapes Matter

Gabriele Corso

Yeah. Proteins are kind of the machines of our body. The way that all the processes that we have in our cells work is typically through proteins, sometimes through other molecules or intermediate interactions, and through those interactions we have all sorts of cell functions.

When we try to understand a lot of biology—how our body works, how diseases work—we often try to boil it down to, okay, what is going right in the case of our normal biological function, and what is going wrong in the case of the disease state? We boil it down to proteins and other molecules and their interactions.

So when we try predicting the structure of proteins, it's critical to have an understanding of those interactions. It's a bit like seeing the difference between having a list of parts that you would put in a car and seeing the car in its final form. Seeing the car really helps you understand what it does.

Brandon

Right.

Gabriele Corso

On the other hand, going to your question of why we care about how the protein folds, or how the car is made, to some extent, sometimes something goes wrong: there are cases of proteins misfolding in some diseases and so on. If we don't understand this folding process, we don't really know how to intervene.

Gabriele Corso

There's this nice line in the AlphaFold 2 manuscript where they discuss why we're even hopeful that we can target the problem in the first place. There's this notion that, for proteins that fold, the folding process is almost instantaneous.

Brandon

Mm-hmm.

Jeremy Wohlwend

Which is a strong signal that we might be able to predict this very constrained thing that the protein does so quickly. Of course, that's not the case for all proteins, and there's a lot of really interesting mechanisms in the cells. But I remember reading that and thinking, “Yeah, that's somewhat of an insightful point.”

Jeremy Wohlwend

I think one of the interesting things about the protein folding problem is that it used to be studied as a classical example of an NP-hard problem, and that's part of the reason why people thought it was impossible. There are so many different types of shapes that these amino acids could take, and so this grows combinatorially with the size of the sequence.

There used to be a lot of theoretical computer science thinking about and studying the protein-folding problem as an NP-hard problem. So it was very surprising also from that perspective, seeing machine learning make it so clear that there is some signal in those sequences through evolution, but also through other things that we as humans are probably not really able to understand, but that these models have learned.

Brandon

And Andrew White—we were talking to him a few weeks ago—and he said that he was following the development of this and that there were actually ASICs that were developed just to solve this problem. Again, there were many millions of computational hours spent trying to solve this problem before AlphaFold.

Just to be clear, one thing that you mentioned was that there's this co-evolution of mutations that you see again and again in different species. So explain why that gives us a good hint that they're close to each other.

Gabriele Corso

Yeah. Think of it this way: if I have some amino acid that mutates, it's going to impact everything around it in 3 dimensions, right?

Brandon

Yeah.

Jeremy Wohlwend

So it's almost like the protein, through several probably random mutations in evolution, ends up figuring out that this other amino acid needs to change as well for the structure to be conserved.

Brandon

Got it.

Gabriele Corso

This whole principle is that the structure is probably largely conserved because there's this function associated with it. It's really different positions compensating for each other.

Brandon

I see. Those hints in aggregate give us a lot of information about what is close to each other. Then you can start to look at what kinds of folds are possible given the structure, what the end state is, and therefore make a lot of inferences about what the actual total shape is.

Jeremy Wohlwend

Yeah, that's right. It's almost like you have this big, 3-dimensional valley where you're trying to find these low-energy states.

Brandon

Yeah.

Gabriele Corso

There's so much to search through that it's almost overwhelming. But these hints maybe put you in an area of the space that's already kind of close to the solution, maybe not quite there yet.

There's always this question of how much physics these models are learning versus pure statistics. I think one of the things I believe is that once you're in that approximate area of the solution space, the models have some understanding of how to get you to the lower-energy state.

So maybe you have some light understanding of physics, but maybe not quite enough to know how to navigate the whole space well.

Brandon

Right. Okay.

Jeremy Wohlwend

So we need to give it these hints to get there.

Brandon

So you get it into the right valley, and then it finds the minimum or something.

Jeremy Wohlwend

Yeah. Yeah.

Brandon

Yeah.

Jeremy Wohlwend

One interesting explanation about how AlphaFold 3 works—which, of course, doesn't cover the entirety of what AlphaFold does—is one that I'm going to borrow from Sergey Chilingaryan at MIT. He sees AlphaFold, and the interesting thing about AlphaFold is that it's got this very peculiar architecture that we have since used. This architecture operates on these pairwise contacts between amino acids.

The idea is that probably the MSA gives you this first hint about what potential amino acids are close to each other.

Brandon

MSA is mult—

Jeremy Wohlwend

Multiple sequence alignment. Exactly.

Brandon

That's what he's talking about.

Gabriele Corso

Exactly.

Brandon

Yeah, yeah.

Jeremy Wohlwend

This evolutionary information.

Brandon

Yeah.

Jeremy Wohlwend

From this evolutionary information about potential contacts, it's almost as if the model is running some kind of Dijkstra's algorithm, where it's decoding: okay, these have to be close. Then, if these are close and this is connected to this, this has to be somewhat close.

So you decode this, and that becomes basically a pairwise distance matrix. From this rough pairwise distance matrix, you decode the actual potential structure.

Brandon

Interesting. So there's kind of 2 different things going on in the coarse-grained and then the fine-grained optimizations.

Gabriele Corso

Exactly.

Brandon

Interesting. Yeah. Very cool.

5. AlphaFold 3 Goes Generative

Brandon

Yeah. You mentioned AlphaFold 3, so maybe this is a good time to move on to that. AlphaFold 2 came out, and it was, I think, fairly groundbreaking for this field. Everyone got very excited. A few years later, AlphaFold 3 came out.

For some more history, what were the advancements in AlphaFold 3? Then I think maybe after that we'll talk a bit about how it connects to Boltz.

Jeremy Wohlwend

Yeah. After AlphaFold 2 came out, Jeremy and I got into the field, along with many others. The clear problem that was obvious after that was, okay, now we can do individual chains. Can we do interactions? Interactions between different proteins, proteins with small molecules, proteins with other molecules—

Brandon

So, quick: why are interactions important?

Gabriele Corso

Interactions are important because, to some extent, that's the way these machines—that is, these proteins—have a function. The function comes from the way that they interact with other proteins and other molecules.

In the first place, the individual machines are often, as Jeremy was mentioning, not made of a single chain, but of multiple chains. These multiple chains interact with other molecules to give those their function.

On the other hand, when we try to intervene in these interactions, think about a disease, a biosensor, or many other ways we are trying to design molecules or proteins that interact in a particular way with what we would call a target protein or target—this problem after AlphaFold 2 became clear as one of the biggest problems in the field to solve. Many groups, including ours and others, started making contributions to this problem of trying to model these interactions. AlphaFold 3 was a significant advancement in modeling interactions. One interesting thing they were able to do was that, while some of the rest of the field had tried to model different interactions separately—how proteins interact with small molecules, how proteins interact with other proteins, and how RNA or DNA have their structures—they put everything together. They trained a very large model with many advances, including changes to some key architectural choices, and managed to get a single model that achieved this new state-of-the-art performance across all of these different modalities: protein–small-molecule interactions, which are critical to developing new drugs; protein–protein interactions; and understanding how proteins interact with RNA and DNA, and so on.

Brandon

Just to satisfy the AI engineers in the audience, what were some of the key architectural and data changes that made that possible?

Jeremy Wohlwend

Yeah. One critical change that was not necessarily unique to AlphaFold 3, but that a few other teams, including ours, proposed, was moving from modeling structure prediction as a regression problem—where there is a single answer and you're trying to shoot for that answer—to a generative modeling problem, where you have a posterior distribution of possible structures and you're trying to sample from this distribution.

This achieves 2 things. One is that it starts to allow us to model more dynamic systems. As we said, some of these systems can actually take multiple structures, and so you can now model that by modeling the entire distribution. But on the other hand, from more core modeling questions, when you move from a regression problem to a generative modeling problem, you are really tackling the way that you think about uncertainty in the model in a different way.

So if you think about, “I'm undecided between different answers,” what happens in a regression model is that I'm going to try to make an average of those different answers I had in mind. When you have a generative model, what you're going to do is sample all these different answers and then maybe use separate models to analyze those different answers and pick out the best. So that was one of the critical improvements.

The other improvement is that they significantly simplified, to some extent, the architecture, especially the final model that takes those pairwise representations and turns them into an actual structure. That now looks a lot more like a traditional transformer than a very specialized equivariant architecture, which was what AlphaFold 3 had.

Speaker 3

So this is a bitter lesson a little bit?

Jeremy Wohlwend

There is some aspect of a bitter lesson, but the interesting thing is that it's very far from being a simple transformer. This field is arguably one of the very few fields in applied machine learning where we still have architectures that are very specialized. Many people have tried to replace these architectures with simple transformers, and there is a lot of debate in the field, but I think most of the consensus is that the performance we get from the specialized architecture is vastly superior to what we get through a single transformer.

Another interesting thing on the modeling and machine-learning side, which I think is somewhat counterintuitive when you look at some of the other fields and applications, is that scaling hasn't really worked the same way in this field. Models like AlphaFold 2 and AlphaFold 3 are still very large models, but at the same time, in terms of parameters, they're actually not very big. They are definitely below 1 billion parameters.

If you hear these days in the LLM space, a model with less than 1 billion parameters, you'd think it can't do anything. But on the other hand, when you look at the computational cost of running these models, they're actually a lot more expensive than it is to run language models because, as Jeremy was saying, we go from having sort of quadratic operations to cubic operations. And so it's interesting how, right now in the field—and this may be related to having less data or needing more inductive biases—we have this ratio of the amount of computation to parameters that is much, much higher than in other places.

Brandon

If I recall, AlphaFold 2 was, what, 70 million parameters? Something like that?

Gabriele Corso

Yeah, it's something like that. It's quite small—less, around 100 or so. Yeah.

Brandon

These decisions around triangular layers and this interesting equivariant architecture in AlphaFold 2 were really priors that baked in a lot of the physics of the system. And coevolution data is—I think people have argued that it is kind of almost like a database lookup of some sort.

Gabriele Corso

Yeah, yeah.

Jeremy Wohlwend

It also sort of provides, in some sense, more parameters as well.

Gabriele Corso

Yeah. I mean, it's definitely more about the amount of pure compute FLOPs, right, that is very high, and it's almost more like reasoning-based, maybe, than just information extraction.

Jeremy Wohlwend

It isn't just because of their reasoning capability; it's also because of the sheer quantity of information that they store. And I think here there's a little bit less of that, and I think it's more about decoding this input rather than maybe memorizing as much of it.

Brandon

So is there a loop in the architecture that allows it to compute more per parameter? How does that work?

Jeremy Wohlwend

Part of it is exclusively this fact: instead of having operations that operate on a single chain, they operate on the pairwise representations. And so instead of having a quadratic number of interactions, you have a cubic number of interactions. That on its own leads you to have smaller representation sizes, but more representations, which leads to more FLOPs but fewer parameters.

Jeremy Wohlwend

Yeah.

Gabriele Corso

On the other hand, there is also this idea, somewhat similar to reasoning, where you recycle this operation. Starting with AlphaFold 2, and also in AlphaFold 3, they have this interesting framework where you start—as we were discussing—the input to the model is sort of this initial understanding of the interactions, either from the evolution of the multiple sequence alignments, but also potentially from what we call templates, which are basically database lookups of similar structures.

And so how the model works is that it decodes these and tries to understand a good potential rough structure for the pairwise interaction. Then what you can do is basically do this recycling, where you feed this understanding back to the input of the model and then try to decode it again. People do this 3 or 4 times, and in some cases I've even tried to do it tens of times. And so you can see it as a very, very early version of reasoning.

Brandon

So, AlphaFold 2, really cool. AlphaFold 3, really cool. But AlphaFold 3 came with a catch, and I think this catch was important for the development of Boltz and so on.

Gabriele Corso

Yeah.

6. Boltz Answers The Open Source Gap

Jeremy Wohlwend

The catch was that it was an amazing Nature paper, but unfortunately they decided not to release the model. AlphaFold 2 was open source, and since then was used, I think the reported number is more than 1 million scientists.

AlphaFold 3, for commercial reasons—DeepMind has since spun off Isomorphic Labs, which is now trying to become a new pharmaceutical company—had decided to keep this model internal and only use it internally. Those of us in the field who were building on top of models like AlphaFold no longer had the base starting point to build on top of.

But even more importantly, everyone in both academic research and industry no longer had access to these incredible models, which were really useful for trying to understand biology but also for trying to develop new therapeutics. I decided to take the matter into our own hands and try to obtain a model that was of similar accuracy.

And so, largely using a lot of the information in the AlphaFold 3 manuscript, we went ahead and built Boltz-1, which was the first fully open-source model to approach the level of accuracy of AlphaFold 3. Along the way—and we can talk about it more—we realized that it was probably too ambitious to have this as an academic project. There were a lot of things that were missing, and so we decided to also start a public-benefit company to push this mission of democratizing access to these models, which we started with Boltz-1.

Brandon

Quick interjection: I remember this. It was actually shocking how fast you got Boltz-1 out. It was just 2 or 3 months, right?

Gabriele Corso

I think we started in late May, and it came in November—

Brandon

Mm-hmm. Yeah.

Jeremy Wohlwend

—if I remember correctly. So, slightly longer, but yeah. It was relatively quick. For what it's worth, we were working on some similar ideas at the time. For example, this idea of having a diffusion model on top of a pairwise trunk was something that we were exploring independently.

When the paper came out, it was really clear, especially on the data pipelines, that there was so much we weren't really doing, and there was a lot to catch up on. But we were already in a place where we had some experience working with the data and with these types of models, and I think that put us in a good place to produce it quickly.

I would even say that we could have done it more quickly. The problem was that, for a while, we didn't really have the compute, so we couldn't train the model. We only trained the big model once—that's how much compute we had. We could only train it once. While the model was training, we were finding bugs left and right, a lot of them that I wrote.

Brandon

Yeah.

Gabriele Corso

I remember doing surgery in the middle of the run: stopping it, making the fix, relaunching it. We never actually went back to the start; we just kept training it with the bug fixes along the way.

Brandon

Right.

Jeremy Wohlwend

Which was—

Brandon

So it's impossible to reproduce now.

Jeremy Wohlwend

Yeah. That model has gone through such a curriculum that it's learned some weird stuff. But somehow, by some miracle, it worked out.

Jeremy Wohlwend

The other funny thing is that most of that model was trained through a cluster from the Department of Energy.

Gabriele Corso

Yeah.

Jeremy Wohlwend

It was a shared cluster that many groups used, so we were basically training the model for 2 days, and then it would go back into the queue and stay there for a week.

Gabriele Corso

Oh, man.

Brandon

Oh.

Jeremy Wohlwend

It was pretty painful. Toward the end, with Devan, the CEO of Genesis, I was telling him a bit about the project and about our frustration with the compute. Luckily, he offered to help, and we got help from Genesis to finish the model. Otherwise, it probably would have taken a couple of extra weeks—

Brandon

Wow.

Gabriele Corso

—which—

Jeremy Wohlwend

Yeah.

Gabriele Corso

Yeah.

Jeremy Wohlwend

Boltz-1.

Brandon

How did that compare to AlphaFold 3? And then was there some progression from there?

Jeremy Wohlwend

I would say that both Boltz-1 and this other set of models that came around the same time were a big leap from the previous open-source models and were really approaching the level of AlphaFold 3. I would still say that, even to this day, there are some specific instances where AlphaFold 3 works better. One common example is antibody–antigen prediction, where AlphaFold 3 still seems to have an edge in many situations.

Obviously, these are somewhat different models. You run them and obtain different results, so it's not always the case that one model is better than the other. But in aggregate, especially at the time, AlphaFold 3 was still having a bit of an edge.

Brandon

We should talk about this more when we talk about BoltzGen, but how do you know one model is better than the other? I make a prediction, you make a prediction—how do you know?

The great thing about structure prediction is that, a bit like what CASP was doing, there is a relatively straightforward way to evaluate the models. Once we go into the design space of designing new small molecules or new proteins, this becomes a lot more complex. You train a model on structures that had been released across the field up until a certain time. One thing we haven't talked about that was really critical in all this development is the PDB, the Protein Data Bank. It's this common resource, a basic shared database where every biologist can publish their structures.

We can train on all the structures that were put in the PDB until a certain date. Then we look at recent structures and ask which ones look pretty different from anything that was published before, because we really want to understand generalization. On these new structures, we evaluate all the different models.

Brandon

And so you know when AlphaFold 3 was trained, and you intentionally train to the same date or something like that?

Gabriele Corso

Exactly.

Brandon

Right. Yeah.

Gabriele Corso

That's the way you can somewhat easily compare these models. Obviously, that assumes you know the training set.

Brandon

You've always been very passionate about validation. I remember DiffDock, and then there was DiffDock-L and DockGen. You've thought very carefully about this in the past. I think DockGen is a really funny story, and I don't know if you want to talk about that. It's interesting—

Jeremy Wohlwend

One of the amazing things about putting things open source is that we get a ton of feedback from the field. Sometimes we get great feedback from people who really like the model, but honestly, most of the time, people share where it doesn't work. To be honest, that may also be the most useful feedback.

Brandon

Yeah.

Gabriele Corso

At the end of the day, it's critical—and this is also true across other fields of machine learning—to set clear benchmarks. As you make progress on certain benchmarks, you need to improve the benchmarks and make them harder and harder. That's the progression of how the field operates.

The example of DockGen was that we published this initial model, called DiffDock, in my first year of my PhD. It was one of the early models to try to predict biomolecular interactions between proteins and small molecules, about a year after AlphaFold 2 was published.

On the one hand, on the benchmarks we were using at the time, DiffDock was doing really well and outperforming some of the traditional physics-based methods. On the other hand, when we started giving these tools to many biologists, one example being the group of Nick Polizzi at Harvard, we started noticing a clear pattern: for proteins that were very different from the ones the models were trained on, the models were struggling.

It seemed clear that this was probably where we should put our focus. So, first, with Nick and his group, we developed a new benchmark. Then we asked, “What can we change about the current architecture to improve this pattern of generalization?” That's what we're still doing today: asking where the model doesn't work, and then, once we have that benchmark, throwing every idea we have at the problem.

Gabriele Corso

There's a lot of healthy skepticism in the field, which I think is great, and it's very clear that there are a ton of things the models don't work well on. But one thing that's probably undeniable is the pace of progress and how much better we're getting every year.

If you assume any constant rate of progress moving forward, I think things are going to look pretty cool at some point in the future.

Brandon

ChatGPT was only 3 years ago.

Gabriele Corso

Yeah, it's wild, right?

Brandon

What?

Speaker 0

Yeah, it's one of those things—even being in the field, you don't see it coming. Hopefully, we'll continue to have as much progress as we've had over the past few years.

Brandon

So this is maybe an aside, but I'm really curious. You get this great feedback from the community by being open source, right? My question is partly: if you open-source, then everyone can copy what you did, but it's also maybe about balancing priorities, right?

All my customers are saying, “I want this.” There are all these problems with the model, but my customers don't care, right? How do you think about that?

Jeremy Wohlwend

Yeah.

Jeremy Wohlwend

So I would say a couple of things. One is that part of our goal with Boltz—and this is also established as the mission of the public benefit corporation that we started—is to democratize access to these tools. But one of the reasons why we realized that Boltz needed to be a company, and couldn’t just be an academic project, is that putting a model on GitHub is definitely not enough to get chemists and biologists across academia, biotech, and pharma to use your model in their therapeutic programs. A lot of what we think about at Boltz, beyond just the models, is all the layers that come on top of the models to get from those models to something that can really enable scientists in the industry.

And so that goes into building the right kind of workflows that take in, for example, the data and try to answer directly the problems that the chemists and biologists are asking, and then also building the infrastructure. This is to say that even with models fully open, we see a ton of potential for products in this space. The critical part about a product is that even with an open-source model, running the model is not free. As we were saying, these are pretty expensive models, and especially—and maybe we’ll get into this—these days we’re seeing pretty dramatic inference-time scaling of these models—

Brandon

Yes.

Gabriele Corso

—where the more you run them, the better the results are. But you start getting to a point where compute and compute cost become critical factors. Putting a lot of work into building the right kind of infrastructure, building the optimizations, and so on, really allows us to potentially provide a much better service for open-source models.

That is to say, even though we’re building the product and can provide a much better service, I do still think—and we will continue to put a lot of our models open source—that the critical role of open-source models is helping the community progress on the research, from which we all benefit. We’ll continue, on the one hand, to put some of our base models open source so that the field can build on top of them. As we discussed earlier, we learn a ton from the way that the field uses and builds on top of our models.

But then we’ll try to build a product that gives the best experience possible to scientists, so that a chemist or biologist doesn’t need to spin up a GPU and set up our open-source model in a particular way, but can just use it. Even though I am a computer scientist and machine-learning scientist, I don’t necessarily take an open-source LLM and try to spin it up; I might just open the ChatGPT app or Claude Code and use it as an amazing product. We want to give the same experience to scientists around the world.

Brandon

I heard a good analogy yesterday: a surgeon doesn’t want the hospital to design a scalpel, right? The surgeon just buys the scalpel.

Jeremy Wohlwend
Gabriele Corso

You wouldn’t believe the number of people who, even in my short time between AlphaFold 3 coming out and the end of the PhD, would reach out just for us to run AlphaFold 3 for them. Or things like that, just because Boltz, in our case—because it’s not that easy to do that computation if you’re not a computational person. I think part of the goal here is also that we continue to build an interface for computational folks, but that the models are also accessible to a larger, broader audience. That comes from good interfaces and things like that.

Brandon

I think one really interesting thing about Boltz is that with its release, you didn’t just release a model, but you created a community. Did that community—it grew very quickly. Did that surprise you? And what is the evolution of that community, and how has that fed into Boltz—

Jeremy Wohlwend

If you look—

Brandon

—as a company?

Gabriele Corso

If you look at its growth, it’s—

Brandon

Yeah.

Jeremy Wohlwend

—it’s very much like when we release a new model: there’s a big jump. But, yeah, it’s been great. We have a Slack community that has thousands of people on it, and it’s actually self-sustaining now, which is really nice. It’s almost overwhelming to answer everyone’s questions and help with the few people we had, but it ended up that people would answer each other’s questions and help one another. The Slack has been self-sustaining, and that’s been really cool to see.

For the Slack part, but then also obviously on GitHub as well, we’ve had a nice community. We also aspire to be even more active on it than we’ve been in the past 6 months, which has been a bit challenging for us. But the community has been really great. There are also a lot of papers that have come out with new evolutions on top of Boltz, and that surprised us to some degree because there are a lot of models out there. People converging on Boltz was really cool.

I think it also speaks to the importance of trying to put a lot of emphasis, when you put code out, on making it as easy to use as possible—something we thought a lot about when we released the codebase. It’s far from perfect, but—

Brandon

Do you think that was one of the factors that caused your community to grow—just the focus on making it easy to use and accessible?

Jeremy Wohlwend

I think so, yeah. We’ve heard it from a few people over the years now, and some people still think it should be a lot nicer. They’re right. But, yeah, I think it was, at the time, maybe a little bit easier than other things.

Jeremy Wohlwend

The other thing that I think led to the community, and to some extent to the trust in the community and in what we put out, is the fact that it’s not really been just one model. Maybe we’ll talk about it: after Boltz-1, there were another couple of models released or open-sourced soon after. We continued the open-source journey and released Boltz-2, where we were not only improving structure prediction, but also starting to do affinity predictions—understanding the strength of the interactions between these different molecules, which is a critical component, a critical property that you often want to optimize in discovery programs.

More recently, we also released a protein-design model. We’ve been building this suite of models that come together and interact with one another. There is almost an expectation that we take very much to heart always having, across the entire suite of different tasks, the best or among the best models out there, so that our open-source tool can be the go-to model for everybody in the industry.

Brandon

I really want to talk about BoltzGen. But before that, one last question in this direction: was there anything about the community that surprised you? Were there any moments when someone was doing something and you thought, “Why would you do that? That’s crazy,” or, “That’s actually genius, and I never would have thought about that”?

Gabriele Corso

We’ve had many contributions. Some of the interesting ones—I mean, we had this one individual who wrote a complex GPU kernel for part of the architecture. The funny thing is that that piece of the architecture had been there since AlphaFold 2.

I don’t know why it took Boltz for this person to decide to do it, but that was a really great contribution. We’ve had a bunch of others, like people figuring out ways to hack the model to do cyclic peptides. I don’t know if any other interesting ones come to mind.

Jeremy Wohlwend

One cool one—and this was initially proposed as a message in the Slack channel by Tim O’Donnell—was basically that there are some cases, especially antibody–antigen interactions, where the models don’t necessarily get the right answer. What he noticed is that the models were somewhat stuck predicting the antibody to interact with the part of the antigen that was incorrect.

And so he basically ran the experiments. In this model, you can condition it; basically, you can give it hints. He gave random hints to the model: “Okay, you should bind to this residue. You should bind to the 1st residue, or you should bind to the 11th residue, or you should bind to the 21st residue,” or basically every 10 residues, scanning the entire antigen.

Brandon

Residues are the—

Jeremy Wohlwend

The amino acids.

Brandon

The amino acid, yeah.

Jeremy Wohlwend

So the 1st amino acid, the 11th amino acid, and so on. It’s sort of like doing a scan, conditioning the model to predict all of them, looking at the confidence of the model in each of those cases, and taking the top. It’s a somewhat crude way of doing inference-time search, but surprisingly, for antibody-antigen prediction, it actually helped quite a bit.

And so there are some interesting ideas that, as you’re developing the model, you say, “Wow, why would the model be so dumb?” But it’s very interesting, and that leads you to start thinking, “Okay, can I do this—not with this brute force, but in a smarter way?” We’ve also done a lot of work in that direction.

Gabriele Corso

And that speaks to the power of scoring. We’re seeing that a lot. I’m sure we’ll talk about it more when we talk about BoltzGen, but our ability to take a structure and determine that that structure is good—somewhat accurate—whether that’s a single chain or an interaction, is a really powerful way of improving the models.

If you can sample a ton and assume that, if you sample enough, you’re likely to have the good structure, then it really just becomes a ranking problem. Part of the inference-time scaling that Gabriele was talking about is very much that. The more we sample, the more the ranking model ends up finding something it really likes. And so I think our ability to get better at ranking is also what’s going to enable the next big breakthroughs.

Speaker 2

Interesting.

Brandon

I guess there’s—my understanding is that there’s a diffusion model, and you generate some things, and then—

Gabriele Corso

Yeah.

Brandon

Then you rank them using a score, and then— I guess it’s just what you said, right? Can you talk about those different parts?

7. BoltzGen Designs New Proteins

Jeremy Wohlwend

Yeah. So first of all, one of the critical beliefs that we had, also when we started working on Boltz-1, was that structure-prediction models are somewhat our field’s version of foundation models. They learn about how proteins and other molecules interact, and then we can leverage that learning to do all sorts of other things. With Boltz-2, we leveraged that learning to do affinity predictions—to understand, if I give you this protein and these small molecules, how tightly is their interaction?

For BoltzGen, what we did was take the foundation models and fine-tune them to predict entirely new proteins. The way that works is that, instead of filling in an actual sequence for the protein you’re designing, you fill in a set of blank tokens, and you train the models to predict both the structure of that protein and, with the structure, what the different amino acids of that protein are.

Basically, the way BoltzGen operates is that you feed it a target—a protein that you may want to bind to, or DNA or RNA—and then you feed it the high-level design specification of what you want your new protein to be. For example, it could be an antibody with a particular framework, a peptide, or many other things.

Brandon

And that’s with natural language or with this—

Gabriele Corso

And that’s basically prompting, and we have this sort of spec that you specify.

Brandon

Okay.

Gabriele Corso

You feed this spec to the model, and then the model translates it into a set of tokens, a set of conditioning for the model, a set of blank tokens. Then it decodes, as part of the diffusion model, a new structure and a new sequence for your protein. Then we take that and, as Jeremy was saying, try to score it and see how good of a binder it is to that original target.

Brandon

You’re basically using Boltz-2 to predict the folding and the affinity to that molecule, and that gives you a score. Is that right?

Gabriele Corso

Exactly.

Brandon

Yeah.

Jeremy Wohlwend

So you use this model to predict the structure, and then you do 2 things. One is that you predict the structure with something like Boltz-2, and then you compare that structure with what the model predicted—what BoltzGen predicted.

This is sort of called consistency in the field. You want to make sure that the structure you’re predicting is actually what you’re trying to design, and that gives you much better confidence that it’s a good design. That’s the first filtering.

The second filtering that we did as part of the BoltzGen pipeline, and that was released, is that we look at the confidence that the model has in the structure. Now, unfortunately, going to your question about predicting affinity, confidence is not a very good predictor of affinity.

One of the things that we’ve made a ton of progress on since we released BoltzGen—and we have some new results that we’re going to announce soon—is the ability to get much better hit rates. Instead of trying to rely on the confidence of the model, we’re actually trying to predict the affinity of that interaction directly.

Brandon

Okay. Just backing up a minute: your diffusion model actually predicts not only the protein sequence but also its folding?

Gabriele Corso

Exactly.

And actually, one of the big differences that we made compared to other models in the space—and there were some papers that had already done this before, but we really scaled it up—was somewhat merging structure prediction and sequence prediction into almost the same task.

The way BoltzGen works is that the only thing you’re doing is predicting the structure. The only supervision is supervision on the structure. But because the structure is atomic, and the different amino acids have different atomic compositions, from the way that you place the atoms, we also understand not only the structure that you wanted, but also the identity of the amino acid that the model believed was there.

And so, instead of having these 2 supervision signals—1 discrete, 1 continuous—that don’t interact particularly well, we build an encoding of sequences and structures that allows us to use exactly the same supervision signal that we were using for Boltz-2, which is largely similar to what AlphaFold 3 proposed. It’s very scalable, and we can use it to design new proteins.

Speaker 2

Oh, interesting.

Gabriele Corso

Maybe a quick shout-out to Hannes Stärk on our team, who did all this work. Yeah.

Speaker 2

Yeah, that was a really cool idea. Looking at the paper, there’s this encoding where you just add a bunch of atoms that can be anything, and then they get rearranged and basically plopped on top of each other, so that—and then that encodes what the amino acid is. There’s a unique way of doing this. That was such a cool, fun idea.

Jeremy Wohlwend

I think that idea had existed before. I think it wasn’t—

Gabriele Corso

Yeah, there were a couple of papers that had proposed this, and—

Jeremy Wohlwend

Yeah, yeah.

Gabriele Corso

—and Hannes really took it to a large scale.

Speaker 2

Yeah.

Brandon

In the paper, a lot of it is dedicated to actually validating the model. In my opinion, all the people we talk about basically feel that this sort of wet-lab—or whatever the appropriate real-world validation is—is the whole problem, almost. Or not the whole problem, but a big, giant part of the problem.

Can you talk a little bit about the highlights from there? To me, the results are impressive both from the perspective of the model and also just the effort that went into the validation by a large team.

Jeremy Wohlwend

First of all, I think I should start by saying that both when we were at MIT, where Tommi Jaakkola was working in Barzilay’s lab, as well as at Boltz, we are not a biolab, and we are not a therapeutic company.

And so, to some extent, we were first forced to look outside our group, our team, to do the experimental validation. One of the things that Hannes and the team pioneered was the idea: can we go not only to a specific group and try to find a specific system, maybe overfit a bit to that system, and try to validate, but how can we test these models across a very wide variety of different settings? Protein design is such a wide task, with all sorts of different applications, from therapeutics to biosensors and many others. Can we get a validation that goes across many different tasks?

And so he basically put together, I think, something like 25 different academic and industry labs that committed to testing some of the designs from the model. Some of this testing is still ongoing, and they're giving results back to us in exchange for hopefully getting some new, great sequences for their task. He was able to coordinate this very wide set of scientists, and already in the paper, I think we shared results from 8 to 10 different labs, showing results from designing peptides to target ordered proteins, peptides targeting disordered proteins, and proteins that bind to small molecules. We showed results of designing nanobodies across a wide variety of different targets. And so that gave the paper and the model a lot of validation that was wide.

Brandon

And so, would those be therapeutics for those animals, or are they relevant to humans as well?

Gabriele Corso

They're relevant to humans as well. Obviously, you need to do some work into, quote-unquote, humanizing them, making sure that they have the right characteristics so they're not toxic to humans and so on. There are some approved medicines on the market that are nanobodies.

Gabriele Corso

There's a general pattern, I think, in trying to design things that are smaller. It's easier to manufacture. At the same time, that comes with potentially other challenges, like maybe a little bit less selectivity than if you have something that has more heft.

Jeremy Wohlwend

Yeah.

Gabriele Corso

There's a big desire to design mini-proteins, nanobodies, and small peptides that are just great drug modalities.

Brandon

Okay. I think we left off talking about validation in the lab, and I was very excited about seeing all the diverse validations that you've done. Can you—

Jeremy Wohlwend

Yeah.

Brandon

—go into some more detail about them?

Jeremy Wohlwend

Yeah.

Brandon

The specific ones.

Gabriele Corso

Yeah. The nanobody one, I think we did—what was it? 15 targets? Is that correct?

Jeremy Wohlwend

14.

Gabriele Corso

14 targets. Typically, the way this works is that we make a lot of designs, on the order of tens of thousands, and then we rank them and pick the top N. In this case, N was 15 for each target, and then we measure the success rates, both in terms of how many targets we were able to get a binder for and, more generally, out of all of the binders that we designed, how many actually proved to be good binders.

Some of the other ones involved a cool example where there was a small molecule that a designer protein binds to. That has a lot of interesting applications. For example, as Gabriele mentioned, biosensing and things like that.

Brandon

Okay.

Jeremy Wohlwend

We had a disordered protein, I think you mentioned also. Those were maybe some of the highlights.

Jeremy Wohlwend

Yeah. I would say that the way we structured some of those validations was, on the one hand, that we had validations across a whole set of different problems that the biologists we were working with came to us with. We were trying, for example, in some of the experiments, to design peptides that would target RexC, which is a target involved in metabolism. We had a number of other applications where we were trying to design peptides or other modalities against some other therapeutically relevant targets. We designed some proteins to bind small molecules.

Some of the other testing that we did was really trying to get a broader sense of how the model works, especially when tested on generalization. One of the things that we found in the field was that a lot of the validation, especially outside of the validation that was done on specific problems, was done on targets that have a lot of known interactions in the training data. It's always a bit hard to understand how much these models are really just regurgitating or trying to imitate what they've seen in the training data, versus really being able to design new proteins.

One of the experiments that we did was to take 9 targets from the PDB, filtering to things where there was no known interaction in the PDB. Basically, the model had never seen this particular protein bound, or a similar protein bound to another protein. There was no way that the model, from its training set, could say, “Okay, I'm just going to—”

Brandon

Like, tweak something.

Gabriele Corso

“—tweak something and just imitate this particular kind of interaction.”

We took these 9 proteins, worked with Adaptyv, a CRO, and basically tested 15 mini-proteins and 15 nanobodies against each one of them. The very cool thing that we saw was that, on two-thirds of those targets, we were able to get nanomolar binders from these 15 designs. Nanomolar, roughly speaking, is just a measure of how strongly the interaction is.

Jeremy Wohlwend

And roughly speaking, a nanomolar binder is approximately the binding strength that you need for a therapeutic.

Brandon

Okay.

8. Boltz Lab Turns Models Into Workflows

So, maybe switching directions a bit, Boltz Lab was just announced this week. Or was it last week?

Jeremy Wohlwend

Yeah.

Brandon

This is your first product, I guess, if you want to call it that. Can you talk about what Boltz Lab is and what you hope people take away from this?

Gabriele Corso

Yeah. As we mentioned at the very beginning, the goal with the product has been to address what the models don't do on their own, and there are largely 2 categories there. I'll split it into 3.

The first one: it's one thing to predict a single interaction, for example, a single structure. It's another to very effectively search a design space to produce something of value. What we found building this product is that there are a lot of steps involved in that, which we need to accompany the user through.

One of those steps, for example, is the creation of the target itself. How do we make sure the model has a good enough understanding of the target so we can design something? There are all sorts of tricks that you can do to improve a particular structure prediction. That's the first stage.

Then there's the stage of designing and searching the space efficiently. For something like BoltzGen, for example, you design many things and then rank them. But for a small molecule, the process is a little more complicated. We actually need to also make sure that the molecules are synthesizable.

The way we do that is that we have a generative model that learns to use appropriate building blocks, such that it can design within a space that we know is synthesizable. There's this whole pipeline of different models involved in being able to design a molecule. That's been the first thing.

We call them agents. We have a protein agent, and then we have a small-molecule design agent. That's really at the core of what powers the Boltz Lab platform.

Brandon

So, are these agents a language-model wrapper, or are they just your models that you're calling agents because they perform a function?

Jeremy Wohlwend

No. They're more of a recipe, if you wish. I think we use that term because of the complex pipelining and automation that goes into all this plumbing. That's the first part of the product.

The second part is the infrastructure. We need to be able to do this at very large scale for any one group that's doing a design campaign.

Gabriele Corso

Let's say you're designing 100,000 possible candidates to find the good one, right? That's a very large amount of compute. For a small molecule, that's on the order of a few seconds per design. For proteins, it can be a bit longer. Ideally, you want to do that in parallel; otherwise, it's going to take you weeks. We've put a lot of effort into our ability to have a GPU fleet that allows any one user to do this kind of large parallel search.

Brandon

So you're amortizing the cost over your users, basically.

Gabriele Corso

Exactly.

Brandon

Yeah.

Gabriele Corso

Exactly. To some degree, using 10,000 GPUs for a minute is the same cost as using 1 GPU for God knows how long, right? You might as well try to parallelize if you can. A lot of work has gone into making that very robust, so that we can have a lot of people on the platform doing that at the same time.

The third one is the interface, and the interface comes in 2 shapes. One is in the form of an API, and that's really suited for companies that want to integrate these pipelines, these agents, directly into existing workflows or existing user interfaces that they have. We're already partnering with a few distributors that are going to integrate our API.

Then the second part is the user interface. We've put a lot of thought into that as well. This is what I mentioned earlier, this idea of broadening the audience. That's what the user interface is about, and we've built a lot of interesting features into it, for example, for collaboration. When you have potentially multiple medicinal chemists going through the results and trying to pick out which molecules we're going to go and test in the lab, it's powerful for them to be able to, for example, each provide their own ranking and then do consensus building. There are a lot of features around launching these large jobs, but also around collaborating on analyzing the results.

Brandon

Yeah.

Gabriele Corso

And so Boltz Lab is a combination of these 3 objectives into 1 cohesive platform.

Brandon

Who is this accessible to?

Gabriele Corso

Everyone. You do need to request access today. We're still ramping up usage, but anyone can request access. If you're an academic, in particular, we provide a fair amount of free credit so you can play with the platform. If you're a startup or a biotech, you may also reach out, and we'll typically hop on a call just to understand what you're trying to do and provide a lot of free credits to get started. Of course, with larger companies, we can deploy this platform in a more secure environment. Those are more custom deals that we make with partners.

That's part of the ethos of Boltz. I think this idea of servicing everyone, and not necessarily going after just the really large enterprises, starts with open source. It's also a key design principle of the product itself.

Brandon

Yeah.

One thing I was thinking about with regard to infrastructure: in the LLM space, the cost of a token has gone down by, I think, a factor of 1,000 or so over the last 3 years, right?

Gabriele Corso

Yeah.

Brandon

Is it possible that, essentially, you can exploit economies of scale and infrastructure so that you can make it cheaper to run these things yourself than for any person to roll their own system?

Gabriele Corso

Oh, 100%. We're already there. Running Boltz on our platform, especially at large scale, is considerably cheaper than it would probably take anyone to put the open-source model out there and run it. On top of the infrastructure, one of the things that we've been working on is accelerating the models. Our small-molecule screening pipeline is 10x faster on Boltz Lab than it is in the open-source implementation, and that's also part of building a product—something that scales really well. We really wanted to get to a point where we could keep prices very low, in a way that it would be a no-brainer to use Boltz through our platform.

Brandon

Good. How do you think about validation of your agentic systems?

Gabriele Corso

Yeah.

Brandon

Because, as you were saying earlier, AlphaFold-style models are really good at, let's say, monomeric proteins where you have co-evolution data. But now suddenly the whole point of this is to design something that doesn't have co-evolution data, something that is really novel. So now you're basically leaving the domain that you thought was—

Gabriele Corso

Right.

Brandon

—that you were good at. So how do you validate that?

Gabriele Corso

Yeah, I agree completely, but there's obviously a ton of computational metrics that we rely on, but those only take you so far. You really have to go to the lab and test: with Method A and Method B, how much better are we? How much better is my hit rate? How much stronger are my binders? It's not just about hit rate; it's also about how good the binders are. There's really no way around that. I think we've really ramped up the amount of experimental validation that we do so that we can really track progress as scientifically sound as possible. I don't know if there's anything else.

Jeremy Wohlwend

I think one thing that's unique about us, and maybe companies like us, is that we're not working on just a couple of therapeutic pipelines with validation focused on them. When we do an experimental validation, we try to test it across tens of targets. That way, on the one hand, we can get a much more statistically significant result, and that really allows us to make progress from the methodological side without being steered by overfitting on any one particular system.

Of course, we choose—we always try to choose—targets and problems that are at the frontier of what's possible today. You don't want something too easy, and you don't want something too hard; otherwise, you're not going to see progress. This is a somewhat evolving set of targets. We talked earlier about the targets that we looked at with BoltzGen, and now we're trying even harder targets, both for small molecules and proteins. We try to keep ourselves on the boundary of what's possible.

Brandon

Do you have infrastructure, or do you just have a lot of different partnerships with academic labs and keep pushing on these and driving them forward?

Jeremy Wohlwend

We do this partially through academic labs. More and more, we do this through CROs, just because, to some extent, we need replicability—often going after the same targets multiple times and seeing the progress from 1 month to the next.

Brandon

Yeah.

Jeremy Wohlwend

And speed.

Gabriele Corso

Speed of execution.

Brandon

What happens if you start getting a bunch of really strong binders against therapeutic targets? What do you do?

Jeremy Wohlwend

Release them.

Gabriele Corso

Yeah. Put them online.

Brandon

Oh, you release them as open source?

Gabriele Corso

Yeah. When we say we have no interest in making drugs, we're serious.

Brandon

Yeah.

Gabriele Corso

With the academic labs, basically, they keep them and do whatever they want with them. With the CROs, so far, we've been releasing them.

Jeremy Wohlwend

I will also say—and I think this has been a bit of the issue that I have with some of the things that have been said in the field—that when we say we design new proteins, or we say we design new molecules that go and bind these particular targets, we should be very clear: these are not drugs.

Brandon

Yeah.

Jeremy Wohlwend

These are not things that are ready to be put—

Brandon

Yes. Yeah.

Jeremy Wohlwend

—into a human. There is still a lot of development that goes with it. To us, we see ourselves as building tools for scientists. At the end of the day, it really relies on the scientist having a great therapeutic hypothesis and then pushing through all the stages of development. We try to build tools that can accompany them in that journey.

Gabriele Corso

It’s not like a magic box where you can just turn it and get—

Jeremy Wohlwend

Get FDA-approved drugs.

Gabriele Corso

FDA-approved drugs. Yeah.

Gabriele Corso

But actually, that brings up an interesting question that I’ve been wondering about. Do you guys see yourself staying in this, for lack of a better way of saying it, layer? Or do you think that you’ll start to, in a physical sense, look at different layers of the virtual cell, so to speak?

There’s also the development process that goes sort of like design, preclinical, clinical, approval, and thinking about improving performance throughout that process based on the designs. Is that a direction that you guys are pushing?

Jeremy Wohlwend

Yeah. So one of the things, as Jeremy said, is that we are not a therapeutic company, and we want to stay that way—not be a therapeutic company—and always be at the service of all the different companies, including therapeutic companies, that we serve. To some extent, that does mean that we need to try to go deeper and deeper in getting these models better and better.

One of the things that we’re doing, along with many others in the field, is that now we’re starting to have models that are good both for small molecules and for proteins, and that can design binders—relatively tight binders. We’re starting to look at all these other properties. They’re called developability or ADME, which we care about when developing a drug, and we’re trying to figure out whether we can design for them from the get-go.

The thing about those properties is that, for some of them, you need to start having an understanding of the cell. That’s, on the one hand, why we need that understanding. But also, the way that we think about all the different and complex diseases is that these models and tools we’re building need to have a good understanding of biomolecular interactions.

At the same time, every disease is often unique, and every therapeutic hypothesis is unique. You may want something that needs to hit a particular target in a virus in a particular way, but you may not know exactly what way you want to do that. In the first set of designs, you’re going to try to target different epitopes in different ways. Then you’re going to test them in the lab, maybe directly in vivo, and see which ones work and which ones don’t.

You need to bring those results back into the models, and then the models can start to have a wider understanding—not just of the biophysics of the antibodies interacting with that target, but also of how that is shaped within the entire cell. First of all, that means, on the one hand, that we need these loops. This is also partially how we designed the platform to be.

But it also means that we need to start understanding more and more higher-level things. I wouldn’t say that we’re working in any way on a virtual cell like others are, but we’re definitely thinking very deeply about how the way that we target certain proteins interacts with pathways that are existing in the cell.

Brandon

One question that has come up is that you talk a lot about user interface and so on, and I think this is really important. My experience dealing with medicinal chemists, when you give them machine-learning models, is that they’re the most superstitious, skeptical, pseudo-religious people I’ve ever talked to when it comes to doing science.

Gabriele Corso

Sorry for the medicinal chemists listening.

Brandon

Yeah. They’re amazing. I’ve worked with some spectacular medicinal chemists who just pull magic out of their hat again and again, and I have no idea how they do it. But when you bring them a machine-learning model, it can sometimes be quite tricky to get them to deal with it. How has your interaction been with this, and how have you thought about building Boltz Lab to work with the skeptics?

Jeremy Wohlwend

One of the great value unlocks for us and for our product has been bringing a medicinal chemist onto the team. His name is Jeffrey. On day 1, he obviously had a lot of opinions about many of the ways that we should change both the way the agents worked and the way the platform worked.

But it’s been really amazing, once we started shaping the platform in a better way with his feedback, how we went from a fair amount of skepticism from him to him actually using a lot more compute than any of our computational folks on the team. At times, he’s running all these sorts of hypotheses: “Okay, maybe I can hit this protein this particular way. I can hit it in that way. Actually, let me look at this particular molecular space. Let me try to optimize for these particular interactions.”

He ends up running several screens in parallel, using hundreds of GPUs on his own. It’s been pretty incredible to see how the way that I was thinking about a problem—“Okay, you’re just trying to design a binder, a small molecule to a particular protein”—is much simpler than the way that he thinks about it. He thinks about it much more deeply, trying all these different things and these different hypotheses.

Once he gets the results from the model, he doesn’t just take the top 15. He really looks over them and tries to understand the different things. Then, when we select maybe some designs to bring forth, he has something where both the models understand that something’s good, but he himself does as well. That’s why we also built the platform to be an interface for this kind of chemist and also a collaborative experience.

Gabriele Corso

At the end of the day, I think that for people to be convinced, you have to show them something they didn’t think was possible. Until you have that aha moment, I think the skepticism will remain. But every once in a while, there’s a result that really surprises people, and then it’s like, “Oh, wow. Okay, this is actually—I can do something with this.”

Brandon

So you just get it in their hands, have them try it out, and they’ll be convinced. Yeah.

Gabriele Corso

Yeah, or maybe once the lab results come back.

Brandon

Or maybe one of their colleagues is convinced.

Jeremy Wohlwend

Yeah.

Brandon

I think it takes going to the lab—

Jeremy Wohlwend

Yeah.

Brandon

I think at some point. There’s no avoiding that. As beautiful as the platform can be, and as nice as the molecules that the model predicted might look, I think what really convinces people is hits.

Jeremy Wohlwend

Yeah.

Brandon

Yeah.

Jeremy Wohlwend

You see the results, and yeah.

Brandon

Exactly. Yeah.

Jeremy Wohlwend

Cool. Thank you for taking the time to chat with us.

Jeremy Wohlwend

Yeah. It’s been really interesting.

Brandon

Um, is there anything that you would like your audience to know?

Jeremy Wohlwend

I mean, first of all, we’re just getting started, continuing to build the team, and so definitely always looking for great folks both on the software side, machine-learning side, but also scientists to join the team and help us shape—

Brandon

On the infrastructure side too? Like—

Jeremy Wohlwend

Indeed.

Brandon

If you think that if you want a new challenge, because this is not just next-token prediction, this is really a new engineering challenge—

Jeremy Wohlwend

Exactly.

Brandon

—that hasn’t been done before.

Jeremy Wohlwend

If you, no matter how much experience you have with biologists and chemistry, if you want to come help us shape what biology and chemistry hopefully will look like in five, ten years, we’d love to hear from you. Go to Boltz.bio and come join the team.

Brandon

Cool. Thank you.

Gabriele Corso

Awesome.

Gabriele Corso

Thank you so much.

Gabriele Corso

Thanks so much.

Gabriele Corso

Thank you.