KYC升级为KYA：Sentinel Bio的Joshua Monrad谈DNA筛查、能力提升研究与智能体监测

1. Sponsor: Lovable

Today my guest is Joshua Monrad, co-founder and managing director of Sentinel Bio, a biosecurity nonprofit and grantmaker working to make pandemics history within our lifetimes. Until recently, he was also a biosecurity adviser at the OpenAI Foundation.

This episode came about as a result of a Twitter post. After noticing that one of the very first agents to use the now-infamous Moltbook message board was working on some sort of protein task, I speculated about how close we might have come to the first AI-enabled bio attack. Several prominent people pushed back, Joshua included. Notably, while he argued that the risk of such an incident remains minimal for now, he does expect that risk to rise very significantly in the not-so-distant future.

Before diving into all that, it’s worth stating clearly up front that Joshua, like so many people who are choosing to dedicate their careers to AI risk management, is at heart a techno-optimist. He’s motivated not only by the noble goal of preventing pandemics, but also by a positive vision that includes eliminating infectious disease, effectively treating chronic illnesses, and, of course, curing cancer. It’s because that bright future is at risk from the possibility of a biological disaster—one that either seriously derails civilization or causes a biotech backlash that forecloses much of the upside—that the bulk of today’s conversation focuses on the nature of AI-related bio risks, the natural barriers that protect us today, and how those protections are likely to erode as we create programmable wet labs and train AIs to use them.

We start with a discussion of what makes it hard to create bioweapons in the first place, with Joshua breaking the risk down into 4 categories. The first is the purely human-created variety. We’ve been protected largely by the fact that it takes serious access and expertise to have a chance of pulling something like this off. Both of those are rare and correlated quite strongly with mental stability.

When it comes to AI-related risks, as we’re already seeing in the cyber realm, there’s first the risk that AI assistance will enable a much broader set of individuals to achieve disastrous outcomes. Studies that attempt to measure this so-called uplift have not found major danger just yet. But, as you’ll hear, Joshua expects the next round of these experiments to show real risk.

Next, there’s the risk that AIs could drive the creation of bioweapons themselves, either because a model has broken bad in a fundamental sense or because it’s gone haywire in pursuit of some narrow goal, as we saw in the OpenClaw incident. Joshua says AI systems will soon be intelligent enough to accomplish this, if they aren’t already. But he remains confident that, for now, they lack the affordances needed to get there on their own.

This leaves open a fourth category: the possibility that AIs could pay or otherwise socially engineer humans to do critical real-world tasks for them. And while this seems to scare Joshua less than it scares me, that’s arguably a small detail in the grand scheme of where all of this is headed. The bottom line, as Joshua sees it, is that we can’t expect to control AI-related bio risk purely at the AI layer. We’ll instead have to create a defense-in-depth strategy that includes both controlling AI systems and hardening the physical world.

With that in mind, I was very interested to hear about Joshua’s conceptual framework for grantmaking, which starts from a theory of victory and works backward to the investments we need to make today. He and his teammates also conducted a systematic review of hundreds of possible interventions before ultimately deciding to focus much of their effort on getting the world’s 200 DNA synthesis companies to implement presynthesis screening, know-your-customer, and signal-sharing measures. To my surprise, Joshua believes that if only 80% of those companies adopt these measures, it will create a major improvement in the overall biosecurity picture.

At the same time, he describes a growing risk associated with benchtop DNA synthesizers. This is just one of many problems that nobody currently owns, but perhaps one that a listener will be personally inspired to take on. The critical problem is that, at anything like our current rate of progress, we’ll have unambiguously bio-dangerous AI systems long before we’re able to do everything we need to properly fortify our position. This, I think, is where the situation starts to look objectively crazy. Joshua agrees.

When I asked him whether, if he were made benevolent dictator, he would impose a pause on frontier AI for biology until bio defenses were developed and deployed, he said yes, he absolutely would. With that, I hope you enjoy this assessment of where biosecurity stands today, and this sobering look at how much worse it’s likely to get if we don’t get our act together very quickly.

It’s great to be here, Nathan.

I’m excited for this conversation. Maybe a little bit scared of what I might learn, to be honest. Let me just set the stage real quick. We connected online maybe a month or so ago, as everybody was still trying to make sense of the latest revelations in the OpenClaw saga.

I had an interesting exchange that honestly still has me a little freaked out. I was looking at the OpenAI report and noticed that one of the very first agents that had used the message board was working on a protein task. As I sometimes do, I let my imagination run with that for a second. We see these agent swarms doing stuff they kind of know they’re not supposed to be doing. They’re very goal-directed in the sense that they have potentially very local, or seemingly trivial, goals, but they’ll go to extreme lengths and do crazy end runs to get there.

We’ve even seen pretty advanced social-engineering-type attacks in the wild by deployed models, although, as far as I know, not exactly in that incident. I put all those pieces together and thought, “Boy, maybe this was even a near miss on a bio-disaster front. Who really knows?” Most people did not respond very affirmatively to that. Helen Toner, no less, gave me a “Come on, man.” Most people seemed to think this is still pretty far off, although maybe coming soon. We’re in one of these weird AI analyses where—

It depends a lot on what we mean by “pretty far off.”

Yeah, the exponent nature of the exponential is kind of weird here, obviously. Maybe for starters, just give me your broad survey of where we are in biosecurity as we’re here, at the end of October, beginning of November 2026.

I’m glad to be talking about this here. I think it’s not always the case, but sometimes something really fruitful comes out of a little Twitter beef. You and I were going back and forth on it, and I’m really glad this conversation came out of that.

In terms of where we are on bio risks, I think we can think of it in maybe 4 phases. There’s a long historical record of people using biology as a weapon, long before AI was even thought of. Then there’s the idea that humans might use AI for bioweaponry. If I were to date that phase, I’d place its beginnings in the late 2010s or early 2020s, when the idea was, “Okay, there may be something here. Maybe it’s just a slightly more effective Google search, but it could provide some uplift.”

Then I think there’s another phase: AI using humans. That’s the social-engineering part you were just talking about. What if a model or an agent tricks a human into performing some task for them that could aid in a plot? The final phase would be end-to-end, with no humans in the loop. I think that’s probably the kind of scenario you may have been thinking about when you were thinking about the end-to-end phase for bio.

I think we can try to put dates on each of these 4 phases. The pre-AI phase was before the 2010s. The AI-uplift phase—we can talk a little bit about the state of AI uplifting humans, but I think we’re still in that phase. It’s really starting to come online now.

2. History of bioweapons (Part 1)

For AI using humans, I’d see social engineering in other domains as a leading indicator. If we start seeing a lot of cases—maybe you’ve already seen them and I just haven’t clocked them—of AI convincing humans to fill out CAPTCHAs, I think I’d be a lot more worried. For the end-to-end phase, with no humans in the loop, I think we’re very close to a place where that’s going to be constrained by robotics rather than intelligence. That’s where I was pushing back on your point, Nathan: I think it’s clear the models are misaligned, or not well enough aligned, that we could have an incident. It’s probably soon the case that they’re smart enough, but they don’t have sufficiently opposable thumbs. That was my take. It’s not clear to me that this is really imminent, but certainly in the 2030s, that’s going to come.

Maybe just briefly, I’m interested in phase one from the standpoint of how much risk there is that people want to do bioterrorism. I think people have very different intuitions about this. You hear the sort of Kevin Esvelt math sometimes: if only one in whatever, then multiply by the population. That’s one analysis, but I’m not sure how often it lands with people who think, “That seems ridiculous. Who would want to do that? You’ll kill yourself just as well as everyone else.” How would you tell a brief history of bioweapons as used by humanity before AI?

Yeah. It’s a super dark history, to be honest. There are probably 3 groups I’d think about. First, state actors. That includes the Soviet and US bioweapons programs. The US had a very well-resourced bioweapons program well into the 20th century.

People often think about this through the lens of game theory. I forget whether it was you or someone else, but I was arguing against the idea that it would never be rational to use a biological weapon because it’s a really bad weapon. I’d say, first, there are ways you could make it more rational. Some of these are not best discussed on a public show, but an obvious one would be to inoculate your own population before releasing some kind of pathogen. Second, since when are warfare and state weapons programs entirely rational? History is riddled with irrational invasions—Saddam Hussein invading Kuwait, Hitler’s venture east. There are a lot of things that, in rational game-theoretic logic, look like brutal miscalculations. So I wouldn’t hang my hat on that.

That’s the state-actor group. Empirically, I think we’re in a better place than we were in the middle of the Cold War, when there were publicly known, well-resourced programs. Now, there are probably some covert programs, but they’re likely operating with much less direct support. So empirically, I think that’s pretty good. From a rational analysis, though, I’m not very reassured that we can count on states not to pursue bioweapons.

Then we have non-state groups. Here, I think it’s really important to break things down by intent. If you’re looking at an actor who wants to cause a spectacle, which is sometimes the goal of terrorists, or has a narrow political aim, I’d agree that biology is a really poor choice of weapon. There are more readily accessible ways to cause harm. Chemical weapons are relatively easier, but even cruder methods are available, and that’s what we’re seeing empirically. That’s what non-state terrorists do.

But if you look at groups motivated by something more sinister, like actual omnicide or ending life, there’s a whole plethora of subcategories of extreme nihilist, violent ideologies. Biology stands out as one of the few ways a relatively resource-constrained group could pursue some of these really dark ends. That’s the lens through which I’m especially worried about biological weapons. One example is the bombing of a fertility clinic in California, either early this year or late last year. The FBI has disclosed that 2 individuals used an LLM at the time to inform the plot. They drove a car with a chemical explosive into the fertility clinic, and their stated motivation was some sort of anti-life ideology. That’s why they targeted a fertility clinic: an antinatalist ideology. I think it’s good that they weren’t pursuing biological weapons, probably because it was too difficult.

3. History of bioweapons (Part 2)

You mentioned the possibility of creating something new and inoculating your own population before releasing it. How do you break down the risk between creating new things and taking known things and putting them into the environment? Smallpox comes to mind as something that was a terrible scourge for a long time and is now gone, but is still out there in idea space, at least. How—

And, by the way, that’s also one of the earlier documented examples of what you might call a biological weapon. There are historical accounts of European settlers in the Americas deliberately giving blankets worn by people with smallpox to Indigenous populations. Smallpox was really devastating to Indigenous populations. It’s sometimes referred to as the Columbian Exchange. I don’t know how much the deliberate transfer of these blankets contributed to the spread, but it’s a really dark example of weaponizing disease centuries ago.

And, yeah, Nathan, you said yourself that you were worried this would be a scary conversation. I noticed your pause before, but this is the material we’re dealing with. How dangerous would something like smallpox be to us now? I’d assume a large percentage of the population has effectively become naive again. I don’t even know if I got a smallpox vaccine. I’m not even sure. Where are we on that?

Yeah.

Things like smallpox and flu variants could absolutely be devastating. I don’t think they would pose outright existential threats for a number of reasons. One is that we have some safeguards to lean on: existing vaccine designs, stockpiled vaccines, or, in the case of flu strains, some cross-immunity from endemic or seasonal strains.

That gives some comfort, but our recent experience with COVID makes it very clear how pathetically vulnerable we are even to a variant of a fairly well-studied viral strain. It’s really important to internalize, especially when we’re talking about all of this in the abstract, how far we are from having some defense in principle to being able to say, “Yeah, we would absolutely handle this.”

I think we are very vulnerable to naturally emerging, but also deliberately released, strains of known, well-studied diseases or near variants of them. But the question of something entirely novel is a slightly different story.

Just to calibrate myself: if somebody were to reintroduce smallpox into the wild, would you expect that to be significantly worse than the COVID experience? It’s a significantly worse disease, right? So if it wouldn’t be worse, it would be because we have some defense that I’m not fully seeing.

I haven’t thought about the epidemiology of smallpox in a while. I guess that’s part of the issue: I don’t think we have a lot of readiness for things we haven’t dealt with in a while. We’ve seen this with the resurgence of measles in the U.S. as vaccination has decreased.

Off the top of my head, I don’t have a super-confident take on how such an outbreak would go. But I also don’t have a really reassuring story for why it wouldn’t be really bad.

I just asked Google’s AI Mode if American kids are vaccinated against smallpox. The answer is no; vaccination stopped in 1972.

Yeah. Smallpox was declared extinct, right? From the perspective of naturally emerging disease, why would you vaccinate against it? For a long time, the disease was a biosafety concern because samples were held in 2 labs in the world, at least officially: one in Russia and one in Atlanta, Georgia, at the CDC.

From that perspective, you might have some concerns about whether someone could break into the lab. It’s only really in the era of synthetic biology and biological engineering that you have to start thinking about creating viruses from scratch. Smallpox has a much longer genome than some of the other viruses we might worry about, so I think there’s a barrier there. But it shows how this threat is evolving: we might have assumed we don’t have to inoculate or even stockpile, because you can’t break into these places, or there’s just not very much there, or whatever.

Turning to the real-world side of this, what makes it hard? You mentioned the long genome. I don’t have a great command of how DNA gets synthesized and assembled, but I understand there are limits to how long a sequence you can synthesize, and then you have to put building blocks together. What are the real hard parts?

I think it depends on whether the threat model is something well-studied and well-known, or designing something entirely novel with novel characteristics. If we take something that’s well-studied, then the relevant factors do include genome length. Smallpox is, I think, 186,000 base pairs. There’s a wide range: some viruses are in the high single-digit thousands. I think polio virus, if I recall, is 9,000 bases. Ebola is 20,000 or 30,000. That’s one crude measure of complexity.

There are also other factors. After you’ve assembled it, you have to boot it up into a live virus, and the difficulty of doing that depends a bit on the strain. Those are some of the factors. I think it’s pretty clear that hundreds of thousands of people in the world could, for example, follow a reverse genetics protocol to reconstruct a well-studied influenza virus and make it replication-competent, able to spread in humans.

Then the question is: of those hundreds of thousands, would anyone actually do it with nefarious ends? If AI creates this kind of uplift by teaching people how to follow lab protocols—which we can talk about a little more, and what that means in practice—is that number going to grow by orders of magnitude? I think we have a lot of indications that it is growing and becoming a lot easier. What might previously have been feasible for someone in the last year of their PhD could now be done by someone with a few weeks of laboratory training. I think that’s increasingly the case for a lot of biological workflows.

4. Measuring wet-lab uplift (Part 1)

If you wanted something entirely novel, we can talk about the uplift studies, or we can talk about the question of designing something novel. I think there are some more fundamental barriers there that, thankfully, are going to take a lot more to overcome.

Let’s do the uplift studies. It strikes me that, to some degree, we’ve been made safer than a naive arithmetic calculation would suggest by the fact that if you go through a biology PhD, you’re probably relatively stable. There aren’t that many people who are so deranged as to want to do something like this.

But the easier you make it, the further out into the tail of crazy people you push such capabilities. In the limit, if it’s just, “Hey, GLM 6.2 or whatever, make this,” then certainly people will do that, even if they don’t necessarily believe it’ll work. Who knows what comes in that scenario?

These uplift studies have been fascinating. I know they’re quite intensive to run. What does the setup look like? How often are they done? How high-powered are these studies, and what do they show?

Uplift studies measure how much using AI lifts someone’s ability to pursue a biological workflow. There were a bunch of studies around the time earlier useful chatbots came out, in 2023 and 2024, that were all just in silico. They looked at how well models could answer multiple-choice questions about virology, for example. People naturally responded that this doesn’t say anything about the physical side of things: the tactile tasks involved in operating a wet lab.

Over the last couple of years, since late 2024 and early 2025, and now in 2026, we’ve had several actual in-lab studies measuring how much AI helps people with different lab tasks. The biggest ones that received the most attention were from last year, and they found modest uplift, bordering on statistical significance. I don’t think that’s necessarily the relevant metric. We’re looking for practical significance: does it really lower the barriers and increase the pool of actors by an order of magnitude?

The studies from last year found modest effects, but I think this will change. These studies are very slow to run and very expensive. My organization co-funded with the Frontier Model Forum what I think is the canonical uplift study, which was done by Active Site last year. It cost more than $1 million.

We put 150 people in a wet lab. They were biology novices, relatively speaking. Half of them were given access to what was the frontier-accessible model at the time—I think Claude Opus 4 and ChatGPT o3—and the other half just had access to the internet. They were given tasks like cell culture, molecular cloning, or virus production, and the study measured how well they did. We found some uplift on some tasks, but not really on the internet workflow.

The mention of o3 is telling in itself. In a properly sane world, we would either find a way not to need to do this, or we would do it a lot and do it before release.

I feel strange sometimes doing this because I’m generally not a pearl-clutcher. Even seeing a few cyberattacks unfolding around us right now—I mean, there are a lot of cyberattacks—I’m not that freaked out about it. But the idea that there could be a self-replicating thing that could kill me and other people crosses a threshold where I actually start to get freaked out about it.

Yeah. We should be running more. I think it shows that, in some sense, the biosecurity ecosystem has done a really poor job. We’ve been caught flat-footed; there wasn’t really an infrastructure for this pretty unprecedented kind of study.

It’s not just that there are contract research organizations and academic labs all over the world that can do biological experiments. The challenge is bringing in people while thinking about who might be a threat actor, how much experience they have, and what workflows would be relevant specifically to biological weapons production—without actually having undergrads in a lab make bioweapons.

I think we’ve been caught a little off guard by the need for a different scientific infrastructure. That’s a lot of what we at Sentinel have been trying to build out. In a really good world, we’d have more of this. There’s a fantastic group called Active Site that’s doing these studies, and we’ve done this one that’s been publicly discussed.

They’ve done a few others that aren’t public, and that’s really amazing. But there should be 10 active sites, right? We should have it down to really easy, replicable science so we could do it every 3 months. Maybe not quite ahead of every model release—I think it’s hard to keep up with—but certainly not just once a year.

Yeah. Does anybody know the up-to-the-minute uplift status right now? Do we know what the Mythos answer is, or the GLM-5.3?

If you look in the Mythos system card, I believe there’s a study that Anthropic did with the biosecurity group at DETER. They had a number of PhD-level participants, either with access to Mythos or with internet access only, and they did a few virology protocol steps. They found that participants with the Mythos model—I presume the rails-free version—were outperforming the internet-only group.

But this was a really small study. I think the total sample size was fewer than 30, compared with the 150 we brought in for the Active Site study. Again, I think it shows the level of resources and infrastructure that would be necessary to do this properly hasn’t been built yet, and we have a really urgent task at hand in building that.

5. Measuring wet-lab uplift (Part 2)

One of the other data points that kind of spooked me was from the UK AISI. This was in, I think, their Q2 State of AI Development report. I’d have to go back and check exactly which model they tested, but suffice it to say, it was a couple of generations behind the current frontier. They found that models were more effective at troubleshooting laboratory experimental setups based on a cell phone snapshot than PhD students were.

Sometimes I feel like I’m taking crazy pills. I don’t know how that could be the case and we still wouldn’t see really dramatic uplift relative to Google. In my daily life, I feel like I’m uplifted in so many ways. Is there a skill issue on the participants’ part, or why isn’t this all coming out very clearly? How do you interpret that?

I think, on the one hand, I genuinely think uplift is a lagging indicator. These studies take a long time to conduct, and then they take a long time to release. It’s easy to lose track of just how quickly the models are getting better and how much more useful they are this year than they were last year.

There’s also this underappreciated point about how complex actual end-to-end biological work is. This is where the argument about tacit knowledge comes up. I think the first people who got really worried about AI and biosecurity were underappreciating this, and now people are being too reassured by it.

If you’ve never done wet-lab biology work, it’s easy to underestimate how much tacit knowledge is required. But if you aren’t using frontier models in creative ways or extrapolating trend lines, you’re going to take way too much comfort in that. I think both things are true.

There’s an academic, Sonia Ben Ouagrham-Gormley, who wrote a book some years ago about barriers to bioweapons. At the time, I think this was a really important thesis and contribution to the discussion. I also think these barriers are coming down.

Not only are they coming down incidentally; I think we’re, in some ways, pretty actively eviscerating them. Obviously, no one is motivated to do that for the sake of bioweapons development, but people are actively trying to make models more multimodal. We’re improving computer vision and speech-to-speech interactions.

One example is a company called Transfr that’s working on collecting huge amounts of visual data of humans conducting lab tasks. They’re setting up sensors in labs so that you can train robots on these very subtle tasks—how to hold a pipette and at what angle—or train smart glasses for teleoperation. Nathan, you can go and watch some of the clips of what this UI would look like: you wear a pair of glasses with a model trained on this lab data, hold up the pipette at the wrong angle, and there’s a visual overlay saying, “No, you’ve got to correct it a little bit in this direction.”

Companies are actively investing in this because it would be really awesome for democratizing the good parts of biology. But I think it’s also going to be a death knell for many of these tacit-knowledge barriers.

Is all of this inherently dual-use? I was fascinated just yesterday listening to Sam Altman on Cheeky Pint. I think Patrick asked the question, “Did we kind of make a mistake with this offensive cyber?” The assumption behind the question—which Sam didn’t address as much as I would have liked—was that it seemed like we’d trained the model to be so good at cyberattacking.

I don’t think we have clarity. That’s one of the things that’s frustrated me about the AI phenomenon recently. It’s unclear to me to what degree the frontier model companies have sent signals along the lines of, “Look, we didn’t train it to be an attacker specifically. We just made it good at programming, and the rest kind of fell out.”

But then you look at some of these behaviors. I was talking to the CTO of Vercel, who I think would know a thing or two about this, and he said, “You look at GLM-5.3, and it’s clear that it’s been trained to be an attacker.”

Yeah.

Or maybe it was Kimi K3; I’d have to fact-check exactly which one he was referring to. His take was that it’s clear the model has been trained to move as an attacker moves, and that’s not something he would have expected to fall out of getting good at general-purpose programming quite so fast. I wish we had more clarity on that. How do you see it in the bio domain? Is it the same, with both the upside and the downside?

Yeah. In both cases, the hard thing is that a lot of it is very dual-use. I’m not a cyber expert, but it seems to me that the prompt, “Show me all the vulnerabilities in this system,” could be what a defender uses to patch a system. It’s also what an attacker would use.

Similarly, a lot of questions and ways of working with biology are intensely dual-use. That’s where the whole trouble is: if it weren’t for that dual-use nature, we could just shut it down. We definitely don’t want to do that. One of the outcomes we’re trying to avoid is failing to reap the fruits of modern bioscience. That’s a huge part of the motivation for my work.

In the case of model training, you mentioned Kimi. I think Moonshot noted that with Kimi 3.1, they had deliberately left out certain cyber data. This is an open-weights model, and I think a lot of people are concerned that you can put in a refusal, but if it’s open weights, it’s not really going to hold.

So what they did instead was just not train on certain data they considered offensive cyber data. On the bio side, even if you do this kind of data filtering, if the data is easily accessible, it’s not that hard or expensive—especially because we’re talking about small amounts of data—to fine-tune the model after its initial training with the missing pieces, whether those are offensive cyber capabilities or dual-use bio capabilities.

So I think it’s better than actively training on it in the first place. But if the data is readily accessible, you haven’t really caused that much friction for attackers. That’s one of the things we’re working on at Sentinel as well: how do we create some kind of managed-access regime, not just for models but for the training data itself—for specific high-risk pathogen data, for example, virology technology data? That’s an intervention that could have some bite even on open models.

I did an episode with Jesse Pannu. Is that the same project?

Yeah, exactly. She’s one of the experts in this. Jesse has done fantastic work. There’s a paper in Science that I think Jesse was also an author on, which people can look at. It outlines a sketch of what this might look like.

But we have an enormous amount of work to do, both to categorize what data we would exclude and to figure out whether it’s enough to put it behind some kind of managed-access protocol. In an increasingly cyber-insecure world, will it eventually make its way into the models? And, if so, do you need to think about whether there’s certain data you shouldn’t generate in the first place?

Right now, a lot of people are rightly seeing data as one of the really big bottlenecks on harnessing AI for biosciences. There are enormous efforts to generate data, both philanthropic and commercial, and that’s necessary—we need to do that. But as we do, should we be thinking about whether there are some data we selectively decide not to create, and experiments we decide not to run, so that this data doesn’t get out there?

I know that the ESM models and the Evo series included some experiments where they trained models with and without certain virus datasets. In at least one of those cases, they showed that the model could still do almost everything else almost as well, but not these human virology tasks, if trained that way.

But I also think that one of the grand things we’re trying to accomplish is figuring out how cells work and where, mechanistically, we want to intervene. I remember a headline from a few years ago: scientists mistakenly put a minus sign in front of the loss function, and then the model was giving out poisons on every generation. It seems like as soon as we untangle all these mechanisms, we’ll inevitably—and probably inseparably—find things we’d want to positively intervene on, but also things that would throw a wrench into the cellular works.

What’s your current calibration on how much withholding data will actually accomplish?

Yeah, it’s super thorny. I think we have to look at these questions in a really honest way and not take things for granted. A huge part of how we operate in my organization is not dogmatically pursuing something just because we set out to do it, but trying to take a more empirical approach: is it going to work?

To do that, I think you need to put enough resources into it to ask: have you tried the best possible instantiation before you conclude that it’s not going to work? Are there creative ways you could segment this data to limit the downside while preserving the positive capabilities? If you try that for several years and pour a lot of really smart minds and models into it, and there’s just no way of setting it up so it works, then I think you need to pursue some other intervention. But the way to do that is to be empirical about it: experiment with these governance solutions, and do that as quickly and iteratively, and in as truth-seeking a way, as possible.

I think my take is that we’re still in that experimental phase. It’s such a recent idea. Jesse’s Science paper came out this year, I think, so we need to do a lot more experiments. I mean experiments broadly: both the empirical assessments you alluded to with Evo and ESM, and experiments in policymaking. What would it look like to write legislation to govern biodata? Can we be sufficiently clever about how we petition for that, or are we going to be bad at it and just unable to come up with any good governance regimes?

If that’s the case, you move downstream to other interventions. There’s a whole slate of interventions that are more in the physical realm than in the data or model realm. I haven’t reached that conclusion yet. The stakes are way too high, we have too few levers and too few good interventions available, and we have too little time to rule something out on the basis of a theoretical concern. I do generally subscribe to a layered defense, or defense-in-depth, approach. So, yeah, we’re pursuing the biodata stuff while also thinking about interventions at the digital-to-physical divide and in the real physical world.

Yeah, I think it feels to me like, in the end, we’re going to have to harden the world, and we’d better get to it.

6. Assessing near-term risk

Yeah, I mean, look, I agree. Going back to the very beginning, the thought experiment I’m running is basically this: let’s say we have a swarm of open-weight-style agents, and they’re not very well aligned. You could even imagine purely helpful versions, which people train for all kinds of experimental reasons. Sometimes people want to put those out in the world for ideological reasons, totally independent of bio issues. Let’s say you had a swarm of those things and they were actually trying to create a new, devastating pandemic pathogen. What can they do? What can’t they do? And how do you see that changing in the next 1 to 2 years? That’s all anybody can see anything of at this point, I think.

Yep. I think the things they would do very well at would be things relating to well-characterized viruses and pathogens. For influenza, there’s an enormous amount of data on both genetic analysis and protocols for reconstructing different strains. I think they’re getting to the point where they would do very well at that.

I definitely expect that. My prediction, which I made when the active-site uplift study we did with 150 participants came out, was that it would probably be the last one. I said it was really important that we be honest about the fact that there wasn’t a lot of uplift here, and I still stand by that prediction.

I think they would struggle more with designing something entirely novel, for 2 reasons. One is that biology is just extremely high-dimensional. This is also why you see big debates about whether we’re going to cure all cancer in a couple of years because of frontier AI. I think people make the well-reasoned case that there are so many dimensions to this and we don’t have nearly enough data to simulate it in one shot.

But, of course, as I think you also pointed out in our back-and-forth, we’ve been surprised by a lot of leaps in capability, and I think we should expect to be surprised here as well. If you’re worried about the end-to-end agent, with no humans in the loop and no social engineering, I think then it’s going to be more of a robotics challenge. At what point are the robots nimble enough? Have we created sufficiently end-to-end lab stations that can do anything you could imagine doing in this kind of virology work in a building without any humans? Or do we have very capable humanoid robots that could walk into a regular lab and execute a workflow?

In terms of timelines, I think what I said to you when we were discussing this a month ago was that a lot of the examples you gave of people being really surprised by timelines shortening were in the purely cognitive and digital domain, like chess or math. While we are seeing advances in robotics, I feel like we’ve seen fewer “Oh my gosh, this came 2 decades before we thought it would” advances in the physical world.

Even then, in the last month, I think maybe to your point, that validates your point. I do think I’ve started hearing more murmurs around robotics being more of an LLM-shaped problem than we might previously have assumed. I think Sam Altman was talking yesterday about the self-replicating data center idea more seriously than we’ve heard him publicly say before. So maybe there too we should expect advances.

But I think it’s very clear that today, when we’re recording this in October 2026, no amount of agentic intelligence would be enough to produce a live physical virus with the current level of robotics.

So that takes me back to this social engineering question. There are a couple of steps in this process where it’s unclear to me, and I think you have some interesting, maybe counterintuitive, analysis around DNA synthesis controls. I would naively think that both at the level of social engineering and at the level of actually getting the DNA synthesized for assembly or whatever, if you’re left to run in your little container for a while and you can try a bunch of different approaches, maybe some start to get through. Then maybe you have a real problem on your hands.

Certainly, there’s some baseline level of success at social engineering, especially if you start hacking billions of dollars out of Bitcoin wallets and then can pay people to do things. People are much less inclined to question the details of a task they’re being hired to do if they’re being really well paid for it, versus just being begged to do it. So that will increasingly be an arrow in the AI’s quiver.

If they’re left to run and try to do this, they don’t have direct API calls now, though that may soon change. But they can go online, message people on Twitter or send them emails, say, “Hey, I need this,” and try to deceive them. Where do you think that falls down? Do you think it would be at the level of social interaction, where people would raise alarms? Models aren’t that good at social engineering yet, although they’re definitely getting better. I’m getting a lot more sophisticated phishing attacks these days myself. I haven’t fallen for any yet, but I’ve come close a time or two. Or would it be at the level of synthesis? This is maybe a good place to tell us more about some of the work you’ve been directly involved in on synthesis controls.

7. Four phases of biorisk

Yeah. I’m very happy to talk about the controls, because I think that’s going to be relevant. Certainly, it’s going to be relevant in this sort of centaur phase, where you’re worried about a human who might be instructed by an AI, blackmailed, or coerced. The playbook we have around “know your customer” will eventually have to become KYA: “know your agent.” But if the near-term threat model over the next couple of years is social engineering, then the playbook we’ve been building around KYC and monitoring the use of different services is going to be extremely relevant. So, yeah, let’s talk about it.

Imagine a swarm of agents. What are they going to run into as they pursue this nefarious goal?

8. Screening DNA synthesis

Yeah. Okay. So, we’re imagining that we’ve failed to constrain the model with classifiers or keep certain training data out of its hands at the training stage. We’re now at the point where you’re trying to take a digital design—you’ve successfully designed some nasty virus—and produce it in the real world. I think there are 2 categories of interventions we’re actively working on. One is restricting access to materials. Synthetic nucleic acids, or synthetic DNA, are the archetypical example here. These are essentially the genetic building blocks we use to assemble a genome of any kind, so we could restrict and monitor access to them.

The second category is various services. You may have designed a protein but aren’t sure how it works, or you don’t have the facilities to test a new interaction you designed. So you send it to a contract research organization or a cloud lab, perhaps just sending them the design and asking them to synthesize it, run some experiments, and tell you how it worked. At both the materials and services levels, there are a lot of levers we can pull to deter, deny, and monitor any kind of bad actor, whether it’s an agent or a human.

So tell me more. I was interested to see that, in the process of zeroing in on a couple of these choke points, you evaluated hundreds of possible choke points. I’m interested in what makes these so natural, and what else you learned from reviewing so many possibilities. Then there’s the question of whether everybody needs to do it. I think you have an argument that we don’t, and that we can get a lot of value even if not everybody adopts. That sounds like it might offer some relief to me. So, I want to hear.

We did look at synthetic DNA as a starting point, but also things like different cell lines, different mammalian models—mice used for testing certain things—pipettes, lots of biosafety cabinets, and a whole range of things. What we’re really looking at is, first, whether there’s a high signal-to-noise ratio. Is this something uniquely used in, let’s say, virology workflows? Second, is it commoditized and decentralized? In the extreme, if it’s something you could easily 3D-print or buy in any local store, it’s going to be very hard to govern.

The things that perform well include synthetic DNA. It has relatively high signal-to-noise, in the sense that the sequence tells you what it is. You can understand something about the intent from what’s being ordered. And if we’re talking about gene-length DNA, it’s relatively complex to manufacture: only 200 companies in the world do gene-length, double-stranded DNA synthesis. Shorter DNA fragments are more distributed and accessible, but also harder to work with.

One lesson from that exercise was that many things don’t have a good signal-to-noise ratio. The only way to control them would be to effectively ban them, which, in my mind, isn’t an option. That would completely hamstring the defensive side of countermeasures and, more generally, life sciences for human flourishing.

You have a handful of things left that you could try to govern. What you need to do is aggregate signals across them. If someone interacts with a DNA provider and orders something that isn’t an outright red flag—they haven’t ordered a gene from smallpox, but they’ve ordered something that isn’t homologous to anything we know from nature, and they won’t explain what it is—that’s one signal. If they’re also querying a monitored proprietary model about how to avoid the attention of law enforcement, those signals together tell you more than either would individually. And if they’re also reaching out to a cloud lab and procuring something that looks suspicious, we need monitoring infrastructure that can aggregate these signals.

A nice benefit of that approach is that it’s less reliant on refusals that create friction. You’re not denying access to a model or a material; you’re aggregating information. In the very, very rare severe cases, you might escalate it to law enforcement for investigation.

9. Outro

How long does this stuff take if you’re doing it as a human? It struck me, as I was thinking through this ahead of time, that one limiting factor for agents might be that they just don’t run for that long. Maybe we could move toward not having multi-month agent life cycles, because we can get a lot of value out of them quickly. But if the same rogue process is still there weeks later, when somebody finally gets back to you, that opens up a whole degree of asynchronous activity that’s pretty normal for humans, but maybe we could cut the models off. What would calendar times look like?

Yeah, I think there are going to be very strong commercial and other pressures to increase reasoning horizons and autonomous horizons. In fact, reasoning horizons have been one of the main benchmarks people have been optimizing against pretty explicitly, and I get why. For anything really useful, you want to do it over a long horizon. So I think we have to plan for a world where models can act autonomously over fairly long horizons. I don’t think we should say, “Let’s pour sand in the gears of the bioscience supply chain,” just to slow things down.

One use of synthetic DNA is to rapidly create diagnostics in the early days of an outbreak, when you need to synthesize short fragments of parts of a pathogen, for example, to create a primer or probe for a diagnostic test. You want to be able to do that really quickly. So I think we should be willing to accept some friction in the name of doing KYC, but we shouldn’t deliberately make things slow as a tarpit for agents. I think there are better avenues available to us.

So, bottom line, where do you think we are today, and maybe over the next couple of years? How should I think about the risk across all of your different scenarios that involve AI? Let’s leave aside the rogue PhD who doesn’t need AI at all, and try to assess the incremental risk AI has brought to us in the bio dimension. I don’t know that I even know the right metric to define it. The percentage risk of a bad pandemic per year would be one way to think about it.

You may have a different way of defining it, but what would you estimate that number to be now? And how do you think it will evolve over the next couple of years by default, let’s say?

Yeah. Look, actions speak louder than words, and I’ve devoted my life to this. I’m going out every day, recruiting people to do the same and fundraising to do the same. I’m here talking to you, Nathan, because I think we desperately need more people and more diverse talent tackling this problem.

I think it’s really, really high on the list of societal challenges. If we’re talking about things that can outright disrupt civilization, it’s also very high up on that list. I don’t think it’s the only problem or necessarily the single biggest problem, but if you look at how many people are working on it and how tractable it is, I think it nets out as a really, really compelling thing to devote one’s time, resources, and attention to.

I mean, I guess I would think that, though, at like a tenth of a percent per year, and maybe even a hundredth of a percent per year. This is why I got into AI safety as a thought exercise, when there were no AIs. I used to tell people, “Look, the dinosaurs got taken out by an asteroid, and I would hate for that to happen to us, and I guess you would, too.” Most people would be like, “Yeah, that sounds bad.” And then I’d say, “Well, good news is we do have at least a few people who are looking out at the skies, trying to identify anything that might be coming our way. We should probably do that for AI, too,” even though it may be a remote thing.

I guess I’m still trying to calibrate a little bit: Is that the regime that you feel we are in, where it’s what most people would say is a very low number, but the severity is so bad that it deserves more resources? I buy that argument, but I’m also not sure whether we might actually be drifting up into a range where we might need to take more drastic measures, like pausing frontier-scale RL on bio tasks, which I wouldn’t do lightly because I share your interest in getting the upside of all this stuff.

You should do an episode with someone just to talk for 2 hours about the amazing things we’re going to be able to do with bioscience. Maybe you could get someone from the OpenAI Foundation’s life sciences team. I think there are a lot of wonders there to explore, so I’m totally with you on that.

Yeah. I think, look, I’ve done my personal preparedness for disease outbreaks. I think that’s actually just general good practice for anyone who’s lived through COVID. In terms of how I’d make the case, Nathan, I think it’s still more of a low-probability, high-severity event. The other challenge is that patching the planet—patching humanity’s immune system—is a relatively clear task. I think we know what to do much more so than for other challenges in transformative AI governance, but it’s a very hard task and a very expensive task.

The end state we’re trying to get to is, in some sense, a culmination of centuries of effort in global health. It’s one we’ll have to defend against much worse threats than just nature’s own, to boot. That’s a state where we have really ubiquitous detection and measures for suppressing transmission in the initial stages of an outbreak: germicidal light, PPE, and extremely flexible, modular platforms for countermeasures that are globally and equitably distributed. That way, when an outbreak is happening, we’re not rolled over immediately, and we can squash it in a way that’s actually protecting billions of people.

I think that’s an achievable end state. It’s very clear and coherent, but, as I said, it’s a decade- or decades-long project. We really have to get our act together and buy as much time as we can while we’re doing that. That’s the core of what we’re doing in the biotechnology governance and misuse-prevention space that I’m working in.

In terms of the organizations or entities that will train the models that present the most risk, do you expect that will be the usual suspects—let’s say, 5 frontier model companies in the United States? Or should we also expect a lot of that to come from pharma companies or startups? In the short term, is this something that will fall out of a $10 billion training run, or do you think you could also see it pop up at Pfizer with a more moderate-scale effort?

I think the risks depend on what you’re worried about, Nathan. If you’re worried about some Millennium Prize-level, completely unexpected breakthrough in the more fundamental parts of biology—being able to simulate things we thought were completely computationally intractable to simulate—then I would probably expect that to come more from a frontier lab.

If you’re thinking more about the baseline competence of biology writ large, where the number of people who can competently reconstruct a pandemic-capable virus grows by orders of magnitude, that’s going to be a death by a thousand cuts. A lot of different actors will contribute by making different workflows more accessible and effective. Of course, a lot of this is going to converge in the limit. There will be a lot of efforts to vertically integrate different parts of biology so that a single institution can do everything end to end.

We see that at the model level, too. One of the more interesting papers that came out of SecureBio and the UK AI Security Institute recently was an effort to see how well frontier models—I think they tested Astra and Opus 5.5, from memory, maybe a slightly earlier generation—could perform if given a dataset and a paper explaining the architecture of a narrower biological design tool. These are design models that are more specialized for biology. Could the frontier model train a model like that?

In that sense, the question of whether a risk is coming from the developer of the narrow biological design tool or from the frontier model kind of collapses, because they’re becoming more and more integrated. I think that’s a general trend we’re going to see in both the model domain and the laboratory domain: this kind of integration over time.

You would know a lot better than me because, as you alluded to, you recently wrapped up an advisory tour of duty at the OpenAI Foundation. There’s an unbelievable amount of money sitting in that foundation, and there’s kind of a mirror image at Anthropic. There’s a demonstrated ability to speedrun certain kinds of problems if they stand in the way of continuing training, right? We’ve seen, “Oh, we paused training for a week because we found more defects in our sandboxes.” Well, that’s not a very long time. You must have felt pretty good about your ability to patch those things in a week.

I obviously don’t think that building what you sketched out—something that sounded like the win condition you imagine for us here—is going to take a week by any means. But when you say decades, I’m kind of like, well, is there a deal here or an ultimatum the frontier companies should be given by the government? Something like: “You guys have hundreds of billions of dollars, and you seem to be really good at speedrunning these problems. Fix it, get us to a good place, and then you can resume training.” That might be—

Yeah. I mean, look, I would love for it to go much, much quicker than decades. Maybe there are going to be transformative things happening that would allow us to move faster. When I say decades, again, I’m talking about the ambitious win state: how do we actually achieve resilience for the entire world?

True.

I think we should double down, yeah. And I think we should go pedal to the metal on that. No one should think we’re on track to doing that. There are subcomponents of it: Could we secure this type of critical infrastructure? Could we protect this group of essential workers? If we really put our minds to it, we can do those things more quickly.

But if anyone feels reassured by seeing some investment in biodefense and thinks, “Okay, therefore, let it rip,” they haven’t really run the numbers or the timelines of how long it would take to actually get to the win state. That’s why I’m so motivated to ask what we can do to prevent misuse in the first place while we’re building resilience.

Some of that is going to happen at the model level, in terms of training runs, but a lot of it is also going to happen through governing the materials and outsourced services. We’re seeing increasing momentum around these ideas.

I think you’re seeing lab leaders and synthetic biology leaders signing on through letters calling for things like mandatory DNA synthesis screening. We still have quite a lot of work to do.

10. Debating an AI pause

Zooming out, do you think we’re acting foolishly collectively by doing this all at the same time and piecing it together? If you were the benevolent dictator, would you take my idea and declare a pause until certain standards were met? We used to have this in RSP form, right? If-then commitments and so on. Would you impose a standard of security that we need to meet as a society? And, by the way, frontier companies should use their foundations to create the public goods we need, but they can’t do this stuff until we’re there. Would you make that call?

Yeah, I would, for a number of reasons. Bio is the area I know best, and it’s where I can most confidently say we’re not on track to making the world bio-resilient on any short timeline whatsoever. We’re trying really hard, there’s a lot we can do, and we need to accelerate the effort. That’s why we’re recruiting and bringing people in to do it.

If anyone’s telling you we’re just a few more grants away from having a completely bio-resilient world, and therefore we can plow ahead with the underlying technology that’s contributing to the risk, I’d say that’s crazy. I suspect it’s probably the same in many other domains of resilience. We’re seeing it now with cyber. So, if I were holding that button, I would absolutely press it.

Yeah. Interesting. Is there anything you could say about your experience as an adviser to the OpenAI Foundation related to that point? Is that sort of trajectory in the Overton window at all, in your experience?

Well, I rolled off my role advising the foundation a month or two ago, and I think there’s been a huge shift in the Overton window even in that short span. More generally, I see a seriousness around these considerations that we didn’t have even a few months ago.

I’m not the best person to say what the official view within the foundation is at the moment. I will say I think it’s a group of very morally serious people who will hopefully wield the potentially very significant resources at their disposal with a lot of responsibility and urgency. I’m definitely rooting for them, and for the work we’re doing at Sentinel in parallel. I hope you’re enjoying your role as the benevolent dictator.

Yeah. What would be your instinct on what to pause until what is accomplished?

I’m less well placed to comment on what specifically to pause. What I’m fairly well positioned to say is where we are on the road to the victory conditions for bio-resilience. That might answer your question about when to unpause in your simple framework.

The victory conditions we’re working with at Sentinel are about getting more than 80% of providers of certain materials and services, such as synthetic DNA or cloud labs, to do the kind of KYC that would allow them to refuse an illegitimate query, or the kind of monitoring and aggregation of signals that would empower law enforcement to intervene capably. I think that’s something we can achieve in the coming years.

I think there’s going to be a time, invariably, when the world wakes up to the seriousness of doing this. What we need to do is build the infrastructure so that we know exactly which DNA sequences we’d be looking for, what signatures of misuse we’d be tracking, and how we’d write it into legislation. Then, when there’s the political will, we can scale it up very quickly.

In parallel, there’s the game of getting to the end state on the bio-resilience side. Like I said, that’s on the order of years if you’re talking about securing specific subcomponents of critical infrastructure and essential workers, and a much longer timeline if you’re talking about actually making the world pandemic-proof.

If you sit down and run the numbers—how many people would you need to get PPE to, and how widely distributed would vaccine manufacturing have to be in order for it to be truly global—I think that, unless you assume very transformative changes in supply chains, logistics, and manufacturing, it’s on the order of many years with anything under normal industrial capacity.

But we should shorten that. That’s what the biosecurity community is trying to do: bring forward this end state in time. That’s how we work: set an end state and work backward from how we get there.

How expensive is this stuff? When you think about PPE, for example, there are obviously a lot of different classes of PPE, and I’m sure they come at different price points. But if you’re looking at the OpenAI Foundation, which has hundreds of billions of dollars today, and its equity could potentially 10x again if its plans come to fruition, there’s plenty of money to buy everybody the PPE they need, right?

They’re sending these demand signals into the compute supply chain as much as they can. Sam was telling Patrick Collison that the invisible hand is failing to some degree because people have been burned by risk or whatever. My guess is that if they were ready to write a $10 billion check to buy everybody in the world—who knows how many masks that would buy, with a volume discount—you could get five for each person and stockpile them for the next moment they’re needed. That wouldn’t even be a big part of their budget, right? How much can we speed-run these things if we have a proper sense of urgency and willingness?

Yeah, I think we can certainly do a lot better than we are now. The state of stockpiling in the United States, let alone in less-resourced parts of the world, is really abysmal. I think we would not fare well in a pandemic today.

I think it’s also within the realm of what’s philanthropically achievable to protect critical workers. One estimate I’ve seen is that for $1 billion, you could equip 30 million critical workers across the U.S. with high-performance reusable respirators. The value there lies not just in dividing $1 billion by 30 million and saying that’s the number of lives you saved. If you enable critical workers to go to work, you can shelter much larger numbers of people from transmission just by staying in.

But of course, that’s a lockdown state that I think there’s very little appetite for. We might do it begrudgingly if we faced the threat of something much more lethal than COVID, but it’s not the glorious future any of us are hoping for. So then you need to get considerably larger numbers of people covered. The costs won’t just be the equipment itself, but also the distribution infrastructure and all the auxiliary work—especially in a crisis where there might be a breakdown of social order—to make sure people are actually using things correctly.

There’s a lot of extra work to be done, but we should absolutely do it.

Yeah, I mean, it feels small compared to the data center buildout. That’s really where I’m coming down on this.

Yeah, a lot of these comparisons will make your mind completely numb if you think about how little we’re spending on extremely admirable things relative to things that certainly have utility, but that a person on the street might not prioritize. There are a lot of priorities that rank higher than data center buildouts.

I’m really heartened by the resurgent interest in bio-resilience philanthropy, but I think it’s also really important to hold ourselves accountable—not for the resources that have been committed or speculated about, but for whether we’ve actually made the world safer for people and reduced the risk of these outbreaks.

At Sentinel, we’ve raised tens of millions of dollars and made around 100 grants to different projects. That’s really awesome. But until we’ve actually made it harder for a bioterrorist or a rogue agent to get their hands on dangerous materials, we have not won.

I think that’s similarly true in the bio-resilience space. We need to hold ourselves accountable for whether people actually have this PPE at home. Nathan, I heard you talking on one of your other episodes about the struggle to get far-UVC installed in, I think, a hospital or a school. That’s the level of accountability we need to focus on: are these things out there where they’re actually protecting people in the U.S. and beyond?

Yeah, the public—the sort of man-on-the-street perspective on this definitely cuts both ways.

On the one hand, I do feel increasingly like there is often some wisdom in the naive take on the macro situation. A lot of times, people, especially if they're hearing about it for the first time, are like, “Wait, you're saying what?”

“What are we doing?”

Yeah. They actually believe that, and then they're like, “Well, if they actually believe that, then this is crazy.” I think they're basically right about that a lot of the time. Then, of course, it also goes the other way, when it's like, “Oh, by the way, I'd like to put this little light in the school classroom,” and it's like, “Whoa, whoa, whoa, whoa, whoa. We're not—we can't do that.”

Yeah.

So, yeah, we can't rely on the man on the street for everything. I appreciate you going down this thought-experiment path with me, though, because I increasingly have the sense that we're playing with a new kind of fire here with AI in general. I've had that sense for a long time, and my sense of the big picture is increasingly that the outside view is, “This is crazy.” Many of us, including myself, who have been relative insiders, if not the ultimate insiders, get frog-boiled and constrained in our thinking by what other people around us think is reasonable.

It does help to have a fresh perspective on this bio stuff. I haven't had as much immersion in it, so maybe I'm coming at it a little more from the naive “this is crazy” perspective.

Yeah. I think, in a nutshell, my take, Nathan, is that my timelines to very, very high-probability AI-driven bioattacks are probably longer than you're intuitively feeling, and I think you can rest easy knowing that they're longer than that. I think my timelines for how long it will take to actually patch humanity's immune system are also longer than some of the simplistic estimates—like, “Oh, you do a back-of-the-envelope calculation, add up the costs of buying PPE, and therefore it's easily solved because this money is potentially coming online.”

I think I have longer timelines on both. On the pausing question, one of our board members was pushing us a lot on what we could do on a 3- to 6-month time frame if we were to reorient all of our work toward that. My answer would be that I would try to spend that time extending the timelines, because we've got some smart people on this case, and we're going to need more smart people. Even so, these are hard problems to patch.

I think we're seeing that with cyber now as well. The idea is to give defenders early access and then think, “Okay, your 6-month clock starts now. Get it in order.” That's a costly gamble to take.

Yeah, and even more so potentially in the bio domain than in the cyber domain. That could be plenty bad, but this is a different kind of thing. Bringing ourselves back into the Overton window of what's probably likely to be possible in the short term, what do you think are the most urgent things, the highest-leverage, lowest-hanging fruit? I know you have this framework of “owning the problem.” Maybe you can present that frame and give us some problems you think are currently going unowned.

11. Neglected biosecurity bottlenecks

I think that frame is the idea of really focusing on getting something all the way to the end state. I wrote an essay about this about a year or a year and a half ago. What I was reflecting on is that, in the social sector, for people who care about having an impact, there are a lot of incentives that spread people thin. One is the idea of, “Oh, you only work on this one issue. Does that mean you don't care about this other issue?” I think that creates social pressure to make a token effort across a bunch of different things.

I think there are professional incentives, too. Another social incentive is that I would love to spend my time reading about a bunch of different things, so when I go on Nathan's show or to a cocktail party, I have a tidbit of information across a whole spectrum of things people care about. I think this pulls people away from saying, “I'm going to pick one thing and just get it over the finish line.” This is really unfortunate, because I think there are incredible returns to focus.

If there's one lesson we've learned in building Sentinel over the last few years, and that we pass on to our grantees and the organizations we incubate, it's that there are nonlinear returns to focus. I could give a bunch of examples of a small number of people who really made an effort to say, “Okay, we have 3 or 4 years to solve this problem of securing synthetic DNA. How can we get laws introduced in the U.S. and in Europe? How can we make sure that the tools exist to make it easy to comply with these laws?” I feel very good about that playbook, and I encourage others to take it on, in spite of some of those social and professional incentives.

In terms of an example, I think we've made a lot of progress over the last couple of years in securing mail-order synthetic DNA. That's where you go online, type in DNA.com or whatever, add the Ebola gene to your shopping cart, and then it gets shipped to you. We've made some progress in making sure the companies are more likely to screen that order and reject it if it's not legitimate.

But that leaves the next frontier vulnerability: the printers that synthesize the DNA themselves. They need some level of hardware security. We need some kind of KYC at the point of sale, and we might need geolocation. If you're really trying to throw money at the problem, you could think ambitiously about buyback programs to buy old machines that are hard to secure and retrofit them.

This is a really great example of an orphan issue. A lot of people will think about it, and a lot of people may have mentioned it in a paper or a tweet. But in terms of people who are waking up every day and trying to say, “I'm going to solve this,” by my account, there are a couple of scattered efforts across a few programs, but it's really an ownerless problem. That's something we're actively recruiting for at Sentinel.

How many similar things are there? You can list a few more if you want, and is there a canonical list you'd point people to?

Within the specific issue of securing synthetic DNA, my colleague Yan Kashmiri and I wrote a piece for the Institute for Progress, which I think is called “How to Secure the DNA Supply Chain.” It's based on a couple of years of work mapping out a whole road map and indicating which of these issues have existing efforts and which don't.

I think there's a bunch there. One thing that feels very neglected is thinking outside of just the U.S. We're increasingly seeing a lot of attention among technologists in the San Francisco Bay Area and a lot of attention among policymakers in D.C. Those are probably the two places you would start, so I'm glad we're covering priority one and two.

But I think there's a huge swath of work in international diplomacy. How do we get the most talented diplomats in Geneva, or international relations people traveling between Beijing and Singapore, to take this issue very seriously? How do you get them to reorient from working on something related to the U.N. Security Council or whatever toward solving these extremely urgent biotechnology and, more generally, technology-governance challenges? I think that should be one of the most compelling and prestigious things to work on. That's one of the really big challenges. We have our work cut out for us.

12. A positive bio future

Maybe one last question, because I think it's important to be intellectually honest about the trade-offs if we're talking about any sort of pause or slowdown. It's also important to be inspired and have a positive vision for the future. I'd love to hear a little sketch of your positive vision for the future. We focus a lot here on the downside and risks, how things could go wrong, and what we might ought to do about it. But you mentioned, at least briefly, that the reason you do this work is that you want to see us actually realize the upside of biotechnology.

In bio, I think there are 2 components to it: the wartime and the peacetime. On the wartime side, there are pandemic outbreaks, and even if we were to eliminate all misuse, we'd still be at risk from nature. If you tally up the deaths from the greatest causes of death in the 20th century, things like smallpox and pandemic influenza dwarf many wars, including the world wars.

Taking that off the table by getting to this end state of extremely widespread, sensitive, and specific detection that is acted upon rapidly, with very fast and modular diagnostics, is important. I think we were beginning to see the inklings of that in how we dealt with COVID. A lot of how we handled testing was not well executed, but being able to produce the tests very quickly and design them within the first few days of sequencing the genome was a really inspiring step forward.

I think we’re seeing the further endgame: We’ve identified the pathogen, we know how to diagnose it—how do we knock it down? mRNA vaccines were a real step change in our ability to deal with emerging infectious diseases, and I’d love to see many dozens of step changes like that.

I think that’s a world where we’re no longer at risk, and we’re no longer trying to haggle with our local school to set up UV. People who are immunocompromised can live with less peril from emerging outbreaks. I think that would be incredibly freeing. A lot of people, because of different conditions, live with perennial exposure and fear of outbreaks, and I think many people still carry that with them after COVID.

Taking that off the table—as we put it in Sentinel’s mission statement, “making pandemics history within our lifetimes”—captures it pretty well. Then there’s the non-infectious disease side of it, which isn’t my area of expertise. But I think anyone who lives with chronic illnesses or cancer feels that any amount of delay in taking those off the table is unbearable. There’s an enormous amount of work to do there.

I think the equity component is often missing from conversations among technologists. People talk about the future being unevenly distributed, and I think that’s definitely the default. I’m extremely suspicious of arguments that we’ll distribute the spoils of this technology by default. We need to build that in as early as possible, and I think we can do a much better job of it. I hope we will, because I hope people will hold power to account on that one.

That could be a great place to leave it. Is there anything else you’d want to leave people with or touch on that I didn’t think to ask?

The candid assessment is that it’s a pretty dire state of affairs when it comes to bio risk, but I think we have many more wins ahead of us than behind us. There’s an enormous amount of clear, tractable work we can do over the next few years. If you’re a legislator daunted by the complexity, someone considering pivoting your career into this work, or a philanthropist thinking about funding work in this space, I think we have to take some courage and hope from how much we can get done. It’s very easy to underestimate what we’ll be able to do over the coming years. We just have to get to it.

Joshua Monrad, thank you.