Bowen Wang
What really blew my mind was when I saw the model make a prediction. You can print out the heat map of the gene expression changes, look at the actual raw data, and line up the linear-baseline prediction, the ground truth, and the X-Cell prediction all together. It’s visually very clear to see that the X-Cell prediction is much more similar to the ground truth than the linear baseline. This is the wow moment I was talking about in the beginning: this is the first time that someone can put together not just 1 Perturb-seq, but 7 genome-wide Perturb-seq campaigns together.
Cichu Chen
Something that jumped out to us biologists right away is that some of the perturbations are context-universal.
Hi, I'm R.J. Haniki, CTO of Mirror OMIX. This is Brandon Anderson who builds RNA therapeutics at Atomic AI and this is the Latent Space AI for Science podcast. One of the themes that has run through the podcast is how the lab and experimentation and the real world have probably the biggest impact and have the most relevance to whether something is AI for science or something like B2B SaaS.
Alessio Fanelli
We’re really happy to have in the studio with us today Bo Wang and Cichu from Xaira Therapeutics. At Xaira, they’re building, with a bunch of other people, an AI drug-discovery platform. They’re using a high-throughput experimentation system to collect very large data sets and then training AI models that can predict the way that your cells in your body will respond to drugs and therapeutics. Really happy to have you. Big fan of your work. Why don’t you 2 introduce yourselves to the listeners?
Bowen Wang
Hello, everyone. My name is Bowen. I’m SVP and head of Biomedical AI at Xaira Therapeutics. I joined Xaira about 8 months ago, and before that I was an associate professor at the University of Toronto in Canada.
Cichu Chen
I’m Chu. My first name is incredibly difficult to pronounce unless you speak Mandarin, so I go by Chu, as in Chewbacca or Pikachu. I think those are your favorite fictional characters. I’m the SVP of AI-Enabled Discovery at Xaira. I joined more than 2 years ago, when it was still in stealth mode. Here, I lead the high-throughput biology group generating the kind of data that will feed our AI models, and I also think about their applications.
Before this, I spent about a decade at the intersection of AI, big data, and biology. Previously, I worked at Insitro, leading the in silico discovery platform there, and before that I was at Verily, which spun out of Google X.
Alessio Fanelli
Okay. So you are at Xaira, the company that’s on the frontier of confusing names and mega-rounds. Xaira came out of stealth a few years ago and raised a really big round, kind of out of nothing. I’m curious if you can explain a little bit about what Xaira’s mission is, what its thesis statement is, what’s special about Xaira, and where you’re going in the future.
Cichu Chen
Yeah, Xaira is an AI-enabled drug-discovery company. At the core of our mission, we’re using AI platforms to generate better therapeutics, to advance patient care, and so we will be making drugs using different AI capabilities.
There are 3 main AI platforms that we’re building here. The first one is protein design work that spun out of our co-founder Dr. David Baker’s group at UW. A lot of the current generation of protein designers are here in the company. The thinking is to use advanced AI technology to develop molecules against previously undruggable targets.
The second AI platform, which I think we’ll spend a lot of time talking about today, is the one that Bo and I have been working on for quite some time and just released a preprint on. That’s the virtual cell, or foundation model of biology, work. There, the hope is to build an AI model to predict biology exactly like you said, and predict what genes and drug molecules will affect cell biology.
The third piece, which we’re beginning to build now, is patient-representation models. The goal there is to have AI models that can understand which patients will respond to which therapeutics. Hopefully, together, these platform technologies will help us make better drugs faster and with a higher success rate than previous technologies, to transform what used to be artisanal trial and error into more and more of an engineering discipline.
I think what sets Xaira apart is not just the $1 billion round, but also that Xaira is one of the very few AI-native companies for drug discovery that works from end to end across all sections of drug discovery, from as early as target identification and protein design, through small molecules and into Phase 1 to 3 clinical trials.
We aim to use AI to accelerate every part of drug discovery so that not only do we increase the success rate of developing drugs, but we also greatly reduce the cycle time. That way, we can have new drugs instead of waiting 10 or 20 years. Hopefully, we can reduce the cycle time so that we have more useful drugs for patients.
Alessio Fanelli
That’s really interesting. I know there’s a lot of interest right now in that third thing, sometimes called translation from the lab to the clinic. Where are the bottlenecks? You have these 3 models. What are the bottlenecks that you’re addressing, and how are you doing that? Why are you doing it that way?
Bowen Wang
As an AI-native company, in almost every part of drug discovery, we’re trying to use AI to revolutionize how we develop drugs. In the early part, we build causal foundation models, or sometimes we call them virtual cells. For proteins, we have state-of-the-art protein-engineering models, and we also have patient-representation learning models.
I think what Xaira is trying to do is not only develop AI models, but also create the right data sets to empower these models. What really makes me excited to work at Xaira is that we always aim to connect the 3 AI models together, instead of letting them work individually on their own.
When we design virtual-cell models, we look for connections. Can we find targets that are easier to apply the protein-engineering models to? Even when we design the cellular causal models, can we connect them to patient representations? What are the right patient data to connect to the cellular models so that we have something to show for clinical utility?
What really makes me excited is that before I joined Xaira, I was a professor in a computational biology or computer science department, where we mostly worked on computers. We looked at the data, arrays, and so on. But once I came to Xaira, what really excites me is that I get to talk to people like Chu and lots of drug hunters—extremely experienced drug hunters—to really understand their pain points.
When we design AI models, we think about questions that really excite biologists. Later, maybe we can talk about how one of the rewarding signals I received after we developed X-Cell is the wow moments from biologists: This is the first time biologists actually found that the model could predict exactly how these unseen cell lines respond to different perturbations.
That’s the part that really excites me: the integration of the dry lab, or AI models, with the wet lab, or biology, and eventually even with the clinical side.
Alessio Fanelli
With this clinical model, I know you guys are aiming to take a drug all the way to FDA approval and beyond. Where do we stand now? I don’t know if you’re able to talk about this, but are you able to collect data from clinical trials and tie that back yet?
Cichu Chen
As Bo said, if you think about the drug-discovery process, it’s easy, right? You just need to find the right target, make the right molecule, and find the right patients to give them to. Of course, each of those steps is incredibly difficult to get right. So far, it relies a lot on trial and error and guesswork. The main issue, I think, is that we don’t have the right biological data to power the training of a predictive model. In the protein-design space, I think that’s where we’ve seen the most rapid progress so far.
Bowen Wang
That’s partially because we have a lot of high-quality data, over 70 years curated by the entire community. People deposit protein structures into a database called the PDB. We also have a lot of sequence data collected over the years from different genomes that can help inform the model as well.
Cichu Chen
It’s these high-quality data that are collected and accumulated that ushered in this revolution in protein design, AlphaFold, and other folding models. In the other domains, such as clinical outcome prediction and virtual cells, we’re nowhere near the same kind of massive, high-quality data. I think it’s mainly a data-limitation issue.
To your question, that’s where we’re very invested in generating these data, particularly causal data in cell biology in a lab. That’s what made it possible to innovate on the algorithm side as well, to usher in virtual-cell models.
On the patient side, it’s a very interesting question. Perhaps that’s some of the hardest data to get, because getting access to high-quality patient samples is difficult in itself. Getting them matched to the right clinical annotation, so that you can actually learn the bridge between molecular data and clinical response, is even harder.
You might be able to do that across different disease severities, but it will be harder to collect the right data to predict which drug treatment a particular patient will or will not respond to. That takes a lot of thought and careful curation to generate data from. We’re beginning to go into that area, but hopefully we’ll be able to share more soon.
swyx
Awesome. Maybe we should switch gears now. You just released X-Cell. Why don't you guys describe it? I'll butcher it.
Bowen Wang
X-Cell is Xaira's first virtual cell model. It is an AI model that can predict the response to genetic perturbations. Certainly, we can extend it to other types of interventions, such as drug perturbations, chemical perturbations, et cetera.
swyx
Can you just describe, for the nonbiologists who are listening, what is a perturbation? What do you mean by that?
In our cells, when we talk about genetic perturbations, a typical human cell has 20,000 genes. Not all cells express every gene equally. That's why your eye cell, your skin cell, and your heart cell, even though they share the same genome, function very differently. A lot of that is determined by selective gene expression, which determines the type and state of the cell.
What we do is build a model where you can, in silico, ablate certain genes from the cell. That is an in silico perturbation. If I reduce the expression of this gene in the cell, what is the implication for the rest of the cell? What's the biological consequence?
swyx
You basically turn the knob down on one gene.
That's right.
swyx
And then what happens to all the other genes in that cell?
Correct. The hope is, of course, to predict the effect on all the other genes, but maybe even more than gene expression, such as the function of the cell.
swyx
Okay.
Bowen Wang
That's important and therapeutically relevant because a lot of drugs are inhibitors, and they function through exactly that: turning down the activity of a protein or gene. If we can start with gene perturbation prediction, the hope is that we can also go to pathway inhibition prediction, so on and so forth.
A pathway is just a set of genes that all kind of talk to each other. This gene expresses a protein; that protein has some impact on another gene, and so forth and so on. There's this long chain reaction of genes and proteins. That's called a pathway. If you interrupt that or somehow change it, then that has an impact on the larger phenotype of the cell—what the cell looks like, what it does, et cetera.
swyx
That's exactly right. Yeah. So you have what you called a virtual cell, or you're creating a virtual cell, and virtual cells are very popular these days. A lot of people are interested in this concept, but I think your approach is somewhat unique or separate from what other people are doing. Can you explain what people broadly mean when they say “virtual cells”? What are some of the distinct strategies, and what is your specific strategy?
Certainly, virtual cell is a very high-level term to describe an AI model that is able to predict or describe what a cell looks like, or predict the cell's expression or cell functions after certain interventions. It's a very high-level concept. It was not a novel idea. We had virtual cell projects almost 20 years ago, but back then sometimes we called it Virtual Cell 1.0.
At that time, people tried to derive differential equations, trying to use mathematics to describe the response to certain pathway interventions. By fitting these equations to different observations, broadly speaking, that was a failed attempt, in the sense that biology is just way too complicated to write in a few predefined sets of differential equations.
Moving forward, with the rise of language models, I think the idea of using AI models to mimic how a cell responds to different interventions by a data-driven approach started to become popular. Almost 3 years ago, just 4 months after ChatGPT was released, our lab at the University of Toronto published one of the early foundation models of single-cell genomics, called scGPT. You can interpret it as a GPT-like model for single cells. It quickly became very popular, in the sense that, for the first time, we had a foundation model that was able to tackle different downstream tasks using the same model. We can use the same model to integrate different batches of single-cell RNA-seq, and we can use the same model to predict multi-omic integration.
swyx
Let's define those things. So, batches—integrating different batches of RNA-seq. You have different equipment; you're collecting data in different labs.
Yeah, different labs, different times of day, correct?
swyx
Different phases of the moon, whatever. Those actually have a big impact on the data that you collect. So there's a big problem of how to even compare this dataset to that dataset when there are all these other differences that have nothing to do with gene expression, and just with how I measured it.
We call that a batch effect. We certainly want to remove the batch effect while preserving the cell types, which are the more important biology we want to preserve.
swyx
This is sort of analogous to the tank problem in image classifiers, for example. The models pick up on these crazy spurious features, which have nothing to do with what you actually care about—the underlying biology. Is that the sort of thing?
Certainly, the core idea of integrating different batches is to keep the biological signals while removing the batch effect. Before these foundation models, what happened in the single-cell domain was that, for every task, biologists had to choose the so-called specialized, state-of-the-art approaches.
With foundation models such as scGPT or Geneformer, what we hope to bring is one model that solves all the tasks in single-cell analysis. With the popularity of foundation models, lots of researchers came together under the Chan Zuckerberg Initiative, and we published a perspective paper in Cell to coin, for the first time, the term “virtual cell”—almost Virtual Cell 2.0.
The idea is: let's use data-driven approaches. If we cannot describe it, let's learn it. That's the idea of virtual cells. That begs the question: can we build a language model, or a language-type model, to predict what the cell types look like and how the cell responds to different interventions? Eventually, we could replace all cellular experiments by simply running simulations on a computer without even running the actual experiments.
swyx
Maybe for a bit more context, you can think about this: a virtual cell is just a general concept. Cells have 20,000 genes in them, and in most human cells, I think roughly 4,000 to 5,000 are usually active at any given time, or expressed at reasonable levels. So, in a normal cell, you might have 4,000 to 5,000 genes doing things.
The question is: in many cases, the way medicine works is you target a protein, or you target some sort of thing that makes proteins more or less common, or you stop the protein from doing something. Given some number of genes in a cell—every cell has a different composition of genes—what is going to change? Will some pathway die off? Will some pathway grow? How does this—based on that—you could predict how a medicine is going to work, by understanding how changing one specific gene or some cluster of genes could change everything. Is that a correct understanding?
Yeah, that's a correct high-level understanding of virtual cell. What's happening in this field is that we're lacking a concrete definition of virtual cells, and people almost equate foundation models with virtual cells. But in my view, a virtual cell is probably a much broader concept than just foundation models.
Foundation models mostly provide reliable, semantically meaningful representations of cells. But I think virtual cells are more dynamic, in the sense that they can predict the development of different cell states across different times, or even describe spatial changes at different cellular resolutions. My understanding is that we're really at the early stage of developing such comprehensive virtual cell models, and the foundation model is really just the starting point.
swyx
AI models always begin with data. You are building—or have built and are continuing to develop—a high-throughput experimentation system. That sounds really cool and really complicated. Can you tell us what that entails? What are the experiments that you're running? How does that inform the building of an AI model? And why do this rather than pick up the CELLxGENE database, which is a collection of gene-expression data that has been aggregated over public datasets?
Cichu Chen
Yeah, great question. I want to pick up where Bo left off. I think Bo said something pretty profound: going from a representation model, the foundation model of biology, to a virtual cell. The key difference there is perturbation prediction, or dynamic processes in biology. That's a causal concept. For that, I think we need causal data.
If you look at CELLxGENE, that's a fantastic dataset that was curated at the beginning with more than 33 million cells, and now a lot more than that. At the time when scGPT was trained on that dataset coming out of Bo's lab in Toronto, that was mostly observational profiling data. It's descriptive data, not causal, and mostly profiling healthy human donors.
The model that was trained on this dataset is very, very good at doing descriptive tasks, such as harmonizing across batch effects and removing effects from different labs and technologies. But I think both of us and many others in the field have found that these models that are trained on descriptive data do not yet outperform linear models on causal tasks, perturbational tasks—what we call counterfactual tasks.
If I did this to the cell, then what would happen? That makes intuitive sense to a biologist because the correlation data in the descriptive data set can be fit with many possible causal structures. In a very simplistic case, let's say you observe genes A, B, and C all go up and down together in your descriptive data set. You can infer that A regulates B and C. That's why, when A goes up, B and C also go up. You might also say that B regulates A and C, and that would be perfectly reasonable as well.
You could also say that A regulates B, and C is completely regulated by something different. You see the problem there. There are any number of ways to fit a causal regulatory network to this group of data. Fundamentally, we believe observational data are underpowered for truly learning causality. This is why we realized pretty early on that we need to start building causal data sets to train a causal model.
So what are the ways to do that? I think the field has come of age to do this at scale, a technique that we call hyperbiology, and there are many ways to generate these causal data at scale. The technique that we have focused on is something called Perturb-seq. For listeners who aren't familiar with that technology, it combines high-throughput pooled CRISPR perturbation with single-cell RNA-seq technology to build 2D data sets. Let me break that down.
swyx
Yeah, go ahead. [laughter]
Cichu Chen
So we just talked about how, in a cell, there are at least 20,000 analytes to measure. These are the genes. These are both the features to measure and also the levers to perturb the cells with. For clarity, let's call them perturbations and gene expressions: perturbations on one axis and the features that you measure, which describe the cell, on the other axis.
Perturb-seq is a technique that leverages the latest breakthrough in lab biology, CRISPR-Cas9. These are bacterially derived enzymes that allow you to disrupt gene expression in mammalian cells—in human cells, for example—and we can do so in a one-at-a-time fashion. So I can take out one gene at a time. Of course, that would be incredibly difficult to scale if I want to do all 20,000 gene-expression knockouts in one single experiment. I'd probably need a huge factory and a lot of robots to do that.
Or you can do them in a pooled fashion. I love pooled experiments. These are hyperscalable. We have lab tricks that allow us to disrupt one gene per cell, but do all 20,000 genes across many, many cells in one single pooled experiment, perfectly scrambled. There are no batch effects. There's no plate-to-plate variation. Basically, you use some sort of combinatorial trick to first perturb all the different genes in different combinations, and then you can read them out and do some math on it. You basically pull out a whole bunch of different experiments in one experiment.
swyx
Correct.
Cichu Chen
It requires barcoding technology, and that barcode is actually achieved by directly reading out what kind of CRISPR guide RNA is present in which cell. For CRISPR-Cas9, for this bacterially derived machinery to work in millions of cells, you just have to deliver 2 things to each cell. You have to deliver the protein—the Cas9 protein that does the job—and you have to deliver an address barcode encoded by a short piece of RNA called a guide RNA. The guide RNA tells the protein where to go in the cell purely via Watson-Crick base pairing: A, T, C, and G.
So it matches a part of the gene. It's sufficiently long to say, “This will match the correct gene,” and then that guides it to connect to the right gene and reduce the expression of that particular gene in the cell.
swyx
Correct.
Cichu Chen
We designed this guide to go to the promoter part of a gene. That's the beginning stretch of every gene, before transcription starts. If we bring the Cas9 protein into there, armed with the right effector—the silencer—that promoter will get shut off, and that gene will never be transcribed again. So effectively, we tune down the expression level of that gene. All you have to know is which guide RNA is in which cell, and that can be done using genomic readouts. That's the barcode, and you can then infer which gene is being silenced in which cell.
That's the way you scale throughput on the perturbation side. On the readout side, it's a 2D data set, right? We just talked about one of the dimensions. On the readout side, we leverage single-cell RNA-seq technologies. These are also recent technologies from the last decade that have been scaled, and they can let you read out the expression levels of all 20,000 genes simultaneously from each cell.
So, armed with both high-throughput CRISPR perturbation and high-throughput single-cell RNA-seq technologies, all of a sudden we can generate these 2D data sets where we systematically perturb, or knock out or knock down, every single gene in the human genome in a cell type, and we read out the impact on every other gene in the same cells. So we generate these 2D rich data sets, not that different from the size and type of PDB data that trained AlphaFold models. That's hundreds of thousands of protein entries. If those are the rows, the columns are the XYZ coordinates of every single amino acid. That's also a 2D data set, and I think it's these types of rich 2D data sets that power the training of foundation models of biology.
swyx
I find it really fun how you've taken a fairly straightforward assay using—I guess this is NGS sequencing, right? Next-generation sequencing, very high throughput—and used it to scale a simple perturbation response, which individually maybe isn't all that interesting, to this massive scale of basically an arbitrary number of cells. I think you did 25 million or something.
Hao Cui
It's actually a lot more than that. So 25 million is what came out of the most stringent quality filtering.
It's actually as much of a scientific challenge to figure out how to do CRISPR and single-cell RNA-seq as it is an engineering challenge in the first part of the experiment. Oftentimes, we have to harvest tens, if not hundreds, of millions of cells, and they go through various quality funnels to give the team the highest-quality data at the end. That's incredibly difficult to do because, as you can imagine, all of these techniques have been published by academia before, and they work very well in small-scale experiments. But when you think about scaling them to a genome-wide perturbation, we're talking about handling hundreds of millions of cells.
Techniques that are published in academia used to be all about handling fresh cells. The cells are still alive, and that may be okay if your entire experiment takes only 1 or 2 hours. It's not quite easy to handle cells across a 14-hour day. That's hundreds of millions of cells. By the end of the day, I used to joke with my team, you can easily detect stress signals from the cells and from your scientists in the lab. [laughter]
swyx
Yeah.
Cichu Chen
We quickly realized that's not the way to generate these data. Machine learning is very quality-dependent, and we want to give the highest-quality data to our AI teams. So we're putting a lot of engineering thought into industrializing the whole workflow step by step. We introduced chemical fixation so that we lock the state of the cells in at the beginning of the experiment, while figuring out ways that it doesn't disrupt all of the molecular-biology steps afterward. It doesn't impact data quality, so we can do all of this data generation in a time-shifted operational manner that's not prone to batch effects.
swyx
One thing that you didn't mention is that you're using some sort of stem cells. Obviously, you don't have brain cells or blood cells, or if you did, you would have a big combinatorial effect on that. So how are you convinced that working on stem cells—which, as I understand it, are actually blood cells whose stem-cell behavior has been unlocked, and that causes some sort of stress on the cell as well—how are we convinced that that is a good proxy for a brain cell or whatever you're studying?
Cichu Chen
Yeah, not quite. We didn't actually start with stem cells. That was more of a later development when we started data generation. We put out the method that I talked about, as well as the first 2 data sets—the world's largest Perturb-seq data release at the time—in a preprint last June. We called the data set Tahoe-100M. That was actually generated from 2 cell lines.
Shawn Wang
Cancer cell line.
Hao Cui
One of them is a cancer cell line; the other is just a cell line. These are immortalized cells. Some of them are derived from cancers, hence cancer cell lines. Others are just derived from primary cells but have been immortalized, grown for many, many years in various labs.
People start with these cell lines in the beginning because, as you can imagine, those are easy to work with. They're easy to scale and easy to grow a lot of cells from. It turns out the ability to grow millions of cells is actually critical for doing these large experiments. We started there first, and they still capture the characteristics of the cell types that are derived from colorectal cancer, as well as hematopoietic cells.
Later on, in the most recent preprint, we actually expanded to many more cell types. Some of these are still cell lines. Our T cells—we chose to use cell lines—but some of these have now gone into primary cells. We did one experiment in iPSCs—induced pluripotent stem cells—and another experiment in…
This is a pan-differentiation, multicell-type stem cell project. So, effectively, we differentiate iPSCs into 10 different cell types in one single experiment without restriction, and we did a genome-scale perturbation across them. You can imagine: instead of just generating 10,000 different biological experiments, we did 10,000 by 10 cell types. It was almost a library-on-library experiment.
Why are we doing this? We think that, in the beginning phase of data collection, as I said, we're just in the early days of virtual cell building. Context, diversity, and richness of the data matter. It's not just the total number of cells or total number of sequencing reads; it's about bits per dollar and information content. So we want to scale not only in the genetic perturbation landscape, but also across biological context, so that we can give our AI teams the best, richest dataset to build a generalizable model on.
swyx
Is there any thinking about—so you say context, but obviously these cells in these experiments have been, I forget the term, separated from their cohorts, right? Is there thinking about using spatial transcriptomics or other technologies, imaging-based technologies, to build models with perturbations, but in the context of the cells that they live near?
Cichu Chen
Great question. So we're thinking about that in a couple of ways. Number one, that's actually exactly why we want to build a virtual cell model in the first place. You might think, well, you can already do exhaustive screening in these cell lines, so why do you still need a model? You can just do the experiment and generate the data. Certainly, if your query is just about cell biology in cell lines, you're right: we don't need a model, at least for genetic screening, because we can just do this experiment.
But you're also correct that oftentimes good targets and biological insights are not about cell lines. They're about primary cells, cells in their native physiological context in organs, or even multiple organs coming together and having some emerging properties. A lot of immunological diseases are that way. You cannot do exhaustive, high-throughput experimentation in animal systems, organs, or all of these complex translational models. You can do some experiments, but these are expensive and high-stakes.
The ability to build a model that can be trained on massive data, where it is possible to scale, and be trained in a way that can be fine-tuned and transferred to make high-quality causal predictions in these complex models—so that we can go into the lab and have the highest-quality hypothesis possible to validate—I think that's the whole point about building a virtual cell model.
But from the AI side, I think you're absolutely right that the future virtual cell model should be able to incorporate multiple modalities, not just RNA expression. Spatial single-cell RNA-seq is already a popular technology, even for scGPT. We actually have an extended version—we call it scGPT-spatial—that is specifically designed for spatial single-cell data. We also have papers on early attempts to take H&E images and predict gene expression. There are already some signals you can find.
Ultimately, what I predict is that a virtual cell model will be able to integrate not only RNA-seq, but also more functionally related modalities—for example, proteomics or other regulatory sides of omics—to overall combine all your descriptive omics datasets and predict the future states of cellular functions. I think that's probably the future for virtual cell models.
swyx
We had Ron Alfa and Dan Beer from Noetik as guests recently, and listeners who want to hear a little bit more about that—we go into quite a bit of depth there. So if you want more background, you can go to that. But can you explain a little bit about what spatial transcriptomics and spatial proteomics are?
Cichu Chen
Maybe a bit of a history lesson here. Before we had single-cell RNA-seq, we had RNA-seq, and before that, we had microarray technologies. What RNA-seq and microarrays used to do was take a chunk of tissue, grind it all up, put it in a blender—imagine making a smoothie out of it—and take all of the RNA from different cells in that piece of tissue and measure all of their expression levels.
It was great because, for the first time, you could measure the expression of all 20,000 genes at a time. We used to have to do them one at a time, but it's not great in that we don't know which RNA came from which cell. This is particularly a problem if you're dealing with a multicellular piece of tissue. You want to attribute RNA to the immune cell, to the skin cell, to the fibroblast, or to the keratinocytes, but you can't because you grind everything up in a smoothie.
What single-cell technology allows you to do is analyze them cell by cell. So now I can attribute RNA gene expression to the cell from which it originated. But there's still a problem: I don't know spatially where the signal comes from. For many diseases, that matters. In immuno-oncology, for example, you want to know when T cells are close to tumor cells, or when a T cell is not able to penetrate a solid tumor. What is the difference between them? Or when a T cell is attacking a tumor cell and when the T cell is not, what is the difference between them?
For that, you need spatial information. You need to observe cells in situ, in their context. Now there are different technologies that solve that problem. Essentially, you take that chunk of tissue. I don't have to grind it up anymore; I just make a cross-section, lay it down on a piece of slide, and I can measure its morphology using standard techniques like H&E staining.
I can then also measure many protein expressions using multiplexed assays and fluorescence assays. Ultimately, I can also look at gene expression, up to genome-wide, in all of these cells in their native spatial coordinates by using some of the latest spatial omics assays. So you have the x-y coordinates of every cell, but also all of the molecular analytes that we talked about earlier. That's an exciting new direction for the genomics field.
Personally, you can imagine, spatial omics adds more difficulty to AI modeling because instead of looking at individual cells, you have to look at the neighboring niche cells to better learn a representation that is spatially cohesive. That is the challenge current spatial foundation models are facing. But that context is going to be crucial for correctly understanding, let's say, cancer, where the interaction of immune cells with cancer cells and nonimmune or noncancer cells is crucial for spatially aware biomarkers and for predicting clinical response. I think that would be extremely important to build such models.
swyx
Getting back to Excel, this presumably can inform a spatial model as well, right? You can have one cell in one place. You can imagine, okay, I can just throw away the coordinates and do inferences on one cell at a time, and now I can create a more complicated model that does that, but it also knows who its neighbors are—is that right?
Cichu Chen
You're absolutely right, but the current version we're releasing is not dealing with spatial omics. However, our ongoing work and the next version of Excel will definitely be able to infer spatially aware representations for different cells.
Shawn Wang
I see. We've talked about the data collection a bit. Let's talk about the architecture. Let's get some red meat for the AI engineers listening in.
Bowen Wang
Sure. Let's get to the history of virtual cell modeling, particularly Virtual Cell 2.0. I think our scGPT kind of sets the foundation for most foundation models of single cells. We adopted autoregressive training, extremely similar to how GPT is trained on language. We use next-word, next-token predictions, so we mimic the way GPT is trained on language to train a single-cell foundation model on all cells.
By doing that, they have to assume an inherent order of genes, right? The way we assume the order of genes is through the attention mechanism. There are many other methods that use different orders of genes. Some are as simple as just ranking the genes based on their expression values. There's also more complicated methods to rank different genes. But inherently, you have to assume an order of genes.
Shawn Wang
And just to be clear, when you talk about genes, those are intrinsically ordered, right? They're a sentence spelled out in ATGC, right? So genes themselves have the nucleotides, and there's this long chain, and that makes a lot of sense to have an order to them. But what we're talking about is something different: that's the expression data, the expression levels. So the expression level means it's just a count for each gene of how many of these genes did I see when I was measuring?
Hao Cui
For DNA sequences, the order of ATGC makes total sense to us, right? But for expression data, they're literally just matrices, so it's really hard to assume an inherent order of genes. Even if you shuffle the order of genes, I think the biology doesn't change much. However, because of the way language models are trained, everybody has preset tricks to train such a model, so it's easy to adopt. That's how all the foundation models are started for single cells.
Then I quickly realized that with diffusion language models, we actually don't need to assume the order of genes. Instead, we can have a bidirectional diffusion process to generate such long, high-dimensional gene expression datasets. Just to think about it, what's the difference between autoregressive training versus diffusion language models? You can think of autoregressive training as typing. For example, “I like coffee”: you have to type “I,” then “like,” and “coffee”; there's an inherent order.
But with a diffusion language model, you can treat it as editing. You iteratively generate a sentence from a very vague, rough sentence, and then you can iteratively refine it. Same thing with gene expression: you can generate a very rough representation of the gene expressions and then iteratively go from a noisy representation to more refined representations.
So you iteratively edit the gene-expression predictions until you minimize the loss. This is a very different philosophy from generatively predicting the response after perturbation, and it turns out it fits single-cell RNA-seq better. That's why we switched from a scGPT-like model to the current ExCell model, which uses diffusion language models.
Shawn Wang
When I think of transformers, they're fundamentally objects that operate on sets. The community spends a lot of time trying to make them things that have some sort of causal ordering to them, but if you just naively take a transformer, it's a set operation, right? Given that, why think about this in terms of diffusion or autoregressive LLMs?
Why not have your initial prediction strategy be something like: take just a set of genes, each of which has its own one-hot-encoded identity, and then use that as a prediction? That seems like a much more natural architecture to me. A lot of people work on things like this, and I've been somewhat confused about why there's this bias in the community against it.
Hao Cui
I think what you were referring to is more related to representation learning, where you can take sets of genes and try to project them to a low-dimensional latent space. But what we care about for building generative models for virtual cells is predicting the dynamics of cells, so we want to have a generative model.
That's why we mostly use decoder-only architectures in order to generate the full transcriptomics instead of just predicting a predefined, small set of genes, because we want to model the whole gene-regulatory network, which is extremely high-dimensional.
Shawn Wang
So, just to be clear, the input is genes plus a perturbation, and the output is new gene-expression levels? Is that right? Gene-expression levels plus perturbation is the input, and the output is—
Bowen Wang
Cells—for each cell.
Shawn Wang
Correct. Correct. That is correct. Yeah. Right, okay. And the way that I think about diffusion language models—you can correct me here, because I don't know a lot about them—is that they're like BERT, but you do it over and over again. Is that a good understanding?
Bowen Wang
Yeah, that is a rough understanding of how diffusion language models work.
Shawn Wang
Yeah. So you just apply the diffusion process. The diffusion process is basically unmasking or editing, correct, over and over again, similar to how an image-diffusion model refines the image over and over again. In this case, I'm using BERT, so it is a transformer. It is a transformer—
swyx
But it's repeatedly updating the sentence, in this case a bunch of expression levels, over and over again.
Shawn Wang
That is correct.
Bowen Wang
Actually, in our paper, we show that as the number of diffusion steps goes on, the loss function keeps decreasing and the fitness of the prediction to the ground truth keeps increasing. This means the model starts to understand how to iteratively refine the predictions.
Shawn Wang
I see. We're talking about diffusion versus autoregressive. I noticed there's a bunch of discussion in the paper about preconditioning and using a whole bunch of other things. Can you talk a little bit about that?
Bowen Wang
Another major innovation we made in ExCell is the way we incorporate prior knowledge into the model. Incorporating biological priors has always been a good idea in biology in general, because biologists spend decades trying to understand some of these biological systems. How do we tell the model some of the prior knowledge and metadata about the cells?
Before ExCell, what people did was try to incorporate a single type of prior. For example, using a gene-regulatory network as a prior to predict perturbations. scGPT sometimes tried to incorporate PPI as a prior as well. To my knowledge, ExCell is one of the first models that tries to incorporate an extremely diverse set of biological priors.
In our preprint, we incorporate 5 types of priors, including literature. Something as simple as asking ChatGPT, “Tell me everything about this gene,” and then using the output as the embedding for GenePT. Exactly—that's GenePT. We also incorporate PPI, or protein–protein interaction networks. We incorporate DepMap, which contains cancer-related essential-gene information, as well as morphology information. We even try to incorporate scGPT embeddings, which are basically cell types.
With this set of prior knowledge as conditions for the model, the model starts to have more accurate context-specific predictions. What's more interesting to us is that by looking at the weights of the different priors, we can actually understand which prior knowledge is more important for particular cell types. So it adds more interpretability to the models.
We find that combining a diffusion language model with a very diverse set of priors, ExCell does much better in generalizing to unseen contexts. Those are some of the AI innovations we made for ExCell.
Shawn Wang
Do you now need to provide all of that context in order for the model to work, or are those preconditionings that it can also do without, if you want?
Jesse Engreitz
We don't need to incorporate these priors anymore, because they're already learnable parameters inside the models. However, what you suggest is more like promptability, or in-context learning, for virtual cells. We can do that as well, basically by adding more conditions to the prior knowledge in order to prompt the model to predict toward certain directions.
Shawn Wang
In other words, the model now takes advantage of the learning from the priors that you provided during training and doesn't need them, but it has some advantage because you provide them during training. You can even get more advantage if you're able to provide those priors during inference.
Jesse Engreitz
Yeah.
Shawn Wang
Okay, wow. Nice. How much does that matter? Whenever I see big machine-learning papers with tons of things thrown in, I'm always wondering: where's the big alpha, and where's the little alpha? Are these just adding a little bit of incremental performance, or are all of these actually crucial to generalization?
Jesse Engreitz
There are multiple factors we have to consider: how much the data contributed, and how much the AI architecture contributed. Even for the architecture, what's the delta from switching from autoregressive training to diffusion? What's the delta from the prior knowledge? All of these require very specific ablation studies.
From empirical experience, we find that the quality and amount of the datasets matter the most. This is why we were extremely excited to publish the PISCES dataset, which has 16 different cell types across 25 million cells, and it's genome-wide. You have a huge tensor if you really think about it from a computer perspective: genome-wide perturbations, genome-wide transcriptomics, the number of cells, and the number of conditions. It's a massive tensor.
Because of the Perturb-seq technology, we don't have batch effects, so the model doesn't need to learn batch effects. That's already an advantage. We find that training on high-quality perturbation datasets already gives a big boost to the models.
We also did an ablation study where we trained all the virtual-cell models out there—including state-of-the-art virtual-cell models such as Cell2Sentence and the original scGPT—on the same datasets to see what delta we observed. We report those results as well. We find that switching from autoregressive training to diffusion language models gives significant improvements over some of the harder tasks, particularly generalization to unseen tasks.
The prior knowledge is more or less condition-specific. For certain cell types, some of the priors make a huge difference, but for certain cell types, the delta seems to be marginal. We're thinking about how to better incorporate the prior knowledge. We still believe that letting the model know a big chunk of existing biology should be helpful, but maybe the way we incorporate the prior knowledge through cross-attention limits the scope of the metadata. It's certainly a research topic.
Overall, if we have to give an order, my order would be the quality and scale of the datasets, then the architecture, and then the prior knowledge. But this certainly only applies to our ExCell model. I'm sure different choices of architecture would have different ranks of contribution.
Shawn Wang
First of all, this is really fascinating—a very cool model. I hope everyone has a chance to look at the paper. A lot of resources were obviously put into doing this. I don't know if you guys can disclose how much, but it's a lot of money. Whatever it was, operating a wet lab is probably very complicated, and the training runs were probably very complicated too.
I think there's a 4-billion-parameter model. Is that right, or—
Jesse Engreitz
4.9 billion.
Shawn Wang
Yeah, a 5-billion-parameter model. A much larger model. It probably took a lot of GPUs to train. What's the lift you get from this effort versus just putting the money into wet-lab work and the traditional pipeline that has basically been the status quo up until now?
Biology is a multiscale discipline. There are DNA sequences at the most fundamental level. There are cells, multicellular pieces of tissue, cocultures, tissues, and animal systems, and finally, humans. I think we'd like to be able to do causal prediction toward the right of the spectrum.
Jesse Engreitz
Ultimately, we'd like to do causal prediction in humans: know what drugs will work in which patients. But that's very difficult because it's hard to collect data on, and so the whole vision of the virtual cell is to generate data in settings where it is possible, so that we can transfer causal prediction towards the right, towards more translational, more complex systems.
You can certainly mine the data we've already generated. We've generated a lot of data: 7 screens, 16 different biological contexts, and genome-scale perturbation. There's a lot of good biology in that already. There's a figure that we put out in the preprint looking at the inactivation of T cells. We already saw some known biology, such as the TCR complex, and we also saw some putative new biology, which we're very excited to validate in the lab. Some of that was also called out in a very recent screen last December, published by Alex Marson's lab, also in the Bay Area. So we're very excited to see that, but the hope is not just to mine the existing data.
The hope is that the model can generalize, and that we will be able to do in silico experiments in the future. Nobody knows beforehand how much data, and what kind of data, are needed to do that. The whole field is waiting for a demonstration that the model can beat a linear baseline in perturbation prediction and generalize out of context—not just within a cell line you have training data on, but out of that context. That's why you need a model.
What's very exciting for us is that, in this preprint, we saw that generalization capability in a few demonstrations. We first did this in T cells. We actually generated the data expressly for this purpose. We generated a resting T-cell perturbation screen. These are T cells in their baseline condition, not activated. Now we have an activated T-cell perturbation screen.
Shawn Wang
So, T-cell activation means I'm trying to kill something?
Jesse Engreitz
No, these are regulatory T cells. We activate their receptors so that they're starting to proliferate.
Shawn Wang
They become more active; they can do their physical job.
Jesse Engreitz
And, critically, we only trained the model on the resting T cell. We told the model, “This is what the active T cell looks like. Now go and predict what all of the perturbations are going to do in this active T cell.” The model had not seen how perturbations work in active T cells.
We set up a couple of rigorous tests. One linear baseline took the perturbational delta in the resting case and transposed it linearly onto the active T cell. That's our linear baseline. Essentially, think about this as a combinatorial perturbation-prediction problem: one of the perturbations is activation of the cell, and the other is all of the genomic perturbations. Can I just linearly add the 2 effects together? That would be a linear baseline.
Second, we applied other models from the field, and last, we applied ExCell. Critically, ExCell had not seen active T cells, and it was able to make accurate predictions—not only on the known biology, the TCR complex, predicting their effects accurately and that these are going to inactivate T cells, which is exactly what we would expect to see, but also on the putative T-cell inactivators that we found in the screen.
So that's very exciting to us. It suggests the possibility that we might be able to use these virtual-cell models completely out of context, in an unseen context, and predict new biology. We're very excited to follow up on those hits and validate them in the lab.
Just very briefly, there are a couple of other cases where we saw exciting generalization capability from this model. Remember, we did a multicell-type differentiated iPSC experiment. There, we specifically held out 1 cell type from training. The model had not seen that cell type; it was trained on the other cell types as well as the rest of the datasets. The model made very good predictions across thousands of genes and thousands of perturbations in that unseen cell type, again suggesting the model's ability to generalize out of cell type.
The last experiment that I think we're very excited about is that we trained this on a T-cell line, but there was a very recent primary T-cell Perturb-seq study published by Alex Marson's lab. That's an impressive amount of work. It's not easy to do screening at this scale in primary cells. Very few labs have that kind of capability. It's much easier to do that in T-cell lines. Again, the model was able to generalize from cell lines to primary cells and make accurate predictions there.
Shawn Wang
So, they actually perturbed primary cells, not cell lines?
Cichu Chen
Primary T cells harvested from donors, and we were able to do this for multiple donors. ExCell, trained on just 1 T-cell line, was able to make predictions across multiple donors from primary T-cell experiments.
Shawn Wang
This is a validation of the whole theory, right? You can train on these slightly weird cells, and it will be good because you're covering the domain well enough—or whatever it is—that you're able to actually predict in real cells that come directly from real people.
Bowen Wang
That's right. I think building a virtual cell is not about replacing biological experiments, as you mentioned. What we're trying to do, really—the holy grail of the virtual cell—is to have a model that can generalize to unseen contexts where it is harder, or even impossible, to conduct biological experiments.
So far, ExCell is focusing on cell lines, and eventually we want to extend to more complicated biological systems, such as animals and organoids, and eventually, as we mentioned before, to patients—to human biology. Bear in mind a few numbers: 90% of diseases have no cure, most drugs fail in phase 3 clinical trials in patients, and the success rate of phase 3 trials is as low as 5% to 10%.
Shawn Wang
And phase 3 means—
Bowen Wang
The final stage of patient trials.
Shawn Wang
So that's when you generalize from toxicity in phase 2 to efficacy in phase 3, right? From a small cohort into a much larger cohort.
Jesse Engreitz
Oh, sorry. Toxicity is phase 1, right? Small cohort.
Shawn Wang
And then a larger cohort. So the generalization problem is: I've very carefully selected my patients, and it works pretty well. Now I get a bunch more patients, and suddenly it doesn't work very well. That's the big problem.
Jesse Engreitz
The central promise of the virtual cell is: can we build such a model that learns all the causal biology, so that it can be used to predict the response eventually in patients? Then, for certain drugs, we can select the right patients to conduct the clinical trials on.
This is a long-term vision, but we're already seeing some early hope that ExCell, trained on a diverse set of causal datasets, can generalize to some unseen cell types. Certainly, there are a lot of experiments to be done to validate this model and to continuously fine-tune it, but we are seeing some early hope.
Shawn Wang
You were talking about linear models, and this brings up the famous—or infamous—ARC challenge about perturbation. There's been this theme about complicated foundation models often not beating linear baselines. I'd like to get your take on that. First of all, is this different? I think some of your own models might also have had trouble beating linear baselines in the past. Is there something different about your current data strategy or where you're going? Where's the field going, and what is the role of foundation models versus virtual-cell models versus these simple baselines?
Bowen Wang
A few things. First of all, those benchmarks, as you mentioned, are conducted on Replogle datasets, which are very small datasets, and the metrics people report are mostly MAE. You can imagine that, because single-cell datasets are so sparse, the average profile of the cells can be a great minimum, a kind of local optimum, for minimizing the MAE. This is why sometimes the average profile of cells has lower MAE even than technical replicates, which are considered ground truth for perturbation experiments. That itself shows that the metric is not reliable.
However, most of these benchmarks are still comparing foundation models that train on static expression datasets, such as cell-by-gene data. scGPT is often benchmarked against linear models. Internally, we also find that, when it comes to MAE, scGPT sometimes fails to outperform linear models, just because of the reasons I just stated.
What sets ExCell apart from these static-expression models, such as scGPT or Geneformer, is that instead of training on gene-expression datasets, we train on causal datasets. We train on a massive amount of genome-wide perturbation datasets so that it learns better about the dynamics of interventions.
In our preprint, we extensively compare with linear models as well. As you mentioned, linear models totally fail to extend to unseen cell types. You can quickly imagine why. I believe that foundation models, or other more complicated AI models that train on the right data, will outperform these linear models in harder tasks, particularly in generalization tasks. That's why I keep mentioning that the right dataset with the right AI model will lead to huge improvements.
But I think the field still needs to see more biological validations to be more convinced that the virtual-cell direction is the right one.
Cichu Chen
I think the field suffers from a lack of consistent and uniformly accepted benchmarks. What gets measured will get improved, and in our paper, we measured—one of the metrics that we put a lot of thought into, and where we saw the model really shine, was metrics around gene-expression changes.
Pearson delta—the similarity between predicted changes and ground-truth changes upon the perturbation—is very hard to cheat on. You have to really get the changes right. What really blew my mind was when I saw the model make a prediction: print out the heat map of the gene expression changes, look at the actual raw data, and line up the linear baseline prediction, the ground truth, and the ExCell prediction all together. It's visually very clear to see that the ExCell prediction is much more similar to the ground truth than the linear baseline. This is the wow moment I was talking about in the beginning.
Bowen Wang
That's right. This is the first time that someone can put together not just one Perturb-seq, but 7 genome-wide Perturb-seq campaigns. Something that jumped out to us biologists right away is that some of the perturbations are context-universal, meaning that the genes do the same thing regardless of the cell types you experiment in. It might not be surprising to you that these are your housekeeping genes, right? Of course, they do the same thing in every cell.
And then there are all of these other clusters of genes that have very context-specific functions. They do different things in different cells. Again, it's not hard to imagine why. In iPSCs, in our stem cells, we saw developmentally relevant genes—genes that are important for neuronal differentiation. They only light up in iPSC experiments, of course. Right? That makes sense.
So you think about biology: it's so complicated. You have to capture these context-universal perturbation effects. You also have to somehow learn the context-dependent perturbational effects. It's not hard to then see why a very sophisticated nonlinear model is able to capture and learn all of that biology much better.
swyx
Are your perturbations always single-gene perturbations, or do you have more? Because my understanding of regulatory networks is that sometimes a single gene does a ton of things. For example, I think males are differentiated due to one gene being enabled at around day 7 of embryo development or something, and that differentiates everything—it's this one gene. But then sometimes you have large networks of genes that are all very redundant, which allows for more subtle feedback mechanisms and so on. So I could imagine a lot of single-gene perturbations as being irrelevant, and that you might want to start having a more combinatorial strategy here.
Cichu Chen
Yeah, that's a great question, and Bo and I have thought about this a lot. Actually, it's interesting that you brought up reproductive biology. I study a lot of dosage compensation mechanisms in female cells. Female cells have 2 X chromosomes; male cells have 1 X chromosome. To match the dosage output from the X chromosomes, the strategy that mammalian cells employ is 1 gene that produces an RNA that does not encode for any protein, just a non-coding RNA. That RNA wraps around 1 of the female X chromosomes and turns down most of the gene expression from that chromosome, shoves it away in a corner of the nucleus, and it's called a Barr body. It is never heard from again.
Bo Wang
I absolutely agree with you: 1 gene can do a lot, but in biology you also have redundancy, compensation, and all kinds of mechanisms where knocking down 1 gene is not sufficient to always see a phenotype. What if 4 genes redundantly do the same thing? Taking out 1 is not going to be sufficient.
Where we started with 1 cell type at a time, loss-of-function, single-gene perturbation, and only RNA expression as the output, we're expanding the platform along all of those axes. That's what we do today to build a scaffold of the data for training models like ExCell. We are now beginning to grow in all 3 axes of the platform: going beyond transcriptomics alone to look at multimodal data; going beyond just 1 gene perturbation to also look at pathway activations and inactivations, turning on and off entire cascades of gene chain reactions; and going beyond just cell lines and monocultures into more and more complex, translationally relevant systems, into primary cells, into organoids, and doing even direct in vivo perturbation screens.
So we believe that with all of that expansion, the data will be all the more exciting to train models on. This is also why we incorporate PPI networks as prior knowledge into our model. Although the model right now is trained on single-gene perturbations, once the model is trained, you can actually predict combinatorial perturbations just on the model, in silico. You can perturb the tokens of 2 genes at the same time and see what the response is. Certainly, without training on the actual combinatorial perturbation dataset, the accuracy may not be there, but at least with the existence of such models, we can start to generate hypotheses using in silico perturbations.
Shawn Wang
How do you see the role of the scientist changing in the age of AI? Because your focus is not language models themselves, or agentic science and things like that, you may have a slightly different take. You're building these very specific models, but still, how are you and your students able to maintain such a high pace? I suspect it may have something to do with generative AI, partly, but also, how do you see the role of the scientist, the academic, changing?
Bo Wang
Yeah, that's a great question. My official split of time is 80% on Xaira, 20% on my university affiliations, but it turns out the reality is 100% on Xaira, 100% on this.
Shawn Wang
You invented a time machine. That's the answer.
Bo Wang
The way I try to keep up is—I mean, my lab uses lots of agentic AI to monitor all the AI papers every day. Every week we have lab meetings where we try to discuss different topics in AI for biology, AI for healthcare, et cetera. To be very frank, even as a professor, I find it extremely hard to catch up. The pace of AI is just so incredibly fast, to the point that sometimes I feel anxiety waking up, saying, “Oh my God, this paper already has so many people publishing, and what happened to our existing unpublished work?” You can imagine students probably face 10 times the anxiety.
Sometimes I try to encourage students to use different tools, trying to stay focused and find a niche area where we can become experts. With the era of generative AI, I find the way people—at least academics—do science is extremely different now. Overall, most professors or students in academia are struggling in terms of funding and the pace of publications. That's why you can see lots of major breakthroughs come from industry, like AlphaFold, for example.
How academics survive or even thrive in such an era of agentic AI is certainly something everybody is thinking about. We see lots of faculty leave university and join industry simply for the resources. If you're doing research on AI, do you have enough GPUs at your school? That's the first question you should ask. When a student joins a professor's lab, the first question they often ask is, “How many GPUs do you have?”
Certainly, in that sense, industry has a major advantage over academic labs. But I think what academic labs have an advantage in is really the pace of innovation, and also the niche areas in which a specific academic lab can become extremely expert. Also, having the freedom of thinking sometimes makes it easier to innovate on ideas that maybe industry people didn't even think about.
Overall, I think the whole field needs to be a lot more innovative to catch up. Hopefully, with the help of different tools, and hopefully the government starts to invest more in academia, because I still deeply believe that academia is the main source of innovation for the whole field, particularly when it comes to biotech. Hopefully we see more investment in academia so that we stay afloat.
Shawn Wang
I agree with you, but why do you think the money shouldn't just go to industry? Why should the government put any money into academia?
Bo Wang
I still believe in the power of academic freedom. This is actually the original motivation for the existence of academic professors, who not only teach but also do research. There are also benefits to teaching and research at the same time, in the sense that when you teach a subject, you actually have to become an expert on it, and that forces you to keep updating your knowledge base and then find easy ways to convey your knowledge to students. By doing that, you actually start to innovate on different ideas.
Myself, for example, I teach a big class at the University of Toronto about deep learning and neural networks. It's a gigantic class with 600 students every year. By teaching that, it forces me to update the slides and lectures every year. I'm reading lots of materials, trying to update myself so that I can find ways to convey all the knowledge to students, right? That's how I keep updating myself whenever there are trends.
I remember vividly: we had GPT-2, and we updated the lecture. Then there were different language models, and we covered how multi-GPU communication is used in training larger-scale neural networks. I force myself to update the models, and by talking to different students, we really generate lots of novel ideas to apply cutting-edge AI models to very specific niche areas in biology or healthcare.
Maybe it's unique to the Canadian academic system. By being a professor in academia, we also have access to lots of healthcare datasets, which are very hard for industry to access due to many legal or regulatory reasons. That's why you see some of the papers we publish through academic hospitals in Canada, where we developed some of the state-of-the-art foundation models for ultrasound images.
So that's what I mean: there's a certain level of freedom of academic thinking that really drives lots of innovation. I still believe—maybe I'm biased, but I myself still believe—that having a certain level of freedom of academic thinking will lead to lots of innovation that is unthinkable in industry.
Shawn Wang
I 100% agree with Bo there. Just thinking about the lab workflow that we do, a lot of these are building upon innovations that were first pioneered in academia as well. CRISPR, of course, was discovered in academia. CRISPR applied to mammalian high-throughput screening was also demonstrated in academia first. Single-cell RNA-seq, this kind of droplet-encapsulated single-cell RNA-seq, was first demonstrated in academia, and then different companies tried to build it up into commercial offerings. Putting all of these together to do Perturb-seq was also pioneered in academia, right? Chris Voigt's lab, Jonathan Weissman's lab, Aviv Regev's lab—many of the pioneers were in academia.
Especially on the lab side, this innovation takes so long, and the discovery process can be so accidental that it's perhaps not ideal for pure industry to take on. But once these things show early promise, scaling them and robustifying them, and generating data that's not only massive but high quality—especially for AI scale—is something that can be done very well in industry, both because of the mindset and because of the resources to support it, which can sometimes be hard for academic labs to match.
Xaira has been very generous with releasing your datasets and your models. You seem to be very committed to open science, given some of the things you were just talking about and the discussion about the best, most important data strategies for virtual cells or understanding human biology. Where do you think academia should go next? Since Xaira has probably a budget comparable to dozens or hundreds of biotech labs right now, what do you think that, if you are an academic—a professor, especially in a wet lab—what would you want to be focusing on?
Bo Wang
First of all, I'm a deep believer in open science. That's why all the models and data we talked about here are open-sourced. You can find all the data and weights on my lab's GitHub.
Shawn Wang
It's very thorough.
Bo Wang
Yeah, thank you. The reason I believe in open science is that, as you mentioned, most of the time academic labs start with an idea and prototype it. It's not scalable; it's not even a good product, and industry can take it and scale things up. This is also why scGPT quickly became one of the most widely used single-cell foundation models in pharma companies. So that's very encouraging to us, and this is why Xaira is also starting to open-source some of the datasets and models.
Part of the reason is that we believe virtual cell is such an early field, and it doesn't help to withhold certain datasets or certain models because it's so early. A better win-win situation is that everybody in this field gets to start contributing data together, contributing models together, and exchanging ideas, so that this field can move forward at a much faster pace.
We see successful examples in the protein space, right? Because of the availability of open-source datasets such as PDB, we have models such as AlphaFold and RoseTTAFold. Again, AlphaFold 2 was also open-sourced, so we can quickly iterate different models. That's why you see a booming situation in the protein space. We want to do the same thing for virtual cells: let's put all the datasets together, have the same standard protocols to generate high-quality datasets, and put all the resources together to generate the next generation of virtual-cell models.
When it comes to academic labs, Chun can comment on how wet-lab academics can survive. But from a dry-lab perspective, I do encourage all the dry-lab AI researchers in universities to start collaborating with industry, so that they can get more resources to develop their own ideas. With the era of agentic AI, now everybody can code. So it's more important to have the right taste about your projects, so that you don't just burn tokens without purpose, right? We want more academic students and professors to have better taste in research, so that we make the right use of agentic AI.
As a professor, it's your job to have taste, obviously, but as a student, how do you develop taste in a world where so much of the scientific thinking process could be essentially outsourced to an LLM? There's not a world where you're forced to bang your head against something and learn taste the hard way.
This is why we need academic training, where you get into a field you know nothing about and hopefully, after you graduate, become the expert about this particular topic in the world, right? This is why you have to go through different programs, talk to your peers, talk to your professors, to get an idea about what good research taste is to begin with.
More importantly, I always teach students that the best way to learn something is just to program it. By programming, you kind of know the details hidden in all the mathematical equations in the paper, which you often omit. But in the era of agentic AI, it's slightly different in the sense that we used to spend lots of time coding and a little bit of time debugging, but now we let the agent do most of the coding and spend most of our time debugging, which seems to be definitely interesting to me. We've had lots of discussions with the students in the lab about what's the best way to spot bugs made by AIs.
How do you find places where AI is particularly good, and also how do you find places where AI is still limited? You need lots of trial and error as well. In the end, you still need to validate your model using real-world evidence. That's why collaborating with wet labs and clinical teams to validate your model and provide feedback signals for developing your taste, as we discussed, is certainly a very important training program.
Cichu Chen
On the wet-lab side, academia has extremely important roles to play. I think we'll enter an era of symbiotic innovation and cross-pollination of ideas. Just like in AI, I think we have great ideas coming out of academia all the time, but industry is now increasingly contributing new ideas on architecture and all of that as well. On the wet-lab side, certainly industry seems to be able to scale this type of data generation quite well.
But biology is so much more than just cell-based Perturb-seq. Beyond RNA-seq, we'd like to measure many other analytes, right? Proteins, metabolites, lipids, protein–protein interactions. How do we do that at scale, beyond individual cells? We'd like to be able to measure cell–cell interactions spatially, cells in their native context, or even whole-animal-level in vivo perturbations.
Again, how do we do that at scale? With lots of great innovation coming out of academia—actually, just one great paper last week. So how do we connect all of these together? I think we have many years of work ahead of us to fully crack data generation for all of biology, and I think we need the scale, industrialization, and innovation from industry; we also need that from academia. Together, I think we'll move this field to the next level.
Shawn Wang
One question that we've been trying to ask everyone is: in your field—which you could say maybe is AI and sort of high-throughput experimentation, or however you want to define it—if you could wave your magic wand and have a bottleneck removed for you, or a key problem solved, what would that be?
Chunyu Liu
Protein.
If there were a way to do protein sequencing or high-throughput protein measurement at the same scale that we can do genomics, that would be amazing. I'm trained in genomics, but if I could do that, I would incorporate that technology in a heartbeat. RNA is amazing: it foreshadows which proteins are going to get made, but proteins by and large are the functional units in a cell.
Not only does their abundance matter, their post-translational modifications matter, and their localization in a cell matters. If we can measure all of those things—their conformational states, their modifications, their abundances, their localizations—at scale, at single-cell resolution or even spatially, I think such datasets will be incredibly useful to train the next generation of foundation models. I know there's a lot of innovation in that direction. I can't wait to see that come of age.
Bo Wang
My hope is—I hope to see a breakthrough in sequencing technology. Not just a reduction in cost, but sequencing technology that can sequence the same cells at different time points. I think this is much lacking right now because, in order to sequence a cell, you have to kill the cell, right? So can we have a technology that can measure cell states at different time points for the same set of cells?
I think that would bring a very different dimension to the dataset, so that we can start to measure the temporal dynamics of cells. So far, everything we measure, everything we model, is extremely static. Can we have a technology that measures different cell responses at different time points for the same set of cells? That would unlock massive opportunities to model the dynamics of cells. To me, that is the real virtual cell.
Shawn Wang
That's a really interesting idea. I never would have thought about that. Wow. Would you be okay with even just partial—small snippets of genes, or maybe 3′ regions of a small number of transcripts?
Bo Wang
Start with a small set of gene panels to begin with, right? But eventually, since we're talking about the magic wand here, if we can have a system that observes how cells evolve at different time points and we have enough data to actually model such development...
I think that would be real virtual cell modeling. There have been earlier attempts—for example, sucking out portions of the cells, taking nanobiopsies from the cell to do a fraction of the cytoplasm measurements. That might be similar to the idea you're talking about. There's also work from a public lab to have the cells secrete little vesicles, and they harvest those in the cell culture media to measure what the cells are producing longitudinally.
But there hasn't been technology that can let you measure the entire cell transcriptome while still keeping a cell. You can't have your cake and eat it, too.
Shawn Wang
Yeah. [laughter] Cool. Thank you for taking the time to chat with us. It's been great. I've learned a lot. There were a lot of really interesting discussions, and I especially appreciate your commitment to open science and all of the cool models and data you've released.
Are there any last thoughts you have, or anything you'd like the audience to know or follow up on?
Bo Wang
Overall, I think virtual cell is such a new and fast-moving field. We hope to have more and more people join us. Our arXiv paper is out, and we look forward to receiving your comments and feedback. We're also hiring. We are always looking for talented engineers, technologists, biologists, drug hunters, AI scientists, and computational biologists.
So look on Xaira.com and look for the open roles. We'll be happy to chat with you.
Shawn Wang
Thank you. Thank you very much. Great. Thank you.