解锁细胞的秘密：与 Squidiff 和 CORAL 的 Siyu He 探讨扩散、解卷积与发现

Thank you, Nathan. It’s great to be here, and I’m very excited to share my research with a broader audience.

Cool. We’ve got a lot to unpack, so let’s start with some big-picture context. I’ve been obsessed with AI for the last few years and studying it intensively, so I feel like I have a pretty good general understanding of the landscape. On the biology side, though, the landscape is even bigger and more complicated, and I haven’t spent nearly as much time in it. I always like to start by getting a sense of where you think we are in the big picture, where you think this work fits, and how you’ve decided to orient yourself within it.

The Squidiff paper is about the transcriptome state of the cell, operating at the level of an individual cell. The CORAL paper zooms out to a larger scope, looking at tissue samples. I’ve had the general sense that the grand challenge in biology involves a couple of different things. One is figuring out the intricate web of what causes what: what upregulates what, what downregulates what, and how an intervention ultimately leads to a change in outcome.

Unfortunately for all of us, that’s not only incredibly complicated, but it also works at multiple levels of scale. We can do a pretty good job of saying, “Here’s a sequence. What does that translate to in terms of the shape of a protein?” We’re even starting to get decent at asking how those proteins potentially fit together. But it seems like we still have a long way to go in understanding how all of that adds up to the function and evolution of a cell over time, and probably even further to go in understanding how that aggregates up to the tissue, system, and whole-organism levels.

I’ve been watching this space because it seems like a critical frontier, and you’ve taken a bite out of each of those problems with these 2 papers. With that preface, tell me more about how you see the big-picture landscape, how you understand the grand challenge, and how you’ve oriented your work toward solving it.

The major motivation behind these 2 projects is understanding whether we can utilize AI models and apply them to current biological technologies. The goal is to understand cellular systems and disease mechanisms, and then provide treatment strategies for disease.

We want to use AI models to work with different types of biological data. For example, I focus on single-cell RNA transcriptomics in the first paper and spatial transcriptomics in the second paper. These are 2 different studies, but they are aimed at a very similar goal: using AI models to accelerate our understanding of cellular activity, figure out disease-related processes, and understand how cells respond to drug treatments.

Let’s get into the first one. The first paper that caught my eye was Squidiff. Am I saying that correctly?

Yes, Squidiff.

I’m interested in where the name came from, just as a bit of trivia before we get into the science.

I had a previous paper called CellOracle, and I was thinking that I wanted more models named after sea animals. We have “squid,” but it’s a diffusion model, so we called it Squidiff. It can also be interpreted as “simulation-related” or “stimuli-related quantitative inferences of transcriptomics using diffusion models.”

The name captures the idea of the model and the goal of the project. CORAL, the second one, is also named after a sea organism. Hopefully, I’ll have more.

You’ve got a theme. You’re building an entire cinematic universe of models.

Tell me about transcriptomes in terms of what we need to know that isn’t obvious. My understanding is that a transcriptome is basically a measure of which genes in a cell are being actively transcribed at a given time. It’s amazing that we can do this at the level of a single cell. The technical achievements and resolution of these techniques are remarkable.

We can take a single cell—I assume this is a totally destructive process, but correct me if I’m wrong—and pull out the RNA that has been transcribed from the DNA at a particular point in time. We can look at how much of these different RNA segments are there, and that tells us the state of the cell at that moment: what active processes are happening in that cell. That’s a rookie explanation, though, so tell me what I’m missing.

A good way to start is with the central dogma of biology. The cell is the smallest functional unit of a living system. In our cells, the chromosomes carry DNA. Almost all the cells in the human body share the same DNA, so what makes them different is the transcription process.

DNA is transcribed into RNA, and RNA is further translated into protein. The protein folds into a specific structure, which is what AlphaFold is trying to predict, and then it performs various functions. The process related to transcription is also called gene expression, and it indicates a lot of the variation between cells in the human body.

One of my postdoctoral advisors, Stephen Quake, has said that a cell is a bag of RNA. That indicates that the complexity of cells can be distinguished by gene expression. Protein expression can also distinguish cellular complexity, but current technology is still more prevalent at the transcriptional level. Proteins are probably noisier, even though we have increasingly advanced technologies for measuring them.

People are mostly studying transcriptomes first because they help us understand cellular activity and disease mechanisms.

The second point I want to mention is the development of single-cell RNA sequencing. I won’t go into all the details of the process, but the major step is that we dissociate tissues into individual cells and then measure the molecular expression, or gene expression, in those cells. We can profile around 30,000 to 60,000 genes, which is a very high-dimensional dataset.

It’s also a high-throughput technology, which makes AI models a good way to analyze the data. We have a matrix with cells and genes as the rows and columns, and it indicates a lot of information about individual cells. This is a phenomenal technology that has opened up new areas of quantitative biology.

Thirty thousand to 60,000 different genes can be measured in a single cell. That’s an amazing level of resolution. It’s always helpful to consider the dimensionality of the inputs, both in biology and in the AI architectures we’re going to discuss.

To be very clear, the starting point for this whole process is a transcriptome: 30,000 to 60,000 numbers representing the expression intensity of individual genes, measured through the RNA pulled from the cell when it’s processed in this way.

Do we have any reason to think that the transcriptome is a complete picture? Are there things that are known to be missing? Is there a way in which the transcriptome doesn’t tell us the whole story of what’s happening in a cell?

Transcriptomes capture some of the heterogeneity between cells, but there is more information. For example, we can look at epigenomics in cells and at chromosomal events that differ across individual cells. There is also protein expression, as I mentioned, which can distinguish cellular states. Sometimes the transcriptome and protein expression share similarities.

There are also mitochondrial-related and ribosomal-related processes. Cells are very complicated, and that’s why people haven’t fully figured out exactly what they’re doing. The transcriptome is one type of data that we can obtain at the whole-genome level, so it’s a very good way to start understanding what is happening.

But there is other information that could also be considered in the model design and in the overall analysis.

Let’s get into the Squidiff architecture. First, what are we trying to make the model do? Both of these papers involve carefully designed architectures that are more complicated than what we see even in frontier language models today. I want to get into why the bitter lesson may not have fully arrived for this kind of work, but let’s start with the goal. What do you want Squidiff to be able to do?

The motivation is that I’m a wet-lab researcher. Obtaining single-cell sequencing data through the whole experimental process is painful because you need to wait for cells to grow, and then you need to prepare the entire experiment. It usually takes months, and it’s also very expensive.

I started thinking about whether generative AI models could create virtual or digital transcriptomes. Then I realized that there might be a way to manipulate the major information in those models. For example, we have many conditional generative AI models. What if we could manipulate the conditions, even when we can’t directly observe the resulting state?

That’s why we designed Squidiff. It isn’t only about generating transcriptomes. It also addresses interesting biological questions, such as what happens when a chemical perturbation or gene perturbation makes cells different, and what those cells look like. In many experiments, we can’t measure those states immediately, and experiments at large scale are difficult. That’s how the model can help the field.

You said it can take months to culture a particular cell type and condition of interest, and that sequencing is expensive. Can you put some numbers around that? Is there an established dollar amount, or is it still a relatively bespoke process where the main expense is the time required to perform the sequencing?

It depends on the tissues or samples you’re working with. I was usually working with organoids, which are engineered tissues derived from induced pluripotent stem cells, or iPSCs. We create engineered brain tissues or blood-vessel tissues and use them for disease modeling.

The whole process usually takes months. Some brain organoids need a year or more to create. There can also be mistakes during the experiments, in which case you need to start over. That makes the process very painful, and it’s one reason I wanted a model that could take over some of the task.

There’s another concern: in some cases, the experiments simply cannot be performed. That’s another reason such a model could be useful.

Let’s do a little more on the actual runtime. You might have a particular cell type and a question of interest: what happens if I apply a drug, apply radiation, or upregulate or downregulate a particular gene?

You want to know what happens, and the answer is ultimately a transcriptome. The system outputs a transcriptome—not one that has been measured, but one generated or predicted by the AI model. Is that the right way to understand it? What does that do for you when you have a transcriptome generated by the model?

When we have transcriptomes from the data, we use different types of analysis for single-cell RNA sequencing to understand what is happening. The first step is usually annotating the cells. People often use dimensionality reduction and manifold-learning methods to see how the cells look, as well as clustering methods to classify cells into different cell types.

Because we have gene-expression information for each individual cell, we know which specific genes are expressed. We can identify the related signaling pathways and understand what is happening. That is the usual way people analyze the data.

There are multiple approaches in the field for quantitatively understanding the data.

How well do we understand that process? If you have a transcriptome, regardless of its source—whether it was measured in a wet lab or generated by a model—what level of granularity, specificity, or confidence can we have about what is actually happening in that cell?

We have numbers for 30,000 to 60,000 genes, but can we aggregate that into the things that really matter? Could you say that a cell is definitely cancerous, that it’s no longer cancerous, or that it’s healthy or unhealthy?

Obviously, you can do more specific things because you can analyze the data gene by gene, but what is our ability to aggregate those results into biologically meaningful conclusions?

If cells are similar and in a similar state, they will have similar groups of expressed genes. We call that coexpression. In that case, we can classify or group cells into different clusters, and each cluster usually corresponds to a cell type.

We can identify highly expressed genes in each group. One method people use is marker genes, which are genes identified by the biological community as important markers for a particular cell type. For example, there are cancer-related markers. If researchers identify those markers, they can infer that the cells are probably in a cancerous state.

Pathologists use similar markers, often involving proteins and antibodies, to determine what is happening in tissues.

There’s clustering, and there are indicator genes that are reasonably well understood. Let’s talk about how you train a model to produce these results.

Where does the dataset come from? How much of it is available in the community from other researchers, and how much do you have to gather yourself? In simple terms, how large is the dataset you used for this project?

It depends on the project. We have a matrix of cells by genes. There are 30,000 to 60,000 genes, covering the whole genome, and we can have thousands, millions, or even billions of cells, depending on the project.

In my models, we use smaller, more specific datasets rather than putting everything together. That is partly because of the complexity of the models and the current stage of development. In the future, I would like to use larger-scale datasets and make the models more like foundation models.

The community shares data by publishing it in public repositories, and we use those publicly available datasets. At the same time, we collaborate with wet labs that help us create novel datasets or provide validation for the model.

The data source is one of the major challenges in the field. We need high-quality data, and although most datasets are good, they sometimes lack important information. That makes it difficult to develop consistent models.

Is the data annotated? When you describe a grid of cells and transcriptomes, I understand that there’s one vector for every cell indicating the strength of activity for each gene in the list of 30,000 to 60,000 genes.

Is there additional metadata about what kind of tissue the cell came from, the patient’s condition, or anything along those lines? Or is it just the raw information?

There is metadata, including cell types annotated by me or other authors. Some annotations are already publicly available. The metadata can include what kind of tissue the cell belongs to and what disease stage it represents. There are multiple ways to label this information, and that is one of the requirements of the model.

How many individual cell transcriptomes constitute the training data for this project?

The model can take a large amount of data for training. I’m not providing a pre-trained model, so researchers need to provide their own training datasets. It depends on the dataset they have.

With around 5,000 cells, training takes about 15 minutes.

You can get this working with as few as 5,000 cells?

Yes, although too few cells can cause the model to be underfit or overfit.

That’s a small number. Is that where it starts to work, while in practice you used more? In the fullness of time, could you use much more?

The future direction is to build a larger model that can cover different types of organoids. In the paper, we use organoids as a case study, but we could encompass different organoids in a foundation model for organoid data. We could also create a foundation model for tissues or real human tissues.

It depends on the exact assets and datasets we want to use. The model is flexible and can take different types of data to address researchers’ needs.

Let’s talk about the architecture. In broad terms, there are 2 core parts: a variational autoencoder and a diffusion-model component.

Before getting into how those work, why not use a Transformer? People in our audience are increasingly familiar with Transformers. If someone came to me and said, “You want to predict the next transcriptome state for a cell given the current state and a perturbation,” my first instinct would be that we could probably use a Transformer. Why did you choose a different architecture?

First, I should correct something in the original manuscript. It isn’t actually a variational autoencoder. It’s a semantic encoder connected to a conditional diffusion model.

There are 2 major components. One is a DDIM model, and the other is the semantic encoder. The generative process takes the semantic variable from the encoder, combines it with Gaussian noise, and uses the denoising process of the diffusion model to produce a final transcriptome.

Why use a diffusion architecture rather than a Transformer?

Transformers and diffusion models are both popular generative AI models, and there are many applications of both in biomedicine.

Diffusion models may capture more stochasticity in the data. Their generation process is flexible, and they can model more complex distributions. Transformers are also powerful, and many related studies use them to generate biological data.

The difference is that the model inputs are somewhat different. Transformers usually take sequence data. In our area, researchers often rank genes first, so the input is not necessarily the continuous-valued data itself. The output is also a ranking of genes, which can be different from the original data.

Many Transformer-based foundation models take an embedding of the data and use that representation for downstream classification and prediction tasks. Squidiff has a different purpose: it directly models continuous gene-expression data and uses diffusion to generate transcriptomes.

On the architecture itself, you described the components briefly, but I want to make sure I understand them. I’ve looked at many different architectures over time, and the one this most reminds me of is MindEye, a project that partially came out of Stability AI. MindEye reconstructed images that somebody was looking at based on an fMRI scan of their brain while they were looking at the image.

That system involved a diffusion model and a diffusion prior. There’s something similar happening here. The first step is trying to understand what really matters in the transcriptome of a cell.

As I understand it, a variational autoencoder typically passes raw data through a narrow bottleneck so that the data can be recreated at the other end, capturing as much of the original data as possible. It is trained with a reconstruction loss, with the goal of compressing the data into its most semantic form while preserving the information that matters.

That gives you a highly semantic representation that can be used for other tasks. Is that roughly correct?

There are different architectures, and a combination of a variational autoencoder with a diffusion model is often called a latent-diffusion model. Related models have also been used for transcriptomics.

The difference is that those models are usually trained separately. The autoencoder reduces the dimensionality and produces a latent representation first, making the generation process more efficient.

Squidiff instead trains a unified model. The encoder and diffusion model are trained together. The whole process includes learning the noise and encoding the semantic information, so the model can extract semantic information from the data while also taking advantage of the stochasticity of diffusion models.

The result is more realistic expression data for individual cells.

So this system is trained end to end under one loss function, whereas in other contexts the variational autoencoder is trained separately and then combined with other components downstream.

That brings us to the diffusion model. People are familiar with this from image generation. I like to explain it procedurally: if the goal is to train a model to generate an image, you can give it an image and ask what the image would look like if it were slightly less noisy. With conditioning, you can also ask what it would look like if it were slightly less noisy and more like the thing you’re trying to create.

The genius of this approach is that it allows you to produce training data at scale. You can take images from the internet, gradually add noise, and train the model to denoise them. Eventually, the model can start with pure noise and go step by step until it produces a high-resolution image. With conditioning, it can produce a high-resolution image of something specific.

The same mechanics are at work here, with the major difference being that the conditioning comes from a semantic encoder. The encoder says, “This is the kind of cell we want,” in a lower-dimensional, semantic representation. The diffusion model then maps that back into the fully detailed picture of a particular transcriptome.

What am I missing?

The data structures are different. Most image data is 2-dimensional, with x and y directions. Single-cell transcriptomic data is 1-dimensional: the object is the cell, and the features are the gene expressions.

Because of that, we can’t use the same image-based diffusion architectures, such as U-Nets, to learn the noise. We use a multilayer perceptron, or MLP, to learn the noise. We also use residual connections to incorporate the time embedding and the semantic features.

The semantic features provide implicit information. They don’t only include cell type; they can also include disease stage, experimental conditions, and other related information. They form a unified condition that can be manipulated in the latent space rather than directly in gene-expression space.

Is there anything special about the noising process? In images, it’s relatively simple. In protein folding, though, you need a specialized noising process because adding noise naively to a protein structure can produce something incoherent. Does that apply here, or can you use a relatively simple noising process?

The way we deal with the noise is relatively simple. We didn’t need to address major problems in the noising process.

The main issue is that most expression data is sparse, with many zero values. We need to account for that. We adjust the data by filtering out genes that aren’t useful and focusing on highly variable genes.

Another advantage of diffusion models is that they aren’t limited to simple distributions. They can generate very complex distributions, which is important because gene-expression data can have complex distributions. The model has the capacity to represent those patterns.

Let’s talk about how you run an in silico experiment. You have a semantic encoder that takes a raw transcriptome and converts it into a more compact, semantic representation that abstracts away details and tries to capture what matters.

The key thing I found in the paper is that, to ask what would happen to a cell type if you gave it a particular stimulus, you need to encode that stimulus as a direction in the semantic latent space.

People may have seen examples of this from language or image models. If you have embeddings for “man” and “woman,” you can subtract them to create a direction from one to the other. Then you can apply that direction to another embedding. The classic example is that if you take the embedding for “king” and add the direction from “man” to “woman,” you may get something close to the embedding for “queen.”

It seems that this is the core mechanism here. You might have cell type X and want to know what happens when you apply stimulus Y. It could take months to culture cell type X and run the experiment. But perhaps you have cell type Z and can apply the same stimulus to it. You can calculate the difference between the original and perturbed state of cell type Z, then apply that vector to cell type X and run the diffusion process.

That would let you predict the transcriptome of cell type X after the stimulus without ever directly performing the experiment on cell type X. Is that what is happening?

Yes, that is essentially what we’re doing. We use vector operations to manipulate the semantic variables.

However, this is an assumption and not exactly what happens in every biological process. Biological processes are complex and usually nonlinear. The latent variables are probably more linear than the gene-expression space, so we use linear approximations to resemble the real cases.

For example, in one of our applications, we study cell differentiation. On day 1, we may have iPSCs, which are stem cells. On day 3, we may have more mature mesoderm cells. If we have semantic variables for day 1 and day 3, we can use linear interpolation to estimate the transcriptomes on days 2 and 3.

Another application involves gene perturbations. We can upregulate or downregulate gene A, upregulate or downregulate gene B, and then ask what happens when A and B are perturbed together. The result may be nonlinear, and the diffusion model can represent nonlinear processes. The latent variables may still be manipulated through vector operations because the latent space is smoother and more structured.

That is an approximation, so it’s risky to rely on the model without validation. We applied it to organoid cases and saw exciting results. We identified transient cell states during development and differentiation from iPSCs into epithelial cells, fibroblasts, and blood-vessel structures.

We’re also considering a variant of Squidiff that uses a time-series analysis. Instead of a simple semantic encoder and semantic variables, we would use a more complicated dataset with temporal information. This is similar to video generation, where the model learns the development process over time and captures more of the data’s nonlinearity.

It’s interesting how techniques initially developed for image and video generation are finding direct applications in biology. I was following diffusion models in 2021, when they were generating strange but sometimes intriguing art. It was unclear how useful they would be.

Now we’re seeing direct applications to biology. Image generation has advanced to the point where it can be difficult to distinguish model outputs from photographs, and video generation is moving in the same direction. It’s not perfect yet, but it has come a long way.

Projecting that kind of progress into the biological domain over the next 2 or 3 years would be a paradigm shift. If these models could produce accurate biological simulations, they could completely change the field.

What do you think is the main obstacle? There’s the nonlinearity issue, which seems fundamental. How extensively has the model been validated? I understood that there was a 3-day cell-differentiation process. Because you can scale the vector, are you literally calculating the intermediate days as one-third and two-thirds of the vector?

Did you measure cells on days 1, 2, and 3 and compare the model’s predictions with those real measurements? How closely did they align?

We have several validation strategies. In the differentiation experiments, we have real experimental data from day 0 through day 3. During training, we use day 0 and day 3, but hold out days 1 and 2 as test data. Once the model is trained, we use semantic-variable manipulation to generate predictions for the intermediate stages and compare the predicted gene-expression values with the real measurements.

For gene and drug perturbations, we use a similar approach. We may have experimental data for perturbing gene A, gene B, and genes A and B together. We hold out the combined perturbation and train on the individual perturbations. Then we test whether the model can predict the combined result.

The third strategy involves real organoid data. We need to culture the organoid, so we can’t stop the process and collect a sample every day. Collecting the sample for single-cell sequencing destroys the entire sample. We might collect data at day 0 and at the final day, then use interpolation to generate intermediate stages.

We also have publicly available time-series data. We compare our predictions with that data to determine whether we can identify similar biological patterns. We discovered novel cell states that had not been previously reported. We then studied which genes were triggered by the differentiation process and compared our findings with experimental data from other publications. We found consistent results, which supports the model’s ability to identify new biological states.

The semantic-space manipulations are linear vector operations, which seems like a problem because biology is so nonlinear. But the diffusion process can capture nonlinearities, so perhaps the combination can work. It remains to be seen where those linear operations break down in the semantic space.

Have you seen validation failures where you concluded that a process was fundamentally nonlinear or that the semantic space wasn’t capturing it?

The cases we’ve tested so far have generally worked, but the intermediate-stage predictions haven’t been as good as we hoped. That may be because we are modeling semantic variables that approximate a factor—for example, differentiation from iPSCs to mesoderm cells.

There may be multiple growth factors and multiple directions that can lead to the same final state. We’re making a linear approximation, but if we had more information—such as the semantic variables for each individual growth factor—the model would be much better.

At this point, we don’t have that information, so this is the way we address the problem.

There is another limitation. Suppose we have data for drug A and drug B and want to understand the result of applying them to another cell type. If we have the vectors for A and B, we can apply them to other cell types and predict the responses.

But what if we have a new drug, drug C, for which we don’t know the semantic vector? We can’t model that unseen drug directly.

To address this issue, we’re considering another version of the model with an adapter that encodes drug information. We use the molecular structure of the drug and its dosage information to create an embedding. That embedding is combined with the semantic variable to create a new semantic representation.

In that case, even if the model hasn’t been trained on an application of drug C, it may still be able to make predictions if we provide the components and dosage of drug C.

Do you think this is already capable of accelerating important scientific work? I understand how it could. In the short term, we have a finite amount of wet-lab work that can be done. We have finite lab space, finite numbers of graduate students and postdocs, and a limited amount of time for cell culturing.

If that throughput is relatively fixed, then what determines how much scientific progress we make is whether we run the right experiments—the experiments that produce interesting results we can learn from. The hope for in silico experiments is that they run orders of magnitude faster.

How long does it take to perform one diffusion process?

The model can help researchers accelerate their work. Obtaining single-cell RNA data may take at least 1 week, from dissociation through sequencing and obtaining the RNA files.

If you want to understand what happens under other conditions, you could use the generative model and spend perhaps an hour generating predictions. That gives researchers some intuition about the direction of a project. I wouldn’t say they should trust the model completely, but it provides useful information.

For example, organoids are useful drug-screening platforms. Researchers study different ways of creating organoids. If we have vectors for individual growth factors, we may be able to predict what happens during organoid development and how organoids respond to different drug compounds.

Another application involves an individual patient. If a patient has cancer, they want to know which drugs will be most useful. It’s risky to randomly try drugs, so we need to test them. The model could predict how the patient’s cells respond, whether a drug is likely to be useful, and whether it may have side effects.

That could provide efficient predictions without requiring the patient to wait a long time to see the outcome.

The individual-patient context is especially interesting. I was mainly thinking about general scientific inquiry: run many simulations, identify the most promising ones, and prioritize the wet-lab work accordingly. That alone could accelerate discovery.

But this becomes even more transformative when applied to an individual with an idiosyncratic situation in their own cells.

Do you think the next steps are mainly scaling up, separating the interventions into distinct semantic encoders, and adding a time-series or video-like component? That seems like plenty of work.

I’m always struck by the pace of research in this area. We’re not talking about just one paper today; we have another entire paper to discuss. When you look ahead to those next steps, do you feel that it’s mainly a matter of doing the work?

At my software company, engineers used to say, “It’s just a simple matter of programming.” They meant that it would take time, and there would be challenges, but that we would eventually overcome them. Are you similarly confident that these next steps will lead to better models and predictions?

It depends on the broader community and on collaboration between scientists. We need a large amount of data, and people have different areas of expertise. Collaboration will be very helpful for accelerating the work.

We need biologists to understand the mechanisms of disease, machine-learning researchers to create the models, statisticians to evaluate them, and other experts to make the models and their applications more rigorous.

This is a collaborative effort. Another important point is that we are applying AI models to health care and medicine, so accuracy matters. We are responsible for patients, and it’s critical to understand exactly what the model can do and not use it in the wrong way.

A lot of work is required before these models can become clinically available and testable. We need experimental tests, clinical tests, and a great deal of preliminary validation before applying them to real patients.

In laboratory settings, there is more flexibility because we can perform additional experiments to test the process.

Any last thoughts on SquiDiff before we move on to CORAL?

I think that covers the main points. I really appreciate the questions. You captured the key ideas of the model.

Let’s talk about the CORAL paper. This one was even more challenging for someone without a biology background to understand.

The fundamental challenge is connecting different levels of resolution or scale into an integrated understanding. In many ways, that is the central challenge of science. We can go down to the lowest level—perhaps string theory or quantum mechanics—and develop theories about how a small number of particles interact. Then we move up through atoms, molecules, proteins, cells, tissues, and whole organisms.

There’s often a gap between those levels. We don’t know how to aggregate the smaller units of analysis into the larger thing we care about. Economics has a similar divide between microeconomics and macroeconomics. We don’t have a complete way to aggregate individual economic actors into an economy, so we use a separate top-down approach based on aggregate statistics.

Biology seems to have this problem at a very high level of difficulty. What motivated this work, and what are you trying to do with it?

This project addresses a technical problem involving spatial data. I’ll start by introducing spatial transcriptomics, which is the main technology in the project.

We’ve discussed single-cell RNA sequencing, but it has an obvious limitation: we dissociate the tissue into individual cells, which means we lose the location of each cell. That information is important because cells aren’t isolated. They interact with one another, and we need to know which cells are neighbors.

Losing spatial information makes it difficult to study tissue structure and cellular communication. That led to the development of spatial transcriptomics and spatial proteomics. These technologies emerged roughly 7 years ago, and now an increasing amount of data is being generated. We need computational methods to understand it.

One significant issue is that spatial data does not always have the same resolution or cover the same tissue section. You can think of it like having 2 photographs of yourself: one is a high-resolution grayscale image, and the other is a lower-resolution, colorful mosaic image taken at a different time or from a slightly different location. The goal is to combine them into a high-resolution, colorful image.

The analogy is imperfect because photographs generally have 3 channels—red, green, and blue—whereas our data can have tens of thousands of channels. Spatial transcriptomics can measure 30,000 to 60,000 genes, but it may have lower spatial resolution. Spatial proteomics can have higher spatial resolution, but it can measure only a limited number of proteins.

Neither dataset is perfect. We want to combine them to obtain a more comprehensive understanding of tissue architecture, how cells coordinate with one another, how they respond to disease, and how we can develop new treatment strategies.

There are newer technologies with high resolution for both proteins and RNA, but there is still a significant need for methods that address these differences in resolution.

There is another difficulty: when we measure spatial proteomics and transcriptomics, we often cannot use the exact same tissue slice. The slices are adjacent and close together, but they can still be shifted relative to one another. That makes the data integration more difficult.

That’s a good primer on the difficulty of doing this, especially when the data comes from a living person. It may be easier when the sample comes from an organoid developed in a lab, but with a living person you have the procedure required to obtain the tissue, measurements taken at different times, and samples that may be nearby without being identical.

Then you have different types of measurements, each with different strengths and weaknesses. All of that creates a confusing picture that isn’t integrated into a single understanding.

This reminds me of applications of the Mamba architecture to biomedical imaging. The goal there was to integrate different types of scans—MRIs, ultrasounds, and other imaging modalities—that each provide different information. If the scans are taken at different times and the person is in a slightly different position, it becomes difficult to determine how they should be aligned.

One application involved deforming scans in space so that they could be combined into a coherent view. A clinician could then see the different signals in an integrated way.

It’s a similar problem here, involving different perspectives on what is happening in the body. This is another example of how AI can help with problems for which humans don’t have good intuitions. At best, a small number of experts can develop those intuitions with a great deal of time and effort, but they can process only a limited amount of information.

AI systems can learn the intuition needed to make sense of these data and then apply it at a much larger scale. What are the major contributions of CORAL? One is taking a higher-level, lower-resolution measurement and deconvolving it—taking something that is lower resolution and determining what each individual cell in the larger sample may look like.

What practical value comes out of this whole setup?

Deconvolution is a major task that the model addresses. But CORAL also provides a comprehensive analysis by integrating different modalities, such as transcriptomics and proteomics. The same approach can be transferred to other modalities.

Once we have high-resolution data, we can explore it at the single-cell level and examine the latent features. We can organize the spatial regions and identify structures in the tissue. We call these functional domains.

Another important goal is understanding how cells communicate with one another. CORAL uses a graph-neural-network-based model to infer interactions between individual cells.

The model can also investigate spatial variability within tissues. Together, these capabilities allow us to deconvolve the data, study tissue architecture, and analyze cellular interactions.

I don’t have a strong intuition for how differentiated the body is. It’s remarkable that we start as a single cell, then become a group of relatively undifferentiated cells, and eventually develop many different types of cells.

Some cell types seem to transition gradually, while others appear to have more abrupt transitions. From my basic understanding of biology, segmentation can occur through gradients of signaling factors.

A key assumption in this work seems to be that these transitions are fairly gradual. The graph neural network models cells interacting with their neighbors. I wasn’t sure whether it also models longer-distance interactions.

There also seems to be a component of the loss function designed to keep things smooth, so that as you move through a tissue, the properties change gradually.

I have trouble reconciling that with certain parts of the body where the boundary seems very clear. If I think about a bone and the tissue surrounding it, it seems like we know exactly where the bone stops and the other tissue begins. There doesn’t appear to be a smooth transition, although perhaps I’m simply not zooming in closely enough.

What am I missing, and what would help me develop a better intuition for this?

It depends on the type of tissue being studied. As you said, bones have clearer boundaries, and blood vessels also have boundaries. In tumors, however, the boundaries may not be as clear because tumor cells can infiltrate normal tissue.

In some cases, it’s difficult to identify structures directly from morphology. If we look at molecular information, though, we can identify interesting patterns that are not visible from the tissue’s shape alone.

The model’s ability to capture boundaries depends on the quality of the spatial data, including both the spatial resolution and the gene resolution. The model represents interactions between neighboring cells. It doesn’t only consider the single most adjacent cell; it can also consider the nearest neighbors.

The number of neighbors is controlled by a parameter. We can choose the value of that parameter and create a larger or smaller graph. It isn’t necessary to look too far away because we’re primarily interested in the closest neighboring cells and their communication.

There are many complicated processes in tissue, so we simplify the problem by focusing on each cell and its neighbors.

The graph loss function encourages smoothness. It is a regularization term intended to make the model more realistic. We also have reconstruction losses and a Kullback–Leibler divergence term.

Together, these losses balance smoothness and sharpness. With single-cell-level data, we can identify clear boundaries because the resolution allows us to distinguish individual cells. We can see the boundary between bone and blood vessels, for example, and potentially identify even more detailed internal structures within organs and tissues.

The reconstruction loss incentivizes accuracy, so when there is a sharp boundary, it encourages the model to capture it. The smoothness term tries to smooth things out and acts as a regularizer. A carefully designed compound loss can give you the best of both worlds.

Did I understand correctly that the parameter you mentioned controls how many nearest neighbors each cell connects to? You can use a small number of adjacent cells or expand the radius to include more neighbors.

Yes, that’s correct.

Let’s talk about synthetic data. In language modeling, the community has been on a bit of a roller coaster. There was concern about a data wall, followed by optimism that synthetic data would solve the problem. Then people worried that synthetic data would produce bad models through mode collapse.

That idea eventually lost traction, and it became clear that having models solve problems and applying reinforcement learning to those solutions could work very well. But I have much less intuition about synthetic data in biology.

What kinds of synthetic data exist? How much can we trust it? How much of it did you use in this project, and what are its current limitations?

We use synthetic data along with experimental data to validate the model.

For simulation, there are well-known models and technologies in the field. For both single-cell and spatial data, these simulations are designed to resemble experimental observations. They model distributions and sample structures that are similar to what we observe in real data.

In Squidiff, we use Splatter, a well-known method for simulating single-cell RNA data. For CORAL, which uses spatial data, we work with a collaborator who is now a professor at the University of Connecticut. He designed a model called scDesign, which was published in Nature Biotechnology around 2 years ago.

That model can sample from designed distributions with spatial information and generate observations that resemble spatial gene-expression data.

We also use real data. The advantage of synthetic data is that we have ground truth. If we are interested in cell interactions or functional domains, we know what those domains are in the simulation and can test whether the model identifies them.

In real data, we don’t have that ground truth. We can still use biological knowledge to validate the model, but synthetic data lets us perform a more direct evaluation.

The reason we need synthetic data is that real data is difficult to gather. But this can feel like a hall of mirrors, and you hope it isn’t a house of cards. If we can generate synthetic data, it can seem circular to ask whether we can learn from it. If we already know how to generate the synthetic data, haven’t we already learned what it contains?

What are you learning from the synthetic data that wasn’t already encoded in the process that generated it?

We use the synthetic data to validate whether the model can capture the results we expect. We don’t use exactly the same generation process in the model and in the simulation.

We generate synthetic data with a well-known or common method, and then test whether our model can recover the underlying structures. For example, we may know what a spatial domain looks like or what the true interactions are. We can evaluate whether the model identifies those domains and deconvolves the data into higher resolution.

In real data, we don’t have that ground truth. Once we validate that the model performs well on synthetic data, we can transfer it to real data. That gives us more confidence in the results, even though we don’t have direct ground truth for the real samples.

The synthetic data is generated independently from the model we are evaluating, so there isn’t a loop in the process.

When ground truth is difficult to obtain, things can become tricky.

There’s a lot of discussion about a Manhattan Project for superintelligence. Sometimes it’s unclear what that superintelligence is supposed to look like or do, and whether we’ll be able to control it.

When I talk to people doing detailed work in biology, I sometimes think a worthy Manhattan Project would be obtaining the data needed to scale this research. A lot of relevant data is probably locked up in medical-record systems and difficult to share because of privacy rules. Those rules have an important purpose, but responsibly making more of the data available could unlock enormous progress.

A major investment, potentially involving government support, could also help. If we’re going to subsidize AI development, perhaps we should make the biological data available so that researchers like you can make discoveries more easily.

The most compelling answer I hear when people get excited about superintelligence is that it will cure diseases and help us live longer, healthier lives. I want that. But perhaps instead of building a trillion-dollar data center to create an undefined superintelligence, we should make sure researchers have the data they need to make discoveries directly. Maybe we end up with both.

How different could your work be in 1 or 2 years if there were a strategic, national-level project to make biological data available and facilitate the next stages of scaling and improvement?

Many projects are trying to make models larger and more capable so that they can represent the human body. For example, Google DeepMind has projects involving virtual cells and unified models that represent different types of cells and organs across different scales.

The Chan Zuckerberg Initiative has also launched projects involving very large numbers of cells, such as the Billion Cells Project. These efforts aim to build large-scale models of biological systems.

This is an exciting period because researchers are using foundation models and large datasets to address these questions. We have GPT models, and people are creating biological versions that can answer questions about cells.

Other researchers are interested in clinical and health questions, so we may eventually have virtual doctors. There are many possible applications.

People are also working on digital-twin projects. We may be able to create a digital copy of an individual and understand what might happen if that person made different choices at some point. That is another very interesting direction.

There are many exciting questions, and I’m looking forward to seeing how these models develop over the next 1 or 2 years.

It’s an exciting time, although also a little scary. This kind of work feels like it has mostly pure upside. Obviously, any technology could theoretically be misused, but I don’t think we need to worry about SquiDiff or CORAL getting out of control.

These are focused systems answering specific problems. They’re a reminder that this work still happens and still has value. It isn’t all about embracing the bitter lesson and maximizing the size of the cluster you can apply to a problem.

I appreciate that about both projects. Is there anything else you want to tease about the next steps, or any final thoughts you want to leave us with?

I’m very optimistic about how AI can help health care and medicine. I know people may be worried about how quickly AI is developing, but at least for now, I don’t think there is a reason to be afraid that AI will destroy the world.

I hope researchers working at the intersection of AI and health care will make the development of health care faster and help people live better lives. That’s what I wanted to share.

Thank you for this great opportunity. I’ve really enjoyed the discussion.