Why Diffusion Will Win AI Inference with Inception Co-Founder and CEO Stefano Ermon
- Ermon’s core bet is that AI economics will shift toward inference efficiency, where autoregressive models remain sequential while diffusion models process many tokens in parallel. That workload maps better to GPUs, improving “intelligence per watt” and “intelligence per dollar” while supporting more test-time reasoning and RL rollouts; his categorical wager is that “the more parallel solution is the one that is eventually going to win.”
- The proof point moved from theory to measurement in 2024: at GPT-2 scale, a diffusion language model matched an autoregressive model’s perplexity with the same data and parameter count while generating text roughly 10x faster. Inception says its commercial Mercury models now benchmark alongside speed-optimized Haiku, Flash, Mini, and Nano models while running significantly faster.
- Latency already has a monetizable wedge, especially in voice and other interactive applications where users will pay for responsiveness. Voice-agent company OpenCall moved from serving its LLMs on Cerebras hardware to Mercury on NVIDIA GPUs, according to Ermon, obtaining comparable speed with greater hardware availability, lower cost, and higher quality; software and hardware acceleration can be “multiplicative.”
- Ermon estimates that 20-30% of workloads could be addressable on latency alone, even before diffusion reaches frontier intelligence. His caveat matters: many tasks still demand the smartest available model, but within a fixed latency budget users will choose the highest achievable quality—and, like broadband, “once you get used to a fast model, it’s hard to go back.”
- Diffusion may eventually offer more than speed, but Ermon treats those benefits as hypotheses rather than promises. Coarse-to-fine generation could permit continuous steering against constraints, while training on many noisy views of the same sample might improve data efficiency; whether either advantage survives scale is explicitly “we’ll see.”
- Inception’s moat is not merely an architectural paper but a proprietary full stack built by roughly 50 people. The company created its own serving engine, SFT/RLHF/RL infrastructure, training recipes, customer evaluations, and feedback loops because existing tools such as vLLM and SGLang do not support its models; keeping much of that stack closed protects IP but impedes community adoption and on-premise deployment.
- The strategic risk is that a larger lab could absorb a successful architecture, but Ermon argues execution and accumulated deployment knowledge may be harder to replicate than the headline idea. His broader evidence is academia’s record of contrarian breakthroughs—early diffusion, FlashAttention, and DPO—and his conviction that, despite AI-assisted research, “human ingenuity is still super important.”
1. Diffusion began as a contrarian search for structure
Ermon started at Stanford in 2014, when generating grainy MNIST digits counted as success and generative-model papers were difficult to publish. Researchers often justified the work indirectly—as unsupervised feature learning for supervised tasks—but he believed building a generative model was the right way to understand structure in data.
His original ambition was a world model: a system able to imagine what would happen if he stood up and walked through a door, then use that imagined future for decisions and model-predictive control. He expected the problem to “keep me busy for my whole career” and admits he was “very wrong” about how quickly capabilities would advance.
After autoregressive image models, VAEs, and unstable, difficult-to-reproduce GANs, Ermon and a PhD student began developing score-based generative models in 2019. Their key move was to train a network to denoise: start with pure noise, progressively refine it, and recover a clean object—an approach that spread across image, video, music, and some protein generation.
2. The 2024 result made text diffusion commercially credible
Text posed a foundational problem because pixels interpolate naturally while “between” two discrete words may mean nothing. Extending diffusion into discrete spaces therefore required “a new science,” not simply applying an image recipe to tokens.
The breakthrough came in 2024 at GPT-2 scale: using the same data and parameter count, Ermon’s group trained a transformer as a diffusion model and matched an autoregressive model’s perplexity. It identified the same amount of structure in the data while producing text roughly 10x faster, giving Ermon enough confidence to found Inception and test the method at commercial scale.
Ermon’s architecture analogy starts with the 2017 transition from RNNs to transformers. Transformers won training because they processed many tokens concurrently, but autoregressive inference remains sequential: “You cannot generate the 10th token until you’ve generated everything that comes before it.” The resulting workload is memory-bound, repeatedly moving weights while performing relatively little arithmetic.
Diffusion makes inference resemble training by processing many tokens simultaneously, mapping the workload to the parallel arithmetic GPUs perform well. Faster, more efficient inference can support more test-time reasoning and more RL rollouts for exploration and scoring. Ermon’s “bitter lesson” is blunt: “The more parallel solution is the one that is eventually going to win.”
3. Mercury turns speed into a present-tense product wedge
Inception says Mercury is comparable on benchmarks to speed-oriented Haiku, Flash, Mini, and Nano models while being significantly faster. These are production systems, not research demos: because diffusion LLMs cannot run on vLLM or SGLang, the company built a serving engine capable of handling real workloads end to end.
The roughly two-year-old, 50-person company still spends heavily on training and inference research, including sampling methods that trade compute for quality. Yet it reuses what transfers: Mercury remains transformer-based, uses attention, public datasets, evaluations, and benchmarks, allowing Inception to concentrate resources where diffusion is genuinely different.
OpenCall illustrates the immediate demand. Its voice-agent pipeline combines ASR, a reasoning LLM for tool use and response planning, and text-to-speech, making latency conspicuous. Ermon says it previously served its LLMs on Cerebras, then switched to Mercury because diffusion delivered similar speed on NVIDIA GPUs—with wider availability, lower cost, and higher quality. Hardware and software gains, he argues, are complementary and potentially “multiplicative.”
Looking two years out, Ermon does not claim diffusion will own every workload; Inception has not yet reached frontier intelligence, which many tasks require. From OpenRouter’s task taxonomy, however, he estimates 20-30% of workloads are highly latency-sensitive—a plausible lower bound for models offering the best quality inside a fixed response-time budget.
4. Control and data efficiency remain upside, not underwriting
Guo’s pushback is that code and other human-created corpora are messy, unlike modalities grounded in physical structure. Ermon answers that both autoregressive and diffusion training are compression schemes: lower perplexity means the model found recurring structure despite the noise. Whether next-token prediction or denoising has the better inductive bias remains “very much an empirical question.”
Mercury preserves familiar deployment semantics—OpenAI-compatible text input and output, instruction following, and structured JSON—so customers can retain their existing harnesses. But Ermon sees a possible diffusion-native interface: an autoregressive system can only score a constraint after completing an object, whereas coarse-to-fine diffusion can evaluate and steer an emerging answer throughout generation. He illustrates the idea with a molecule whose solubility can be scored and used as a steering objective.
A second possibility is data efficiency. Denoising exposes a model to many corrupted views of the same example, effectively augmenting its training data; academic evidence suggests that might help when data is scarce. Ermon is careful not to overclaim: speed is the demonstrated wedge, while greater intelligence, controllability, and scaled data efficiency remain possibilities—“we don’t know what we’re going to find.”
5. Inception’s defensibility rests on the stack and learning loop
Guo surfaces the central startup risk: a compute-rich incumbent could absorb a superior architecture. Ermon’s initial defense is research IP and trade secrets, but his deeper answer is deployment accumulation—the proprietary serving engine, customer feedback, sometimes-collected data, and evaluations derived from production experience. Training a diffusion LLM is insufficient if a competitor still lacks “the vLLM equivalent” required to operate it.
Inception also built its own SFT, RLHF, and RL infrastructure. Keeping much of that machinery closed preserves more IP, but Ermon openly acknowledges the cost: fewer community contributions, harder adoption, and more difficult on-premise deployments. Training details—including model sizes and FLOPs—remain undisclosed as trade secrets.
The organization splits between a product team serving today’s best model and a research group building the next one across training, RL, and inference. Agents have accelerated iteration, and Ermon often finds compute more constraining than headcount, but he says that today “human ingenuity is still super important,” even if that assessment could change within six months or a year.
His confidence in contrarian research comes from academia’s track record: early diffusion beat GANs at the scale his group could train academically before spreading through industry; FlashAttention emerged from work he co-advised; DPO began as a rotation project in his group. Limited compute is real, but academia permits unusual bets and mathematical insights—“there are gems”—that can later have major industry impact.
Full transcript
1. Research Background
Hi listeners, welcome back to No Priors. Today I’m here with Stefano Ermon, a longtime Stanford professor and now co-founder and CEO of Inception Labs. Stefano has an extraordinarily broad body of work around generative modeling, but is especially well known as one of the fathers of diffusion. We talk about his company challenging the large labs and why speed and efficiency are going to be the name of the game in AI over the next few years. Welcome, Stefano. Thanks so much for being here.
Great to be here.
I would love for us to start with a little bit of your research background and how you ended up starting your company.
2. Inception and the Broader Industry
For sure. I’ve been doing research in generative models for basically my entire career. I started at Stanford in 2014 as an assistant professor, and I was working on building generative models. Back then, the research area was not particularly hot. The models were not working particularly well. We were still building little generative models over MNIST, and it was a big success if you could generate these grainy images of digits.
It was even hard to publish papers on that topic. You had to justify training a generative model as a way to learn features from unlabeled data that could maybe help you do better at supervised learning, because that was the thing that everybody cared about. But then things took over, of course, and I was at the right place at the right time, working on the right thing. I’ve been doing research in that space since the beginning, basically.
Did you have, besides a curiosity in the area, a personal hope for what the models would do back in 2014 and 2015?
I always felt that was the right way to think about learning from unlabeled data: building a generative model is really the right way to make sure you understand the structure in the data. That was what I was getting at. I wasn’t even dreaming about the kinds of capabilities that the LLMs we have today could do.
I was thinking more from a world-models perspective. I was working a lot on images and thinking, “Okay, I have a world model. I can imagine what’s going to happen if I were to stand up and walk out the door. I can picture that in my mind.” That’s important to make decisions, and in model-predictive control, having this kind of model of the world requires some generative capabilities.
I always felt, “Okay, that’s the right direction to work on.” I felt like this was going to be very hard, was going to keep me busy for my whole career, and was a good problem to work on. Then, of course, I was very wrong, and things evolved much faster than I was expecting.
3. Starting Inception
I think that’s kind of universally true, though. Walk me through the state of your research and how that led you to start the company.
I was working on generative models of images, initially working on autoregressive models, which were very slow and kind of blurry, then VAEs, and then GANs took over.
Yes.
Back then, we were very unhappy with the state of generative models for images. The GANs worked, but they were very unstable to train and very hard to reproduce results with. We were trying to see if there was a way to build something that was as good as GANs but more principled.
We started working on score-based generative models, which are basically what eventually became diffusion models, back in 2019 with my PhD student. We came up with the idea of training a neural network to denoise images. If you can denoise an image, then you really understand enough about the structure of the image that it should be possible to build a generative procedure based on these denoisers.
That basically became the underlying technology of modern diffusion models. Instead of generating images left to right, 1 pixel at a time, you start from pure noise and gradually refine the object until you get a clean picture at the end. That started back in 2019 in my lab, and then it took over the space.
Even today, the best models for image generation, video generation, music to some extent, and a lot of the protein work are based on diffusion. My group has worked a lot on various kinds of diffusion-model techniques for accelerating them to generate samples very quickly and improving the quality of these models.
Since we were able to get diffusion models to work on images, I started thinking about how to get them to work on text or code generation. Can discrete objects be handled in a way that moves beyond autoregressive models to something more parallel, with built-in error correction? I’ve been doing a bunch of research at Stanford on getting discrete models to work on text and code generation.
We had a breakthrough in 2024. We published a paper showing that, for the first time, it was possible to match the quality of an autoregressive model at the GPT-2 scale. It was less than 1 billion parameters and still fairly academic, but we were able to train basically the same transformer model as a diffusion model on the same data. We were able to match the quality—the same perplexity—and we were fitting the data just as well as an autoregressive model with the same number of parameters.
But the diffusion model was significantly faster because it was diffusion: You’re outputting many tokens at the same time. We were able to generate text 10 times faster compared to the autoregressive model. That felt very exciting, and I really wanted to see what happened if you scaled up and trained bigger models. So I started Inception Labs, a company to basically scale up the technology and try to build commercial-scale, diffusion-based language models.
4. Why Diffusion Beats Autoregressive
Everyone has now seen the outputs of diffusion models, in particular in images. I would argue that increasingly, generated short-form video from diffusion models is a dominant form of entertainment in other parts of the world, and it will likely become so here. It’s kind of unbelievable to me, even having followed the field for the last decade-plus, the quality that’s possible today.
That seems obvious, right? It’s such a big use case in images and video generation that people are even creating hardware to support better performance here. It’s not intuitive that this would work for other fields, or that this is an interesting competitive direction against the current full-transformer-focused AGI labs. Can you offer some intuition on that?
It’s a very interesting state of the world right now from a researcher’s perspective, because there are 2 main paradigms—2 ways of building generative models. There’s autoregressive modeling, where you have a model that predicts the next token or the next pixel, and then you generate left to right, 1 token at a time. Then there’s diffusion, which is coarse-to-fine, iterative denoising generation.
As you said, we have continuous modalities where diffusion dominates. There are discrete modalities—text and code—where primarily all the big labs are betting on the same architecture: autoregressive models. As we move toward more and more multimodal models, there’s this idea that maybe we’ll have a model that can handle all modalities and know everything about the world. What architecture will that be? Will it be an autoregressive model or a diffusion model? Nobody knows. I think the jury is still out on that.
At Inception Labs, we’re betting on diffusion models because we believe that what matters eventually will be inference-time scaling, and there are fundamental reasons why diffusion models are better than autoregressive models at inference time. Even if you think about the story of autoregressive models, there was an inflection point in 2017 when people switched from RNNs to transformers.
Why was that? The problem was that RNNs had to essentially process tokens sequentially, 1 at a time, and training was very slow. People came up with the idea of an architecture that allows you to process many tokens at the same time, in parallel. That was a transformer, and it was the thing that scaled better for training and enabled a lot of the successes behind LLMs.
But if you think about inference—not training, inference, generation—autoregressive models are still sequential. The computation is left to right, 1 token at a time. You cannot generate the 10th token until you’ve generated everything that comes before it.
That kind of workload does not map well to GPUs. It’s extremely memory-bound. You’re spending most of your time moving weights across the memory hierarchy and doing very little arithmetic, and that’s a fundamental problem with autoregressive models.
The equivalent, if you think about RNNs, transformers, and autoregressive models, at inference time is a diffusion-based model. A diffusion model is built to have, at inference time, a workload where you process many tokens at the same time. The workload that we have at inference time in a diffusion model is very similar to the workload you have for training, where you’re processing many tokens at the same time in parallel.
And so it's built to essentially have an inference workload that maps really well to the kinds of things GPUs do really well. We bet on trying to build the architecture and the kind of models that will scale best at inference time, because economics are dominated by the kind of intelligence per watt and intelligence per dollar that you're able to get from the models.
If you think about a lot of the advances with reasoning models, a lot of it is scaling test-time compute, right? Being able to scale better along that axis will also matter. And even if you think about RL post-training, a lot of the bottleneck is generating rollouts—letting the model explore, scoring the trajectories, and then improving the model based on the kinds of things it finds. Inference is again a key bottleneck for RL post-training. If you have a model that scales better at inference time, then automatically you're going to get better scaling during RL post-training.
5. Discrete vs. Continuous Modalities
That's why we decided to bet on a diffusion-based LLM, because it's inherently more parallel, and the bitter lesson is that the more parallel solution is the one that is eventually going to win.
How did you think about applicability, or what experiments did you run in terms of cracking the nut on discrete versus continuous modalities? Because I think people have also shaped the existing dominant paradigm through new tokenization efforts or methods to make video and voice work, for example. This is not the same token-oriented paradigm. How do you make it work here?
There was a lot of research that went into figuring out how to apply a technology that was inherently very tied to continuous structure in the data. If you think about a diffusion model, it's learning how to denoise images, and it kind of makes sense for continuous data because, if you think about even 2 pixel colors, you can interpolate between them and it will still make sense.
But if you think about 2 words, there isn't necessarily something in between them, right? It's all discrete, and so it required a lot of R&D and a new science that had to be developed to figure out how to extend those kinds of ideas to discrete spaces.
What can you claim about how well it works today?
We think it works really well. We've been able to train diffusion-based LLMs that are comparable in quality with the speed-optimized models from frontier labs. Our Mercury models are on par with the Haiku models, Flash models, and Mini and Nano models from OpenAI, if you look at benchmarks, while being significantly faster.
6. Inception Today
We've gone, I think, from pure research prototypes to things that are actually used. We're serving these models in production today. We did all the work of figuring out how to even build a serving engine, right? You cannot run these diffusion-based LLMs on vLLM or SGLang; we had to build our own serving engine, and we can handle a lot of the complexity of real production workloads. We've solved all these challenges, and we can deliver this new experience end to end to real customers today.
Actually, a great time to just talk about where Inception is as a company. How many people? What are you guys actually serving? Sort of the state of the research?
Yeah, so we're about 2 years old, with around 50 people, spending a lot of time still on R&D—figuring out what's the right way to train these models and how to accelerate inference. It's not obvious how to do that. If you think about an autoregressive model, it's pretty clear there's not a lot of things you can do there: you generate 1 token at a time, and that's it. In a diffusion-based model, we know that there are a lot of different possibilities for trading compute for quality at inference time.
If you think about image diffusion models or video diffusion models, there are a lot of techniques that you can use to accelerate sampling, from distillation to fancy differential-equation-solving techniques that allow you to sample very quickly from these models. There is a lot of research on the training, on the inference, and then engineering—just thinking about data mixes, evals, and RL post-training infrastructure.
There is a lot of work that needs to happen to figure out how to build recipes that work for this new model, and we try to leverage existing things as much as possible. For example, it's still a transformer-based model, so you don't have to throw away a lot of the work that has been done on good architectures. We still use attention, and we still use a lot of the public datasets that people have created, as well as evals and benchmarks.
We're a startup, so we try to be scrappy. We try to use existing things as much as possible and focus on the aspects where we can have the highest impact and where we can be the most differentiated. Right now, it's speed. In the future, who knows? It's possible that a diffusion-based language model will be maybe significantly more intelligent than an autoregressive one. Nobody knows that.
That's why I think this is very exciting, because we're developing these really powerful AI systems, but it's all very fresh. It's all very new. I doubt we've discovered the best way of building these systems. There's got to be alternatives. There's got to be other ways of building these models, and eventually, efficiency will be very important.
If you think about the AI factory, how is that going to work? I think nobody really knows, and just being able to play in that space and think about alternative ways of creating intelligence, I think it's exciting.
Absolutely. I also think that in an increasingly, fundamentally compute- and supply-constrained environment, I wouldn't say that there wasn't very focused research on the efficiency of models, but it was a second-order consideration for many of the industrial research efforts versus pure capability scaling, which is what we've thought about, right? And new methods in that.
7. Where Speed Wins
But if you fundamentally believe that we can use all of the power we have and there will be economic reasons to do so, then I think the focus on efficiency is going to go dramatically up. Many people now argue that one of the most important decisions for a company in the AI space is: How do we use the power we have between training and the most valuable use cases for any watt?
On that topic, here in September 2026, where does speed win? Where do people care enough about this already?
Yeah, it's basically applications where latency matters. Generally speaking, I think everyone cares about speed in the sense that if you can give me the same quality but faster, people will always pick the faster solution. We're seeing it with these faster versions of even the models from frontier labs. People are willing to pay more to get access to faster models, right?
8. Inception Customer Base
I think once you get used to a fast model, it's hard to go back. It's kind of like broadband, right? It gets faster and faster, and if you're able to try it, people cannot go back once you get the fast model.
Are there customers that you can talk about publicly that care about this today?
9. Interaction with Hardware Landscape
There are a few that we can mention. In the voice space, for example, OpenCall is an example. They're building voice agents. The pipeline is: You have an ASR model, you have an LLM that's doing all the tool calls and figuring out what to say next, it has to be a reasoning LLM typically to have the highest level of quality, and then there is a text-to-speech component at the end. Speed matters a lot to them.
They were previously serving their LLMs on Cerebras. They were using custom chips to get to the kind of speed that they need to deliver the best experience to their customers. Then they switched over to our diffusion-based LLMs because they can essentially get the same speed as what you would get if you were to run an autoregressive model on custom hardware.
If you have a diffusion-based LLM that's built to be parallel and it's accelerated at the software level, then you can get the same speed on NVIDIA GPUs, which means much more availability. GPUs are scarce, but there's more of them than custom chips, as well as lower cost and higher quality. So that's an example in the voice space.
Yeah, I was actually going to ask you how you think this interacts with the hardware landscape as well, given we've seen enough demand now from use cases that are like, yes, I want a big, expensive chip with a lot of SRAM, and people will pay for the outputs of that—in coding and other use cases.
Yeah, for sure. And I think hardware is one way to accelerate things.
And software might be better if we can use the existing hardware.
Exactly. Exactly. And especially, they are complementary. That's the exciting piece: To some extent, the gains that you get from the software are multiplicative with the gains you get from the hardware.
And maybe someday people will develop custom hardware that fits even better the models that you're building.
For sure.
If we zoom out to Inception and the broader industry, I think there is a vein of concern—and correct me if I’m wrong—that it’s very hard to invest in new architectures today because if there are advances in architecture or methods, they will simply be absorbed by large players with the resources to scale compute. Talk to me about how you think about competing as David in this situation.
That’s a very valid point and something that is also top of mind for us. Initially, for us, the moat is the IP, the trade secrets—the ideas that we and our researchers have to build these models and make them better. As we mature as a company, one of the reasons we’re not just doing pure research but have also developed a product and have real customers is that we’re getting feedback on the models from the real world.
By doing that, we’re also developing components that are very important to deploying these models. For example, a serving engine: if you don’t have the serving engine, you can’t really serve these models in production. By forcing ourselves from the very beginning to go out and deploy something end to end, we’re learning a lot about how to serve these models and how to build software that’s needed to run them. That again becomes IP. Even if you train a diffusion-based LLM, if you don’t have the serving engine or the vLLM equivalent to serve it, you’re still stuck and you still cannot use it.
Along the same lines, we’re working with real customers and getting feedback on the models. We figure out what works and what doesn’t, we collect data from them sometimes, and we create evals based on what they’re seeing. That again becomes part of the technical moat because those things are a little bit harder to replicate.
10. Data Compression and Structure
You can tell me if this doesn’t make sense as a question to ask technically, but one of the things that diffusion models benefit from structurally in image or video generation is that you’re replicating something where there should be some consistent structure in the world. Voice as well, right? It is whatever is really possible and most likely.
Some of the fields where AI has been most valuable to date—I’d say a lot of the input data you use to train, like code data, for example, is very messy. One could argue that a lot of it doesn’t actually have the correct real structure you’re looking for. How do you think about that when it’s human-generated input data versus images, video, or voice?
It’s a good question. Fundamentally, whenever you train a generative model, whether it’s an autoregressive model or, to some extent, even a diffusion model, you’re trying to identify structure in the data by essentially building a compression scheme. That might not be obvious, but whenever you train these models, you’re effectively trying to identify common structure by finding an efficient way of compressing the data.
The more you can compress the data, the more structure and patterns you’re identifying. That’s how these models work, which is the amazing thing: by predicting the next word, you’re learning something about the structure of the data. It’s the same whether you’re using a diffusion model or an autoregressive model. Both methods are essentially trying to learn a compression scheme.
When I mentioned the original 2024 paper, when we showed that we were achieving parity with autoregressive models, the metric we were using was perplexity, which is a notion of how much structure you have identified in the data. Even though it might not seem obvious, we were actually able to identify, at the GPT-2 scale, the same amount of structure as an autoregressive model.
Yes. I think that empirical result is there, but the intuition would be: code is not grounded in physics. There’s a lot of noise in the data set that you’re working on, and it sounds like you believe that’s a manageable problem.
There is noise in everything. To the extent that the numbers don’t lie, if you’re able to drive the perplexity down, then it means that you can actually build a compression scheme, and that will get you that level of compression. The structure must be there, and the model must have been able to uncover it.
11. Controllability of Diffusion Modeles
It’s more a question of inductive bias. Is a transformer a better way of identifying those patterns, or is something else? Is next-token prediction the right modeling framework, or is it more like denoising? That’s very much an empirical question that, at the moment, we don’t even have the tools to understand.
Can I ask a question just because you used a voice customer as the example here? One of the benefits that some people building these AI products have identified in having an LLM in the middle of a voice pipeline is that it can handle alignment a little bit better, or improve controllability. I imagine that has to look different for a diffusion-based model. Can you talk about that?
Yes, that’s a key value proposition. One of the things they always look into is the extent to which a lot of the value they provide comes from the harness and making sure that the models are indeed doing the right thing.
The interesting thing about a diffusion-based LLM is that we’ve built everything to be backwards-compatible. The API is the same. It’s still OpenAI-compatible: text in, text out. It so happens that the models we’ve trained are good at following instructions. They’re good at outputting structured outputs if you’re using JSON, and they can handle all of those things.
It was good enough. It was better, in fact, than the models they were using before. They’re still able to provide that level of service to their customers by using Mercury.
Well, it’s very simple if the interfaces are the same and you can just use your same stack.
Yes, but I think that’s actually a very interesting point. We know that diffusion models are typically easier to control compared to autoregressive models. The reason is that, with an autoregressive model, you kind of have to wait until you’ve generated the full object to know whether or not it satisfies a constraint, whether or not it’s aligned, or whether it meets whatever objective function you care about.
Maybe you’re generating a molecule and you care about solubility. You have to wait until you have the full object to be able to score it with some reward function. But a diffusion model is more coarse-to-fine generation, so from the very beginning you can progressively ask, “Is this object the kind of thing I want or not?” You can steer the generation in the direction provided by an external reward function or some set of constraints.
You can progressively do it.
From the very beginning, you know whether the object is the kind of thing you want, and you can steer the generation accordingly. There’s a lot of evidence in the academic literature, at least, that diffusion models are easier to control, and there are different ways of steering them that are just not possible with autoregressive models.
That would be a different interface for the model that might not even be available for autoregressive models. We’ve been thinking a lot about what the right product experience is that we can build around new capabilities that are simply not provided by autoregressive models.
12. Emergent Capabilities at Scale
Are there capabilities that you imagine Inception’s models having at scale that today’s models don’t have, beyond performance?
That’s the thing: we don’t know. It’s emergent. Right now, the wedge is speed. We know they’re much faster. That was the initial bet because it was easier to test, it was easy to measure, and it’s obviously valuable.
But that’s why I find it so exciting: as we learn more about how to train these models, we don’t know what we’re going to find. There’s a decent amount of evidence in the academic literature, for example, that diffusion-based models are more data-efficient compared to autoregressive models.
The intuition is that when you train a diffusion model, you’re learning by denoising. You start with an image, add noise, and then learn how to remove the noise. It’s effectively doing data augmentation in the sense that the same image is augmented by many noisy views.
Okay.
They tend to be a little bit more data-efficient. If that holds up at scale, then maybe we’ll get into tasks where we have less data.
Yeah, where data becomes the bottleneck, then it becomes more interesting, right?
13. Future Workload Split Between Diffusion vs. Traditional
We’ll see. That’s why it’s so exciting: things are changing, and this technology is so important and valuable that having something differentiated will create value.
If we project out 5 years—which is actually way too long in the AI world—if we project out 2 years, what do you think is the workload split between diffusion and traditional models?
I think we’re still not at the frontier level of intelligence, and a lot of workloads do require frontier-level intelligence. But in my estimates, even if you look at OpenRouter, it has a very nice way of looking at all the different use cases. You can see research, conversational use cases, coding, software engineering, and log processing. They have a nice, hard taxonomy of tasks.
14. Adoption Challenges
I was doing some estimates, and I think there’s somewhere between 20% and 30% where latency is really important. At the very least, as a lower bound, I think that could be addressable by models that, within a given latency budget, will give you the highest possible quality.
All technology approaches have trade-offs. What are the challenges of working with diffusion models?
It’s a different stack, so one of the challenges is that we had to build a lot of things in-house. There isn’t a mature ecosystem of serving engines or kernels. A lot of those things had to be developed in-house.
There isn’t really much that’s open source. There are some open-source models, but they’re not particularly good. That makes it a little bit more difficult to deploy and to get customers to try things. They’re not used to it, so that’s been one of the challenges.
I imagine that also reflects externally. In a landscape where folks have some sophistication and care about cost and performance, for certain use cases you’ll care about cost and performance from the beginning. There’s an increasing amount of interest in post-training, and I imagine that in a new architecture, that’s even more challenging.
We had to build our own stack for doing SFT, RLHF, and RL. That becomes IP to some extent.
15. Hiring and Team Organization
It’s one of the reasons we decided not to open-source everything: we wanted to keep the IP a little bit closer to us and not open-source it. But there are downsides, like there’s less opportunity for the community to contribute. It’s harder to adopt, and it’s hard to do on-premises deployments, so there are pros and cons to the two choices.
Can you talk about the scale of your own training, current or aspirational? At 50 people, I’m sure you’re continuing to hire. Why should researchers, engineers, or others consider investing in this direction or working at Inception?
We’re not able to share much about the training—the size of the models, the FLOPs, or any of that. It’s kind of a trade secret, but we are continuing to push the frontier, and Inception is a great place to be if you want to have an opportunity to shape the direction of the field.
16. Recursive Self Improvement
It’s still a relatively small field. There’s a lot to be invented, and a lot of the people who decide to come to Inception instead of joining one of the other labs want to have ownership and like to invent new things. They like to be in a space where there’s more of a green field and more opportunities to try things. There’s less that’s known or available out there. It’s a little bit more open-ended, and so we tend to attract those kinds of people.
I think one thing that is both exciting and causes some despair among my research friends is the ability to use models for recursive self-improvement in the research field itself. Given that you’re working in a very different direction, what is your view on this?
Yeah.
I mean, explicitly, it sounds like you still feel there’s work for you and your team to do.
Oh, yeah. I think we’re not there yet. Maybe we don’t have access to the models that other folks have, but I feel like there is still work to do. We use models a lot, of course, and they have accelerated the speed at which we can iterate and try ideas. We use models from frontier labs, and it’s been great. It has accelerated our development process a lot.
17. Resource Allocation
At the same time, I think that, at least right now—I don’t know what it’s going to be in 6 months or a year—human ingenuity is still super important. The ability to come up with the right ideas, prune the space, and identify directions that are more promising has been really important to us.
Fifty people is not that many people for, let’s say, a full-stack research, serving, and product company—or however you would think about describing it. How do you organize, and how do you think about allocating your resources here?
It’s a small team, but everyone is very talented, they work very hard, and we have access to agents that are making us a lot more productive. I think the numbers are sufficient to do a lot. In fact, often I feel like the bottleneck is more compute than people.
18. Impact of Academia
The team is organized so that there is effectively a product team handling the platform and working with customers to make them successful with our models. There is basically a team serving the current best version of the model, and then there is a team building the next version of the model. That includes training, RL, and inference, so that’s more research.
Stefano, one last question for you. The 2024 paper was a super interesting result and made a big splash. You’ve been working in this field for a long time. A lot of folks would say—or would claim—that academic AI research is very challenged in this era of being able to scale resources a great deal. This is certainly true to some degree, given that you started a commercial company around it as well.
How did you gain confidence that the directions you were working in would have an impact or be promising before you really had those results?
I think it was a collection of results that I had been working on in my lab. Of course, there’s the early diffusion work that we did in the lab. We showed that, at the kind of scale of models that we could train academically, we were able to beat GANs while being much more stable. Then the whole thing took over and became Stable Diffusion and Midjourney; all of that started from ideas that were developed in academia, in my lab.
But that’s not the only one. I was a co-advisor, so I worked on FlashAttention, for example. That’s another thing that came out of academia and eventually had a huge impact in industry. Another example is DPO, which was a project that started as a rotation project in my group. It’s an algorithm that is used to align LLMs and diffusion models everywhere.
That’s again something that was developed entirely in academia. It was based on a clever insight—some interesting mathematical structure in that problem—that allows you to come up with a very different and more efficient way of post-training and aligning these models.
There are gems and lots of opportunities for finding new and better ways of solving important problems. One of the nice things about academia is that it allows you to take these contrarian bets. As you said, the challenge is that maybe we don’t have enough resources, and there are never enough resources. If we had more compute, we could be more efficient.
But you have access to amazing students, everyone is trying to develop the new thing, and people aren’t scared of taking bets. That’s why academia has been so impactful, I think. Even in the AI space, a lot of the important ideas have roots in—or were even created in—academia.
Awesome. Super inspirational. Thanks so much for being here, Stefano.
Thank you so much for having me.