Cohere 首席 AI 官 Joelle Pineau：Scaling Laws 为何会延续，以及合成数据的未来

The scaling laws have been remarkably robust. There's a lot we don't know yet in terms of the vulnerability of these systems.

Maybe you don't need to buy the Galácticos. Why do you have an Andrew Tull, a Daniel Gross, an Alex Wang, and the Galácticos assembling?

I used to be quite skeptical that neural networks were necessarily the ultimate solution to machine learning. I seem to have been quite wrong on this one.

Knowing what you know, what do you not let your children do?

Eat too much sugar. I don't have a lot of patience as a scientist for people who are predicting the extremist scenarios—the catastrophic risks of AI, where AI becomes our overlord.

1. The Rising Cost of Data

If I gave you $10 billion, what would you spend it on first?

Ready to go? Joelle, it is so great to have you in the studio. I've heard many great things from Nick, Aidan, and Shrep[?]. Thank you so much for joining me.

2. How Meta Shaped How I Think About AI Research

Thank you. Happy to be here.

You spent over 6 years at Meta, and I want to start there because it's a very transformative time and place. What are the biggest takeaways for you from that time, and how did that shape your mindset and how you think today?

I was there from 2017 to 2025, and you have to see just how much AI changed over that period of time. What we were really focused on was fundamental AI research. One thing that I've learned is how long it sometimes takes to prove out a hypothesis. We feel like AI is moving at the speed of lightning, but in fact, there are some things that just take a few years to mature—to get the right optimizer, the right compute, and the right data for that to really make a difference.

I look at where we are today, and everyone says, “It's here, it's here, it's here.” Then you actually look at what a lot of the leaders have been saying recently. Andre was saying it's not the year of the agents; it's the decade of agents. Sam is pulling back, too. Have we got over our skis, and are we actually all pulling back and realizing that time is the factor we need to rely on?

3. Challenges in Reinforcement Learning

I'll give you an example. I've been in research for a couple of decades now. I've been working on reinforcement learning for over 20 years, and suddenly everyone's been talking about reinforcement learning since the advent of reasoning models, agents, and so on. Sometimes you have to be a little patient with these ideas, and the right algorithmic tweak, the right context, or the right problem domains just opens up the magic.

I was listening to Andre yesterday, and he said on this show that reinforcement learning is terrible.

Less terrible than 20 years ago.

Have we overinvested in RL-based methods at the expense of more scalable alternatives?

I'm still super bullish on RL. The concept itself is so fundamental: this idea of training a system through a system of rewards, indicating what's valuable and what's not valuable through numerical values. That is so fundamental, and it's not going away. Where we're maybe getting a little bit ahead is thinking that RL out of the box is going to give us AGI. That part is a lot less likely.

If you look at the curve of progress, RL is terribly inefficient, and the amount of signal you need to really shape the behavior of a model is far from where we are today. We'll need to figure out how to deal with this learning-efficiency problem.

You're probably thinking, “What did I get myself in for?” I don't blame you. I ask questions that I think everyone else thinks, but I'm not afraid to say I don't know. Why is RL so inefficient?

4. AI in Enterprise: Efficiency and Adoption

You're going to get me on a deep topic. There are a few reasons. One is the fact that RL is about sequential decision-making. Think about starting at a point: you need to figure out what you're going to do next, and you might pick the right side of the branch or the wrong side of the branch, and then the road keeps splitting.

Every time you make a mistake, it compounds through the length of the series of actions you're making. That means the amount of error you can make can be very large, and getting it right is quite difficult. Sometimes people compare it to finding a needle in a haystack—finding the right solution in RL.

The other part that's hard is the fact that, to train the system, to train the models, you essentially have to take actions to learn. You can't learn from static data. You can learn some things from static data, but to get the right policy, you need to test it out. That means you need a simulator, you need to get the synthetic data, and all of that can be really expensive. We have difficulty getting a variety of environments and simulations to test RL.

When we look at the cost curve for RL—you said you've been working on it for 20 years—have we seen that dramatically come down? Will we see it continue to dramatically come down, or is it fundamentally an expensive method of training?

It's come down, especially in domains where we have good reward functions. The place where most people started hearing about RL was around the time of AlphaGo. The game of Go was one of the goals for AI. Many people thought we were still a decade away from having machines play Go at the level of humans. Then a team from DeepMind went off, played against the world champion, and showed that RL could basically do it.

In cases where we clearly know what the goal is and can write down the reward function precisely, we're good. We can make a ton of progress. That's why you're seeing progress in mathematics, very well-defined reasoning tasks, games, and these kinds of things.

Using RL to shape the behavior of models and get them to be social creatures—we have no idea how to do that. I don't know if you have children, but shaping their behavior is difficult. The number of times you can repeat the same thing and they still do something else is remarkable. You don't know how to write that out mathematically, and that's where I think we're still in for some hard work.

Okay, so we still have some hard work. When we look at the training-versus-inference market today, we've put so much weight on training so far, and it's been incredibly costly and expensive. Then I hear everyone say, “Actually, inference is 95% of the market. That's where it's all going, and that's where Nvidia will make most of its money.” How should we think about the cost curve applied to training versus inference, and where it sits today?

I think there are a lot of different variants. If you'll allow me, I'll pivot to where I'm going with Cohere. I joined Cohere less than a month ago, and it's a super exciting company.

One of the things that Cohere is doing is developing AI models that run on-premises. That means enterprises bring them in and run them locally, so the company has to worry about training the models. Obviously, we want world-class models for the needs of the enterprise, but the enterprise doesn't have to worry about the inference cost. The client's customers have to figure out what's the right way for them to digest the AI.

That means there's a lot of motivation to have very efficient models so that they can run really efficiently on-premises. We get caught up in one paradigm, but there are other paradigms as well.

If they're the ones paying for the inference, is there less incentive to make the models efficient? You're not the one paying for it. If you're the one paying for the inference costs, you want it to be as efficient as possible because it's your dollar going to that. But if it's IBM's dollar, I would love it to be efficient, but we're not paying for it.

We're still in the early days of AI adoption in the enterprise, so what's good for the client is good for us.

5. Is It Possible To Be Capital Efficient in AI

Totally. What's the biggest challenge about capital-efficient AI today? I know that sounds strange when you look at the economics, so to speak. What's the biggest challenge?

There are a lot of challenges today. In terms of the economics of AI, I think one of the biggest challenges is the fact that it's very hard to have predictability. Everyone wants to know when we're going to hit the breakthrough. Everyone wants to know how many GPUs they actually need. Everyone wants to know what return they can expect.

There's just a lot of uncertainty built into the system. A lot of that is because there's a lot we don't know about this technology. That means we have to take on quite a bit of risk when we're building out a data center, building out a workforce, or trying to figure out how much data to curate. That makes it difficult for a lot of people. People want answers, and this is a world where we don't have that level of predictability compared to other industries.

Does progression happen in a linear fashion, or does it happen in step functions, like AlphaGo or DeepSeek, which, depending on what you believe, suggests a lot of efficiency in terms of model improvement? Is it a step function, or is it linear?

I tend to decompose the different ingredients that lead to progress. People often talk about the algorithms, the data, and the compute. I think, in general, compute and data have a more linear effect on progress. You build more compute, you run bigger models, and you can typically get better performance. You feed in more data; it's not just quantity—you need to worry about quality and diversity as well—but roughly, it's more linear-ish with respect to the data.

The algorithms are the ones that have the nonlinear effect. You can explore lots of ideas, and then something like the Transformer comes along and just changes the paradigm. It's not just the Transformer: on the optimization side, suddenly we hit upon Adam, which is a technique to optimize your model, and it changes the paradigm. With reasoning, suddenly we start thinking about how to put that in the loop, and it changes the paradigm. Those ideas tend to have a nonlinear effect.

The challenge with these algorithmic ideas, though, is that they may take a long time to prove themselves out. The paper can be sitting out there, and there are thousands of papers coming out. The idea is sitting out there, and we may not think to try it with the right data, at the right scale, and with the right combination of hyperparameters, so you don't notice that effect for a while. It's hard to predict, and it's nonlinear more on the algorithmic side than on the data, compute, talent, or other sides.

With respect to Google, Transformers were obviously birthed at Google and sat as papers for maybe a couple of years. When you mentioned compute, algorithms, and data, if we just go through them to understand, everyone suggests that there are 2 different worlds. Scaling laws exist—just throw more compute at it. When you look at data-center investment and the desirability of compute, and then you have GPT-5 seemingly focusing on efficiency and other signals, do scaling laws play out from here, and if so, for how long?

The scaling laws have been remarkably robust. They don't play out exactly as we expect, but they've still been remarkably robust. Lots of people have bet against scaling laws in the past, and overall, we've seen a pretty robust effect. They don't work alone; we also need these algorithmic innovations. Most of the time, I wouldn't bet against it.

On the algorithm side, is that the hardest thing to innovate on? You could think about buying more compute; it might be hard, but you can buy more compute. With data, there are different ways to get it, whether it's synthetic or human. Are algorithms the hardest thing to innovate on?

It's certainly the most creative work to be done. The space of ideas is so wide that I would say it's the hardest in the sense that, as a researcher at heart, you can move in so many different directions. Picking the right one is something you don't know until you get there.

It's a little bit like reinforcement learning. In that sense, I think it's the most interesting one, the most frustrating one, and certainly the most difficult one from an investor's point of view, because you don't know where to put your chips.

Speaking of knowing where to put your chips, and moving from purely a research lens with Meta to now also building product, is there ever this inherent conflict between intellectually interesting research and the need to productize and monetize? How do you think about that?

One of the reasons I'm really excited to be joining Cohere is that we're at a stage where AI is really starting to be useful. Maybe not as useful as people think it is, but we are there. By working on AI that's going into enterprises, I feel we're going to get such an interesting signal of what works and what doesn't work.

We keep talking about AGI and AI for the masses, but when you need to sell AI to a business, you get a real signal of what works and what doesn't work. That's what I'm most curious to see. We've been using academic benchmarks for many years. You get some signal, but it's not the same as getting this to do productive work.

I'm curious to learn from that. We're going to get new types of data and, I think, a lot of insights that are then going to drive the research ideas. That's the other thing to think through when you have a large space of ideas to explore: getting that feedback signal from the real world is super useful to guide you through that search of ideas.

I just had a great chat with David Khan at Sequoia, who said that he thinks a good barometer for utility value within enterprises is: does it have the ability to replace the work of your bottom 5% in any category? He says we overestimate a lot.

Can it replace the bottom 5% in any function? If it can, that's a very meaningful improvement. Do you think that's a good barometer, and how would you advise an enterprise on whether something is useful as a yardstick?

I prefer, in terms of a barometer of productivity, something a little bit different, which is to say: can most of your employees do 10x the amount of work with AI versus on their own? That, to me, is actually a better barometer. I think humans and AI have very complementary abilities, so to just flat-out replace a portion of your workforce is actually pretty unrealistic. Some may try, and some may be slowing down their hiring, but I actually think—

Respectfully, I think 10x-ing your work feels more unrealistic. Is that not a bigger ask? I'm almost more intimidated by 10x-ing my work.

I don't think that's unrealistic at all.

Wow.

Yes, within a timeline that is the next couple of years.

Yes. How does that actually shape out, then?

I think you have to identify very concretely the types of work that you are delivering. We're starting to see Hollywood-quality productions being made in a matter of hours. We're seeing, to take a super-concrete case, machine translation. If humans are doing the translation compared to machines doing it, you go from hours to seconds on long-form text and multipage documents.

For a lot of work, it's not that AI can do all of the work. Humans still need to ask the right question, verify the information, and shape the tasks. But once the task is well-defined, the parameters are clear, and all the design considerations are fed into the prompt, you press the button and you've got an answer in seconds for something that used to take weeks or months.

I completely hear you and understand that. I'm just trying to reevaluate a belief that I've had for the last few months. I'm a venture investor, and for all of us to make money, we need to see the transition from a human labor budget to AI spend. It's with that transition that we obviously see the TAM massively increase and make a lot of money.

But when I hear you say that, I suddenly question that assumption. The barometer for whether we make money is that you're suggesting we don't replace the human labor budget; it just makes us 10x more efficient. Is that correct?

Yes. There's a lot of nuance to all of that. Some work will be harder to get that same level of efficiency gain, whereas with other work you'll see 100x in terms of efficiency gain. But I do think that, for a lot of the work that's happening right now, that's absolutely feasible.

Where do you think the efficiency gains are most tangible? It goes back a little bit to this notion of what the tasks are that we can specify.

In any case where we can be very precise about what a great result looks like, we'll be able to make that task automatic much more easily than tasks that are much more nuanced and have a lot of complexity.

So it's ambiguity.

Ambiguity in the specification of the task is what's hard for our machines.

How have you seen enterprise reaction to this? There's fear from workers sometimes, excitement from leaders, and apathy sometimes. How have you seen and measured enterprise response?

A lot of the workforce can be reasonably fearful about job displacement. There are also a lot of individuals who have an instinctive reaction to change, and change can be hard for a lot of people. We're seeing a lot of change in a very short time span, so I think there's also a generational effect. For some generations, that change is more jarring.

I have teenagers and young adults at home. For them, they're just natives. They're going to grow up with that technology in a different way than some of the older generations.

It's interesting. Speaking of children at home and how they engage with it, Sam Orman said that young people engage with it as an OS to the world, and AI is that companion for them, while older people use it as a next-gen Google. Do you agree with that, and do you see that in your work?

I see a lot of people using it as a tool more than as a companion. People have this Swiss Army knife in their work life all of a sudden that can be super helpful, but that's really most of what I see.

Totally get you. What are enterprises' biggest challenges with AI adoption at scale?

For many enterprises, one of the challenges is making sure that the AI comes in and can be integrated into their workflows, their processes, and their information.

And so the challenge is to deploy in a way that allows them to exploit all of the information systems that they already have. Some of them have accumulated these over decades. That's some of the work that remains to be done.

Integration with existing systems and data flows.

That's, of course, something we see a lot at Cohere because we do on-premises deployments. One of the things we focus on the most is data confidentiality and security, so that enterprises can exploit all of that information. That's top of mind for us.

But it's also a huge opportunity. I would say there's a big interest in that, but making sure to get that compatibility, I think, is a challenge. In many cases, change is hardest for people, and so you have to get them curious about using the technology. Many people feel they have to get it right the first time, and I really think a spirit of exploration and curiosity is much better suited to the phase of maturity of the technology that we have today.

We don't have all the answers about how it should be used. That's going to come from people in the field.

6. Security Concerns with AI Agents

Security is a topic that we quite often glaze over, especially when investing in application-layer AI tools. What does no one know about AI security that people should know?

With respect to AI security, I think there's a new front that's opening up with the development of agents. Frankly, there's a lot we don't know yet in terms of the vulnerability of these systems.

With LLMs, we're starting to get a better understanding. We've had quite a few red-teaming exercises and jailbreaking, and so on. People have identified different risk vectors—prompt injections, things like that—which are vectors for malicious actors to interfere with a system.

With AI agents, we haven't seen that. One of the features of computer security in general is that it's often a bit of a cat-and-mouse game. There's a lot of ingenuity in terms of breaking into systems, and then you need a lot of ingenuity in terms of building defenses, so we just have to stay very active in that sense.

What are the potential vulnerabilities in an agent world?

In terms of agents, we worry a lot about hallucinations in LLMs. The parallel in agents is impersonation: agents that come along and are essentially impersonating entities they don't legitimately represent and, in doing so, taking actions on behalf of those entities. That could involve infiltrating banking systems and so on.

I do think we have to be quite lucid about this, develop standards toward it, and develop ways to test for that very rigorously. There are ways to reduce that risk drastically. You can run your agent completely cut off from the web, and you're reducing your risk exposure significantly. But then you lose access to some information.

Depending on your use case, depending on what you actually need, there are different solutions that may be appropriate.

Totally get you. That's a really hard one because then verification becomes the most important thing. But then who's the arbiter of verification? Is it governments? Is it companies?

Mm-hmm.

How does one think about that? Who says you're a valid agent versus an invalid agent?

Governments can be good for defining standards on which we all agree. Companies are much better at building the solutions at scale and deploying them.

Do you think governments are good at setting the standards when you look at AI and where we're at? And then when you look at the sophistication levels of government programs or decision-makers, with respect, they're just a little bit behind. Do you think they are actually equipped?

I don't think you should look at where governments are in terms of AI regulation necessarily. AI as a field is so incredibly young and fast-moving, and by nature—and there's some good in this—governments are moving a little bit more cautiously and usually need to benefit from our knowledge to make good policies.

I do think you can look at other fields in terms of regulation. You look at aviation: the security record for aviation today, compared to where we were 50 years ago, is just incredible. Governments have played a role in defining that in terms of standards and what the norms are, and so on.

I'm quite hopeful. I'm an optimist about this—maybe it's my Canadian side—that governments can play a useful role. In many cases, clear standards actually mean reducing uncertainty for a lot of companies in this space.

But we shouldn't expect that to be ahead of the technology. I think that would be the wrong order of things, in some sense. We need to develop that technology with enough creative space, and we need to learn fast. Then we need to develop the right guardrails for that technology from the real learnings we have.

We mentioned governments and their role. When I had Nick Frosst on the show, he was saying that there are actually benefits to not being an American company, given some geopolitical challenges sometimes. I'm just intrigued: do you think we will have these kinds of sovereign models for each geography? We have Mistral in France, and Cohere, obviously, was founded in Canada, but I know you've got global headquarters. Do you think we will have these sovereign models and regionalized winners?

I do think it's healthy that there are models getting built in different places around the world, not just in the US and China right now. I think this is healthy in terms of diversity of thought. I think it's healthy in terms of having a greater number of people with access to technology.

For Cohere, the vision isn't to be a Canadian company. The vision is to be a global AI company. We have headquarters in Toronto, and we have teams distributed around the world. We have a great team here in London, as well as in the US, in France, and other places.

Having the ability to deploy models that operate across the world is going to be an important part of the strategy for Cohere. I think there's a great opportunity.

What being headquartered in Canada gives us is a sensitivity to the fact that it's not always a one-size-fits-all solution. I go back to the research we've done. We've done leading work in terms of multilingual models, and it turns out it matters. You go to Japan, you go to Korea, and they do want models that work well in their language.

7. Can Zuck Win By Buying The Superstars of AI

People in the workforce are still operating in the language of the country. Having a company that's attuned to that, that values that internationalization of models, is actually important in the global market.

Totally get that. On the team-building side, obviously Canada has great talent. You mentioned some in London as well. What have been your biggest lessons and observations on team-building in this talent frenzy that we're in? Also, how do you analyze that?

One of the things that's important when you're building a team for AI is that you need people who have vision, who have a sense of what we can create, because we're in a space where there's so much innovation that is still needed. You need an ingredient of vision. That can be 1, 2, or 3 people who bring that ingredient of vision.

You need people who have amazing execution muscle. They don't care that it's their idea. They care that, if the team agrees on an idea, they're just going to push it and get it done. They're going to build the system and run the experiments. They have the technical rigor to execute.

Then you need people who keep the team together, who have a sense of who needs what to operate well, and who are that social glue. Humans are still social beings, and that social glue in a team matters a lot.

Where I've seen it fail is when you have just one type of person inside the team. I don't think it becomes that productive to put a bunch of AI superstars all together in a room without the execution machine and without the social glue. I don't think you necessarily get the same result.

I'm a big believer in building teams with diverse, complementary skill sets.

So you can't just buy the Galácticos?

I don't think you need to. You really have to be thoughtful about putting people in a group.

The other thing that helps a lot is for the team to have focus. If it goes in all sorts of different directions, you'll lose that power that you get from people working together. Having a lot of clarity—what's the North Star, what's the goal, where are we going—even if over time that needs to change, that level of clarity is required for everyone to be working in the same direction.

Can I be so blunt? If you don't need to buy the Galácticos, why do you have an Andrew Tull, a Daniel Gross, an Alex Wang, and the Galácticos assembling? Is that wrong?

You do need a few of these uber-talents on the team. There's a relatively small number of people who just understand this technology very deeply. You do need some of this talent, and if you can afford it, you should get some of that talent.

But you don't need all of your team to be that way. You need a team with complementary skills as well.

Does that create a good team? If I gave you $10 billion to go build a team and you could buy a couple of these luxury star players...

I feel like it's Top Trumps cards for sports teams: you can buy a couple. Does that create a good team when one is a $3 billion person and the rest are just average $50 million people?

Yeah, I wouldn't say no if someone offers me the opportunity to hire. There's definitely some really talented people in the field, and they deserve to be fairly compensated. This technology is going to probably make a lot of people very rich and have major effects in terms of society, and so we should be rewarding the talent.

But I'd be very thoughtful about what teams I put together and how they work together, rather than just hiring a roster of superstars without being thoughtful about how they're going to work together.

It's so funny, because of the impact that you can have in these teams, the multibillion-dollar price tags that you see can even be justified.

Time will tell. I don't think it's necessarily needed to go at that scale, but time will tell.

If I gave you $10 billion, what would you spend it on first?

One of the things you need is a balance between talent and compute. If you have too much talent and not enough compute, you're wasting your time. So usually, there's an equilibrium between those 2 pieces.

I think we often underestimate the importance of data, and data is getting more and more expensive, so I would certainly spend a good chunk of it on data as well.

So many things to unpack there. Do you feel you have sufficient compute today?

I think we are reasonably well-resourced in terms of compute and in building the models that we want to build. Yeah, access is not a massive problem.

No. Okay, great. Why is data becoming more expensive?

Data comes in different forms. On the one hand, the days of having data labelers who can say, “This is a cat and this is a dog,” are somewhat over. The easy tasks are things AI can do, so we're getting into a space where we need more specialized tasks.

Imagine you're building AI for enterprise. There's a particular business logic, and you need to make sure that you're catching the errors. You're going to need someone with a deeper understanding of the tools. So that's more expensive talent to come in and actually prepare the data.

There's also a lot of data that's synthetic. When you're building agents, you need to build environments. To build environments, you need some pretty creative folks who are going to build you synthetic simulators.

We've seen this on the robot side for many years, with people building robot simulators. Now you're building AI for enterprise, so you need to think about how you're going to simulate these work processes in a reasonably realistic way that the AI can train on. That generation of environments, benchmarks, and dynamic domains can be pretty expensive too.

When you look at the expansive data, and then you said, “Cat, dog, lamppost,” you've got these CAPTCHAs.

I get them wrong.

I legitimately get them wrong. I'm like, “Jesus, it's getting harder.”

They're getting so hard. They are. The other day, I called up my CFO and said, “I failed the reCAPTCHA. I'm so sorry. I'll try again in half an hour.”

Please let my AI agent answer that one for me. It's embarrassing. But the question that I have is, when you look at (likely Mercor), (likely Surge), and (likely Turing), how do you evaluate that market, which is providing a lot of that talent?

Is that an ongoing, enduring market, or is that just, “Hey, for the next 3 to 5 years, we'll need it in the training phase of these models, but I don't know what it looks like beyond that”?

I don't think it's a phase, in the sense that I do think this partnership—we'll call it—between humans and machines, where humans provide guidance to machines, is here for a long time. What will change is the nature of the information that the AI provides versus the information that the humans must provide as a complement.

Some of these firms may not be around in 5 years, but this notion of having humans guide and train the behavior of AI systems is here to stay.

It's super interesting. As an investor in one of them, I see all of them converge around needing to do 3 things now. They used to just be talent acquisition: “Oh, we'll get you these people.” Now they're like, “We'll get you these people and we'll get you high-quality data that you can really use.”

And now it's like, “Oh, shit, we need this third pillar, which is, we'll also help you implement that data into your models, do training, and help you with benchmarking.” Now they need all 3.

Are you seeing that third one, where it's the implementation of their data as well? They don't just hand it over the fence.

8. Synthetic Data and Model Degradation

There's definitely some of that happening. I think, for me, the even bigger trend we're seeing is the move from just labeling data to crafting environments to produce new tasks.

You said about synthetic data and that also being a very important segment to consider. Do you get model degradation when you get this kind of reinforcing loop of models learning on synthetic data, which creates more synthetic data? Does it actually degrade, or does it improve?

It really depends on how you're generating your synthetic data. In some domains—if you think of images, language, or LLMs talking to each other—at some point, you definitely get degradation. That degradation is due to essentially a loss of diversity in your data.

You can make an analogy: you take a bunch of people, put them on an island, and let them reproduce. At some point, the genetic diversity is going to keep shrinking. You get a reasonably similar phenomenon with models because you're not injecting diversity into the data. So there are domains where a lack of diversity means you get a collapse of the distribution.

There are other domains where you don't need diversity. If you think of playing chess or playing Go, these kinds of games, we know exactly how to generate board configurations. We can generate tons of synthetic data—not endless, because it's a closed world, but still tons of synthetic data—and through that, learn for a long time.

Then there are domains that are sort of in between. If I think of coding, we can generate synthetic code. You take normal code, and we know how to inject diversity into the code: I can take a couple of repositories, mix and match, apply an LLM to transform it, and so there's a way to generate synthetic data.

The language is predictable enough, and there's enough structure that I also know how to inject diversity so that you don't get that collapse. The hope is that, especially in these domains, we can use a lot more synthetic data and do it without suffering from degradation of performance.

9. Why AI Coding is Akin to Image Generation in 2015

Do you worry that we are creating a world with just much worse code? A lot of people are concerned about the quality of the code that's being output and about how we're relying on it so haphazardly. Do you worry about that?

Let me make an analogy in terms of the quality of generation. You ask about code generation, but let me take you back to 2015 and image generation. I don't know if you have it in your mind, but the quality of the images that were generated—we had image-generation models in 2015—was really bad. The resolution was bad, the composition was bad, and so on.

From 2015 to about 2022 or so, we saw huge progress in terms of the quality of image generation. So if you think of code generation, right now we're in the phase we were in for image generation 10 years ago. Yes, there's a lot of bad code that's getting generated. There's a lot of code that will get thrown away. But wait another 10 years, and I think the quality of the code that's produced is going to be excellent.

What will the developer world look like in 10 years when that is the case?

Well, if I carry my analogy further, I don't know if it's a reassuring scenario, because if we look at where we are today in terms of image generation, the volume of images getting generated is huge. What matters now is picking the quality out of the volume.

If I fast-forward 10 years on code generation, when we have the ability to generate a ton of code to do a ton of different things, we're going to need some selection mechanism to decide what code we actually want, where there's actually value. That's going to come.

There's still going to be some sort of editorial design choice. Someone needs to decide, of all the code we can generate, what's the code we want to generate? What do we need to be running in terms of our digital world?

So it's like a chief curation artist.

Yes.

With AI generation, curation doesn't go away. Curation, verification—this is work that doesn't go away.

Does the structure of teams fundamentally change, then? It's funny, kind of playing that back to you, and also playing back to you what you said earlier about the human and AI: if that is the case, there's not much of a partnership, is there, between humans and AI? It's a chief curation person sitting on top of a huge amount of artificially created code.

Well, that's your 10x productivity improvement there.

It is. You're ticking that box. It removes the human.

You still need people with intent. That's one thing that you need to decide: what do you want to build, and what purpose does it serve?

And so that intent is still there. That role of critique is still there. The team composition does change significantly once you suddenly have designers who, in their hands, have amazing tools to go directly from the ideas in their heads to the digital world, maybe eventually to the physical world. That equation definitely changes.

Do you think prompts, and the way that we interact today with prompts and with ChatGPT, are the enduring interface for human engagement with AI?

It’s awfully limited. Prompts can mean a few different things, but the idea of typing in a box, to me, is very limited, and we’re already going to break out of that box. We’re seeing a lot of cases where voice is a much more natural interface. I do expect we’ll see gesture, eye gaze, and these kinds of much more multimodal ways to interact with AI rather than just sticking in that prompt box.

But language is incredibly powerful. If you think of a prompt as language, as a way to express ideas and communicate with a machine, that’s a powerful paradigm. As humans, so much of our communication is based on language. I don’t think we’re going to move away from that because it encodes information. Language and words are symbols that encode so much information so efficiently, and I don’t think we’re close to getting away from that.

It’s funny, this conversation has changed a lot of previously held assumptions for me. When you think about what you did believe that you’ve now changed your mind on, what’s most prescient? I’m genuinely curious to know.

I’m a scientist who is happy to be proven wrong at any time, as long as there’s new evidence. Other scientists are much more likely to hold on to very strong convictions. I have weak conviction, but very strong respect for the scientific method and rigor—experimental rigor and theoretical rigor as well.

There are a ton of things. I used to be quite skeptical that neural networks were necessarily the ultimate solution to machine learning. I’d seen enough cycles of neural networks peaking and then becoming less useful. I used to think that every time you changed the scale of the data—from hundreds of examples to thousands, to hundreds of thousands, to millions of examples—neural networks were the first thing we tried because they’re a universal function approximator, and then something else would come out that was better. That was true for the previous generations. Some of you may remember support vector machines as being better than neural networks in the early 2000s.

I seem to be quite wrong on this one. Neural networks seem to be here to stay, and the ability to do backpropagation and gradient descent and all that seems to be a really powerful way to learn.

What does everyone else believe quite strongly that you think they’re quite wrong on?

I don’t have a lot of patience, as a scientist, for people who are predicting extreme scenarios—whether it’s the catastrophic risks of AI or the winner-takes-all, AI-becomes-our-overlord kind of scenario. I don’t have a lot of patience for that. I wouldn’t say it’s necessarily widespread, but I think you lack scientific rigor to analyze these kinds of scenarios.

I’m much more pragmatic and grounded. I’m pro-innovation. I’m excited to see where AI is going and the problems it can solve, but I’m not so interested in going around and making up science-fiction scenarios.

You’ve been on the most incredible journey. You said there about image generation in 2015 and partly how much it’s improved. We’re seeing this unbelievable capital supply go into the space in a way that we haven’t seen for many years. Is it a good bubble, where we’re getting incredible improvements and fundamentally advancing technology, or is it a bad bubble, where costs are becoming too exorbitant, teams are too difficult to build, and compute is too difficult? Is it a good bubble or a bad bubble?

I think about it as a bubble with bigger variance. The upswing is going to be bigger, and there are going to be big downswings as well. There’s a lot of variance in the system right now.

As long as people have a tolerance for risk, I think AI is a great investment. We should continue to support risk-taking, new enterprises, and new ideas. There are a ton of exciting new startups being created, and we should continue to support them. You just have to be tolerant of risk.

I’ve had some people on the show suggest that evals are, to put it delicately, bullshit, and that they don’t actually mean anything anymore. Humanity’s Last Exam—what does that really even mean? We have these new tests that come up, and leaderboards—what is this? Is that fair, or do you think they actually serve a very effective utility to the ecosystem?

I do think they’re really good indicators. You do need to take evaluation seriously in terms of knowledge, but you shouldn’t take it seriously in terms of the ultimate goals. There are lots of different benchmarks and so on. You have to decide: What type of model are you building? What are the characteristics of your system? Then think of evaluations as unit tests for the performance of your system.

Software engineers will know what that is, right? You run through that evaluation, and that gives you a signal of how the system is doing in a particular dimension. But as we’re building systems that are more and more general, you don’t optimize for these, right? We build AI systems that go into enterprises. None of our clients ask, “Are you able to win the Math Olympiad with this model?” That’s not what they care about. They care about bringing value to their business.

We’re curious to know how well we do on math problems because it can be predictive of behavior on other things, but you don’t obsess over specific benchmarks. You look at the return on investment in terms of what you’re trying to build.

Can I ask—we mentioned access for enterprises. Enterprises have money, and that’s a great luxury in a lot of cases. Research institutes and universities often don’t. With these bubble-like tendencies, people with money are able to afford the compute and the talent. Are we seeing this lack of access or democratization for great educational institutions that now can’t afford to compete in this new world?

Certainly, a lot of universities have a lot fewer resources than companies today. That’s not completely new. When I joined Meta in 2017, one of the reasons I did that was because I could already see the disparity in terms of access to compute, and I was really curious to see how you could do research with a lot more compute.

There’s still amazing research being done in universities. You go to the major international conferences—NeurIPS, ICML, and others—and often the best paper awards are actually won by researchers from universities. There are a lot of good ideas that you need to test out at small scale. In a university, you have a lot more freedom to pick pretty risky ideas at a small scale. No one is asking you to justify your research in ways that often happen in companies.

I think they play different roles in the ecosystem. What’s especially good is that talent flows between them. University students come in, do internships, and take jobs at companies. We’ve also seen a movement of people coming out of these large companies, going back to universities, teaching, and sharing with the next generation what they’ve learned.

How important is it to have seen success, and how valuable does that make you? When you look at people like (likely Mira Murati) raising $2 billion out of $10 billion, it’s like, “Well, no one’s seen the success that she’s seen with OpenAI, so it’s valid.” Help me out as an investor. Is it valid to place that much of a premium on access to people who have seen it at that level, or is that slightly overpricing it?

In many cases, when it comes to deciding where to invest very early on, when you don’t have tangible information, you look at people’s track record. There’s a part of that that’s about what they’ve learned in terms of the core recipe. But the other thing is also the achievement of having put together amazing teams who are building world-class models, and there’s a lot of subtlety to that.

I think both of these ingredients are important to consider.

10. If Joelle Was a VC Where Would She Invest?

If you were investing today and you were joining my team, which category would you most like to invest in—security, generative AI, compliance, you name it?

There are a lot of verticals, whether healthcare or scientific discovery, that I think have incredible promise. We’re going to see real, tangible progress within 5 years that is going to completely change the face of what we can do. That’s probably where I’d push.

11. Quick-Fire Round: Lessons from Zuck, Biggest Mindset Shift

That’s very exciting on the healthcare front in particular, when you think about that timeline as well. I’d love to do a quick-fire round with you, if that’s okay. I’ll say a short statement: What would you most like to do but, because of technical or financial limitations, you’re not able to?

I’m super keen to figure out how we build societies of AI agents. We’re doing it implicitly, but how do we look at populations of AI agents interacting together and have a sandbox for doing that? Maybe that’s something I’ll get to do.

Is it a lack of time, resources, or something else? There's just a ton of different things to do, but I'm keen to see what happens there. When you think about that ecosystem of agents, you have children.

Yes.

How does AI impact social friendship and connection? Do you think

It definitely does. There's a sense that we spend a lot of our time in the digital world. When I look at 2 of my children, they spend a lot of time in the digital world playing online games with their friends. It's still very social. There must be some AI, there's the digital platform, but it's still a very social experience.

Others have more individual experiences. There's definitely a shift in the time we spend towards that platform where we go look for that social element.

Knowing what you know, what do you not let your children do?

Eat too much sugar.

Totally. That's the physical diet. Completely agree with that. Is there a technical diet?

I spend some time discussing settings. You get an Instagram account. Great, you can have an Instagram account, but what are the settings on that account? Making sure they understand.

I mean, they'll go and change them, won't they? We're going to discuss settings. I know that was not a popular one. Do they listen?

The thing with children is you don't know until later.

Do you limit screen time?

I spent a lot of energy, especially in their younger years, limiting screen time. My kids did not have a cell phone until they were 14 or 15.

Did you see Adolescence?

I have not.

Okay. Watch it. It's fascinating. Basically, a little boy goes up to his bedroom and gets lost down rabbit holes of TikTok and Reddit.

It does not turn out well.

I've heard about it. I just haven't had time to sit down and watch it. Do you worry about the loneliness pandemic and the mental health crisis that we have?

I do worry a lot in general about making sure that people are mentally healthy. I think we have to be careful about taking shortcuts and saying, because suddenly we have certain platforms, we have AI, and so on, that is causing mental illness.

There are a number of people who are suffering, and they deserve to have good answers to the situation. They deserve—we deserve—to find real solutions to that. There's a lot of people looking for shortcuts and short answers, but I think more research into that is definitely warranted.

What's your biggest lesson from working with Zach?

He is incredibly deep into understanding the work. He does not coast. When he started getting into AI, the depth of the questions that he'd ask was incredible. He just gets really interested in the topic and goes super deep, and that then informs everything he does after.

You can have the most amazing team, but as a leader, you need to go deep and understand the work.

Did you see him change?

As anyone gets more knowledgeable about a topic, they get more decisive. There's a phase where you're really learning and trying to understand, and there's a phase where you understand a lot of things and then make your decisions faster. So certainly, that shift happened.

What 1 AI buzzword would you ban if you had a magic wand?

Existential risk.

Why?

Because it just makes people afraid. It's not out of fear that we do our best work and make good decisions.

Do you find the cost of talent acquisition prohibitive?

Talent is costly. Talented people deserve to be paid well. Someone coming in just because of money rarely is going to be the right person. But you do need to compensate people fairly.

Totally get that. Final one. What are you most excited for? You don't like existential risk. I don't like doomsday planning. When you think about the positivity that can come, what are you most excited for when you look forward to the next 3 to 5 years?

I do think some of the work in terms of AI for scientific discovery is going to be pretty fascinating to see, just in terms of the doors it's going to open up—the ability to explore the combinatorial space of solutions. So I'm curious about that. And then I'm super curious to see: how can we actually make our models more efficient?

There are larger and larger and larger models. No one wants to run these models. I spent a lot of my career building open-source models. I'll give you 1 example. We were in the frenzy of large language models, and I pulled the stats on the most downloaded models of last month.

We had a model like RoBERTa from 2019, a small language model, getting 20 million downloads a month. People want efficient models that they can use, that they can run. So I'm also super keen to see what we're going to be able to do at a scale that runs on 1 or 2 GPUs.

Final, final one. You said there about being open. We seem to be reverting to a closed world now. Is that the world we should predict and plan on?

That's a mistake. That's a deep mistake. I will continue to believe that, especially for research, the ideas need to circulate, and this thought that you can just close us down was absolutely false. People are circulating.

Do you not think we are, though, moving into that world? Everyone seems to be closing systems and closing access.

There are definitely a number of places where people are closing down access. I don't think that is going to be effective. Ideas will circulate, and I also think it's a mistake from a point of view of fostering innovation.

This has been such a joy. I've learned so much from this conversation. Thank you so much for putting up with my very basic questions, but I've loved having you on the show.

My pleasure. Thank you.