当 AI 发现下一个 Transformer——Robert Lange

I think a lot of analogies from evolution transfer to scientific research, right? In the sense that we traverse a tree of different ideas or different experiments, and then in the paper we report 1 path through that tree.

When we run LLMs autonomously—

Yeah.

—they tend to just—nothing interesting happens.

But oftentimes, innovation for a specific problem might require first inventing a different problem, right?

Automatically coming up with this reduction—or, let's say, this recursive nature of problem-solving—is something these systems right now don't necessarily have built in intrinsically, right? Oftentimes, it's easier to generate a lot of solutions than to actually hard-verify them, right?

The reason why I'm not that worried yet about labor-market disruption is that I still believe deeply that humans are the source of deep understanding and creativity in the world. If I didn't believe that, I would be very worried.

So I think it's going to be an amplifier of these latent dimensions humans are great at, right?

And I think one of the Rubicon moments is when the new Transformers architecture, or something massive, is discovered by AI and we're all using it. Nvidia GTC starts Monday in San Jose and it's free to attend virtually online. There's already been a leak this week of something called Nemo Claw, which is an open-source agent platform and if it's real, it could be one of the bigger announcements this year. So it's definitely worth watching Jensen's keynote for that alone. I'm giving away a DGX Spock. Nvidia just hiked the price $700. You probably heard about these memory shortages, right? So yeah, it's now $4700, which is very very expensive. And Merve from Hugging Face, by the way, she got one for her birthday and she said she literally cried. So it's a really cool bit of kit. If you register through my link in the description and you attend at least one session, then you are in the draw. This is a massive conference. Physical AI and robotics are going to be the breakout theme and Jensen does the keynote Monday at 11:00 a.m. Pacific. The link is in the description. Don't miss it. Robert Lange, it's amazing to have you on MLST.

Thank you, Tim. It's a pleasure to be back.

So, you're working for Sakana? Tell us about that.

Sakana is a Japanese AI startup working mostly on AI for Japan and, at the same time, exploring novel or ambitious ideas on the research side.

It's been around for over a year now. You're one of the founding researchers, right?

Exactly. Sakana has been around for almost 2 years now—1¾ years, I would say. It's fascinating to look back at the early days and see how much the company has changed organizationally. But in spirit, we're trying to embrace Ken Stanley's open-endedness idea and explore many different ideas that might not get resources right now in the ML community more generally.

We've got a few interviews coming out with Sakana that we filmed here in Japan. I won't spoil the surprise, but the CEO is David Ha. David, you know, there are these epic giants out there, like Clune and Stanley. David Ha is one of these people.

David's work has had a lot of influence on my personal PhD. He did a lot of fascinating work on hypernetworks and modulation in neural networks, as well as on evolutionary computation and evolutionary optimization. That also painted my path during the PhD.

You've released a paper called ShinkaEvolve, and we were just saying that it kind of means “evolve evolve,” because in Japanese, shinka means “evolve.” That's quite common. It's a common thing to do to have these multilingual, double namings in Japanese.

Just before we get there, we interviewed the AlphaEvolve team, and I also interviewed Jeremy Berman a few weeks ago. Your paper is very much like a more sophisticated version of those, in the sense that it's using language models to generate programs and taking an evolutionary approach: We generate the program, refine the generated program, and have an evaluator, and we do this over several steps. Your approach does many things that the other ones don't do. Tell me about the paper.

First off, of course, this was partially inspired by AlphaEvolve. I think it's great work. I know Alex and Matei, and I think they were doing incredible science.

One thing that's important about using all of these evolutionary, LLM-driven methods is sample efficiency, right? Many of these systems sample, let's say, 1,000 programs for a given task. What we tried to do with ShinkaEvolve was essentially cut down costs, as well as computation and evaluation time, by introducing a set of technical innovations to this evolutionary search.

We showed that it's possible, with very few program evaluations, to improve upon, for example, the circle-packing canonical result that they showed in their paper. More generally speaking, I think we're right now at a point—or an inflection point—where these evolutionary-driven LLM systems can really revolutionize scientific discovery. We hope to have made a step forward in making this more democratically accessible.

The code is open-source and available, and because of its sample-efficient nature, we hope that many people can interact with the system and make their own scientific discoveries as well.

Yeah, that's actually a really important point, because I suppose we can use these foundation models. First of all, isn't it fascinating to reflect that we have these amazing models out there that we can access, such as GPT-5 and Grok 4, and they are so much better when you get them to refine their solution in several steps? Why is that? I suppose a naive question would be: Why aren't they just good out of the box?

Potentially, with enough random samples, right? It's sort of this monkey typing on a keyboard. They would potentially be able to get there, right?

But in principle, it's coming back to the principles of evolution, right? In the sense that you need to collect a bunch of stepping stones first and then build on top of them to really find innovations or tune innovations down the line. I think language models with the right evolutionary harness are extremely powerful in terms of scaling up to make discoveries.

I think Jeremy's work, as well as the AlphaEvolve paper and work we've done on the DeepMind Go machine, for example, show that this stepping-stone accumulation, plus iterative verification and collecting information and evidence from a real-world or synthetic evaluator, is really important for that.

Very cool. Stepping-stone collection came from Ken Stanley. It's a wonderful paper, “Why Greatness Cannot Be Planned.” He said that it's better to have systems that don't converge. In natural evolution, we're just trying all these different things, and greatness quite often follows a diverse path, which means you have to do things that initially seem quite stupid, and then later they turn out to be incredibly useful.

We're trying to design algorithms that can allow for a population of slightly weird things, and then we kind of lock in and converge a little bit. So we're still converging, though. We're still building systems that don't diverge forever. What are we losing?

Yeah.

One thing I find extremely important after having done ShinkaEvolve is the problem of the problem, right? With all of these systems so far—maybe except for The AI Scientist, which we can also talk about—the problem is given, right? You have an evaluator, a correctness checker, and you sample programs only on that single problem, right?

Oftentimes, innovation for a specific problem might require first inventing a different problem, right? For example, I think in the matrix-multiplication result that the AlphaEvolve people show, you can recursively apply the algorithm to larger matrices. So it's actually an important result, right?

Automatically coming up with this reduction—or, let's say, the recursive nature of problem-solving—is something these systems right now don't necessarily have built into them intrinsically, right? Going forward, I think it's going to be really important not only to do open-ended optimization of solutions, but to co-evolve the problem and solution together in order to collect even more diverse stepping stones and really kick off this open-ended process.

To me, one of the big life goals or achievements I would want to see is a process that can run not only for, let's say, a week or many weeks, but potentially for years, collecting even more diverse, interesting stepping stones.

Yeah, I spoke to Joel Lehman, and he was talking about Knightian uncertainty, which is that machine-learning algorithms are very good with unknown unknowns. In a sense, the unknown unknown is talking about these stepping stones that might be useful later.

When we run these algorithms at the moment—the same is true with LLMs and reasoning systems—they're very, very good when we give them a specific thing. What you're pointing to is that we might need to invent new, unrelated problems and find solutions that might then be related to what we're trying to do.

So, that feels like a bit of a catch-22 situation, right? We're saying, circle packing: here's my evaluation function, and I want you to diversify and then converge toward the solution. I had the same thought with Genie, by the way. It gives you exactly what you ask for. You put a prompt in—“a Swiss lake with boats on the water and mountains on the side”—and I was thinking, where are the birds? I forgot to put birds in the prompt. So how can we meaningfully build systems that actually bring in other unknown things that might be useful?

I think one inspiration, or something I would personally want to research, is systems outlined in PowerPlay or POET by Jeff Clune and others. There is essentially a set of tasks and a solution generator, and both of them co-evolve in an almost auto-curriculum-like style. In POET, the natural first application was reinforcement learning, but I think this can now be broadened to science more generally, at least when there's a simulator available for running these evaluations.

By doing such a co-evolution, you always try to max out the capabilities of that generator while increasing this convex hull, or potentially even more diverse problems, while doing so.

I know there's always the leading thought that, even with POET—which was this thing where you had a population, a lot of environments and agents, and the environments were complexified—the agents would have an effective curriculum to learn things of increasing complexity. But even then, isn't there a kind of design bias in the system where there's some code somewhere that complexifies the environment step by step? Wouldn't that also just be designed by the human? So it would also just give you exactly what you ask for?

Ultimately, this comes down to the hypothesis that language models can potentially do extrapolation or interpolation. Even though these things might, in the end, be designed by humans, there are many unknown unknowns that we humans didn't think of while designing them. So potentially, it is possible for an LLM to find a novel discovery simply by us not having thought about it before.

When we run LLMs autonomously,

Robert Tjarko Lange

Yeah.

Depending on the prompt you give them, they tend to just go a few steps in that direction, and then no new, interesting novelty emerges. I think even if you wire them agentically with environmental feedback, they still seem quite parasitic on their starting conditions. With an LLM, could we build a system that actually adapted to novelty—one that could discover new things?

I think it really also depends on what you give the LLM as a starting point. For example, in ShinkaEvolve, we saw from time to time that if you give an initial solution program that's already pretty optimized for the problem at hand, you still get stuck in local optima, where not a lot of novelty is introduced. If you start from an impoverished solution, there's much more room for diversity.

I think this comes back to what I did before in my research, namely meta-learning. It's this classical trade-off where you can either start out from something very, let's say, unconstrained—from a very simple solution—and give much more room for the optimization. But this might actually require open-endedness and a long time to find a good solution.

Or you start out from something that's already very constrained by inductive biases, let's say, and then you might be much more efficient in terms of convergence, but you don't get the open-ended, big-novelty benefit from it.

Yes. I suppose where we want to get to is building systems that aren't designed by humans. For example, if I'm leveraging my deep understanding—LLMs are really good if you understand something deeply—and similarly, we could kick off ShinkaEvolve and put a starting solution in there that leverages my understanding.

We want to have AI systems that anyone could use, so a non-expert could say, “I want to solve this problem,” and it will solve the problem. We should talk about the evolutionary approach, right? To maintain diversity, you had a population of programs, and they were separated into islands. Tell me about that.

The way ShinkaEvolve, similar to AlphaEvolve, works is that you keep an archive, like a database, of programs, and then sample parent programs with a set of inspiration programs. You ask an LLM to make an improvement to that program: provide code to edit or rewrite an entire program, or potentially even cross over different programs. Then you query the LLM, get a program out, and evaluate it on the problem at hand. For example, you might increase the sum of the radii of a bunch of circles in a square.

You run this each time, collecting evidence from the evaluator, adding it to the database, and repeating the process. You don't do this sequentially; you do it in parallel for many different programs. Each time a program is added, you essentially try to diffuse the knowledge collected by that program across the entire database.

One way to think about this is that you have a tree where each node represents a program, and then you branch off of it based on the parent nodes. Interestingly, these approaches do tend to scale, but ideally we can make the scaling happen at a faster rate. This is something we tried in ShinkaEvolve by making a bunch of innovations, including model ensembling. We're not using just Gemini; we're using all frontier model providers and figuring out a smart way to use each model for a given parent.

If you have a certain program, in some situations it might be better to use a GPT model; in other settings, it might be better to use a Gemini model. We introduced an adaptive prioritization scheme that can adapt the evolutionary algorithm on the fly while running the algorithm.

This also comes back to the naming, right? Shinka kind of means “evolve,” so this evolutionary algorithm that we apply using LLMs also co-evolves at the same time while we optimize the programs.

And on this circle-packing problem, you had this plot showing how it converged, and it seemed to converge quite quickly. We'll show the plot on the screen now. Very quickly, the performance jumped up, and then it slowly converged. You said in the paper that it was using, I think, 3 core innovations, and my thinking was: if you ran this 50 times, would it be the same every single time?

To what extent is it thinking outside the box? Sebastian Bubeck is always posting on Twitter, talking about how GPT-5 has just discovered new things, and there's always the question: is it just searching the internet? Is it just finding things that have been found before and combining things together in a new way? Could it really think outside the box?

Mhm. I think this is almost a subjective question. First off, I don't know all the problems on the internet that try to do circle packing. But what I can see in the tree that we also depict is, for example, a crossover operation between 2 programs, where different concepts are combined.

One important part is, for example, the initialization of the circles. Another is the optimization: basically, a constraint-optimization program is executed. Then the final part is a reheating stage, where noise is added and more strategy is squeezed out. To me, this propagation of information through the tree is really fascinating, where, in some sense, the stepping stones are actually used in a complementary fashion.

With regard to rerunning the program multiple times, of course, there's some stochasticity in it. We're using language models, and due to queuing and device scheduling on the server side, we can't get rid of all the noise. We've seen that, at least for the general quality of the solution—the result that is arrived at afterward—it is possible to re-obtain this, but sometimes with a different program, or most of the time just due to stochasticity.

It's not as if, for many problems, there's 1 solution that achieves that score; there's a spectrum, or a region, let's say, in program space that resembles the same thing. I think one thing that was very interesting about the circle-packing problem, also coming back to the problem I discussed initially, was that originally we used a formulation where correctness was checked with a very tiny amount of slack. The circles could overlap a tiny little bit, and afterward we reduced the radii, and the solution was exact.

This didn't change the score by too much, so it's still state-of-the-art, but it was essentially a proxy problem.

We then re-ran ShinkaEvolve on the exact setting, and we found that it took a little bit longer to actually obtain the same quality of a solution. So, I think this already points a little bit in the direction of what I discussed in the beginning. Sometimes surrogate problems might actually be extremely valuable in making such discoveries, and having an automated way of designing these surrogate problems efficiently might be something really important going forward.

Yeah, that's absolutely fascinating. It reminds me of support vector machines, where we make the optimization tractable by introducing slack variables, and you can think of that as a kind of surrogate problem. But then I'm thinking: would ShinkaEvolve or AlphaEvolve know to introduce a surrogate problem? As designers who understand the problem, we can think outside the box and do stuff like that. Presumably, if the fitness function had the constraint that there were no circle intersections, it wouldn't occur to the algorithm to come up with the surrogate problem.

Exactly. Yeah, this is a big limitation right now. At this current point in time, we take the problem to be fixed, and we optimize for that problem. But when you think about humans, we're really good at inventing our own problems, or reformulating the problem so that we can actually work with it.

I think a lot of the innovations in mathematics come from taking a very different perspective on a problem—taking number theory and applying it to linear algebra, or the other way around. Right now, these systems are not yet at the point of achieving that level of transfer.

Yes, and it reminded me: I spoke to Leon about this. You've got this Sudoku benchmark. A lot of folks watch the Cracking the Cryptic YouTube channel, and that's exactly what they do. They invent new problems based on abstractions that capture the essence or aspects of the problem you're solving.

Then they do something similar to ShinkaEvolve. They do this kind of evolution where they take these different solutions, combine the best aspects of both of them, and forge a divergent path to a new solution.

Robert Tjarko Lange

Yeah.

And that seems to be the essence of what we need to do.

Robert Tjarko Lange

Yeah, for sure. There is some work also by Jeff Clune, Shangwen Yu, and Song Lu on automated capability discovery. They look at language models that generate tasks, but it's unstructured in the sense that it's not done in order to enable the solution to one target problem.

I think making these connections is going to be very fruitful down the line.

Very cool. Now, the other thing—we'll show the graph on the screen, the evolutionary graph—for the circle-packing problem, I was looking at that, and first of all, it looked incredibly parsimonious, which is good. It looked like it had found an optimal path to the solution very quickly.

I was thinking in my mind, maybe there's some natural pattern, something about that which we could use in the abstract to guide the evolution in the future. But the other thing I'm thinking about is that, right now, the problem with machine learning is that we don't really have semantics baked in.

What we're doing is using a verifier, looking at the reward, and doing patterned exploration. We're taking steps toward the target. I love mechanistic forms of reasoning where we actually know something about what the program components mean.

The reason this is important is that when we're merging together the best-performing programs from 2 different islands, that's a kind of first-order interaction. Does it make sense to merge them together? It's wonderful that, with LLMs, you can give them any pair of programs and they will find a way to merge them together. But wouldn't a more principled way be to say that there are some semantic primitives here and we know they fit together?

There's this LEGO analogy: we're building up based on principles rather than forging a path based on performance.

Yeah, that's a good point. One thing we do in ShinkaEvolve is keep essentially a scratchpad. Each program is summarized, and from the program summaries we keep a set of global insights that were shared or extracted from these programs. Then, based on the scratchpad, we construct meta-recommendations that become part of the system prompt.

That way, you can try to semantically grasp some of the discoveries. But a general problem, which is again task-dependent, is that you thereby diffuse that knowledge across the tree. Sometimes you want things to be much more isolated.

It's always a trade-off where you somehow have to find, for your problem, the right position on the spectrum: how much knowledge diffusion do you want to have, and how many hard islands of programs do you want to have? We're trying to take steps in the direction of automatically adjusting this in an optimal way, but again, it's very program-sensitive.

Another point that you're already going into is Jeremy's solution to ARC-AGI, doing solution evolution in the instruction space instead of the program space. I do think that this is something important, and we're trying, with the construction of this meta-scratchpad, to do both at the same time.

Again, it's problem-dependent. I played around a little bit with ARC-AGI-1 and ARC-AGI-2. On ARC-AGI-1, I think the transform-program direction is actually quite effective. As Jeremy said, it's deterministic, and it's easier to get a clear signal to improve on during your evolution process.

On ARC-AGI-2, this whole semantic evolution seems to be more efficient. Ideally, we can get a system that can automatically decide whether it wants to take a programmatic approach in settings where that's feasible and easier to bootstrap from, or whether it takes the semantic approach of evolving instructions or LLM-driven input-output mappings.

Yeah, it's so interesting because a symbolic AI person would say, “I don't like connectionism because it doesn't understand. The only semantics in connectionism is this notion of similarity; it doesn't really understand things.”

They would say, “Just start with an entity-relationship graph and build up using composition and first principles.” But that doesn't work, right? We're using neural networks because they're incredibly flexible and understand a lot of things about the world, but they don't have the kind of constraints that we want.

So, we use these tricks. Jeremy evolved program descriptions. In your program selection, you had semantic novelty detection using embedding-based similarity.

Yeah, embedding-based similarity.

Yeah, so you had a kind of self-similarity matrix based on the cosine similarities. And indeed, you've got this meta-scratchpad. What we're seeing is this fascinating spectrum of possibilities where, still using neural networks, you can imbue semantics using all of these different tricks, but they all come with trade-offs.

Yeah, for sure. I think it's kind of interesting. We've had a long period of computer science where algorithms were designed by humans. Then we had this Andrej Karpathy Software 2.0 paradigm, where we had neural networks that performed a certain function.

Now we're at this point where we're using LLMs to design algorithms or solutions more generally. Even though large frontier language models are extreme black boxes, or it's very hard to get a full mechanistic understanding of them, the outputs can be understood: the programs, the instructions, and so on.

I think it opens up a very new paradigm for doing research, or basically doing anything. But I think we're just at the starting point of figuring out the right user interface for that.

The other innovation in the paper was using UCB, which is upper confidence bound. It comes from the multi-armed bandit literature, which is this problem where you can pull these levers, and at the beginning you don't know which levers to pull. Over time, you reduce your uncertainty and can pull the ones that work, but there's this exploration-exploitation dilemma.

You've implemented that for figuring out which LLM to use. It could be Gemini, or it could be Grok-1 or something, to figure out which one to use.

We're using a model ensemble to propose program mutations. Intuitively, one could say that the best frontier model on SWE-bench is always the best mutation-proposal model. But in practice, that's not always the case.

In general, it's extremely hard in this evolutionary setting to assign clear credit to a single model. For example, one improvement is implemented by GPT-5, and then the next one is implemented by Sonnet 4.5. It's unclear whether the performance gain from the second mutation actually originated from GPT-5 collecting the first stepping stone or from Sonnet 4.5.

Instead of uniformly sampling models, we implement a bandit-based approach where each model is one arm of a bandit. We look at how often each model improves the performance of a parent node by creating a mutation. We then adjust the posterior probability to first explore all arms once and subsequently change over time to prefer models that have yielded improvements for similar nodes.

The great thing about using a UCB-like algorithm is that it actually has a theoretical regret, which means it’s only logarithmically worse than the optimal switching path, if that makes sense. But if I understand correctly, UCB is based on a global rating, like a mean score for every single LLM. I think what we want is more of a contextual switching decision, which means we know that for this particular program, Gemini is better. Do I understand correctly that, at the moment, it might converge to a single frontier model, and then in a nuanced situation, we might still get the wrong model?

In general, there is some probability allocated to all models, right? It’s not like it can just pick one model and then stop using the others. There’s still a chance for open-endedness and serendipity, if you will. In general, for the problems we considered, we haven’t seen one model clearly dominate all the others. We’ve seen that it really depends on the course of the evolutionary process which model is better, and UCB, or the bandit approach that we take, dynamically adjusts this in an efficient way.

Would it be possible in the future to use an LLM to make this judgment?

Potentially. In some sense, in that case, you think of the LLM as a surrogate model, right? You can think of a Gaussian process as a surrogate regression model, and there has been some work showing that language models can act as surrogate models. The real question to me is: how do you represent the information to the LLM?

If you use the raw programs and their fitness evaluations, you quickly run out of context. You need some amount of compression in order to present the information in the right way to the LLM to prioritize the models.

I hadn’t appreciated how long the context is. When I was thinking, could we use an 8-billion-parameter Llama model while doing active fine-tuning? We could say, “I just ran this program on Grok, and it got this score.” Then, over time, for the given run of this evolution, it would know that Grok is good at these problems.

Potentially. I’m not sure how efficient this fine-tuning is if we’re only evaluating 150 programs, but in principle, one could imagine it. I think it’s on the engineering side, not necessarily the prettiest thing to do. It could in fact happen.

But I think for all of these things, we started out with the most intuitive algorithmic component that we had, and UCB was one that really did the job here. Much credit to Eduardo Santana, who introduced us to Shinka.

Let’s talk about the diffs and the mutations. We generate programs, and I think you folks were inspired a bit by AlphaEvolve. They had this gating where you gate the part of the code that is mutable. Tell me about all of that.

A program is just a long string. In order to make sure that certain parts that are essential to the evaluation—for example, the imports and so on—are not deleted by the LLM mutations, there are so-called markers that state which parts of the code are mutable and evolvable. It’s easy to programmatically make them immutable when you get a diff proposal, and those parts will not be changed. Only the rest of the code snippet will be changed.

We implement a type of rejection sampling with reflection, where if an LLM happens to try to mutate this part, the proposal is rejected and we resample a new one. Thereby, we can somewhat mitigate certain security or safety problems and get a robust mutation.

One of the bigger questions is how you can turn this from a single-file mutation setup into a multi-file mutation setup, working on entire codebases. In principle, you can represent many codebases in a single file, but the hierarchical structure might actually be useful. There are some ideas from Aider, this coding tool, where you construct a repository map and have some level of abstraction, but they also come with positive and negative trade-offs.

I love Aider, by the way. It feels as though, in the future, code-generation systems will actually resemble Shinka overall. If you think about it, they’ll be using some kind of Git repository. Maybe Cursor already does this, because in Cursor you can restore previous checkpoints. But it can explore different branches and merge checkpoints together, and obviously, you just say in natural language what you want to do.

We didn’t talk about mutation, by the way. We just spoke about diffs, and there’s also an option to do a full-file rewrite. But there’s also this notion of crossover. How does that work?

A small innovation on top of AlphaEvolve, where I believe they only use this diff-based mutation, is that here we wanted to have more flexibility to entirely rewrite the program, to come up with a completely different stepping stone, if you will. Again, you can make part of the code mutable, but instead of proposing a patch to change certain parts of it, we essentially rewrite the entire program.

This is sometimes helpful. It’s not always a clear benefit, but it allows you to get more diversity into the search. This is one type of mutation alongside the diff- or patch-based approach.

The other one is a crossover mutation, where we sample not only a single parent program, but two different ones, and ask the system to make a complementary improvement. On some problems, this is really helpful, and on others, it’s not. In general, we found that having diversity in terms of operators is also helpful in discovering new things.

I wanted to follow up on the point you made before about this being a new paradigm. I think so, too. I’m really convinced. Right now, we’re at the beginning, where we still think a lot about the chat-assistant interface as the way we interact with LLMs, but it’s inherently single-threaded most of the time.

We’re sitting in front of the computer, interacting with the chat. We’re seeing changes as they occur in the editor, accepting them, and so on. But I think this is also just a stepping stone toward a more distributed way of thinking about research, optimization, and so on.

I like to think of vibe coding and vibe chatting. On the other hand, we have vibe optimization and vibe researching, where my ideal future scenario is one in which, during the day, you as a researcher could work with a system like ShinkaEvolve or The AI Scientist. You steer the ship like a shepherd, in some sense, and then during the night you press play and go to bed.

In the background, you have multiple experiments running, with new ones automatically being proposed by LLMs and evidence being accumulated. In the morning, you come back and have a multithreaded system running in parallel. You’re more like the shepherd of the ship than the person actually executing experiments and analyzing them. You’re still analyzing, but you’re not executing; that’s happening through the system itself.

Yes, and increasingly this might be semi-supervised or even proactive. There’s that new product from OpenAI where it knows what you’re interested in and, while you sleep, it goes off and finds your pulse. That’s right.

We’re in a situation now where we’re reasonably technical people. MATLAB and Mathematica are supremely powerful, but you need to know how to express problems precisely. I can imagine a future where we express problems just in natural language, or perhaps, based on our interactions with language models, the platform knows what we’re interested in and can go and find things on our behalf.

This is about democratizing the technology for people who perhaps don’t know exactly what they’re looking for.

I think one of the bigger problems there is the verification aspect. Oftentimes, it’s easier to generate a lot of solutions than it is to actually verify them rigorously. Language models are capable of doing soft verification, looking at code and latently running a stack trace of execution, but it’s not exact.

I think notions such as reward hacking, and not doing real discoveries but shortcutting them, are areas where we need to put more time and effort into figuring out how to make sure this actually moves in the right direction. I would hope that language models can eventually do this efficiently themselves, either by implementing it in code or by doing it latently.

But this is also part of the problem, right? It's not only coming up with the problem, but also with the automatic verification at the same point.

Isn't it a tantalizing idea that there are natural patterns in the world, and the building blocks to construct novel solutions are already there? And maybe they're there for a reason. Maybe they just reflect natural regularities in the universe, because there's always this question: intelligence is about adapting to novelty.

The world is always changing, and the world tomorrow will have things that we can't explain with our knowledge today. But we do have abstract knowledge that could be easily recombined to explain the future, and LLMs might already have those building blocks.

For sure. I think, in some sense, the more you think about Occam's razor applying to everything in our world, whether it be language or science, it's pretty interesting, because these artifacts now go into our language models of today, and potentially there is some amount of this being captured.

I think, though, it might also be an inductive bias that leads to a local optimum at some point, right? You need more complexity, but I do think that with systems that do this evolutionary mutation-style approach, you might still push the system out of these local optima eventually.

Yes, and then there's also the notion of the importance of adaptivity. This is what Chollet says intelligence is. Since we've had these models that actually do adaptivity at inference time—things like test-time adaptive fine-tuning, reasoning models, and so on—they started getting nontrivial performance on ARC.

Now, it's very, very expensive to have adapting huge foundation models. It's just a practical concern where we haven't done that yet. But what we can do is build systems like ShinkaEvolve that leverage the best of both worlds. They leverage frozen foundation models, but they give you adaptivity.

The purpose of adaptivity is to respond to novelties, to create new building blocks, to synthesize new building blocks in this principled, tree-like structure that allows us to adapt to novelty. So we are having our cake and eating it.

I have to say, I found it very interesting that Jeremy, basically in your podcast, when you asked him about Shinka, was saying that he doesn't believe there are a lot of percentage points to be gained by using a system like Shinka, but you can make it much more efficient. That was the gist of his answer.

To me, once you've made it much more efficient, you can scale it up again, right? So if you essentially have a cheaper system that can generate many more instructions, I would expect that, by the nature of open-endedness, you might get some amount of improvement out of it.

Right now, I don't have any evidence for it. I would love to collect that evidence. It's again the magic of open-endedness that comes into play: as long as these training examples of ARC-AGI give you a good signal for your final test submission, you should be able to progress.

Yes, and that is a great segue, because on the circle-packing problem, it was so sample-efficient that in fewer than 200 interactions with an LLM, you converged on the solution. But I was thinking, great, but it's still quite dependent on the starting conditions. We talk about this design bias and so on, so what we put in is very important.

But now what we could do is scale out. We could run this 1,000 times, and we could have another process which prompts, generates, and breeds the starting conditions, because every time we run ShinkaEvolve, what it's doing is searching parts of the epistemic tree. What would happen if we just scaled that out massively?

We haven't tried, but you could even start with an empty program, right? That would be basically the same. Then you would branch off of that empty program, I would expect. We haven't done this simply out of cost and time reasons.

But I do think, in many ways, this is the question that will push us toward this true open-ended vision of running a system for a month or so, really trying to squeeze this out. I'm not sure if we're entirely there yet, but I will do my best to make sure that we will.

And the reason this is interesting is we know, as a practical matter, that we can't start with nothing. If we were just starting from the most primitive building blocks, the search space would just be huge and there'd be no learning signal. So we know we need to start a little way up the stack, but we can massively parallelize that.

Let's say we have 1,000 different instantiations of ShinkaEvolve. It doesn't have to be embarrassingly parallel; we could still have some sharing. During their execution, we could still have a little bit of crossover, and maybe then we could run all the ShinkaEvolve instantiations in a similar kind of meta-evolution loop.

My suspicion is, contrary to Jeremy, I agree with you: we know there are diverse stepping stones out there that could dramatically improve many of these solutions. We simply haven't scaled it up yet.

I also believe that using a system like ShinkaEvolve could automatically detect whether an instruction-based optimization approach for a given problem or a program-based approach is actually the right thing to do. Sometimes, potentially, it's even a mixture, right? There are some things you can probably articulate more easily in Python than you can articulate in language. I would be really interested in exploring that.

Yeah, I mean, you said earlier about Jeff Clune—what was Jeff Clune's paper? The thing that generates problems.

AI capability discovery.

I did speak to him about this at NeurIPS, but something like that could be fascinating as well, where we're also generating the problems and solutions and then kind of moving the back end.

I think the way this will land commercially is there'll be a new type of GPT where everyone is solving different types of problems, and the system—it'll be like a kind of ShinkaEvolve with a massively distributed version—where mathematicians are using the platform over here to solve this problem, and it will see commonalities and kind of link them together. Because you need to leverage human creativity in this process as well, I think.

The big challenge going forward is going to be: how do we change our incentive system for this to actually scale? I think, for example, some amount of economy will be needed, or some amount of mechanism design, in order to make sure that everyone is still happy to engage in it.

So maybe we're going to have many more leaderboards for whatever is numerically scorable. And I think this will be really interesting to see how compute, these automated agents, and human shepherding and steering will ultimately change and revolutionize science and, I guess, society more generally.

And Rob, looking at the future, we've got a load of people in San Francisco that want to scale language models, and they are adding in implicit forms of adaptivity and composition, so they're building controllers and doing reinforcement learning with verifiable feedback and so on.

I think that you subscribe to the slightly different idea that we need to be far more open-ended and use evolutionary algorithms and so on. But do you think that they're on a path to nowhere? Do you think they might change tack? I mean, where is this going?

I actually think that these things can be complementary. Let's say you find your own model to be a circle-packing expert. I do believe that mixing in different RL fine-tuned models into the ensemble of models, and then having a good way to adaptively select which model to use, is not a bad idea.

To me, I fully subscribe to this philosophy of open-endedness, and reading Ken Stanley's and Joel Lehman's book was really a fundamental moment in my life. I want to see how far we can push this, and I think we're not yet at a convergence where either the capabilities of the models, the way we scaffold around them, or the way we humans interface with them have converged.

To me, there are really these 3 points: model capability, model scaffolding, and the user interface. We still have a lot to push on all 3 angles.

Beautiful. The only thing we didn't talk about was that we spoke about the circle-packing problem, but you also applied it to a few other things. Can you tell us about that?

So one thing we did was use a framework called ADAS—Automated Design of Agentic Systems—where basically, instead of manually writing an agent scaffold, you use an LLM to write agent scaffolds for a specific task.

We looked at mathematics tasks, such as AIME, and we used ShinkaEvolve to evolve, basically, an agent—using an agent to evolve an agent. We found that we could dramatically improve the performance of very cheap models like GPT-4.1 Nano, but the agent scaffold was actually able to either generalize to other language models or to different years of AIME. That was one application.

One important other application was AIDE Bench. AIDE Bench is work done by other folks at Sakana, including Yuki, who's also part of the paper. It considers heuristic programming contests previously run by AtCoder, which is a famous Japanese competitive programming organization, and we showed that ShinkaEvolve can also work very well as a co-scientist.

Basically, we took initial solutions obtained by an AI agent that had previously been designed, and then optimized on top of these initial solutions with ShinkaEvolve. We showed that on one of these programming tasks, if the combination of this agent and ShinkaEvolve had competed in the challenge, it would have ranked second place. So I think there's some evidence that ShinkaEvolve can work as a co-scientist, not only for LLM agents but potentially even for humans, like we discussed before.

Finally, the last application that we looked at was designing mixture-of-experts load-balancing loss functions. At Sakana, we've done some previous work called DiscoPOP. I think we discussed this during the last podcast, where we're using LLMs to design objective functions. Back then, we did it for preference optimization and post-training, and here we did it for load balancing in mixtures of experts.

There, too, we found that within even only 20 generations, we were able to explore not just a single objective function but a convex hull where there are different trade-offs between performance and load balancing, and so on. I think this is another application of ShinkaEvolve where it's not only about finding the best solution but essentially illuminating a program space where there are always potential trade-offs—for example, between runtime and the quality of the circle packing. Having a system that can explore all of these is important as well.

I'm very excited to see you apply this to the ARC challenge. What are your thoughts about that?

I still need to collect results, so I don't want to make any hard claims before having done this. But I would hope that there's some chance of improving the cost of these systems and then potentially even their performance. But, yeah, it's still to be seen.

Oh, very exciting. So you've done some experiments. Exciting news is potentially coming.

I've started looking into it.

What are your thoughts in general about ARC, though?

I think it's good. I think it's really important, and it fills an important gap. I deeply respect François and read the paper when it first came out. No one thought it would actually be possible to get numbers above 10%, right?

It's also pretty fascinating on a societal level how far we've come since then. Sometimes, while you're deep in battle mode or work mode, you can forget where you were 1 year ago. Then, just looking back, it's pretty amazing how far we've come since ARC-1.

Yes, and saying that, I think François doesn't get enough credit because it's such a good benchmark—not necessarily for the reasons people think. François is always saying that we need to have a benchmark that's easy for humans and hard for AIs. In a sense, that's not quite the case.

I said when ARC-2 came out that it's actually very difficult for humans. There was one task where Doug was stumped for about 15 minutes. There were 3 of us looking at it, and we just couldn't solve it. It's one of those things where, depending on your perspective, you might get it straight away or you might not. So there's that criticism. People have said that ARC-3 is even harder, but I think that's rather missing the point.

I think he's saying that with a lot of these competitive-coding problems, the data set is contaminated. These are problems that have been solved before, in part or in whole, which means that when you look at the epistemic tree, many of the building blocks for solving them are very high up in the tree.

He's looking at problems where there's very little data-set contamination, and they need to be solved from very abstract building blocks. You're starting much lower down the tree and synthesizing a model by composing very abstract building blocks together, which is the essence of intelligence. For that reason, I think ARC is really pushing us to build adaptive systems that we could say are intelligent.

Yeah, I agree. In many ways, I'm really looking forward to the next few years and seeing how far we can push this, and also how much generalization we can get afterward. When you look at the more recent models, they're getting much better at program-style code generation, or outputting code for ARC, than they are at the instruction-based level. I think this might already be a small sign of some amount of overtraining on ARC-AGI-1, at least.

I do believe there are some aspects of work that will be automated before we get to full science automation and the type of work I'm doing. But I could imagine that certain parts of the dimensions I deal with every day are for sure going to be hit by AI. Then the question is: Are there going to be new dimensions opened up that we as humans will fill in?

I think what I said before about shepherding and so on—I really hope that's the way forward. In the sense that humans are the ones steering the ship while being massively amplified in their productivity.

Right now, I'm not really seeing the kind of job-market disruption that was being predicted. I know from personal experience that, in a sense, it's made it very difficult to hire people. Scriptwriters use ChatGPT; I can spot it instantly. Writers and copy editors are actually in more demand than they were before, fixing all of the crap that has been generated with ChatGPT.

There's the cloud analogy as well. IT system administrators who were earning £60,000 a year in the UK rebranded themselves as cloud and DevOps engineers, and they more than doubled their pay. People are very adaptive. They see new trends and new bandwagons, and they just adapt and add value on top. That has been the trend for a very long time. Do you think AI is going to be so transformative that it will transcend people's ability to adapt?

I think it's just a question of speed and rate. I was talking about cultural evolution and technological evolution, and it seems like we humans need more adaptation and more time to get used to the technology, to carve out these niches where we can fill in and be complementary.

First off, I think we're still not at the ceiling of technological progression. Maybe in a couple of years we'll need less slop editing, as you said, but I do think we need some more time to adapt to the different modalities of interacting with these systems.

Everyone can interact with a chat assistant, but I think this is the most naive form of interacting with AI agents, for example. So, yeah, I think we need to get the pacing of all of this right. We need to do much more exploration in human-machine interfaces, UI and UX design, and how to make sure that humans feel fulfilled during this experience.

This is particularly relevant because you were behind the AI Scientist paper, and there's now a version 2 of that. Allow me to be a tiny bit skeptical. We were talking about when we evolve systems to do a particular thing, and at the moment, it feels like, as good as they are, they're still quite parasitic on the instructions and intentions of the human supervisor.

It's very much an exchange between the humans and the system. The implication is that, in the future, we might have systems that are so autonomous and so open-ended that they can figure out valuable things to research that humans wouldn't be needed anymore.

The reason why I'm not that worried yet about labor-market disruption is that I still believe deeply that humans are the source of deep understanding and creativity in the world. If I didn't believe that, I would be very worried.

I agree. To me, the AI Scientist v1 and now v2 are glimpses into a potential transformation, but I fully agree that, in order to make really big scientific breakthroughs—multiple breakthroughs every day, or whatever—you still need humans in the loop to either seed or guide the direction in which to explore, or to verify, check, and actually transfer these insights.

I don't think it's going to be that all PhDs will be unemployed. It's more going to be a core evolution of humans with this technology and, potentially, in an ideal future for me, it will allow humans to focus on what they're really great at.

I think it's going to be an amplifier of these latent dimensions that humans are great at. Something that's critical is that we as humans try to interact with these systems as early as possible in order to have influence and ownership over this development process. Ultimately, it's collective intelligence that will shape all of these systems together.

Do you think these systems can become incredibly sophisticated, such that they are somewhat detached from humans?

Well, with the AI Scientist v2, we released a paper that we submitted to an ICLR workshop, and it was able to pass the acceptance threshold before meta-review.

I do think, at least for workshop-level contributions, we're getting there. While not every submission in the AI Scientist papers is reaching that threshold, we're at the point where we can even talk about noisy review processes and this actually being something that, as long as you have a large budget, you might get something out of. I think going forward, for the bigger innovations and so on, for now you still need humans, but we're at the GPT-1 moment of making this sort of a reality, and potentially in 10 years this is going to look very different once the infrastructure for it has been built up.

There are places like Periodic Labs, which are now building real physical labs with robotic systems to automatically execute experiments. This will take some time, but it is imaginable that, as we do RL on these types of systems and actually account for negative results and actual hypothesis testing, we might be able to unlock many more capabilities. Getting these systems to be really good hypothesis testers, with verifiers in the loop, could enable that.

I don't want to sound like a Luddite. It's entirely possible that I just don't have the imagination to think about the future, so it is possible that, in the future, these systems might understand very deeply and be creative. I think right now the problem is that they only understand things a few levels down in the epistemic tree. They can do some surface-level recombination, and they can discover new things in the basin of things that are already discovered, but we understand things very deep down in the epistemic tree, which means our cone of creative potential is much wider. It's possible that gap might be closed. What would happen then?

Chris Lu

The way I think about the scientific process is like a tree search, ultimately. I think a lot of analogies from evolution transfer to scientific research, in the sense that we traverse a tree of different ideas or different experiments, and then in the paper we report one path through that tree. As I alluded to before, we need much more full-tree datasets for training these LLM systems to actually learn how to do this exploration and this foraging, basically.

At the same time, I feel like evolution will also take place on the cultural level for us. We will get better at steering the ship, and I can imagine that, in the future, the way we do research will be completely different. I'm pretty sure that right now, already, 99% of machine learning research is done with AI assistance. Think about ChatGPT brainstorming, Cursor coding, Claude Code, and so on. In the long run, we're going to move along that spectrum from “with AI” closer to “by AI,” and then toward more high-level orchestration and oversight by humans.

There's also the notion of how intrinsically coupled to humans the value function is. One school of thought is that AI will develop a mind of its own and basically transcend humanity, and it will just have agency that is not parasitic on ours. I personally don't subscribe to that view, but the other view is that, let's say, the AI Scientist, version 10, is going to be continually doing epistemic foraging. It's going to be finding new things that are useful, and they kind of have to be useful to us, because if it finds things that are not useful to us, then we just won't use them, and nothing will happen. Do you think there'll always be a kind of value function coupled to humans?

Jeff Clune had this work on OMNI, using LLMs as amortized notions of interestingness for humans. Ultimately, the way we train these systems is coupled to human data, and going forward it will also be coupled with human data that is collected using verifiers. I have a hard time believing that, in the long run, when you run this open-endedness paradigm with AI Scientist agents, it's going to completely divert to something that's either fully uninterpretable or unrelated to problems we as humans care about.

Humans can steer, to a certain degree, where the search happens. You can tell a system, “Okay, try to do cancer research,” and work on problems that we care about. Ultimately, we're the ones who control how many FLOPs are being pushed into this.

As a thought experiment, I can imagine, let's say, in the world of mathematics, what if an AI scientist could come up with entirely new problem formulations and then solve them? These are things that humans had never conceived of before, and maybe we'd be less interested in the answer because humans hadn't spent time thinking about it. We could just explore the phylogeny of mathematics to the nth degree. At some point, maybe we just wouldn't care anymore. Maybe we can just carve out that space forever and ever.

Maybe down the road there is a stepping stone that enables a new innovation in a different field that we actually care about. It's very hard to say a priori whether or not something is interesting.

Yes, and there's also the notion of diverse intelligences and diverse minds. I love this idea. Maybe we could just create artifacts in a space that's completely alien to us. We might even ascribe moral value to them, and we might not want to turn off the power because we want these alien artifacts to stay alive.

Chris Lu

Maybe. I read a lot of science fiction, but I would shy away from speculating about all of this. I do think one thing I'm extremely certain of is that the way we conduct research and science is going to fundamentally change in the next 5 years, 10 years, and 20 years. I hope that we're going to be able to tackle some of the biggest problems that are still seemingly unreachable right now with and by AI.

Terence Tao has posted that he's been using GPT-5, and it's been speeding him up. It's taking away a lot of the drudgery. But the cynical take is that, as Scott Aaronson posted something similar, maybe laziness is stepping in and, in some pernicious way, using AI models is actually stopping us from thinking outside the box. It's encouraging us to search in the neighborhood of things that are known. It's very useful to have an artifact that knows all of the experiments, all of the things that were ever done by people 20 years ago, but now we don't have people really applying their brilliance and talent in completely new areas.

First off, it's great that these experts are already using the technology in their day-to-day work. I think it's also important that really top-level scientists try to push what's possible with these systems, or squeeze out where there might be blind spots or things that these systems can't do.

Second, I think it comes down to discipline and how we raise the next generation. On a personal level, how much do you just tap “accept” on everything that's being proposed by these systems? There's also responsibility in terms of educating the next generation. We need to teach our kids that what comes out of these systems might not always be true, that facts can be subjective, if you will, and that there needs to be more research into what's being given to you. I think this will be the cultural evolution that we have to step through and try to make the best out of.

The autopilot thing is very interesting because there is a tendency, using Cursor, just to accept things. At some point, the models are generating so quickly that you can't even read the tokens coming at you, and then you just press accept and press accept. It's the same thing in cars: as soon as you have too strong of an autopilot, you just completely switch off. Then you see a divergence, because there's something about thinking that must be grounded on your path. There's this path dependence. When you start becoming parasitized by this other train of thought, you stop thinking about your path, and then you're not in the driver's seat anymore.

This is a bit of a harsh statement, but sometimes I wonder if these systems, like these coding assistants, are almost like drugs. You become addicted, you use up all your budget, and then you need to load up again. Once you fully reach the budget limit, you feel like, “Okay, what am I going to do now?” I think once that happens to you, you should really rethink the way you work.

To me, right now, there are certain parts where auto-accepting is acceptable, and there are certain parts where it's definitely not, and you really need to go deep into it. We're in this weird non-equilibrium state where things are moving constantly. The systems or models are changing, the features are changing, and the parts where the systems are good are changing all the time, so we humans need to constantly adapt to that. I think it's a big cognitive challenge, and we just all need to be aware that there are certain problems and certain challenges that we have to adapt to.

I think the best way to do so is just to interact with this technology as much as you can and maybe find new research ideas out of that experience. How is AI Scientist V2 different from V1?

In V1, we used a template-based approach. We had a base experiment, and for that base experiment, we asked an LLM to generate ideas with Semantic Scholar API calls and literature search. It then implemented these ideas based on the template. It basically did code diffs, then linearly executed an experiment plan and wrote a paper in the end.

What could happen was that there was an idea, and that idea didn't work out. But then, in the end, the experiments were still executed linearly, and you wrote a paper. This was already impressive in the sense that it looked very much like science, but if you think about human science and the scientific method, it's much more like research, as I said before. You adapt what you're going to execute next and refine based on the evidence that you accumulated.

This is the notion of falsificationism from Karl Popper: we collect evidence for hypotheses and reject others, and we do so in a loop, basically, until we want to publish or find something. We tried to take this notion and directly build it into the agentic scaffolding for AI Scientist V2.

Now it's basically a parallelizable agentic tree search, where there's no longer a template experiment needed; it's drafted by the LLM itself. AI Scientist V2 can thereby be applied to many more settings, if you will. At the core is this new agentic tree-search paradigm. We also made a couple of minor technical changes, such as using a VLM reader to figure out whether the captions of a paper are aligned with the figures. We scaled this up to many more computational nodes and then wrote a paper in the end again.

I'm trying to say this in the most polite way possible, but a critic might say—I don't want to use the word “slop”—that we are producing papers that appear like papers. They have figures, results, and things written in a certain way, but they're not grounded deep down in the epistemic phylogeny. Near the top of the tree, we're seeing some novelty in composition happening, but it doesn't reflect a deep understanding. What would you say to that charge?

It's for sure that not every paper that comes out of AI Scientist V2 is a Nature-worthy publication. That's for sure the case. Definitely, there is some amount of, let's say, slop or content that is not a big scientific discovery being written up by the AI Scientist. But ultimately, we showed that it was possible to obtain a workshop-level paper.

And you think this is sort of the first time, basically, where we can see that, at least now, we're able to fully autonomously spend compute and API calls to obtain some amount of scientific insight. For me, at least right now, it's a good way to prototype ideas or investigate a certain field, get a starting point and initial results, and then work on top of it. But for sure, more work needs to be done to make this entire process more robust, more efficient, and essentially produce many more true positives, as you will.

Yeah, and there might be one of these things, like when we moved from GPT-3 to GPT-4, where there was just a massive increase in fidelity. With slop, to me, it simply means a lack of deep, grounded understanding. There's no reason in principle why these things couldn't have a deep, grounded understanding. They just don't have it yet.

Yeah.

So it's something that could improve over time, but it's likely to improve quite slowly. Then, at some point, we might just think, “Oh my God, we've got an AI scientist.”

Yeah, this kind of comes back to what we were discussing before. First off, there is a verifier in the loop, in the sense that experiments are actually executed on a computer. The numerical results are fed back into the system to come up with the next thing to explore.

But we haven't made a discovery like a residual connection or something that has diffused into everything in machine learning. I think what we really need is to make these systems much better at integrating knowledge over multiple experiments and becoming better at formulating the next hypothesis based on previous insights.

This might require some amount of post-training on these traces, basically, but I'm pretty positive that we might also get there with just diversity and scaling these systems up in an efficient but scaled-up way.

I'm just thinking that the first breakthrough discovery—would it resemble the AI Scientist paper, or would it resemble ShinkaEvolve? For example, we could do a massively scaled-up ShinkaEvolve and say, “I want to discover a new architectural design.”

Robert Tjarko Lange

Yeah.

And would that happen, and then we'd get the AI Scientist to write it up and do ablations and stuff? Maybe that would be the pattern of it.

To a certain degree, I've been thinking a lot about how you can potentially even combine these 2 paradigms: the AI Scientist and ShinkaEvolve- or AlphaEvolve-style optimization algorithms. I do think there's some amount of work to be done on the auto-verification aspect and on the problem-formulation aspect.

The paper-writing part is actually the least important part of the AI Scientist. It's a form factor that we humans are used to, and it helps anchor our mental model of a scientific discovery. But ultimately, I'm not sure if the paper is going to be the knowledge-transmission medium in, let's say, 20 years.

Something I've also been thinking about a lot is whether we can make papers much easier to access agentically. Right now, it's a LaTeX document, but you could imagine equipping every paper with several Model Context Protocols so that every figure is reproducible, data is accessible, and it's much easier for LLM agents to either replicate the work or work from it afterward. They could do epsilon improvements and ablations themselves through that interface to the paper.

To be entirely honest, I'm not sure if it's going to happen, because there have been many great ideas for improving the format of scientific artifacts out there, and people still seem to like the paper format, which has existed for hundreds of years. I think it's a question of incentives again and really showing that, if something like that existed, it would enable much faster progress by AI agents in scientific discovery.

Yeah, paper is a great human interface. It's a similar thing with automated driving: we could revolutionize the road network to have sensors and dramatically improve monitoring, observability, and optimization.

But I'm fascinated by that idea. You're saying it's not just the reproducibility of the experiments, but also the way that the figures are designed, the code, and so on. Then we could create this huge playground where agents can repurpose, recombine, and restudy work that has been published by other scientists. It also made me think: does having an automated scientist make peer review more or less important?

Robert Tjarko Lange

I do think it actually makes it more important, at least for now. We now have—or could have—a mechanism that generates many, many papers. First, it increases the workload on human reviewers, and we need an effective way of filtering and then essentially taking only the cream of the crop for human verification afterward.

For now, the ultimate verification is still the human and the diffusion of the result through the community. We need better tools for doing this automatic filtering and verification. We have the AI reviewer that comes with the AI Scientist, but you probably need some form of experiment execution to verify everything.

There is, for example, work by OpenAI on PaperBench, trying to go in that direction using LLM software verification and these types of things. I'm hopeful that we're going to figure this out in the next few years.

Yeah, and I think one of the Rubicon moments is when a new Transformer architecture or something massive is discovered by AI and we're all using it. My worry, I suppose, is that folks like Google, who have enough compute power, are going to be running AI Scientists and owning many of these discoveries. That's why it's so important to have work that can efficiently discover new things in science.

It's important to have work that's openly available, too. With the AI Scientist and Shinka, we're really trying to make sure that we can apply the collective intelligence of all of us to shape how this might look in the future.

Amazing. Well, Rob, this has been so fantastic. It's been great to have you on the show.

Robert Tjarko Lange

Yes, and I hope to be able to talk to you in the future again about some of this.

Absolutely. Rob, thank you so much for coming on.

Robert Tjarko Lange

Thank you so much, Tim.