第014期｜为了乐趣而非利润，寻找错误编译（AI 基础设施）｜Justin Lebar & Jordan Nanos

Hello, everyone. Welcome back to SemiAnalysis Weekly. This week, I'm with Justin Lebar. We're going to talk about an article that he put up last week called “Finding Miscompiles for Fun, Not Profit,” also known as “You Don't Need Access to Claude Mythos to Spend $10,000 in an Afternoon.” Justin, welcome to the show. Hot off a $10,000 afternoon, how are you feeling? How are you doing?

Happy to be here.

At a high level, can you talk me through what you covered in the article? It wasn't one of our longest articles recently, but it's gotten the most likes on Substack of anything we've put out in a while, so I think people really enjoyed this one.

Happy to be winning some internet points here. What I went through in the article was the process of finding bugs in 2 compiler stacks. The first one was NVIDIA's closed-source compiler for PTX. PTX is this low-level assembly—or maybe I should call it high-level assembly language—that's emitted by various compilers and goes into ptxas. The output from ptxas is the machine code that actually runs on the GPU.

I went through the process of finding bugs in that, as well as the process of finding bugs in LLVM. I looked at 2 backends in LLVM: the AMD GPU backend, which is what you would use if you wrote HIP code and compiled it through Clang, and the x86 backend, along with the shared code between them that's used by most of the backends in LLVM.

I looked at 2 different ways of finding bugs in compilers. The first was a more traditional method called fuzzing, where you generate random programs, run them through the compiler, and then somehow check that the output after compilation is the same as before. There are some subtleties to that that we can go into if we want, but essentially, that's all you're doing.

I did some fuzzing, and then the other thing I did was LLM-assisted bug finding, where I literally unleashed the agents on the code and said, “Please read the code and find bugs.” We talked about the difference between these 2 approaches and speculated a little bit on what might be possible in the future.

Could you talk a little bit about the motivation to do a project like this—where it came from in your mind? Then we can dig into what exactly you did in more detail.

1. GPU Compilers Face More Bugs

I've been working on compilers for a decade, really compilers for GPUs and machine learning, and bugs are terrible in compilers. I think it's actually worse in the machine-learning world than it is in the CPU world.

If you think about a regular C++ compiler, it's had hundreds of millions, maybe billions, of lines of code run through it. If there's a miscompile, there's some chance that someone is going to notice. That chance goes up the more lines of code you run through the compiler.

In contrast, when we're compiling for GPUs, there's just so much less GPU code in the world. Think about an LLM: there are relatively few programs, relatively few GPU kernels, that go into an LLM compared with all of the code that has to go into Google Search.

That's one big motivation for doing it. My suspicion—which is somewhat borne out and somewhat not borne out—is that it would be easier to find bugs in the machine-learning compilers because they're less mature or less well-tested in terms of the amount of code that gets run through them.

Can you talk about why that might be true or might not be true? Maybe give a few more details on what you actually went through with the fuzzing and then the other approach, with the model directly reading the code?

2. Compiler Bugs Are Hard to Measure

It's really hard to say with these approaches when you get the results. It's hard to interpret them in terms of the conclusion you want to draw. The headline you want to write is, “Compiler X is so bad. It's a completely buggy piece of trash,” and it's really hard to arrive at a conclusion like that.

When I look at the rate of bugs that I was able to find in the AMD GPU backend versus x86, the LLMs reported a lot of bugs, but you then have to ask: What's the severity of these? How likely is it that someone's going to hit one? That's really hand-wavy and hard to say.

I also haven't honestly looked into the AMD GPU bugs as much as I've looked into the x86 bugs, because I'm letting the AMD people work on the AMD stuff. In terms of the x86 bugs that I found, I found 1, maybe 2, very high-severity bugs where, if code hit them—which is a little bit iffy, because is code going to do that?—it would be really bad. It's reasonable that code might hit them.

The one that we link in the article is a bug where, if you construct an atomic operation in a certain way, the compiler will split it into 2 non-atomic operations, which is really bad. Part of what makes it so bad is that most of the time, it will work fine. If you don't have contention on this particular line in the CPU, you would never notice that you even had this bug. But 1% of the time, it's going to do the wrong thing.

We did find that, but that was the worst one that we found. In the AMD GPU backend, we actually did find a number of miscompiles. Again, I don't want to extrapolate from that to say that the AMD GPU backend is actually buggier. It's just what we were able to find with the tools that we have.

That makes sense. Can you talk a little bit about how you actually went about this in terms of writing the fuzzer with an LLM and then having the LLM read the outputs itself?

3. LLMs Build Better Fuzzers

My first instinct was to do what I've done before, which was to write a fuzzer. In that sense, you're writing a program that does 2 things.

First of all, the program has to generate well-formed inputs—well-formed random programs—because you can't just throw any arbitrary program at the machine and expect it to do the right thing. You can have undefined behavior in your program, among other things. You have to generate well-formed programs that are also weird enough that they might trigger a bug.

The second thing that the fuzzer program has to do is have some way of checking equivalency after compilation versus before compilation. Usually, the way to do that is to run the programs somehow, but you could run them on a GPU, which is what I was doing. You could also interpret the programs by looking at the output from the compiler, going line by line, and writing an interpreter for that.

That was the first thing I did for both NVIDIA and AMD, and I had the LLM write the fuzzer for me. What was remarkable to me was how much better that fuzzing experience was compared with just a couple of months before. I know I'm not saying anything new—we all know that the models have gotten better—but I think a less well-told story is that not only have the models gotten better, the harnesses have also gotten better.

When I first did this, from December to January 2025–2026, a lot of the time I spent was literally going into Codex and saying, “Good job. Keep going.” I was repeating myself over and over. It would find 1 or 2 bugs and say, “I'm done,” and I'd say, “No, you're not.”

Now, all the harnesses have this `/goal` feature. You just type that in, and it works. In some sense, it's not the most complicated thing in the world. I could have written that myself on top of Codex back then if I had really cared, but now it's built in, tested, and supported.

That was one thing, but the quality of the code that the LLMs were generating was also much higher. I was able to vibe-code the whole thing, and that hadn't worked when I tried it in January.

What do you think the big unlock was for the LLM? Was it the fact that you were able to do this faster, or would you just not have wanted to pursue it because you didn't have the time or the motivation in the past? Or was it more enjoyable?

I think all of the above, honestly. These are all related things.

Whenever you're embarking on a project with an unknown payoff—I'm looking for bugs, and I don't know what I'm going to find—you always want to be skeptical of the amount of time you're going to put in. When I was negotiating this project with Dylan, he said the same thing to me: “Yeah, do it if it's going to take you a couple of days, but if it's going to take you a couple of weeks, we should really think about it.”

I said, “I don't know how long it's going to take me, so I'll spend a couple of days on it.” Then, in a couple of days, I had these long lists of bugs. That was the motivation for fuzzing.

But inevitably, with fuzzing, you hit 2 problems. Problem number 1 is that it becomes harder and harder to expand the universe of programs that your fuzzer will generate, because you're always running up against these limits of undefined behavior or other ill-formed programs. Problem number 2...

Maybe there are 3 problems. Problem number 2 is that, as you do this, you're generating random programs and doing, in some sense, an uninformed search. You would expect that, in the beginning, you're going to find lots of bugs, but then, as this universe of valid programs expands, it takes longer and longer to find things at the edges that actually trigger bugs. That was another problem that I was hitting.

The third problem is that, after you have a list of 40 bugs that the fuzzer has found, it becomes increasingly hard to teach the fuzzer not to keep finding the same bugs over and over again. You have to go into the fuzzer code and say, “Don't generate this pattern because this is a known buggy pattern.” Now your program that's generating random programs gets increasingly complicated as you find more and more bugs. Are you really able to exclude X pattern or Y pattern, especially if you don't really understand what is and isn't the buggy pattern?

With both the NVIDIA fuzzer and the AMD fuzzer, it started slowing down. The NVIDIA fuzzer kept finding the same bug over and over again, and it was hard to exclude it. When I say it was hard to exclude it, I mean it was hard for the model to exclude it, because I didn't try. I just asked, and it wasn't doing a good job.

4. Reading Code Finds Hidden Bugs

Then I had the idea—and again, this isn't a really novel idea—of, “Why don't we try reading the code?” It's exactly the same thing that Anthropic was doing with Project Glas to find security vulnerabilities. I was just using it to find compiler bugs. To me, it was shocking how well it worked. It was also shocking how expensive it was.

The update to the article, which I think I just added yesterday or the day before, is that we published on day X, and on day X plus 1, Anthropic released Opus 4.8. In addition, they updated Claude Code to have this new UltraCode mode. They claim that UltraCode is for exactly this sort of thing: big, orchestrated projects.

I tried to use that to find bugs, and my initial read is that it's way better. In particular, it's way more token-efficient. I was able to do big scans over LLVM using UltraCode that used about a tenth of the tokens I was using before. I would say the quality of the bugs is actually higher. The number of bugs is lower, but I'm okay with that because I was throwing away a lot of the bugs that I was finding before anyway.

How much of that is different, and how much of that is due to 4.8 versus UltraCode mode, is really hard to say. My guess is that a lot of it has to do with ultrathink, based on the fact that I don't feel like 4.8 is a huge intelligence step up from 4.7 in terms of what I can see. Therefore, I would attribute this big step to the harness, but I really have no idea.

Okay. Really interesting. I got a lot of questions that come to mind there. Maybe the first one is to take a step back: prior to the article update and the 4.8 release, would you have had a way to estimate—I know this is hard—the amount of time it would have taken to do a project like this without the assistance of the coding models, and then possibly extend that to the new model and the ultra code feature?

A project like this meaning reading the code, looking for bugs, and actually finding meaningful bugs?

Maybe start with writing the fuzzer itself, running it, and then reading the code itself.

5. Bug Hunting Is Economically Practical

We'll start with reading the code, which is basically impossible. I don't think that any person could do it. It would fry your brain. If you really, really, really cared, you could try, but actually, you wouldn't do it this way. I think the only way to do it would be to write a formally verified compiler, which people have done. The CompCert project, for example, isn't really used in production. I think that's really the only way you could have done it.

In terms of writing the fuzzer, it would have taken me weeks, and it took me days instead. Again, it's a little hard to say what the quality difference is between the fuzzers. I didn't look at the new one, but it found a lot of bugs.

Especially for the fuzzer for the closed-source NVIDIA compiler, at some point, that's kind of all you can do: find bugs. You can't really fix them, and you also can't really evaluate how many root-cause bugs there are. It's even hard to evaluate how bad a bug is because, in the end, you don't really know what the buggy pattern is. You just know that you were able to trigger it with this particular code. It could be much bigger than that, or it could be really, really specific—ultra-specific—to what you wrote. You don't really know.

That's why I hesitate to make big comparisons. It was really easy to find a list of 40 bugs that were easily reproducible. Of the 4 or 5 that I looked at, every one had no undefined behavior. It was very clearly a bug. You have a series of, I think, max and subtract operations: max, subtract, max, subtract, 4 times, and then you get the wrong answer. It's very clear that there's something wrong.

I think the most interesting or telling statement that you just made is that it would be roughly impossible to take the approach of reading every single line of output from a compiler and then identifying the—

Not even a line of output—I'm talking about a line of code inside the compiler. Even if you read them all, are you going to be smart enough to notice that there's a bug there? Most of the bugs that I've gone through—and I've gone through more than 30 of them in the x86 backend—almost all of them I never would have noticed. Even if you had said, “There's a bug in this 100 lines of code. Find it here,” I never would have been able to find it.

Does this leave you feeling optimistic or pessimistic about the state of open-source and closed-source compilers, and about the state of the models as far as how they can contribute to this infrastructure that underpins a ton of software that we're using right now?

I feel super optimistic, so long as someone is going to be willing to spend the tokens. Even at $10,000, it's not that expensive for a small company like SemiAnalysis, much less a big company like OpenAI or Google. Whether they're actually going to do it is the other question.

In that sense, I feel really optimistic because we are finding real problems. Even when I've been writing the fixes, I have the model check them. First of all, I have the model write the fixes itself. But then, if I go in and change something, which is happening about 50% of the time, I have the model check my work, and it frequently finds bugs in my work. In this sense, too, it's dramatically improving the quality of the code that we publish.

Part of the reason I wanted to publish the article was to get other people to do the same thing, because it still takes human time to filter through the bugs. It also takes human time to convince a boss to let you spend time and money on this project. But if we do, I think it can dramatically improve the quality of the software that we're shipping.

I don't know if you have plans to work on this project going forward, but is there any next step or any ideas that have sprouted because of the work you did with FuzzX?

One obvious next step is actually fixing the bugs. The LLVM people have been really great at turning around reviews, so I've fixed most of the high-priority bugs that I've been able to identify—and again, I should say, that the models have identified for me—in the x86 backend that it found. That's been really great.

One interesting next step that I haven't tried would be to apply the same “read the code” approach to the NVIDIA closed-source compiler. But the code that you're reading there is the assembly language, as opposed to their source code. I would be really fascinated to try that, especially with a more powerful model like Mythos. Maybe even 4.8 would do a good job at it. I literally haven't tried. I didn't want to blow Dylan's token budget on it again.

I got comments about that, but I'll keep them to myself about people feeling free to spend tokens currently. Token-maxing, man. It's a virus.

If there's anything that listeners, or people reading the repo or trying to do a project like this themselves, should be taking away—or for the more general audience that may be learning about what a compiler is for the first time—what do you think the key takeaways are?

6. Software Quality Is the Bigger Prize

The takeaway for me is less about compilers than about software development in general right now. We know that anytime you make a new tool for finding bugs, you're going to find bugs. It's not surprising that LLMs were doing this, or that my fuzzers were doing it.

But I do think that the rate at which I’ve been able to fix them has been a lot higher than it would be otherwise. The rate at which I’ve been able to triage them has been a lot faster than it has been otherwise. I mean, a tool that flags 1,000 pieces of code as suspicious, and 1 of them is an actual bug, isn’t actually that useful. It’s a huge amount of work to go through it.

The LLMs so far have been quite good, especially 4.8 with UltraCode. We’re talking about a couple of days old at this point. Part of the callout I would make to people is that if you have a code base that you care about, it’s worth throwing an LLM at it and seeing what it finds. If you have the budget to spend $10,000 on it, you should.

If you have the budget to spend $1,000 on it, that seemed to be what it was costing me to throw UltraCode at it, which is actually extremely reasonable. I don’t know—maybe this is the compiler engineer in me talking—but I always feel like bugs that you find yourself are much better than a user finding them. On the other hand, there often isn’t an incentive to do that because nobody gets handed a medal for preventing a fire.

Instead, you get handed the medal for, “The customer filed these 10 high-priority bugs, and I got right on it.” But I think we have an opportunity to really improve the quality of software that we’re shipping if we choose to use the tools and if we choose to spend the tokens on it. I’m hopeful that people will do that, and that’s part of why I wanted to write this article.

Yeah. That’s certainly optimistic from my point of view, with the discourse right now being something along the lines of “slop AI,” a slop cannon producing a bunch of buggy software. But from your point of view, the fundamentals of this, if used correctly or for a specific purpose, would allow people to ship less buggy software. You just need to focus on the right things, I guess.

Yeah. Again, I try to be scientific about this and not make larger claims than I have evidence for. I haven’t seen the evidence that the AI slop cannon is actually buggier than the human slop cannon. People often say, “How can I possibly trust that this model code doesn’t have bugs in it?” Well, it’s nice that you’ve never written a bug yourself.

I do agree that right now, the AI is slop in that it’s not as beautifully designed as I think my code is beautifully designed. That’s also true for code that anybody other than me writes. But in general, I think it’s worse than what I expect from my best collaborators.

When I say it’s not as beautiful, I mean that I don’t enjoy reading it as much. But I haven’t seen the scientific evidence one way or another showing that it’s actually buggier. It could be that the AI slop is harder for LLMs to find bugs in. It could also be that it’s easier for LLMs to find bugs in it, or it could be equally hard. I also have no idea.

Yeah, super interesting. I think it’s certainly more bloated, but I agree that it’s a toss-up whether it’s actually producing more or fewer bugs. Per line of code, maybe more. Per hour or per day that you’re shipping features, probably less, because at least I can fix them—or do different things and fix them quickly.

No other questions jump to mind for me right now. Is there anything that you think is left unsaid about this article or this project that you want to cover?

I’m excited to see what other people do with these tools, and I hope that people go and use them and also report on them, because what I did was extremely unscientific. It was a case study. It’s a story: I did X. We’re going to get better data if more people do it and talk about their experiences.

Even that, of course, isn’t scientific. The plural of anecdote is not data. But I’d love to see more anecdotes about this, and then maybe we can even try to approach it scientifically. I’d love to see us focused on finding bugs—not just security bugs, but bugs in databases, in other compilers, and in web browsers that aren’t security bugs. I think there’s a lot of potential for this kind of thing.

Agreed. I hope you continue to work on this and continue to provide these anecdotes in the open and develop in the open. It’s been really fun following this project from the sidelines and learning a little bit along the way.

Thanks.

Really cool stuff.

Thanks. Good talking to you.