[BidClub_]
Machine Learning Street Talk · · 63 min

The Thermodynamic AI Chip · Thomas Ahle

Thomas AhleTim Scarfe

YouTube ↗
TL;DR
  • Normal Computing’s work spans an AI-assisted EDA stack for a closed, punishingly expensive chip-design toolchain. Thomas Ahle describes a path from human intent through Verilog/RTL, optimization, formalization and verification to tape-out. A commercial tool may cost roughly “$10,000…for one CPU kernel”; a million-agent data center would imply about $10 billion in licenses. His swarm generated more than 500,000 lines in 43 days, while roughly 20 GPT agents have kept progressing for around six months.
  • Agentic output is scaling faster than grounded understanding, making correctness—not code generation—the constraint. Tim calls the result a possible “spaghetti monster” and a source of “understanding debt.” Thomas admits the bet is “escape velocity from code complexity,” with models improving faster than their code deteriorates. ProgramBench’s roughly 150–170 programs initially yielded 0% complete passes, and even 70–80% test scores prompt the harder question: “did it get any of them actually right?”
  • Formal proof only protects a design when AI has faithfully translated human intent into the formal specification. Proof or disproof can provide clean verifier rewards, but auto-formalizing thousands of pages is brittle: a few wrong words or numbers can produce an irrelevant theorem. Separate design, test and coverage teams provide partial independence; repeated runs add useful disagreement but not true orthogonality, so adoption remains “a trust exercise.”
  • Normal’s released CN101 chip makes noise itself the computation for a narrow class of probabilistic workloads. An array of capacitors connected by programmable resistances is driven by noise so the circuit follows stochastic differential equations; the resulting behavior is related, “sort of,” to the inverse of the encoded matrix. The test is whether scaled benchmarks validate the approach beyond today’s narrow use cases.
  • Hardware acceleration is a co-design problem, so a spectacular component speedup can still be a false economy. If inference currently uses only about 10% of a GPU, eliminating that bottleneck could offer roughly a 10× gain in that slice before another stage dominates. Diffusion hardware likewise matters most if models are redesigned to “go all in” on cheap randomness. The guest favors hybrids: learned intuition plus optimized classical search, as in Stockfish’s fast neural evaluation inside a traditional engine.
  • Continual learning could turn agentic systems into abstraction-building machines, but it disrupts alignment and serving economics. The guest says models form abstractions during pre-training—Mistral’s code intuition is the example—but do so poorly live. He thinks Dario may regard on-the-fly learning as a safety risk because it can drift from a safe checkpoint. Live weight updates also break shared-model serving, though per-user LoRAs or always-adapting hardware may offer routes. Tim argues that Claude already behaves adaptively as a broader ecosystem of users, tests and retraining.
  • The largest execution risk is human and organizational: AI can raise performance while eroding competence, review norms and shared code. Tim calls it “the most deceptive thing ever created in human history”; the guest’s sharper formulation is, “it’s not just that it’s getting smarter, it’s also that humans are getting dumber.” Their mitigations are deliberately frictional—type code by hand, limit parallel projects, preserve expert review, and find teamwork structures better than everyone watching ten private agents.
Digest · the substance, structured for research

1. EDA economics slow AI adoption in chip design

  • Thomas Ahle describes a background in theoretical computer science and algorithms for high-dimensional data, including locality-sensitive hashing, before moving into thermodynamic computing. Normal Computing’s broader stack aims to take a designer from intent through RTL, optimization, formalization and verification to tape-out—something he compares to “the Lovable for chip design.”

  • He describes Verilog as a “super-parallel programming language” whose constructs map onto hardware. Engineers compile RTL into netlists and schematics, then simulate and formally verify the design before sending it to a fab; unlike software, fabricated mistakes cannot simply be “fixed in prod.”

  • The toolchain lacks software’s open-source ecosystem. A commercial simulator or verifier may cost about $10,000 “for one CPU kernel,” so the exchange estimates that scaling to a million agents could mean roughly $10 billion just for licenses—before paying for the computers themselves.

  • That scarcity also starves hardware AI: little open-source code is available for pre-training, and agents cannot cheaply access the tools needed for reinforcement learning. Python and JavaScript therefore feel “day and night” ahead of hardware workflows.

  • The paranoia is economically rational. Thomas cites the famous Intel division bug of the 1990s as costing between $500 million and $2 billion, with other tiny escaped bugs nearly bankrupting companies; hardware is consequently a natural home for formal methods.

2. Half a million generated lines create an understanding-debt bet

  • Thomas’s Verilog-simulator project produced more than 500,000 lines in 43 days; he also describes roughly 20 GPT agents running for around six months and still progressing. His explicit bet is “escape velocity from code complexity”—models improve faster than their generated code becomes unmanageable.

  • Tim questions the upgrade illusion: Fable suddenly makes 4.8 look terrible, just as 4.8 displaced 4.6. Thomas concedes it could be “smoke and mirrors,” but says three days of Fable access cleaned up some GPT-5.5 mistakes and produced more progress on objective tests; that evidence earned some trust.

  • ProgramBench, which Thomas thinks Facebook released, contains roughly 150–170 programs, including FFmpeg. Models initially scored 0% because none passed every test; reporting 70–80% of individual tests obscures that a program failing 20–30% is probably not correct.

  • Tim doubts deep structure can be recovered from behavior alone. Thomas counters with codec reverse engineering: people infer an unknown format from encoded blobs, screenshots or clips, and priors about transforms or chunking, iteratively visualizing output until the movie appears. Tim reframes this as a “generation–discrimination asymmetry”; Thomas likens it to A* search pruning.

3. Intelligence needs reusable abstractions, not just longer searches

  • Tim argues that LLMs can display intelligence through “combinatorial closure”: given useful primitives, a model can hill-climb and compose computational structures. The guest agrees that higher-level abstractions in a better pre-trained model can make subsequent reinforcement learning more effective, but live systems rarely add discoveries to a reusable library.

  • The guest thinks abstraction does happen during pre-training—Mistral’s apparent intuition for bugs in Normal’s codebases is his example—but agrees that continual learning is missing during live use. Tim also describes Claude as an adaptive ecosystem: users run tests and counterfactuals, those experiences are retrained, and later users inherit the resulting abstractions.

  • The capability is fraught. The guest thinks Anthropic may be one of the companies wary of continual learning and recalls Dario saying that live adaptation could be a safety risk because an agent might drift away from its aligned “safe checkpoint.” The attribution and wording are presented cautiously in the conversation.

  • Serving would need redesign too. Updating weights per user prevents providers from sharing one fixed model across customers; the guest mentions a shared base with LoRAs, similar to something he associates with Thinking Machines, while acknowledging that continuous updates remain operationally difficult.

  • On chain of thought, the guest focuses on giving a model state and workspace in which to operate. He suggests that a simple LLM-like system with access to such a persistent process could, in principle, have the representation capacity of a universal Turing machine; whether it can learn to exploit that capacity remains the empirical question.

4. AI could turn custom silicon into the next CUDA kernel

  • Tim frames an ASIC as intelligence crystallized: compress a complicated task into its best representation, then bake it into non-adaptive hardware. The guest agrees that this can make current models extremely efficient, but warns that locking hardware too tightly historically constrained software innovation; Nvidia’s flexibility left room for experimentation, even while steering AI toward matrix multiplication.

  • AI-assisted EDA shortens that lock-in cycle. The guest imagines developers creating specialized circuits as casually as CUDA kernels: invent an algorithm, ask AI to design and optimize its circuit, check correctness, then exploit improved batching at the fab.

  • The resulting loop qualifies as “recursive self-improvement”: AI helps build better software and hardware, which then improves AI. The guest remains drawn to the opposite endpoint too—flexible chips that learn on-device instead of freezing every abstraction into silicon.

  • Analog memory may make continuous learning practical rather than optional. Basic capacitors require refreshing, while analog resistive memories can struggle to retain values; if inference and learning continue together, adaptation itself can refresh the state, closer to the guest’s picture of synapses that “are always adapting.”

  • Tim connects this to the performance-versus-competence debate through Chomsky’s bulldozer analogy and his comparison of Deep Blue to a bulldozer winning an Olympic weightlifting contest. The guest responds that if AI is genuinely doing language, systems that do it differently from humans could still illuminate the abstract concept of language.

5. Formal verification is only as good as the formalized intent

  • The guest separates auto-formalization from proof. The first converts a human specification into something like Lean; the second proves that code satisfies it. Verifiers create strong reinforcement-learning environments for proving or disproving a formal statement, but they cannot tell whether that statement captured what humans intended.

  • AlphaProof’s useful training trick, as the guest tells it, was requesting “a proof or disproof.” A mistranslated statement could still generate training signal because the model might prove it false; for the 2024 IMO problems, however, correct formalization mattered and was performed by hand.

  • Tim adds that AlphaProof used Lean 4 despite limited Lean 4 material, requiring a Lean 3-to-Lean 4 converter and substantial human repair. He also notes that a newer model that won gold at the IMO did not use formal verification in the same way, reinforcing the idea of a spectrum rather than a binary.

  • Hardware raises the difficulty from a paragraph to thousands of specification pages. “A couple of words wrong” or a few mistaken numbers can make the resulting proof irrelevant. Chipmakers respond with orthogonal design, testing and functional-coverage teams, effectively testing the tests and checking every requirement off.

  • Multiple AI runs can expose disagreement because long generations contain enough entropy, but the guest does not call identical models genuinely orthogonal. Tim’s broader point is that assurance may remain a spectrum of tests, proofs, visual inspection and independent perspectives.

  • The conversation also rejects the idea of one uniquely correct formal representation. The same mathematical or hardware intent can be represented in multiple ways, and alternative representations may help both model performance and the legibility of the final artifact.

6. System-level hardware needs more than cycle-perfect proofs

  • Cycle-level properties—exact behavior at every clock—fit SystemVerilog Assertions, including for adders, arithmetic circuits and cryptographic hardware. Protocol-level questions such as deadlock, livelock, timing consistency and shared-resource conflicts call for higher-level formalisms such as TLA+ and timed Petri nets.

  • The guest’s DRAM example makes the distinction concrete: activating a row requires waiting for data to travel before reading, capacitors need periodic refreshing, separate banks can work concurrently, and shared circuitry introduces dependencies within banks, across banks and across bank groups. A timed Petri net represents this parallel choreography without specifying every gate.

7. CN101 makes thermal noise do probabilistic math

  • The guest says he came to Normal Computing after working at Facebook on probabilistic computing and Bayesian neural networks. Posterior inference could require many repetitions with different random seeds or analytical approximations. His provocation is that manufacturers spend enormous effort eliminating electrical noise, while probabilistic software adds randomness back: “why not try to build a chip that’s just inherently random?”

  • Normal’s first silicon, CN101, uses an array of capacitors linked by programmable resistances. Once noise is injected, the physical system follows stochastic differential equations; the guest says its behavior corresponds “sort of” to the inverse of the matrix encoded in the circuit, which can be captured through averaging.

  • The present target is narrow, and the guest emphasizes co-design over universal claims. CN101 is an early implementation of a new computational paradigm; the decisive evidence will be benchmarks as the architecture scales and as models are redesigned around abundant physical randomness.

  • Bayesian uncertainty also becomes less straightforward in generative AI. A distribution over one output once had obvious value; for a sequence that explores ten options, backtracks and produces a final answer, users care about trust in the conclusion—not uncertainty over one particular token.

  • The guest’s earlier experiment made models gather evidence, estimate its likelihood under competing answers and combine it with Bayes’ law. A manually calculated neuro-symbolic version performed well in an internal prediction game, but telling the model to apply Bayes itself did slightly better—perhaps, he hedges, because it discounted probabilities it had “bullshitted.”

8. Co-design determines whether thermodynamic speedups survive

  • The guest’s warning is that specialized hardware can accelerate the wrong slice. If inference currently uses only about 10% of a GPU, fixing that bottleneck might yield roughly a 10× gain in that slice; afterward another component dominates. A diffusion accelerator similarly disappoints if a conventional neural network elsewhere sets total latency.

  • Existing algorithms were shaped around GPUs, where sampling huge numbers of Gaussian variables is unattractive. Faster random hardware offers an immediate gain, but the larger opportunity requires new models that “go all in” on the newly cheap operation rather than preserving an architecture built to avoid it.

  • The guest rejects using LLMs for every computational layer merely because they could emit binaries or assembly. Cryptography, big-number arithmetic, compilation and synthesis contain brute-force loops where optimized classical algorithms remain far faster; learned knowledge should guide search, then hand work to the right mechanism.

  • Stockfish is his model for the hybrid: a traditional high-speed chess search whose evaluation function became a shallow, wide neural network that updates extremely quickly. It outperforms the best open-source AlphaGo-type chess engines by combining intuition with structured search.

9. AI productivity is breaking competence and collaboration

  • Tim argues that software once canalized complexity into shared interfaces such as spreadsheets; agentic coding instead creates personalized webs “like bash scripting on steroids.” The guest sees open source fragmenting as people regenerate code from scratch rather than converging on maintained libraries, even when the private result is locally more efficient.

  • The conversation also touches product and tool economics. Tim uses Notion through an MCP or CLI interface rather than paying for agent calls at API prices; the guest says API pricing is “killing innovation” in that space and mentions account-based access as a possible alternative.

  • The guest adds that specialized tools encode accumulated domain knowledge, so agents extracting and distilling that knowledge could make the tools themselves strategically valuable.

  • The epistemic failure is sharper outside one’s expertise: Tim describes “AI psychosis,” in which Claude convinces users that mediocre papers or code are excellent while experts spot the defects immediately. The guest says this breaks the social contract—why spend time reviewing something its sender may never have read?—and floods technical communities with unread AI-generated work.

  • Their core warning is dependence masquerading as understanding. “It’s not just that it’s getting smarter, it’s also that humans are getting dumber,” the guest says: people summarize papers instead of reading them and inherit outputs without their brains “going through the motions.”

  • The guest’s countermeasures include typing generated code by hand, warning employees before copy-paste and resisting the fifth parallel project until the first few are understood. These frictions matter because Normal must train AI engineers in hardware and hardware engineers in AI rather than merely maximize immediate output.

  • Teamwork remains unresolved when everyone watches ten private agents. The guest recalls an Anthropic team saying everyone independently coded a version and they selected the best—an effective human ensemble, but “it does remove all of the teamwork.” Tim closes the episode’s governing tension: the industry is collectively betting that performance matters more than competence.

Tim Scarfe

Yeah, so why not try and build a chip that's just inherently random?

Meet Thomas Ahle. I caught up with him in Zurich, and he's one of these rare galaxy-brain people who's comfortable in probabilistic machine learning, formal verification, and chip design. However, there's a small problem: when the token god hands you something that looks like it works, how do you know it's actually right?

Thomas Ali

My background is in theoretical computer science. I used to do algorithms for high-dimensional data, locality-sensitive hashing. Then I moved to thermal computing to develop thermal computing, also to speed up Bayesian intelligence. Sometimes I think about it as the Lovable for chip design. We take it all the way from your intent through the design, through optimizing your design, to formalizing and verifying your design, all the way to tape-out.

Tim Scarfe

Now, I didn't fully appreciate this before. These days, a chip doesn't necessarily start in a factory. It can start as code. Engineers design the whole circuit in a language called Verilog, almost written like software, and only much later does any of it become physical silicon.

But first, that code has to be simulated and formally verified. It has to be proven correct. Because once a chip is fabricated, if there are any bugs, you're in big trouble.

A few months ago, Thomas blogged about building a Verilog simulator using a swarm of AI agents collaborating with each other, and it generated over 500,000 lines of code in 43 days. The reason he needed to do this is that commercial software costs a ridiculous amount of money and isn't very friendly to using agents—$10,000 per seat or something.

Thomas Ahle

Yeah, probably for one CPU kernel.

Tim Scarfe

If an AI can generate a chip design, a proof, or a working program, how do you know it's actually correct?

Thomas Ahle

You can look at the percentage of tests that it got right, or even the Fable. It was like, “Oh, yeah, it got 70–80% of the tests correct.” And it's like, “Yeah, but I talked with the people who made the benchmark, and they said, ‘But did it get any of them actually right?’”

Tim Scarfe

Now, in chip design, noise is the enemy. Manufacturers spend a fortune getting rid of it, but in thermodynamic computing, the opposite is kind of true: the noise is the computation. It comes straight out of probabilistic machine learning, which already runs on randomness and uncertainty, and thermodynamic computing tries to make the chip itself a stochastic differential equation. Let the chip's own noise settle into place, and then it can land on answers that would normally cost a fortune to compute.

Thomas Ahle

Then you infuse all of this noise, and it'll start to behave according to these stochastic differential equations. It actually behaves sort of according to the inverse of that matrix.

Tim Scarfe

They've already released their first version of this technology. It's a chip called CN101, all the way to silicon. It's still early days, and it's aimed at a fairly narrow band of probabilistic workloads. But the real test is what the benchmarks can say once they scale this up. Something of huge interest to us at MLST is: if we're going to let AI build everything, what kind of understanding do we actually keep in the overall process?

Thomas Ali

It's not just that it's getting smarter. It's also that humans are getting dumber.

Tim Scarfe

Now I should disclose that Normal Computing very kindly offered to cover our production and travel costs for this show, but we kept full editorial control.

The Thermodynamic AI Computing Chip

These are tools for device manufacturing. They're things like compilers for RTL. RTL is like the programming language you use. If you're building a chip, you typically don't draw all the different gates and so on; you write it in a programming language like Verilog. It's like a super-parallel programming language, with a lot of constructs that map well to hardware.

But there are actually no good open-source compilers for this system. The whole hardware industry doesn't have the same open-source feeling as software, where all the good stuff is free and there's this big, vibrant market of tools. Here, it's much more locked down to big providers.

People then compile this, and they get a netlist. They can start building their schematics. They can also look at it. You get these giant things that you can eventually send to the fab. But they also need to do simulations, for example, for testing. You can do formal verification.

Tim Scarfe

Let me just play that back so I understand it. Verilog is a bit like a programming language for describing a circuit. Before we go and get that fabricated, we probably want to do simulations because it's very expensive to get it fabricated. So, we can design the circuit and do simulations.

Thomas Ahle

Yeah, exactly. It's super expensive, too. I think they said there was this famous Intel division bug in the '90s that probably cost them between $500 million and $2 billion. That's a lot of money in the '90s.

There are other examples that nearly bankrupted some of these companies—tiny bugs that managed to make it through to the fab. It's a really different world from software, where people fix stuff in production. You just move fast and break things. People are much more paranoid in the hardware industry, which is also why it's a cool place for formal verification and formal methods to thrive.

Tim Scarfe

So you're saying that at the moment there are these commercial verifiers and simulators, and they cost a ridiculous amount of money. Is it something like $10,000 per seat or something?

Thomas Ahle

Yeah, for one CPU kernel.

Tim Scarfe

Right.

Thomas Ahle

Say you want to scale this up in a data center with a million agents running—computers are already expensive, but not that expensive. What is that? $10 billion or something, right? Just for those licenses.

I think that's actually one of the reasons AI still isn't as popular in the hardware space. They haven't been able to train the models for these kinds of workloads because it's not feasible. You don't have all of the open-source code out there to start the training, but you also don't have the tools that they need to learn to use. You don't have all of the tasks—you can't just do all of the reinforcement learning on top of it.

They clearly are doing some of that. We can also see from model generation to model generation that they're getting better, but it's kind of night and day compared to Python or JavaScript. We've been developing these EDA tools in-house using AI. I think I probably have the world record for the longest-running agents, having some 20 GPT agents running for around 6 months now. They're still making progress.

Tim Scarfe

So, as I understand it, similarly to Anthropic, which had this blog post out—they had a functional specification of a C compiler, and about 40,000 agents reproduced a C compiler—you did a similar thing for an agent for an—

Thomas Ahle

4.8, by the way, right?

Tim Scarfe

Well, I mean, that's an interesting thing as well. This is what I want to get to, right? It's tantalizing that there are some domains that are so well evolved that we have—they might be very complex, but we have a functional specification, and it's reasonably coherent.

The idea is that we get a shitload of agents and reproduce the function of this software based on these tests. We do it recursively, agentically, and so on. My contention with this is that I think it's not about where you end up. It's not about the functions and the tests passing. It's about how you got there and how structured it is.

There is this tendency with agentic coding to build a spaghetti monster which seems to work. If you think about it, in this project you're talking about, I think you said there's 500,000 or more lines of code. In 5 years' time, there's all of this code, and you folks probably haven't read most of it. That sounds like a problem.

Thomas Ahle

Yeah, I think for a lot of this stuff, we're relying on the hope of some kind of escape velocity from code complexity: that the models are going to keep improving faster than our code gets messed up. I had those 3 days of Fable access, and it was really good. It definitely cleaned up a few things that GPT-5.5 had messed up.

Tim Scarfe

But did it, though? Do you think it could be deceptive? Because when Fable comes out, all of a sudden 4.8 looks terrible. And when 4.8 came out, it looked amazing compared to 4.6. There's something deceptive about it. It's like a parlor trick.

Thomas Ahle

Yeah, no, it's true. In principle, it could all be smoke and mirrors. But I think that's why it's good to have the hard tests, right? I could also see that I was suddenly making more progress on the more objective tests than I had seen for a while with the other models, so I guess that gives me some trust.

But it is weird to not have the same in-depth level of understanding of everything in your code. I definitely miss it.

Tim Scarfe

And I’m interested in what the consequences of that are because you said you put this blog post out earlier in the year. When we had a chat on the phone, you said that you noticed there were some things that were wrong. There is a tendency, though, to accumulate understanding debt, and when that happens, I think from an evolutionary point of view, you’re stuck. I think deep, grounded understanding of how things are is the basis for the next design decisions and the next evolution. So, do you get into this kind of fishbowl thing where now you’re in no man’s land and you don’t really know what to do?

Thomas Ahle

Yeah, I do think it’s important to understand as much as you can and have time for. I think for compilers, there are some very important architectural design decisions, and then there’s a lot of just implementing 100 different functions. They all just—especially modern compilers—it’s layer by layer by layer. You have to lower these things from the front-end level to the intermediate-representation layer and then to the back-end layer. A lot of that, you definitely don’t want to have to write; it also parallelizes well, I think. I’m sure this could be one of the reasons why Anthropic also picked it.

We talked about this ProgramBench thing, I think Facebook released, where it’s a benchmark. The task is—I think they took 150–170 programs, and some of them are really complicated; FFmpeg is one of them. The AI just has to reimplement them without internet access. Basically, when it came out, all of the LLMs got 0% because none of them were able to pass all of the tests.

But I’ve seen people—I very rarely see that when people post benchmarks for this thing. They always post, “Oh, the percentage of tests that it got right,” or even, like, the FABLE. It was like, “Oh, yeah, it got 70%–80% of the tests correct.” And it’s like, “Yeah, but I talked with the people who made the benchmark, and they said, ‘Yeah, but did it get any of them actually right?’” Did it? Because if the program only passes 70% of the tests, it’s probably not right.

Tim Scarfe

I know, in a way, this is the thread that we were talking about before. For the last 60 years, going back to behaviorism, there’s been this thread between structure, competence, and prediction. Essentially, ProgramBench is making the argument that you can learn the physiognomy of a program from its external behavior. You can learn the deep structure, the constraints, and so on. I suspect that isn’t possible unless, of course, the LLM already knows about the source code because it’s in its training data or whatever. What do you think about that?

The Thermodynamic AI Computing Chip

Yeah, I think humans do it all the time, actually. There’s the whole field of reverse engineering, where people—actually, I listened to something with the FFmpeg people where they were talking about all these codecs that they had put in there, and a lot of the time they have no idea what they do. It’s just this obscure blob of code; they maybe don’t even have access to the code.

They just have a couple of example videos that were encoded with this, and maybe some people had shared screenshots or clips from the movie. Then they had to go and try to reverse-engineer this encoded blob, asking, “What is it doing? How could we write a program that decodes this thing?” Somehow, people are able to do it.

We must have some sort of prior, right? What are the typical things people put in a video codec? There are probably Fourier transforms of some stuff, or you probably chunk things up. I don’t know; I haven’t done that much video encoding. But you can just treat it as a completely black box: try to keep improving your program and visualize the output until the end of the movie comes out, and you have an idea of how this probably looks—how the movie is supposed to look.

Yeah, well, that’s an interesting thing as well. We can hill-climb in a partial-knowledge regime, right? We might have a verifier. As you say, a lot of science is a form of intelligence where you’re using MATLAB and you do, say, a Laplacian or an image plot. You take a distribution, and it takes shape one step at a time. So, what you’re doing is taking steps into the unknown, and every single step of the way, it’s like there’s a generation–discrimination asymmetry, right? We can discriminate well, but we can’t generate yet. We take these steps, and then, when we get there, we kind of collapse that into a model, and that’s the artifact of the intelligence.

The Thermodynamic AI Computing Chip

Right, it’s like an A* search or something, where you prune your search tree as you go based on what seems most successful so far.

Exactly. But to your point, every single step of the way, we’re using the toolbox. Some people are really good at riddles or intelligence tests, and a lot of it is skill, right? They have a toolbox of abstract tricks that they use.

But then there’s this question: maybe that’s all it is. Maybe there are these natural patterns, these abstractions, that we can compose together and use to handle any kind of novelty in an intelligence setting. Therefore, maybe language models could do this autonomously. I think they possibly could, yeah.

The Thermodynamic AI Computing Chip

This is a good point, because I tried to get an answer from Noam Brown at some point on Twitter about this. You see how the better pre-trained model you start with, and then you start reinforcement learning on it, the better the reinforcement learning seems to work, depending on how good the model was to begin with, in a sense.

Yeah, super interesting. I agree with you. I think it’s wrong to say that LLMs aren’t intelligent, right? There is some kind of combinatorial closure, and that means that from the primitives in an LLM, we can do hill-climbing and build some computational structure to solve problems. But that’s not quite it, and it can happen at different levels of abstraction.

If the LLMs have higher abstractions, then they can traverse the combinational closure of those higher abstractions and solve the problem. What seems to happen, though, is that there’s no continual learning, so those abstractions don’t get added to a library and reused later. But there’s also no abstraction. What humans would do is look at this computational graph and say, “Ah, I see that that’s just an analog of this thing over here.” And I’m now going to compress it into a new variable that screens off all of that complexity. Why don’t language models do that?

The Thermodynamic AI Computing Chip

I think you’re getting into a continual-learning-type thing, which is obviously a big thing we lack. I do think language models do that during pre-training. That’s why, for example, Mistral is so good at doing stuff in our codebases: it has somehow seen a lot of code, and it just seems to have this intuition for bugs and problems and so on. That must be because it’s seen the patterns before.

But it’s true that when you’re actually running them live, they don’t do it very well. A lot of people these days are thinking about this: how can we do better continual learning so it can build all of these abstractions really well on the fly? I know some companies are actively against trying to do that.

Tim Scarfe

Why?

The Thermodynamic AI Computing Chip

I think Anthropic is one. I think Dario said he considers it a big safety issue, too, because you could easily lose all of the work that has gone into alignment if the agent is learning too much on the fly. It gets further and further away from the safe checkpoint.

But I think there are enough other people working on it that it’ll probably happen. It’s definitely a big unknown, right? I think it’ll also have to change a lot of things in how we serve models, for example, because suddenly you need to be able—if by continual learning we mean that we actually update the weights live—it’s going to cause a lot of problems for the current paradigm. You suddenly can’t use the same weights for all customers, either. I guess Thinking Machines has this thing where they have one shared model and then some LoRAs on top.

Mhm.

The Thermodynamic AI Computing Chip

Maybe something like that can work, but you still have to somehow keep doing this all the time. We’re also working in non-von Neumann computing on alternative forms of computation.

Tim Scarfe

Yeah.

The Thermodynamic AI Computing Chip

Like these thermodynamic chips or just unconventional computing. It actually becomes sometimes hard to keep the memories in these analog resistors unless you keep learning at the same time as you’re doing inference. You have to use these fancy substrate-like memories if they need to be permanent. But if you’re just using basic capacitors, like you have in DRAM or something, they require constant refreshing. So either you have to spend a lot of energy on that, or you want to keep the learning going forever.

The Thermodynamic AI Computing Chip

That way, they kind of automatically refresh. And that's probably the most similar to how brains work, right? They don't—they never just freeze. I don't know what Chomsky or somebody would say, but they aren't just frozen. I think the synapses and stuff are always adapting.

Tim Scarfe

Yeah, exactly. I think, out of all of the words we can use to analogize intelligence, adaptivity is number 1. In a sense, when we look at Claude, you can ask: Is Claude the model? Is it the ecosystem? I think it's the ecosystem, right? It is adaptive because—

The Thermodynamic AI Computing Chip

The personality or the persona or something.

Tim Scarfe

Well, you know, there's a tendency to anthropomorphize, but no, millions of people are using Claude Code. This is tangentially related to how we come up with causal abstractions in the world. The way we do it is, we are in the world and we are agents, and we can make decisions, reconcile uncertainty, and say, “What would have happened if I did this?” I can share it with my friends, and there's this wonderful percolation process where the abstractions just become embedded.

Now, that does happen with Claude Code, right? Because there are people using it on their machines. They're actually running tests, they're doing counterfactuals, and then all of this gets retrained in the next version of Claude, and then someone else will reuse that abstraction. So, we have the adaptivity as a system.

The Thermodynamic AI Computing Chip

It's a shared brain where all of the experiences go, where it's called the bulk or something.

Yeah, so it's interesting to discuss whether that's effectively the same. If we actually had some hypothetical, real-time adaptive, divergent Claude, would it be much better than the Claude that we already have?

It's also related to the work that you guys do because, in my view, the process of intelligence is the creation of these coarse-grainings—skills. That's kind of like what you guys are doing with ASICs. You're building this customized hardware for making certain types of computation go really, really quickly. It's almost like that's the result of intelligence. You say, “I'm going to take a very, very complicated thing, whittle it down, represent it in the best way I can, and then bake it into a non-adaptable hardware substrate.” Is that fair?

Alon Eyal

I think that is what we're working on right now: this idea of trying to build hardware that best fits the models we have right now and really make a super-efficient inference.

You could say, in the past at least, the issue has been that you stop yourself from innovating on the software side if you lock down your hardware too much. The NVIDIA chips have been pretty good for innovation. They're pretty flexible. Of course, they have also guided the way we do AI in a lot of ways, toward matrix multiplications and so on, but still, people have been able to innovate.

On the other hand, I think hardware is also getting easier and easier to make with these kinds of AI-for-EDA tools, so it's sort of shortening things. Maybe now people are thinking about making their CUDA kernels, right? It's not so unreasonable to think that soon we'll just be doing AI for making—instead of CUDA kernels, we'll just make some custom circuits for every single thing we want.

So, if you come up with a—if you want your new algorithm to run really fast, you just design a specialized circuit for that. AI helps you optimize it and check that it's correct, and of course, you still have the fab, but people are getting better at batching things for the fab.

I also think having really flexible hardware is going to be very interesting. The stuff where it's learning on-chip and, ideally, the more adaptability that it has and the less we need to bake in—the better. I mean, I don't know; it's just cooler, I think.

Tim Scarfe

Yeah, but isn't that a wonderful example of this recursive self-improvement? As you say, we're building the AI, and then the AI is helping us build better kernels, better software, and better hardware, which then in turn makes the AI better, and then you get this kind of loop.

Alon Eyal

I mean, this is recursive self-improvement in a way, right?

Tim Scarfe

Exactly. Exactly. But we should bring this to life. There's this concept called autoformalization. On your website, you say basically that you have done something similar to AlphaProof with respect to building chips.

Roughly speaking, AlphaProof is the system that won silver at the IMO in 2024. What they did back then was use a language model to generate a bunch of Lean code. It was obviously a little bit messy, so some of the formalization was done by hand, and they did multiple renditions and so on. Then they would do verification with Lean. You're doing something like that for chips.

Alon Eyal

Yeah. Yeah, it's interesting because there's autoformalization, which we define as taking human specifications and writing them up—turning them into formal specifications in Lean, for example. Then, of course, you also need the proof step, where you provide the proof: prove that your code, whatever you have, actually satisfies that specification.

I think some companies, like Axiom, for example, are very focused on this part, right? Also, in some sense, AlphaProof—that's also the main thing it did. It started with a formalization, and then the hard part was training the model to provide a proof or disproof.

I think a really nice trick in AlphaProof was that when they did the formalization of the proof, it didn't really matter if they got it right or wrong, because if they just asked the model to provide a proof or disproof, and they got it wrong and it was no longer true, then it would just prove that it was not true, or disprove it. You could still use it.

Of course, when they actually did the IMO challenges, they wanted the autoformalization to be correct, so then they did it by hand. But they didn't need it for the training, which I think helps scale it up.

I think we can do a similar thing with hardware, by the way. It's pretty easy to take some chip design and then come up with some properties that may or may not be true, and then train the model to try and prove or disprove that this thing holds. But that's all about creating the proof. Autoformalization, in some sense, is harder because it's harder to create the training data for it, right?

That's kind of also a story about AI in the last 2 years since reinforcement learning: anything you can create a good RL environment for, you can probably learn, but anything else is out of reach right now. Some of these chips have thousands of pages of specifications. If you want to turn that into a formal model, and you get just a couple of words wrong somewhere or a couple of numbers, then it doesn't work, or what you prove is not important or relevant.

I think it's always been an issue in the chip industry, and they've kind of tried to solve it by having orthogonal teams. They have one team designing the chip, one team designing the tests, and another team designing tests of the tests, where they look at coverage. They call it functional coverage, where they measure what the tests test and then check everything off.

Hopefully, if all 3 teams have read something the same way and understood it the same way, they have the right idea of it. You can try and do something similar with AI. You can argue whether it's really orthogonal if it's the same model that's doing each of the 3 jobs.

I don't think it is quite orthogonal. I do think there's enough entropy in these models when you do long runs, and you can definitely find a lot of bugs by just doing things many times and seeing if there's agreement between them. But it's definitely a trust exercise, and a human exercise, to figure out how we get hardware engineers to trust it and how we make it easy for them to verify that our formal model fits with what they thought it was.

You can come up with all kinds of different tricks, both to make it more visual—to show what your understanding is and whether it matches—and to ask questions and see if you agree with the questions, or show examples. You can also try to do some kind of back-and-forth. There are lots of tricks you can try, but it's an interesting problem.

I feel like AlphaProof and all of these other people never really solved this, because their math statements were just 1 paragraph; they were not thousands of pages long.

Tim Scarfe

Oh yes. Yeah, interesting. With AlphaProof, I think they wanted to use Lean 4, and there was hardly any stuff for Lean 4, so they created a converter from Lean 3 to Lean 4. They needed lots of people to fix it, but I guess my point is that they needed to fine-tune a language model on a ridiculous amount of Lean code, and then they were using Lean as an intermediate.

Interestingly, with their new model that won gold at the IMO, they weren't doing any verification at all—not in a formal sense. But is there a spectrum? The way you were just describing it, it's not binary, right? You were talking about test coverage and different perspectives—the blind men and the elephant—so there are functional tests and descriptions, and we can do visual inspection and so on.

Will it end up being a case where we're always wrestling with something we don't completely understand, but we're using as many signals as possible together?

Alon Eyal

Yeah, I think where we're going now is also trying to encompass more of the spec creation. Obviously, when we start out, people have already written down all of these specs, and we want to help them with those. But when you do that, you also miss out on the whole intent that went into it and the whole process of creating the spec.

You don't know if there's just some numbers somewhere in there, for example. You don't know why they chose those numbers and not some other numbers. I think that by internalizing more of the process, at some point you can at least be as safe as any human could have been about this chip. There might also just be some ambiguity that no one cares about.

Tim Scarfe

The ambiguity thing is interesting. There's a wonderful talk by Eric Curiel called “Math Does Not Represent.” He was talking about general relativity and four completely orthogonal representations of it. It was a similar thing with AlphaProof: How many ways are there in Lean to represent nonnegative numbers? Apparently, there are quite a few.

The Google guys were just representing the problems in different ways to stimulate the model. Is this almost the training data, in a way? If they auto-formalize the same problem in different ways, they actually get different problems, right?

Alon Eyal

Yeah, both in terms of the intelligence—how can we bootstrap the model to do better intelligence—but also in terms of the legibility and abstraction of the final output. I guess when people think of auto-formalization, we have this almost idealistic view that there is one true representation, and it's just going to be legible and everything. But it seems quite vague.

Tim Scarfe

Is there one true representation?

Alon Eyal

No, I definitely don't think there's one true representation. With smaller chips, like a floating-point or crypto chip, the specification is actually pretty simple for those. It's more when you get to the big, system-level stuff.

I think software is in many ways ahead of hardware. In terms of formalization, they're behind, but in terms of thinking about architecture together with AI, I think a lot of people now run their architecture documents by Claude or somebody and are like, “Hey, what do you think of this? Should we move things around here?” Boom, boom, boom.

I think this whole discussion helps the models understand what your intent is, what you care about, and what you don't care about.

Tim Scarfe

Just to give us an example and bring it to life, there was that DRAM article you published, Alon Eyal. The article was talking about timed Petri nets, and I did look this up on the internet. Apparently, it's a thing from the 1960s for describing distributed systems. Explain that.

Alon Eyal

What we talked about before with the formalizations is very much at the RTL level, at the very cycle level, where you care about proving that the exact thing happens at every single clock cycle. But, of course, there is another kind of formalism that's very important for things like adders, arithmetic circuits, or crypto.

A lot of the hard problems people have are more system problems, or protocol-level problems. Things that have maybe been more popular in software are proving things like, “This system can never deadlock or livelock,” or proving protocol-level properties. You want to show that all of the timing requirements between different things make sense and that there are no internal inconsistencies.

There currently exist different formal languages for these things. For the cycle-level stuff, people use SystemVerilog Assertions, or SVA. For the higher-level protocol things, there are more classic tools like TLA+, I think by Lamport or something. We try to formalize things in both ways because they're useful for different things.

It's possible that at some point they can all merge. If you supply it, it's kind of like doing very high-level math, and then you can reduce it all the way to the actions if you want to. But I think it's also interesting. It's a very new field, so we're trying to explore different ways, and timed Petri nets are one way to represent these superparallel systems that you have, for example, in these memories.

All of these different banks can be operating at the same time. If you activate one of these rows, you have to wait for the data to run down to the bottom before you can read it, for example. Then, after a while, you have to refresh it because these capacitor DRAM cells have to be refreshed all the time.

But you can also be working in a different memory bank. There might be lots of them on the chip at the same time. There are certain things that are shared between the banks, so you have interbank dependencies and intrabank dependencies, as well as bank groups. People build really crazy stuff in there.

Tim Scarfe

Very cool. What's really exciting to me is that it's possible to build chips that can do certain types of things orders of magnitude faster. That's why I want to talk a little bit about thermodynamic computing.

Instead of forcing transistors to settle at 0 or 1, you let noise do a random walk and bias it so the chip is a stochastic differential equation, right? That sounds crazy. How does that work?

Alon Eyal

Yeah, this was one of the things that really got me to Normal Computing in the first place. Before Normal Computing, I was at Facebook, as we called them, in the research group that does probabilistic computing.

We were doing Bayesian neural networks, where you assume probability distributions for all your weights and try to infer the posterior from the prior and the data that you look at. A lot of these techniques were slow because you had to either do lots and lots of repetitions with different random seeds, or you were trying to do it analytically. That maybe takes a little bit of the point out of the probability.

But then you have these chips, and the chip manufacturers spend so much time getting every single little piece of noise out of their systems and having extremely sharp margins for everything—so much precision. It's probably the most precise business in the world. And then what do we do with them? We just add randomness everywhere.

Why not try to build a chip that's inherently random? The brain probably has a bunch of randomness. The first chip we made was basically an array of capacitors with programmable resistances between them. You inject all of this noise and get it to behave according to these stochastic differential equations.

Then you think, “What can we do with that?” It's a new computational paradigm that I found very interesting to explore. One of the things you could do with it is that the matrix you put onto the chip in the stochastic differential equation actually behaves according to the inverse of that matrix. We could try to capture it and average it out.

Tim Scarfe

When we spoke about this on the phone, you said something very interesting. We often talk a good game about this. I was talking with Michael Jordan the other day, and we were saying, “We need uncertainty quantification. We need adaptive computation.”

Alon Eyal

Yeah, I think this was one of the issues we had. Bayesian machine learning was really strong for a certain amount of time, at a certain point before generative AI, because you had one output, and then it made sense to have a distribution as the output.

But now you have these sequences. You keep putting these tokens out, and I think no one really cares about the uncertainty of one particular token or about having a better distribution for that. You really want to know, after the model has thought about 10 different options, backtracked, and done all this stuff, and it comes out with a final answer, how much can I trust this answer?

You either have to go really deep into mechanistic interpretability to try to carry all of the uncertainty all the way through that, or you have to try to use some more anthropomorphic methods based on how humans would estimate their uncertainty and apply that at a really high level.

Tim Scarfe

Yeah, and—

Alon Eyal

And inside, you lose some of that beautiful math.

Tim Scarfe

Because it's actually really cool that you've got a Bayesian background. The way I introspect about this is: when you have a thought and an intuitive notion of how confident you are, it seems to be because you have a deep structure. You can introspect and rationalize and say, “Okay, well, there’s this component and this component, and there are these constraints I’m not quite sure about.” That seems to be the missing link.

Alon Eyal

Yeah, we actually did some experiments back when transformers first became popular. This was before we knew exactly what the company was going to do, or that it was going to be AI for hardware, so we wanted to do predictions, and we just sort of built Bayes’ law into the model. For a particular question, it would try to find lots of pieces of evidence, and then it would say, “What is the probability that I would see this evidence if the answer is yes, and if the answer is no?” It would do that lots of times, and then at the end you could use Bayes’ law to say, “Okay, then what is the probability that the original statement was true or false?”

It actually worked really well. I did an internal prediction game, and it beat everyone. We had 2 versions. One was more neuro-symbolic, you could say, where you would just have the model come up with all these probabilities and then manually calculate it. I also tried another version where I just told the model, “Now you use Bayes’ law and do it,” or something, and somehow it actually did a little bit better.

I don’t know how it did that. Maybe it was just able to look back and think, “Actually, a couple of these values I probably bullshitted, and I should just ignore that.” It’s interesting. I know there are some people training LLMs, and there are some benchmarks now where people are trying to predict Polymarket and this kind of stuff with LLMs. They have benchmarks for that, but I’m not quite sure whether they use these sorts of techniques or whether it’s all just end-to-end reinforcement learning and they hope it just picks up a good methodology by itself.

Tim Scarfe

But it’s a wonderful example of this kind of activity-specialization loop.

Alon Eyal

Yeah.

Tim Scarfe

So, for the first month, there was a problem with vibe coding. There was a false positive every single day, and I just created a skill. Then Claude would retrain it, and eventually it just kind of converged. This is the bull case of vibe coding, right? You fix it every single day, over and over, and eventually you’ll land on the right track. It works really well.

But in a way, this is similar to what you guys are doing, right? You have this outer loop that takes something complex, then compresses it and optimizes it, and you bake it into a kind of crystallized hardware.

Alon Eyal

Yeah, I was thinking about this older tweet by Elon Musk. I don’t know if people were laughing at him. He tweeted it out, but he was saying, “Why don’t LLMs just write the binaries directly, or the assembly?”

Tim Scarfe

I saw that.

Alon Eyal

Yeah, I don’t see why we wouldn’t be able to do it. The question is whether we would want to do it. I think there are some fundamental computational problems everywhere in the world. In recent years, we haven’t focused so much on them because we’re so excited by AI and interested in all the stuff that AI can do really well.

But obviously, things like cryptography or some basic algorithms—you never want the LLM to just do it, even if it can do very large-number multiplications. It’s just super-inefficient; you might as well optimize a circuit or a piece of code for that. Compilation has some of the same problems, and also chip synthesis, where you’re trying to explore all these different designs and so on. You don’t necessarily want to have the LLM do it because it’s so slow versus some super-optimized loop.

Not to jump too much into things, but it’s interesting to think about chess again. Of course, you have the AlphaGo thing where everything ran through a neural network, right? But today, the state of the art in Stockfish, of course, is that they took a more hybrid approach. They took the neural networks and made a certain type of neural network that can update really fast when you change the state, and then they combined it with just super-fast search. It actually outperforms the best open-source AlphaGo-type chess engines.

Tim Scarfe

Oh, really? Using adaptive fine-tuning and structured inference?

Alon Eyal

I don’t know if you’d call it adaptive. It’s more like they took the classic chess search engine and replaced the evaluation function with a neural net—a very shallow, wide neural net that is really, really fast to evaluate.

I think that for something like synthesis and compilation, there’s a similar thing where you could try to do it all with LLMs, but at some point the speed is a bottleneck. You can get a benefit from having more knowledge and more intuition and all of this stuff, but at some point there’s also just a hard computational problem where you want to brute-force some stuff. At that point, you want to be able to switch to a more classical algorithm.

Tim Scarfe

Absolutely. Now, before we were talking about thermodynamic computing, and you folks have done some work that has huge potential for things like Markov chain Monte Carlo, I think, and diffusion models. But you did say to me when we spoke last time that in some cases it can be a false economy. For example, you could have a diffusion model that might have a different type of neural network on the end of it, and you might find that the benefit you get from doing the diffusion might be bottlenecked by another part of the model. So, in practice, where can we see a huge uplift?

The Thermodynamic AI Computing Chip

I think it’s always interesting when you build hardware, because there’s always this co-design problem with the algorithms, right? The algorithms are so based around the GPUs and the hardware we have now. You can try to target bottlenecks. You can make hardware, like we talked about with the thermal DRAM, that’s super-efficient for memory. That might speed up inference, but it can only speed it up so much.

Say we have 10% GPU utilization now for inference: you could hope to have a 10× improvement there, but you don’t know, if you suddenly have access to that much with a different architecture, what new algorithms you could build if you really went all in on that.

There’s a similar thing with diffusion, right? The architecture isn’t that efficient in terms of randomness. With GPUs, people don’t really want to sample tons of Gaussian random variables everywhere. So they may also build architectures where they try to remove that bottleneck by not focusing too much on these things. We can build new hardware that makes that more efficient, and that has some performance gain. But to really make the most of it, you then also have to come up with new models that go all in on that.

It’s interesting, these Notion apps. Have you thought about this? All of the apps now want to be your central AI. Notion has this—they want to integrate with the other apps—and Linear has an AI, and they want to integrate with the other apps. Everyone is trying to capture being your central AI assistant, and then all the other apps will just be tool calls.

I actually use it a lot because it’s got an amazing agentic interface. It’s got a CLI interface, so I use it from Claude. What they want you to do is pay them to use the agents that are built in at API prices. I don’t really see the reason for doing that when I’ve just got a Notion MCP. I’ve got a Notion CLI.

The Thermodynamic AI Computing Chip

API pricing is really killing innovation, I think, in that space. I think Codec—they’re actually opening up more so that you can use your accounts. I saw just yesterday, I think maybe Anthropic also opened up something where you could use more of the SDK with your accounts. But the price difference is so big that if you have to use API pricing, nothing is competitive.

Tim Scarfe

Well, we don’t have to go too deep into this, but I think creativity is all about respecting constraints. You studied linguistics at Oxford, and Chomsky always says that there’s a difference between linguistic competence and linguistic performance. He’s been using this bulldozer analogy for ages. He said, “I love bulldozers too.”

They’re great for clearing snow. They’re not a contribution to science. He even said about Deep Blue that a computer winning at chess in the way it did is a little bit like a bulldozer winning the weightlifting competition at the Olympics.

The Thermodynamic AI Computing Chip

I think it was, in a way. With Deep Blue, maybe modern chess AI is a bit different. I actually worked on chess engines for 15 years. I remember him saying that too, and I was a bit surprised, because you’d think that if your science is biology, or you want to understand how humans do language, then maybe LLMs aren’t so relevant to how they learn language.

But if your science is more about the abstract concept of what language is, I thought you would be really interested in seeing different systems developing language, comparing them, and seeing what’s common, what’s the same, what’s different, and so on. That would give you a wider understanding of the concept of this thing.

Of course, you have to have some kind of respect for AI language to even include it. If you think it’s completely a stochastic parrot thing, maybe you don’t really care about it and don’t want to include it in your model of language. But if you do think it’s actually doing language, then I don’t think it really matters if it does it the same way as humans or not. Maybe it’s more interesting if it does it in a different way.

Tim Scarfe

You mentioned chain of thought, by the way. How much can we read into chain of thought? Some people just call it “chain of thoughtlessness,” like Subbarao Kambhampati. But actually, it is probably the modus operandi now for doing interpretability—for actually understanding what they’re thinking.

You could argue that chain of thought is like the press secretary, not the orchestrator, so it’s almost a post hoc confabulation. But that’s not quite right, is it? Because that sounds a little bit like there’s no causal link between the chain of thought and what the language model outputs. That’s not true. So how much can we read into it?

The Thermodynamic AI Computing Chip

You want to give the model somewhere to think, right? When you do reinforcement learning, the chain of thought before reinforcement learning and after reinforcement learning is very different. Before, you tried to prompt it, and some tricks worked and some didn’t. But once you do reinforcement learning, it needs to have infinite memory and be able to have something to operate on.

It’s like a Turing machine, right? Whereas when you just had the transformer and single-shot output, it was, in the Chomsky hierarchy, a completely different type of system—an automaton or something like that, where there’s only a finite amount of computation it can do. Now it can do as much computation as it wants; it just has to learn how to do it.

You could imagine building a really simple LLM-type system, and with access to chain of thought, it would be a universal Turing machine. At that point, it’s just—no, because it can have this state, and it can keep reading and outputting. I think it would be pretty easy to make a structure like that. The question is whether it can learn it, but at least now it has the representation capacity, so it can do it, and clearly something is improving and working.

Tim Scarfe

This is actually the bigger problem with the ecosystem: software is always designed to reduce complexity and introduce canalization. Spreadsheets are a great example of this. Accountants use spreadsheets in finance, everyone uses spreadsheets, and they create an interface that everyone uses, reducing complexity in the system.

Agentic AI just creates spaghetti everywhere. This is part of the reason why people aren’t shipping, because it creates some ephemeral complexity. It’s a little bit like Bash scripting on steroids. I’ve now created this web that only I understand, and it’s becoming more and more specialized over time, so I can’t share it with other people. The entire ecosystem is becoming very messy.

The Thermodynamic AI Computing Chip

I definitely think a lot of things are breaking now because of that. A lot of the open-source stuff is like—everyone just writes new code from scratch instead of trying to come together and hone these shared libraries. For a lot of things, it is more efficient than going through everything else, because you can get everything just the way you want it.

I actually think it’s very interesting, what you’re talking about with spreadsheets and this canalization, because there’s a lot of domain knowledge inside these tools. People who build these tools might be worried about how easy it is to extract that knowledge, with people building clones and so on.

Everyone these days is locking down all of the data, because they’re realizing that data is becoming so valuable. They want to keep it themselves or build something themselves rather than have other people use it. I think it could be a similar thing with a lot of specialized tools, because the tools basically are data. They’re developed over so much time, with all the right patterns found and everything built in there. So how do you stop other people from going in and agentically extracting and distilling it?

Tim Scarfe

I’m sure this has happened to you. Random people on the internet will say, “I’ve just generated this paper. I’ve just generated this code. Have a look at it.” There’s a massive amount of AI psychosis out there, where you do things that are slightly outside your domain of expertise, and Claude will convince you that your stuff isn’t mediocre—it’s great. Then you share it with other people, and they immediately see through it.

I think this is a serious problem as well. There are experts out there who have really clear ideas about things. They’ve been doing software engineering, and they’ve been in science, and when they use Claude, it’s brilliant if they’re diligent, because they can use good abstractions and representations. But there’s now a tsunami of pollution out there.

The Thermodynamic AI Computing Chip

Yeah.

The Thermodynamic AI Computing Chip

And yeah, and it breaks the social contract, right? In the past, if I wrote something and asked you to read it, you could at least assume that I had spent 10 times more time writing it than you would spend reading it. Now you’re really skeptical about anything, because why would you want to spend time reading something that the other person didn’t even read themselves, maybe?

The Thermodynamic AI Computing Chip

Yeah, and it’s not really fun. Have you talked with anyone about that? People talk about needing a GitHub social-credit system—a karma system—so people can downrate you. If you don’t have high enough karma, maybe they just won’t read your pull requests.

Tim Scarfe

We do need that.

The Thermodynamic AI Computing Chip

Yeah. archive recently put gates on people uploading things there as well.

Tim Scarfe

Right. Yeah, and they had this ban. It’s like a one-year ban if you have hallucinated a—

The Thermodynamic AI Computing Chip

Yeah. But it’s sad in a way, because it makes it harder, especially for new people and young people, to break into that stuff. They don’t have any karma or anything to show.

I know, but I think the broad reason we have this problem is that this technology is the most deceptive thing ever created in human history. It’s a serious problem because it’s all about epistemic subjectivity: you generate things that you don’t understand, it convinces you that they’re correct, and you can’t see the glitches.

Obviously, an expert can look at it and see the glitches straight away. But it also creates dependency. When you start posting things that you know you don’t understand, you want to be consistent. You’ve now made a statement that you know about this thing, so you’re going to keep doing it.

Weirdly, people are very defensive about it as well. If you criticize work that they’ve done with Claude, they take it personally. It just creates this perpetuating cycle.

The Thermodynamic AI Computing Chip

It creates a feeling of understanding that you might not actually have. It’s like doing an exam or copying somebody else’s work: you might feel like you wrote it, but you didn’t. Your brain didn’t go through the motions.

I think it’s dangerous if you’re worried about AI taking over, because it’s not just that it’s getting smarter; it’s also that humans are getting dumber.

Yeah.

The Thermodynamic AI Computing Chip

We get lazy in terms of understanding things. We don't read the papers; we just put them into AI and say, “Explain this paper to me,” or something. But, on the other hand, it can speed you up so much that it's hard not to use it.

It's about finding out when you should stop and when you should start using it again.

Tim Scarfe

I know. It's reminiscent of Elon Musk's tweet where he said that there are no researchers at Tesla, that there's only engineers. This, for me, is the big problem, actually. It's a paradox because you can use language models to increase your knowledge. That's a fact, right?

If the purpose is to increase your knowledge, then, if you're a curious person, you can just dig and dig and dig, and you can learn a hell of a lot. So why is it the case that, on average, they erode our knowledge?

I think it's Elon's fault—but not his fault, you know. He says that engineering is a means to an end. We are building this thing, and it needs to pass these tests. In a sense, I don't care if your knowledge erodes during the process, because that's not what I'm measuring you on.

We shouldn't be binary about it, because clearly engineers trip up on things and learn a lot along the way. But it seems to be quite convergent: when you're pursuing knowledge for its own purpose, you seem to build deeper foundations and discover new things.

The Thermodynamic AI Computing Chip

I guess that's a tension with capitalism or something. Companies aren't necessarily trying to develop their employees. Maybe they will if they can see a profit motive in it, but at the end of the day, they just want to get the job done.

Of course, if you spend a lot of energy developing your employees and then they leave, it's always a bit tough. I don't know if there's a way to force it.

I think it's also a little bit self-inflicted, because learning things is just hard. It can be frustrating, and it doesn't always get you into the same kind of flow as just prompting or something.

I've been trying to use this idea that I think Capaci suggested. When he was learning new things from an LLM, he wouldn't copy and paste them. He would have side-by-side windows, but he would always write all of the code by hand.

Yeah.

The Thermodynamic AI Computing Chip

I made an internal app to try and help with this, because we hire a lot of AI engineers. It's hard to find people who are really good hardware engineers, really good AI engineers, and also software engineers, so of course you have to be all of those.

We have to scale up hardware people on AI, and we have to scale up AI people on hardware. So I made internal training tools and things like that, where I try to have the same blocks in there, if you like, while trying to block copy and paste, and at least give people a warning saying, “You should maybe try to type it yourself.”

You wouldn't think it would make a difference. If I'm just looking at it and typing it myself, why would I learn it better? But it's just empirical that, somehow, our neurons and stuff make it important to go through the motions.

You have to figure out which projects you need to understand in depth and which ones you're just making, where it doesn't matter as much. I think we also get tempted by AI to start way more projects in parallel, right? I can just start another one while I'm working on that, and I'll start another one, and another one, and another one.

Then suddenly you're thinking, “There's obviously no way I could actually understand all of these in depth, because now I've started so many projects.” So you have to stop yourself and say, “Maybe instead of starting the fifth project, I'll go back and try to understand what's actually going on in the first couple of projects.”

I think maybe there'll be some evolution. We're still learning all the social stuff around using these tools, what's effective and what's not effective. Maybe we can even build some things into the tools. Is that what you're saying also with the scoping, to try and make them better?

Another really hard social problem right now is how to do teamwork. That's something I also think about as a manager. If everyone on my team is staring at their 10 agents working on something, when are they going to go and talk to their colleagues about it?

If they don't understand the code themselves, how are they going to explain to the other people in the company what the code does and build together? I don't think anyone has really solved it well at scale.

I heard a podcast with the Anthropic co-working team, and at least the way I remember it, the podcast asked, “How did you make this so good?” They said, “Everyone on the team coded their own version, and then we picked the best one.”

I was like, “Damn.” If that's the best we can do in teamwork, it's kind of—well, in a way, it's cool, because it's like the ensemble method applied to people. You can explore so much more.

But it also removes all of the teamwork. At that point, it's just everyone working in isolation; you're just duplicating. I'm curious how we're going to solve that.

Tim Scarfe

It's performance versus competence, and I guess we're collectively making a bet that performance is all that matters. The Thermodynamic AI Computing Chip, it's been amazing having you on the show.

The Thermodynamic AI Computing Chip

Yeah, it's been so much fun. Thank you, Tim.