AI:AM：这就是我们想要的 AI 吗？令人警惕的加速、AI 选举押注、GPU 交易与 AI 原生风投

1. Imminent means now (Part 1)

This week on AI in the AM, the reports of acceleration keep coming. Is recursive self-improvement already underway, and is dramatic acceleration imminent? Friday morning, the day after METR published its report on how much AI is already speeding up AI research inside Anthropic, the pace of things is definitely pretty relentless. It might be time to spend more time with our family, because it does feel to me like basically the AI 2027 timeline, more or less, is coming true.

Pash, my co-host, and I were talking about the local ballot measures against data centers and how the developers have been going about buying land.

One of the things the companies have been doing is the old-school oil-and-gas landman way of acquiring land. They go in and acquire land at pennies on the dollar using LLCs, and after that they merge the pieces of land. Then they're like, “Oh, Meta is here.” And people are like, “If I knew I was selling to Meta, I would have asked for more.” So people feel cheated, right? If you have political willpower, all contracts are worth dust.

Mike San Miguel of Luxor Technology. He sells GPU servers through the company’s hardware desk and set up its KYC department.

There are some compliance issues. These things are traded more closely than some nuclear secrets. Every OEM is a little bit different. Nvidia works closely with all the OEMs around allocation and around greenlighting orders. Nvidia also typically checks every 6 months and makes sure those GPUs are where they’re supposed to be. They’ll physically come on-site and make sure they haven’t left.

Brian Zhou, a researcher at Thoughtful, helped build BallotBench: 9 frontier agents, each given a week to forecast the midterms, and trading agents putting real money on their forecasts.

2. Agents forecasting the midterms

It takes forecasting agents a bit of time to see an election result, communicate it to their trader, and so on. By that time, more programmatic or deterministic algorithms and bots have already traded on this information. This is pretty indicative of what we were seeing: the traders were doing quite well the week before Election Day, and then on Election Day they simply didn’t react fast enough. Sometimes they made up incorrect information, which resulted in the traders losing a lot of money.

Bryan Bischof, head of AI at Theory Ventures. Pash asked about the firm’s 2026 builder survey. Bryan expected open-source models to gain ground in production. Instead, the balance shifted toward closed models.

There is a lot of acceleration in model drops. Right now, tying your production service to open-source models requires a little bit more thrash in the engineering. I heard from a couple of folks specifically, “Yes, but the open-source models change so fast. It’s a little bit more convenient to pin to an OpenAI model.”

Part one: Imminent means now. Friday’s show opened on two documents, the METR report and an amendment to the Anthropic S-1, the filing for its IPO. The new language: Anthropic says it may imminently meet its own threshold for dramatically accelerated progress attributable to AI.

Anthropic updated its S-1, the IPO filing, and in that update they added language saying they may imminently see recursive self-improvement. I’ve always wondered why they’re doing the IPO, and why so quickly. My take has always been that I think within the company there’s a sense that you need more controls, and more people have to have exposure and more control over this. One way of getting this out there is getting the IPO done, so that you give a larger part of society hooks into it and the ability to express their desires through the market. To me, “imminent” is basically a month.

Usually, what I assume when I hear these sorts of statements is that it means now. The best model I have of Sam Altman and OpenAI leadership is one I formed during the 6 months between when I started testing GPT-4 and when it was finally launched, watching very closely all their statements during that time to try to figure out how to read them in general, given the inside information that I had.

The best way I can interpret them is that they are speaking very literally and dressing it up as vagueness, or layering on uncertainty to try to—I don’t know what exactly—keep the secrets a little bit longer while telegraphing the main message. It’s a strange strategy to me, honestly. I really don’t know why they do that. I would interpret this the same way from Anthropic. I think you can see how this would happen. We have so many results now with all these galaxy-brain zero-day exploits. OpenAI just dropped some huge number of vulnerabilities in the Linux kernel—I think about 1,000 of them.

You know, so much for “many eyes and many hands make light work.” The idea with open-source software over time is that because there are so many eyes on it, vulnerabilities won’t last. They’ll be spotted and patched. That was the whole promise and virtue of open-source software. It basically worked while the attackers were all human.

It’s almost certainly the case, by analogy to Linux and by analogy to math, that there’s a lot of low-hanging fruit. A lot of low-hanging fruit means acceleration. It means you’re going to get more value from compute and find more ways to get value from data. A big part of the reason I think they’re sharing so much of this stuff is to signal that the same thing is about to happen on the ML stack. This is now a really credible threat, and “What are we going to do about it?” becomes a sociopolitical question as much as a technical question.

In my mind, the technical question is mostly answered: there’s going to be a period of pretty dramatic, fast progress. Does that top out on an S-curve? My guess is still probably yes. I don’t think it goes exponential forever, but who knows? As I think it was Rosie Campbell who said, the S-curve can stay steep longer than you can stay relevant. My expectation for the next few months is that we pick a ton of low-hanging fruit—the models pick a ton of low-hanging fruit in the stacks used to train them. As a result, we do see really fast progress in the short term. Maybe that levels off, but it probably levels off where we have AI R&D at a weekly superhuman level. How that plays out over the course of 2027 is really, really hard to predict.

I agree. I think “imminent” means now. I was also sandbagging with a few weeks to a month. What it means is that they have the tools available, and they can see a path to implementing RSI pretty much any time they want. A lot of the pieces have been built, but they haven’t pushed the button yet. They waited for the election. They’re going to do the IPO, and then they’re going to announce some of the new things that they’ve discovered. Then you’re going to have this gap up, and we’re going to have the Christmas break. During that Christmas break, we’re all going to digest all this new technology that we have access to. Then we’re going to come back in January, and we’re going to see the takeoff as people deploy. That’s somewhat similar to what happened at the end of last year. Well, here we are in the early takeoff. I’m somewhat wistful, but nostalgic for the human era. This is the end of our dominance, at least.

I’ve also made my peace with the idea that intelligence in the world makes the world better for me. If I had people of Apple’s intelligence—people like those who designed the iPhone—design my bathroom, cars, or home, I would be happier. That infused intelligence in all the products you see makes your life better. So, yes, it’s going to make everyone’s lives better, but it is the end of the anthropocentric era.

Mhm. It’s going to make everyone’s lives better as long as we can keep everything on the rails. I do share that optimism. I just got back earlier this week; I went to Bletchley Park, as we talked about a little bit yesterday. I was sharing with this audience of enterprise tech leaders just how much work we still have to do to keep the AIs behaving well.

I shared with them my son’s cancer story and how much I used AI in that experience, and how valuable it was. I would have paid a hundred times as much as my $200-a-month subscription because the value to me at that time was absolutely at that level—$20,000 a month. Literally, there was no question I would have happily paid, because I was getting that much value. I do believe in the upside, and I hope that’s beyond question at this point. But there are so many things we’re really still just beginning to grapple with.

Another example from OpenAI yesterday: they showed that models sometimes prompt-inject and fool their graders. You have models creating, in some instances, fake files that they label as having come from an instructor, trying to present some of their own output as if it were human instruction to the grader.

The grader picks up on that in the context of evaluating the model’s work and takes on board the instructions that the model itself was trying to trick the grader with. This apparently hasn’t happened that often, but they found instances of it. To Anthropic’s credit, they did say in the filing that they don’t believe they can do it safely on a fully automated basis. I think it’s honestly quite obvious that they can’t.

How are we going to show the collective self-control and restraint not to tip over into that regime? I really don’t know. On the one hand, it’s super easy to just not do it. Nobody’s putting a gun to anybody’s head saying, “You must do recursive self-improvement here.” We can’t remember that often enough. And yet there’s so much momentum, and it seems like they’re very determined to do a version of it, at least, that still seems pretty risky to me, or certainly a lot riskier than I would personally endorse.

We recently launched and have a paper coming out about BallotBench, where we had AI try to forecast the 2026 midterms that just happened, as well as upcoming elections around the world. Maybe let’s start with the methods. We had 9 agents—basically all of the frontier models offered by each lab—get their own Modal sandbox and work for 7 consecutive days leading up to the election. We gave them additional resources in the form of a $500 budget they could use to request access to paywalled data, news sources, subscriptions, and so on.

They were given the overall objective of submitting predictions for all the races on a given list: who would win, the odds of victory, given an 80% margin of error, and additional predictions for the margin of victory. For example, would the Democratic candidate win by 3 points or 7 points? The agents were tasked with giving incredibly granular and detailed forecasts, and there are a lot of interesting things we’ve observed from this. It’s a really interesting test of long-running agentic behavior.

In particular, what we see with some of the Muse Wonder models is that they’re often incredibly bad at long-horizon work. When given a task in the Muse Code harness, with a list of, say, 800 elections to predict, they’ll create a random forecast, which is just a random skew off the 2024 election results, and submit it within the first 2 or 3 minutes of working.

I’m already intrigued by your description of the Muse agent seemingly phoning it in. One initial question on that point: how do you reconcile what you saw with what people seem to be reporting online, which is that they’re having pretty good experiences overall?

No, for sure. Our initial theory is that this is mostly due to part of the harness design. We’re not talking about Meta’s personal agent; we’re talking about their coding agent. One thing we noticed is that the design of the Muse Wonder model, when paired with Muse Code, Meta’s native harness, essentially includes a task list.

Even though the prompt instructs the agent, “You have 7 days, unlimited API usage, and full permission to fan out subagents—do whatever you need to get the best possible results on this task,” what we observed is that the harness creates a list: “We need to check and make sure we’ve submitted all 800 or so different races. We need to check and make sure that all the forecasts have been accepted.” Then there’s essentially a goal condition that, once met, ends the turn prematurely.

We actually had to force the agent to continue working. Whenever it chose to stop, we revived it with a new sandbox, a new container, and access to its old runs and messages. I think this is more of a harness-engineering question, as well as a general model-behavior problem. But it’s definitely a different result compared to what we see with much more frontier models. When we see Opus 5.5 or GPT-5.6 Astra, their behaviors are vastly different.

For example, on a given day, Opus 5.5 would fan out something like 170 or 180 subagents and manage them by delegating work. It might ask one to look into polling results for a specific state, or to do a detailed review of what each ballot initiative in California, like Prop 40, is doing. Then it uses those subagents to help parallelize a lot of the work it wants to do.

I think this raises a general question about what agents do, especially with longer-horizon, open-ended tasks, and about their proactivity and creativity. One thing we were excited to see was whether agents would do something outside what we’d expect an expert forecaster to do, like putting out their own poll or finding more creative ways to get information instead of just fetching data and building models. We don’t see any major instances of that happening yet, but we see a lot of interesting approaches from individual agents.

For example, one thing GPT-6.1 Soul does is take the forecasts of every election modeler it looks at, then use the past decade of their forecasts to calculate how right or wrong each forecaster was. It can then use that result to aggregate the different election models those forecasters create, whether it’s Nate Silver’s model or others like 50+1. It can adjust those models based on how it expects them to underperform or overperform, as they have in recent years. We’re seeing a lot of creative methods like that.

Tell me about the results. How did the models compare against each other? How did they compare against top humans? Do we now have superhuman forecasters?

The overall result we’re seeing right now is that, in general, models perform just as well as the best forecasters. For races in the House and Senate, agents perform slightly worse than the markets but better than the modelers.

And in the Senate, the best agents, like GPT-6 Astra and Claude Fable 5.1, perform below the best forecaster, Nate Silver, but ahead of the markets. But all of these are within a very close margin of error. It’s important to note that a lot of the results for very competitive races have not necessarily come out yet.

Right now, there are still around 100 to 200 races across the House and state governorships, and there are still 2 races in the Senate that have yet to be called. So these are very preliminary results. It’s important to note, though, that for a lot of these agents, the methods they use include aggregating Nate Silver’s decision model or prediction markets. So it’s not as if they’re suddenly just spitting out a number that isn’t based on the numbers they’re seeing.

But we also find that these models are very well calibrated. For example, if a model assigns an 80% chance of a candidate winning across all the elections where it assigns a similar 80% chance, it’s right around 80% of the time. That’s essentially how you measure the calibration of these models, and we find that the best models are quite well calibrated.

We’re more excited about what this means for later models, because unlike experts or prediction markets, there may be only $1,000 of spread on what we might see in a very local race down the ballot. Expert modelers may not be forecasting individual state Senate or state House seats. This is where agents shine: modelers don’t have the attention, and there aren’t enough people betting on these much smaller markets.

I was going to ask about the relationship between prediction and trading. I saw a note that indicated Fable 5.1 had the highest Brier score on its predictions but still didn’t do so well in the markets. That sounds like the kind of thing we might learn something from.

Yeah. Part of the report will also cover this multi-agent behavior, because we essentially have one agent working as the forecaster and one agent working as the trader. We set up infrastructure that allowed them to communicate with each other.

We’re seeing a lot of instances of models making up information, parsing information incorrectly, and other broader issues, which partly explains why the traders—the actual trading agents—didn’t do too great.

Thoughtful also leads Post-Train Bench. Give an agent a base model, a GPU, and a time limit, and see how well it can post-train the model. With new results still to come, I asked Joe what he expected next in light of the METR report.

What is your expectation, as somebody who’s spent serious time looking at what models can do to train other models? Can you add a little more color or nuance to your expectations for the next couple of quarters?

Yeah. I think one of the general trends we’ve been seeing, both with yesterday’s report and with earlier reports, whether that’s Anthropic’s measurement of how much individual researchers have been uplifted by AI or metrics like lines of code written and experiments done, is that today—and this has already been true for many months—researchers have agents that can do the vast majority of the implementation work, which is what we’d consider research engineering.

The realistic limitations researchers face today aren’t necessarily their ability to execute on their ideas. As many people have talked about, it’s their research taste in finding what problems and experiments they want to pursue, as well as the speed at which they can come up with them, given that agents are already autonomously building and executing many of the experiments they want to run.

So, regardless of whether we get to recursive improvement, we’ve already seen a huge uplift, and that will only continue to get better. I think the bottleneck ultimately becomes resources and the quality of ideas and intuition, which is, of course, the next hill everyone is trying to climb.

Have you seen anything that suggests the distribution of idea quality is starting to overlap between models and truly top researchers?

I don’t think so, necessarily. But I do think we’ve observed that better models are more willing to fan out and do experiments, and are better both at thinking of interesting experiments and at executing them. With worse models, we see more of a general linear approach, whether that’s in post-training or other tasks.

With more capable models, we see this behavior where they spend their first few hours trying to fan out, do a bunch of different experiments, allocate their resources in an intuitive way, and do what we might expect. None of this is necessarily a Move 37 moment, because a lot of it is just the agent thinking, “Let me try an initial SFT batch and see how that affects the results,” for example.

So we don’t necessarily see anything indicative of superhuman taste. But we definitely see that more capable models are better at long-horizon work and at the sort of autonomy we’d expect from better researchers.

3. Paying up for compute (Part 1)

Great to meet you. We’ll be looking out for the next version of Post-Train Bench. You’ve got me wondering what the agents will say about how much acceleration we’re going to see in AI over the next couple of quarters. If they can predict the election as well as the experts, we should probably start asking them about the future of AI itself.

4. Paying up for compute (Part 2)

Part 3: Paying up for compute. Thursday’s show came 2 days after Election Day, when voters on 41 local data center measures sided against the data centers in all but 1. Prakash went first.

I actually think AI data centers have been very unpopular precisely because they’ve been seen as, number 1, Trump; number 2, corporate corruption; and number 3, this miasma of rich people making money and having plans to throw the rest of us out of work.

I think this is an entanglement with the population just not feeling well taken care of. If AI data centers are such a big issue, do you really believe that if Trump had a 60% approval rating, unemployment was low, and gas prices were low, you’d see this? I think the answer is no.

So I don’t think data centers were actually a big issue, but I think they were an issue to tar the president with.

But at the same time, we do have these ballot initiatives. From what I’d read, around 40 ballot initiatives is one of the bigger waves of ballot initiatives. These sorts of things only come around when you have something like gay marriage, where there was a whole wave of initiatives. There was a similar wave to affirm or protect abortion rights in the wake of the Roe v. Wade overturning.

This does seem significant. What I like about ballot initiatives—I’ve long been a big fan and advocate of direct democracy—is that, not to say people never make mistakes, you can unbundle your concerns about a particular issue from the broader political conversation. The fact that we saw so many of them, and that they all won, I guess with one exception, seems to me like a sign of things to come.

We’ll see. Maybe we’ll get companies cutting people in on the deal. We’ve talked about that a few times, too, and people should get paid. I’m definitely on board with that. But doesn’t it seem like there’s a pretty real signal there to you? It’s not the biggest thing, but it does seem like there’s a serious groundswell of people willing to work, willing to go collect signatures.

That stuff is not easy. You have to stand on street corners and get strangers to sign petitions. And then it still has to pass on top of that. So the fact that so many of those were organized and so many of them passed does seem like a big signal of what we can expect.

I agree, but why did they sign, right? Do you think they signed because it’s the end of the world?

No. Yeah, I think your diagnosis was pretty right.

Right. So I think the outcome of this is that if you’re going to be Google or Meta, you’re going to be paying Google or Meta prices for everything: the land, the labor, the construction.

Our Thursday guest was Mike San Miguel of Luxor Technology, which came up in Bitcoin mining and now trades new and refurbished GPU servers. When he says $550,000 to $650,000, he means one 8-GPU server.

So what is trading frequently?

Right now, it’s mostly B300s.

B300s are trading frequently?

Yeah.

Do they trade as a chip, as a node, as a rack, or as 8 together? How does that work?

Great question. Typically, for these servers, it’s an HGX system: 8 GPUs per server, a validated build built by an OEM. So you’ve got the HGX baseboard, dual CPUs, 32 sticks of RAM, depending on the size, as well as shared storage and east-west NICs.

Do people trade used systems with you right now?

Mostly for B300, it’s new. Occasionally, we’ll see abandoned orders. They’re new enough and profitable enough to rent out that we really haven’t seen anything used hit the market yet.

How much does one of those cost, off the top of your head? What’s the asking price or clearing price right now?

Depending on the specifications, the OEM, warranty terms, and all of that, you’re looking anywhere from $550,000 to $650,000. Typically, for bulk orders, it’s on the lower end.

People are trading new systems. What’s the typical lot size? Are people trading one rack system, 10 rack systems, 100 rack systems?

The units per rack come down to the data center configurations, because that’s about how much power can be delivered to each rack. Typically, most data centers will be able to operate 2 per rack right now, unless you’ve got a really nice, dense, supporting DLC data center. In terms of sizes, we trade everything from 1 or 2 units up to about 128 or 256.

$50 million trades?

Yeah, they get pretty big. The numbers are pretty aggressive, but it’s the scale of the buildout that’s happening.

Prakash asked whether those prices track the hourly rental indices.

One of the areas where we help our customers understand the market is the difference between a contracted rate and an hourly rental rate on these GPUs. If you look at an AI factory contract rental for B300, the typical market price right now is $4.20 to $4.50 an hour, depending on the market you’re in and the configuration.

If you’re only getting $4.20 per GPU per hour, that’s a very different rate from what you’re going to see on those indices. That’s $8 to $9 per hour per GPU. So if you have the capital to go out and buy a lower number of nodes and deploy them, typically you can get a better rate of return, as opposed to buying a full AI factory and negotiating an offtake contract with a neocloud.

Why is there excess inventory? These things are pretty expensive. I assume that in order to buy them, you have to put down a prepayment before they start manufacturing them for you.

Yeah, you do. Part of it has to do with data center availability. This market has looked like a series of different bottlenecks. Right now, it’s liquid cooling and data center readiness. That number I gave you, with most data centers being able to support, let’s say, 2 nodes per rack, has to do with constraints not on the actual hardware side, but on the data center’s power delivery. The cooling buildout is also a bottleneck right now.

We’re starting to see liquid-cooled data center inventory become available in January, in Q1, but most of that is already sold out. A lot of data centers are now telling us it’ll be Q3 2027 or 2028.

Are people trading the market? Are they buying systems without having places to plug them into?

No. Typically, what happens is you’ll have a project push at the last minute, and that’s why some of these OEMs will reserve the right to resell inventory. If one of their hyperscaler sites isn’t ready, they’ll be able to sell that inventory and then backfill it when that data center is ready. Typically, you have to have an address.

At some point, we will see these H100 rental prices dip. And now it sounds as though if the prices go down, you’re going to see more retirement. So you’re going to see more of those enter the trading market. Are there any places to deploy them?

Great question. I agree that as these rates go down, you’ll see more get decommissioned. In terms of sites to deploy them, not every site can operate every GPU. Vera Rubin is strictly a liquid-cooled solution. That’s NVIDIA’s next generation, right? The cement slabs those go on have to be reinforced. Typically, those are, what, like a ton per rack? The weight is insane. The power draw is about 230 kilowatts per rack.

To give you some comparison, for 2 B300s, you’re looking at 30 to 35 kilowatts per rack. Not every site can do Vera Rubin. Not every site can do GB300, which is similar to Vera Rubin in terms of power. So it’ll make sense to convert some sites; it won’t make sense to convert other sites.

AI isn’t one single workload. AI is a family of different workloads, and they all have different requirements. There are some sizing differences right now, but it’s mostly RAM, shared storage, local storage, and some back-end networking. We need to get more efficient with how we size to the workload, because the whole point of this hardware isn’t to own the hardware or to support NVIDIA’s next trillion-dollar milestone. The point is to run a specific workload that has real-world implications for end users and helps create new businesses. As we get out of these very hardware-constrained environments, we’re able to look more at the software, and then the hardware gets more efficient.

5. Hardware compliance and KYC

Part four: Who’s checking? Mike San Miguel on the rules that travel with the hardware.

Do they do KYC on the person buying the thing, or on who owns the person buying the thing? Do they do checks on that kind of stuff?

Yeah. Part of my role here was spinning up our KYC department. A lot of the OEMs won’t even quote without KYC being done. Some won’t ship without it being done. Internally, we basically don’t accept projects without KYC being done, except in some rare circumstances where they’re big names or companies we’ve worked with for years.

There are limits on what percentage can be owned by China.

What percentage of that company can be owned by China?

I think they might be going down to 49%. I know typically, at 50% or more, you can’t sell to companies that are around 50% owned by a Chinese entity.

Do they do this when a lot of Chinese entities will go through the Cayman Islands or BVI or whatever? Do they trace back through that?

Yeah, they do. At least, it’s something we do. Every company is a little bit different. We do international orders; we’re not just geographically bound to North America. For us, we’re going to trace back not just who the hardware purchaser is, but also whether they have an offtake agreement with a neocloud. You cannot legally sell this equipment with the offtaker being a Chinese entity, for instance.

Mike signed off there. As a precaution, I kept talking. We’ve talked to people like Samuel Hammond and others at the D.C.-facing national security decision-making level. But then you go down to the deployment guys, and these guys are running around getting permissions. Every 6 months they’re expecting an NVIDIA checkup, and they’re chasing KYC. They go to the owners and the users, and maybe they don’t know the user of the user. They’re saying, “Okay, maybe that is unclear.” I wonder to what extent the policy discussions at the top end are divorced from how in-depth the execution is at this point.

Yeah, I bet Everything Everywhere All at Once is probably true here, because a lot of what he described sounded to me kind of like the fracking boom. There are a lot of things coming together all at the same time. It’s capitalism at work, right? Wait a minute—there are a lot of moving pieces, from the sites and approvals to the local politics and the grid, and of course the chips. You’ve got to reinforce the concrete, and not every site is ready to take on the latest super-heavy rack.

So many little details. So many people coming together to do their part to create something that, at the end of the day, at the consumer level, gives you some appreciation for why the cloud occasionally has downtime. But in general, it’s pretty amazing how smooth it all is. I could see both: enforcement could, in many cases, be pretty legitimate, especially among people who care about it or who are conservative enough that they want to make sure they’re absolutely on the right side of the law.

At the same time, it sounded chaotic enough that if you told me 20% of the GPUs somehow don’t get their regular NVIDIA six-month check-in, and some percentage of those found their way to wherever, that wouldn’t shock me at all, based on what he said. It sounds like there’s a lot of sincere effort going in, and also enough noise and chaos that the opportunity for things to slip through is probably pretty substantial, I would bet. Then Prakash turned to China and the pressure on American companies over their Chinese customers.

For example, Oracle selling capacity to Tencent: $7 billion of capacity that was canceled, right? The national security infrastructure is pressuring these companies to cancel that, right? It’s an American company doing business. It’s outside China. They’re not sending chips into China. At the same time, there’s no pushback on the political infrastructure that is allowing this. Why don’t you just make laws? Why pressure the companies? I just don’t like it. If you want to make laws, just make laws. But if you don’t have the political willpower to make those laws and you try to slide this under the gate, why?

You have a mandate now, right? You have a mandate in Congress. Do something, right? We say compute is the new oil, right? So there is also this impact. Let’s say you decide not to sell compute to China—zero Chinese content. Do you also mean that YouTube cannot show AI-generated video to Chinese participants, Chinese viewers, Chinese customers? That’s an AI product, right? Google needs to generate that AI product. Google has to use compute, and that compute is going to be used on behalf of a Chinese customer, right?

You can restrict it at the data center level. You can restrict it at the raw compute level. You can restrict it at the product-of-compute level, right? But at some level, you’re going to end up selling to the Chinese because they are global consumers, right?

I had said we get carried away with these restrictions. Here’s why. I really do think we’re going to need cooperation with the Chinese. That’s one reason why all this matters to me: I generally don’t want to see tensions ratcheted up between the US and China, but specifically on AI, I really think we’re going to need cooperation between the US and China on AI governance. I really don’t see how—we’d be extremely pleasantly surprised if we can get through all of this without the two countries coming together in some meaningful way.

6. AI-native venture investing

When you have, as we recently talked about, a super-senior person at a frontier company telling me at the curve that there’s a level of intelligence we probably just shouldn’t go beyond, and that view has now taken hold at the top levels of frontier model companies in the US, the only way to achieve that is going to be through some sort of agreement. We’re really not putting ourselves in a great position to make that agreement. A lot of this stuff seems ultimately quite self-defeating, and it makes me sad.

Part five: What investors are superhuman at? Back to Friday and Bryan Bischof of Theory Ventures, an early-stage venture firm. First, the compute. I asked how much slack there really is in the market when we keep hearing both that you can’t get GPUs and that you can get them on demand if you’ll pay a multiple.

Totally. I think there are a couple of things at work here. One, if you go to a neocloud and say, “You got compute?” they’re like, “Nope, it’s all locked up. It’s all contracted out for the next 2 years.” They’re not lying. They’re not telling you an untruth, but there’s also been quite a lot of discussion, and even Satya talked about how there’s a lot of compute that is locked up. This is essentially underutilized compute due to either its geolocation or the available energy.

These are both true at the same time: there is locked-up compute that actually isn’t super useful. There’s also the reality of GPU utilization. Yesterday, I was chatting with some folks working at the infrastructure level around GPU utilization, and one of the big problems is that you actually can’t run at maximum capacity all the time because it’s unsafe from a scheduling perspective. You actually want to build in slack. There are so many more dimensions than simply whether you have contracts on the available compute.

This is why you see many different routes to big success and embarrassingly large numbers of ARR in the compute market. This isn’t just because of FOMO. This is because there are so many profiles that you need to maximize the value of whatever compute is available to you. We run thousands of jobs here at Theory Ventures that are powered by AI agents. If you ask me what the distribution of immediacy is for those jobs, I’ll tell you: maybe 5 to 10% is right now, maybe 40 to 60% is “I’d like to see it before my next meeting,” and the rest is, “I don’t know, maybe this week would be neat.”

Even in this one niche of the kind of work that we’re doing, we already have quite a broad variance of inference needs. That’s true of almost every AI application-layer company that I’ve spoken to since 2023. From that alone, you can see that if someone tells you there are simply no GPUs to be had, well, that’s not really accurate. But they’re speaking from a particular bias, and the bias is probably about the workloads that are in demand right now for them.

Then we got into how an AI-native venture firm actually runs: how they think about token spend, run experiments, and build research agents.

Everyone is technical, all the way from our talent partner to our GTM partner through the investors. Everyone is using AI. Everyone’s been using AI since the day that I joined. More and more, the way that we think about token spend is that you should be thinking very clearly at all times about whether there’s a way to accelerate by using AI tools. You should be thinking about experiments that you want to run, and we’re a really experimental bunch.

7. Episode Outro

Every quarter, we do our little off-site where we talk about everything that happened over the quarter. One of the things that we talk about every single quarter is what experiment every individual did that quarter with AI. That’s not an exaggeration. This isn’t, “Oh, I asked Claude to do something new this quarter.” No, you preregister what kind of experiments you want to do related to your job that use AI, and then you check if they worked.

This happens every quarter because we’re so insistent on rethinking what it looks like to work in a venture firm this year, next year, and the year after. How are we adapting to those things? As the head of AI, my engineering team is obviously still operating at another level: How can we build tools that use AI to transform other people’s workflows? Our goal is not to replace an investor with an AI system. Our goal is to make that investor 10 to 100 times more effective and efficient by using AI tooling, and to build the AI tooling that will make that possible.

Let me give you a quick example. Everyone talks about company brain, no matter what kind of business you’re working in. I’ve used a couple of company-brain products and, candidly, found them all to be useless. The main reason is that they’re trying to build what we used to think of as an internal knowledge system. You can think of Notion, for example. Notion is a stack of files, and maybe you can build a good search on top of it. Is that really your ambition?

My ambition is a lot higher. My ambition for a company brain is agents who are consistently working to connect the disparate nodes in the graph of understanding from your business. That’s why I can easily say, yes, we don’t have an agent for every human at Theory. We don’t have 10 agents for every human at Theory. We have thousands, because there’s a lot of work to get done.

My ambition is that every piece of work an investor does during the investment process, the research process—because remember, we’re very research-focused—is automatically linked, enriched, and expanded. This is why, when people say, “What are you building?” I say, “I’m building research agents.”

How do you evaluate success? Obviously, one of the huge challenges in venture historically—and this hasn’t changed yet—is that it takes a long time to find out if you made the right bets or not, right?

Totally.

Sometimes I do wonder, including for myself. I’ve periodically done a cron job cleanup where I’m like, there were ideas I had about taking every bit of data exhaust and processing it in some other way, finding connections, being super galaxy-brain about it, or scouting this or reporting that or whatever. And I’m like, probably two-thirds of those, if I’m real about it, I never actually look at the outputs of. So it’s easy to have these ideas, spend a bunch of tokens, and just end up logging a bunch of text somewhere without really making an impact.

How do you know what is actually working and what is potentially subject to leaving on the cutting room floor?

Let me tell you a story about Blue Bottle. When I was at Blue Bottle, we wanted to build inventory forecasting. Specifically, we wanted to build a machine learning system that could help us predict how many pastries we were going to sell on a given day, and other things like coffee and milk. But pastries were the first focus. What was our North Star, our guiding light?

It was actually not that we’d sell a lot more pastries or waste a lot fewer pastries if this was successful. Those were 2 metrics that we tracked, but what would we have believed was success? We would have believed it was a success if the cafe manager, who’s already got about 70 jobs, essentially went into the system when it came time to order and just clicked, “Okay, okay, okay, okay, okay, okay, okay.”

Now, we don’t want them to just not think and go on autopilot. We actually don’t want that at all, and there’s a reason. This will surprise you, but the variance in what a cafe lead understands about demand in their cafe is shocking. It would blow your hair back how much they understand about demand in their cafe that isn’t just in the numbers. We have these weirdly deep, tacit understandings.

So what was our North Star? Our North Star was that a cafe manager has more opportunity to spend that time with customers. A cafe manager isn’t superhuman at looking at data and saying, “I’m going to need 25 cookies.” A cafe manager is superhuman at understanding hospitality. That may sound weird because you can’t manage that on a dashboard. You can’t look at hospitality on a dashboard. But you can ask them periodically, “How much more time do you have available to spend with customers, to spend on the floor, and notice that there’s a coffee spill? To understand that your baristas are slammed right now and the staffing was off?” That was also my fault, but these are the things that we always cared about.

Why do I bring this up? You could say, “Okay, it’s 7 to 14 years to find out if this diligence was good.” Sure, but it’s a pipeline. First of all, you want to ask yourself: How fast can you get through the research pipeline? How fast can you go from identifying a potential opportunity or market to a clear understanding that lets you make a decision?

But more importantly, what does hospitality mean in venture? It means the investor has time to talk to human beings: potential or existing companies, founders, competitors, and employees, to understand the human element of the market and what’s going on. Investors are superhuman at a few things, but I don’t personally think they’re superhuman at looking at websites. They are superhuman at understanding potential and upside, understanding the relationship between customer need and willingness to buy, and framing what the vision for the future can look like. That’s what they’re superhuman at.

So when you ask me how I measure success, I measure success when an investor tells me, “Yeah, actually, I get to spend 30% less time staring at the website and trying to figure out the important details I look for every single time.” Or, “I get to spend 20% less time filling out these specific answers to questions that are important for diligence, and instead pick up the phone 5 more times and talk to 5 more people who have credible opinions on this deal.” That’s success, and you don’t need to wait 7 years to find out.

Do you find there’s a spectrum between “Let’s do more research” and “We need to cut off the research and make a decision now”? The research process doesn’t really have a defined endpoint. You could always dig deeper.

Correct. Totally.

How do you make those decisions? Are the agents the ones making the decisions? Is there a stepping-off point where they have to come back and a human makes the decision? How does this loop work?

Totally. I want to reference something Nathan said earlier. He’s got this list of things, and he’s like, “Some of this should have been left on the cutting room floor.” We would call that taste. Knowing what should be cut and what shouldn’t be cut is taste.

And yet again, here’s where I’ll point to why it’s important to have exceedingly bright colleagues. When you have exceedingly bright colleagues, they’ve got great taste about what research needs to get done. There’s some research that we always want to do. We know we need to do it; this is the structure. And there are opportunities for agents to say, “Let’s spin off some subagents to continue doing this kind of research. Let’s dig deeper on this.” That’s fine. That’s automated.

But ultimately, the alpha is in the taste. We can burn more tokens; we can burn more compute. But it’s a pretty focused laser, and you really want to point these resources at things that people have the ability to consume. If I produce an extra 1,000 pages of research on some deal, who’s going to look at it? I guess other agents could look at it, but is that valuable? It’s not obvious to me that it is. Again, pure volume of research is not a metric that we even track.

How’s your teaching of data science at, I believe, Rutgers?

Yeah.

Students are probably asking some similar questions there. What do you say to them, and what do you think the future of that field is?

Yeah, great question. I still make them do gradient descent by hand. In my opinion, I have yet to hear a single person give a cogent explanation of what the hell we’re supposed to do as teachers.

I believe, and have believed for a long time—I came up in pure mathematics; we’re not the most practical bunch—but I’ve always believed, even as a pure mathematician, that you should be constantly thinking about what your students’ futures look like and why every minute in your class is relevant to that future. I’m not just phobic but allergic to wasting students’ time.

So I reassess every semester, much to my own suffering, because that means I have to rewrite my curriculum every single semester. What do I teach them? Three years ago, when it was obvious that AI was going to change data science, and I started working at Hex to change data science, I rewrote my entire curriculum to introduce why AI might be relevant. Every semester, I go through the whole course and ask, “Can AI just one-shot the course?” The answer has never been yes, but that’s okay. The answer has always been, “Okay, these things are now not just easy for the model, but easy for the model to obfuscate the learning.”

I don’t think the question is whether the model knows all the content I plan to teach. The question is instead whether the model can collaborate with the student when that’s going to be relevant for the student, in a way that yields the best outcome. If Astra can train an SVM on the coffee survey data at a better loss rate than Brian can, neat. But can Astra say, “This feature is highly predictive, but when we think about it, if we put this on the website, it’s going to upset some of the people going through the flow, and they might churn”? Astra can look at the data afterward and say people were churning more on this question, but does Astra know ahead of time to look for that? Does it know to instrument that in the product? Does it know this is a weird question and people might be bothered? These are the kinds of things Astra Ultra is still only mid at.

Ultimately, I don’t want my students to only be users of AI. I want them to be the things that make data scientists exceptional, and that is ultimately curiosity. Every data scientist I’ve ever met who’s in the top decile, and certainly the top percentile, is curious as hell, almost to a fault. The absolute best data scientists I’ve ever worked with are borderline annoying with their curiosity. You kind of want them to just leave it alone sometimes, and that’s what makes them so excellent.

Mostly, what I’m trying to indoctrinate my students into is the propaganda of curiosity. Be curious. Ask the questions you can’t get out of your brain, and when you do that, you’re going to be an excellent data scientist. Whether you’re writing pandas by hand or using Matplotlib from memory, I don’t really care. Do you remember the indexing for NumPy? I don’t. I used to know it, but I don’t remember it anymore. Do I remember the slice operators for pandas filters? Not anymore. I forgot them all. But I used to know them. It doesn’t matter, though. What I know now is when you look at a distribution, whether it makes sense. Sometimes Astra doesn’t.

After Bryan Bischof signed off, I kept going. His answer about what he tells his students about their futures was really interesting, and it sort of contained the pessimist and the optimist take at once. When he was first going through the example of a survey question that might offend people, I was like, man, that’s bleak. The idea that we’re just going to be sitting here being the offense detectors while the AIs do all the real work.

I was like, “Oh God, that doesn’t sound awesome.” I did like the curiosity framing a lot better. Being relentlessly curious has certainly been indispensable in my experience working with data. You really want the difference between somebody who deeply wants to know the truth the data contains—to the degree that the data even contains truth, which is often a problem unto itself—and somebody who’s just trying to produce an artifact they can say is done and hand off to whoever the consumer is.

I do like curiosity as an answer to what we’re going to bring to the future, but I would not be telling my kid to go into data science today. I’ll tell you that. So, that’s a tough one. The real question of whether there’s going to be a job in data science that looks anything like the current job in the next few years—I feel like that’s pretty tough.

It seems like we’re going to have data science agents, and it’ll be up to the non-data scientist, general-purpose business operator to provide that little spark of curiosity, anomaly detection, or whatever special sauce the models don’t quite have. If you look at a company, you can kind of cut it apart into sensing, analyzing, decision-making, and then the reverse, which is operationalizing and taking action. I think AI is going to be very good at sensing and analyzing, and also at operationalizing, but for decision-making, you end up with a human in control.

You can have a business with a single human in control. What’s the number of decisions that human can make? Those decisions are decisions with taste. For example, if you’re running a tech company, maybe you don’t need to make decisions about the accounting framework. You’re looking at the product, you’re looking at the technology, et cetera, but accounting and the back office are the back office. In the same way, if you’re running an accounting company, accounting is front and center, but technology is kind of the back office. You’re not there building technical tools.

I feel like you can separate companies based on their ultimate purpose. Then the ultimate purpose has all of the humans with taste in there. Basically, everyone gets to do what they really want to do and what they’re really good at, and everything else gets taken care of.

Earlier that Friday morning, before the guests, a new benchmark had been making the rounds: AIs playing Blood on the Clocktower, a social deduction game of bluffing and collusion. The timeline was cheering about how good they’d gotten at it. Here’s what I made of the game. If you take it in isolation, all else equal, it’s interesting to know how good the AIs are at playing this game. Are they getting good at deception? Are they getting good at collusion? Are they getting good at sniffing out when other people are trying to deceive them or collude against them? These are all interesting questions.

What bothered me, though, was the sense that—then Nolan Brown comes in over the top and says, “Wow, this is great work. You want to come here and make evals?” That embodied what I thought was problematic about this whole thing: we’re just going to hill-climb on this now, right? The mentality of the field as a whole seems to be, “Oh, isn’t this exciting?” Let’s have a horse race to see who can be the best Blood on the Clocktower AI player.

I think what we’re all kind of forgetting as we get wrapped up and excited about this, and want to go play the AIs ourselves, is that this is not good behavior from the AIs. We do not want AIs deceiving and colluding. I’m sure all these people have a thoughtful response to this, but zoom out: the situation is bad. We have AIs that are untrustworthy in all these ways, and we’re not even taking a beat to say, “Hey, is this what we want? Do we want AIs that are going to out-connive and out-collude us?” I don’t think we want that. This is very similar to my idea that we should not train AI generals. Let’s not train AI master manipulators.

So simple. But here we are. I didn’t hear or see any corrective commentary on the timeline saying, “Hey, is this really what we want?”

Back to the end of that same Friday show, after Brian had signed off, Prakash shared a picture of where the economy goes from here.

I feel like the going-forward economy is an agentic economy. That means agents building businesses and software for other agents, suited to their affordances, which are going to be massively parallel and high-speed, and able to absorb and distribute information at very low latencies.

We’re going to end up with a Salesforce primarily used by agents, a Ramp primarily used by agents, all this software—a Stripe primarily used by agents. Maybe Stripe or other companies will be the ones that create the software. I’m very doubtful, because I don’t think they’ll move that fast. I don’t think they’re capable of moving that fast. I don’t think they understand the affordances. And I know Stripe is basically a billion times smarter than I am. I’m not trying to downplay how effective, intelligent, and competent Stripe is.

You’re going to get 50 million Patrick Collisons. You’re going to get 50 million Elon Musks. You’re going to get humans who give them capital to deploy. You’re going to get agents coordinating investments into businesses built by agents for agents. I largely see the future of the economy as primarily agent-driven, with us just receiving free benefits. A CRM call from a human is so, so costly in comparison to the massively parallel CRM calls an agent would make.

The abundance basically comes from us building all this stuff for this agentic economy, and as a result, we just have all this excess because we don’t consume that much. We’re just not that consumptive in comparison to what the agents are going to consume. That’s where this abundance is going to come from.

My reply leans on a side project I’d launched the week before, Ftopia, which finds the best fruit at your local grocery store and around the world. Here’s where I came down on the economy Prakash had just described.

On the one hand, it’s cool if there’s abundance in API calls because the agents are using so many and building great systems they can use, and then we get to piggyback on them. But there’s a lot more to life than API calls. I feel like we’re especially quick to forget that in the AI space, or we’re contorting ourselves to like this future.

That’s why I’ve been holding myself to the idea that I want to get outside more and get more exercise. If I’m not doing those things, am I really winning, or am I just plugging myself into some AI-powered machine as a human-integration-layer component that’s too subservient to the machine?

This is why I decided to do this Ftopia project recently. If we are in a bright future, then one deeply rooted thing we all like—we evolved to like—is eating delicious, fresh, in-season fruit. Everybody loves that. It goes back to when we were monkeys; we loved that.

When I tweeted that, I got basically no play on it, compared to the Blood on the Clocktower thing. Everybody was like, “Oh my God, look how great AIs are getting at deception and conniving and winning these games of social manipulation.” I’m just like, I don’t really want that from my AIs. What I want is not an abundance of API calls, but an abundance of things that are more biological. Give me the fruit. That’s what I really want.

The fruit of the AI revolution should, at least in part, be actual, literal fruit. If we’re losing that, we’re losing so much of what has mattered to us. If we can’t even notice that we’re losing it, I think that worries me and makes me think maybe we’ve already lost the plot.

I mean, it’s pretty obvious to me that we’re going to get vat-grown food that’s far more nutritious and far tastier. In the 20th century, we discovered chemicals. When you look at these food products from Doritos or whatever, they’re so scientific. There are thousands of PhDs who have worked on Doritos—thousands, literally thousands.

The old Onion article, remember? “Doritos Celebrates Its 1 Millionth Ingredient.”

They’ve figured out your flavor palate so well that they can pinpoint exactly what flavor they want to produce.

And all of that technology is going to get deployed into bio, right? We’re going to produce fruits and vegetables. Hybrid fruit is already basically GMOed, but we just took a very long time to GMO it. Part of abundance is going to be your 99th-percentile wine available for two bucks, right?

My favorite apple is the Honeycrisp. It’s a relatively recent creation, not GMO as far as I understand, but definitely the product of large-scale, systematic crossbreeding, which is sort of—

GMO before there was GMO. So I’m not afraid of GMO-enhanced fruit by any means. I’m all for it, but it is still something that’s a biological construct that feeds us as biological creatures and delights us in terms of our biological senses and the ephemeral qualia we get from it. This stuff is all too often lost because it’s not very measurable. It’s not so hill-climbable, and it’s not what the agents need, right?

I think we’re going to need to be very careful not to create an economy that really serves agents and creates abundance for us in things that are core to what agents need. I kind of agree with you that that might be what happens by default. But I would challenge everybody to think: What if the stuff that agents need in abundance isn’t the same stuff that makes our lives good?

One of the annoying parts is that the frontier lab guys are very serious, right? When you’re very serious, you only talk about curing cancer. No one talks about making better Doritos. No one comes out and says, “We’re going to make a better football game.” Where are the things like, “We’re going to make a better burger” or “We’re going to make a better football game”—the things that people enjoy?

I think they want to be serious. They want to show that earnestness, and they don’t want to come off as foolish. Maybe they should focus on making a better Doritos chip, one that’s healthier for you, or a Doritos that’s naturally grown from the ground, where you just need to cut open the potato and it’s Doritos. This sounds frivolous and idiotic, but it would catch people’s imagination about what’s actually going to be possible, right?

I think it’s also because they have this kind of World War II mindset about their work. They genuinely believe time is short and the stakes are high. Their own allocation of time is very skewed toward work. There’s just not a lot of space for leisure in the lives of people at the frontier companies. They’re so pattern-matching at this point: “Here’s a hill. Let’s go climb it.”

And I do think there should be a little more discretion about which hills we really want to climb. Are we really happy that we’ve created AIs with their current shape? I don’t know. Unfortunately, there’s just not enough time and energy going into those sorts of questions, so I’m stuck on that a little bit today. The Blood on the Clocktower thing really did it for me because, again, it is good work and it is interesting, all else equal, but I think we need some thinking that challenges the all-else-equal paradigm on so many levels.

Yeah, it’s a really tough one because I don’t think agents can operate in an economy without some level of subterfuge, right? You can’t tell your pricing to the other guy, for example. You can’t tell your internal costing to the guy who asks. You have to reserve that, and that involves omission. You have to know when to omit.

This is also why I think Palantir had this idea of the ontology: Who’s allowed to know what? That’s pretty important. Who’s allowed to know what, who’s allowed to say what, and who’s allowed to do what? I also think you need some kind of interrupt from government, or from regulation, which means that the agents have to obey when the government decides, “Hey, you can’t do this.” There are a bunch of things that need to be worked out in terms of governance. I’m sure we’ll get there, but we’re going to have some trial and error along the way, I guess.

Yeah, no doubt. Maybe in closing, I’ll cite one sentence from the METR thread on its report on acceleration. They say, “The full analysis estimated how much of the acceleration in capabilities was due to compute, data, and algorithmic progress.” That’s the first sentence. Now I’m editorializing: In a thread where they’ve caveated everything, saying they’re not confident and that it’s very unclear how to interpret the data, they’ve gone around in circles with themselves and with the Anthropic team. There’s so much fog of war, and even the core estimate of 2× is really something where they wouldn’t be surprised by anything from 1.3× to 3×, so caveats everywhere.

The next sentence is the only one I’ve noticed so far that begins with “We are confident.” It says, “We are confident that the acceleration was not due to an increase in the growth rate of training compute.” That suggests that, in terms of government interventions, we may see some data center buildout, and maybe slightly slower data center buildout than we would have seen counterfactually if the election had gone a different way. But it’s striking that the acceleration they’re reporting on here isn’t just a raw-material-input story. It’s an algorithm story and a data-quality story, and those things are being driven to a large degree by the intelligence we already have. Again, that’s where it goes from not just bigger and bigger capital to actual recursive self-improvement.

So if we’re going to try to change that trajectory with a government intervention, it’ll probably have to be more focused and pointed right at the practices of the frontier companies than something as diffuse as a broad slowdown in data center buildout.