[BidClub_]
SemiAnalysis · · 35 min

Claude Code Psychosis: How SemiAnalysis Is Token Mogging Meta | Ep. 008

YouTube
TL;DR
  • SemiAnalysis is turning Claude Code into an agentic research stack whose economics are attractive against manual analyst work. Dan’s Wags system initiated AOI coverage, returned a partially garbled contract-related result after roughly five minutes, and cost $3.52 for that run—perhaps $10–$15 including prior setup, versus having “an analyst spend like a day on it.” The ambition is covering 60–100 companies while humans focus on “why it happened and what it means.”
  • A major constraint is context architecture and durable memory, not merely model intelligence. Specialized agents handle transcripts, news, events, and financial models; because they can only communicate through the lead agent, each handles a discrete part of the process while a “pristine” company agent ingests their outputs. When subagents shut down, context disappears, yet memory files cannot grow forever. The unresolved failure mode was stark: a generated balance sheet did not balance, requiring supervisory checks and an intern whose job is to “whip the agent.”
  • Low task costs and compounding usage support strong token and GPU demand, but enterprise adoption remains the larger uncertainty. SemiAnalysis says it consumes more than twice Meta’s tokens per employee, and Dan would still run the demonstrated job at $15–$20. Yet IT approvals, weak incentives, and distance from customers may turn 50% productivity into finishing work 50% earlier rather than producing 50% more—so token spend need not map cleanly into corporate earnings.
  • Model quality has crossed from a careful expert tool to something users can address with “the same instructions that you would give to an intern.” Sam said software work a year ago demanded test-driven development, stepwise scaffolding, and tight context management; now he would introduce the product to parents or grandparents and expects “every finance bro, every lawyer” eventually to rely on it. The counterpoint is that adoption remains weak even though the output is already good enough.
  • Mythos might mark a cybersecurity leap, but Dan was unconvinced that raw capability alone explains it. He noted that coding and cybersecurity gains far exceeded broader improvements and questioned the “gigantic marketing campaign” around zero-days. He suggested persistence, post-training, harnesses, and reward signals may be the decisive stack. Dan also said many other models may be able to exploit a vulnerability once the finding is explained, while discovering it through a long-range task is different.
  • The model race may be converging while the real moat shifts toward interface, workflow history, and user inertia. Meta’s rushed-looking Avocado release was described as roughly in the ballpark of “Opus, 5.4, and a Gemini,” raising a “four-horse race” question. Repointing Claude Code might technically require one URL change, but users accumulate knowledge files, adapt workflows, tolerate fixable errors, and value Claude Code’s ability to control a computer rather than merely chat in a browser.
Digest · the substance, structured for research

1. The intern now commands the research swarm

  • Dan introduced Terrence, an NTU AI-and-accounting major, working across five terminals and three screens. The running joke carried the organizational point: “The intern’s the boss now,” teaching agents to perform financial modeling rather than performing every task himself.

  • Wags, the team’s “agentic director of research,” supervises separate agents for earnings transcripts, news, events, company briefs, and models. Because the agents can communicate only through the lead agent, each handles a discrete part of the process. That division lets SemiAnalysis span networking, TCO, several switching-related companies, and dozens of optical-transceiver suppliers without forcing one context window to ingest every source and operating instruction.

  • The architecture keeps the company agent “pretty much pristine.” Other agents gather and summarize material, then save an index covering models, transcripts, and briefs; the company agent wakes up and ingests those outputs, retaining what matters without carrying “how we got to it” inside its context.

  • A live AOI query returned a partially garbled result mentioning “324 million,” “1.2,” and likely Meta and Amazon, mostly forward-looking; Dan said he would “have to check this against the actual thing.” The $3.52 run—perhaps $10–$15 including earlier initiation work—made his economic case: even at $15–$20, he would choose it over a day of analyst labor.

2. Discrete skills are compounding into a research operating system

  • Three months earlier, Dan was not working this way. A Chinese New Year slowdown created time to learn tools that could reduce the team’s workload; isolated requests—summarize an event, build a financial model, monitor press releases—became reusable skills for earnings, news, events, modeling, and monitoring, then components within Wags.

  • The rationale was mundane: Dan said there was still no reliable way to ingest financial data, and checking and fixing imported data can take longer than typing it in. His target is coverage of “60, 80, 100 companies” without creating so much baggage that analysts lose sight of the industry. Humans should spend less time on data entry and recounting events, and more on “reviewing, editing, thinking, drawing connections.”

  • Claudia, the conference agent, addresses the material nobody can physically consume. SemiAnalysis attends 50–60 conferences; GTC alone produced 621 sessions and 830 presentations, while the team attended “exactly zero” sessions because meetings took priority. Claudia transcribes presentations and YouTube links, indexes the conference’s speaker “talkability,” and can find talks about CPO, particularly the use of DWM for CPO. She can also assemble a briefing on ScalaCross for Julian, who then briefs Dan.

3. Trust improved faster than reliability and memory

  • The host dated a change in posture to roughly November, when SemiAnalysis began pushing people to use the tools: users previously distrusted every output, whereas they now tend to trust by default and verify when stakes warrant it. His emerging frustration is that “I am the constraint now”—the model wants to code, analyze, and build, but the instructions and task design limit it.

  • Dan’s counterexample was a generated balance sheet that did not balance. He had to investigate whether the agent mixed FactSet MCP data with SEC filings, and the team requires every completed model iteration to go through supervisory checks. The host’s pushback was precise: this is “not one that an analyst would ever make,” so treating agents exactly like junior humans obscures their distinctive failure modes.

  • Memory is the harder structural problem: “Once the subagent shuts down, like, that’s it.” Lessons must be memorialized, but an endless memory file is impossible. Sam experienced the inverse problem while learning about neoclouds—his knowledge lived inside Claude conversations—so he built a skill that moved those lessons into a flashcard app for deliberate retrieval.

4. Token demand can surge before enterprise productivity reaches earnings

  • SemiAnalysis says it consumes more than twice as many tokens per employee as Meta, a comparison the team called “token mogging.” Dan linked the demonstrated task economics to the GPU rental-price inflection: even if the total cost reached $10–$20, he would still prefer it to a day of analyst labor.

  • Broader adoption remains early. Dan sees fund managers using models for narrow summaries, while many Fortune 500 employees lack IT approval or workflows that compound over time. If Meta, Amazon, Unilever, P&G, and Boeing used Claude as deeply as SemiAnalysis, he argued, “the only way this gets relieved is like, honestly, a lot of chip fabs.”

  • Inside SemiAnalysis, more efficiency releases a “pent-up dam of ideas.” One employee reportedly ran Claude Code for 24 hours straight; other ideas became shareable Slack links within an hour, while an agentic coding-tracing benchmark and Accelerator or ClusterMAX dashboards reached working form within two to three hours.

  • The host’s caveat—worth keeping—is that SemiAnalysis links better research relatively directly to subscriptions, whereas someone six layers from a P&G customer might use 50% efficiency to stop 50% earlier. Dylan then said improving model quality made him pessimistic about adoption: “The bottleneck is not actually the quality of the output,” but perhaps apathy or misaligned incentives.

5. Frontier gains depend on persistence, while workflow incumbency becomes the moat

  • Dan found Mythos’s improvement unusually concentrated in coding and cybersecurity; its broader gains were “significantly less,” and the system card reportedly said that outcome was unintended. He questioned whether cybersecurity messaging was partly marketing: “If I wanted people not to use a model” maliciously, he would not run a campaign emphasizing zero-day discovery. A Stanford security researcher warned him about long-term agentic attacks, while that researcher and his friends remained suspicious that Mythos represented a true step change in model quality.

  • Dan offered GPU-kernel authoring and inference-system architecture as parallel frontiers. AI was producing many leading kernels in a way it had not six months earlier, and he described human-to-AI crossover as generally “a one-way street.” Experienced kernel authors using AI still had an edge, however; token budget alone had not yet commoditized the task.

  • Long-range persistence remains weak: models may stop investigating documentation and source code, suggest opening a maintainer PR, and fail to resolve a logged bug. Dan’s synthesis was that both layers matter—model post-training must reward extended coding behavior, while a well-designed harness supplies the tools and structure needed to sustain it.

6. The model race may be converging, but workflows create inertia

  • Meta’s Avocado release looked rushed, including an erroneous chart, yet landed near “Opus, 5.4, and a Gemini,” suggesting the hard catch-up work may be done. Dylan judged it not quite as good as the top two or three models but in that ballpark, while Dan said Meta appeared to have completed the difficult part and had relatively little left to close.

  • Switching a Claude Code configuration could be technically trivial—Dylan described it as one URL change—but accumulated knowledge files, expected behavior, and tolerance for fixable mistakes create inertia. Dan’s question remained open: does differentiation migrate from raw capability to “ecosystem,” “incumbency,” and user experience?

  • The hardware anecdote pointed in both directions. Dan said Mac minis had roughly three-month U.S. lead times and about one month in Asia amid the Claude Code rush. Dylan countered that a Mac mini was unnecessary because most model compute runs in the cloud, and considered using a Chromebook instead.

Daniel Nishball

I just have a lot of things I prepared to show and tell and that I’m going to share. This is our intern.

Dylan Patel

[laughter] Oh my goodness. I’m like, “Hey, you know, I know you guys say we don’t need interns, but the script is flipped. The intern’s the boss now.”

Daniel Nishball

Yeah, man. How the tables have turned.

Dylan Patel

Hello, everyone. Welcome back to SemiAnalysis, episode number 8. We’re going to talk about all of the ways in which Claude Code psychosis can be used for profit today.

With me is Mr. Daniel Nishball again, calling in from Singapore. We’re going to run through exactly how many agents the team has built at our biggest office. And a new member of the team, Sam Harsh, calling in from San Francisco, just south of me. Guys, welcome to the show.

Daniel Nishball

Thank you.

Sam Harsh

Excited to be here.

Dylan Patel

Awesome. Cool. So, look, it’s been a big week for Claude Code. We’ve been using it a lot. We put out some bait. I tweeted today about how we’ve been spending on Claude Code, how we’re consuming more than twice as many tokens per employee as Meta. We clearly have much more psychosis than those guys do.

Token mogging, I believe, was the technical term. And then we also saw a nice release from Mythos from Anthropic, which we can run through in a little bit of detail. Dan, what’s new, man? You were just telling me you got the agents ready to go.

Daniel Nishball

Yeah. We’ve been stockpiling equipment at a scary pace, mostly because Ray’s been scaring us into buying everything—laptops, iPhones, and Mac minis, right? We’ve got 2 of them in the office, which we’ve been playing around with. We’ll probably move them to proper service at some point.

But it’s all about how we’re using Claude Code. A lot of people say Claude Code means you don’t need interns anymore. The script is kind of flipped: now the intern is the boss of the agent swarm.

This is Terrence. He’s at NTU. There are 3 universities in Singapore, and NTU has a great AI program. He’s an AI and accounting major—dual majors. His job is teaching the agents how to do actual financial modeling.

I’ll introduce you to a couple of the agents we’ve been using.

Dylan Patel

While you’re pulling this up, that was just a picture—we got to see the nice back of Terrence’s head, and he had 5 different terminals up across 3 different screens, I think. So now we get a live demo.

Daniel Nishball

Yeah, this is the kind of stuff Terrence does all day. He’s the boss. This is what we call Wags, our agentic director of research.

The problem statement is that we have to cover a really broad industry across networking and TCO: four to five switching relay companies and dozens of optical-transceiver supply-chain companies. We can’t do a full model on each of these companies every quarter.

On the left, you’ll see Wags, the director of research, doing a supervisory function. Right now, we’re initiating coverage in AOI. If we talk about agents before we step back and talk about why we’re doing this, you’re mainly limited by context.

You can see in the bars that things are already filling up. By the time you load in all the context you need—how to grab this information, how to scrape things, and how to do all of that—you’ve already overwhelmed the context.

This is “read all the transcripts, process them, generate summaries.” This is “grab all the news digest.” Then there’s the event engine, the brief agent’s metadata, and the model agent, which isn’t here—it’s actually wrapping up now. The model agent is responsible for building the model.

The problem is that all of these fill their context up with the task at hand. What we do is structure all of this. It’s actually finished, so I can show you a little bit of the process.

I said, “Okay, let’s initiate. Here are the steps we’re going to take: conference summaries, key takeaways from the coverage, and the final scorecard deliverable.” I’ll ask it, “Hey, Wags, can you summarize all of the agents you used and what they did?”

The key thing is that the one agent we keep pretty much pristine is the company agent. What the company agent does is read in all the results and all the summaries. Then you can ask it questions.

It’s important to have that in the context, but not necessarily how we got to it. You can see what they did.

I’ll take a step back and say, why did we build this? Other than covering more ground, there’s one really silly problem statement. In my past life, before this, I was in public-markets investing. I built so many of these models and covered a lot of companies.

It seems silly in this day and age that there’s nothing to reliably ingest financial data. By the time you ingest financial data, you spend more time checking it and fixing it. You might as well type it in at the beginning.

It’s only now that this has allowed us to do this. The point is, we’re going to have this initiate coverage, and we’re going to be able to ask it questions.

We want the analysts to spend more time reviewing, editing, thinking, and drawing connections, and less time on data entry and telling us what happened—but more on why it happened and what it means. Our analyst team is fixing my view. It just means we cover more ground and cover it smarter. Make sense?

Sam Harsh

What are your impressions watching this happen? Have you been using agents at this level?

Not at this level. It’s a bit bewildering to me, certainly. I was burning some tokens yesterday. It felt good to break that in under the SemiAnalysis banner for the first time, but nothing like this.

It strikes me that if you’re using that much context on each agent, maybe it’d be better to be more aggressive about spinning up subagents. How do you think about structuring this whole thing and passing information from node to node to make it go as quickly and reliably as possible?

Daniel Nishball

Yeah, it’s a good question. One problem we had was moving data between the agents. They can only talk to the lead agent, so we have each of them doing a very discrete part of the process.

You can see I’ve woken up a company agent. When it wakes up—when we finish the initiation process—it saves an index file of what we create. There’s a model, the earnings transcripts, and the company briefs.

When you wake up the company agent, it’s told to ingest all this stuff. It’s ingesting all this context. I’ll ask it, “What was the large optical contract that AOI won, and what impact did it have on their financials?”

Quarter has a good product, but it generally only scans earnings transcripts. It won’t necessarily have all this other stuff in its context. I’m not even sure this is going to work because I just initiated it, but broadly, the question is: how do you keep the right amount of context and the right amount of information in the right agent?

We have an agent for earnings, an agent for events, and an agent for just updating financials. Does that answer your question?

Sam Harsh

Yeah, that gives some good insight. I’ll need to go to school on this over the next few days, but it’s good to see the expert at work.

Daniel Nishball

Actually, look at this. Imagine how long it would take my analyst. I just ran this, probably starting 5 minutes before the podcast. Think of how long it would take your analyst to initiate on this and come up with this answer.

AOI landed 324 million, million 1.2, likely Meta, Amazon, mostly forward-looking. Obviously, I’ll have to check this against the actual thing. We’re still developing it.

Let’s see how much this cost. This cost me $3.52. You can see why, like, what we've written about is we've written about, probably I think there was some initiation work I did in other sites. So, maybe the total cost is like 10 or 15 dollars.

But it’s either this or I have an analyst spend a day on it. You can imagine that if it was $10, $12, $15, or $20, I’d still do it. That’s why I think the end demand and the end ROI are so strong.

And that's why you're seeing the inflection in GPU rental prices, which we talked about last time.

Dylan Patel

Dan, can you talk about how quickly you learned this? I've been using coding agents for a while, and I feel like I'm almost more cautious than you, slowly dipping my toe into agents or OpenClaw sort of stuff. How did you get into it? You weren't using this even 3 months ago, right?

Daniel Nishball

3 months ago, it was sort of like we had a Chinese New Year break. We had wound down a lot of the projects we were working on, so we had a little bit of headspace to get into this. The ironic thing is that we were so busy, we didn't have time to learn new stuff that would make us less busy.

With that break, we started playing around a bit. What really helped was the concept of how you want to build a research team: what the tasks are and how you would divide them. It also started from a collection of very discrete things we wanted to do.

Let me switch windows real quick. What we realized is that there are a lot of discrete tasks. There are 2 important things to realize: there are a lot of discrete tasks, but to build that into something that's an ongoing, living agent with living coverage—not just, "Let's build this project, let's build that project"—you need to build something that compounds itself and actually uses all these functions.

I'll fire up Claude. Here, I'll show you the skills we built. All of these skills are used by Wags: event summary, earnings, financial model build, press news, and pure monitor. All of these were started as very discrete things: "Can you summarize an event? Can you build me a financial model? Can you do a press news monitor?" They all stemmed from things we were doing that were really manual, things we couldn't have even imagined doing now.

For example, "I want to know what all the press announcements from Broadcom are ahead of OFC."

Dylan Patel

Yeah, makes sense. You talked about the job of an intern being different. Has your job changed meaningfully? Do you feel like you have to spearhead learning this stuff, even though in some ways maybe younger people who are going through school right now might be more AI-native and better prepared to learn how to do some of this stuff?

Dan

I would say for me, a lot of this is really trying to figure out what the end goal is. I think all the analysts get it because they've all done these kinds of tasks. For me, it's really trying to figure out what we want to get to and what we need to be as a team.

Where we need to be is a place where we can cover 60, 80, or 100 companies without creating so much baggage that we lose sight of what's really going on. So, making sure that Wags and everything else does things in such a streamlined way that it really enables that cadence.

I'll give you one more example of another agent family within Wags. What do we call—

Dylan Patel

Oh, there are families now?

Dan

Yeah, Claudia is our conferences chief. As a company, we attend 50 to 60 conferences. The thing is, when we attend, most of the time we actually spend it in meetings these days, in targeted demos.

Dylan Patel

What do you cover now?

Dan

I know, George, you were at GTC, probably in back-to-back meetings with all the neoclouds.

Dylan Patel

I went to about 3 sessions the whole time. I was just there having meetings with people in person because they happened to be there, too. Every time I go into a conference, I earmark all these amazing sessions that I want to go to, and then I go to exactly 0 of them. You can see there are 621 sessions at GTC and 830 presentations, of which I went to exactly 0.

To be clear, I was at GTC while Dan was at OFC. We talked about that a couple of episodes ago. I actually got the debrief kind of live on that podcast because I tried to read some notes, but I hadn't talked to you about it since you'd gone.

Dan

Yeah, and so you're not always able to view every presentation. It becomes an obstruction, having to stop whatever you're doing and say, "Okay, now I've got to go to the website and dig through that." You might get interrupted.

Once we have the file URLs, what Claudia helps us do is take presentations and YouTube links, transcribe them, and index the talkability index of each conference for speakers. Then I can say, "If I'm researching CPO and DWM for CPO, find me every talk that discussed CPO, particularly the use of DWM for CPO." And there it is. If I want to do a research project on CPO, that's the starting point.

I'll show you another output that we did. I said, "Tell me what to do for ScalaCross." Here it is. Julian's leading our work on ScalaCross. He just joined us last week—or actually, this week is his first week—so now I've got a bunch of stuff for Julian to read. I'm saying, "Go read this and tell me what it says."

Dylan Patel

So the agent is going to brief an analyst to brief you? Is that the workflow?

Dan

Yeah.

Dylan Patel

I like that. Yes, yes, yes.

Sam

When I hear Dan say, "We've got to cover 60 to 80 companies," and I think about joining SemiAnalysis and what it was like coming up to speed—trying to figure out what's a neocloud, what's an AI lab, what's a tokenomics model cover—it would have been a qualitatively different experience.

I was thinking about this a couple of days ago because there may be some skills I've left behind that it would be good to pick back up with the upgraded tool of Claude Code. Right now, my first thought when I have something new to learn is to spin up Claude and get into it that way.

I built a skill for Claude that added the content of my conversations with it to a flashcard app, so I could go back through all the lessons from the neoclouds. I realized that all of my learnings were in conversations with Claude, and those were going to be hard to surface if I wasn't more deliberate about keeping track of them. It was completely Claude-based, and I don't know how I would have done it otherwise.

Dylan Patel

What do you guys think about the trustworthiness? It did cross some space for me. It was roughly November when we started pushing people to use it. Doug was definitely spearheading that at SemiAnalysis.

Now we implicitly trust the output by default and have to double-check things when it could potentially be wrong. Before, everybody implicitly did not trust every output and had to keep prodding the model until it got to a point where it was trustworthy.

Do you think there's some level at which this stuff can get better? I'm thinking about the Mythos release that came out today—or a couple of days ago—which is to say that I feel like, at some level, I'm the constraint now when using Claude. The instructions that I give the model are the constraint.

It wants to write code. It wants to do analysis. The model wants to build web apps for me, and I just need to get out of its way so it can build the thing that I want. It's strange to think about how much better coding models can really get from here in certain scenarios.

Does that make sense to you guys? Do you think there's a meaningful gap in your Claude Code experience where the model really needs to get better at a specific task that it's just bad at right now?

Dan

Yeah, and it's almost like training a human analyst. One of the common problems I see—and I was troubleshooting this today—is when it builds a model and doesn't always balance the balance sheet. So I was looking at this example here. Let me share my screen real quick.

Okay, can you see the screen? If you zoom in, you can see that the balance sheet doesn't actually balance. The assets and liabilities don't balance. This is the sort of thing where I'm saying, "How can you input something that doesn't balance?"

I have to go back and figure out whether it was mixing up FactSet MCP or mixing up SEC filings. We're building more of a supervisory function with Wags.

Daniel Nishball

So actually, every time a model finishes an iteration, it has to go through these checks. That's what Terrence's job is: to whip the agent, whip Wags, and make sure that we don't see this. It's just like teaching an analyst team, really. So that's one of the things.

Dylan Patel

Yeah, but it's not necessarily just like teaching an analyst team, because that type of mistake is not one that an analyst would ever make.

George

That's a good point. Analysts don't forget; the challenge is memory. Once the subagent shuts down, that's it—the context is gone, right? You have to memorialize everything in a memory, but then you can't have an endless memory file. Having those learnings persist over time is one of the biggest challenges, actually.

The balance-sheet problem is not a new one, right? Mostly, we've solved it, but we have to be vigilant for these things. It may take us another couple of weeks or months to iron out everything, but it's like learning stuff. It's learning as we go.

Dylan Patel

Can you talk a little bit about, based on your personal experience, what you think the impact is going to be on the broader market? Last week, the Core Research guys were commenting on how shocked they are that others in their friend groups—people they consider smart and plugged in—are just stuck in different jobs or different mindsets, where they're not even trying this stuff right now.

They're asking, "How do you get bigger adoption, more people using it?" My sense is that everybody feels there's going to be a lot more growth in the use of Claude tokens this year, as not only more people sign up, but individual people also start to use it more. Do you guys agree with that? Sam, what do you think?

Sam

I think it's reached the point where I'd be comfortable introducing it to my parents, grandparents even. I was using it for work, writing software, about a year ago. As you said, it has gotten a lot better at just getting things right the first time. A year ago, you had to use test-driven development, be super careful in managing context, and scaffold projects up step by step.

But now it's good enough that it sort of uses the terms that we use. You can just give it the same instructions that you would give to an intern, and it works often enough. So there's some point at which these quantitative improvements in model quality—piecewise improvements on benchmarks—net out into a qualitatively different experience using the model.

A lot of that is obviously the tooling Anthropic has built up around it, giving a nice interface with the Claude co-work app and so on. I think it's only a matter of time before these things are not such an intimidating tool that only code people use, and every finance bro, every lawyer, is just going to be relying on it to do the bulk of their work.

Dylan Patel

Yeah, yeah. Big changes.

Daniel Nishball

Yeah, I agree, Sam. I think about it: it feels like everyone in our circle is using it. Everyone I've met in the Bay is using it. But we know that, for Fortune 500 companies, they probably don't even have IT approval.

I talk to a lot of fund managers, and they're still only starting to use it for just summarizing really small, single-task stuff. It's not really a tool that compounds and improves over time. I really feel like if GPU prices are shooting up now, and everyone in your circle is really the tip of the spear, can you imagine if everyone at Meta or everyone at Amazon was using it?

I think you guys had a great post as well, comparing Meta's usage per employee, where we're completely mogging them right now. The gap's only going to become bigger. But can you imagine if everyone at Unilever, P&G, Boeing—all these companies—were using it in such an embedded way? The only way this gets relieved is, honestly, a lot of chip fabs. You'll have to bring Jeff in next time for that.

Dylan Patel

Yeah. Well, I think the other thing is that we have a relatively direct impact on the company's revenue from the time at which we start producing better products and better research with these models, and have that flow through to people spending money on SemiAnalysis research and signing up for the products. That's a pretty direct, clear link.

Whereas somebody who's 6 layers removed from a customer, working at P&G, may have a different experience of their contribution. In some ways, I think there are a lot of people who, if they were to get 50% more efficient at their job, would just stop working 50% earlier in the day as opposed to doing 50% more. Whereas it's almost the exact opposite at SemiAnalysis.

One of the guys on the team told me the other day that he had a 24-hour straight Claude Code session. He just did not go to bed.

Daniel Nishball

Yeah, I think I know who you mean, and he doesn't go to bed anyway—Jordan.

Dylan Patel

It's a different guy who doesn't go to bed. I work a lot with that guy in my time zone. I think you know who it is. It's a different guy, actually, somehow.

Daniel Nishball

Okay. Well, it's probably true of the guy I'm thinking about as well. He's not the only one. A lot of guys don't want to be named. But, yeah, it's certainly night owls. This is affecting sleep patterns, but I think it's been an unblock for these people who have this pent-up dam of ideas that they want to get out into the world or into the products that we have to sell at SemiAnalysis.

It's just awesome, right? It's been an awesome experience that people can go from an idea to a link sent in Slack for others to click and view in the span of an hour. I've had that experience multiple times this week. The agentic coding tracing benchmark with the guys there, the dashboards for Accelerator, and tracking some of the ClusterMAX participants and stuff—I mean, these are things where we got the idea to do them, and then 2–3 hours later we're in a working format.

It's really interesting to consider what improved productivity would actually do to companies that are not named SemiAnalysis. I'm not sure exactly that the productivity is going to flow through directly to the financials of a company if they spend a lot more on Claude tokens.

Dylan Patel

Yeah, I agree with that. There's some funny way that the quality of the models nowadays makes me pessimistic about their adoption long-term, because it's become clear that the bottleneck is not actually the quality of the output. For some reason other than the fact that it's not good enough, all these people at Fortune 500 companies are not using them.

It is good enough. It makes me 5 times as productive, and I'm still figuring out the right tool set. But something else—whether it's apathy or foreign incentive structures or whatever—is keeping everyone else from hopping on board.

So, what do you think about the Mythos announcement that had the comments about cybersecurity issues, and obviously Nicholas Carlini's YouTube video, where he's reviewing some of the bugs that they found in the Linux kernel and, I think, the NFS driver—stuff that's been around for 10-plus years and running on production systems—with just a little bit more focus and effort and a new model?

Daniel Nishimoto

Maybe I'll fire away with some hot takes. I've been working most of the day on a note and going through the system card. It's weird to me how good it is at coding. Its all-around improvements are significantly less than its improvements in coding and cybersecurity in particular. The system card says that they did not intend for this to be the case. It sort of fell out, which is a bit odd.

I'm not sure to what extent the yap about cybersecurity is a marketing ploy. If I wanted people not to use a model for malicious cybersecurity uses, I probably would not have this gigantic marketing campaign talking about how good it is at finding zero-day exploits.

I was speaking to a security researcher at Stanford yesterday who said, "You're out to lunch as a researcher if you're not worried long-term about agentic attacks from LLMs." But he and all his friends are very suspicious that this actually is a step change in model quality. So we'll see. They've done a brilliant job of making us all wait with bated breath for it, at least.

Dylan Patel

Yeah, we're all excited, for sure.

Daniel Nishimoto

I think I can throw out 2 other examples of stuff that seems similar to cybersecurity in this case: kernel authoring on GPUs, as well as system architecture for inference. We've been working on both of those recently. There has been some progress in the GPU MODE-style leaderboards for kernels, where AI is generating a lot of the best kernels in a way that it wasn't even 6 months ago.

That seems like a notable change, because generally the way this stuff works is not that AI catches up, overtakes humans, and then suddenly humans land a counterpunch and get better than AI. Generally, it's a one-way street.

Dylan Patel

Mhm.

Daniel Nishimoto

So it seems like the dam is breaking there, but it hasn't fully gotten through yet. Or the people who have the most experience with the kernels, in addition to using AI, are still able to win these competitions instead of just anybody who has enough budget for tokens being able to spend the tokens to author these kernels.

The other thing I was working on this week, which the model seems to really not be able to do that well, is long-range tasks: consistently digging into documentation and source code to find bugs, even with extra-high reasoning effort turned on and lots of prodding. A lot of these models will just kind of give up and tell you to open a PR with the maintainers of the repo and not dig into the details of like actually lying or the LLM to fix something that is clearly broken, clearly have a log for.

I wonder if the cybersecurity use cases they’re describing are not necessarily something intrinsic about the models’ understanding, like the weights having a good encoding of or understanding of how these systems work. It seems like those representations are already there. It’s almost more about the ability to train the model to be persistent on long-range tasks, then give it the right harness and the right instructions and let it go.

A lot of people claim, “Well, if you were to describe the exploit in great detail that Mythos found, many other models can also exploit that same vulnerability, right?” That seems roughly correct to me. If it was explained in great detail to me, and I had all the source code and enough time, I feel like you could just give it enough tries to do it.

Why wouldn’t a less capable model be able to understand something that seems relatively easy to understand when it’s explained to you by the model that found it, if that makes sense?

Dylan Patel

Yeah, interesting. Does that imply to you that if we start building better harnesses for other models, we should be able to do something analogous? If it’s not some fundamental improvement in model quality per se, but something more like agency or persistence, we should be able to get that by other means.

Daniel Nishimoto

It just seems like both are going to be important. In other words, the model’s training includes—the post-training includes—the ability to use these harnesses effectively, pursue these tasks, and get a reward signal. The more that you train it to do something long-range and coding-related, the better it’s going to perform at that stuff.

You need to develop the harness in addition to developing the model.

Dylan Patel

Yeah, we’ll see. We’ll see, man. There’s a lot to think about here. Do you guys have any hot takes on the others?

Obviously, Meta had a big release with their Avocado model, kind of catching up. It feels almost like it was rushed to come out. They screwed up one of the charts announcing some of their benchmark performance, and it was just a tweet: “Here you go,” right? Then suddenly, they’re showing up on all of these things.

My impression initially was that they were almost surprised to have a model that performs this well. Even though it’s not necessarily a counterpunch, based on my personal usage so far, it’s not as good as the top 2 or 3—naming like Opus, 5.4, and a Gemini—it does seem like it’s in that ballpark now.

More RL, more data, and off they go. They certainly have the compute, and the team has produced this thing. So, four-horse race now?

Daniel Nishimoto

Yeah, I think you would know better than me, but that’s how it seems from here. It seems like they’ve done the hard part just to get to this point and that there’s not a whole lot required. Easy for me to say sitting on a freaking podcast, but there’s not a whole lot, all things considered, left to close.

Dylan Patel

You speak, man. We’ve got to send you down to Menlo Park and start checking out what’s going on down there.

Daniel Nishimoto

I’m always happy to go to Menlo Park.

Dylan Patel

Dan, let’s leave it on this note, actually. I’m really curious: everything you shared earlier in the pod—what would it take to use a completely separate model? What would it take for you to switch off of Claude?

Daniel Nishimoto

Yeah, I was just going to ask you guys that, because this question gets asked a lot, right? Philosophically, everyone’s always catching up with each other. Does that mean that the moat is less high? I wonder: does the moat become all capabilities, or does it become how you capture the eyeballs and the users? Has Claude Code found the right formula?

People get comfortable, and I always thought ChatGPT would be like everyone would kind of have their $10 thing and they’d just be happy using it. But I think people have been switching out of it to Gemini and other things.

I don’t know. I would think it’d be annoying to switch out, because you’d have to get all your knowledge files reloaded and retransferred to another set of agents, another set of models that know how to work properly with it. I have the same thought: I probably wouldn’t want to go and retry everything and have 2 models working.

What do you guys think? Do you think this becomes the way you differentiate—not just raw models, but the ecosystem around it, the incumbency, the user experience, and the user history with it?

Dylan Patel

I’m surprised. I think, at a high level, it’s easier to switch models than you think in the ecosystem of using Claude Code. Pointing it at a Codex model would be one change to a URL in your config, but I think that doesn’t matter.

Daniel Nishimoto

Would it work the way you expect, though, or would it behave a different way?

Dylan Patel

I don’t know. You gave the example of the balance-sheet stuff, right? I feel like everybody using Claude Code right now does not have very high standards for the model that it needs to pass to be wowed by it and keep using it. When the thing doesn’t work with the balance sheet, you don’t just throw it away and stop doing it. You give it another prompt, and then it fixes the issue.

Shawn Wang

Mhm.

Dylan Patel

I feel like there’s a lot of inertia that would need to be overcome to actually motivate people to switch to a new model. That’s my impression, hearing people like you describe their initial experience using Claude Code, which is more of a change not in the model you use, but in the interface.

It’s not a web browser on a remote machine with this chat application or interface. It’s more about being able to control your computer. That’s the change in the interface that Claude Code seems to enable—or Cowork, or whatever that is—that’s motivating people to pay people to install OpenClaw for them, or something like that.

You have people lining up for OpenClaw installs and teaching lessons, right? Because that’s the hurdle.

Daniel Nishball

You know what the lead time for a Mac mini is in the U.S. now?

Dylan Patel

No.

Daniel Nishball

It’s like 3 months. Fortunately, we’re in Singapore, so we’re closer.

Dylan Patel

Huh? You don’t need a Mac mini.

Daniel Nishball

I know you don’t, but that’s what everyone’s doing. You go to the Apple Store and say, “Oh, you’re doing all this Claude Code stuff?” “Yeah, we are.” Unfortunately, it’s only a month in Asia.

Dylan Patel

That’s funny. I was thinking about getting a Chromebook, because it doesn’t matter whether the file system is local at all. Almost all the compute is in the cloud anyway when I’m using these models, so I might as well just run it out of a notebook.

Daniel Nishball

And you’ve got a 3-month wait for the Mac mini.

Dylan Patel

Don’t tell Paul, because we’ve been buying $3,000 MacBooks to log into virtual servers. I can guarantee you that Paul does not listen to this. It’s okay.

Paul’s awesome. He lets me get the 48 GB configuration, so I’m not complaining. He adds people to Slack channels at a moment's notice at all hours of the day when they are restricted that are called Paul.

Paul’s amazing. Let’s go. Shout-out, Paul. If he’s listening to this, we love you, Paul. Awesome.

Dylan Patel

All right, guys. This has been a fun episode. Rambling a little bit. Hopefully people found it interesting. Dan, I love the demos. I love looking at the back of Terrence’s head. Shout-out, Terrence. Shout-out, Paul. We appreciate you all.

Daniel Nishball

Next time. We’ll check out Terrence another time. We need live demo. Live demo of five screens, six or seven agents just all going.

Dylan Patel

Yeah, that’d be fun. Sounds good. Go, guys. Thanks for coming. Good job.

Daniel Nishball

Bye. All right. Thank you.

Claude Code Psychosis: How SemiAnalysis Is Token Mogging Meta | Ep. 008 | BidClub