Ep. 038 - Is GLM 5.3 a Mythos-Class Cyber Model From China? (InferenceX)
Kimbo ChenWenyao GaoJordan Nanos
- GLM-5.3’s efficiency is not a straightforward bearish signal for memory demand. Kimbo’s distinction is that sparse attention eases bandwidth pressure but does not eliminate KV-cache capacity requirements: “you still have to keep all the tokens.” HiSparse moves cache between HBM and DRAM to support more concurrency; Jordan expects cheaper serving to drive “a lot more HBM and a lot more bandwidth” over time.
- AMD’s MI355 challenged NVIDIA’s serving economics within weeks of GLM-5.3’s launch. At 150 tokens per second per user, Wenyao reports costs roughly matching B200; Jordan also highlights MI355 throughput at 100 tokens per second per user. InferenceX showed “the red AMD curves overtake the green Nvidia curves,” before NVIDIA improved again—making inference-engine optimization a moving competitive variable.
- Ultra-low latency has different economics from maximum aggregate throughput. TailRT reaches up to 470 tokens per second at batch size 1 through a decode megakernel, but cannot serve large batches. Kimbo calls memory bandwidth “the most important part” in this regime, while Wenyao says he does not use ultra-fast modes himself: “It’s too expensive.”
- Post-training, not a larger base model, drove GLM-5.3’s reported capability jump. Wenyao reports more than 10% improvement in agentic benchmarks; Kimbo calls the cybersecurity change from GLM-5.2 a “night-and-day capability difference.” Jordan describes GLM-5.3 as a “frontier cybersecurity model,” based on his own experience of Anthropic and OpenAI refusals in CVP threat-teaming work and his fallback to GLM-5.3. Kimbo says it is “definitely not Mythos class,” though he tentatively calls it near Fable level.
- Z.ai’s edge may lie as much in research throughput as in new algorithms. Jordan’s pushback is that familiar post-training methods are not unique; the advantage can be running more, cheaper experiments on equivalent compute. Kimbo highlights critic-based SAO training and SLIME, “arguably one of the best-designed” frameworks, while cautioning that no detailed technical report has appeared since GLM-5.
- Chinese-chip deployment is a public claim; optimization for a particular supplier remains a hypothesis. The GLM-5 paper claimed day-one adaptation across seven domestic platforms, and Z.ai says OxAlpha/GLM-5.3 Flash ran entirely on Chinese hardware—Kimbo’s caveat is “assuming everyone’s honest.” His calculated arithmetic intensity around 120 roughly fits Moore Threads, but he explicitly says that proves neither optimization for those chips nor actual deployment.
- The next gains might come from scaling hybrid attention or post-training for specialized workflows. Kimbo calls GLM-5.3 Flash “almost like a Frankenstein model,” combining DSA with Kimi Delta Attention, and suspects scaling it to GLM-5.3’s size could yield further gains. Wenyao’s enterprise route is post-training on proprietary data rather than financing pre-training from scratch.
1. Sparse attention relieves bandwidth pressure, not cache capacity
- Kimbo Chen’s distinction: linear attention changes the attention algorithm; sparse attention preserves it while selecting the most relevant tokens. Since GLM-5, Z.ai has used DeepSeek Sparse Attention, introduced with DeepSeek-V3.2: a Lightning Indexer selects tokens for attention built on Multi-Head Latent Attention. Kimbo says dense attention is infeasible for million-token sessions; Jordan frames efficiency as important to serving them affordably.
- Kimbo rejects the idea that sparse attention solves KV-cache capacity: “you still have to keep all the tokens” in the relevant context. HiSparse, the sparse-attention version of HiCache, instead offloads cache from HBM to higher-capacity, lower-bandwidth DRAM and brings relevant portions back. The bandwidth saving and capacity workaround are different mechanisms.
- Wenyao Gao’s B200 example shows a substantial difference in host-memory percentage between concurrency levels 8 and 16. The effect is more pronounced on B200 than B300 because B200 has less memory per chip. Recalling the memory-stock selloff around DeepSeek V3 and R1, Jordan argues efficiency permits more users rather than automatically less memory demand. His Jevons paradox expectation is “a lot more HBM and a lot more bandwidth.”
2. AMD catches up as serving software changes the cost curve
- Wenyao reports NVIDIA led at launch, then AMD reached roughly B200’s cost at 150 tokens per second per user within weeks. Jordan corrects the chronology—GLM-5.3 arrived in early August, not October—and highlights MI355 throughput at 100 tokens per second per user. InferenceX’s history shows “the red AMD curves overtake the green Nvidia curves,” followed by NVIDIA improving again.
- Jordan points to peak efficiency around 45 million tokens per $1 of TCO. Wenyao’s explanation separates fixed chip capabilities from inference-engine development: AMD engineers are “working hard overnight,” repeatedly optimizing software and catching the cost per million tokens of many NVIDIA chips.
- Wenyao’s low-latency counterexample is TailRT: a decode megakernel reduces launch overhead and reaches up to 470 tokens per second at batch size 1, but cannot serve large batches. DRAM offloading adds latency, particularly in the tail. He contrasts Jane Street wanting answers within seconds with his own ordinary agent work: “I can just wait and maximize the output as long as the cost is affordable.”
- Jordan links that trade-off to Rubin Ultra’s discussed capacity reduction from 1 TB to 192 per GPU while preserving bandwidth. Kimbo interprets it as prioritizing bandwidth: in ultra-low latency, “you’re fighting against how fast you can load your model.” Capacity still matters, but caching, networking and overlapping transfers with computation offer ways around shortages.
3. Post-training delivers the capability jump, with limited technical disclosure
- Jordan calls GLM-5.3 a “frontier cybersecurity model,” based on his experience of Anthropic and OpenAI models refusing work he was doing in CVP threat-teaming; he says he falls back to GLM-5.3 and notes its weights are downloadable. Kimbo pushes back on the strongest comparison: “technically it’s definitely not Mythos class.” He tentatively says it is near Fable level, while unsure of Anthropic’s exact term.
- Wenyao calls the release a “wake-up call”: agentic benchmark results improved more than 10% through post-training rather than a fresh pre-training run. Kimbo describes GLM-5.2 versus GLM-5.3 in cybersecurity as a “night-and-day capability difference.” For them, this shows how much capability can be gained through post-training without scaling up the model.
- Kimbo describes synchronous GRPO first, prioritizing stability. Because agentic workloads have longer sequences, where reward signals can be diluted, the second stage switches to a more REINFORCE-like variant that tolerates policy staleness. That lets them use an older or most recent policy without extensive importance-ratio and policy-staleness calculations.
- Kimbo also highlights PPO-style training with a critic grading rollouts and co-training alongside the actor: Single-Rollout Asynchronous Optimization, or SAO. His caveat matters: GLM has published no detailed technical report since GLM-5; his account combines researcher discussions, announcements and published research.
- Jordan challenges whether those familiar algorithms amount to a distinctive advantage; Kimbo initially agrees they are not unique to Z.ai. Jordan then clarifies that efficiency itself enables more experiments and research jobs from equivalent compute. Kimbo calls SAO a clear innovation and SLIME “arguably one of the best-designed post-training frameworks, if not the best.”
4. Chinese-chip deployment is claimed; supplier-specific co-design is unproven
- Jordan reads the GLM-5 paper’s claim that “from the first day” it was “full-stack adapted to Chinese GPU ecosystems,” spanning kernels through inference frameworks across seven platforms: Huawei Ascend, Moore Threads, Hygon, Cambricon, Kunlunxin, MetaX and Enflame. His point: infrastructure optimization need not mean NVIDIA or AMD.
- Kimbo adds that Z.ai publicly claimed GLM-5.3 Flash, under the name OxAlpha, was fully served on Chinese hardware. He recalls tests suggesting OxAlpha served “100 trillion tokens or something”—a figure that led people to doubt a Chinese lab could serve it on GPUs. Jordan dates OxAlpha’s OpenRouter appearance to August 20. Kimbo’s condition is explicit—“assuming everyone’s honest”—but he thinks Z.ai has worked through kernel and system blockers sufficiently to serve efficiently.
- Kimbo calculates GLM-5.x decode arithmetic intensity around 120. DeepSeek had fitted its model for H800s; his estimate for GLM seems to fit Moore Threads GPUs. He says GLM’s configuration would not be optimal for B200 or H200: it may target something else or sacrifice hardware–software optimization for undisclosed reasons. He calls the Moore Threads idea a “conspiracy theory”: the calculation does not prove optimization for those chips or actual deployment.
- Kimbo also cites Z.ai’s account of models helping bring up and verify GPU kernels for unfamiliar, non-GPU hardware while monitoring jobs for system efficiency. Wenyao argues that with the right harness, tools and sufficient budget, agents can write faster kernels than humans. Asked what AMD’s TileRT megakernel cost in tokens, Kimbo answers: “I don’t have a good idea, honestly.”
5. The next gains may come from hybrids and proprietary data
- Kimbo guesses Z.ai will probably keep the architecture for further GLM-5.x releases, perhaps GLM-5.4, but sees another path in GLM-5.3 Flash, “almost like a Frankenstein model” mixing DSA with Kimi Delta Attention. He suspects Z.ai can scale this hybrid to GLM-5.3’s size and probably extract more performance or capability.
- Kimbo says GLM-5.3 arrived in early August and GLM-5.3 Flash in late August; he estimates Flash may have been trained in two or three months, with two months the aggressive-case estimate. He notes that betting on KDA was a major bet before Kimi K3 arrived in mid-July.
- Wenyao argues proprietary-data post-training can improve specific workflows without funding pre-training from scratch; he cites Harvey’s Kimi-based agent and Periodic Labs’ AI-for-science work. He also imagines large two- or three-trillion-parameter models alongside highly efficient, “good enough” models whose cost companies need not worry about.
- The unresolved input is human labor. Kimbo says Z.ai discussed “a lot of human effort” in GLM-5.2 and GLM-5.3, but will not clarify its nature. He is “pretty sure” it concerns labeling; what kind of labeling and effort remains unanswered.
Full transcript
This week, I’m joined by Wenyao and Kimbo. We’re going to talk about everything GLM-5.3. They put out a great article recently. The model is about 2 months old, and we’ve got everything from InferenceX performance results to model architecture, post-training pipeline, and what people are using it for. Spoiler alert: there are lots of discussions around this model’s cybersecurity capabilities. Kimbo, Wenyao, welcome to the show.
Thanks for having me, Jordan.
Awesome.
Excited to be here.
I can hear it in your voice, guys. The title of the article was “How GLM-5.3 Sparse Attention Affects HBM and Memory Usage.” For those who don’t know, maybe we can start with an introduction, Kimbo. What is sparse attention? Why do we care about HBM memory usage when we talk about these new open models, especially ones coming from Chinese companies like Z.ai?
Sparse attention is a category of attention mechanism. In contrast to linear attention, which tries to change the core operation of attention—usually called scaled dot-product attention—sparse attention changes the way you select the number of tokens while keeping the core attention operation. Linear attention tries to change the fundamental algorithm into a linear-time algorithm, whereas sparse attention is about inventing some sort of token-selection algorithm.
The idea is that, regardless of how long the sequence or context becomes, you attend to only the most important, most relevant tokens at every forward pass.
Awesome.
This makes the model a lot more efficient, right? A lot of different labs are pursuing things like this to make attention more efficient. Can you give a comparison of what Z.ai is doing with the GLM models and what other labs like DeepSeek or Kimi are doing?
Like you said, in some sense, it’s almost inevitable because of physics. We just don’t have enough memory or compute. Since GLM-5, GLM has used DeepSeek Sparse Attention, the attention mechanism DeepSeek invented for its DeepSeek-V3.2 model. GLM adopted that instead. The core components are a Lightning Indexer, which is the token-selection component, and the sparse attention itself. That sparse attention is based on the more standard Multi-Head Latent Attention invented by DeepSeek and popularized with DeepSeek-V3.
Even though DeepSeek invented DSA—DeepSeek Sparse Attention—with V3.2, they continued innovating and changing their attention variants. For the V4 series, they introduced CSA, or Compressed Sparse Attention, and Heavily Compressed Attention (HSA), which, roughly speaking, build on top of DSA and add more token-compression modules. For V4.1 Flash, they simplified HSA and CSA into a slightly different design.
Sorry, I don’t mean to interrupt you. The reason they’re pursuing this is efficiency, right? People speculate that Chinese labs may care about this more than American labs because of the hardware restrictions they face. But roughly, the outcome of all these efforts in sparse attention is that the cost to serve tokens—in this chart, a million tokens—goes way down. It’s really cheap to serve if you can use sparse attention or additional techniques like HSA and stuff like that, right?
Yeah, totally. It’s infeasible to fit that many tokens with dense attention anymore, given that our usage easily goes above a million tokens of context for a single session.
Okay, Wenyao, let me bring you in here. When it comes to serving this model and analyzing the cost to serve a million tokens at 150 tokens per second, this involves running benchmarks that simulate a workload a real provider like Z.ai—or somebody else downloading and serving the model, like an inference endpoint provider—would actually incur. In other words, they want to serve dozens, hundreds, or even thousands of concurrent users on a single model endpoint. Can you talk about the restriction Kimbo just mentioned? If you’re going to support all these users with a million-token context, that’s going to put a lot of pressure on your system and the resources you have available, like memory, right?
Absolutely. For serving, one of the factors people care about most is aggregated token output, because it represents how many total outputs you can produce for your users. There’s a trade-off here: we only have a fixed amount of memory on the GPU, and to get more concurrency and serve more users, we need more memory.
There’s a technique called HiCache, which serves as the prefix cache we use. When concurrency goes up, it offloads memory from HBM to DRAM, and that makes the percentage of host memory go up as concurrency increases. If you look at the B200 charts we have on HiCache, the percentage of host memory at concurrency levels of 8 and 16 is very different. This is even more pronounced on B200 than on B300, because B200 has less memory per chip. That’s a really interesting observation from our side.
Kimbo, do you remember when HiSparse and HiCache were first introduced? These are clearly optimizations meant to save people money when they’re serving these models, which implies the people introducing them have a lot of users using their models that they need to worry about. It’s not a theoretical thing.
HiSparse is the sparse-attention version of HiCache. The whole idea of HiCache is to offload the KV cache from HBM to a lower-tier, higher-capacity but lower-bandwidth memory, and only load the most relevant set of tokens during computation.
I think HiSparse is a good example of an explanation for a misconception: that sparse attention fixes all the problems of the KV cache consuming too much memory or having high memory requirements. Sparse attention doesn’t mitigate the memory-capacity bottleneck. It helps a lot with memory bandwidth. But for capacity, during sparse-attention computation you still need to select the tokens, so you still have to keep all the tokens in the relevant context. You cannot discard them during computation. At some point, they still have to be loaded into HBM, so memory-capacity consumption won’t be significantly reduced. That’s where HiSparse becomes useful, because it can apply typical caching rules so that you can load the most relevant parts into HBM.
Okay, Wenyao—
Yeah, definitely.
Talk to me about some of the initial performance results here. The ability to use HiSparse versus HiCache on this model, GLM-5.3, is obviously giving some great performance results, especially for particular vendors, when you look at total tokens per $1 of TCO. Forty-five million tokens at peak up here—very, very efficient.
It is. I just want to add to what Kimbo just said. One thing we want to emphasize in the article is that even though GLM is compute-efficient, that doesn’t mean it’s efficient on the memory side, because the model still needs to read through the relevant context history. HiSparse becomes very important. It offloads from HBM to DRAM, and offloading back is a very tricky technique that we’ve also introduced in the article.
As for the performance, I think Nvidia’s chips generally get ahead of all the other chips at the very beginning. A few weeks later, when we were about to ship the article, it was very surprising that AMD’s interactivity at 150 tokens per second was relatively the same cost as Nvidia’s B200 chips.
Let me actually show that animation as we go through it right now. The models were released in early October—sorry, they were released in early August. We’re now in October. As you shift through and start to see these results show up on the dashboard, you can see the red AMD curves overtake the green Nvidia curves, which is quite impressive. Now Nvidia starts to come back up. Explain your experience here, because you guys are running this stuff every day. Lots of people have been anti-AMD for a long time, especially because of the AMD software ecosystem, including us.
But this is a relevant model that’s not even 2 months old, and the best curves for the highest throughput—even at 100 tokens per second per user, which is a real interactivity range—are beyond what the inference endpoints I can see on OpenRouter are approaching. Those endpoints are below that. That’s a lot of high performance from AMD’s MI355. Good for them.
Yeah, they’re definitely working hard overnight and trying a range of optimizations. I think it shows that there are 2 sides. The first side is the hardware, where hardware vendors like NVIDIA deliver their chips to customers, and the performance of those chips is fixed. On the other side is the inference engine, where you have to catch up, continue optimizing, and iterate over and over again to make sure your performance stays more cost-efficient. This is a perfect example: we see AMD engineers working hard on their engine, Adam, and over time its cost per million tokens has caught up to a lot of NVIDIA’s chips.
Yeah. We have a live chart specifically looking at the prefix cache reuse you were talking about earlier. If you click this at the bottom, you can see, for a given configuration like MI355 on FP4, how much of the host cache—like the DRAM—is actually being used, and compare it with GB300 at the corresponding points on the curve.
1. The Capacity Bandwidth Tradeoff
Maybe dig into the point you were just making about the implications for DRAM. If we go back to the first big open-model release from a Chinese lab, DeepSeek taking the world by storm almost 2 years ago now, V3 and R1 came out, memory stocks sold off, and NVIDIA sold off. Everybody was really concerned that a more efficient model would mean less use of memory. But maybe the opposite is true: as you make models more efficient, you can increase concurrency and serve more users, so you still have just as much demand on the memory itself.
Could you put a finer point on the capacity-versus-bandwidth thing? A lot of our listeners may have tuned in for the conversation with Myron a few weeks ago, where we talked about the de-spec of the Rubin GPUs for Rubin Ultra, from a terabyte of GPU memory per GPU down to 192, matching the R200. That keeps the bandwidth the same but reduces the capacity. Can you explain the trade-off in the context of inference, where HBM bandwidth and capacity, as well as host DRAM bandwidth and capacity, are all used to serve users at a lot of points on this curve?
I can start on the inference side. If we think about memory offloading and capacity, we definitely want to serve more users with more concurrency, because that maximizes aggregate output tokens. But we still need to think about the trade-off, which is the downside of offloading to DRAM: longer latency, especially long-tail latency.
We’ll customize based on different use cases. For example, we introduced another inference engine, TailRT, in our article. It focuses on low-latency cases. Essentially, it uses a megakernel for decode to reduce launch overhead, and in smaller-batch cases, such as batch size 1, it delivers blazing-fast decode speed, up to 470 tokens per second. The trade-off is that it can’t serve large batch sizes.
We’ve seen more and more inference engines trying to serve these special use cases on general-purpose GPUs rather than server products. People who want high interactivity and low latency will definitely love it. The trade-off really depends on how much you’re willing to pay and what kind of use cases you’re dealing with. If you’re Jane Street, you definitely want your request to get an output in several seconds. But if you’re a user like me, dealing with a normal agent task, I can just wait and maximize the output as long as the cost is affordable.
You’re not using ultra-fast mode, Wenyao?
No, I’m not using it. It’s too expensive. Also, it’s hard for me to feel the difference somehow.
Yeah. Kimbo, how about you? Are you using ultra-fast on any of the coding agents right now?
Not really. I don’t think I can even use fast right now, so, yeah.
Okay, so what do you think about TailRT and the trade-offs there? Like Wenyao said, you’ve got to pay significantly more for that faster token speed. It’s a fun engineering problem, right? But maybe not a—
Mm.
—best thing that’s realistic.
Back to what you were saying, Jordan, about the Rubin de-spec: keeping memory bandwidth and prioritizing that over memory capacity. I think that’s a good example of the design decisions we saw as well: memory bandwidth is more important than capacity. Concretely, sparse attention doesn’t help with memory capacity, and you still need memory bandwidth to sustain the performance you want.
In the ultimate speed-of-light performance, which we can see, in some sense, in TailRT or the ultra-low-latency regime, you’re fighting against how fast you can load your model or how fast you can load the relevant data to finish the relatively small amount of computation you have. In the ultra-low-latency regime, memory bandwidth is the most important part.
The problem with memory capacity is that we still need a lot of it, but dealing with insufficient memory capacity is a very standard problem in computer architecture. That’s why we have a cache hierarchy, and there are numerous ways, through hardware, to overlap computation with lower-tier, lower-bandwidth memory transfers. In Rubin’s de-spec, I think they’re betting on keeping the memory bandwidth while trying to connect GPUs through networking and, again, trying to overlap them.
2. Post Training Drives Capability
Yeah. When Z.ai keeps the model architecture the same from 5.2 to 5.3 and gets such a performance improvement strictly from its post-training pipeline, what do you think they’re thinking about right now? Do you think they’re experimenting with even more efficiencies in future attention architectures, or are they just going to double down on DSA and keep it the same for some future GLM-6 release, or whatever comes next?
My guess is they’d probably continue with the same architecture, pushing further for 5.x or maybe 5.4. But based on their releases, it seems like they’ll continue trying different architectures. Not too long after 5.3, they released 5.3 Flash, which isn’t really a Flash model. In my opinion, it’s almost like a Frankenstein model where they use KDA, or Kimi Delta Attention, with DSA, as far as I remember. They’re mixing all the efficient alternative attention mechanisms, putting them into a model, and actually training it. The model is pretty good, so I think its success could probably translate into a scaled-up version with even better capabilities and a similar architecture.
Yeah. Let’s talk about the post-training pipeline. As a bit of a backdrop, before that, we can talk about the capabilities. The first time I remember people talking about this was a tweet from Tor Taxes[?], which got Elon to respond, and then got Jia Tang, the founder of Z.ai, to respond. The question was, “When do you think China’s going to get a Mythos-class model?” This was back in June, when GLM-5.2 had been out for a little while. Elon responded, “Probably Q1,” and Jie Tang said, “Won’t take that long.”
I made a comment on one of the episodes we recorded yesterday. I said to Dan, “To me, GLM-5.3 is a frontier cybersecurity model, because I get classified out of using the Anthropic or OpenAI models, even though we’re in the CVP threat-teaming program and we’re doing stuff with the neoclouds to help them do this. It’s not even the classifiers; it’s the stuff trained into the model that I can’t get around. So I fall back to using GLM-5.3, and I need to be more creative in my prompting if I want the frontier models not to refuse me and pollute the whole context window, where you have to rewind if you want to try anything again.”
Anyway, this is an incredibly capable coding model, especially for cybersecurity tasks, and it’s openly available. Anybody can download the weights. This isn’t the biggest model on the market, and all the improvements from 5.2 to 5.3 came through post-training. I kind of agree: just keep post-training going. What are the unique aspects of their post-training pipeline that you think are letting them do this?
Yeah, just to comment on the tweet.
I think Jia Tang, the CEO, is pretty vindicated, right? It took only about a month after that. I can’t believe Elon got out-Elon’d by someone. It was an out-Elon timeline. The thing Elon is most known for—not the only thing, but the thing he’s most known for—is an aggressive timeline.
Yeah.
But, yeah, I think GLM-5.3—
This is Mythos class right now—or Fable class—but it’s certainly approaching that. It’s nipping at their heels, getting pretty close, I’d say, right?
Yeah, true. Technically, it’s definitely not Mythos class, but I think it’s still nontrivial to get Anthropic themselves to admit that 5.3 is—what do they call it?—Fable class, I think. Near Fable level.
Let’s go find it. Yeah, good point. I’ll go find that—
Yeah, so—
—blog post from them.
Yeah. I think it’s a nontrivial feat, all within a couple of months of post-training on the same model.
Among all the Chinese model labs, I’d say Z.ai is the one that invests most heavily in post-training and is really laser-focused on it. You can see other model labs changing their architectures every once in a while. GLM has stuck with the same architecture since GLM-5, despite DeepSeek already going through 2 iterations since DSA. Kimi has kept releasing new versions of its attention mechanisms over the past year and doing scale-up experiments. AntLing is doing all kinds of linear attention as well. StepFun is doing some very cool things, and MiMo is being even more conservative on that side.
GLM has mostly stuck with its architecture while focusing on post-training. The basic techniques in its post-training aren’t too far off from DeepSeek V3, which is still GRPO-based. But I think GLM’s post-training pipeline has very clear steps and reasoning behind how it adapts its algorithms to the efficiency it needs.
For example, in the first step, they use typical GRPO for general post-training. They do it synchronously to get the most stability out of training. After that, because they need to extract more system efficiency from training, and because agentic workloads typically have longer sequence lengths, they change approaches. GRPO—and RLVR in general—has always had the issue of a weak training signal. Roughly speaking, as your sequence gets longer, the strength of the reward signal gets diluted across the sequence.
Because of that, in the second stage, they switch to a more REINFORCE-like variant of a post-training algorithm that tolerates other system issues, such as policy staleness. That way, they can use an older version of the policy, or just the most recent version, instead of having to deal with a lot of importance-ratio and policy-staleness-related issues during calculation.
Give me a quick summary. You’re describing algorithmic decisions here. These may be unique decisions they’ve made, but nothing I’ve heard you say before or during this talk would lead me to believe GLM-5.3 is doing something so unique algorithmically, or so completely different from the other labs, that they have a big advantage in terms of algorithmic work in the post-training pipeline. Is that fair to say?
Yeah. I guess what I’ve said so far isn’t too unique to Z.ai. Yes, that’s correct.
Okay.
Actually, I wanted to say what I felt about GLM-5.3 when it first came out. It was completely based on post-training with the previous models, and I felt like it was a wake-up call for me. GLM-5.3’s agentic benchmark results improved by more than 10% just by doing post-training. It’s more GPU-efficient to do post-training than to pre-train a model from scratch.
I think that’s a wake-up call for a lot of other companies as well. They don’t have enough budget to train a model from scratch, but they still want to improve their company’s workflow in a specific area, and they happen to have proprietary, private data. They can just do post-training themselves and achieve a better result than some closed-source models.
If I remember correctly, GLM-5.3 was focused on agentic reasoning and general RL, and we can see that all 3 areas improved quite a lot in the benchmark results. When I saw the tweet that GLM-5.3 had come out, along with all the benchmark improvements, I was like, “No way. They did this just by post-training?” That’s quite amazing to me.
Yeah. So—
Yeah.
Mm-hmm.
Yeah, go, Kimbo.
I totally agree with Wenyao about how much juice there is left to squeeze just by post-training. 5.3’s post-training effort is definitely very impressive. In our article, we also compared GLM-5.3’s cybersecurity capabilities with 5.2, and the difference is really notable. It’s a night-and-day capability difference, and that’s all just post-training.
To be fully honest, GLM hasn’t released a detailed technical report since GLM-5, so I’m working from conversations with researchers and their blog announcements. I think there are 2 things that are most notable. One is the research they published about doing PPO-style training. In some sense, they’re reverting to the typical RL training format, with a critic model grading rollouts along the way and co-training the critic model with the actor—the main model. They call it Single-Rollout Asynchronous Optimization, or SAO.
The second thing is that, in their public speeches and at conferences, they talk a lot about—thanks for putting up the meme, by the way.
Take the meme.
They talk a lot about—thanks. I don’t know; it’s probably too Taiwanese, too local. Even my Chinese friends—the SLIME authors and people in the Chinese community—don’t understand that meme, unfortunately.
Maybe explain that a bit, Kimbo. It’s so funny.
Wenyao, have you watched Sword Art Online?
No, I haven’t, but I saw your explanation on Slack. That was fun.
Do they understand this meme, Kimbo? “10:15,” something like that.
I don’t know. I haven’t asked anyone about that yet.
Okay. I jumped in earlier and said something that might get misconstrued here: that they didn’t have—
Mm.
—necessarily super-unique advancements in algorithms. You just said the word SLIME, which I was waiting for. There’s both the Single-Rollout Asynchronous Optimization, but then also all the infrastructure work related to training efficiency. That seems to me like it allows them to run a lot more experiments, potentially many more jobs, and potentially much more compute-efficient jobs. That’s itself an advancement that sets them ahead of anybody with equivalent or even more compute who just can’t do as much research as them.
Yeah, definitely. SLIME is highly regarded in the open-source community. It’s arguably one of the best-designed post-training frameworks, if not the best. I’m sure they have very good infrastructure to support that.
Z.ai also released an article, I think probably not too long ago—about 2 weeks ago—on how they use their models to collaborate with one another and further improve system efficiency. That includes bringing up and verifying GPU kernels for their crazy GLM-5.3 Flash model on unfamiliar hardware with a different architecture. I’m not sure exactly what the hardware was, but it wasn’t a GPU. They use their models to improve that, and they have ways to improve the improvement loop and monitor system-level jobs to squeeze more efficiency out of the whole infrastructure.
So, yeah, I think SLIME is definitely notable. Algorithmically, SAO is a clear innovation, applied in 5.2 and 5.3, alongside the infrastructure work, even just based on what they’ve open-sourced.
It’s a very good improvement and definitely would help—
Yeah.
3. Training Across Chinese Hardware
And let me throw one thing in here that I think a lot of people have missed, or that I’ve at least been focused on. If we go back to the GLM-5 paper—
Mm-hmm.
I want to read a paragraph from this. Remember, this is almost a year old at this point. They say, “Last, one more technical contribution lies in the fact that from the first day, GLM-5 is full-stack adapted to Chinese GPU ecosystems. We have successfully completed deep optimization, spanning from underlying kernels to upper-level inference frameworks across seven mainstream domestic chip platforms, including Huawei Ascend, Moore Threads, Hygon, Cambricon, Kunlunxin, MetaX, and Enflame.”
When you’re talking about the model helping bring up the infrastructure for serving GLM-5, it’s not necessarily just Nvidia GPUs, kernels, and the serving stack for SGLang on the AMD or Nvidia GPUs that we test in InferenceX. They’re serving this model, at least they claim, for many months on seven different domestic Chinese accelerators. What do you guys make of that?
First of all, they also publicly claimed that GLM-5.3 Flash, under the name OxAlpha, is fully served on Chinese hardware. I remember when people tested it, they saw OxAlpha performing very well while also being able to serve, I think, 100 trillion tokens or something—a ridiculous amount of tokens compared to what you’d expect.
Based on the number of tokens they can serve for free, people thought it was impossible that it was being served on GPUs by a Chinese lab, because Chinese labs aren’t that GPU-rich. But it turns out they’re using Chinese hardware. Assuming everyone’s honest, I think they’re able to plow through all the blockers—the basics of the kernels, the systems, everything—and serve those models efficiently.
Yeah. I’ve got it on screen. OxAlpha was a stealth model on OpenRouter. It was only a month and a half ago, on August 20, when this happened. It feels like a while ago now, much longer. But anyway, things move fast.
Yeah, they did. I think they brought it up in 1 or 2 months. They released GLM-5.3 in early August, and then GLM-5.3 Flash in late August.
Yeah.
So that means that even if we try to bound when they started developing the GLM-5.3 Flash model, it probably wasn’t too long—more like 2 or 3 months, as far as I can tell. Betting on KDA was still a big bet before Kimi K3 came out, because no one really knew whether KDA scaled that well. I think Kimi K3 came out in mid-July.
Yeah.
So the most aggressive timeline would be that they trained GLM-5.3 Flash within 2 months. If that’s true, that’s actually pretty crazy.
Yeah, it is really crazy.
One thing I want to add about the hardware side is that I think I also wrote about it in the article. We can make a guess. First of all, the arithmetic intensity of the attention operation during decode is typically matched with the arithmetic intensity of the hardware. This is shown most notably in DeepSeek V3, which DeepSeek talks about openly.
That means there’s some sort of hardware and software co-design: you try to change the model configurations to fit the hardware properties of the GPUs you serve on. DeepSeek fitted its model for H800s. That got me curious, so I tried to calculate the number for GLM-5. One of the most interesting changes they made is to the number of heads. I can’t remember the exact configurations they changed, but because of those changes, the arithmetic intensity is very different from that of Nvidia GPUs—
Yeah, I’m pulling this up here.
Yes, thank you. I think my calculation was somewhere around 120 for GLM-5, or the 5.x-series models. I did some basic calculations, and it seems to fit with Moore Threads GPUs. That’s somewhere in the range.
That doesn’t prove they’re optimizing for that. It doesn’t prove that Z.ai is actually serving models on that hardware, or that the models are optimized for it. But I think it at least shows something. My conspiracy theory is that they’re probably optimizing for it, but I couldn’t get confirmation. So, Z.ai, if you’re watching this, I’d love to hear your thoughts.
We are high alpha here.
Yeah, but at least I think what we can—
Led by Kimbo.
Yeah, we can calculate the arithmetic intensity, right? But I think it at least shows that they’re definitely optimizing for something else, or they’re sacrificing quite a lot of software-hardware optimization for other reasons that aren’t public. Obviously, it wouldn’t work well on—or wouldn’t be optimal for—B200s or even H200s.
Yeah. It’s a sign of what’s to come: somebody might make a software-level co-design decision based on hardware that isn’t Nvidia GPUs. I think that’s quite a big statement.
Mm-hmm. Yeah. And adding to that, as the models’ capabilities improve, we’re beginning to see the R&D cycle shorten drastically, right? Take kernel development as an example. Nowadays, agents can write kernels that perform much better than humans, with the proper tools, like a kernel development agent or Cake, which was just released recently, and the right harness and tools.
We also have leaderboards for agent-written kernels. As long as you have the budget, an agent will write a faster kernel than a human. It’s pretty interesting to see how different model companies will compete with each other with the help of agents. It’s human plus AI competing.
How many tokens do you think the AMD TileRT megakernel for GLM-5.3 cost them to produce?
It must be a lot, I think. What do you think, Kimbo?
I don’t have a good idea, honestly. I wonder. Maybe we should ask.
Yeah, maybe we should. We’re coming up with good ideas here. Okay, guys, is there anything you think we’ve missed talking about? I think we went through InferenceX’s performance results, some high-level discussion about the cyber capabilities of this model, what we’re using it for, high sparsity, high cache, KV offloading, model architecture, and the post-training pipeline.
What else is left? Rumors and vibes about what’s to come with GLM-5.4? What do you want to speculate about as we sign off?
Yeah, I think we briefly talked about it. GLM-5.3-Flash is definitely a success for them. They’re able to mix DSA with KDA, using a very new architecture while still having a very good model with great capabilities. I suspect they can scale up the GLM-5.3-Flash model architecture to the size of GLM-5.3, and probably squeeze out even more performance or model capabilities.
GLM-5.3-Flash has that crazy efficiency as well, right? It uses linear attention and sparse attention.
I’m curious about what they’re going to do next, because GLM will definitely explore something else. Through post-training, they show us that you can improve model capability not only by scaling up, but also by squeezing more out of your current parameter size. Even if you don’t have a trillion-parameter model, you can still perform as well. For companies that do inference for the model and are highly sensitive to cost, I think it will definitely be a good call.
We’ve seen Harvey in the Bay Area with its own post-trained agent based on a Kimi model. We also see Periodic Labs—maybe Kimbo can expand on that—doing post-training for AI for science. I’m really excited to see how this trend goes, and especially what GLM-5.4 will bring us.
One last thing about post-training that’s important but not discussed very much—and Z.ai isn’t willing to clarify it further—is human effort. In their public conferences, they talked about spending a lot of human effort on, supposedly, data when training GLM-5.2 and GLM-5.3.
The question is, what kind of human effort? I’m pretty sure it’s related to data labeling, but what kind of data labeling, and what kind of effort goes into it? I think that’s also pretty important when it comes to post-training: being able to get quality data.
Makes sense. So what do you think comes next? Do you think we get SpaceBunny Alpha, MiniMax, and 3.1 Flash? Do you think it’s GLM? Do you think Kimi comes out with something new? DeepSeek? We had a big release from Reflection AI.
I don't know. What are you guys looking forward to most with the model releases coming out soon?
I'm looking for an efficient model. Actually, Reflection's model is really efficient. I'm thinking maybe in the future we'll have large models, right? Two trillion, three trillion, capable enough for people to do powerful stuff.
But we'll also have models that offer good enough performance and are extremely efficient, so companies won't care about their cost and can still do certain things well, like the models Reflection releases.
Makes sense, guys. Okay, well, thanks for coming on. It's been a whirlwind tour of everything GLM-5.3, why we don't think model releases like this mean people should sell their memory stocks, and why we expect Jevons paradox to continue to result in more people consuming a lot more tokens, a lot more HBM, and a lot more bandwidth over time. Thanks for joining, guys. Great job.
Thank you, Jordan. Thank you, Wenyao.
Thank you, Jordan. Thanks, everybody.