AI 视频模型 Fal 不得不测试两次
Jennifer LiGorkem YurtsevenBatuhan Taskaya
- Fal 的 Gorkem Yurtseven 和 Batuhan Taskaya 介绍了如何通过后训练,将 MiniMax 开源的 H3 视频模型打造为「H3 Max」:速度达到原 MiniMax 端点的35倍,成本低一个数量级,在 Elo 评分相同的情况下几乎没有明显质量损失。 为了把竞争对手放进同一张图,对比图的时间轴不得不采用对数刻度,质量轴则无需如此;结果「好得有点不像真的」,Fal 因此花费数万美元进行内部评测,并为外部验证推迟上线3–4天。
- 战略逻辑是,生成式媒体与 coding agents 一样,已经进入 Fal 所称的「Token market fit」阶段:单个专业用户每月能有效消耗约1万美元的 token;而自4月以来,整个行业一直受算力约束。 仅靠系统级优化,在触及性能屋顶前,同一模型最多提速2–3倍;后训练与模型协同设计则能突破这一屋顶,进一步提升一个数量级。
- 增益来自复合效应而非单一技巧:后训练和 RL 先在优化栈介入前抬高质量;对扩散模型而言,将50步降至20步原本会损失质量,而 kernel 工程则把许多推理工作负载中通常30–40%的硬件利用率提升至理论值的70–80%。 此外,团队还分别优化了提示词扩展 LLM、潜空间扩散和 VAE 解码。模型部署在单个8-GPU节点上;公开版 Turbo 以成本降至原来的1/2为代价,在1.5秒内生成5秒视频,质量达到「97分位」。
- 实时视频并非预先规划的产品,而是一次意外涌现:上线后,工程师 Rehan 从笔记本向 Twitch 持续直播 H3 Max 生成,Levels IO 做出了无限流媒体网站,Fal 的 ML 团队则开发连续版本——3项独立行动都在 Twitter 上走红。 Fal 的分布式团队连续3–4天实行16–17小时接力轮班;Gorkem 说,他觉得那天可能打破了 Slack 的记录。
- 连续版本 H3 Max Director 可生成最长60分钟、由动作控制的视频,依靠2分钟高度压缩的原始视频记忆,以及一个在此后维持场景连贯性的「不断演化的系统提示词」——「镜头可以转回最初那个人,而那个人仍处于同样状态」。 Gorkem 对消费者市场的判断是:「所有人都在等待 AI 迎来一个大规模消费时刻。我相信 H3 Max 让这件事成为可能」——它现在已经足够便宜、足够好,能够支撑一种真正全新的社交 AI 体验。
- 速度问题基本解决后,未来1–2个月将聚焦可控性:按时间戳用 JSON 指定机位、控制光照、唇形同步和动作迁移,目标是99.9%的可靠性,而基础提示词只能达到80–90%。 上线一周后出现的 Blender + GPT Astra 参考视频工作流,已经让 VFX 艺术家接近「100%可控」。
- Hollywood 的使用量在一年前还不存在,如今已成为 Fal 增长最快的业务板块;随着 IP 解锁系统和美国托管的 CDance 上线,Gorkem 预计制片厂「未来几个月会把 AI 使用量提高10倍、100倍」。 Amazon MGM Studios 的 Nara 工具大部分运行在 Fal 基础设施上;上线近3周时,H3 Max 已成为 Fal 平台上最受欢迎的视频模型,调用量略超2倍。
1. 「Token market fit」:为什么一家推理平台开始做模型后训练
- Gorkem 对市场的定义构成了整期节目的主线:生成式媒体与 coding agents 一样,属于「Token market fit」赛道——「单个人能不能有效消耗大量 token,规模大概是每月1万美元」。专业用户整天坐在电脑前生成视频,每月花费数千美元;自4月左右开始,Fal 和整个行业都受到算力约束,增长速度几乎与新增算力同步。
- 触发点是 MiniMax 将 H3 开源。Gorkem 称,这是「第一款真正开源、能力很强、代表最新一代的视频模型」,支持参考输入,架构也足够熟悉。Fal 此前为其他实验室提供推理服务,但「从来没有权利在它上面叠加这项能力」;开源终于让团队可以全力投入。
- 这并不是从零开始:Fal 已经在开源图像模型上验证过后训练方法论,分别用 Ideogram 和 FLUX 做过一版,并搭建了相关基础设施,「拿一个现有模型,在它周围做 kernel 和系统设计,让它跑得非常、非常快」。
2. 突破性能屋顶:35倍速度究竟如何叠加出来
- Batuhan 认为,纯系统工程存在明确上限:在模型和架构不变的情况下,「你也许能让模型快2倍、3倍,同时产出相同质量……这里确实有一个性能屋顶」。后训练与系统协同设计「让我们能够突破这个屋顶,达到一个数量级」。
- 对扩散模型而言,把50步砍到大约20步通常会牺牲质量。Fal 的做法是先通过后训练和 RL 提升质量,再套用自身的优化栈,最终以远高于原模型的速度实现同等甚至更高质量。Kernel 工程则把许多推理工作负载中30–40%的常见利用率,单独提升至理论值的70–80%。
- 视频是一条流水线,而不只是一个模型:包括通过大型 LLM 扩展提示词(单次调用、不做缓存、低 batch,完全不像 agent 工作负载)、潜空间生成、VAE 解码和可选的放大处理。「默认情况下每个环节都没有经过优化」,团队逐一检查并继续优化。模型采用单节点8-GPU部署;超过8张 GPU 后,通信开销会侵蚀效率。Hopper→Blackwell 本身可带来2–3倍提升,但成本相近,因此主要缩短墙钟时间,而不是提高效率;这对显著跑快于实时尤其有用。
- Turbo 之争体现了价格底线:团队起初拒绝把主模型称为「Turbo」,因为它的质量已经超过原模型;一周后仍然发布了 H3 Max Turbo——1.5秒生成5秒视频,成本降至原来的1/2,质量达到「97分位」。Batuhan 的判断是:「我不认为人们还需要更快或更便宜……把速度问题解决后,我们应该推动质量和可控性。」
3. 无人规划的发布:实时视频如何意外涌现
- 结果出乎预期,Fal 因此花费数万美元进行内部评测,随后又将发布推迟3–4天,等待外部评测平台确认结果。「结果好得有点不像真的。而事实确实如此。」
- Gorkem 称,这再次体现了 Fal 反复出现的一种现象——「每隔几个月……整个公司会抓住某个东西,创造力突然爆发」。工程师 Rehan 开始用笔记本向 Twitch 直播 H3 Max 的连续生成;Levels IO 独立联系团队时,已经准备好一个无限流媒体网站;ML 团队则同时在内部开发真正的连续版本——「3条并行路线……完全独立地在 Twitter 上走红」,没有一条是事先规划的。分布式团队连续3–4天实行16–17小时接力轮班,Gorkem 说他觉得那天可能打破了 Slack 的记录。
4. Director:2分钟记忆、60分钟流媒体与消费者市场判断
- H3 Max 首次跑得快于实时速度时,ML 团队的反应是「这太疯狂了」。此前的尝试带来了新的经验:生成仍能关注前5秒的10秒片段,把高度压缩的原始视频记忆延长至2分钟。注意力成本「呈指数级上升」,因此需要「极其聪明的优化」;一个「持续、渐进演化的系统提示词」则把世界状态从第2分钟延续到第60分钟。最终诞生了 H3 Max Director,Batuhan 认为它是「唯一能够生成最长60分钟连续视频、且受动作控制的模型」。
- 这次演示让技术变得直观:Fal Live 频道让观众通过聊天投票决定接下来发生什么——「进入办公室的不是一个女人,而是一根香蕉」——场景却能持续保持一致:「镜头可以转回最初那个人,而那个人仍处于同样状态。」Fal 正在与拥有 Instagram/TikTok 节目的 AI IP 持有者合作,训练 LoRAs 并运行直播版本,已有几家排上日程;创意技术专家现在可以通过语音进行导演,几乎像在真实片场由真正的导演调度。
- Gorkem 最大的判断,更应被视为一项论点而非事实:「所有人都在等待 AI 迎来一个大规模消费时刻。现在它已经足够好、足够便宜,可以在其上构建一种真正全新的社交 AI 体验。」下一步基础设施工作包括将流水线拆分到不同硬件类型,并把部分任务推送到用户家中的消费级设备上。
5. Hollywood 是增长最快的板块,但它要的是点状解决方案,而非登月计划
- 发布近3周后,H3 Max 按调用量计已成为 Fal 最受欢迎的视频模型,「略超2倍」,并逐渐成为许多其他平台的默认模型。Hollywood 的使用量从「一年前还不存在」变成增长最快的板块;Amazon MGM Studios 的 Nara 工具「幕后大部分由 Fal 基础设施支撑」。
- Gorkem 对市场缺口的判断是:「Hollywood需要的、创作者真正需要的,与研究实验室正在做的之间,存在一点错位。」制片厂想把视频稍微延长、改变机位控制或调整光照,而不是从头生成全部内容;Fal 的后训练项目正好针对这些小型点状需求。
- 可控性路线已经逐级推进:文本生成视频、图像生成视频、带逐角色声音的参考素材生成视频;现在又加入 JSON 机位控制——「我希望 T0 时相机位于0、0、0,T1 时转到90度角」——以及光照方向、唇形同步和动作迁移。由于基础模型原生并不「遵守」机位指令,这些能力都需要大量调优。目标是99.9%的可靠性,相比提示词只能达到的80–90%;通过统一基础设施,目前可以套用到任何模型上,包括闭源模型。上线后出现的 Blender 工作流,则是渲染低分辨率场景并将其作为参考输入,已经让专业用户接近「100%可控」;使用 GPT Astra 的 Blender 内 LLM 流程进一步加快了这一过程。
- Hollywood 面临的另一半问题是法律问题:Fal 现在已经提供 IP 解锁申请系统和美国托管的 CDance——这是每家制片厂都在等待的「缺失部分」。Gorkem 的预测是:「现在这些 Hollywood 制片厂面前已经没有障碍……未来几个月,它们会把 AI 使用量提高10倍、100倍。」他引用的证据是:Fal 第二届 Generative Media Conference 现在已经「由制片厂主导」,而去年还只有几名抱着好奇心参会的高管。
完整逐字稿
Generative media, along with the coding agent market, is what we call token market fit. Everyone's waiting for a large consumer moment in AI. I believe H3 Max makes it possible.
Were you surprised by the speedup and the gain you could get from post-training this model?
We have a public version called H3 Max Turbo that can generate a 5-second video in 1.5 seconds. From a cost standpoint, it's also 2× less expensive.
People started creating these beautiful scenes using an LLM model, GPT Astra, in Blender, and all of a sudden, it unlocked a whole new workflow for Hollywood and professional people.
We've been very, very focused on speed, performance, and quality, and now we have a really good base model. The next month or two is gonna be fully focused on
Welcome, Gorkem and Batuhan, to our podcast again. We did the last one last year. This is long overdue, and we have such an exciting model to talk about, which is Hailuo 3 Max. When it came out, I called it, “It's really in a league of its own.” It's so funny to see the benchmarks where you have the dot for this model on the far left or far right, and everything else is on the other half.
That graph is actually on a log scale. It's actually farther out, but we had to fit it in, so we had to use a log scale. The time portion is on a log scale; the quality is not.
That is hilarious. The internet noticed, for sure.
Yeah.
There were so many viral tweets about it. People really played around with this model. Maybe just give us the backstory of what inspired you to post-train this open-weight model from MiniMax, and how you got the quality and speed to—
Yeah.
—to where it is?
First of all, the Minimax H3 model is the first truly open-source, very capable, latest-generation video model out there. Even though we work with some of the other model labs to run inference for them, we never had the right to add this capability on top of their models.
So when MiniMax came up with their open, very capable, open-source model that is truly latest-generation and can take references, with an architecture that is very familiar from other video models, we thought this was a great opportunity to go all in and see what we could do. We did many different things that we're going to talk about, and together they produced the results that you see on the graphs. The biggest reason why everything came together for this particular moment was that Hailuo 3 was the first truly next-generation video model that was open source.
Given that fal has been known as a generative media inference-serving platform, what was the idea behind getting into post-training an open-weight model? You talk quite a bit about it in the blog, about combining the systems work with the model itself. Maybe talk more about the work behind that.
Generative media, along with the coding agent market, is what we call token market fit. The way we define it is: Can a single person productively spend a lot of tokens? The amount is something like $10K a month.
There's an incredible amount of demand in the market to generate video and many other things at the same time. A person who is doing this for their daily job spends all day in front of a computer doing this, and they spend thousands of dollars—lots of tokens.
Since around April, the whole industry, including fal itself, has been compute-constrained. We're growing as much as we're adding compute. There are things we do here and there, but the whole industry has been compute-constrained. We've always been looking for efficiencies that could relieve that a little bit, so people can use these models more.
That has been the idea behind everything we've been doing since April. This came at the right time because it makes everything maybe an order of magnitude more efficient. It gives us more compute for other models, or allows more tokens to be generated using H3 Max. I think Batuhan would agree on that—
Mm-hmm.
—on that, like—
From system-wide optimizations, which is what we've been doing for the past 3 or 4 years, you can maybe make the model 2× or 3× faster while producing the same quality. At the end of the day, it's the same model and the same architecture. You have the same constraints; you're just trying to optimize what you can get out of the chip itself.
There is a roofline there. We've been approaching that roofline more and more, especially lately, because our entire team has been focusing on how to get as many video pixels as possible out of a single chip. This new set of post-training-related optimizations, with system-model co-design, enables us to go beyond that roofline by an order of magnitude.
We just felt the pressure. We had been working on this with open-source image models before the video models. We did one version with Ideogram and one version with FLUX. We've been experimenting with how to build post-training infrastructure to take an existing model, build kernels and systems design around it, and run it very, very fast for a specialized version that can beat anything else we would get just by running the model itself.
The combination of that, getting a frontier video model in our hands, and all this expertise allowed us to go up by an order of magnitude in terms of speed.
Incredible. Let's dig into that. I may get some of the numbers wrong, but—
There are efficiency numbers, cost numbers, and speedup numbers.
Right.
Not everything means efficiency, but it all adds up to being very efficient.
Yeah. What's stunning to me is that there's a magnitude-lower cost and also a much faster speedup. I think it was a 35× speedup, right?
Yeah, compared to the original Minimax H3 endpoint.
Compared to the original. At the same Elo score, you didn't really sacrifice quality. Maybe reveal a little more of the secret sauce. Is this mostly the kind of systems work you've done? Did you have to change the model architecture?
Mm-hmm.
Is it the systems work that really brought down the cost and latency? And how do the next generation of chips, like the GB200, fit into the whole story?
It's just a compounding effect of multiple different optimization variables that we've been targeting. The first one is that you go from the base model to a model that's been post-trained to be more efficient. For diffusion models, this is essentially how you go from running 50 steps to running something like 20 steps. You're trying to optimize that pipeline, but as soon as you go from 50 steps to 20 steps, you lose quality.
Right.
So in the optimization scheme, you need to say, “I want to improve the quality, and then I want to apply the optimization.” We have checkpoints that are significantly higher quality but obviously slower.
What we initially did was run our post-training and RL pipelines so we could improve the model's quality, and then apply the optimization stack on top of it. The end result gets you to the same quality, or even higher quality, than the original model, but at the same time, you're an order of magnitude faster.
Most of the gains come from post-training this model to make it compatible with running on fewer steps. On top of that, you add all the kernel and systems-engineering work that brings your hardware utilization from 30–40%, which is standard in many inference workloads, to 70–80%.
You're essentially trying to get to the maximum possible output. Seventy to 80% of theoretical MFU is impossible to reach, so you're essentially at the roofline of what you can get out.
These models aren't just a single thing where you give a prompt and get a video back. There are actually pipelines underneath. You need to take a prompt, run an LLM—a very large LLM—to expand that prompt into the format the model was initially trained on, generate the video in latent space, and then decode those latents back into pixels.
Right.
Depending on the workload, there might also be an upscaling component involved. There are multiple components, and every single component is unoptimized by default. There's still a lot to be gained there, and we looked at it from the perspective of getting the maximum out of every single component.
This made us run LLMs at super-high speeds. That is another component.
But for a different workload, this isn't like an agent-decoding LLM workload, where you have very high cache rates and higher sessions. It's a single shot: You give a prompt, you get an output back, and there's no caching. You're operating at low batch sizes. So there's a completely different set of optimizations on the prompt-expansion side, a completely different set of optimizations on the diffusion model, and a completely different set of optimizations on the VAE that takes you from latents to pixels. You combine all of these to create an effect that compounds.
From a hardware standpoint, going from Hoppers to Blackwells, you see a 2–3× improvement by itself. But from a cost standpoint, it's pretty comparable because the cost is also in that league. So I would say it only reduces your wall-clock time, but not the efficiency itself. It obviously helps if you want to go significantly beyond real time. If you want to generate 5 seconds of video in less than 2–3 seconds, then you need some of this latest-generation hardware today to unlock that possibility.
Maybe this is a detailed question: Is the model being served on a single GPU, or is it—
The majority of video models today run on a single-node configuration, which is 8 GPUs, because once you start scaling beyond 8 GPUs, the efficiency gets lower and lower because of the communication overhead. Existing MiniMax Hailuo 3 endpoints, as well as other video models, are probably being served in a single-node configuration. It's the same with this. It's running in parallel across 8 GPUs.
And do you think there will be more efficiency gains there that you can either optimize more of the steps in between by sacrificing some of the user experience—say, narrowing down the different types of inputs and outputs—or, as you're thinking about parallelism, is there more juice to squeeze? That's the question, too.
Mm-hmm.
Yeah. We released a turbo version of H3 Max. The initial idea was to call this H3 Turbo, and we were like, "We don't want to call this Turbo because the quality is better than the original one." This needs to signify how good of an achievement it is. So we released H3 Max, but a week later, our team was like, "We can run this 2× faster at 97th-percentile quality." We ran evaluations, and they're almost the same. There's still a small, noticeable loss in quality, but we have a version called H3 Max Turbo that's public that can generate a 5-second video in 1.5 seconds, which is insane. From a cost standpoint, that's also 2× cheaper.
It depends on how okay you are with losing quality. You can go down from there. Today, these models are so cheap and so fast that I don't think people need them to be any faster or cheaper. They're already at a point where, from a cost standpoint, compared to the frontier itself, they're an order of magnitude cheaper. From a speed perspective, they're more than an order of magnitude faster, and they enable all these experiences.
I think we would need to see what other levers people would need, but my bet today is that we need to improve quality more than speed at these speeds. Let's fix the speed, and let's try to push for quality and controllability of these models, which is what we've been pushing over the past 2–3 weeks.
I think controllability is key. When we first did it, we did text-to-video and then image-to-video, and references came later. That adds a ton of controllability, and it's basically the default mode in which people use these models these days: references.
We're now also adding different LoRA fine-tunes of the base model. We're working on a lip-syncing version, different camera-angle LoRAs, and different style LoRAs. Again, open source adds a whole ecosystem around the model, and it really, really helps.
Were you surprised by the speedup and the gain you could get from post-training this model? I saw it as a bit of a surprise that one day—I think it was a Saturday—you launched the model, and on Sunday people put it on Twitch. It became a real-time model.
Yeah.
That's the interesting part of why you released it.
We did evaluations. We spent a ton of money doing evaluations on our own—tens of thousands of dollars, even. The results were unbelievable. The plan was to release the model without doing external evaluations, but then we decided to hold off. We didn't want to tell people that this was so much faster and so much better before we had some external validation.
So we waited 3–4 days for all these other evaluation platforms to actually run the evaluations. We matched the results we had externally as well, and that's how we launched it because, as you said, the results were a little too good to be true.
Yeah.
And it was.
I guess, were you taken by surprise by the real-time use case that came out of it? Or what are some examples that you think this model has unlocked—experiences that prior models couldn't?
This—
This happens at fal once every couple of months, where the whole company gets hold of something and the creativity just explodes. Everyone starts working on a new little app, a different optimization LoRA, or whatever it might be. The whole company gathered around this model, and some front-end engineers started working on interesting applications.
We can talk about our world model accelerator team, which is brand new. They started working on the live—
Experience.
The WebRTC live experience. There were 5 or 6 different parallel little projects within the company. I think we broke a record on Slack that day for how many messages were sent in the company.
We have a distributed team, with people all around the world—most of them in San Francisco. It's incredible when you see the 24-hour development: people working 16 or 17 hours, and then someone else waking up and picking it up. That went on for 3 or 4 days, and that's when we released all these projects.
Take me into that. It's so interesting, because you imagine a model or product launch being planned out, with all these evaluation vendors ready and lined up, and then you ship something out. Then you let the world—or the external users—take it, experiment, build experiences, and put them online.
It seems like people internally who are very creative just took this, dropped everything they were doing, and launched an experience that got really popular on Twitter. Do you want to tell us about that one?
Yeah, of course. One of our engineers, Rehan, just by himself, started streaming continuous generations of H3 Max from his laptop. He was just—
His computer.
His computer, exactly. He was doing some prompt tricks, trying to keep a coherent story, and then he started livestreaming that on Twitch.
In parallel, Levels IO, a famous Twitter influencer at this point, had a similar idea. He reached out to us and said that he already had a website ready. He wanted to host the streaming himself and have a website that does infinite streaming.
Internally, we also had another team working on a continuous version of H3 Max. The Rehan and Levels IO versions were independent clips. It's still very fast, but the clip starts and ends. Then you take the last frame of the clip, try to put it into the next one, and try to create a continuous—
You need to put some work into—
Yes.
—the last-frame prompt. Yeah.
And there's no memory. The second clip doesn't really remember anything from the first clip other than the last frame.
But internally, the ML team was working on a version where the transition is more seamless. There are 2 minutes of memory. You're in a scene, and when you direct the model or someone else enters the room, everyone looks at that person entering, and the scene is continuous.
Internally, we were working on that, and another team was working on an experience we called fal Live for the continuous version. So we had 3 parallel efforts going on that were all independently going viral on Twitter, by the way.
And these were all spontaneous. You didn’t—
We didn’t plan for it at all.
…plan for any of them.
Yes. Exactly. Yeah.
And they just became products and experiences.
But then let’s talk about how we made the model more continuous.
Mm.
That was very surprising to me because I’ve never seen that actually work on a video model before.
Going back, we have been very focused on role models and essentially action-controlled, or action-driven, real-time, continuous streams of video. The problem, until something like H3 Max, was that the quality was not good enough at all. It degraded a lot and didn’t remember the past.
Before that, we built the infrastructure to stream video, have people control it in real time, multiplex it to multiple people, and keep latency very low. At the same time, our ML team was trying to take every video model and apply a set of optimizations and tricks: How can we make this generate a 15-second video or a 30-second video instead of a 5-second video? But we were always below the real-time factor, where we could never generate 5 seconds in under 5 seconds.
Once H3 Max unlocked it, the ML team—which is a separate team internally; we have a research team, an inference team, and an ML team—saw this and said, “This is insane.” They could apply the learnings from previous models where they had attempted to do this. Instead of trying to generate a 5-second chunk, they could try to generate a 10-second video, and the 5 seconds from the previous one were still attended to. The model still remembered that context, and as the video went on, we could extend that memory up to 2 minutes.
You need extremely clever optimizations because attending to a 2-minute video is extremely compute-intensive, and it goes up exponentially from a compute standpoint. We did lots of optimizations there, but at the end of the day, we were able to remember back 2 minutes, which is generally good enough from a memory perspective. With prompt tricks, you can still continuously remember more fine-grained details beyond the 2-minute mark. You can essentially stream infinitely; we capped it at 1 hour from that perspective.
That team then released the model as H3 Max Director, which is public for people to use. I think it’s the only model that can generate up to 60-minute continuous videos that are action-controlled. You can start with a prompt—say, an office setting where someone is working—and then 30 seconds later it imagines by itself. Thirty seconds later, you can say, “A woman walks in through the door,” and it can take the prompt and reflect it immediately, which is the most fun part.
And the office is still the same office.
Same office.
The camera can pan back to the original person, and the original person is still there in the same state. Yeah.
So—
Yeah.
We released that, and it got a lot of attention. We built this fal Live website to demonstrate it because people need to see how cool this is. This is a new technology, and I don’t think people are really aware of it. It also went viral immediately because we let people vote on what the next section would be. It was a form of—
Crowdsourcing. Yeah.
Crowdsourcing, where the chat was controlling whatever was happening. Which is fun, but obviously we limited the options, and they could pick, say, a banana enters the office instead of a woman, which is more fun. We started adding more channels, and every channel had a concept.
There was a channel that was full chaos. There was a channel with cartoons from the ’80s, and the model was extremely capable: It remembered so many different concepts and had a big memory from a style and concept perspective. It just became a very fun experience underneath.
Again, there are so many incredible experiences coming out of this. H3 Max Director was another huge surprise to me. I found it interesting in the generative media market that it’s not like with a language model, where you have this linear graph of continuously compounding intelligence and capability.
It feels like in the field you’re operating in, there are always a few months of quiet time, with a lot of things bubbling, but then in a very short period, everything bursts. All these things come together: The base model is good enough, and the latency is low enough that you can—
References. Yeah.
Yeah.
Yeah.
Get the real-time experience, but also apply controllability on top of that real-time experience. This opens so many opportunities for live experiences where the end user can control what’s happening on the screen, which is incredible. We’ve imagined many of these experiences, but never been able to really play around with them.
Maybe just tell us more about what you’re seeing in the market. How are people using the director capability? What are you seeing creators create that you haven’t seen before, and what do you think that unlocks in terms of what people can do with this medium?
It’s been almost 3 weeks since we released H3 Max, and already it is the most popular video model on the fal platform by almost double—a little more than double—in terms of volume. On a lot of other platforms, it’s also becoming the default model that people interact with because it’s so fast and cheap; it just makes sense.
If you come to a platform, this is the experience you want to see. In terms of popularity and volume, it’s taking over, at least from our vantage point. For Max Director, there have been, I don’t know, tens of different versions of these live streams. Some are still going on and becoming more and more popular.
We’re trying to work with some AI IP holders—people who have AI shows on Instagram and TikTok—and train a LoRA on their style and do a live version of their show. We have a couple lined up already, so that’s going to be very exciting.
If you talk to a creative technologist, prompting with voice has already become something they use all the time, using Whisper Flow or ChatGPT’s voice mode. And now you can keep talking to the model, and it’s almost as if it’s a real director on a real movie set, directing the camera and directing people where to go. You can do that.
Our creative engineers started using these models that way. We’ll see. A lot of interesting experiences are being built as we speak.
Very interesting. The video is playing on the screen—
The video is playing, and you are—
And then—
You are talking to the video, and what’s being displayed changes accordingly. Yeah.
That’s incredible. And talking about how the memory works now, this may be a technical detail: the capability of remembering what happened in the last—
Mm-hmm.
…scene or in the last couple of minutes of the scene—are you remembering that through the frames, the images, or is it through text—
It essentially—
…condensed—
No, it’s essentially the raw video, obviously very compressed because you can’t attend to the full video. But it essentially knows most of the details that happened in the past 2 minutes from its own generations.
Above the 2-minute mark, think of it as having an evolving system prompt covering minutes 2 through 60; it knows the overall structure and overall detail. It remembers the last few scenes. If you think of a scene as 15 or 30 seconds, it remembers the last 4 to 8 scenes.
On top of that, there’s a continuously, gradually evolving system prompt that keeps remembering the overall coherence of the world.
And everyone's waiting for a large consumer moment in AI. Now it's good enough and cheap enough that a truly novel social AI experience can be built on top of it.
Maybe let's talk more about the economic side of this. What is, I guess, first, just talking about serving cost for—
Yeah.
—the same minutes of video with H3 Max, and how has it changed your thinking around your footprint, your inventory of chips? How do you want to have different steps of experiences serving to the end user?
Yeah. Batuhan mentioned this a little bit. Everyone talks about how complex the next-generation LLMs are, but video models are actually very complex as well because the pipeline has different components, and sometimes they require different hardware configurations for efficiency, things like that. If we were to make this even more efficient, maybe go even cheaper, we would probably run different parts of the pipeline on different types of hardware. Another interesting thing would be to run it on consumer hardware, so people can run it on their own machines at home. Optimizations don't translate 100%, but they translate somewhat close to that, and then we can do extra work to translate more of it. Doing these optimizations on different types of hardware and combining the pipeline in a way that's even more efficient is what we're going to do in the next coming weeks.
Amazing. So you will have people like Rohan that can stream part of the experience from his computer, but also have the director and the control plane more living in the cloud.
Exactly, yeah.
Makes sense. So we talk about all the consumer experiences this model could unlock, and it seems like Batuhan is happy with all the efficiency squeezed out of the GPUs. Now we're talking more about how we improve the quality and controllability of these models so that the high end of the market—the Hollywood creators and directors—can take this to the next level.
I saw some demos. Coincidentally, this model came out the same week as, or a week prior to, Astra. People were combining the Blender experience with H3 Max from fal. Talk about how it's going to impact the Hollywood world.
Using Blender with one of these AI models together is an extremely popular workflow for professional work. Basically, you render a low-resolution version of your scene, what you want to do, using Blender and previous non-AI technology. Then, once you add that video as a reference to an AI model, you basically get close to 100% controllability.
This is an incredibly popular workflow for VFX artists and people who are doing this professionally, because they want exact results. They want to get exactly what they put into the model. As you mentioned, a week after we launched H3 Max, people started creating these beautiful scenes using an LLM model, GPT Astra, in Blender. All of a sudden, it unlocked a whole new pipeline: using an LLM to create a Blender scene and then passing that to the H3 Max model, or any video model.
It works very well with H3 Max because it's extremely fast, and you can try many things all at once in parallel. That unlocked a whole new workflow for Hollywood and professional users, and it gets you close to 100% controllability.
As I said, we have been very focused on speed, performance, and quality, and now we have a really good base model. I think the next 1–2 months are going to be fully focused on how much controllability we can add to these models so that professionals at studios—professionals who want to actually produce content that fits their use case perfectly—can leverage them.
The team has been working on an amazing lip-synchronization model. You can supply the audio, you can supply a video or an image reference, and then it can synchronize the lips perfectly. The same goes for motion controls. You can take the motion of someone dancing and apply it to your AI-generated character, and it fits perfectly.
You can get these results with basic prompting, and you're going to get 80% or 90% reliability. What we are targeting is 99.9% reliability in the outputs, so that you can actually trust the model to have done every single aspect of the generation perfectly. That's what we have been pushing.
One big launch that we had last week was camera controls. Essentially, you can direct where the camera is going within the video perfectly, down to the degree.
And this is by describing it in the prompt, or—
It—
—generating the scene?
You essentially give it a JSON that says, “I want the camera at 0, 0, 0 at T0. I want the camera at a 90-degree angle at T1.” You supply a structured description of where your camera needs to be at any point in time, and then the model is perfectly conditioned to regard it as the only source of truth. It doesn't hallucinate about where the camera should go.
You can essentially reconstruct 3D scenes from a single input because the model itself is a very good video model. At the same time, it perfectly adheres to the camera controls.
And this is because the base model itself already has an understanding of the camera angle that you can—
It doesn't respect it; it just understands it. You need to tune the model to a significant degree, and this is what enables post-training infrastructure at large scale.
We now have the infrastructure to take H3 Max and add any capability to it. The same applies to any new model, right? If there's a new video model, we essentially spend most of our time building it as infrastructure rather than just doing one-off training runs, so that we can build services around this—not just for open-source models, but for frontier closed-source models as well.
We see in the market that the biggest gap is how controllable these models are. First, we started with text-to-video, where you put in a prompt and get a video back. It was good, but you never could describe the perfect character for you.
Mm-hmm.
Then we had image-to-video, where you use an image-editing model and then generate the first scene. The model was obviously much more fitting, but you still couldn't say, “I want this new character to appear at second 3.” You needed to put it into your first frame. You could prompt it, but these models are never perfect.
Then we added reference-to-video, where you can provide an initial starting frame, and you can also provide, “I want these characters with these voices.” That's also a big unlock, where you can essentially say, “This is the voice for this character.”
Now we're adding, “Within this scene, I want the camera to look at this degree at T0. I want the camera to look at this degree at T3.” Then we're adding lighting controls, where you essentially say where the light is coming from.
These are all compounding on top of each other, and we have the unified infrastructure to apply this to any model at this point.
That's incredible.
Hollywood is our fastest-growing segment, and there's a lot of noise about how AI might disrupt Hollywood. Hollywood usage was nonexistent a year ago, but in the past year it grew, and now it's the fastest-growing segment.
Amazon MGM Studios, at their conference, released their Nara tool. It's mostly backed by fal infrastructure behind the scenes, and we are seeing incredible pull coming from Hollywood. It's exactly what they need: these small point solutions rather than generating everything from scratch.
They want to be able to extend the video a little bit. They want to be able to change the camera controls. They want to change the lighting, and someone has to build these solutions for them.
What Hollywood needs, what the creators actually need, and what the research labs are working on—there's a little bit of a disconnect there. We believe we can come in and do these little post-training projects to close that gap, because we work with all the Hollywood studios and hear from them what they need.
These are exactly the things they need: small point solutions that make them more efficient, push out more video, and allow AI to close that gap very nicely.
Maybe say it in a little bit different way. We've been staring at this problem for the last 3 years as well. We see companies trying to build a movie director, a video model, by—
Yeah.
Either pre-train or post-train on the video side. But what I'm hearing is different people expressing the way they want the output to come out very differently. Consumers talk about it and then write the prompt and generate the results very differently from a Hollywood director—
Professionals—
—which is obvious, right?
Yeah.
Yeah, yeah.
Professionals wanna talk about these camera angles. They wanna talk about the lighting. You've built a library or a collection—
Toolkit, yeah, yeah.
—of post-training data and toolkits that can apply to any model that you can grab the weights of, so that they are adapted to a different audience, where they can express their creativity in a bit different fashion to control the model—
100%.
—which unlocks a lot of capability underneath.
Half the problem was the capabilities of these models. We are solving that. The other half of the problem was legal and data residency, things like that.
Yep.
We made a ton of progress there as well. We now have a system where people can apply with their own IP, and we unlock their own IP in the models. We are gonna grow that, and that's gonna be a very powerful thing we do with Hollywood studios.
Also, we now have CDance U.S.-hosted as well. We already had other Chinese models; CDance was the missing part. Every Hollywood studio wanted us to have it U.S.-hosted. Now that's available. So there are no obstacles in front of these Hollywood studios now. Everything is ready, and we believe they're gonna 10x, 100x their AI usage in the coming months.
It's such an exciting world for movie lovers, consumers, people who consume a lot of video and creative content.
We have our Generative Media Conference next week. This is our second time doing it. Last year, it was mostly consumer AI. There were maybe a couple of Hollywood executives here and there, just curious about it, and now it's dominated by studios.
There are new AI studios that are offshoots of the bigger studios, trying to do only AI shows, but also the biggest Hollywood studios are there because now they have big plans to integrate AI into their workflows and their existing systems. So you can see the change in attendance at the conference as well.
That's awesome. And thank you so much—
Of course.
—Gorkem and Batuhan, for coming onto our show.
Thank you for having us. Yeah.
It's a super exciting time for Gen Media.
Thank you.