[BidClub_]
Latent Space · · 49 min

Mistral: Voxtral TTS, Forge, Leanstral, & Mistral 4 — w/ Pavan Kumar Reddy & Guillaume Lample

swyxVibhuPavan Kumar ReddyGuillaume Lample

YouTube ↗
TL;DR
  • Mistral is positioning Voxtral TTS as a low-cost, open-weight wedge into voice agents: a nine-language, 3B model designed around low-latency streaming. Its novel stack combines an autoregressive transformer, a 12.5 Hz neural audio codec, and flow matching rather than a second K-step autoregressive decoder. Pavan’s broader call is that audio remains unusually open terrain because “there is no winner model yet.”
  • The product strategy favors small specialists where scale adds little, even as Mistral Small consolidates mature capabilities into a sparse general model. Mistral Small combines instruction following, coding, reasoning, and vision with 6B active parameters and 256K context, but Shawn argued that transcription should remain a dedicated 1B–3B workload: a giant model carries “a lot of capabilities you don’t really need.”
  • Forge makes enterprise customization—not a generic endpoint—the commercial center of Mistral’s pitch. Companies may hold decades of proprietary knowledge amounting to trillions of tokens, yet a closed model leaves them using “the same model as all your competitors.” Mistral says continual pre-training or fine-tuning can eliminate repeated 10K-token context payloads and, in some production migrations, produce a better deployment at roughly one-tenth the cost.
  • Voice fine-tuning converts commodity speech into an enterprise-specific asset across language, acoustics, and brand. Examples included putting an Asian language at 50% of a training mix instead of roughly 0.1%, adapting ASR to medical terminology or noise, and building a 3B offline audio model with function calling for cars or kitchens. For TTS, the target is not celebrity cloning but voices matched to each company and use case: “You don’t want the same voice for all the enterprises.”
  • Leanstral is less a niche math product than a laboratory for verifiable, long-horizon agency. Arthur explained that Lean can accept any valid proof that compiles, avoiding the ambiguity and reward hacking of an LLM judge; agents can decompose a theorem into lemmas, send them to parallel sub-agents, and receive partial progress. Arthur also reported some transfer from formal reasoning into broader math, while the team expects formal software verification to expand beyond today’s aircraft, robotics, and safety-critical niches.
  • Open weights remain both Mistral’s stated mission and its ecosystem strategy, but deployment support is essential. Guillaume credited access to models such as Llama with enabling outside work on post-training methods including DPO-type preference optimization, while rejecting a future where the smartest systems sit “only behind closed doors.” Yet Mistral also admits it initially underestimated the difficulty of connecting models to company data, tools, agents, and multimodal workflows.
  • The next bottleneck is training infrastructure for trajectories that take hours, with customer deployments supplying the evals public benchmarks miss. Arthur said pre-training still has substantial room, while methods such as GRPO that worked for thousand-token math solutions do not readily handle six-hour paths to a reward because the model becomes completely off-policy. Forward-deployed teams close the loop through “real-world eval,” including cases as narrow as grading a child’s one-word pronunciation, while the science group explores similarly under-served AI-for-science problems.
Digest · the substance, structured for research

1. Voxtral TTS extends Mistral’s audio stack to speech generation

  • Voxtral TTS is Mistral’s first speech-generation model: nine languages, 3B parameters, native text capability, and what the team describes as quality competitive with the best systems at a fraction of GPT-4o’s cost. Alessio called it probably “the best open-source model”; Guillaume replied, “You’re just hyping yourselves, right?”
  • The release follows a deliberate capability sequence: Voxtral, Mistral’s first audio model, was released in July last year as an ASR model, followed by a transcription-only model family in January and production features including context biasing, diarization, timestamps, and real-time transcription. Speech generation is next; an integrated full-duplex audio-in/audio-out system comes later.
  • Guillaume’s operating principle is “step-by-step”: optimize transcription, speech generation, and then real-time capabilities separately before merging them. The joking destination was a “super omni model,” but the team does not want integration to obscure the economics or quality of each component.

2. Flow matching turns speech entropy into a streaming architecture

  • For understanding, Voxtral resembles Mistral’s Pixtral vision design: an encoder turns audio into continuous embeddings that enter the decoder as tokens, while the output remains ordinary text. Generation is harder because the model must produce audio rather than merely interpret it.
  • Pavan’s in-house codec represents audio at 12.5 Hz, with each frame carrying one semantic token and several acoustic tokens. Their embeddings are summed on input; on output, each transformer state feeds a flow-matching head whose velocity estimate moves noise toward the latent for an 80-millisecond audio frame, then a vocoder reconstructs sound.
  • A conventional depth transformer predicts K acoustic tokens autoregressively inside every time step. That works, but adds K sequential operations. Mistral instead trained its codec to support both discrete and continuous representations; both approaches worked internally, but “the continuous stuff works just better” and permits substantially fewer inference steps.
  • Pavan hedged the novelty claim carefully: audio already has flow-matching models, but he had not seen this exact autoregressive combination and “could be wrong.” His larger point was firmer: unlike text, audio has not converged on one accepted architecture, making it “pretty exciting to explore.”

3. Speech quality depends on choosing among valid performances

  • Guillaume’s intuition behind flow matching is that a known word still has many legitimate pronunciations, inflections, and rhythms—even in one speaker’s voice. He described these as clusters of possible performances: “You can’t predict the mean of it,” because an average across them would resemble blurred speech; the model must choose one sharp realization.
  • Disfluencies such as “ums,” repetitions, and filler words are only one part of this entropy; intonation and pronunciation remain variable even without them. A depth transformer models those dependencies conditionally, but Mistral’s flow head can work in roughly four or 16 steps, with a possible path toward one-step generation that the team is not yet using.
  • The architecture was chosen around voice agents: “We want real-time streaming.” Whole-utterance diffusion was therefore rejected early, while autoregressive framing lets audio become another head on a decoder transformer and leaves a cleaner route toward eventual native audio-text modeling.

4. Long-form generation inherits text-model context scaling

  • At 12.5 audio tokens per second, Pavan estimated that one minute consumes roughly 720 tokens. An 8K window can therefore accommodate about ten minutes; 32K covers half an hour, and the team is already comfortable training at that length.
  • Extending to 128K suggests hour-long generation without an architectural break. The unresolved work is the data recipe and the whole algorithm required to preserve coherence across that span; conceptually, however, it resembles long-context text modeling with autoregressive flow prediction replacing next-token classification.
  • In a later discussion, Shawn raised combining voice with video and spatial audio. Guillaume contrasted a standalone artifact consumed after generation with conversational audio, where extremely low latency and streaming are primary constraints.

5. Voice demand is real, but natural conversation remains unfinished

  • Guillaume said customer demand drove the audio investment, yet even transcription exposes the gap between benchmark competence and human interaction. English is ahead; he specifically noted that users talking to French models still slow down and over-articulate, while French, Spanish, and German remain weaker despite not being low-resource languages. He suspects people of the current generation may retain that learned habit even after models no longer require it.
  • Alessio’s comparison with Google Assistant underscored the progress: within roughly four or five years, systems moved toward end-to-end audio input, audio output, and function calling. His hedge matters, though—current voice modes still do not feel like speaking to a person, even if the existing stack “should be able to get” much closer soon.
  • Alessio framed audio as a productivity interface when the user knows what they want but would otherwise type a long specification, particularly for coding. Audio is “a natural interface that’s just not there yet,” making the opportunity an interface replacement rather than merely better transcription.

6. Mistral Small merges capabilities without abandoning specialists

  • Mistral previously developed instruction following, Devstral coding, Magistral reasoning, and Pixtral vision as separate artifacts owned by focused teams. Mistral Small is the first broad merge of those mature branches; function calling was among the capabilities not fully integrated at the time and is expected to improve substantially in Mistral Small proper.
  • The resulting mixture-of-experts model is highly sparse, with 6B active parameters and a 256K context window. Guillaume also confirmed that larger versions exist, preserving a scaling path beyond the “Small” label that prompted the hosts’ laptop jokes.
  • The hosts pressed the omni-model thesis. Shawn’s economic argument was that pure transcription does not need a model this large: a dedicated 1B or 3B system could deliver essentially the same performance far more cheaply.
  • Guillaume said future general models should absorb stronger coding and reasoning alongside neglected enterprise domains such as legal work, finance, and computer-aided design. These areas often lack fashionable benchmarks, but his prescription was blunt: source and process the data, because “you just have to do the work.”

7. Forge turns proprietary corpora into the enterprise moat

  • Guillaume’s sharpest commercial argument was that off-the-shelf closed models leave companies unable to leverage decades of accumulated knowledge—sometimes trillions of domain-specific tokens absent from the public internet. Inference-time context can expose the data, but training the model on the corpus is better and avoids resending perhaps 10K tokens on every query.
  • Forge packages the same data pipelines and efficient training infrastructure used by Mistral’s science team for continued pre-training, fine-tuning, and SFT. Guillaume called it “battle-tested” after roughly two years of internal use: customers are effectively receiving the tools Mistral uses to build its own models.
  • Privacy is another adoption vector. Enterprises divide information into sensitivity tiers, producing awkward workflows in which some data may enter a public cloud and some may not. On-premises or private-cloud deployment removes that routing problem, while customers also value having one partner span text, code, and audio rather than distributing sensitive data across vendors.
  • Production economics often become visible only after a successful prototype. Guillaume said customers arrive after discovering a closed endpoint is too expensive to scale; tailored models can sometimes be “10x cheaper,” better on the target workflow, and hosted on their own servers.
  • A separate host caveat on cost comparisons was that one figure represented inference cost while another included API-provider margins; such comparisons are not pure inference-cost comparisons.

8. Customization reaches languages, acoustics, and brand identity

  • One customer needed an Asian language that occupied perhaps 0.1% of a general model’s mixture. Mistral trained a model with that language at 50%, capturing dialects and slang rather than mere grammatical coverage. Another request combined audio and strong function calling in a 3B offline system for cars or kitchens where connectivity is unreliable.
  • Speech-to-text fine-tuning can add tail languages, specialized medical or technical vocabulary, and robustness to particular noise conditions. TTS support is expected soon, with enterprise voice adaptation focused on tone and personality rather than novelty cloning.
  • Guillaume framed the target situationally: an empathetic healthcare assistant should not sound like a routine customer-support bot, and each enterprise wants a voice representative of its brand and safety requirements. Alessio’s example was that two customer-service lines should not answer with the identical voice.

9. Leanstral makes formal proof a verifiable reasoning laboratory

  • Guillaume said Mistral’s open-source commitment has been there since the beginning and connected it to his earlier work at Meta releasing Llama. He credited access to such models with enabling outside post-training research, including DPO-type preference optimization. Detailed reports on reasoning, audio, and real-time encoders serve the same objective—avoiding a world where the best intelligence is “only behind closed doors.”
  • Leanstral grew from a small internal formal-math team that Guillaume said began working on reasoning too early, before LLMs, when formal-proof data and the surrounding community were negligible. Its advantage now is verification: numerical AIME answers are easy to check, but ordinary proofs can be correct in many forms, making reference comparison brittle and LLM judges vulnerable to reward hacking.
  • Arthur explained that Lean removes this ambiguity because any valid proof can be checked mechanically—“like a program, if it compiles and it’s correct.” That makes formal reasoning useful beyond mathematicians, especially for software verification that today is concentrated in aircraft, robotics, and other settings where failure threatens lives.
  • The agentic connection may be broader still. A model can propose three lemmas, dispatch them to parallel sub-agents, and prove the main theorem using their outputs; even one failed lemma may leave reusable partial progress. Arthur said transfer from formal reasoning into broader math is already visible, supporting Shawn’s thesis that proofs may proxy for planning and long-horizon coherence.

10. Deployments are becoming Mistral’s research feedback loop

  • Arthur said pre-training still has large gains available, but the harder frontier is algorithms and infrastructure for extremely long trajectories. Methods such as GRPO were workable for thousand-token math solutions, but when reward arrives only after six hours, the model can be completely off-policy by the time of the update.
  • The AI-for-science work is looking for “low-hanging fruits” in physics, materials, and other fields where domain researchers may not know current model capabilities. Mistral’s customer relationships provide the matching function: pair model researchers with specialists who hold unusual, proprietary problems and data. ISM Électronique was one concrete partner example.
  • Forward-deployed work ranges from CAD to possible vision use cases such as defect detection, as well as an evaluation that grades a child saying a single word. Those narrow cases expose failures no academic benchmark would contain; the lessons then flow back into base-model training, which the team called “real-world eval.”
  • Research and deployment therefore share tools, data pipelines, and feedback rather than operating as separate organizations. Mistral is hiring across Paris, London, Palo Alto, Warsaw, Zurich, New York, soon San Francisco, and remote roles, while trying to retain what Guillaume called a small, very agile team culture.
Guillaume Lample

When your customers use an off-the-shelf closed model, what's very sad is that they are not leveraging all the data that they have been collecting for 4 years or so, but for decades. So much data—sometimes trillions of tokens—in a very specific domain, their domain, which is data that you would not find in the public internet. So, if they are using those closed-source models, they are basically not benefiting from all these insights and all this data that they have collected through the years.

Shawn Wang

Okay, welcome to Latent Space. We're here in the studio with our trusty co-host, Vibhu. Welcome.

Vibhu Norby

Hey, I'm excited for this one.

Shawn Wang

As well as Guillaume and Pavan from Mistral. Welcome.

Guillaume Lample

Excited to be here.

Pavan Kapanipathi

Thank you for having us.

Shawn Wang

Pavan, you are leading audio research at Mistral, and Guillaume, you're a chief scientist. What are we announcing today? We're sort of coordinating this release with you guys.

Pavan Kapanipathi

Yeah, so we are releasing Voxtral TTS. It's our first audio model that generates speech. It's not our first audio model: we had a couple of releases before. We had one in the summer, Voxtral, our first audio model, but it was a transcription model, an ASR model. We released some updates on top of this, supporting more languages and a lot of full-stack features for our customers, like context biasing, diarization, and timestamping in the transcription.

We also had some real-time models that can transcribe audio not just at the end of the audio. You don't need to feed them your entire audio file; they can also work in real time. This is the natural extension in audio: speech generation.

We support 9 languages. This is a pretty small model, a 3B model, so it's very fast, and it's also state-of-the-art in terms of quality, at the same level as the best models, but much more efficient in terms of cost and latency. In terms of cost, it's also much cheaper—it's only a fraction of the cost of GPT-4o. That's something that we like: this model really exists.

Shawn Wang

Yeah, memory length?

Pavan Kapanipathi

That is that.

Shawn Wang

Yeah, what's the decision factor?

Guillaume Lample

It's a good question. There will be more. There will be more.

Shawn Wang

Ooh. Yeah. For one, any sort of research notes to add on what you have?

Pavan Kapanipathi

Yeah, maybe we'll dive into it later in the podcast, too, but it's a novel architecture that we developed in-house. We iterated on several internal architectures and ended up with an autoregressive flow-matching architecture. We also have a new in-house neural audio codec, which converts this audio into 12.5 Hz latent tokens: semantic and acoustic tokens.

That's the new part about this model, and we're pretty excited that it came out with such good quality. Like you were just mentioning, it's a 3B model. It's based on the Mistral model that we released just a few months back, and it's a trunk. It's mainly meant for the TTS stuff, but the native text capabilities are also there in the model.

Shawn Wang

There's a lot to cover. I always love anything to do with novel encodings and all those things because I think that's obviously increasing a lot of efficiency, but bugs also sometimes happen. You were previously at Gemini, and you worked on post-training for language models. Maybe a lot of people will have less experience with audio models in general compared to pure language models. What did you find that you had to revisit from scratch as you joined Mistral and started doing this?

Pavan Kapanipathi

At least when it comes to the two buckets, I guess: audio understanding and audio generation. The audio understanding, like the Voxtral models that Guillaume was mentioning, were released earlier. They were released in July last year, and the follow-up transcription-only model family was released in January. That could be one bucket, I guess, and generation is another bucket.

I think you can also treat them as a unified set of models, but currently the approaches are a little different between these two. To your question about how audio is fed to the model: in the understanding model, it's very similar to the Pixtral model that we also released. That was the first project I worked on after joining Mistral. It was pretty nice, and Voxtral was very similar in spirit.

We feed audio through an audio encoder, similar to images through a vision encoder, and it produces continuous embeddings, which are fed as tokens to the main transformer decoder model. The model output is just text, so on the output side there is nothing that needs to be done in these kinds of models.

The interesting part about the generation stuff is that the output now has to produce audio. We have a neural audio codec, which converts audio into these latent tokens. There is a lot of existing literature and a lot of models based on this kind of approach, and we made slightly different design decisions around this.

At the end of the day, the neural audio codec converts audio into a 12.5 Hz set of latents, and each latent has a semantic token and a set of acoustic tokens. The idea is that you take these discrete tokens and feed them on the input side. There are several ways to fuse this at each frame, but we just sum the embeddings, so it's kind of like having K different vocabularies and combining all of them because they all correspond to one audio frame on the input side.

The output side is the interesting part. I don't know if it's the most popular, but one popular technique is to have a depth transformer because you have K tokens at each time step. With text, you just have 1 token at each time step, so you predict the token from the vocabulary and get probabilities.

Shawn Wang

This is a very straightforward technique.

Pavan Kapanipathi

Very straightforward. But if you have K tokens, then the main thing would be to predict all of them in parallel, and that doesn't work—at least, it doesn't work that well—because audio has more entropy. One of the techniques people use is this depth transformer, where you almost have a small transformer, or it can be an LSTM or RNN as well, but people use transformers and predict the K tokens autoregressively in that. So, you have 2 autoregressive things going on.

The thing we did differently is that instead of having this autoregressive K-step prediction, we have a flow-matching model. Instead of modeling this as a discrete token set, we train the codec to be both discrete and continuous, to have this flexibility. We did try the discrete stuff, too, and it works well, but the continuous stuff works just better.

There is a flow-matching head which takes the latent from the main transformer and, kind of like in diffusion, denoises it. But in flow matching, it's a velocity estimate. You go from this noised latent all the way to the audio latent, which corresponds to the 80 ms of audio, and then it's sent through the vocoder to get back the 80 ms audio frame.

Shawn Wang

Yeah. Is this the first application of flow matching in audio? Usually, I come across this in image generation.

Pavan Kapanipathi

Yeah, actually, in some sense, there are flow-matching models in audio, but I think this specific combination—I could be wrong. There could be some work I haven't seen.

Guillaume Lample

I haven't seen much work in this. So, I think it's novel, and a lot of it is just a way bigger community today. I think they pioneered a lot of this diffusion and flow-matching work, and it's interesting to adopt some of the ideas there into audio.

Personally, that's the key part. Trying things out is one of my main points: unlike text—even in vision, I think this is true, but in audio it's definitely true—there is no winner model yet. There is no, "Okay, this is the way you do things." It's still evolving. People are still iterating and figuring out what's the best overall recipe, I guess.

I'm pretty sure there are models which are also completely end-to-end, like native audio [?] and native audio [?], but it's still not like the field has come to a convergence point with this being the right way to think about it. That also makes the space pretty exciting to explore.

Shawn Wang

What are some of the ways to look at it? There are ways where you can do diffusion for audio generation, but if you want real-time generation, that's a big thing with the approach I'm assuming that you took. Also, how do you go about evaluating different axes of what you care about?

Pavan Kapanipathi

Good point. You can do just flow-matching diffusion for the whole audio. We didn't even go down that path because one of the main applications is voice agents, and we want real-time streaming. That's the use case. That's not the only use case, but it's one of the primary use cases we want to get to.

So, we picked the autoregressive approach for that. Within the autoregressive space, again, you can do chunk by chunk, or you can do—so, we picked the—I think, at least personally, I prefer the approaches which are the simplest, I guess. We tried to see whether we could just add audio as another head to our regular transformer decoder model, because that kind of makes it easier for eventual end-to-end modeling of audio-text-native modeling.

Yeah, and it works pretty well. So, I guess we went with that. We experimented a little bit with the flow-matching head itself. We had a discrete-diffusion kind of approach, which also works well, but the flow-matching one worked better.

Alessio Fanelli

I was just curious about how you think about this overall direction of research. When you work with the audio team, do you set some high-level parameters and then let them explore whatever, or how does it work between you guys?

Guillaume Lample

No, I think the way it works is that we are prioritizing together what the most important features are. There are many, many things you can do in audio, so we try to decide how we should do things. For instance, ultimately what we want to do is build this full-duplex model, but we are not going to start that directly.

Alessio Fanelli

Just to confirm, does full duplex mean it can speak while I'm speaking?

Guillaume Lample

Audio in, audio out.

Alessio Fanelli

Yeah, yeah.

Guillaume Lample

So, ultimately, we are going to get there, but for us, we decided to take it step by step. We start with whatever is the most important, I think also for our customers, which is transcription, the most popular use case. Then there's speech generation, and real-time just a bit beyond that. Then we're actually going to try to combine everything together. But we thought it was also important to separate things and optimize each capability one by one before we merge all of that together.

Alessio Fanelli

And then the super-omni model.

Guillaume Lample

What's interesting is that, as I was saying, when you work on some other domains of this LLM and other things, there are many areas where I think it's not as interesting. For instance, in many places it's essentially just around data or creating new environments, and a lot of easy things—things where I think the research is maybe not as interesting.

While in audio, there are so many ways to actually build this model, so many ways to go around it. This space is, I think, really interesting. What we also tried for speech generation is a mixture of multiple approaches. What was interesting is that even though they were extremely different, they ended up being, at the end of the day, pretty close. But flow matching turned out to be quite more natural, so we are happy with it.

Shawn Wang

Is there an intuition for why flow matching maybe just models speech better in some natural, fundamental latent dimension?

Guillaume Lample

No, I think the main thing is that, even at a particular time step, there is a distribution of things to be predicted. You already know the word that you're speaking. In text space, let's say the word maps to just a single token, for simplicity. In most cases, it does, so you just pick the word.

But within audio, even the same word, with your own voice, could be inflected in so many different ways. I think any approach that models this distribution—and flow matching is one of them; it's not the only one at all, but it's the one that works pretty reasonably well—is better. You have to pick across several different possibilities.

The intuition I have is that there are several different clusters, each corresponding to some specific way you would inflect or pronounce that thing. You can't predict the mean of it because that corresponds to some blurred-out speech or something like that. You have to pick one and then make it sharp.

Shawn Wang

Conditional inference.

Guillaume Lample

Yeah, exactly.

Shawn Wang

Is that all covered under disfluencies, which I think is the normal term of art? Disfluencies, pauses, intonations. By the way, I have to thank Sophia for setting all this up, including some of these really good notes, because I'm less familiar with audio. So, maybe—

Guillaume Lample

No, no, no. I think disfluencies are definitely one such phenomenon. Disfluencies are more like—

Alessio Fanelli

Which is ums and ahs—

Guillaume Lample

Yeah, ums, ahs, and also repeats. You feel like you're doing these filler words while you're thinking, so you repeat the word.

Alessio Fanelli

Okay. Whereas intonation is like a way of speaking?

Guillaume Lample

And yeah, I think there is a lot of entropy in modeling it as a distribution, and any technique that helps with it—the depth transformer is a conditional way of modeling this, and transformers are actually good at it, but it's a mini-transformer, so I think that worked pretty well for us too.

The main consideration is that when you have a depth transformer, if you have K tokens, you need to do K autoregressive steps. So even though it's a small thing, it's K steps, which is very big—heavy, to say the least. But with flow matching, we were able to cut it down significantly, so we are able to do the inference in 4 steps or 16 steps, and it works pretty well. There are more novel techniques to bring it down even further, in the extreme case to 1 step. We're not doing it yet, but at least the framework lends itself to more efficient—

Alessio Fanelli

Yeah, and the image guys have done incredible work. Now you just send the prompt and you get an image.

Shawn Wang

Yeah, surprisingly, not enough image-model labs use those techniques in production. I feel like it's a lot of research demos, but nothing I can use on my phone today.

Guillaume Lample

The thing that's interesting here is that, since there has been so much work done in the vision community compared to audio, to me there are so many learnings and so many things we can do to actually improve this paradigm further. I don't know if it's our first version, but we have so many ways to make it much better and much more efficient, cost-efficient, so—

Alessio Fanelli

Yeah.

Guillaume Lample

So certainly you feel that, of course, but there are still so many things that can be done. I feel that it's just—

Shawn Wang

I should also mention, for those who are newer to flow matching, I think the creator is this guy named Alex. I think he did, at NeurIPS maybe 2 years ago, a very good 1-hour workshop on flow matching. I recommend people look that up.

That's the other thing, right? Efficiency-wise, I imagine the reason it's open weights, the reason you picked a 2.4B backbone, is that you're trying to fit some kind of hardware constraints, some kind of business constraints. What are they?

Guillaume Lample

Not necessarily. I think something we care about in our models is that they are efficient. We have a lot of separate models, for instance. We have this model that we know is very small and very efficient. We also have a small model that we think is very, very good and very efficient as well.

I think an approach that maybe other companies are going to take is to have a very general model that will do a bit of everything, but that is also going to be expensive. The other thing I want to say is, if you care about this specific use case, you can actually use this model. It just does that. It's extremely good at it, but it's also very efficient.

That's why we can actually offer models, or you'll be able to access our APIs, that are really, really good at that and much more cost-effective than another general model that will contain a lot of capabilities you don't really need. So, that's all. We're doing general models, but also more customized models that exist.

Alessio Fanelli

How does it compare to other TTS models? You're going fully open weights; you're just dropping it like—

Guillaume Lample

I think it's pretty good.

Shawn Wang

Yeah, I think it's pretty good. It's definitely one of the best.

Alessio Fanelli

For sure, for sure. It's probably—I would say it's the best open-source model, right?

Guillaume Lample

You're just hyping yourselves, right?

Alessio Fanelli

Yeah. Why now? How does it fit into the broader Mistral vision? How do you see voice agents? How do you see voice—

Shawn Wang

I think every year I've heard, “Okay, you're a voice, you're a voice.” There's a lot of architectural stuff. There's a lot of end-to-end latency that you're solving, but where do you see voice heading?

Guillaume Lample

We had so many customers asking for voice. That's also why we wanted to build it. What's interesting in this domain is that, in a sense, if you take something simple like transcription, it doesn't seem like something that should be very hard for a model to do. It's essentially pattern recognition; it's classification. These models are very good at classifying, right?

Nonetheless, when you talk to them, it's not there yet. You don't talk to them the same way you talk to a person or something. Maybe people don't realize it. In English, it's still much better than in any other language. Even compared to French, for instance, if you talk to this model in French, when you see people talking to this model, they will talk very slowly. They will articulate as much as they can. So, it's not natural, right? We're not yet there.

Maybe the next generation won't know this, but people who are maybe our age will always keep this bias of speaking very slowly when they talk to this model, even if probably in a couple of years—maybe next year—it won't be necessary anymore. What's interesting is to see that even for languages like French, Spanish, and German, which are not low-resource languages, you have a lot of audio with this.

They still aren't as good. I think the reason for this, I suppose, is that there hasn't been as much energy or effort put into some other modalities, such as vision or coding. There is still a lot of progress to be made, but I think it's just a question of doing some work on this. There is a clear path, I think, to get there.

Alessio Fanelli

It's really fascinating because I worked on Google Assistant a while back at this point, but when you take a step back, it's fascinating. It's not that long ago—it was 4 or 5 years ago—and now it's completely audio-in, audio-out, and the function calling and the whole thing happen completely end-to-end in a very natural way. There are still ways to go, like you were saying. Even despite all the progress, it's not like you're speaking to a person when you talk to any of these agent bots or voice-mode kind of situations. There's still a gap. I think that's the great thing, and I feel like, even with the existing stack, we should be able to get to these very natural speech conversational abilities soon enough, I guess, and we'll also hope to get there.

On the customer side, when you talk to these agents, usually people are just writing to them. Sometimes it will be very clear—for instance, you want to write code, but you have a very clear idea of how you want the model to implement what you had in mind. So here you have to spend a lot of time writing. It's not really efficient. Audio is really a natural interface that's just not there yet, but I think it's just going to be there, at least.

Shawn Wang

How is it like building, serving, and inferencing? We see a lot about how it's very easy to take LLMs off the shelf, serve them, fine-tune them, and deploy them. I know you guys have a whole—you have Forge—you have a whole stack for customizing and deploying. Is there a lag in getting that distribution channel? Are you helping there? With prompting LLMs, you can have them be concise, verbose, and all that. They're built on LLM backbones, these models. So how do you see all that?

Guillaume Lample

Yeah, I think this is a lot of what we're doing with our own customers. Very often, they come to us for different reasons. One reason is that they sometimes have a lot of privacy concerns: they have data that is very sensitive, they don't want it to leave the company, and they want it to stay inside the company. So we help them deploy the model in-house, either on-premises or on a private cloud, so they aren't worried that it's being given to a third party or that there is some leakage.

Many companies have different levels of data sensitivity. They have, sometimes, Tier 1, Tier 2, and Tier 3 data. Tier 3 can be sent to the cloud; Tier 1 has to stay there. So it creates heterogeneous workflows where it's annoying: you cannot send some data to the cloud, but you can send other data. When we deploy the model for them, they don't have these considerations; they're not worried that this is going to leak. Everything is much easier, so we help them do this.

That's one of the value propositions, but the other is that, very often, when customers use these off-the-shelf closed models, what's very sad is that they're not leveraging the data they've been collecting for 4 years or for decades. It's so much data—sometimes trillions of tokens—in a very specific domain, their domain, which is data that you won't find on the public internet. It's data that a closed-source model would not actually have access to, and on which the model is going to be really good.

If they're using closed-source models, they're basically not benefiting from all these insights and all this data they've collected over the years. They can always put it into context at inference time, but it's still not as good as if you actually train the model on it. So that's basically what we help them do.

We actually provide them with Mistral Forge, basically what we announced at GTC this week. It's a platform with a lot of tools to help them process data and train on it. It's actually the same thing we're using in the science team, so it's battle-tested infrastructure, with a lot of efficient training code for continued pretraining, fine-tuning, and even doing SFT well. We help them do this using the same tools that our science team is building and using.

Since these are tools that we've been using for 2 years now, they're really battle-tested and sophisticated. It's the same thing we're giving them—giving companies the same thing our science team uses internally to build their own AI. It makes a really big difference.

I think sometimes customers, and many people in general, don't realize how much better the model becomes when you fine-tune it on your own data. You can have your model here, and you start from there. You have a closed-source model that is starting here, but if you actually fine-tune it, you really go much further than this. Then you have a very big advantage. The model is trained on your entire company knowledge, so it knows everything. You don't have to feed 10K tokens of context at every query. So it's much easier.

I think using a closed-source model is really sad because it basically means you are not leveraging all this data, and you are going to be using the same model as all your competitors, when you're actually using everything you've been collecting for years, which is really valuable. So we help customers do this.

We have a lot of solutions deployed, and we have engineers who go into the company and basically look at the problem customers are facing, what they're struggling to do, and what we should do to solve it. So we help them solve these problems together. I think our approach is a bit different here from that of some of our competitors. We don't just release an endpoint and put some stuff on top of that, and we don't just give them a checkpoint. We really look very closely with customers, look at the issues they have, help them solve them, and make a tailored solution for the problem they're facing.

One example is that sometimes customers really want a model that performs well on some Asian languages. If you take some of the Mistral models, they can't speak or write in this language, or at least it's not amazing. This language might be 0.1% of the mixture, so it has been included during training, but very, very little. What we did here is actually train a new model for them in which this language was 50% of the mix, which is much stronger. It knows all the dialects and the slang. That's an example of things we can do, and it's really custom, I think.

Some customers, for instance, wanted a 3B model that could do audio and was very good at function calling—something you want in the kitchen or in the car. In particular, they wanted this to be offline, because in a car you don't necessarily have access to the internet. We can build these solutions. There is no model out of the box for this on the internet. You have very general models that are good at reasoning and are strong models, but for things like this, customers always want specific solutions.

Sometimes they come to us because they experiment with a closed-source model, get a prototype, are happy with what they built—it works well, and they're happy with the performance—and then want to go to production. Then they realize it's extremely expensive. You cannot ship this. So they come back to us and say, "Can you help us build the same thing as this, but using something much cheaper?" We can sometimes build something 10x cheaper just by fine-tuning a model on it. It will be better on-premises, on their own server, and much cheaper as well.

swyx

That's the missed opportunity right there.

Alessio Fanelli

Take all the money.

swyx

Outside of that, you do put out open-weight models so people can do this themselves. It's just that not enough people go out of their way to do it. They're not going to; they're going to ask you to do it.

I see how it works.

Guillaume Lample

Initially, we didn't know—I mean, we weren't competitive at the beginning of the company because I think our strategy was not exactly the same as what it is today. What we underestimated initially was the complexity of deploying these models, connecting them to everything, and making sure they have access to the company's knowledge. We were seeing customers struggling with this, but that was 2 years ago, and now things are much more complicated.

Now you don't just have text-only LLMs doing simple instruction following; you have reasoning, agents, tools, and multimodal audio. It's much more complicated than before, and even back then it was hard for customers. So they really need support, and this is why we're providing that as part of the proposition as well, to help them deploy this.

swyx

I'm curious: is there also voice fine-tuning that people do?

Just as with Forge, we also have a unified framework. The hope is that, like with the Voxtral speech-to-text model that we released earlier this year, and even with Vox chat, which we released last year, there will be a big, rich ecosystem of people fine-tuning Whisper. People want the same thing with Voxtral.

It's much stronger than Whisper, and the platform offers that kind of fine-tuning, which could be any kind of fine-tuning. For instance, sometimes people want to support new languages—long-tail languages—which we hope to cover ourselves natively. But if there is a language where you have data and you want to fine-tune it, I think this is a good use case. The other use case is the same language, even English, but in a very domain-specific way—

Alessio Fanelli

Yeah, terminology, jargon, medical stuff.

Guillaume Lample

Exactly. And also the specific acoustic conditions, like if there's a lot of noise. The model will do decently in most conditions, but you can always make it better, and those are some of the use cases where you can improve it even further. That's one good use case for this.

For our text-to-speech, we're just releasing it, so we'll have support for that soon, too. I think it's a similar use case, but it's a little different—the kinds of things that you want to extend a text-to-speech model to could be voice personalization and voice adaptation for enterprises. Many enterprises need a very specific kind of tone and personality for this kind of voice, and all of those are good use cases for fine-tuning.

swyx

This is what I was going to ask you. We never talked about cloning—voice cloning—here. How important is it? I can just clone a famous person's voice, okay, but—

The main use case would be enterprise personalization. Enterprises need a lot of customization. You don't want the same voice for all enterprises. Each enterprise wants something customized and specialized, which is representative of both their brand and their safety considerations.

The kind of thing that you would deploy as an empathetic assistant in the context of a healthcare domain would be very different from the kind of thing that would be in a customer support bot, and that would be different from more conversational applications. Those are the customizations you would expect from enterprises, and that's the main use case, at least from our side.

Alessio Fanelli

My base example is that you don't want to call 2 customer services and have the exact same voice. It's going to be weird. But also, on the technical side, there are a few things about Voxtral that I thought were pretty interesting. He's a big fan of this paper.

Guillaume Lample

Oh.

swyx

He said during the paper that it's the best ASR paper he's ever read.

Alessio Fanelli

Yeah, I've hyped up this Voxtral paper enough. We covered it somewhere.

swyx

Whisper is known for 30-second processing. You extended this to 40 minutes. There was a lot of good detail in the paper about how this was done, even little details about the padding. It is very much needed; you need to have that padding in there. There was also the synthetic-data generation around this.

I'm wondering if you can share the same about the new text-to-speech model. How do you generate long-form, coherent speech? How do you do that, and are there any gems? Is there going to be a paper?

Guillaume Lample

Yeah, there would be a technical report. I think it could have a lot of details. The summary of it is that some of the considerations in the paper came from the fact that we started with the Whisper encoder as the starting point.

Now we have in-house encoders, like the real-time model that we released in January. We also released a technical report for that real-time model, which uses this dual-stream architecture. It's an interesting architecture; you should check it out. We have a causal encoder, and I don't think there's any strong multilingual causal encoder out in the community, so we thought it would be a good contribution. It's a good encoder that other people might want to adapt, and we trained it from scratch.

I think our full stack is now mature enough that we're able to train very strong encoders. Some of these considerations, like striding and related details, are a function of the Whisper encoder. Now that we train encoders in-house, the design considerations are different.

As for our text-to-speech model, it also leans on the original autoregressive decoder backbone. The considerations are almost identical. The model processes audio at 12.5 hertz, so 1 second maps to 12.5 tokens. One minute is about 720 tokens, so you can get up to 10 minutes in an 8K context window, and half an hour in a 32K context window.

A 32K context is something that we're very comfortable training on. We can extend it even much longer—to 128K—so you can naturally see how it can extend to hour-long generations. We need the data recipe and the whole algorithm to work coherently through such long contexts, but the techniques are in some ways very similar to text long-context modeling. The key difference is that it's doing flow matching autoregressively instead of text-token prediction.

Alessio Fanelli

Okay, I think that was most of the voice questions that we had.

Shawn Wang

I have a big question on Mistral Small.

Alessio Fanelli

Mistral Small. Let's go.

Shawn Wang

What is Small? How do we define Small? What is this?

Alessio Fanelli

I remember the days of Mistral 7B on my laptop. It's not fitting on my laptop.

Guillaume Lample

I could run it on the big laptop, but it's just a different question. If I tell you the equivalent of this model is 6B active parameters, which is true, we did let you infer that in the same name. We could have called it Medium, but I think it's just a naming question. I'll trouble you with this, I suppose.

It's a model that we released, Mistral Small. It's a mixture-of-experts model that combines different models. Before, what we were doing was having one general model for Mistral doing instruction following, with a separate model, Devstral, that was really good at coding and specific to code. We had another model for reasoning, Magistral. These were separate artifacts built by different teams at Mistral.

Now what we're doing is basically merging all of this. Pixtral, the first vision model we had, was also a separate model. The way we do things internally is that we have one team focus on one capability and build one model. Once it's mature enough, we decide to merge it into the main picture. This was the first time we basically merged all of this into one.

There are some other things we didn't have time to merge at the time, such as more capabilities and function calling, which I think is going to be much, much better in Mistral Small proper. But, yeah, it's our latest model and the one we're working on. Of course, there are larger versions of this.

Alessio Fanelli

The key things are that it's very sparse, with 6B active parameters, so it's pretty efficient to serve, and it has a 256K context window.

Shawn Wang

What's interesting is this general theory of developing the individual capabilities in different teams and then merging them. Where is this going to end up?

Like, we've seen the 5 things put together in this. What are the next 5 teams?

Guillaume Lample

I think, actually, OpenAI has kind of gone away from the original vision of the omni model. This is what they were selling, right? All modalities in. But I feel like you might do it.

Shawn Wang

I mean, I think there are some modalities where it's not completely obvious. For instance, for audio, if you want to do transcription, I think it makes no sense to use a model that's this large. If you just want to transcribe text, it would be very inefficient, actually. If you want to do audio, you probably just want to build a 1B or 3B model. Performance would be essentially the same, and it's going to be incredibly cheaper.

So that's why we want a separate model that just does this. The question is just: if you are talking to your model by speech and asking very complex questions, how do you do this? On the other hand, you want to cascade things. Do you want to put a 1B model on the stack, not a competitor to this? I think, in a way, if you do that, that's the right possibility, of course.

But, yeah, for us, the next capabilities we want to try to integrate into these models are going to be more coding and more reasoning. I think there are more capabilities that people don't talk too much about but that are important for our customers in different industries—for instance, legal, finance, and computer-aided design.

These are things that are hard to put out of the box because people really don't prioritize them. There are no new benchmarks for that, but it's not hard to make these models better. Coding is just a matter of doing the work, like sourcing some data and processing it. If you can do that, it makes sense. We have other things to merge into this.

For voice, the key thing over maybe the last year or so with Veo and Google Imagen and all these things is joining voice with video, right? People don't understand spatial audio because most text-to-speech is just, “I'm speaking to a microphone in perfect studio quality.” But when you have video, the voice moves around.

Guillaume Lample

That’s true. The consideration is also a little different in the sense that there, it’s a standalone artifact where you get the whole thing and consume it. But in a conversational setting, you need extremely low latency.

Shawn Wang

Yeah.

Guillaume Lample

Streaming would be one of the primary considerations.

Shawn Wang

You can build a giant company just doing that, so you don’t need to do the voice stuff. But I was just saying, on the theme of merging modalities, that is something where I’m like, wow. Everyone up until, let’s say, mid-last year was just doing these pipelines of, “Okay, we’ll stitch a TTS model with a voice thing and a lip-sync thing and what have you?” No.

Guillaume Lample

Just one giant model.

Shawn Wang

Yeah. I have a two-part question. One is, it’s still open. It seems like open source is still very core to what you guys do, and I just have to plug your paper.

January 2024—when you rolled out Mixtral of Experts, it was very fundamental research on how to do good MoEs. The paper came out, and it’s a very good paper for anyone, but that’s just a side tangent.

I mean, Mixtral 8×22B was like the nuclear bomb for open source.

Guillaume Lample

I think it beats 70B models.

Shawn Wang

So, 70B or more?

Guillaume Lample

Yeah, yeah, yeah.

Shawn Wang

But this is a big upgrade for 7B.

Guillaume Lample

Yeah, yeah. I don’t remember this. I don’t think it was January, right? It was NeurIPS.

Shawn Wang

It dropped during NeurIPS, and then everyone at NeurIPS was there. It was December—

Guillaume Lample

2023, but I think, yeah, the model was updated as well. I think it was a bit earlier.

Shawn Wang

Yeah, it’s just a little update, probably.

Guillaume Lample

Yeah, I know, but you have a point to make.

Shawn Wang

No, I mean, you have to check that. But I just want to hear more broadly about open source for you guys. When you were talking earlier about what’s next and what the other side teams are working on, you put out Leanstral.

This one was a surprise. I was like, I don’t—this doesn’t fit my mental model of Mistral.

Guillaume Lample

Yeah, I mean, first, for open source in general, I think it really relates to the journey of the company. We’ve had an open-source thing with us since the beginning, and even before this. When I was at Meta, we released Llama.

What was really nice to see was that, before this, for most researchers, like universities, it wasn’t possible to work on LLMs. There were no LLMs available outside. If you look at many of the techniques that were developed after Llama was open-sourced, like all these post-training approaches, including DPO-type preference optimization, all of these were done by people who had access to the model. It would have been impossible to do without the model.

So, it really helps us move faster. We want to contribute to this open-source ecosystem. I think it’s a deep signal, and it has a lot of impact. All these papers in the open-source community are really helping the science community as a whole move faster.

That’s why we are releasing very detailed technical reports on Leanstral and our first reasoning model. I’m publishing a lot of research—things that worked, things that didn’t work as well, and so on. I think it’s helpful. For the audio model, we also want to share a lot of details, including for the real-time model.

We really want to continue this. We belong to this community of people who share science. We really don’t want to be living in a world where the smartest and best models are only behind closed doors, accessible only to some company that has the power to decide who can use them. I think that’s a scary future we don’t want to live in.

We really want these models to be accessible to anyone. We want intelligence to be used and accessible by anyone who can use it. That’s why we’re pushing for this mission. Leanstral is open source—not the first model, and therefore not the best—but it’s one step in this direction.

It’s a bit different from what we usually release, but we have a small team internally working on formal proving and formal math. I think that’s a subject we care about in general. We’re working on reasoning. I think we started too early, before LLMs. Doing reasoning without LLMs is very hard, especially when you work with formal systems, because the amount of data you have is negligible. It’s a very small community of people writing formal proofs.

Arthur Mensch

The reason we like it is that, if you look at what people are doing with reasoning, the problems you can use are usually problems where you can verify the output. For instance, all these AIME problems where the solution is a number between 1 and 1,000—you can compare it with a reference. Or if it’s an expression, you can compare the output expression generated by your model with a reference.

But for many—most of—the math problems and most of the reasoning problems, there’s no easy way to verify the solution. If the question is, “Show that f is continuous,” you cannot compare it with a reference, right? If it’s “Prove that this is true” or “Prove these properties,” there’s no way to easily verify the correctness of your proof.

So, it’s hard to apply reinforcement learning here because there’s no verifiable reward. You could provide a judge that looks at your proof, but it’s very hard, and you could have some reward hacking happening there. You could provide a reference proof, but there are many ways to prove the same thing. If the model gets a negative reward because it’s a different proof, maybe it’s still a legitimate proof—just different. So, it’s not going to work well.

What’s nice with Lean and formal proving is that you don’t have to worry about this whatsoever. They’re all functionally equivalent as long as they compile in Lean, right?

Shawn Wang

Exactly. It’s like a program: if it compiles and it’s correct—

Arthur Mensch

Yeah, it’s very easy, and you can apply this to any kind of—

Shawn Wang

It’s just way too small. No human will actually go and do it.

Arthur Mensch

Yeah, that’s kind of exactly it. The only people who can do it are a very small community of people doing a PhD on it. So, it’s super small, and it’s kind of sad because it’s actually very useful—not just for math, but also for software verification.

Software verification today is a really tiny market. Very few industries work on this and need it. It’s usually companies building airplanes, robotics, or things where they absolutely want to be sure because lives depend on it. But it’s very rare that people formally verify the correctness of their software.

One reason for this is simply that it’s super hard to do. Are you thinking of TLA+?

Shawn Wang

It’s the language that some people use for software verification.

Arthur Mensch

No, I know. It’s what people use in some contexts, but Coq—but, yeah, I think the reason people don’t use it more, and why this industry isn’t as big as it could be, is because it’s very hard.

But now, with coding agents, it’s going to be very different. We’re going to see much more of this. I think this industry is going to be much larger in the future with these formal methods. So, we’re anticipating this a little bit and wanted to work on it, because it’s proving a theorem and putting a function in it. It’s actually the same tool.

Shawn Wang

One of my theories is that, because the proofs take so long, it’s actually just a proxy for long-horizon reasoning, coherence, and planning, maybe.

A lot of people will say, “Okay, it’s for people who like math. It’s a niche math language. Who cares?” But you use this as part of your data mix, I’m sure, for post-training and reasoning. It might actually improve everything else. I think that’s underexplored, or no one has really put out a definitive paper on how this generalizes.

Arthur Mensch

Absolutely. I think that’s kind of what we’re seeing already. For instance, if you do some reasoning on math and then ask the model to do it in Lean or Coq, we’re still in the early stages, but it definitely shows some transfer—some sort of emergence that happens.

Some of what’s also interesting is that, in the long run, there’s a lot of connection with coding agents. Sometimes a model can see a theorem that it has to prove, and it’s very complex. Then it can take the initiative and say, “I’m going to prove these 3 lemmas. I’m going to suggest 3 lemmas, and I’m going to prove each lemma in parallel—3 of them in parallel with sub-agents—but I’m also going to prove the main theorem, assuming the 3 lemmas are done.”

This sub-agent approach is pretty interesting. Even if you fail to prove one of the lemmas, you can maybe use it to prove another lemma. So, you get some partial reward here. It’s a bit less sparse than if you just get a zero reward for the entire thing. I think we can actually stop here.

Alessio Fanelli

Yeah, it's also an interesting case just for specialized models in general, right? The cost thing you show is pretty interesting. Similar score-wise, you're looking at $30, $70, $150, and $300.

Arthur Mensch

Compared to something like that.

Alessio Fanelli

I think cost is a bit unfair, right? Because this one is at inference cost, and this one is there with their margins on top of it. But we don't know anything else.

Shawn Wang

I did want to push on that more. Not on cost, but you mentioned that it's a great way to have verifiable long-context reasoning. What are other frontiers that you guys are working on internally? There's a lot of people pushing back on pre-training and scaling RL, pushing compute toward having more than half of your training budget all on RL. Where are you guys seeing the frontier of research in that?

Arthur Mensch

You mean with RL?

Shawn Wang

Just in foundation-model training in the next few years. One thing that you guys do is fundamental research from the ground up, right? So you probably have a really good look at where you can forecast this out.

Arthur Mensch

Yeah, but I think for us, we are still working a lot on the pre-training side, and we are very, very far from any sort of saturation in pre-training. I think any sort of improvement in pre-training would be a big step up compared to everything we have done before. So, we are pretty excited about this.

On the other side, I think now we have more and more to think about algorithms that will actually support these very long trajectories. I think when it was, for instance, GRPO, it didn't really work with this kind of policy, which was okay initially because you were solving math problems that could be solved in a few thousand tokens. So, the model could generate them pretty quickly, and when you were doing your updates, the model was never too far off—still not too far off.

But now, when you are moving toward these kinds of problems where something takes hours, like 6 hours to get a reward, then your model is completely off-policy, you see. So, you actually have to bring in new infrastructure that supports this because of your algorithm. Everything we're doing internally now, we're trying to build some infrastructure that will anticipate what we have in, like, a 6-month window, which is these extremely long scenarios, at a pace that they can handle.

When we started Mistral, part of me, and maybe also Timothée, wanted to recreate this very nice environment where people are there so they can do the research they like, with a lot of resources. So, it was nice. I think things changed a lot when ChatGPT came out. After that, it was very difficult. This time is also the same again, but it was nice, and we also want to recreate part of this culture we had before.

Alessio Fanelli

Coming to the end of it, I think you guys are doing incredible work. You have a very impressive vision for open source and for voice. What are you hiring for? What's next? What are you looking for in people trying to join the company?

Arthur Mensch

Yeah, so we are hiring a lot of people on our science team. We are hiring in all our offices. Our headquarters are in France, in Paris. We have a small team in London and a team in Palo Alto as well. Recently, we opened offices in Warsaw, in Poland, and one in Zurich. We also have some presence in New York, and soon one in San Francisco. So, we are a bit everywhere, and we are also hiring people remotely.

We are growing the team and trying to hire very strong people. I think we want to stay fairly small. We find it quite efficient to have a small, very agile team.

Shawn Wang

Okay, let's focus on science for a bit. We actually are strong believers in science. We started a new science team that focuses specifically on AI for science. What areas do you think are the most promising?

Arthur Mensch

What we are most excited about right now, and something we have started doing and will probably be able to share more about in a couple of months, is exploring AI for science. There are a lot of areas where we think you could get extremely promising results if you apply AI in these domains. There are a lot of low-hanging fruits. You just have to find these domains where AI has not yet been applied.

It's usually hard to do because the people working in these domains don't necessarily know the capabilities of these models. They don't know how well AI would work. You have to pair them with exactly the right researchers, matching them with exactly what to do. We are kind of doing this matching naturally with our customers.

We have some companies we work very closely with. For instance, ISM Électronique is one of our partners. We are doing some research with them, and they have tons of extremely interesting problems—problems in physics, science, and material science that they are essentially the only ones working on because they are doing something no one else is doing.

There are many domains where AI can actually revolutionize things. You just have to think about it and be familiar with what it can do well and how to apply it. So, it's something we are exploring with our partners and customers. AI for science is very interesting.

Alessio Fanelli

Yeah. And then for forward-deployed engineers, what makes a good one? What do they need? Where do people fail?

Arthur Mensch

I think you usually need people who are very familiar with the technology—not necessarily with a lot of research expertise, but who are actually pretty good at using these models. They know how to do fine-tuning, and they know how to start an RL pipeline. It's not easy. It's something that the majority of customers and companies would not be able to do on their own.

Michele Catasta

So, we need people who like to solve problems and are excited about complex, very concrete problems. It's applied science, basically. I don't think it's too different from the skills you need in research, because essentially you are trying to find solutions to problems that customers have not yet solved.

Sometimes it's easy, and sometimes you have to do the work. You have to create synthetic data and find edge cases. It depends on the problem, but you need a bit of patience as well. You have to be creative. I think the skill sets are actually very similar.

Shawn Wang

The diversity of the work they do always surprises me. It goes all the way from the kinds of things they encounter in industries. It's just very interesting, I think.

Alessio Fanelli

Any fun success anecdotes?

Michele Catasta

I mean, yeah, it can be training this small model on the edge that just does one specific thing. We are also training some very large models with some specific languages. We are making models really good at using tools, like computer-aided design, for instance—these kinds of things.

Shawn Wang

Is that paired with vision as well?

Michele Catasta

Yeah.

Shawn Wang

Like defect detection for chips or in factories, identifying things. The diversity could be anything where you can deploy these foundation models. The work to make it work in that specific setting is basically whatever it takes to make it add value in that specific workflow.

Yeah, and it goes kind of across the stack, right? Even just pulling up the website, you have to choose—

Michele Catasta

It's so broad. Compute is so broad.

Shawn Wang

We didn't even touch on Mistral 5, the live-coding CLI tool. One thing you guys were actually, I think, the first to—

Michele Catasta

Mistral Agents.

Shawn Wang

Mistral Agents. Yeah, the agent builder—you can serve it via API and all that. I'm guessing forward-deployed people will—

Michele Catasta

Yeah.

Shawn Wang

—help build that out and stuff.

Michele Catasta

It's also why we are doing many things. I think that's part of the value proposition. Customers are always extremely careful about their data, and they don't want to trust so many partners—trusting one partner for code, giving their data to another third party for audio, and so on.

They like this. What they really like about our approach is that we can help them with anything, so they don't have to send all that out to so many clouds.

Alessio Fanelli

I think there can be many orders of magnitude more forward-deployed engineers than research scientists. They don't need your full experience, but they're still super valuable to customers.

Michele Catasta

In practice, these 2 teams are still quite intertwined. First of all, they are using the same tools, the same data pipeline, and everything. It's very helpful for the same team to get feedback from the solutions team, because they can say, "Look, these customers are trying to do this. This is not working. Can we maybe solve this in the next version?"

Shawn Wang

Yeah, so this is basically your real-world eval.

Michele Catasta

Yeah. It's real-world eval. It's not something where, for instance, if you're just working in the lab, you simply ship a model. If you don't do this work of examining it with customers, you have no idea whether your model is good at an edge case.

Even in your work before this, right, there is a very big gap between the public benchmarks, which are very academic, and the real cases.

Shawn Wang

The real cases are just very diverse.

Michelle Pokrass

In the specific context of a customer, you can fine-tune and make it better. First, you evaluate, create a solid eval benchmark, and then measure it in the context of their kind of audio. For instance, one use case is literally just the word for kids, and they have to say it out. It's a very specific thing: you're just saying one word, and then you have to grade the kid on whether they did it right. It's like RL for kids.

There are very diverse use cases and ideas that the applied scientists and engineers will use to make it better. Then, from those learnings, we incorporate them into the base model itself, so it's just better out of the box.

Shawn Wang

Yeah. It's a good full-circle system. The foundation-model evals are all just proxies for what you really care about. It doesn't make sense for there to be a one-word transcription like that. It's not something you want to fit on.

Perfect. Well, everyone should go check out everything Michelle has to offer and try the TTS model, which we'll link in the show notes. Thank you so much for coming.

Michelle Pokrass

Thanks. It's such a pleasure to be here, guys.

Mistral: Voxtral TTS, Forge, Leanstral, & Mistral 4 — w/ Pavan Kumar Reddy & Guillaume Lample | BidClub