语音 AI 的世界：与 ElevenLabs 的 Mati Staniszewski 对谈

Mati Staniszewski co-founded ElevenLabs in 2022 and has since scaled it to the $11 billion leader in AI audio. He's credited with capturing the humanness of speech through realistic emotional inflection, and they're now expanding into everything from agentic workflows to music.

Cheers. Thanks for having me.

Maybe a good place to start is: describe to me how—I know how an LLM works at a high level. Describe to me how an audio model works. If we were looking, Karpathy-style, to build a toy one from scratch, how does it work?

1. How Audio Models Work

In the early days, you would try to replicate it exactly as you would replicate it with the human body. You would completely try to reproduce an analog machine that would create a vocal tract, effectively. Then that progressed into trying to create a digital signal for speech. Bell Labs was one of the first to try to create a structured set of signals that would represent speech. That is a first precursor to what we would do today.

Then you would try to stitch in phonemes—effectively, different sounds of how we would speak as humans—and then try to concatenate them together. It's another important part of that equation. Based on the most probable next word, you would try to bring the phonemes from your library together.

Then you get to modern history, where we now do similar neural nets in other domains. You predict the next sound based on the context of the previous sounds if it's streaming speech. If it's, let's say, a context of audio, you use a combination of predicting the phonemes, but you also use the contextual text element of that work.

Here, credit to my co-founder, Piotr, who effectively came up with that new idea of how you can now create voice models that are reliable, high-quality, and quick. You would bring a lot of the ideas from transformer models and diffusion models into the speech space. That prediction of the next token in the phoneme space wasn't something that was possible. We spoke briefly about how you can operate in the text or waveform space. There's also mel-spectrogram space. Usually, you do text, mel-spectrogram, waveform.

Sorry, what's the spectrogram space?

It's a visual representation of how speech sounds across pitch and energy, and then you transform that into a waveform. When WaveNet and Tacotron models came along, they would effectively use text to mel-spectrogram—that visual representation—and then decode and encode that into the waveform.

Piotr figured out how to abstract some of those steps and decode and encode them a lot better. That prediction of the next phoneme was one of the big pieces. The second big piece was: how do you bring that context into the equation? What I mean by context is, if a voice actor was reading a textual copy, you would know, okay, this is a dialect sequence. I need to produce a dialect. If it's a happy sentence, I might need to pronounce it as a happy sentence. But what happens before and after comes into the equation, and you need to bring that across.

Then there's a last big piece. A voice model has the sound of how you intonate a given fragment. But the second big part is the voice itself: the characteristics of accent, style, and prosody across that voice. When you actually try to vocalize something, when you create that voice model and turn text into audio, you need the text. You also need the voice reference for how you want it to be spoken.

Here was the second big innovation. Apart from context, it's how you decode and encode those features. When Bell Labs came up with their initial representation of speech, the big piece there was that you would have hard-coded parameters for that speech. With ElevenLabs models—

Hard-coded parameters for an enthusiastic speaker, British accent, that kind of stuff?

Exactly, that kind of stuff. The set of pitch elements that you can select, the set of energies, and spectrograms you can select from. In our approach, you would give the model open-ended ability to select what those parameters should be. It's not going to be a British, Polish, Spanish, or English speaker, but the model will deduce them itself. The same goes for other sets of parameters that are not hard-coded, whether it's enthusiasm, sadness, et cetera.

You're saying Britishness is an emergent property in your voice models?

Exactly. Those two big parts are the encoding and decoding of how you create the voice. It was a super-hard problem before, and we figured it out, too. How you then construct that in a sentence, how you get the context across so you can predict the next phonemes, and how you bring them together in a reliable and stable way while doing it quickly—these were the two first big innovations in the voice models that continue today.

But if LLMs reason about text and word-subpart tokens as the way they think about the world, what is the equivalent of a token in a voice model? You mentioned phonemes a bunch. What is that representation?

We start with the voice embedding for the speaker. You need that reference when you produce and create the speech. Of course, in the input to the voice model, you still get the text, and you bring in the speaker encoding. Then, when you produce speech, you operate on the waveform, or effectively at the phoneme level of that speech.

Sorry, what is a phoneme? Fill in my understanding.

It's like a syllable deconstructed into even smaller elements. These are effectively the human sounds you can produce. These would be the closest to that representation. But of course, in our models, it's now going to be a combination of not only operating at the phoneme level; you also operate at the text level. You operate on both in sync because when you are predicting the context, you need to understand how that sentence will get constructed. Especially if it's more of a streaming, real-time use case in a voice-agent setting, you need both parts to work together. It's similar to how you operate at the token level on the text side; we operate at the token level on the audio side.

It feels like a big part of the magic of Eleven was that your voices were much more human-sounding. How did you accomplish that?

2. Making Voices Sound Human

I'll give you a quick synopsis of how we think about the models on the text-to-speech side today. In any model, you need the architecture, you need compute, and you need data. Architecture innovations were one thing. The data part was the second big thing.

With audio, you will have a lot of audio data available, but frequently you will not have it annotated in the right way. You won't have who is speaking when. Some of the “what” is annotated, but the “how” isn't. As we are speaking now, what are the emotions that we're using? What accents are we using? We would invest a lot internally in creating our own data labelers, our own team, to be able to create better data sets. That was a combination of semi-automatic techniques and manual techniques.

Actually, a lot of the models that we did afterwards spun out from a lot of that research, too. The speech-to-text model was initially a model we did for ourselves because the models on the market just weren't good enough to annotate that data. Then another brilliant research area on our team was being able to construct it so we could spin it out as a model that we brought to the customer.

You've just been doing useful stuff in voice, and that has emerged with a whole bunch of products that you might not have expected because you find you're building useful stuff.

Exactly. That's a combination of data and being able to do it automatically. Creating a team that's coached on voice and on how to describe it helped us a lot, because most of the labelers out there just aren't as well-versed in understanding audio and voice. Then, of course, deploying those models in production, seeing how customers interact with them, and having them annotate all of the data helped us refine those models over time.

A very interesting thing on the side: we spoke about the speech representation. The first guy who created the speech synthesizer is a guy called Wolfgang von Kempelen. He created that analog machine that would represent, effectively, a human vocal tract and try to produce that sound. He had to spend decades on that, and that started producing vowels. But that was the same person who created a chess machine, the first viral, let's say, chess machine that would simulate playing chess.

Is this the Mechanical Turk?

It was called the Turk. But the crazy thing behind it was that it was operated by a human, and it was all a fluke. That's where the Mechanical Turk came from, which we actually use in our data-labeling production to make that work.

Sorry, we jumped right in. But if you describe the Eleven business today, people think of you as the speech company. How should they actually think of your business, to the extent you can describe the big areas? Text-to-speech, speech-to-text, voice agents.

Just break down the business for us.

3. Deploying Voice at Scale

In a nutshell, I'll describe ElevenLabs as a research and product deployment company. We build foundational audio and voice models, and then build a platform for businesses to transform how they communicate with their customers and employees. That will apply through AI agents, from customer support, sales, hiring, and training all the way through to marketing and storytelling for our creative tools.

In that set, we've created all types of foundational audio models. Text-to-speech models for producing speech, speech-to-text models that work over 100 languages and happily beat others on benchmarks, all the way through to conversational models, how you loop them together with music, and other domains of audio. Then, of course, beyond the models, when you actually bring them to production, that's where the second level of the platform comes in, where that meets the businesses on the specific use case.

On the agent-specific example, it would be how you now connect those models to the knowledge base, to telephony, and to the integrations that you need to perform the actions; how you evaluate and monitor the agent so that it behaves in the right way; and how you build the right safeguards.

On the creative side, on the marketing side, it's how you create a good ad, so you can create a good video voiceover for one of the campaigns. How you create an article that's narrated with a specific voice that represents the brand in a good way. That's where we combine the models and our understanding of the customers we work with into one unified platform.

Every platform company has this question about how far they go into applications. How do you think about where you go horizontal and power the whole ecosystem versus where you develop applications? Because you can imagine there being a whole ecosystem of closed-captioning tools that grow up and are built on the ElevenLabs technology. It's not necessarily a space that you would have to go after yourself.

I think the big difference is that today, we see ourselves as a platform where, if you're building a horizontal use case in your business, we're a great place to come. If you have a lot of domain specificity, that's where I see a lot of the application companies forming over time. That's specifically not the space we will go into.

I think it also is interesting when the technology is moving as quickly as it is here. It's one thing with SaaS where you get these vertical-specific providers, but I would imagine one of the biggest risks for you guys in being intermediated is if there is, in this example, a closed-captioning service that is on a version of ElevenLabs that's two versions old and hasn't upgraded. That's a problem because you want people to be using the latest and greatest model that you've developed, and you'll be deploying new capabilities every week. I presume that's part of your thinking: when it's moving that quickly, you need to go direct in a lot of cases.

That's right. In closed captioning, we already know that our service is going to be able to tackle 99.9% of the cases that customers have. Then there's the added benefit that we work with healthcare customers, where we will create a custom model for those customers and get that transcription perfectly.

The context is the tricky thing in closed captions, because we talk about a lot of technical stuff on this.

Yeah, for sure. That's where you effectively need a dictionary of the technical terms beforehand, which, as we work with businesses, we know we need to embed in that creation process.

We're talking a bit about product here, and one thing I note is that LLMs are amazing. You have the usage stats of ChatGPT, Gemini, and all the popular LLMs, where they're working and people use them a ton. It feels like there's a big product overhang when it comes to voice, where the leading-edge voice models are incredibly capable.

Yet I was driving home the other day, and I needed to read a PDF, but I was driving. I said, "Okay, I'll just have my phone read the PDF to me." You can try to hack it with the iOS screen reader, but it doesn't really work with the scrolling. Then, in theory, you can upload it to Gemini, but you're trying to get it not to summarize it, and it just hung when I tried to press the "Read this to me" button. There was no way I could get my phone to read me something, which seemed like a fairly basic feature. All cars advertise voice control, and yet it sucks.

Separately, if you want to input something to the navigation, no car has a good version of that yet. Maybe Tesla does. Why does it seem like, with LLMs and Claude Code and everything, we are using all the capabilities of the intelligence, whereas with voice, we're living 10 years ago somehow?

I'm thinking about whether I agree with the premise that we are 10 years behind.

In the lived experience of people day to day, they're using Siri's transcription, which has gotten better, and it's still way behind the leading edge.

There's definitely a piece of it where I think the technology in many of those cases already exists. There's a deployment gap in what you're saying. It's like automotive: some of the big companies are not adopting that quickly enough or bringing it into production. But there are plenty of different problems that you need to fix along the way. The quality of voice models for them to actually sound good—this has only been a thing for the last 3 years.

Yeah, but it's 3 years.

It's a 3-year-old thing.

Cars have over-the-air software updates now.

That's 3 years for the first voice model that can narrate text asynchronously. Two years ago, we started seeing the real-time version of that. I think the real break was a year ago, when you could start seeing that in production. Then I think over 2025, the big piece that hasn't been possible is how you connect the real-time voice interaction with something that I think you're referring to. It has the context of what you want to do, what is the material that you want to read, how it connects to a set of your preferences from the past, and gets that across.

I think that only recently became possible, and we've seen big adoption across the enterprises leading on the technical side. I think this year, it should be in the automotive side too, or in some of the applications.

You think we'll start seeing great voice models in cars this year?

This year, for their own cloud use cases. In-car, without connectivity, not yet. There's a deployment gap, of course, in how you bring that into the cars. But I think in the next 2 to 3 years.

How about the PDF-reading use case?

That should work. Yeah.

But how should I have done it?

Back in the day—I'll preempt this with a story to cue ElevenReader—we had this problem. We had so many audiobook authors come into ElevenLabs. In 2023, we released the first software. We had a lot of creators, and then a lot of audiobook authors or book authors who couldn't afford professional narration and wanted to create an audiobook. However, none of the companies accepted AI audiobooks.

You can't sell an AI audiobook on Audible or something?

Exactly. Audible would block AI content. We had no choice. We needed to create an avenue for them to—

Ah, because there was no distribution for AI audiobooks.

Exactly. We created ElevenReader, and that came with a functionality where you can upload your PDF, you can upload your text, and have it read out loud with a number of incredible voices, whether it's Sir Michael Caine, all the way through to working with his estate and with Richard Feynman's estate, where you can have that—

You are working with the Sir Michael Caine's of the world?

Exactly. Then you can actually read it out loud. That works extremely well. Now, how can you do it?

Gosh, I do want everything read to me by Michael Caine.

It's a great voice.

Shouldn't you guys have a consumer app where I can just do the common voice things? I want to be able to have an Eleven app on my phone, and then if I upload a PDF to it, it can do the common things that I would like, such as have it read it to me.

Yeah, that's exactly ElevenReader. That works.

The phone makers allow third-party keyboards. Do they allow third-party transcription engines? Will they, do you think?

The phone makers, you said?

Like Apple and Google. The operating-system makers.

Yeah, not all of them. With Android, you can work through it. There are variations of that, Nothing.tech and others.

I feel like if you had a popular Eleven app that allowed for transcription, people would use it a bunch, and maybe eventually Apple would say, "Oh, we should allow third-party transcription engines if that's what people want."

It seems like they might be going in that direction.

Recently, they announced that they will open up the LLM ecosystem. Hopefully, they will do the same with the voice ecosystem, which is similar.

I think it's rational to do when it's moving so quickly. The voice assistant paradigm is one of the oldest UI paradigms in computing, like “Open the pod bay doors, HAL,” from 1969. I will claim it's not working yet. Siri doesn't have the intelligence.

Then, on Gemini and ChatGPT and those apps, I want to use the voice mode, but I don't know about you—it just doesn't work. Sometimes I'll be using my phone, and I'll use the iOS keyboard transcription to type in the field and then say a bunch of stuff, and then send it off. But this suggests to me that consumers really want voice mode that works, and yet it's just not working for the major LLM apps or for anyone. Why isn't it working yet?

4. The Voice Turing Test

It is pretty hard to do because you want two things. You want to be able to say things that you want, but sometimes you want it to execute them, and sometimes you want it to wait for you to finish and add something to the sentence. Sometimes you want it to be interactive, so it asks you questions back to clarify and get some of the additional detail. All of that is actually pretty hard.

That's where the magical ideal version of a voice agent for us comes through. You need the speech-to-text element, the transcription side, and then the turn-taking mechanism. When do you finish sentences? When is it likely, based on silence and the context? Then sometimes you want it to speak back and clarify, or at least give you the text back to clarify, and then maybe execute a set of instructions.

That problem is still very hard. I agree with the claim that this orchestration side has not passed a true conversational-agent Turing test, where it behaves as you would expect from another person, where you can say—

That's the simpler way of saying what I'm saying: We have passed the Turing test with text LLMs a long time ago. We're actually nowhere near that on voice LLMs. It's interesting how that's a final frontier.

I feel like it's going to work in specific domains. In customer support calls, it passes the voice Turing test. It works well. Let's take another spectrum of that: an interactive gaming experience, truly interactive as you would have with another human in that game. It's so hard and further out there. We haven't passed it yet there.

Yes, that makes sense.

But I think that's a combination of an even simpler version within that. Sometimes you might give a response immediately back. Sometimes you need a tool call to get additional information from the database. How do you orchestrate that?

That's probably the most common thing we see as we work with some of the companies out there. You want those systems to orchestrate extremely well. If it's a conversational use case, pretty simple, you can route the agent to speak with it. But if you need to authenticate, if you need to pull additional information from the database, what do you do? How do you handle that graciously?

That's where it gets tricky.

To that extent, I would say that we're just getting there. We'll hopefully see that. Our goal is to pass the voice Turing test in all those cases, or the Turing test for all conversational agents outside of voice, too. I hope we will all be there in the next year or so.

5. Personalized and Controllable Voices

For subscription businesses, a lot of revenue is lost in that last few seconds before the checkout. Someone has to get up and find their wallet, or they mistyped their card number, or they hit an error, and they just give up, and you lose the sale. For a company like ElevenLabs, adding hundreds of thousands of subscribers, even a tiny bit of friction like that, it would really add up. But that's why ElevenLabs uses Link from Stripe. Customers save their details once, and then they can check out in seconds across more than a million businesses with saved credentials. If you want a faster checkout for your customers, you should turn on Link from Stripe.

Are you guys working on personalized voice transcription, where it feels like part of the way we're making it hard for ourselves is when I speak to Siri, I have a bit of an accent, and so it sometimes has a hard time understanding me? But my accent doesn't change. It could just get good at listening to John. But my understanding is it's not; it's just running the global voice-recognition model. I'm guessing it's the same for ElevenLabs, where you're running the global voice-recognition model.

But again, you have an accent. If you walked up to someone in a coffee shop and said two words, they might have a hard time understanding you because they're not putting it through their Mati Polish accent filter. Where's this going with actually interpreting the person that you know to exist on the other side?

I have a very tricky accent to detect. My voice is frequently used in the tests.

You're part of the test suite.

For text-to-speech, for speech-to-text, for everything. It's pretty—

But again, trying to parse your voice in a global model is just making life hard. It's like, have a Mati-specific model.

On the speech-to-text side, by transcription, exactly. The big part now that we are bringing in is that you have two parts. One, effectively, is person- or voice-specific detection, which is true for the accent side, but it's also true for a crowded room. We have an incredible research team that's able to continually keep both the accuracy high and add things like speaker detection, of course, and noise reduction.

But then the second part is also keyword detection. There are specific words that you would want to say in those settings that you want to effectively monitor for. We spoke about, let's say, I'm going to the coffee shop and ordering things. The set of actions the coffee shop would expect me to do—

This is information theory. It's like they can just listen out for the coffee words.

Exactly. Then try to match it to the closest proximity. Both things will help. In a setup where you have my voice, perfect—you can decode it and code it on that. If you don't have my voice, or even if you want to double-amplify it, we already support keyword detection, which is useful for real-time and asynchronous settings.

Back to Cheeky Pint transcription, you could effectively pre-generate that from the previous podcast and look for a set of words that you would traditionally use in it.

You do the keyword detection already, but how hard is it—I want to get superhuman transcription performance by feeding it an hour of Mati audio before it listens to Mati, and then it should be able to do a much better job transcribing. Is that just a really hard research problem?

No, it's solvable. We think we can roll it out in one of the next versions, hopefully in the next month.

You think this year you're doing person-specific transcription?

Person-specific transcription. We can already diarize our speakers extremely well. If we are speaking, we can, of course, discern who is speaking when, which, on the transcription side, apart from accuracy, is one of the harder problems. We do that extremely well.

Now it's going to be effectively what you're saying: fine-tuning based on the speaker that I want to listen to, which we know will be important. In a healthcare setup, it's such an important part. You're in an operating room, you're a doctor, you want to say a command, and then you want to really be able to listen to that one person's specific piece.

You have a hardware device at home, let's say it's a pilot that helps you control the TV. Here, too, you will want it to listen to you versus, let's say, the family running around. Or maybe you want it to listen to everyone. You could decide that, but in many cases, you want to be able to specify that.

That's really exciting.

It's great because there's still so many unsolved research problems.

Yeah, there's just breakthrough after breakthrough coming in the domain of voice models. How about on the flip side, when it comes to speech generation? In the Zoom “touch up my appearance” feature, I've always thought about that in the context of voice. Should you offer a de-accenting filter for voices?

There's one podcast that I like to listen to, but the voice is a little mumbly, and I always thought they should put it through a de-mumbling filter just to make the—

Or slow it down.

Yeah, make the enunciation a little better. But all these things, again, like Photoshopping an image, there's no reason that the— Have you thought about voice-to-voice, basically, rather than voice-to-text or text-to-voice?

Yeah. There are two big parts. One on the speech-generation side, similarly, so many innovations are still there. There's a wider piece, and we released the V3 model that we're solving that for the first time. It's like, can you control speech? You can have the text-to-speech. You generate something that sounds emotionally great.

Previously, until the end of last year, effectively, you would rely on the model to decide what the best performance was.

You could regenerate it, but ultimately, the model decides the best performance. That's where controllability came in: we can finally give it cues, like saying it in a slower way, changing how you deliver a dramatic pause, or any other cues that you give.

To be able to do that, you need the architectural changes and the data that we created over time, where you annotated what was said and how it was said, so you can actually train the model to do that. Today, finally, you can have both speech generation and an entire voice-agent experience with what we call expressive mode, where the agent knows the emotions on the other side.

If the person is stressed, it can react and be reassuring. That's generating an LLM response on the reassuring side and responding in that set of emotions, too. That breakthrough was super hard to do. That, of course, stretches to a lot of what you said. It could be some version of speech enhancement, either in real time or in a post-processing setup, to change how that's delivered. That's a relatively recent innovation, and we know it can still be so much better. The edge cases of how you want to describe it are pretty large. That's one.

Then the second part of the question, which is a huge question, is speech-to-speech models. As you said, our approach, as you think about a voice agent or conversational agent, is effectively a cascaded approach. You use transcription, or speech-to-text, an LLM, and text-to-speech, and orchestrate all of that together. Then you have speech-to-speech, which goes directly from speech, and there's a speech response on the other side.

When we say speech-to-speech, is that the idea that it doesn't go through text as an encoding in the intermediate?

Exactly.

Oh, interesting. For performance reasons? For accuracy reasons?

You usually do it for latency.

It is faster to run a model that does not have to transcribe and then generate.

Exactly. It's quicker, but on the flip side, you lose reliability. You lose all visibility into the parts of the pipeline. In emotionality, we think you can deliver both sides extremely well, and maybe you can make it more controllable, too. Today, we are optimizing heavily on a cascaded approach.

I'm sorry, a cascaded approach is what?

It's speech-to-text, going through the text layer. As we work with a lot of businesses and enterprises, they will need that visibility into what happens. They will want to execute certain tasks on top of that. They want good visibility into each of the steps and great accuracy from all of the models.

Beyond that, they can abstract away what's the LLM layer and what's the intelligence layer. The integrations are easier in that system. That's where we are betting a lot of the research work: on how you can make that great. We think we can make that great.

In speech-to-speech, as you think about more of a companion version of the applications, that's where it will flourish, because maybe hallucinations aren't as important, but latency matters a little bit more. Maybe hallucinations are even a feature.

Maybe in the future-future, just to finish that part, you will have some version of a combination of the models. For low-complexity, easy models, you will have speech-to-speech, and for higher complexity, we will have to cascade it.

I was going to ask about this. You know the way there is research on how the invention of writing changed human brains and changed the neural pathways in ways beyond the actual written language. Do you observe that speech-to-speech models think differently than cascaded models? It sounds like they're dumber.

They are definitely dumber. You need a smaller model.

But that's interesting, that forcing models to reason about text. I know they just have much more in there as well, but they're smarter.

Yeah, but if you are getting speech-to-speech, usually you will use smaller models, so it's still quick.

I see. It's also just a model-size thing. But are there interesting differences beyond correlates like size?

What I can say is slightly different from your question. The people interacting by voice, and the performance we see in how they interact with the business, changes just by nature of interacting with us.

A good example: you can contact ElevenLabs and register your interest. You go through the form, and at the end of that, we supplemented it so that instead of going through the form process, you can speak with our agent and leave more details.

What happened there were two things. One, people were actually much more keen to complete the forms by speaking with the agent, so we would go through the form a lot more easily. But second, they would be a lot more open-ended in terms of what the use cases were. They would start giving us information about the wider set of use cases and the complexity of the use case. Writing it out was tedious and tricky—

This is like an open-ended adventure game.

Open-ended. You could ask follow-up questions and clarify. But people were just more at ease and could trust the system while doing that, that it was working. That helped us a lot.

And 3, which is perhaps more of a technological barrier, is that it also works across all languages. Now we have leads from all parts of the world coming in and leaving their details.

We did that use case, and now we have a few different companies building their SDR versions of that, too, to help them capture the leads coming in, from banks all the way to one of the automotive companies. People are just more keen to speak through voice.

I want to ask about these second-order effects. You've talked in the past about how, growing up in Poland, I guess the dubbing of TV shows was cheap, and so they would only have one voice actor for a TV show. No matter the parts, male and female, they're like, "I love you." "I love you, too." There was one voice actor doing all of them.

Now, thanks to better voice models, you'll be able to have really good AI-generated voices for all the dubbing. Again, it's not like it's taking jobs from great dubbing that was happening previously. It was awful dubbing happening in Poland previously. That's one example of the second-order effect. What are the other second-order effects you're seeing from ubiquitous good text-to-speech and speech-to-text?

It seems like across a broad array of languages, because whatever exists in English, this just didn't exist in Polish or Irish or, pick your language.

6. Voice Breaks Language Barriers

One is breaking down the language barrier. The inspiration came from the movie side, but it also applies in any communication setup. In the future, could I travel to another country and speak Polish or speak English, and have that language understood in the local native language? From The Hitchhiker's Guide to the Galaxy, this is a version of the Babel fish. You can actually understand the world. Voice, of course, will be an interaction layer.

Similarly, all of us will have our own extensions and voice agents that can help on our behalf. There are very clear and great examples of people who lost their voices and can get them back for the first time. We see that everywhere, whether that's people who lost their voices due to ALS or throat cancer and can get them back.

Just recently, there was an example of a patient who had Neuralink. We worked with them to bring back the voice that person could speak with, using their own voice, with the family around. We worked with a lady who lost her voice before she got married. Then, finally, the technology became possible. We were able to recreate that voice. For the first time, she could replicate the marriage ceremony and speak the vows together, which was such a heartfelt moment.

That's really sweet.

It's probably the most important of all the work that we do.

When you guys talk about voice agents, is a voice agent just the idea that you have some long-running or persistent agent that is going out and interacting with the world through voice? Customer service would be one example of it. In the other direction, your OpenClaw going and making you a restaurant reservation and actually calling up the restaurant—is that how I should think about voice agents?

That's right. Exactly. Whether it's the reactive side, being able to interact with the customer, or the proactive side, calling it back.

We recently had a very interesting one, topical because it was a Guinness-related one, where there was a developer developing a Guinndex.

Oh, I saw that. They were calling all the pubs in Ireland to check on the price of a pint.

You could ask that or report information.

The Guinndex was built with ElevenLabs technology?

It was built with ElevenLabs, too. People could actually do both sides: proactively reach out, reactively reach out, and always capture the full voice. Then 3,000 different entities could report their prices and get that across.

Have you, by the way, hooked up your OpenClaw to ElevenLabs? Is the OpenClaw–ElevenLabs combination something that a lot of people at ElevenLabs are doing?

As you know, OpenClaw will look for the most popular tools frequently when it tries to cook things up.

ElevenLabs is one of the recommended ones.

It's the top option for voice. Can you tell me a bit about the business of voice models? I think people have an intuition around big LLMs where there are these very expensive training runs, and yes, they depreciate quickly, but there's so much usage that all of the models trained to date have paid off their training runs and then some.

Then there's this ever-larger CapEx going into—I mean, a lot of it is inference these days, but also training. People have some intuitions from the LLM world. I'm curious just how I should think about voice: how expensive is training the voice models? Is the expense in the researchers? Is the expense in the training runs? The economics is presumably simple, where it's just per usage. But yeah, just talk us through the business.

7. Scaling the Voice Business

Definitely cheaper than the LLM and image-video models. Significantly smaller models.

The models are smaller?

Smaller.

What's the parameter count for a leading-edge voice model?

A few billion to tens of billions of parameters per model.

For context, I think CPUs moved away eventually from gigahertz as the metric as they moved to more cores. I think we've mostly moved away from just raw parameter count, but I think the leading-edge LLMs are in the hundreds of billions of parameters.

I think the leading ones, yes, but of course, you have the variations that you will use at lower scale. CapEx is still pretty high. We've, of course, raised recently half a billion at an $11 billion valuation.

Makes sense.

Makes sense. To continue being able to build the best models in the world, researchers, of course—you want the best people in the world. I think we have those people working in audio, and my co-founder is leading that work. That's definitely a big piece—not financially, but even in how you maintain the ambitious deployment, so you continue building leading models. That helps you attract more talent to build that.

Then, on how we service, of course, inference is correlated with how the models are used. For us, we've seen incredible growth across the work. Mostly, this is charged per—if it's input text or text-to-speech, it's usually per text token. If it's a voice agent or transcription, then it's per minute. We see that being the bigger part. But usually, broadly, it's on a per-token basis. Of course, as we work with businesses, it's like an annual agreement. The bigger the spend, the bigger the commitment, the bigger the discount to get it across.

The way we usually do it is, when we have a new model, we try to give it at cost to a lot of the customers so they can experience the best. It's still usually not as reliable—

That's interesting. The newest thing is often the most expensive, whereas you make the newest thing the most economically attractive one?

We try to make it attractive so the customers are—it's more expensive for us than any previous generation. The quality is higher, so we try to keep the prices still competitive.

I see. You subsidize it, but it's inherently more expensive as a bigger model.

Exactly. Over time, we might do some tricks to optimize it, but we want the customers to experience it. Because of research, the big thing that we've seen is that the reliability of the model in the early days might not be there. Then, two, people don't even know what's possible with that model. You want the widest set of distribution, so people can show the world what's possible. You can have it, of course, as the distribution mechanism, learn yourself what to improve and what to change, and then get it out there.

Are the voice models just getting bigger and bigger? Will we have voice models in the hundreds of billions of parameters, or have we found— it seems like for certain types of model architecture, there's an upper limit on the natural size. Have we found that upper limit for voice models?

It feels like for specific use cases, like audiobook narration, you probably found that size. You probably don't need to stretch it much bigger to make the quality much higher, but for certain use cases, that will probably grow. The thing that I hesitated on with the question is, in a cascaded approach, you probably will not see dramatic size changes.

You inherently want the models to be quick and reliable. You want to orchestrate them in a smart way. In a fused approach, probably that will get into tens or hundreds of billions of parameter models because you combine, of course, the LLM side and the voice side, so that will get bigger. But on just voice, I think it will keep being small.

There are certain domains where we'll see bigger models. That's so interesting. It does seem fun, from a research point of view, how there are still these various unsolved aspects, and how you guys are just making technical breakthroughs and then releasing them down the product pipeline. That's a really fun stage of a company's life cycle.

For sure. It's fun because it feels like we can do innovations on both sides. There's so much on the research side, so much on the product side. Ultimately, the biggest part is how we deploy it to the customers, where SMBs will have a very different dynamic than the enterprise. It's not a vendor-SaaS relationship where you just give the product out there for the biggest companies out there, but you are more of a partner in their AI transformation.

You want the resources to work alongside them, to work on the frequently very new use cases that were impossible to help create, and bring those voice agents to production. That's a big shift. The biggest focus is how we bring the conversational agents out there to businesses around the world.

When you say bringing conversational agents is the biggest priority, is this for customer-service-type use cases? What are the most popular use cases for conversational agents?

We want to be a partner for full interactions between businesses and their customers or their audience. I'm saying their audience because that will apply in support. Support is the easiest one because that's where it's most ready. That's maybe the big difference in how we see ourselves compared to some of the other companies in the space.

This can also apply to sales. You can have the proactive side of reaching back. You can have AI SDR versions of that. Then you can have all the way to the marketing use cases where we are your partner for working even outside of the conversational-agent space, in how you create a great marketing campaign.

How does this break down between—we had Des Traynor from Intercom on here, and they have Fin, their agent, and it's a thing on the website that you can go talk to. He described a very similar phenomenon that you described, which is, you start maybe thinking, "Oh, this will help me answer customer-support queries." It becomes like a generic UI for the website, where it's a box you can type in to go do things and understand things. Why wouldn't you read the docs and design your integration that way? Whatever.

Will I have one for text and then one for voice? Will you guys do text too? How does that—because it seems like this is also succeeding at the text level with Fin and Sierra and all these things.

The places where we know we will be able to provide the biggest value are where, ultimately, today you will have either a big portion or most of the interactions coming through voice. If that intersection is there, that's where we can provide higher value.

Of course, if you need a text chatbot there—if you fix the voice agent, you have fixed the text piece inherently as well. The place where we do optimize today is going to be how you select the right voice for the right customer interaction and how you pull that in there.

In the pretty complex case of what you mentioned earlier, of how you orchestrate that to pause or look for something deeper in the docs, how it can be an extension of the entirety of the business, so not only in support, but across the entire user journey.

The bottom line is we want to be able to provide for you across the entirety of the interactions. Voice is usually a big part of those interactions. We need to solve the integrations, we need to solve the knowledge, we need to solve text as part of that. We wouldn't, for example, go very deeply into what I think will happen in a lot of those cases—the reasoning version of those use cases where you maybe need the multitouch.

Yeah, and a lot of complex actions.

A lot of financial analysis. That would not be something we optimize for.

Can we talk about your revenue ramp? You're just one of the fastest-growing startups, period, of the past few years. What's your most recently announced revenue figure?

Most recently announced was the end of 2025.

Whatever number you want to give us.

Most recently announced was $350 million at the end of 2025. The best proof of the technology working…

Recently, we were working with Deutsche Telekom and T-Mobile, Revolut, Klarna, Meta, and IBM—a wide set of use cases. This quarter was one of the best for enterprise growth, where we had our first quarter hit $100 million in additional ARR growth, which is crazy.

In net new ARR.

In net new ARR.

If you're thinking this quarter was $100 million in net new ARR and $350 million at the end of the year, I'm no mathematician, but it's up in the $450 million range. Versus this time last year, that's a several-fold increase. What's working? From the outside, I would assume that there's really strong cohort growth within accounts, and then you seem to have self-serve and enterprise businesses that both contribute a lot. I don't know how big self-serve is, but as a user, I like to be able to fiddle with ElevenLabs and not have to go talk to sales. Maybe you can just talk about what worked to reach $450 million-plus of ARR so quickly.

Over 50% is now sales-led enterprise. I think, largely, that technology powering a lot of their agentic interactions just became reliable at the same time as high quality over the last year and a half. Frequently, you will start the account, and then, of course, it continues expanding. We see there's definitely a land-and-expand motion across ElevenLabs. We bring—

What does that expand look like? Is it new departments? Is it just the usage starts taking off? When a customer expands—

Both. Usually, the first part is that we try to make it very easy for our customers, maybe against ourselves, where we give the technology pretty attractive economics because we so much believe in the technology providing value. You can actually try it and test it. Then, within that one department—

You think you'll make it up in usage, basically.

Exactly. That usage, the commitment continues increasing because you know it's providing value. Then it's so much easier to make that a choice. Of course, cross-department pollination is there, too. Our work with Deutsche Telekom started with the marketing side. We did Magenta work and podcast generation. Then it expanded to customer support. Then it expanded to us working on an agent across the entirety of the network, so people can call in and have the agent. You could see those step changes across.

We are now 470 people as a company, so we keep on growing. Some of the things that stay consistent are small teams. We have fewer than 10 people on each of the product or research initiatives. Even as you think about sharding some of our go-to-market strategy, there will be smaller teams understanding the industry in depth, understanding the market in depth, and going independently and going quickly. That definitely contributed largely to that.

Especially on the biggest enterprises, what we found works is that we have the full spectrum: self-serve, PLG motion that helps drive distribution and awareness of ElevenLabs. On the completely other spectrum, we have high-touch, deployed engineering working side by side with the customers to customize the entirety of their work together.

Why did you guys do self-serve? Because I presume you have a lot of competitors where they have technology, and it's behind a contact sales form, and you have to go talk to an SDR and then talk to an AE. You guys just offer the technology available on the side.

I'm a huge believer in this. A huge part of Stripe's growth has been driven by the fact that we just made Stripe available to anyone and built a lot of products around that adoption pattern, but so many companies seem to skip it. I'm curious how you guys came to—

So many reasons. The quick ones that come to mind are the feedback loop. You have immediate understanding of how good your technology is. Two, which is an extension of that: we stand behind our technology. We believe it's the best in the world for models, for voice agents, and for deployment. We want people to experience that.

I think you do that the same way at Stripe, where the best version of the technology is available to everyone, which is so attractive to actually try it out. We always try to make everything we built for the highest-end use cases available back to the ecosystem for free.

Frequently, the newest use cases for enterprise will need reliability, compliance, and the ability to scale, which we deliver. Frequently, as you develop new technology, it might not be ready for a lot of those parameters, but it's definitely ready for developers and SMBs. We love what they are doing because they are showing us the future and effectively helping us find a trajectory for where ElevenLabs should go.

I'm totally convinced. I'm just always amazed that more companies don't pursue it, where it feels like they're really shooting themselves in the foot by not. Did you guys self-serve on Stripe?

We self-serve on Stripe.

For example, ElevenLabs is a huge company, and yet you started on Stripe on a self-serve basis.

Initially, there were two of us at the beginning. You try to see what's working in the industry, but you try to think from first principles. You want to try it out. You want to understand how it works. The more friction elements there are before you're trying it out, the less you trust whether it's available, whether there'll be additional payment hidden behind some of those steps. You don't want to go through them. It's so much.

Speaking of Stripe, do you have any Stripe feedback for us? Anything you want us to fix?

My most common feedback until recently was, why don't you give us pay-as-you-go, usage-based billing? One of our finance leads, Maciej, was speaking with your team the day before. He was great. He had been thinking about it for a long time.

Was it he who said, “You guys should buy Metronome?”

“You should buy Metronome.” Then the next day, the Metronome acquisition was announced. Now you have it. That was my most common feedback, and we'll be launching. That's a good announcement for this podcast: we'll be launching usage-based billing for everyone.

Sorry, I'm shocked. Previously—

Pay-as-you-go.

Previously, you had it on an enterprise basis, but everything on a self-serve basis was—

We had the subscriptions, yeah. Subscription plans—you could go over them—but now we are launching a full pay-as-you-go experience. You can just try out the voice engine, which is effectively this orchestration loop all the way through to any of the models directly.

Going back to self-serve, I think a new thing in AI is that all self-serve products should have pay-as-you-go as an option. Maybe you want to have a subscription with some unlimited tiers, but I don't know if you've had the experience of using Claude, typing away your queries, and eventually hitting some rate limit. It says, “Sorry, you've hit your usage limit,” and you want to be able to do the thing that you can do with Claude Code, which is just pay per API. It's like, “I'll pay for it.” It's very funny as a consumer not to have the option to pay more to use the product more.

I think every AI product will need to have some all-you-can-eat subscription with limits, and then the ability to pay for overages. It sounds like that's what you're doing.

Yeah, exactly. That's what we're doing.

The only thing I wanted to ask you about is that I feel like all CEOs of larger companies today are trying to figure out how all these AI advancements change the nature of the organization and how you redesign your organization around all this new intelligence. That could be about the scaling factor of the number of people you need to do the work.

It also should be: do you need more senior people because they're better able to direct the AIs, and the AIs maybe can do the work that previously would have been done by junior people? Do you need more junior people because they're going to be more AI-native in how they work? Do you want smaller teams? Do you want bigger teams? How do you actually do the process engineering of it? Your finance team should be using Claude extensively.

Finance teams do not historically have a lot of home-built software. There are all these questions that are floating around, and you have very rapidly built a much more AI-native company. I'm curious what lessons we should all be learning from ElevenLabs as a large business recently built, without the baggage of decades of “how we've always done it.”

We started in 2022, which is a year when the two topics of the day were crypto and metaverse.

Yeah, because you scaled in AI.

Exactly. We had the privilege of scaling through the world when it was all happening. For us, what works—and we really believe in that being a big part of the future—the first is small teams, keeping the teams small and super flat.

Both my co-founder and I will have over 15 direct reports each that we'll work with. Most of those people will have that same span of direct reports.

Your span of control is way larger than a traditional company. Normally, it would be 8; you have double that. Obviously, that's exponential.

Exactly. Of course, there are some teams which, in the short term, might not do that, but ultimately, that's where we think it's going to be headed. It's roughly 10 in team size within each of those workstreams.

Startups, no offense, but startups often have pretty wacko management ideas. There was a funny tweet: “Lord, grant me the confidence of an early-stage startup founder blogging about their management theories.” You think this is not a startup effect? This is an AI effect where basically—

No, it's definitely a little bit of a startup effect. I figured out it's the hindsight benefit.

I'm canceling our Stripe changes.

No, it's like I need to preempt it. The hindsight of this may be working. We'll see in the next 5 to 10 years.

Much flatter org.

Much flatter org.

Smart teams.

It works for us; it might not work for all companies. There are some parts, like go-to-market, where we're still trying to figure out the best way. But smaller teams, flatter org. I think there are two paradigms, but generally, people being more technical—or, if not technical, even in non-technical teams, having a technical resource.

We will have a person in ops or in talent who will effectively be a tech lead for that team. That helps them automate a lot of the work and helps up-level the rest of the team, too. There are two parts that are helping.

Talk me through this in talent or something like that. Is it that you are building your own software where other companies might have bought software, like Workday or Greenhouse or something? Is it that they are using the existing software you have better? Is the process that would be spreadsheets in a traditional company built with software? How do you use the software in these organizations?

Sometimes, but we still use a lot of the traditional vendors. One pattern is, of course, LLM-ifying everything, making the data explorable so you can interact with it: who's in the pipeline, what worked, who does the best references, all of that work, so you can double down on that.

Two, it's frequently things that you manually do. There's a gap between where the agents are today versus what you could do if you have the technical skill set. A good example is: How do you scrape all the right profiles to be able to reach out to the right candidates? You analyze whether it's—I don't know how much I want to say—but try to detect specific things that we know worked. You bring that across to the people.

On the go-to-market side, there are just so many things you can do with additional amplifiers. It goes from understanding what case studies are relevant and creating a good pre-read for you before you go to the meeting, through creating the AI SDR experience that we spoke about, to creating an entire deck experience. You have a pre-populated deck with the right numbers that's customized to that customer, which you still want the person to go through and develop, but ultimately is in there.

There are plenty of those additional things that you know will amplify the work of the people around you, potentially replacing some of the easier tasks that are done. Then we wanted people to explore the culture at ElevenLabs. We created a voice agent that people can speak with and see what the culture is, but also get prepped for the interviews.

I think across many of those teams, there's an additional benefit to what they can do. An interesting piece, of course, is in Ukraine, where we're doing ongoing work. They need to rethink a lot of how their development, their systems, and their support work for citizens across the country.

People who are in the war zone don't have the same access to information. They cannot rely on the same phone lines. They cannot rely on the same physical services around the country.

Is it your employees in Ukraine?

We had a few, but they reached out because they were developing their central app called Diia. They developed it over the years, but now they were doubling down on how this can be a way of supporting citizens. Of course, there's an easy part of how you create a first agentic government where you have help with benefits, what's happening on the front line, or education.

That's delivered to everyone. Or healthcare, so you can book your checkup or appointment. How do you create all of that? Of course, we traveled to Kyiv, worked with them on bringing that and making it available by voice so everybody can access it.

But the thing we've learned while being there was that model we speak about, where you have technical resources in each of the teams, is actually the same in every ministry. Every ministry had technical resources working on creating that agentic version of their work.

Then it was a central digital transformation team that would assemble this all together to deliver it through the central citizen support, which I thought was brilliant.

That's very tech-forward for Ukraine.

So tech-forward. The most advanced set of work we've seen. We got a little bit validated. Maybe technical resources in each of the teams is a good idea. That works heavily for us.

You mentioned some of the other parts, like whether you hire senior or younger people. The main thing we try to filter for, of course, is the culture piece, which is so important. You could scale people, but scaling culture is much harder. You want to optimize for that being right.

In our case, it's first principles, taking ownership, striving for excellence but staying humble. The main thing that's in that ownership part that I think works well for the AI world is agency. If you have that agency to explore, regardless of where you are in the experience cycle, it's going to be a tremendous amplifier to your work.

My biggest takeaway from all this has been around agency, where I feel like high-agency people are the winners of the advances in AI, and within organizations, low-agency people will lose out.

Yeah, completely agree. It's probably the thing that Piotr and I are most proud of as we scaled ElevenLabs: the people who are at ElevenLabs. It's been the culture and seeing the expansion of the culture, where culture builds the company now rather than any single person or any single product building the company. That is probably the biggest validation and happiness.

There's the other angle of that, where I think people are striving to be incredible in their craft and their work, but at the same time, have fun in a lot of their work. That combination of agency and just enjoying what you do is probably the best thing we've been able to do today at ElevenLabs.

It sounds like a really fun stage, like we were saying. Interesting research breakthroughs, really fast-growing business. I'm sure you're enjoying it. Mati, thank you.

John, thank you so much.