ElevenLabs CEO：为什么语音会成为下一个 AI 界面

We don't want to become the same as the previous generation of the editing suite. So instead, let's solve it at the research level, where it will know, based on the voice, exactly how it should speak and at what speed. To cater to all those different use cases, you need such a big array of different voices, languages, accents, and styles.

So we launched a voice marketplace where you could create your voice and then share it. When that voice is shared, you earn money in return. Today, we have almost 10,000 voices, and we've paid $10 million back to people in the community. There are some crazy stories from the voices. Just speaking through exactly the technology, showing the examples, and avoiding this initial knee-jerk reaction that AI is bad has been tremendous.

I'm excited to welcome our first speaker, Mati, co-founder and CEO of ElevenLabs. [Applause] All right, so good to have you here, Mati.

Thanks so much for having me here. Great to see everyone, and good morning.

And that was the walk-on music generated by ElevenLabs, was it?

It was. We expand continuously across the audio space. We started with voices, then created orchestration for building voice agents, and now also create a fully licensed music model that can produce amazing music to go alongside it.

Awesome. We'll talk about all of that. I've had the opportunity, and also the luck, to get to know you from the very early days, when ElevenLabs got started, and to partner over the last 3 years, just to see your execution everywhere—from product launches to shipping new lines and models, like you just mentioned: everything from text-to-speech models and speech-to-text, and then we started doing music and sound effects, and now the AI agent platform.

I'm very curious. First, I'm still in awe of the shipping speed after all 3 years. But I want to ask: how do you actually maintain both the speed and quality when you have such an expansive product roadmap?

So, first of all, we partnered almost 3 years ago, and it's great to hear all the kind notes. But what they didn't realize when we partnered was that the infrastructure team was 3 people, and of course now at ElevenLabs, the company infrastructure team is 11 people, so we're seeing the growth on the other side as well. I hear that the companies here have raised $66 billion in total fundraising, so the number 11 is everywhere here. [Laughter]

But I think the first piece is that I have, I think, the smartest person I got to know as my co-founder. He has been the research brain for creating a lot of the models and for assembling what we think are the most incredible researchers in the voice space. That allowed us to create the first text-to-speech model that could understand context in a better way and turn that into emotion and intonation.

Then we found a way to capture the characteristics of the voice, so you have the voice sound with the right style, age, gender, dialect—everything in 1. The researchers have, of course, now expanded that to speech-to-text, music, and other work. That's our foundation.

Then, the way we structure it to be able to ship quickly, especially with so many things happening in the AI space, is through a lot of small teams. Today, we have roughly 20 product teams, each 5 to 10 people in size, with full independence to go ahead and ship products.

Of course, that carries some issues, like duplicative work, or sometimes people going at different speeds. But on the positive end, the ownership of each of the teams is extremely high. People know that it's down to them to really deliver and ship, and it allows us to move extremely quickly.

We bucket our work into the creative space: a creative platform where we help with narrations, voice-overs, and dubs for creatives in the media and entertainment space. Then, on the agent side, we help people create voice-agent experiences and conversational-agent experiences across customer experience, all the way through to immersive media.

Great. ElevenLabs has “Labs” in the name, very similar to many of the other big labs, which means you're doing first-party R&D and model development, but also building all these 20 products. How do you think about balancing both—continuing to progress on model research, but at the same time not delaying product launches?

Yeah, it's very tricky. I'm sure many of you have the same thing: do you build a product when you don't know if the research innovation will displace the product you just built? We had this in the early days, too. One of the simplest examples was that we had a model that worked, and one of the most common requests was: Could we do different speeds for voices?

Could you have an additional slider to modify the speed of how audio gets generated and how quickly it speaks? We were very against this idea: We don't want to do any sliders or toggles. We don't want to become the same as the previous generation of the editing suite. So instead, let's solve it at the research level, where it will know, based on the voice, exactly how it should speak and at what speed.

We resisted this for a good 9 months, and we couldn't solve it on the research side. Then the product was a super-simple solve that got all the users across. Now, the approach we take in looking at this is: if we think the research work will take more than 3 months, then the product team can do anything they want to start adding other models and some of the extensions.

Of course, sometimes the timeline is tricky to predict, but roughly, the guidance we have from our internal research team is: What are the initiatives we hope to ship this quarter, and what are the long-term initiatives? For anything long-term, you can use any other work to close that gap and make it better.

I guess first you kind of have to figure out if the research commitment is going to meet the timeline, and then go on to align with the product teams. That makes a lot of sense.

As everyone is moving to San Francisco and building in person and locked in, in the same space, ElevenLabs has always been building globally and having people more distributed. But you now have centers, I guess, in different locations—from London, Warsaw, San Francisco to New York and other places. How do you think about building this global expansion and finding talent globally versus the trade-offs of building in the same place?

Yeah. My co-founder and I started between Warsaw and London at the time. I think ElevenLabs wouldn't have existed if we weren't starting from Europe. It's a very peculiar thing, but in Poland, if you watch a movie in Polish—a foreign movie in Polish—all the voices, whether that's a male voice or female voice, get narrated by 1 single character, with no emotions and no intonation, as you can imagine. It's pretty terrible, and it's still happening today for most of the content out there.

I've had a similar experience growing up in China, where we have a lot of Western movies dubbed in Chinese in a monotone.

So bad. [Laughter] And in Poland, of course, as a post-communist country, it's a cheaper way to do it. You don't have to hire as many people. You have 1 monotonous audiobook reading of a movie. And that was where the company started.

We started initially in Europe, and we realized that if we wanted the best people to solve what was a research problem at the time, we needed to hire wherever they were. We couldn't lock ourselves to just San Francisco or look at the West Coast. We knew that we needed to find them across Europe and Asia and bring them into the company. So we started fully remote and started looking at those people.

On engineering, we were also very against the traditional hiring method of looking at LinkedIn, looking at traditional backgrounds, and trying to figure out whether we could find a different method to hire people. That led to some very interesting hires. We hired a person who had an incredible open-source text-to-speech model and was working in a call center at the same time as a recipient of the calls to make money.

Wow.

And he's now on the team, one of the most brilliant researchers we have, doing all the data processing. But the same pattern kind of followed. Of course, the early team was very distributed, and then as we started scaling beyond 30 people, we realized that for new people joining, there's a benefit to having a space to be next to others, get deeper into the culture, and understand what all the projects are that are happening in the company.

We started hubs where you can go to London, Warsaw, and San Francisco, where you can work with others in person. That's how we try to marry those 2. If you are early in your career, we try to hire you in the hub so you can immerse yourself in the company. If you're used to remote work, that's completely fine, but if you want, you can always come and join us in the hub, and that worked really well.

Currently, we continue hiring people from very untraditional backgrounds in some places in the company, and then fusing that with very traditional backgrounds, which can teach the others. In sales, for example, we've done some of those experiments, too, where that combination worked really well.

The lesson is you can really find talent everywhere. It's just how hard and how you look for them.

And I think in Europe, also, people are—this was an interesting one—in the U.S., people are very keen and excited to work, and if you go to any social event, it's like you want to talk about work. In Europe, I didn't have this feeling; most people don't want to do that. The cultural piece is different, but then you do have pockets of people that actually strive for it, too. They just don't have the companies where they could do that.

I feel like our team from Europe is the most motivated and passionate set of people that we are lucky to have.

Yeah, I can attest to that, given that I’ve met some of them. They’re very hardcore and have a very good work ethic, for sure. You’ve also maintained a pretty flat structure and have people own a lot of responsibilities quite laterally. Can you talk about the rationale behind that? I guess there was also a no-title policy.

We removed titles a year ago, and it’s going well. It still works. I do think we said we did it, but a lot of AI companies kind of do it already, with “member of technical staff” being the usual title you have for engineering. Then, in a lot of go-to-market, you’re just go-to-market—not VP of sales or other roles. So I think it’s actually a pretty common pattern.

But in our case, we had a small-team approach, where you have an extremely small number of people, usually 5 to 10. We wanted to make it very clear that every team we create has 6 months to prove itself. If it’s proven, that team will stay and continue working. But from the moment you join, you can have any impact on the company. You can have any role in that team. Tenure will not define your position in the hierarchy. If you’re smart, quick, and passionate, you can elevate yourself very quickly, which really helped.

It’s also a common layer for the external world. Everybody looking at ElevenLabs knows that our go-to-market team is the go-to-market team. There’s no positioning to the same extent. What this allows us to do is, when we speak with a lot of our partners and customers, they know they’re getting the best people, always. We can also send people to different conferences and events regardless of that positioning.

I think the tricky thing with the flat structure is that, in the way we currently have it, there’s effectively a set of leads for the subdivisions: research, creative, voice, agents, go-to-market, self-serve and sales-led, and, of course, ops. That’s the only layer of leads, and under that there’s a pretty flat, small-team approach across the world.

You really want the leads to be able to carry the complexity around the team. They can suggest things between one team and another if they see that there’s something valuable happening between them. I think picking people who can context-switch between teams is super important, and then letting the team fully focus on that.

We also had an interesting learning: if you put a person in all the Slack channels and give them transparency, they actually get frequently distracted because they read all the messages. You can still choose not to read them, but you still do. So you kind of need to cut access to a lot of those pieces to force their attention, and that kind of works. All those small things work really well.

Maybe we can borrow some of those lessons too. [Laughter]

Let’s switch gears a little bit. You’re on the front line, seeing a lot of the creative work—whether it’s from art, music, or advertising—that’s starting to adopt AI tools. In the beginning, that wasn’t the case. There was a lot of resistance, and now we’re seeing the adoption and the welcoming of more generative AI tools, including AI audio.

You’ve done some really smart things, from marketplace payouts to working with these creative industries since day 1, actually. I remember how much you stressed that we had to find a way to work with them, while also observing the market shift over time. How do you adapt to these changes and find ways to work with the industry in its infancy? How did you navigate some of those challenges?

I think the first piece is actually spending time with the industry and trying to understand its priorities and incentives. Of course, it’s sometimes tricky. Sometimes you end up being starstruck. We had the honor and pleasure of working with Jared on some of his incredible work and learning from him about what’s important— which parts of the production process you can actually use AI for, which ones you want to keep, and where it’s actually helpful.

I think that’s the key across all the partnerships in the space. In our case, we tried to figure out how to do that in the voice space, which is, of course, about how the voice-acting space will look in the future. To cater to all those different use cases, you need such a big array of different voices, languages, accents, and styles.

So we launched a voice marketplace where you could create your voice and then share it. When that voice is shared, you earn money in return. Today, we have almost 10,000 voices, and we’ve paid $10 million back to people in the community.

There are some crazy stories from the voices. One of our first voices was a deep Spanish voice, and the magic of the technology is that the same voice is now available in all different languages. At the time, it was available in 30 different languages; now it’s 70. We had the Spanish voice join us, and it wasn’t picking up in Spain. Nobody really liked it as much, and then it picked up in an English-speaking country—that same voice—because of its deepness. Now it’s one of our top 3 voices for all the use cases. So you can all register to our voice marketplace and maybe earn some money too.

The second important thing is figuring out how we can be part of the industry and bring it together to disrupt together, rather than just disrupt. With labels, I think I’m still learning how to interact. We’ve worked with labels, including Merlin and Kobalt, as well as 4 of the majors, to bring their music into the music model so we can do it in a licensed way. You can generate that music and have commercial rights, so you’re fully protected.

That was a hard process. It took us 18 months to figure out an agreement that worked. In the end, I think the main thing was adding a set of forcing functions, or forcing timelines, to find a trigger: This is when we do it, and we either do it together or we do it separately.

Those forcing functions really helped add urgency. We needed to move that forcing function a few times, but it still worked to a large extent to go after that. Finding the compromise wasn’t easy.

In our case, working with the labels was about protecting what they care about. They also care about how they continue doing well by their members and by the artists they work with. We spent a lot of time working with their members and speaking about how we think about the technology and what’s going to happen in the next couple of years. That really helped.

Just speaking through exactly what the technology does, showing the examples, and avoiding that initial knee-jerk reaction that AI is bad has been tremendous.

Tying back to the earlier question, as you’re navigating this landscape, how do you think about bringing in the right talent that can head and lead some of these functions? These are mostly unknown territories in terms of how to navigate them. Where have you been seeing success in bringing in the right people?

For spaces that are completely new to us—this, and legal is another example—we would always bring at least 1 or 2 people who had been in that space and had interacted with the same parties full-time in the past. Then we would augment that with a lot of consulting people who would help us in a specific conversation.

In this case, in music, we had music lawyers who worked very closely with us and consulted across a few of them. The good thing is that they know all the players, and they effectively served as a bridge between both sides. So we could speak the same language, and that was really helpful.

You’ve had a very specific taste for people who are risk-tolerant enough and also understand the commercial business opportunities to help guide the right chain of actions in each of those domains. I found that very fascinating.

100%. Legal—I don’t know how many of you are trying to find your first legal counsel, or have found a number of them. For us, this was one of the trickiest roles to hire for because you’re hiring into a space you know very little about.

The first couple of legal people were clearly not a fit, so we parted ways. Then we hired a third person who came from a number of Fortune 500 companies. They had never worked in the startup space or in venture. What resulted was that every conversation pointed out the risks we faced. Anything we wanted to do came with a list of risks that it could carry.

It was really tricky to work that way because you kind of get risk advice like, “Okay, this is where we should draw the line,” but everything was a bad decision. Now we’ve hired a person who previously worked as counsel at a number of companies—and don’t poach them. They’re amazing. [Laughter]

They understand the risk equation a lot better. They’re not only a counterpart in figuring out what the risks are; they can also say, “This is what other companies do. This is what we should potentially do.” They’re a true thought partner, and the change has been tremendous.

ElevenLabs started as more of a creator brand, from individual creators to creators who were building businesses. But now you've been having a lot of success moving into enterprise, not just with the AI agent platform, but even with the text-to-speech and speech-to-text models. How have you been navigating that transition? That's one of the very common places where a lot of really great consumer and creator brands fall down, but you've had a pretty smooth transition so far.

When we launched, we had a lot of early inbound. When we started with the classic PLG, we had a lot of inbound from enterprise, and I remember speaking with the a16z team when they joined us. Our initial take was, of course, that we wanted to be an engineering company. We didn't want salespeople; we wanted to reinvent that and have engineers do the sales.

We did hire 1 traditional salesperson and 1 nontraditional salesperson—an engineer—and we told them, “Do sales now.” That, as you can imagine, didn't work out in this specific case. But we learned our lesson, and we now invest in a combination of that. It's 80% sales and 20% engineering, so there's still a little bit of that.

This was a super-important lever for understanding who the customers are, what they care about, and working deeply with them to bring that back. That kind of working with them opened up what we actually needed to do on the product and research side.

Munjal Shah from Hippocratic AI is here. It was one of the earliest incredible use cases in the healthcare space, where they would create, effectively, agents that would take inbound calls to hospitals and schedule appointments. Beyond that, they would do all the other parts of outbound to patients—to remind them about taking medicine or to remind them about an appointment that was happening.

To be able to do that, you suddenly shift from using 1 foundational model to combining speech-to-text, the LLM, and text-to-speech and orchestrating them together. Then there are the integrations you need to build, and you actually need to deploy it. They were one of the earliest examples, in 2023, but we've seen this repeated pattern across a number of other customers in the customer experience space and many others.

We decided to invest more in helping with the entire orchestration. Instead of just doing text-to-speech, we can help combine our research to make this whole combination more fluid. But if you're thinking about enterprise, you need to build the combination of a knowledge base inside a system, and you need to help deploy that with telephony providers, Twilio, and SIP trunking. How do you do that in a templated and easier way?

The biggest gap, and the most common one, is that it's easy to do a demo, but how do you actually build it into production? How do you test, version-control, evaluate, and monitor it over time? How do you fine-tune it over time based on the results? All of that has been a big part of what we've done.

Underlying all of that, as we spoke a little bit with Matt before coming here, the foundation needs to be there: security, compliance, and serving the customers that will rely on that infrastructure. That's something we want to shine through at ElevenLabs. If you're using the software, it's always going to be reliable, and hopefully one day the 4 nines or 5 nines will be there, which is tricky in the AI space. But that's the goal.

The obvious difference between PLG and sales is that the cycle to work through and identify the right customers is much longer. I think that's where the differences within our internal team were interesting to observe. You had a lot of people who hadn't worked in an enterprise setting, and then you had another side of the company that had.

The side that hadn't worked in enterprise was very skeptical about going enterprise and waiting 6 or 12 months for results. In the early days, we needed to shield them from that information and say, “Trust us. We'll do this, and it will work.” They were very skeptical, and then, of course, after 12 months, it worked out. But that was probably the hardest cultural part: how you still keep everyone jumping on the same train.

That's exactly right. A lot of companies, at least from what I've observed, sort of slowed down after they started adopting more of an enterprise product—launching and building for customer requests. That started to delay product launches. Is that something you're seeing, or is there still a good balance where you want to be able to put out demos, POCs, and early teasers quickly, but at the same time deliver a very robust and reliable product?

There are 2 parts. The first part is that we have a difference in the team structure, and then we have a difference in the external product structure. On the external product structure, we want to ship very quickly. But of course, if you're shipping to enterprise, you want to make sure that it's stable and reliable, so we delineate very clearly what's alpha and what's not alpha. Then we go through that transition over that period.

As we work with the customers and our partners, they can decide whether they want access to alpha in the first place. When they do, it's clearly shown that this is an alpha product that might not be as stable, so they get a choice. I think that choice has been the most important lever: Do you want it or not? Some customers are incredible at doing that innovation, showing some of their work, or experimenting with that work.

Deutsche Telekom, with John here, is creating some incredible new podcast experiences. That came from testing early models of turning text into a NotebookLM-style podcast, with incredible voices that you can select—German-speaking voices, English-speaking voices—that sound good.

The second piece is the team structure, and that's something we didn't do until later, when we had more than 100 of us. We delineate inside the company between products that are pre-product-market fit and post-product-market fit.

With post-product-market-fit products, you're working for the long term. You test and evaluate a lot, and you only deploy when that's truly ready. With pre-product-market-fit products, your mission is to ship until you think you've hit product-market fit. Usually, we give it a 6-month period to prove itself out. If not, we kill the product, and we've killed products in the past this way.

That's the main important piece: Until we know there's a big potential user base, we'll continue iterating.

I've been able to observe some of those hard decisions in the moment, but they were the right decisions later on, to let go of some products. This is one of my favorite questions. My partner Martin Casado always says companies go through 3 phases: There's a product phase, a sales phase, and a scaling phase. Given that you've been through some of those phases, what has been the hardest transition for you as a CEO?

There have been a lot of transitions. Of course, I have my co-founder next to me through each of those. I've known him for 15 years; he's been my best friend since high school. I've been very lucky to have that combination. And, of course, you, Jennifer, and all the partners have helped us through those transitions, which has been incredible.

I think the recent realization was that we're now a 350-person company. Of course, that means our go-to-market team and the incentive structure around it have evolved pretty strongly. What wasn't clear to me, and in hindsight is obvious, is that in the early days everybody would operate on a passion basis. They would operate based on what they thought was best for the company.

As our go-to-market team grew, we realized that the incentive structure really matters if you're building that machine. That transition—from having a lot of the people who are helping create that machine become part of that machine—means those incentive structures will eventually drive behaviors that might be slightly different from what you had in mind if you don't make it extremely clear.

In some ways, quota and commissions are effectively a lagging indicator of strategy, and strategy is a kind of leading indicator of what will happen in the future. You need to find a way to resolve those 2 together. You want to make sure the quota and commissions and the strategy that you want to drive are closer together, and that the disparity is as small as possible.

For me, the biggest realization was that we're becoming a bigger company, and there are clear behaviors that happen based on commissions. To actually resolve those, we need to be very upfront about making it explicit that sometimes, even if commissions are incentivizing something and you think it's the wrong thing, you should come back to us, let's speak about it, and just course-correct.

Now we're explicit with all our sales teams that if they're seeing a deal that might be competitive in nature, and our pricing table would suggest that they can go very low and earn a higher commission, but they think it's wrong, it's better to come to us.

We’re happy to still grant commission, but kill the deal and go. We had this case recently where one of our foundation model competitors came to us wanting to license our models for demos. Of course, the incentive would suggest that you should sell to them, but luckily we didn’t.

Yeah. You granted commission, though.

Yeah, in the early days, you can definitely do that.

And we’ve adjusted that. Now it’s in the policy, so you cannot sell to the foundation model companies.

So it’s clear to everyone internally.

That was incredible, Mati. Thank you so much for sharing all the lessons and learnings with us. Let’s give a round of applause to Mati.

Thank you.