George Sivulka, Co-Founder & CEO @Hebbia: The Future of Foundation Models | E1250

You can bucket great founders into 3 backgrounds. The most common is that you had a messed-up childhood. The second most common would be that you're gay, and the third most common would be that you were adopted.

Look at a list of the all-time greats: Elon Musk had a messed-up childhood; Jeff Bezos and Steve Jobs were adopted; Peter Thiel and Sam Altman are publicly gay. All of these early-life experiences end up giving you some desire, some deeper passion, to go out and prove yourself.

George, I am so excited for this. I've been really looking forward to this one. I spoke to Kevin Hartz, Sangin[?], and Corey; I found out all the shit there is to know. Thank you for joining us.

It sounds like you did a lot of research, so thank you for diving deep. I'm really excited to meet you as well.

As a venture capitalist, it's amazing the amount of free time you have. This is going to be a show. Talk to me about your childhood. I spoke to Sangin, and he said this was a really interesting part of getting to know you, so talk to me about your childhood. I'm leaving that deliberately open for you.

It is fair. The first time I met with Sangin, who's one of our Series B investors, it was a 30-minute lunch that turned into almost 2 hours of us talking in depth about the dynamics that I think made me have a chip on my shoulder.

In short, I was born in Staten Island, New York City, which means you already have a chip on your shoulder from that. I grew up around New York City and in New Jersey primarily, which is a second chip. My mom is probably like a Mafia child, born and raised in Staten Island, and my dad is an immigrant from Slovakia who grew up under the Iron Curtain and then immigrated—really escaped—to the United States.

They both fully intended to be professional athletes, and they had 4 children, of which only 1 was a boy. You can imagine their dismay when I was chasing butterflies on the soccer pitch or falling on my head many times. I have plenty of stories of literally falling over while trying to dribble a basketball.

My whole childhood, I was really just a math kid. I wasn't very out there, wasn't really talkative, and was only really good at math. My parents barely even knew what Stanford was, so growing up, you have this whole misalignment between who I was and who I wanted to be, and who they wanted me to be.

I think that gave me this drive, desire, and passion to go out and prove myself in a way that was really tangible—maybe not only to them, but hopefully to my own kids one day.

Did you have friends?

I was very popular, thank you very much, Harry.

I had a lot of friends who were incredibly nerdy. We went to a public school. I was the type of kid who would hack the school tablets to put StarCraft on everyone's computer, and then we'd all not be paying attention in public school, playing StarCraft.

1. Three Traits The Best Founders All Share?

There was a large enough contingent of kids who were also not athletes that there was some involvement there, and I think they also had a strong effect on me.

Before we were chatting, you said there are 3 archetypes of successful founders that you found as a trend. Can you talk to me about the different profiles?

I'm happy to. I always joke around and say that you can bucket great founders into 3 backgrounds. I think probably the most common is that you had a messed-up childhood. The second most common would be that you're gay, and the third most common would be that you were adopted.

If you look at a list of the all-time greats, Elon Musk had a messed-up childhood; Jeff Bezos and Steve Jobs were adopted; Peter Thiel and Sam Altman are publicly gay. I think all of these early-life experiences end up giving you some desire, some deeper passion, to go out and prove yourself.

I actually very much agree with you. I always felt like a disappointment. My brother was always incredibly talented, good-looking, tall, and smart, and I was pretty average. I was fat, and my dad didn't really hang out with me; he hung out with my brother. I always just felt like a disappointment.

What a mistake that was, Papa. What does your brother do now?

He works for me upstairs. That's what I'm talking about.

Did you feel like a disappointment?

I think the answer is yes. I felt physically unable to do the things that I wanted to do, or I thought that I was good at things that weren't valued or weren't as important.

All of my sisters are amazing athletes. They're all about 6 feet tall, and they're incredible athletes. I was just not. I was kind of the ugly duckling in many ways.

2. How Cold Calling NASA Changed My Life

I heard that you built lasers and cold-called NASA. Can you talk to me about these very strange but cool early influences in your life and how they shaped you?

Those are completely separate stories, but I think—

How do you cold-call NASA?

The story is actually very good. I wanted to be an astronaut. That was my number-one goal, and I was hell-bent on that.

By the time I was around 15 years old, I was going to high school in New York City, an all-scholarship school where the alumni paid for everything. I was tracking academically really strongly, and I wanted a NASA internship. They were offering them to college undergrads or graduate students, so obviously I applied and got rejected 5 times.

Then there was a snow day in February. My school was closed, but I commuted into the city. I showed up in front of NASA's New York City office, the NASA Goddard Institute for Space Studies, and demanded that they let me in.

The front-door security guard was like, “Kid, get the heck out. What are you doing? You don't have an appointment.” I was like, “I printed my résumé on the nicest paper. I'm wearing a suit. You've got to let me up.” He kicked me to the curb.

I sat outside at around 110th Street in Manhattan. It was snowing, and it was so cold. I didn't know what to do. I started crying and called my mother because I was going to come home.

She's a salesperson who works in medical sales. She picked up the phone and said, “Listen, no, you're not going anywhere. You sit your ass down and call every single number that you can get into the building.”

I sat on the curb and cold-called every number on Google from my old phone. Finally, someone picked up. It was one of the only people in the office that day. They came down, met me in the lobby, and I pitched them on myself for 2 hours.

They gave me an interview. I botched the interview because I didn't know anything about linear algebra and didn't know anything about physics. But I remembered and memorized all of the titles of the posters on this professor's wall.

I came back the next day, showed up again cold, and told him basically everything I could possibly know about his specific research. He was impressed enough to let me work for him for free. They paid me the next year, and then I published internationally recognized research the following year.

By that time, I think that was impressive enough to let Stanford let me in, which was a life-changing moment for me.

That is incredible. It is also an incredibly heart-wrenching moment, thinking of a little boy on a street, crying.

You and I are both young. We've been taught that you win through persistence and going for it. When is that true, and when is it not?

I have an unhealthy obsession with driving really hard. I think you can never give up. I just don't think that's an option.

You can look at every company ever. Some get to $100 million in revenue in whatever span of time—which their marketing team has probably hacked—and some end up taking really long periods of time. The only thing that actually changes is the rate at which you get there.

Sometimes things go in your favor, and sometimes they don't. But if you're so persistent that you just continue, you can bring a lemonade stand to $100 million in ARR. There's nothing that's actually stopping you. You can brute-force your way as a founder. Screw product-market fit—you could literally brute-force anything in the world.

3. From Stealing Food From Stanford to Pitching Peter Thiel

You just have to have that chip. You have to continue to pound away at whatever is in your way.

Stanford was a big one for you. I imagine it was a really big personal validation to get in, correct?

Yes.

How did it feel when you got in?

On to the next one. I was like, “Okay, that's done.” The next day I was like, “Okay, how do I become the youngest PhD student in my school's history?”

It wasn't even a moment. I was excited for a moment, but it faded very quickly.

I spoke to Corey before the show, someone who's known you since you were 18, probably even earlier. Take me to the founding of Hebbia. You were doing incredibly well at Stanford; you were the wonder child. How does Hebbia come to be in that situation?

I was one of the youngest PhD students in the history of my school. At the time, one of the areas of research that was most interesting to me was meta-learning: this idea of teaching machines to learn to learn.

In June 2020, OpenAI released GPT-3. If you remember the title of that paper, it was “Large language models are multitask or meta learners.” I was sitting in my lab one day, playing around with this new technology, and I thought, “Wow, they just stole the most important thing I could work on from under my hands.”

I said, “If I can't build the most important technology, how can I build the most important product?” Those are 2 very separate things. Obviously, at the time, GPT-3 was not a product, and I don't even think ChatGPT is a really good product. It's like a calculator: it has the technology encapsulated in a very simple form, but it's not a product like Excel that lets you build whatever you'd like with it. That's very human-first.

Stanford always pounds into your head the idea that you've got to start a company where there's a lot of pain. A lot of my students or friends would go into investment banking or private equity if they were really lucky, and they would come back as the least happy versions of themselves. They'd lost 50 pounds, hated their lives, and were miserable.

It seemed like there was more pain in financial services around processing unstructured data than anything I'd ever seen. I thought, “There's a great company to be had here. Let's give it a shot.”

We're sitting in that lab, saying, “There's a great company to be had here. Let's give it a shot.” What now?

I heard—and I saw pictures of this wonderful bedroom—that this was Four Seasons finest. You made me feel like such a diva when I saw that room. You were unable to make $300 in rent and were sneaking into Stanford dining halls for meals when you weren't studying there.

Yes.

George, I have no comment. Off the record, you raised 2 rounds of financing with clothes hanging behind you on Zoom. Take me to the next step after deciding to do this in the lab.

I was on a PhD salary. You were making $38,000 a year, or $42,000 if you had the Stanford Graduate Fellowship, which I had. Big dollars.

I said I was going to go on leave. I originally went on leave and told my adviser, “I'll be back in a year. This coronavirus situation—just give me some time.” I didn't have anywhere to go. There wasn't a logical next step, and I wanted to work on this company.

I asked my friends, who were renting out a house in East Palo Alto, to let me rent the cheapest room they could possibly find. They were fully booked, and it was over $1,000 in rent. I think it was actually $500 or $600, not $300, so I'll give my broke self some credit for not being able to afford the rent.

They said I could rent out the master-bedroom closet. I brought in a mattress from the dorms and had a folding table from a nearby Home Depot. I would rotate whether the mattress or the folding table was on the floor.

I sat there and worked all day, 16 to 18 hours a day. I'd go to sleep, wake up, and do it again. No weekends. I turned into almost a monk, obsessively building Hebbia.

I was training models at the time, so I'd wake up in the middle of the night to check on them and continue to use my GPU credits because I didn't want to spend any money.

Is there a period where more work is not effective? When I think about working 16 to 18 hours a day in that environment, I wouldn't function well. I need fresh air and exercise.

I'm masochistic to the extreme, to the point where it's unhealthy—an alcoholic, bipolar, tortured child. I'm like Lindsay Lohan's “Adventures in Babysitting.”

Ultimately, I probably went too hard. This was 2020, and I definitely left nothing on the table, to a point where it was detrimental to my health. At the same time, that was a crucible that helped form me.

It's very hard to be a founder. Those are the moments when you're eating microwave meals every day, losing weight, and trying to will something into existence.

I was trying to pitch one of my former bosses at a professional-services firm. He looked at me on the Zoom call and almost cried. He said, “You need to just come work here. What are you doing to yourself? Come back. We'll give you a proper salary. You don't have to do this.”

There were so many low points like that. I just kept chewing through it.

You raised money in the closet?

I got on with it. We raised a pre-seed from Peter Thiel and Floodgate, and then our seed from Mike Volpi at Index.

Mike said, “You have to—we're going to do a partner call, just as a formality, with a few partners to close.”

What round is this?

This was for the seed, a follow-on to the pre-seed in November 2020.

I got on the Zoom call. I literally had clothes hanging behind me. All of a sudden, Mike showed up, then 4 other partners, then 80 partners. The Zoom screen tessellated with hundreds of faces.

I was horrified. Mike was like, “Look, he's living in a closet,” and everyone was like, “Great founder.”

4. Lessons working with Peter Thiel

I was so embarrassed, and then I pitched my company. Honestly, it was hilarious.

What was the story of driving to Peter Thiel's house?

Two months prior, I was just about to leave Stanford. One of my best friends in the world had interned at Founders Fund. He said, “I hear you're raising financing. You should talk to Peter.”

I wasn't going to say no to that. He introduced me on an email thread with Peter, and I said, “Peter, I'd love to do a lunch or dinner anytime soon.” I'm not a morning person, so I asked for lunch or dinner.

Peter said he could do breakfast. I said, “I really want to do lunch or dinner. Can we do brunch?” He said, “I'm going to do a breakfast.” I said that was fine, and he gave me a slot on a Saturday.

I got in my car. It was an old, beat-up 2006 Audi convertible that I had fixed up from Craigslist and bought for $4,000. At 3:00 in the morning, I drank a bunch of coffee—18 cups of coffee and a 5-Hour Energy, all the disgusting stuff—and drove from 3:00 to 8:00 to his house to pitch this guy.

He showed up 45 minutes to an hour late. He was just waking up, and I was wired, sitting in my chair ready to go. It was supposed to be a 30- or 45-minute breakfast, so I thought it was kind of already shot.

We ended up talking for 4 or 5 hours about the company and all the flaws in my business model, but also about math, deep esoteric philosophy, and the world.

He said, “I'm not investing,” because it was coronavirus and there were a variety of other factors, “but I'd love to put in a check.” I left the conversation feeling like I'd made a friend or had been seen by someone incredibly bright.

I felt like I had been inducted into the Illuminati. My whole Audi drop-top was in the sunshine, I was playing Kanye West, and I drove out. It was my first offer from a venture investor.

How much did he invest?

I don't even recall. The total round was about $1 million, so it was nothing.

What makes Peter so incredible?

There are 2 things. He is incredibly ontologically smart. He can build this worldview or perspective where he knows how to pattern-match to a variety of other things.

He's also phenomenologically smart, which is the idea that he understands processes and how humans behave really well. He's always thinking, “If I'm looking at something that's about to unfold, could I have predicted this ahead of time?” He always asks himself that question.

He's built up a rich perspective of the fallacies that human society has, and the mimetic behavior that people copy each other with.

We have money from Peter and the Peter stamp of approval. Does that open every door in the Valley?

We were in late discussions with a lot of investors, and then everyone else was like, “Let's pile on in here.” It's some of the best money that you can get, and it was a game changer for us.

Then we closed with Maples and Floodgate?

Yes. It was at Floodgate, with Mike Maples and Thiel, and then we got back to work. We had about $1 million.

Two months later, Mike heard about Hebbia from his daughter, who was a Stanford student. I think she had seen the product and was friends with us. Mike came in and said, “This is completely different from Elastic or all these other search technologies that I've seen and invested in.”

He's on the board of Elastic today, so he said, “Let's add some fuel to the fire.”

How much did he invest?

I think he invested an additional $2 million or $2.5 million at the time.

Where did the $130 million come from?

That was years later. This was all in 2020. We ended up building Product Studio, which was the first to productionize retrieval-augmented generation. Also in 2020, we built the first semantic-search engine.

Can you just help us understand what RAG is?

RAG is an acronym that stands for retrieval-augmented generation. If you look at large language models today, they're really good at thinking if you give them the right context. But they hardly ever have the right context, and RAG was the first real attempt to give them the data to answer questions correctly.

We were building on RAG to start. We were one of the first people to productionize the idea of putting a search engine behind an LLM. You would ask a question, and instead of it just replying from its memory, it would go and do a search and then reply with that context.

In an enterprise, where you have a lot of offline data, we were really the first people to hook up that offline data to large language models to answer questions.

You're one of the first, and we're seeing that in action now. It's working. That's a bit of a plot twist.

We actually don't think RAG works at all. It's one of the most-used AI architectures in the world, pioneered at Hebbia in a very meaningful way. I think every enterprise is experimenting with it, but it has a lot of different failure modes.

A lot of the time, the questions people ask these systems aren't explicitly in the data. They're about the data. For example, if you're asking an AI system, “Is this company a good investment?”—which is a very common thing that people ask Hebbia over marketing materials—it's never going to say, “This company is a great investment,” unless that's something the CEO says in a pitch deck or a recording.

What you actually want from that system isn't to search the data. You want an answer about the data. You want to know the customer concentration, the strength of the management team, or whatever criteria are fundamental to your specific investing process.

That's a process. It's not explicitly stated. The marketing materials are often a load of crap, and you have to distill what's true out of them.

That's what Hebbia does. It's not finding something that already exists; it's taking all the things that already exist and starting to answer questions about that information.

Take me to that transition. You were building on RAG, saying, “Great, we're going to productionize this,” and then you moved off it and realized that it was shit and wasn't as good.

We deployed it at some of the largest finance firms in the world. We went from $0 to $1 million in revenue sometime in 2021 or 2022, and we raised a Series A from Index and Mike Volpi—$30 million.

We started to see that all these customers now knew what ChatGPT and LLMs were. Hebbia had this mature enterprise product in the market, and we were by far the first to get there.

We looked at all the queries people were asking. The questions weren't, “Find me the quote,” or, “Find me the Command-F answer.” They were more like, “Read all the documents and tell me all the times they mention AI,” or, “What is our exposure to Silicon Valley Bank during the regional banking crisis?”

5. The Future of AI and Business Applications

Almost 90% of the questions people were asking these systems weren't answerable by searching through the documents. They required work to be done on top of the documents and then an answer to my question.

What happens to the rest of the landscape if you're saying RAG isn't the right approach and they all love RAG? It couldn't be hotter right now.

I don't think they're loving RAG.

You don't think they are? What makes you say that?

I think 90% of enterprise AI right now is vaporware. People say, “We swear it works. Look at this amazing demo where we ask what the CEO says about the investment.” The minute they try to use it in a real-world example, it completely fails.

I think the majority of AI usage—and a lot of these usage statistics—are fugazi. One of my favorite phrases is “fugazi.”

One of the things that Hebbia tries to put forth in the market is: change will take time, but we have a system that's starting to drive real, measurable value over very specific, defined use cases. Our tagline is always, “Stop experimenting with AI.” Everyone is experimenting and excited about it. Start driving value and getting value out of it.

To what extent is this RPA versus agentic AI?

I'm not a big believer in RPA. I think RPA is almost not an AI application in the new sense of AI. It's AI in the old, 10-years-ago sense of AI.

RPA is effectively very simple computation. Some of the things people ask Hebbia about are 800-page credit agreements, 230-page confidential-information memorandums, and marketing materials.

They're not asking for things like copying numbers. They're saying, “Tell me what the inconsistencies are in this document,” or, “Tell me where there's an event of default that we can trigger.”

There's an open-endedness and a new level of computation that people can do.

Daniel Dines, who we had on the show—it comes out on Wednesday—said it very well. He said, “RPA is low-skilled, low-level cognitive processes, and agents are high-skilled, ambiguous, great decisions.”

That's a nice phrasing. It's incredibly clear. We are very much capturing the agent—the high-level, ambiguous decision-making—and trying to trace it all the way down to individual citations or individual characters that led the model to that decision.

I thought about SaaS's statement the other day: the notion that business apps that exist today will all just collapse into agents. Do you agree with that? Will apps just be the predecessor to agents?

I don't agree with that at all.

You think he's completely wrong?

It depends on how you define a business app. If the new business apps are platforms, you'll see those platforms really take hold. Hebbia is a platform. It lets you build whatever agent you'd like.

Here's a bit of a mindfuck. When building Hebbia, or building all these foundational primitives for how people use AI over the last 4½ years, we've always asked ourselves, “What are the apps that AGI would want to use?” Or, “What are the apps that agents would want to use themselves?”

What are the tools we could build that would assist LLMs or the really smart foundation models of the future to get to an answer more quickly?

It's interesting. Hebbia Matrix orchestrates lots of smaller LLM calls. It's scaling at inference. It's running massive amounts of compute at the orchestration layer, and we think an AGI system would prefer to use Hebbia Matrix to diligence a company or look through thousands of documents rather than read them all by hand in a really long context window.

Is the future of business apps actually business platforms? We've got platforms, agents, or apps. What is the future?

I ultimately think it will be a mix of all 3. History doesn't repeat itself, but it often rhymes.

60 years ago—or even longer—the foundational unit of compute, doing a calculation on a computer, was introduced to the enterprise. There were plenty of people tallying things or bookkeeping in actual books, and their jobs changed.

There were apps for bookkeeping, and then there were platforms like Excel that let people build better bookkeeping apps. Excel was then unbundled again into better and better bookkeeping apps.

There's opportunity not only in verticals, but also in the entire industry: building platforms, building cooperatives, and even building new types of quote-unquote agent employees.

I think that opportunity is the exact same size. If $100 trillion of value was created in the stock market from the introduction of the computer, or the fundamental unit of compute, I think $100 trillion of value will be created in the next 60 years from the introduction of inference, or AI.

Will that be additional value, or will it be value that denigrates from the existing value of alternatives?

I believe it will be additional value. Maybe I'm too techno-optimistic, but I think the S&P 500 is completely undervalued by trillions of dollars of additional value.

I can't quite get my head around that, to be blunt. How does adding $100 trillion of value even exist? Is that just because we will see GDP and productivity grow so much that it takes place?

I genuinely believe that more than 50% of GDP will be contributed by what you can call agentic applications in the next few decades.

I think it will happen faster than the next few decades.

Do you?

Daniel Dines said we consistently underestimate how long it takes enterprises to adopt new technologies, get comfortable with data security, and get comfortable with processes. Is that right, or are we past a tipping point?

It's a good point. If you're cutting costs, which I think Daniel Dines and UiPath are one of the best examples of—using AI to make companies more efficient—then maybe that's true.

But look at finance and how quickly Excel went to 90% market penetration in finance. From 1985 to 1986—literally 18 to 24 months—Excel took over all of finance. Everyone switched from using a calculator, the HP-12C, to using Excel.

If you look at how quickly finance started using credit-card data to value public companies ahead of their earnings, that was also a 2-year period. More recently, finance is the slowest-moving, most lethargic industry. It's the worst possible customer base to go after unless you're providing outsized alpha or real value.

In that case, the minute there's something real, finance moves faster than any other industry. I'm making a bet by going into finance and trying to get to my own—

What I worry about is losing the education process. A lot of GPs or managing partners in a senior firm have been through the shit of analyzing companies, staying late, understanding what makes a great business, and all of those things.

Then we'll say, “Don't worry about that shit. We do it.” We have no graduation pathway for the next generation, so we have decision-makers who don't have that graduation.

I'm less worried about that. Maybe it's naïve, but one of the best things about having years of experience is having depth of knowledge about investing.

If I'm a junior trying to price an asset, I haven't seen that many other companies that look like this company. I might say, “I think it should be priced at X, Y, or Z.” Then someone in the investment-committee meeting will say, “No, I've seen 20 other companies in my 40-year career that look exactly like this, and they all went nowhere.”

That person is leaning on prior experience. With Hebbia, juniors who are really smart can say, “You might have remembered 20 deals, but I'm looking through every deal in our company's history in a giant Matrix. I can tell you quantitatively that when a company is performing here, it's in the 90th percentile across all these investing criteria. We should pay a 90% premium to market.”

6. Debunking the Myths of AI Job Displacement

I'm using more deals than that person has ever seen because I know their name is on this many investment-committee memos. That kind of structured thinking and additional information can make juniors better investors. I don't think it takes away from the experience.

Do you really think it will be a tool for usage, not a tool for replacement?

I genuinely believe it makes humans better. In 5 years, I think it will change the way people work, but I genuinely believe it will increase the AUM of the firms that use it. I think it will drive more employment.

There will be some jobs that change. There are no more bookkeepers who do tabulations in spreadsheets on 2 sheets of paper. What changes are the cognitive tasks that are lower in cognition—more back-office and middle-office tasks, and maybe some of the more junior front-office tasks.

Those tasks will start to move toward, “How can we manage AI juniors?” rather than doing everything ourselves by hand. But just as Excel didn't take away jobs from people, it changed people's jobs and required them to learn Excel. The exact same thing will happen with AI.

You don't think we'll see team sizes reduce as a result of agent integration into the enterprise?

There are all these stories, like [likely Klarna] positioning to investors that it's firing half its staff.

I think it's bullshit.

There might be some reality to it, but it's an amazing marketing story. Anytime I hear something put out as a marketing story, I almost negate it in my head and think about the actual implications.

When you're saying something and screaming it from the rooftops, it almost always means that internally you're freaking out about something. I look at that loud behavior, and I think the behavior itself negates the content.

That's my position on this sort of stuff.

How do you feel about competition? There are several players now in the Hebbia slipstream.

If $100 trillion of economic value will be created by AI and agentic applications, there will be so much room and opportunity for many different players.

When Excel came out, Marc Benioff released Salesforce, and people created TurboTax and all these unbundlings of Excel. That didn't make Excel any less valuable. I think it made Excel more valuable.

I view Hebbia as a platform that will get better the more people are inspired by it and build increasingly verticalized applications.

What models do you use? You sit on top of what?

We are completely model-agnostic. We use all of the major model providers and some of our own models.

The foundational difference that Hebbia is capitalizing on right now is a fundamentally new and important idea. On the order of creating RAG, agents, and decomposition, there's this idea that we pioneered over the last year or so: scaling at inference.

Talk to me about this.

OpenAI is starting to do this with o1, where they'll have a model recursively think about a question over and over again before producing an answer. Instead of training a larger model, they're using a similarly sized model and telling it to run multiple cycles—to compute more—before answering.

Hebbia pioneered something different. About 18 months ago, we said, “We can't wait for these models to catch up. For a simple, single question, let's run hundreds or even thousands of submodels of the best models in the world to compute over every document and answer the same question.”

If you can't train larger and larger models fast enough, you can take whatever is state-of-the-art or cutting-edge and run it more times to get more compute and more computational power, resulting in better decision-making for the same user.

This is an idea we pioneered. It doesn't matter if you're using Claude 3.5 or o1 itself. Scaling at inference at the orchestration layer was something that had previously been scaled at inference with the training layer, but you get much better results. It's a way to drive toward more accuracy.

Which model provides the best results? Des Traynor from Intercom recently spoke about the movement away from OpenAI to Anthropic.

For certain types of documents, like dense legal documents or even more colloquial documents, Anthropic works better. For other types of documents, o1 or GPT-4o works better.

7. The Future of Models: Many specialised or few generalised?

It's always a trade-off between accuracy, speed, and all kinds of things. A lot of the time, when we're decomposing a task, we'll use a mix of OpenAI, Anthropic, and even Gemini.

Do you think we live in a world moving forward with many models specialized in different things—as you said, some do legal and some do other things—or with generalist monolith models that own the whole stack?

There's a story that makes me think of Bloomberg, which has the best financial-services training set of all time. They trained a GPT-3.5-class model called BloombergGPT and released an arXiv paper. Everyone on LinkedIn was saying, “Wow, Bloomberg is cutting-edge, and they're going to steal finance AI.”

Why did they not? They had the best data in finance. Then GPT-4 was released a few weeks later—it might not be the exact timeline—and it destroyed BloombergGPT at every finance task.

You saw the idea of post-training, refined verticalized model creation, continually losing to scaling laws. Maybe we're at the end of the scaling laws of training, but Hebbia, OpenAI, and a variety of other companies are pioneering the idea of scaling laws in inference.

8. The Impact of Scaling Laws on Foundation Models

I think nothing that other players can do to fine-tune models will ever catch up.

I need to break that down. Everyone is asking, “Are we at the end of scaling laws?” Reid Hoffman and Daniel Dines are saying yes. The upper end of LLMs. Reid Hoffman is saying no, there's so much more room to run.

Can you break down the difference between scaling laws at inference and scaling laws in training?

It's a bit of a marketing distinction, but ultimately, the way we got here over the last 5 to 7 years of training models was to build a bigger and bigger model, give it more and more data—more and more clean data—and then perhaps do some RLHF or reinforcement learning to fine-tune it after pre-training.

That worked well to get us here, but we're running up against the amount of good data that exists in the world.

Are we? People push back on this and say there is so much data we haven't used yet, whether it's video data that can't be translated or synthetic data. We're not at all exhausted in terms of data supply.

I think we're starting to run up against the constraints of it. That's a gut feeling. I'm not looking at data collection, and I'm not particularly in data collection myself, but I think we're starting to run up against the limits of really good data that we can use.

What's the problem, then?

Ultimately, that might mean we're training larger and larger models. xAI just created the largest GPU cluster of all time, and they're going to try to train larger and larger models.

Regardless of how the scaling laws for training larger models—for parameter count, accuracy, or performance—play out, I believe you could still get better compute not by building a larger engine, to use a metaphor, but by putting a bunch of smaller engines together.

Hebbia, by orchestrating large amounts of inference to answer a single question, is building something like a Tesla made of a bunch of smaller electric motors that make a lot of torque in a really amazing larger engine.

Doesn't that make it incredibly capital-inefficient?

The one thing people in my position will always tell you is that the cost of intelligence will go to zero.

Since Hebbia started, the cost of inference over a fixed number of parameters has decreased by about 7 orders of magnitude in 4 years. I genuinely believe scaling compute is a no-brainer.

Yes, we run more large-language-model calls than anyone might say is necessary, but we have the best accuracy in the business. We can answer much more complex problems, and we're driving real value for enterprises.

Every quarter, our margin improves.

You're not spending money fast enough?

Exactly.

You mentioned xAI's GPU cluster. What they've been able to do in such a short amount of time is miraculous. What do you think that tells us about the model layer itself?

I think the model layer—and this isn't a hot take anymore; I've been saying it for a few years—will become commoditized.

A lot of value will accrue at the hardware layer, especially. We could talk about what that means for NVIDIA. NVIDIA has a stranglehold on training, but not as much of a stranglehold on inference.

You might see other chipmakers start to have their chips used in a more meaningful way. CUDA is what all machine-learning scientists were trained on in their PhDs, but for inference, it doesn't matter as much what you're using.

I think it will be the infrastructure layer and the application or agent layer that accrue the most value.

Why doesn't it follow the same vein as cloud? Cloud is commoditized. Google Cloud and AWS are completely commoditized, but it's a great business for them.

It might. There are probably fewer and more entrenched players in cloud. Ultimately, those players have an OPEC-like oligopoly, where they can control pricing.

Cloud could end up following the same path, but I think cloud is more complex than training larger and larger models. Cloud providers are basically using models as a loss leader to build stronger moats in their cloud businesses.

You see this with Anthropic and Amazon, and with Microsoft and OpenAI.

Absolutely. Whoever has the best models will continue to attract the right amount of investment.

The difference with cloud is that the cost of switching is much higher. I can switch models readily. There may even be an entire industry built around switching models from OpenAI to Anthropic when OpenAI goes down.

Switching clouds, for any substantially sized startup, can be a $10 million to $20 million investment. It's almost never worth it. Cloud is much, much, much stickier.

With models, it's a very simple API key. It's very simple to switch models, and I think that's an important differentiator.

OpenAI is at 160, Anthropic at 40, and xAI at 50. Which one did you buy?

I think xAI is the most undervalued company. This is a really spicy take, but I think xAI might overtake OpenAI and Anthropic in value over the next 12 to 24 months.

That's crazy, but you think they're all undervalued?

I think they're all undervalued. I genuinely believe all AI companies and the S&P 500 are undervalued.

Elon is very well-positioned geopolitically. He can run a more efficient business without as much administrative bloat or friction from employees.

9. The Geopolitical Influence on AI

How important is geopolitics in winning this?

Geopolitics is very important. Governments will be some of the largest users of AI, especially with what the new administration in the United States has been talking about regarding increasing government efficiency.

Energy is a very big bottleneck. It's common in Silicon Valley to talk about needing nuclear reactors to flatten the duck curve so that we can continue driving toward larger and larger data centers.

Those are ultimately geopolitical resources. All of these things are important.

Elon is also operationally talented. If this becomes commoditized, whoever can operationalize model creation and serve models the fastest might start to win.

You think xAI is going to win, and you would invest in them?

I would, if I had the opportunity. Ultimately, though, I think all of them are undervalued.

10. The Commoditization of AI Models

I genuinely believe all AI companies—and the S&P 500—are undervalued. If we're about to create $100 trillion of value, I think this is a real, tangible technological shift: a massive unlock on the order of what computing did for the entire economy over the last 60 to 80 years.

I think AI will do that for the next 60 to 80 years. All these companies are massively undervalued, including non-AI companies.

Unpack the last bit, including the non-AI companies.

Computers made legacy businesses better if you used them correctly. AI is a massive disruptive force, but if you can ride the wave of change, AI agents and this new fundamental paradigm are a massive unlock.

Do you think there's a difference here? People talk about different technological transitions, whether it's the agricultural transition and the dependence on human labor, movement, and machinery, or computers and workforces. Those were at least 10-year transition periods.

This is: “We use AI tools now because we bought them today.” The transition period is instant. Doesn't that change enterprise-value accumulation and whether these technologies are good or bad for businesses? Your business could die instantly if you don't have them.

I like technological revolutions. You have fire, and then someone invents the torch many years later. You invent the engine, and then someone invents the car. You invent the wheel, and then someone invents the chariot.

Encapsulating and building a useful product on top of a technological change takes more time. Hebbia has built that product for AI. If Excel was the product for compute, I think Hebbia has built that product for AI.

When you have a good product, the transition will be very quick. Right now, we have chatbots and surface-level search engines that give you surface-level value. They might help your kid cheat on their homework, but determining whether something is a good investment is a much richer problem.

Is chat the right interface for many of these applications?

I don't think so. Chat was always a useful feature and interface, but it's like a single cell in Excel. It's like asking whether the TI-84 was the right interface for computers, or whether the terminal was the right interface.

We haven't even started to explore the opportunities for interfaces. Hebbia is the Bell Labs of defining AI interfaces. I think of ourselves as the Bell Labs of defining AI interfaces.

RAG was one of them: the idea that you could find things in data really quickly. Decomposition and agents are another. Scaling at inference with our Matrix product is another.

You can look at many other things, such as agents controlling 4 screens at once, where you're watching someone use a computer, or computer use, where AI models are moving cursors. I think almost all of them have potential.

If agents are efficient, does the interface become irrelevant?

The better agents are, and the more work they do, the more important it will be that they're easily understood by humans.

Let's say we have 10,000 employees or 10,000 AI agents dropped into a company. They're all experts at doing something. That becomes less a problem of giving them the right tasks and more a management problem.

There's an entire infrastructure and orchestration layer—the thing I always come back to—of making these things work together. That's going to be a challenge, and it will require a very human-first product. That's what we're trying to build.

Do you think Elon will be successful with DOGE?

I think it will be his greatest challenge. There are many self-reinforcing and self-protecting mechanisms in the largest organization in the world, which is the U.S. government by spend and headcount.

It's a massive, unruly organization. It's not going to be as simple as Twitter.

Are you more excited post-Trump?

The thing I care most about in the world is that, as an industry, we have clear guardrails that we can follow and understand. I want to build the best possible tools, get them out into the economy, and make sure that everyone transitions in the best possible way.

Does your business not thrive on a better financial system? We're seeing a financial system in the United States that, from afar, would seem to be thriving objectively. It would appear that Trump is good for business.

I won't make a comment here.

I went viral before the election because I said, “It's so interesting: 99% of CEOs come on the show, and they either shut up or say they vote for Kamala.” Then it ends, and they're like, “By the way, I'm so Trump. I am so Trump.”

It's totally cool, but it's fascinating.

For sure. I totally understand why they're not answering.

You are not alone. It's okay.

You mentioned NVIDIA before. There's a big question around its ability to sustain its monopoly. You've seen Google, Meta, and Amazon all wanting to move into the chip space. How do you think about NVIDIA's ability to sustain its unwavering monopoly so far?

The best moats aren't technological moats or data moats. They're people moats. People and networks have the most friction to change.

One of the things NVIDIA does best is that it made an early bet on machine learning and created CUDA, which is how almost everyone learns to train models. They learn how to interface with NVIDIA chips for training.

As you start to see the shift from training to inference as a fundamental macro shift in how people deploy AI, I think that will slightly destabilize the dominance of NVIDIA chips.

You can start to use AMD chips or custom architectures, which all the major model providers are exploring, to do inference. You have academics and researchers training large models on NVIDIA chips, but the minute they deploy them, they can deploy them on cheaper infrastructure.

I think that will be a big change. I'm still bullish on NVIDIA, but I'm even more bullish on other chipmakers and custom ASICs for inference because I think there will be a larger shift toward inference.

Do those other chipmakers exist among the incumbents—Google, Meta, Amazon, and so on—or is this a new generation, Cerebras-style?

It will probably be large technology providers and AMD. I don't know about Intel, but I would probably bet on them. There's definitely an opportunity in the market, but chips are hard.

Before we do a quick-fire round, do you want to resurface back up to the agent layer?

Sure.

Are we out of the experimental-budget phase?

I think 90% of the market is still in the experimental-budget phase, but we're starting to see early promises of actual value. My entire business is focused on those repeatable use cases.

Everyone thinks they're a master of agents and agentic workflows. What do they think they know that they actually don't know?

I think they believe they know how to use AI in a business context. The people in the enterprise who are most excited about AI and position it most strongly are CTOs and information-technology people.

The people who understand how to use AI in a business context are those closest to the business. CTOs and IT people actually know the least about the business.

We're jumping the gun a little bit, with the CTO trying to build the CRM before it's been invented. You need business people to build the CRM and Excel first, in that order of operations.

There's a lot of unbundling of AI applications, with CTOs trying to build a very specific vertical application. I think building a platform like Hebbia Matrix is what will unlock users' ability to discover what they can use AI agents for.

What will be the pricing mechanism for the future of agents?

There are about 4 canonical pricing models: consumption-based pricing, per-seat pricing, paying a salary to rent an employee—which seems ridiculous but will seem less so—and flat pricing.

It depends on how you're driving value. Hebbia is building human-centric AI, the human layer for orchestrating an AI agent staff and scaling at inference. We use per-seat pricing because it's ultimately always back to the human.

You'll see all of these new business models and pricing mechanisms.

You use per-seat pricing because it's human-centric, or just because it's what customers know as a buying mechanism?

We're human-first and business-user-first. CTOs like to pay for consumption or API usage, while business users like to pay per seat because that's how they map back to value.

We also want to incentivize change. Technology isn't the hard part of all this. It's hard, but the hardest part of AI is change management. No matter what company you're in, the hardest part is people—actually getting people to use the software.

When you charge for consumption or API pricing, you're disincentivizing change. You're saying, “I'm going to penalize you monetarily every time you use an AI application.” What the heck?

11. Quick-Fire Round

With a per-seat fee, it might be expensive, but use it more. You can run more LLM calls on Hebbia effectively for free than on any other platform if you're actually driving real change, and that's what I love to see.

Are you ready for a spicy round?

Give me the spicy round.

We've got the tissues out here, too, in case you cry. These are questions from friends of yours.

Would you sell for $2 billion today?

No, I would not.

What was the worst VC meeting you've ever had?

The worst VC meeting I've had—I can't say this one.

Which one? Was it one where you thought, “They're douchebags”?

I would never say. Our customers are VCs. I love almost every VC I've ever met.

What was the single best VC meeting?

It's somewhere between Peter talking to me about anything but the business—deeply academic things—and Mike taking me on a walk around the Woodside Horse Park, which was pretty great.

Do you trust Sam Altman?

No.

Let's do a quick-fire round. I'll say a short statement, and you give me your immediate thoughts. Does that sound okay?

Sounds good. Let's do it.

What do you believe that most people around you disbelieve?

I have a crazy one. I believe UFOs are real. It's a little more on the nose right now, but I actually believe there's fundamentally different propulsion technology and that the U.S. government has access to it.

Wow. Conspiracy theory.

I have a lot of spicy takes.

What trait are you slightly ashamed of, but that has contributed to your success?

I don't think I'm ashamed of it per se, but one thing I always hid was that I'm deeply religious. In an industry that's very atheistic or agnostic, it was something very personal to me.

How has it contributed?

When you're doing hard things, chewing glass, or working late hours, believing in something larger than yourself—or believing in what you do as a vocation or something deeply purposeful and meaningful—is additional fuel.

It helps you in a way that's good for the soul. It charges you up.

Do you pray?

I do. I pray for an hour every morning, believe it or not.

What?

I wake up and sit on a meditation cushion. I used to meditate, and I think meditation is great, but praying, putting something out into the universe, or having a dialogue with whatever you believe in is even more powerful.

Do you talk out loud?

Sometimes. I live by myself, but sometimes it's all in my head.

I think it's incredibly good for the human mind. It's almost an antivirus for the human mind.

Exactly. People meditate, so why is it weird to pray? When you dive into the human psyche and aren't looking at your phone, a lot of the time it's also a great channel to think.

A lot of the best ideas I've had at Hebbia have come from moments of silence.

I get up at 8:20, my first meeting is at 8:30, there's an espresso ready for me, and I'm saying, “Where are my shorts?” We have different morning routines.

What's your gym routine? You're a fit dude.

I try to work out every day. I end up channeling the startup pressures, anger, and anxiety into lifting heavier and heavier things.

Is Silicon Valley back as the center of all this?

There was a podcast that recently came out where everyone said, “If you're going to build an AI company, you've got to build it in Silicon Valley.” But there's one company in New York doing something really amazing, and that company is Hebbia. It seems like they're doing something interesting.

I do think we're the exception rather than the rule, unfortunately.

I'm a big believer in Silicon Valley. Why are you the exception?

I think we're a Silicon Valley company in terms of our style of work, how hard we work, how we pursue new technology, and how we invest in technology.

At our core, we started in Silicon Valley, and we have almost only Silicon Valley investors.

What have you changed your mind on in the last 12 months?

Longer than 12 months—probably 18 months ago—it was the scaling-at-inference idea. It was the belief in a new set of scaling laws and that they would be really important.

ServiceNow, Salesforce, or UiPath: shag, marry, kill.

I wouldn't do any of them. I don't think traditional “shag” is my thing. That's short-term excitement.

I probably kill them all. I don't think traditional enterprise B2B applications are sexy. We're an enterprise AI company.

Are you a buyer of Salesforce?

We are. I don't think Salesforce is shit in this next generation.

I think Salesforce has built a very, very, very sticky network effect with people. People are the switching function at the end of the day. It's not a technology problem.

Claude can build a Salesforce. Klarna had another fugazi story about not using Salesforce because Claude built them a CRM. I think the switching cost and network effect of changing human beings' habits are too high.

Salesforce is one of those monopolies with so much habitual stickiness.

You can buy one company in the public markets that will benefit most from the next wave of AI. Which company do you buy?

I would probably buy NVIDIA—or AMD, rather. It's a lame answer, but I think AMD will benefit from the shift to scaling at inference in an outsized way.

You can be CEO of any other company for a day. Which company?

Probably not a company. I'd love to be mayor of New York. It's a fascinating job, and I'd love to make some change there.

What question are you never asked by investors, angels, advisers, employees, or journalists that you think you should be asked?

One of the most interesting questions is: where does creativity stem from? Where do you get inspiration from? How do you come up with new ideas?

I don't think people come up with new ideas by brainstorming or in conversation. I think that's fugazi, fugazi.

I'm also a painter. I do large-scale, 10-foot-plus oil paintings.

I heard about this. Where did that come from? Are you a poet as well?

I love to write. I'm probably not as good at poetry, but I think other creative outlets are really important.

I'm an annoyingly uncreative kid at school. Painting? If you want me to try to run, I will fall on my head.

What do you find about painting that is good for you?

It's one of those activities where you can channel emotion, intuition, or latent thoughts that are somewhere in your subconscious, and connect things in a meaningful way.

In a world full of stimulus, where you're always thinking or churning through something and dealing with distraction, you can stand in front of a canvas for 10 hours with some nicotine and just get lost in the art.

Great artists will tell you they don't even know where paintings come from. You're channeling something. It's one of the best places to think.

It gives you connections and brings up parts of your subconscious—connections you can't really access without being creative, whether you're making music, writing, or painting. I think that's one of the best ways to process anything.

Do you feel that your parents are proud of you now?

I think so. They've heard about it. There was one moment when I think my father's boss called him and said, “Your son's kicking ass.”

That was a very happy moment for me.

That's a special moment.

The chip remains, though. It's not going anywhere.

George, I so appreciate you being so open, and I so appreciate the conversation. You've been fantastic to have. I've loved it, and I appreciate all the research you've done and all the crazy lines of questioning.

Thank you, Harry. I appreciate it a lot.