AI is Making Enterprise Search Relevant, with Arvind Jain of Glean

Hi, listeners. Welcome to No Priors. This week we're speaking to Arvind Jain, CEO and co-founder of Glean. Glean is an AI-powered enterprise search and knowledge management platform, which allows you to not only access all the different internal documents and Slacks and other things that your company may have, but also allows you to enhance workplace productivity by using different applications on top of that. Prior to Glean, Arvind had a really storied career. He co-founded Rubrik, he was early at Google, worked on search there, amongst other things, and so we're very excited to have him here today. Arvind, welcome to No Priors.

Thank you for having me.

I'm really excited about this. I've known you for years, and Elad's known you for maybe 15 more years than that. You're an amazing, repeat successful founder with Rubrik and Glean. I want to start by asking you about search. You've been a search guy since before it was cool, for a long time when it felt not solved, but not as dynamic. How broadly has search changed because of LLMs?

1. LLMs Rewrite Search

I've been working on search for almost 30 years now. A long, long time. The paradigm has completely shifted.

Search had been static for a long time. It was this keyword-based paradigm: people ask questions, you find words, try to find them in documents, and bring them up to the users. What LLMs have completely changed is search itself. The main thing they have done for search is allow us to deeply understand a question that a user is asking, and similarly, to deeply understand what a document is about. You can actually match people's questions with the right information conceptually, and that gives us so much more power.

It's not brittle anymore, and I think it's been a foundational technology to really evolve search into these new experiences that you're seeing these days, where you can go far beyond just surfacing a few links to an end user and actually deeply understand their questions and answer them directly using the knowledge that you have.

If I remember correctly, Glean got started in the more traditional search world. As these foundation models and LLMs have come to the fore, you've really shifted how you think about both the capability set that you provide and how you approach things. Can you tell us a bit more about how you started building the systems, how that's shifted, and then how you've mapped new use cases against it?

You're now effectively this really interesting platform that can be used in all sorts of ways inside an organization, around the corpus of information they have. I'd actually love to hear about the technology transition. How did you think about that? When did it happen? I think you really lived through it in a meaningful way.

Yeah.

You were super early to it, actually.

2. Glean Builds Enterprise Search

We had good timing, I would say. We started thinking about building Glean in late 2018 and started the company in early 2019. The interesting thing is that transformers as a technology had emerged by then. The whole world was not talking about it, but in search teams at Google, we saw the power of embeddings and how they could fundamentally change search. We had the luxury of actually seeing this in action.

Version 1 of our product already used transformers for semantic matching. We didn't have these terms. Nobody used to call it vector search. We didn't have RAG. These terms had not been invented yet, or generative AI, for that matter. Internally, we used to call it embedding search, and it was a core technology that we started out with.

The models at the time were not as powerful as today. We started with the BERT model that Google had put in the open domain, which was trained on all of the internet's data and knowledge. We would then take those models and, for every customer of ours, build custom embeddings on their business content. That would power the semantic part of the search system.

But remember, search as a technique—there's been a lot of focus on embeddings and vector search over the last few years—but that's actually only one part of building a good search system. If you think about an enterprise, imagine a company that has been around for a few decades. They have tons and tons of information spread across many different systems. A lot of that information has become obsolete because it was written many years ago.

When you build a search product, it's not enough to say, "I want to understand somebody's question, and I'm going to match it with the right information that semantically or conceptually matches what the user is asking." You have to solve for other problems, too. You've got to pick information that's correct today, that's up to date, and that has some authority—someone who's an expert on the topic has actually written that document. You have to do all of those other things, too, to truly pick the right knowledge and bring it back to people.

So we started building the product in that shape and form. It was a very different product, actually. Nobody had really approached enterprise search as a problem before. The interesting thing I remember is that even though I was coming off a successful company—we'd had good success with Rubrik—I don't think people really wanted to invest in enterprise search, or in me, for that matter, because this problem was not exciting.

It was traditionally a very bad problem, right? There were all these search engines—FAST. I remember in the early Google days, there was sort of an enterprise search engine based in Norway. There were lots of attempts at this.

A lot of attempts and no successes.

Why do you think it didn't work? It felt like an awful market.

It was like a graveyard of all these companies that tried to solve the problem and didn't. Part of it was just that search is a hard problem. In an enterprise, even getting access to all the data that you want to search was such a big problem.

In the pre-SaaS world, there was no way to go into those data centers, figure out where the servers were and where the storage systems were, and try to connect with the information in them. It was a big challenge. SaaS actually solved that issue.

Most search products—the companies—started in the pre-SaaS world. They failed because you just couldn't build an end product. But SaaS allowed you to build something. My insight was that the enterprise world had changed. We have these SaaS systems now, and SaaS systems don't have versions. Everybody—all customers—has the same version, and they're open and interoperable. You can actually hit them with APIs and get all the content.

I found that the biggest problem was actually solved: I could easily bring all the enterprise information and data into one place and build this unified search system on top. That made it possible for us, for the very first time, to build a turnkey product. That was a big unlock.

So it was the rise of these connectors and APIs internally. You're using Google Docs instead of older-school systems, or you're using Slack, or these new tools that now provide you access to the data or underlying content.

You guys must remember Google Search Appliance.

Yeah.

The idea of, "I need to slurp your data continuously into a hardware appliance in order to actually do search," is ludicrous.

It was a challenge. By the way, the origins of Glean were at Rubrik.

At Rubrik, we had this problem. We grew fast, had a lot of information across 300 different SaaS systems, and nobody could find anything in the company. People were complaining about it in our pulse surveys. I always ran IT in my startups, so it was a complaint that came to me.

I had to solve it, so I tried to buy a search product and realized there was nothing to buy. That's really the origin of how Glean got started as a company. Search in SaaS made it easy to connect your enterprise data and knowledge to a search system, which made it possible for us, for the very first time, to build a turnkey product.

There were a lot of other advances as well. Businesses have so much information and data. One interesting fact: one of our largest customers has more than 1 billion documents inside their company. Now hear this: when Elad and I were working on search at Google in 2004, the entire internet actually had 1 billion documents.

There’s a massive explosion of content inside businesses, so you have to build scalable systems. You couldn’t build a system like that before, in the pre-cloud era. I would spend all my time just trying to build that scalable distributed system, which we don’t have to anymore because of all the great cloud technology.

And then, of course, transformers. That was really the big unlock: we could actually understand enterprise information more deeply. It was very necessary in the enterprise compared to on the web. On the web, even if you don’t have good semantic understanding, there’s so much that you can learn from people’s behavior because you have a billion people coming and using your product. In the enterprise, you don’t have that luxury, so you have to make up for that lack of signal from users with other techniques, and transformers are one of them.

It sounds like you feel a combination of more traditional IR and search techniques and embeddings is relevant. Do you think that persists? Where would you want bespoke infrastructure or signals like freshness and authority, and how much do models just do in the end?

3. Models Need Organized Context

Yeah, I think there’s always this thought that the models will have near-infinite context windows, and you can just give them everything and they can figure things out automatically. But I don’t think we’re anywhere close to that happening.

I’ll give you an example. Let’s say that models are mimicking human intelligence, right? They’re getting more and more capable of working the way humans do. But as a human, imagine if I were to give you a question and then say, “Here’s everything,” and, in a completely non-organized fashion, give you a whole bunch of, let’s say, 1 million documents and let’s imagine you have the memory powers and speed to process them. It still feels like a very complicated thing.

It’s very hard to make sense of information that is being given to you out of order. Can I give you 1 document that is something from today, something from 4 months back, something from 3 years ago, and then something again from 2 days back? If I give you information in a manner where it’s not organized in any shape or form, then as a human, you’re going to have a lot of difficulty reasoning over it.

So we think about the models the same way. There’s a good amount of work that you have to do to present the information to the model in some organized fashion. That’s when they’re going to do a much better job reading that information, reasoning over it, and giving you the answers. Sure, you can give them more and more over time, but it still matters how you provide them with the right information.

Now that you have this sort of corpus of information, right? You’ve basically aggregated all the internal documents of a company, which in itself is incredibly useful just for search. But you’ve also gone down the route of enabling applications to be built on top of it in different ways. Can you talk a bit about that, and what are some of the common use cases that you’re seeing?

4. Search Becomes An Agent Platform

We started with this vision of building a Google in your work life. But then, as models got better and developed reasoning and generation capabilities, it changed our product. Our new product, Glean Assistant, looks and feels more like ChatGPT.

Instead of me asking questions and seeing a bunch of links come back to me, now, of course, you converse with Glean. You ask questions, and it works just like ChatGPT. You come and ask a question, and it’s going to take all of the world’s knowledge and, additionally, all of your internal company’s data and knowledge, and use that in a safe and secure manner. It knows who you are and what information you can use within the company to answer questions for you.

That was the first progression in terms of our product. We evolved from being a Google to something that looks more like ChatGPT—a more powerful version of ChatGPT inside your company.

As we built Glean Assistant, you could think of it more like a personal assistant that you’re giving to every employee in your company. It’s a tool, your sidekick, and it’s always available to help you with whatever questions or tasks you have. It’s going to use all of your company’s context and data to help you with your work.

But businesses are actually a lot more interested not in that, but in thinking about how they can transform their company with AI, or how they can take specific business processes where they’re spending a lot of money and bring automation into them with AI.

We started getting asked for that before agents became the talk of the day, before everybody started building agents. Early last year, when agents had not yet taken off, people were asking us, “We need to build more curated applications using this data platform that you have.”

As an example, HR teams would come to us and say, “Look, we love Glean Assistant. People come in there and ask questions about benefits, PTO, vacation policies, and whatnot, and it works great. But sometimes it uses content that’s not authorized or blessed by us. If somebody’s asking questions on people-related topics, we want Glean to use only the curated content that our people team has created, and we want it to behave in a particular way, with a particular tone, and all of that.”

That was a request we started getting last year: Can we create more specific, curated experiences, function by function, for different use cases? So we started to build that. We weren’t calling them agents; we were calling them apps.

Now, of course, people think of them more as agents because it’s no longer just asking questions and getting answers. You want these specific functional experiences to actually replace a business process, which also involves doing some work in those systems—not just answering questions, but actually doing some work.

Arvind, when you talked about access to the right data with the right authority, it really begs the question of access control, right? In a platform like Glean, when you have all this unstructured data, this seems much more complicated. What’s your overall stance, or how do you think this is going to work in the future?

5. Enterprise AI Needs Trust

Yeah. Enterprise information, in some sense, is governed and protected. Most of the knowledge inside the company—90% of the knowledge, I should say—is private in some shape or form within your company. You’ll have a document that’s private to you, or that you share with a few other people. But that’s the nature of enterprise knowledge. That’s the fundamental way it works.

You can’t build, for example, a model inside your enterprise, dump all of your internal company’s data and knowledge into it, and then make that model available to everybody in the company. Because if you do that, you’re leaking information inside your company. You’re letting somebody on the engineering team see sensitive information that probably only the HR team should be able to see, as an example.

So any AI experiences that you build inside the company have to think about security, governance, and permissions at a fundamental level. That’s what we do in Glean. When we connect with all these different systems inside an enterprise, if we index a particular document from Google Drive or a conversation from Slack, we also keep track of which users can access that information.

This is fundamental. Any access to data that happens through our platform has to match the users’ permissions. The users have to be signed in, and we will only let them use information they have permission to access. This is an important problem to solve. Unless you have infrastructure like that, you cannot roll out AI safely inside your enterprise.

I learn a lot from people who work on search, especially search at any sort of scale, because you get all sorts of weird user behavior. Related to your idea of AI as a personal assistant, what are some behaviors you see from end users in terms of how they’re using Glean or AI in general that you think we should just do more of?

I’m always very surprised when I learn from Google people about the behaviors around navigational search, how many queries are 1 word, what the popular queries are, and those sorts of patterns.

And so I'm sure you see Glean and AI superusers.

One of the biggest surprises for me was that I always felt we were building such an intuitive product. It's like this little—there's no UI. There's one box, and you ask a question or put in a search. What's the big deal? Why do you have to learn how to use this?

Yeah.

We realized that as we added more and more natural-language capabilities, including the ability to ask a really long question—a paragraph-long set of instructions—people wouldn't do it. I think everybody has been trained over the last 20 years to type in 1 or 2 keywords. Google has taught us what search can do.

With Search, we never had a problem. We would launch our product and see immediate, high usage. Nobody was confused about how to use it. With Assistant, people didn't know what to do with it. Some people were more curious and would ask all kinds of questions that we couldn't answer. For example, somebody might say, “What should I do with my life?”

Coming back to this, that was one of the key learnings: AI is actually very unintuitive. For most people, you have to expose them to these capabilities in an incremental fashion—things that are more meaningful to their day-to-day work.

For example, if I'm an engineer, you can prompt me sometimes: “Look, you can learn about a new piece of technology. I can create a 2-page tutorial for you right now.” You have to understand what people's core work is, and then give them prompts to start experimenting and get excited about trying something with AI.

One thing I would also add is that a lot of the time, businesses are excited about AI. They have a lot of dollars to spend on it, but they're also asking for ROI: “We're going to make all this investment—what are the returns? What are the efficiency gains I'm going to get? What top-line improvements can I make to my business?” There's a lot of focus on that.

I think one thing that often gets overlooked is education, because the world is changing. Imagine that 3 years from now, you wake up and you're the CEO of a large enterprise. What do you want to see in your workforce? You want to see people who are trained and AI-first. They're experts who know how to leverage the strengths of AI, because this is a difficult technology. It's not perfect, it's not easy, it makes mistakes, and it hallucinates, but it's powerful. If you become an expert, you can get a lot done with it.

That has to be the objective today. As leaders think about AI, how do you give people tools that motivate them to bring AI into their day-to-day work?

You had an amazing career between being early at Google, starting Rubrik, and now starting Glean and running it. What was unexpected about doing Glean? You'd gotten to so much scale and done such amazing things in the context of Rubrik. What was hard, unexpected, or just very different about Glean that you didn't anticipate?

6. Founders Must Create The Market

From a product side, one of the most interesting things for me was how hard it was to roll the product out to our customers.

We had a very different journey at Rubrik compared with Glean. At Rubrik, we were in an established market. There were buyers and dollars, and you had to replace an old technology with a new technology. Here, we were in a market where there were no budgets. There was no concept of buying a search product in the enterprise.

Everybody thought, “Yeah, this is an important problem, but it's not a line item in my business priorities. It's a vitamin, not a painkiller. People are living without it.” Well, that's true. You live without something you don't have. That's by definition true.

We had a lot of challenges. We had to do a lot of evangelism to get the right people—those who wanted to be innovators—to make that bold call and buy a product they weren't used to buying. That's the first part of it: you have to create the market for this, which was difficult.

The second thing, which was actually very interesting, is that our product was working well. It was doing good search and letting people find things. But then we started to hear from businesses, “I'm scared of good search. I don't want a good search product in my company because I have all these governance gaps. I have sensitive information all over the place, and now people are discovering these things.”

When we launched, for example, people found the salaries of other employees. At one of our customers, somebody found a sensitive M&A document about something that hadn't happened yet. People were very scared of actually having good search.

That was an interesting challenge. We were doing good work and doing it safely and securely, but if you don't have good governance, you can't sell the product because it's so good.

It seems like LLMs should be able to help with that, right? They can classify documents and say, “Hey, this one may be sensitive. Do you want to secure it?” Et cetera.

Exactly right. We were forced to build that. We had to go above and beyond respecting permissions in individual systems to understanding who you are and what you're asking. You should have the right to ask the question, and when the information comes back, does it even feel safe enough for us to show it to you?

In that sense, we ended up becoming a security product. A lot of companies buy us to fix governance in their data and systems and become AI-ready—not only for Glean Search and Glean Assistant, but also for all the other AI products that you can buy inside the enterprise. That was a very interesting journey.

Personally, at Rubrik, I wasn't the CEO. I ran R&D as one of the founders of the company. Here, I had to learn how to become a CEO, and I don't think I've learned it yet. That's a constant challenge and a set of learnings that I go through because fundamentally, I'm still an engineer. Everything I do is shaped by that mindset.

Growing out of that into being able to run a large business is a personal transformation that I'm going through.

One thing that I think is striking is that, from a go-to-market perspective, you all are really focused on big enterprises, right? You mentioned some of these enterprise data needs. A lot of people always want to do PLG, and you've really done the top-down sale. It's been incredibly successful, and you've done it twice now, because Rubrik was largely that as well.

Yeah.

Could you talk a little bit more about when it makes sense to do big, direct enterprise deals versus the PLG motion, and how you think about that as you build businesses? I think it's very differentiated, and most people just can't pull that off. I'm curious about how you think about when to do it and then how to do it.

To be candid, when we started Glean, my dream was to do PLG. I'm an engineer, and I wanted the company to have engineers, with the product selling itself on the web. Who doesn't want that? It was something we desired.

But the problem is that our product is, by definition, a company-wide product. We cannot offer the product to one individual inside a company. Even one person requires us to search across the entire company's information to meet their search needs. It's expensive. You have to index all of your company's data and knowledge.

So we never had the concept that we could make it available to 1, 2, or 10 people inside the company.

So we're sort of forced, structurally, to build in that fashion where it is an enterprise product. We roll the product out company-wide to every employee. That's what makes it cost-effective.

But coming back to your question, the standard approach that I think companies prefer now is that they think of PLG as basically lead generation as a funnel. You sort of nurture and then expand using an enterprise sales motion. The right recipe for me, if I had a choice, would be to start both motions simultaneously. I wouldn't say, “For the first 3 years, I'm going to focus just on being PLG and then bring enterprise sales later,” because you're leaving a lot on the table. Timing always matters, and you have to start the motions at the same time.

Arvind, one thing that we have talked about that I feel must have been hard—the priors on this market were not great, right? We talked a little bit about the rationale for you feeling like you really saw the problem internally anyway, and understanding that there were these architectural, foundational things that had changed in terms of the movement to SaaS and API-based integrations and such.

Still, I think it's a really big question of advice for founders or maybe people joining startups: When should you agree with the priors that something is a bad market, or how should you think about that question?

I'll share a few things on this. Number one, I think as engineers, there are always doubts. The more you look at priors, the more likely you are to ultimately kill your own idea. For any given idea, there are 10 reasons why it won't work as you start to go into the details.

Sometimes a simpler approach is helpful: There's a problem. You talk to people, they have and feel this pain, which clearly means that nobody is solving it yet because the pain exists. So don't go into the details anymore. Just do it. Things will get figured out over time.

At least for me, this was unusual. I'm an engineer by training, and I'm naturally trained to question things. There's a lot of self-doubt in my mind. I don't know what happened to me when we started Glean, because there were all these people saying, “Don't do it,” and somehow they couldn't discourage me. I just felt that this was an exciting problem.

I knew everybody in the world had this issue. Even at Google, it was always a big joke internally. All of us were spending all of our time making it easy for people to find things, but we couldn't do it internally at Google. It was super hard to find anything inside the company.

I somehow found that conviction. I was being lazy, not willing to go into the details and look at all those priors. I just wanted to do it and solve it. I think that's what worked for us in this particular case.

I feel like Glean had 3 big components that all came together, which you mentioned earlier. There was the need that you identified as somebody running IT for your own company. To your point, it goes back to Google: This was a need, and every company that I've talked to has always wanted to build search and directories and all this stuff.

The second thing is the rise of connectors and APIs in the context of existing enterprise software that everybody's using, so you can extract the data more easily. The third thing was the big shift in terms of the underlying technology—the shift in terms of what search is capable of, these foundation models, embeddings, and so on.

Given the latter 2, are there other big opportunities that Glean is going to work on that you've identified as really interesting areas that suddenly are tractable again?

7. The Personal AI Workforce

I think for us right now, the focus remains on the 2 core products that we have. The way we think about our company is that we have this really powerful end-user AI assistant that helps every person work differently in the future. Then we have this agent platform that you can use to bring AI into every one of your business processes, making them better and more efficient.

We've been making big promises on both to our customers. The way I describe and pitch our product to our customers is the following: Come to Glean, ask it any question, or give it any task. Glean will use all of the world's knowledge and all of your company's internal data and knowledge in a safe and secure way, answer those questions for you, or complete those tasks.

I just promised you that Glean does everything. You don't have to work anymore. We're a long, long way from solving even the pitch that I just mentioned to you. We have to understand knowledge properly. We have to pick the right, correct information and throw away the old information. There are so many challenges there, and there are so many issues.

People talk about hallucinations as a big problem with AI models. We feel like a bigger problem for us isn't even hallucinations. Most of the time, you can't find the right information. Sometimes it's not there: People are asking questions, but nobody wrote it down. Sometimes we're not able to find the needle in the haystack. We pick the wrong thing.

There are a lot of challenges, and I think we will be working on this problem for a long, long time. I don't see us having any need or wanting to do something different. Just solving this one problem itself is a big success. We're going to stay focused on these 2 products.

Let me also talk to you a little bit about the vision for the future. I think the way we all work has sort of been accepted as something that AI is going to change completely. AI is going to change how people work. AI is going to change how businesses even look and feel, and what kind of workforce you have in the future.

One thing that's going to fundamentally happen is that each one of us is going to have this amazing team of assistants, coworkers, and coaches that are truly personal to us. You're always surrounded by that team, and this team knows everything about you—your work life, what you need to do today—and proactively helps you, does 90% of your work for you, helps you get better at your work, upskills you, and is your coach.

That's the world that we want to be living in. Today, there are some people who already live in that world. For example, as a CEO, you have the luxury of having all of that. You have assistants, a chief of staff, an executive team, and a coach. But in the future, that's going to be something that all of us are going to have, regardless of how senior we are. You may be a new graduate joining the workforce.

That's what we're trying to solve for. We're trying to build that amazing personal team around every individual that's going to make us all 10Xers. That's just a natural extension of continuing to evolve our Glean Assistant product and make it better and better over time.

Yeah, Arvind, thanks so much for joining us today.

It was excellent. Fun questions.

It's always nice to see you.

Likewise.

Find us on Twitter at @noprior spad. Subscribe to our YouTube channel if you wanna see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at know-priors.com.