Arvind Jain 谈 Glean 的打造与企业 AI 的未来
Glean 的创立逻辑是企业搜索，但早期押注 transformer，让 2019 年的产品在生成式 AI 到来时占据了异常有利的位置。 Arvind Jain 从一个普遍痛点切入——研究显示，员工有 1/3 的工作时间花在寻找信息上——并使用基于 BERT 的模型匹配概念，而不是关键词。随着生成和推理能力提升，Glean 从“工作生活里的 Google”演变成了“工作生活里的 ChatGPT”。
Glean 在通用能力已经成熟的地方使用 frontier models，同时自行构建企业级检索能力。 它在客户语料上训练小模型，生成定制 embeddings；同时单独微调小型开放域模型，用于拼写检查、同义词处理和缩略词扩展等窄域搜索任务。综合回答和多步推理则交给 GPT、Gemini 或 Claude。Jain 的原则很直接：“不要重新发明已经被发明的东西。”
权限和数据新鲜度，而不只是模型质量，才是企业 AI 的核心约束。 Glean 从 Google Drive、Slack 和 Salesforce 等系统导入治理规则，将权限写入索引，只检索当前登录用户有权访问的文档。由于 embeddings 本身可能泄露受限信息，针对客户的模型只会在 Glean 判断安全的子集上训练。
Jain 的连续创业模式，是在其他人已经放弃的市场里押注普遍存在的问题。 Lukas Biewald 追问 Rubrik 和 Glean 是否体现的是非凡执行力，而非新颖想法；Jain 同意，企业搜索曾经“只有失败”，并成为投资人不愿投入的“死区”。他的信念来自两个变化：SaaS 让碎片化数据问题恶化，却也让数据更容易获取；transformer 则让语义理解在技术上成为可能。他从 Google 带来的运营模式是把创新放在第一位，招募聪明的工程师，并基本放手让他们构建。
一个具体的 ROI 案例，是对企业 95% 的非结构化数据进行推理。 在出现客户流失后，Glean 财务团队一名非工程师员工要求一个 agent 汇总 Salesforce 客户名单、共享 Slack 中的情绪信息和产品使用数据，将客户风险划分为绿、黄、红三档。Jain 认为，这份结果优于传统仪表盘，因为它能够纳入带有主观判断的文本证据。
Glean 评估的是完整的回答链路，同时承认企业 AI 不可能消除错误。 它从真实互动中提取“黄金”问答集，例如获得良好反馈的 Slack 回复，用来测试检索和模型变更，并使用 LLM 作为评审。针对幻觉，它会将回答逐行与提供的源材料核对，并可能压制缺乏依据的表述或直接拒答；但 Jain 表示，过时、缺失或检索不佳的知识，比模型凭空编造本身造成更多失败。
Jain 不认为减少人力是最有价值的 AI 策略；他更希望每名员工身边都有一支可规模化的“梦之队”。 他设想 assistants、coworkers 和 coaches 帮助每个人完成所需工作的 90%，同时企业保留甚至扩大团队，让成员能够完成“10 倍的工作量”。这场变革可能显得渐进——一次只改变一个任务——直到员工发现，自己已经与 2 年前截然不同。
1. 企业搜索成为 Glean 切入生成式 AI 的入口
Jain 于 2019 年初创办 Glean；据他所知，当时“生成式 AI”这个词还不存在。他要解决的是一个更早出现的问题：信息散落在各种企业系统中。研究显示，人们有 1/3 的工作时间花在寻找信息上，因此最初的目标是打造“工作生活里的 Google”——一个集中提供文档、数据、答案和相关同事的目的地。
Transformer 已经在改变 Google 内部的搜索。2018 年底，基于 BERT 的模型进入开放领域后，Glean 得以在企业语料上训练小模型，在几乎没有词汇重合的情况下识别文档之间的语义接近度。Jain 谨慎地表示，Glean “可能是第一家在企业中使用 transformer 的公司”，因为这项改进是被实际观察到的，而不只是预测出来的。
2019 年的模型既不能生成内容，也不能进行推理，但基于 embedding 的匹配为后续发展打下了基础。随着模型能力到位，Glean 从呈现文档转向阅读文档并生成精准答案：“过去工作生活里的 Google，如今更像工作生活里的 ChatGPT。”
2. Glean 构建企业级检索，同时借力 frontier intelligence
Jain 对技术团队的要求是“最大限度使用外部正在发生的创新”。Google、OpenAI 和 Anthropic 已经提供了能力强大的模型，重复造轮子既帮不了 Glean，也帮不了客户：“不要重新发明已经被发明的东西。”
在企业语境真正重要的地方，Glean 会训练小型、客户专属的模型。这些模型学习公司的语言、代码词和内部概念，生成定制 embeddings，比通用语义搜索更准确地将问题匹配到相关文档。
检索完成原材料组装后，由 GPT、Gemini 或 Claude 综合生成答案；最大的基础模型还负责推理，以及将复杂问题拆解成多步骤的 agent 工作流。对于拼写检查、涉及同义词和缩略词的查询扩展等窄域搜索操作，Glean 会微调小型开放域模型，因为使用 GPT 虽然有效，但属于“杀鸡用牛刀”。
安全机制被嵌入检索层。Glean 从 Google Drive、Slack、Salesforce 等系统同时读取内容和治理规则，再将权限写入索引，确保每名用户只能检索到获授权的材料。对于定制 embeddings，它不会在完整语料上训练，而是通过启发式方法选取安全子集，因为模型内部状态可能以不可预知的方式泄露受限信息。
3. 技术条件改变后，失败品类也可能变得有吸引力
Biewald 认为，Jain 的履历很难仅仅归因于运气：按他明确保留余地的回忆，Rubrik IPO 时的估值超过 200亿美元，而实现 100亿美元以上的结果，概率可能是“千分之一”或“万分之一”。但他的追问也很关键：Rubrik 和企业搜索都是市场熟悉的故事，其他人也曾反复失败，这说明真正的差异化可能在执行力。
Jain 的回答依旧刻意克制，但选题逻辑很清楚。Rubrik 于 2013 年进入构想阶段，并于 2014 年初启动，解决的是保护数据这一普遍需求；当时勒索软件和其他攻击不断增加。Jain 表示，最优秀的技术人才和注意力集中在消费科技，企业市场却相对落后，现有企业产品往往已经存在超过 20 年。Glean 同样瞄准几乎每名员工都能感知的问题，但包括 Google 在内的数十年失败经历，已经让投资人相信企业搜索要么无法解决，要么并无必要。他还指出，过去 10 年里没有一家知名搜索公司成立。
让一个糟糕品类变成反共识机会的，是两项技术变化同时发生。SaaS 让信息问题严重得多，却也让问题变得可处理，因为 SaaS 系统允许 Glean 读取企业数据；transformer 则提供了概念理解能力。Jain 仍觉得自己的判断不可思议，因为“怀疑是我存在的一部分”，但最终认定，大型公司必须解决“具有普遍性的”问题。
4. 企业 AI 正从搜索框走向运营层
Jain 预计，无论资历高低，每名员工都将拥有 assistants、coworkers 和 coaches，帮助他们完成所需工作的 90%，同时帮助他们持续提升。如今，这样的支持团队主要是 CEO 的特权——assistant、coach、chief of staff 和高管团队——但 AI 可以让它普及到所有人：“你不再是一个人。”
Glean 目前有一个 agent，可以在不召开进度会议的情况下产出最新技术教程。它读取设计文档，检查已经解决的 Jira 项目和后来提交的代码，再将过时描述与当前实现对照。Jain 将其称为“常青文档”：系统生成的是最新解释，而不是返回最显眼却已经过时的文档。
客户流失案例展示了这种运营杠杆。出现一定程度的流失后，一名没有软件工程背景的财务员工要求 Glean 从 Salesforce 枚举客户，检查共享 Slack 对话中的情绪，结合产品使用数据，并将每个账户划分为绿、黄或红三档。
Jain 认为，这份报告优于仪表盘，因为仪表盘主要处理结构化指标，而 agent 可以在分析使用数据的同时解读主观文本。他更大的判断是：“企业数据中有 95% 是非结构化的”——其中包含大量洞见，只是过去没有人手提取。
支撑这一愿景的组织模式，部分来自 Jain 在 Google 的经历：把技术和创新放在首位，招募聪明且有动力的工程师，提供资源和自主权，并基本放手让他们组织和构建。在 Glean，他表示自己的角色往往是少干预，同时维持以创新为先的文化。
5. 可靠性更取决于证据质量，而不是自信生成
Glean 把自己当作“客户 0”，但评估也延伸到客户部署。真实对话本身就包含潜在的黄金数据集：一个 Slack 问题、一条人工回答，以及积极反馈或点赞，可以帮助确定优质回答应当是什么样子，而不必手动构建每个基准。
每次搜索、检索、模型或其他系统变更，都会在这些问答集上运行测试。由于质量部分取决于主观判断，Glean 还使用 LLM 作为评审，将生成回答与黄金答案进行比较，并衡量端到端系统，而不是孤立地评估模型表现。
Jain 坦率承认，Glean 不会“始终以完全正确的方式回答问题”。信息可能缺失、过时，或来自错误的来源；模型本身也可能产生幻觉。Glean 会将生成的主张逐行与输入证据核对，为其附上支持性的人工撰写材料作为引用；如果找不到依据，就压制回答或直接拒答。
因此，模型幻觉只是众多失败模式之一，而且在 Jain 看来并不是最大的那个。“大多数损失”发生在更早的环节：从多个系统中找到正确的信息，判断信息是否最新，以及确认信息是否来自该领域专家。如果不存在答案，Glean 也可以转而找出正在处理相关议题的人；Jain 还认为，未来 agent 主动向人类追问缺失事实是合理的发展方向。
6. 增强人力的回报高于削减人数
Jain 认为，关于微型团队或“一人独角兽”的预测“从根本上偏离了重点”。一家企业的价值、智力和产能都来自人，因此 AI 应该增强这套基础，而不是主要用来取代它。
他的策略十分明确：“我不想真正缩小团队规模，我其实想扩大团队”，同时让每个人都能完成“10 倍的工作量”。企业可以选择用 AI 缩减团队，也可以继续投资于人才，大幅提升收入端。
持久有效的成功因素仍然是“努力、投入和成功的欲望”，但任务组合会发生变化。工程师可能不必再记住那么多 API 和框架，而是投入更多时间进行思考和设计，减少写代码的时间。Jain 预计，适应会以渐进方式发生——“每天都有一个新任务”——直到这些累积变化最终带来转型。
You're listening to Gradient Dissent, a show about making machine learning work in the real world, and I'm your host, Lukas Biewald. This is a conversation with Arvind Jain, whom I've known for a while. Arvind is the CEO of Glean, one of the most successful enterprise companies using LLMs—maybe right now the most successful enterprise company that foundationally uses LLMs.
And I really want you not to be fooled by Arvind's humble style. He's one of the most successful founders in Silicon Valley. Before Glean, he founded Rubrik, which IPO'd for, I believe, a valuation over twenty billion dollars. So he's been phenomenally successful twice. Very low-key, but clearly very driven and has some real insights. I hope you enjoy this conversation.
Arvind, thanks for taking the time to talk. It's been a while. I think you have an incredibly successful company, so I feel maybe embarrassed asking you this, but I think it's not a consumer brand, so people might not have heard of it. Can you explain what Glean does, please?
Yeah, absolutely. First, Lukas, thank you so much for inviting me to this. Glean is an enterprise company, an enterprise AI company. Think of us as ChatGPT, but inside your company.
Glean connects you with all of your internal systems, your data, and your knowledge inside your company, and then gives you a ChatGPT-like experience where people come and ask questions, and Glean will answer those questions for them. It's going to use all of the world's knowledge, but additionally, it's going to use all of your internal company data, knowledge, and context to help answer those questions for you. Simply put, that's what we do.
Of course, as AI continues to advance, the same platform that we use to build our Glean assistant can also be used as an agent-building platform to build all kinds of AI agents on our platform.
What's interesting about your company is that it's been one of the real successes, I think—or at least, from my perspective, one of the real successes—in terms of real LLM applications for the enterprise that work well. But if you look at the timing of when you started it, it was before it was obvious that LLMs were going to work so well, at least to most people.
Yeah.
How did your vision for the company start, and how did it evolve through this period of LLMs taking off?
1. Enterprise Search Before LLMs
Glean started in early 2019. In those days, nobody was talking about large language models, and the term generative AI did not exist, to my knowledge, at the time. We were not thinking about that either. We had a problem at hand that we were interested in solving, which was search.
This was my experience in every single job that I had before Glean: It was so hard to find things inside the company. When you do your work, you need some information; you're looking for a document. It's extremely hard to actually find information.
As enterprises, we have so much information and so much data, and it's spread across many different systems. In enterprises, we tend to have hundreds of thousands of different systems, and stuff is all over the place. When I'm looking for something—when I have a quick question I need an answer to, or I'm looking for a doc that I know exists but still cannot get to it—those things cause a lot of frustration.
In fact, there have been a lot of research studies. People would spend 1/3 of their working time just trying to find things. That was the first problem that we wanted to solve. We wanted to build a Google for people in their work lives: one place where you go to ask your questions, and we bring the right information back to you.
Sometimes, it may not even be the information. It may be that you're just looking to connect with the right people who can help you on a given topic. So we become that one place where you come and find data, documents, and people all in one place. You don't have to think anymore about where to go and look for things. That was the first product that we wanted to build.
The interesting thing is that in early 2019, transformers were already a thing. The rest of the world didn't really care as much, but inside search teams at Google, we were seeing a really good impact from transformer technology on some of the core components of a search engine.
We saw this real promise in those days, in early 2019, where we could conceptually understand information. We didn't have to actually go and build a keyword-based search system anymore, which is quite dumb. If you think about how we match information to people's questions, it's happening at a very surface level with keywords.
Google had actually put BERT-based models in open domain in late 2018, and we were able to use them and then actually go and pretrain what we now call small language models on your enterprise corpus. We could start to deeply and fundamentally understand what your business is about and what any given document is about, and then do this conceptual, semantic matching of information to people's questions.
Interestingly, transformers played a foundational role in Glean's evolution. We were probably the first company to use transformers in the enterprise in those days because it was really, really applicable to building a good search experience at the time. That's how we got started.
Over the last 6-plus years, as these models have advanced, what used to be a Google for you in your work life became more like ChatGPT for work life. In 2019, transformers had no capability to generate information. They had none of the reasoning capabilities that models have today. They had this embedding-based, conceptual sort of matching capability, which we were able to use.
As these new capabilities came, they allowed us to evolve our product. Now we don't have to just surface information to you; we can actually read that information using AI and give you precise answers to your questions. That's how the evolution has been over the last 6-plus years.
So you're politely telling me that you did see the power of transformers, at least in 2019, when you started the company, which is super cool.
Well, yes. We saw it, actually; it wasn't just a vision. We could take 2 documents—documents that had no real word overlap. If you look at the actual choice of words used in those 2 documents, they were very different, but still, a BERT-based model trained on your enterprise data could tell you, “Hey, these 2 documents are very close to each other semantically.”
That was it. Remember, in 2019, today we expect a lot from AI, but AI was not there in 2019. That itself was still a big step function in terms of our ability to really conceptually match people's questions with knowledge. We saw that power, and then we decided to bring it to enterprises.
You're in this world where these labs are investing massive resources and constantly coming out with new models. Do you still fine-tune a custom model for every customer, or is that important to you? How do you think about which parts to buy or use and which parts to build in-house?
2. Building Versus Buying Models
The thing I keep telling our technology team is, “Do not reinvent things that have been invented already.” That wouldn't do us or our customers a service. Our approach is always to maximally use innovation that's happening outside because it's available to us.
The good thing is that all of the innovation happening in the LLM space—all of these models from companies like Google, OpenAI, and Anthropic—they're all making those models available for us to use. We like to use those models, and we do; we use them quite a bit in our stack.
The way our technology works is that we actually pretrain and build some models for semantic matching. We have to do that ourselves because these models are all about your enterprise and your data, and sort of building an understanding of what your business is about.
And so there, we actually will go and connect with your enterprise data and knowledge. We'll take all the documents and actually build and train the model on them so that it can start to understand how your business talks and speaks, what code words you use, and what's your lingo. So for that, it's actually important to go and train a model on your enterprise corpus.
But these are very small models. We don't actually train large models. These models allow us to build custom embeddings for your enterprise content, so that when people come and ask for things, we can match their requests with documents in your corpus in a much better way using semantic matching. So those models we build ourselves.
But then when it comes time to actually do something with the information, imagine the workflow of when you come and ask a question in Glean. The first step is, well, we see your question and we're trying to understand what it's about. We're going to use our semantic embedding models—the ones that we train ourselves—to assemble the right pieces of information and get the best knowledge within your company that we think can answer those questions.
And now we're going to take this information and ask AI to synthesize an answer from it. For that, we don't actually use our own models. We use out-of-the-box models like GPT, Gemini, or Claude. Similarly, for reasoning, when you take a complex question and you're trying to break it down into a multistep agent workflow, we use the reasoning capabilities of the largest foundation models.
So that's the architecture. It's more, “Don't train, don't fine-tune when you don't need to.” But there are certain things in the search stack where it makes sense to do that. Maybe I'll give you 1 or 2 more examples.
Please, yeah.
Well, one thing—so in search, of course, part of it is that you get a question from a user and you're trying to semantically match it with the right information. That's what I described just now. But then there are smaller parts of the stack, for example, query expansion. In retrieval systems, you have to spell-check users' questions, find synonyms, and find acronyms.
For those models, what we've realized is that you can take these really small, open-domain language models and fine-tune them for those specific tasks. For example, for spell-checking, we have a fine-tuned version of a language model that will do it for us. For some pinpoint tasks—small, very specific things—these fine-tuned models actually help us do those things faster and cheaper.
They don't add more capabilities. You could take GPT, a large GPT model, to do a spell-check for you, and it would do a fantastic job. It's just overkill to use that model for that kind of task. So there's a little bit of that fine-tuning happening in the search stack for us. But otherwise, for reasoning and generation, we use out-of-the-box models.
3. Security For Enterprise AI
I think another one of your advantages, or parts of your technology that you talk about a lot, is the security model that you have. In enterprise search, there are such complicated rules around who can access what data. As these models get more complicated, if you're building an embedding model, you could imagine it somehow leaking information across embeddings. Can you talk a little bit about how that works and how you think about it?
Yeah. The way Glean works inside your enterprise is that, as we connect with all of these internal systems like Google Drive, Slack, or Salesforce, we're reading and understanding content that lives in those systems. But we're also understanding the governance of that data—who are the people who can access that information? For any given document, for example, we'll look at the list of people who are authorized to see it.
This is fundamental because if you're going to deliver AI in the enterprise in any way, you have to do it in a safe and secure way. You can't have AI-based systems start to leak information internally to your employees who didn't have the right to see that information. So the way our system works is that we connect with these individual applications, understand their permission model, and bake that into our core indexing technology.
The permissions are baked right into our indexing system. So now, when you come and ask questions in Glean, we'll retrieve information for you that can answer that question, but we'll only retrieve information that you have the right to use. We know who you are, you're signed in, and we'll use our enterprise identity to establish which documents you have access to and which are relevant to this topic.
Then we make the models work on it. That's the fundamental idea: in a RAG-style system, you can make AI safely work on data that you have permission to use. But you also mentioned this interesting question around embeddings. If you're building custom embeddings, what content do you actually train them on?
Because if you train them on all of your enterprise corpus, they will indeed start to leak information, and you won't realize in what form or shape they actually do it, because ultimately you can't understand, behind the scenes, how AI works. So when we're training custom models, we typically train them on content that we think is safe to train on. There are a lot of heuristics that go on behind the scenes to understand what information is actually safe to train on.
So if you use it only on a subset that is safe, then your embedding models are not leaking information.
I see. Speaking of security, I think a really interesting fact about you, at least from my perspective, is that you're also the founder of Rubrik, right?
Mm-hmm.
Which maybe not everyone has heard of, but it's an incredibly successful company that recently IPO'd. I feel like you're kind of a two-time unicorn, verging on a decacorn maybe, as a founder.
And it's so amazing. You're so humble and understated. I just wonder if you have any insights into what you're doing differently from other people. What do you attribute these 2 successes to?
I would think that a $10 billion-plus exit has got to be 1 in 1,000, 1 in 10,000, depending on where you start. So the fact that these 2 are correlated, I think, is a powerful indicator that there's more than luck involved, at least in your case.
Well, I think it is correlation, and no more than that. But what I would say from our experience is that when we started Rubrik, that was early 2014, and we picked a problem that we knew every business faced. It was really hard to protect your data, especially as the attacks were on the rise with ransomware and other things.
We felt that, in 2013, when we were contemplating starting Rubrik, all the energy and attention, and all the best tech people, were focused on consumer technology, and enterprise was sort of lagging at the time. We didn't have a lot of great companies. The SaaS revolution was still fairly early, and we felt businesses were being ignored.
Just helping them with this very fundamental problem—“Hey, we'll help you keep your data safe”—was something where there were no products being built that were less than 2 decades old. So we saw an important, large problem with a large market, and after that it was about just going and doing a good job building a product and doing a good job building a business.
I think we were fortunate in that we were able to go build a good team that then, of course, generated all this success. I would say that with Glean, similarly, we picked a problem that is not a niche problem. You can go and talk to every single person, anybody that you know, and ask them, “Hey, is it easy for you to find things inside your company?” And they'll all say no.
And so it was a pretty obvious problem. Again, nobody was working on it at the time, and we felt that we had to be the pioneers and solve this. I got a lot of advice, by the way, at the time, not to actually get into this business.
Well, yeah, because—sorry. I knew you were going to give me something really humble like this. But I think what these 2 have in common, in my mind, is that they're ideas that a lot of people have that don't work. I feel like I've heard those pitches for your 2 companies quite a lot in my life.
But then you were wildly successful doing both of these different things, where I think it must be really execution-oriented. It's not like you came in with some wildly new product that no one was expecting. It seems like you did a phenomenally good job in a market that didn't seem so great, or that seemed broadly crowded and specifically hard to execute in. I'm not sure, but I want to draw some insight out of you.
Yeah. I think I'll add 1 thing. Look, think about search. A lot of people actually told me not to work on search and enterprise search.
Mm-hmm.
Because there were only failures. In the 2 or 3 decades of enterprise search history, including products from Google, every time Google tried to build an enterprise search product, it always failed.
When you see that kind of history, people started to feel like this problem was maybe too hard to solve, or maybe it wasn't even worth solving. If nobody builds a big business, maybe there's no need to build a product like that. That was the mindset, and that's why there was no innovation in search inside the enterprise for a long time.
There were no startups. We started in 2019, and there was no search company that you would know of that got started in that 10-year period. This was a dead area, probably one of the worst areas where no investors wanted to invest.
But this is interesting: you see a problem that everyone has. Everybody faces this pain point, yet there are no good products. Then there are also technology trends. We saw some new things that happened at that time. One of them was SaaS, the whole SaaS transformation, which made the problem much worse, but also made the problem tractable in the sense that SaaS systems actually allow you to go and read data inside those systems. Before, we couldn't even do that. It was just so hard to get hold of data in the enterprise.
We also saw transformers as a really core new capability that would help you deeply understand content and actually build a better search experience. We observed those trends, which gave us the confidence to take that contrarian bet and say, “Look, we think we can now go and build a good product and build a huge business.”
I think part of it is that I’m an engineer, by the way, in my background and in who I am as a person, and doubts are part of my existence. That’s how I think most engineers are. I was surprised at myself when everybody else told me not to do this. I don't know what it was that actually gave me that conviction, so I was surprised at myself.
Maybe that's 1 thing to think about: if you're going to build large companies, of course you have to tackle problems that are very universal in nature—broad problems that have broad impact.
Do you think that you operate differently? Another thing that's interesting is that your product helps other companies operate better and more efficiently, and I assume you're dogfooding your own products.
Yeah.
If I were a new hire asking you, “Should I join?” would you tell me, “Get used to a special kind of culture that we have here at Glean”?
That's a good question. I don't know if we have a particularly unique culture. A lot of our learnings, and my learnings, have come from Google, where I spent a lot of my working life.
Google was a really special company. It was very different from the 3 companies I'd worked at before Google. It was so unique in the sense that it put technology and innovation in front of everything else.
As an engineer at Google, you were the king. Nobody got to tell you anything. You could go and build whatever technology you wanted to build. You got to decide whether it was important, whether you wanted to build that system, and you had all the resources to go and build those things.
That was the Google model: you hire really smart people, really good engineers who are very motivated to do something special, and then just let them be. They'll figure out the right things to figure out, they'll figure out how to organize with other team members, and they'll do a great job.
Frankly, for a manager, that's also a pretty good situation to be in. You don't have to do any work.
I love that model, and we've always followed it. I do it now at Glean, where we have a really amazing team, and my role is generally to stay out of the way. I can't help myself—I do get in the way sometimes—but often, it's mostly about building an innovation-first culture, and success will come.
4. The AI Powered Workplace
Do you think that having much better search—and now you're building these agentic systems, which sounds like they could be customized to what a company is trying to do—will cause companies to function dramatically differently in the future?
The way we work is fundamentally going to change, both at an individual level and at an overall business level. In fact, this is the vision for the future. I believe this is what our work lives are going to look like in the future.
For any person, regardless of how senior they are, you will have an amazing team of assistants, coworkers, and coaches that are going to help you not only do 90 percent of the work that you need to do, but also help you get better at the things that you do.
If you think about today, there are certain people in the company who have that luxury. If you're the CEO of a company, you have your coach, your assistants, a chief of staff, and an executive team. You have this really amazing set of people around you, which helps you become a true multiplier for the company.
AI is going to change that. Today, only the CEO or the senior executive can have that luxury and that help. AI democratizes that and brings that help to every single person.
Everybody's going to have this dream team around them, and you get to build that dream team with AI-powered agents. You can have 1 agent that works like your assistant, another that acts like your coach, and yet another that acts like your coworker and shares your workload with you.
That's the world that we're going to be in. You're no longer alone. You have this team continuously around you.
Similarly, for companies, most of the business processes that we have today could be envisioned as having parts that can easily be handled through some kind of AI agent. There's a big transformation ahead of us. It's going to reshape how individuals work, and it's going to reshape how a company—how an organization—looks.
People’s roles are gonna change. I think about software engineers. You hear that in the future we’re gonna become more code reviewers as opposed to people who write code. That kind of change in where we spend our time is gonna happen to all of us.
It’s funny. I think about what’s in our, at Weights & Biases, in our Notion and Google Drive and all these things. We have a lot of, for example, OKRs and our quarterly plans. I keep being tempted to paste them into Gemini. I know the context window is big enough—
Yeah.
And put in my current OKRs as I get through the quarter and just be like, “Which ones do you think we’re gonna miss?” Or, “What do you think about this strategy? Do you think it’s a good idea?” If you look back at what we’ve done, do you ever do stuff like that? Not tactical coding stuff, but sort of start to ask about broader strategic questions?
Yeah. Actually, today with Glean, these are the kind of things that it does a really good job at. Glean is connected to all those systems, so you don’t even have to—
Right.
Even have to cut and paste those OKRs from that—
Right.
Notion or Google Drive doc. It already is in that overall enterprise-wide search index that Glean has.
Some of my favorite activities with AI now are, number one, when I want to learn about technology, what a given team is working on inside our company, or how we designed a particular system. Instead of setting meetings and asking them to give me an update and walk me through some kind of presentation, I ask Glean to generate the latest, up-to-date tutorial on that topic for me. And that’s all I say.
I have this agent where we’ve told it that when I’m looking for a tutorial on a given topic, it should make sure to go and look at all the design docs that have been written. But design docs often become obsolete, so it should read the design docs, look at the updated results, and then start to look at more recent Jiras that got resolved and code that was committed into that component. Get the latest view of how things work. Don’t give me the stale doc; give me the latest on this system and how it works.
This is actually pretty cool, because now there’s this concept of evergreen documentation. I don’t have to deal with stale stuff anymore, because AI is doing that human-like thing where it’s reading stale information and new information, deciding what’s right and up to date, and giving it back to me.
Similarly, an example similar to what you mentioned: we’re a new business, and if you build a good product, you don’t have a churn problem. We actually didn’t have any dashboards around churn, and nobody had built any analysis on it because it wasn’t relevant.
I love how humble you are. That’s amazing. You’re old enough that you must have a very sticky product if, six years in or something, you’re not worried about churn.
Yeah. But guess what? We had some churn last quarter, and we certainly got worried: “Okay, what’s happening? What’s gonna happen to all the other customers? Where is the customer health?” I’m exaggerating a little bit. Some teams have it, but we didn’t have really good versions of it.
We wanted to do an analysis, so one of our team members in finance—she’s not an engineer and hasn’t built anything from a software-system perspective—went and asked this question in Glean: “Go and look at every single customer of ours.” You can ask Salesforce to get a list of all the customers. For each one, go and look at our shared Slack channels with them. Go look at what the conversations look like and what the sentiment is. Go look into our product dashboards and look at the usage.
Then take all of that data and give me a risk profile: whether it’s an account that is healthy—green—or red or yellow. Sure enough, AI is pretty good at this, and it generated a report that tells us all the top-risk accounts and the ones that are not.
Frankly, I would say it was better than any dashboard we would’ve built, because it was able to make use of subjective textual information, which dashboards obviously can’t. Dashboards can only look at usage trends and some structured information. These are certainly new capabilities, where you could tap into the 95% of your enterprise data that is unstructured. It’s full of insights, if only somebody had time to extract them. That’s where AI comes in super handy.
Totally. Wow, that’s such a compelling story. I want to do that as soon as we get Glean installed here. Wow. How do you—one thing I think about with you guys is your customers probably don’t let you pull metrics on their data.
Yeah.
Or maybe they do, but how do you know if you’re shipping a new model that’s better or if your embedding is better? How do you actually measure your progress?
Yeah. First, we have customer zero, which is Glean, our own instance. As we bring new models in, we have an evaluation framework. In any customer deployment, there’s plenty of information available, and you can have question-and-answer pairs: people have asked questions and somebody’s answered them.
You don’t even have to create these golden datasets. Sometimes you can obtain them from actual, real conversations that people have. For example, in Slack, somebody’s asking questions or people are responding with answers, and there’s a lot of reaction and thumbs-up to that information. That’s a really good example of a golden set of what a question is and how AI should be answering it.
We build these golden sets for each customer, and then we’ll bring a new model in. We would run all those questions through the new model. It doesn’t have to be a new model; it can be any change that we make in our own systems.
Our system has a lot of things happening in it, including changes in the search and retrieval system and the models that we use. So you measure how the answers are looking on the golden set with this new change that you’re rolling out. And then you also use AI to—because these are subjective answers, you also have to use LLMs as a judge to see how close you are to what the golden response is. That’s the fundamental way for us to measure how well our end-to-end system is working.
Hmm. Do you have issues with hallucinations? I feel like in almost every domain that kind of drives us—
Yeah.
Crazy. Do you take special measures about that? I would think of it from an enterprise perspective—
Yeah.
They might feel more sensitive to hallucinations than even a consumer.
Yeah, 100%. Hallucinations are there. We have some safeguards against them, but that’s not to say that Glean is always going to answer questions exactly the right way.
I think it’s not there because there are so many things that can go wrong when somebody asks a question. Sometimes the information isn’t even there to answer. Sometimes the information you’re using to answer people’s questions is stale or out of date. So the failure scenarios are numerous, and one of them is the hallucination of the model itself.
For the hallucination part of the model, remember that in our system, when you ask a question, we’re the ones who first assemble the raw materials—the knowledge that we’re going to use to answer that question. We give all of that knowledge to the model at prompt time, along with the question, and the model comes back with an answer. What we do after that is take the model’s response line by line and try to see if we can find that same piece of information in the input that we gave to the model.
We use that for our citation framework, and we cite, on a line-by-line basis, where the human-generated information was used to produce that particular line. When we don’t see it, we either suppress it or don’t show the citations. That way, you can tame the model a little bit. If you see information being produced that isn’t present in the input we gave it, then we can give up and tell the user, “Look, you asked about a relevant piece of information, but we weren’t able to figure out an answer.” That’s how we suppress hallucinations a little bit through citation and reference checking.
But the bigger problem in the enterprise is not that. The bigger problem, when people ask questions, is whether you can even get the right information from all the different systems you have, and whether you can get information that’s up to date, correct, and written by a subject-matter expert. Most of the losses actually happen on that side.
I see. Do you imagine a world where these agents start asking people if they can’t figure something out—routing to someone? I think I would kind of like that if I could deploy a bunch of agents to go collect information for my employees.
That’s an interesting use case. In Glean, one thing we do is that we don’t think of our product as, “Take all the information in the company, understand it, and then answer questions for people.” It’s very much a people-oriented product.
When we deploy Glean inside a company, we build a deep understanding of the enterprise itself: what the business is about, who the different people are, and what they work on. In fact, it’s a common scenario that when somebody asks a question and we can’t answer, sometimes our answer will be, “Look, we don’t have the information, but these are the people who are working on this topic, and you can go and connect with them.”
But with agents, you’re flipping the question. Instead of humans asking agents to do work, agents are actually pinging people to retrieve information from them. That also seems quite legitimate because that’s probably the future world, where there are deeply intelligent agents that work alongside humans. Humans can ask agents questions, and agents can also ask humans questions.
Do you have other thoughts on where organizations go as they interact with AI more deeply?
5. AI Augments Human Talent
I would just add one thing. There’s a lot of this ROI conversation that tends to happen with AI: “I’m going to have AI come and do a lot of the work that humans do today, and we’ll be able to replace them. Maybe I’ll have a small team. Maybe we’ll have a single-person company that does a billion dollars in business.”
There are all these viewpoints about how AI replaces humans, and I feel like that’s missing the point in a big way. Ultimately, a business and its value come from its people. That’s where the intellect is, and that’s where the capacity to do work is.
It’s much better to think of AI as an enabler for all of those people. As a business, I don’t want to shrink my team. I actually want to grow it, but I want everybody to be able to do 10 times more work so that we can succeed in a much bigger way.
So when you think about enterprises and how they change with AI, people are going to have one of two mindsets. Either they’re feeling the pressure and want to reduce their team size significantly, or they can keep investing in people and improve their top line significantly.
Do you think that the skills that make people successful will change in this world?
I think the fundamental skills remain the same. What makes one successful, in my opinion, is hard work, dedication, and the desire to succeed, and I don’t think that fundamentally changes.
Of course, all of us have to adapt. Think about software engineers. Maybe it’s no longer necessary to remember all the different APIs and frameworks that are available because AI can always bring that information back to you. You can spend more time thinking and designing, and less time writing code, perhaps.
These changes will happen incrementally, and we won’t realize that we’ve transformed ourselves by the time it happens. It won’t feel like a step-function change. Every day, there will be one new task that we start using AI for, and before you know it, you’ll be fundamentally different from what you were 2 years ago.
All right. That seems like a nice place to end. Thanks so much for your time, Arvind.
Thank you, Lukas. It was really fun to have this conversation.
I appreciate it.
Thanks. Yeah.
Take care.
Thanks.