让你的 Agent 自我进化的瑞士军刀:Composio CTO Karan Vaidya 谈打造智能工具
Karan Vaidya 认为,真正的护城河来自使用驱动的工具迭代,而不只是覆盖超过1,000个应用、超过50,000个工具的目录。 他表示,运行时故障可以实时触发新版本工具,而曲折的执行轨迹会沉淀为可复用技能,让 Agent 下次走上“直线路径”。在他看来,跨 Agent、跨应用积累的经验才是护城河;Nathan 则表示自己过去几周一直在使用 Composio。
Agent 系统最稀缺的资源是注意力,因此暴露更多工具反而可能让产品变差。 Karan 警告,Agent 可能“用错刀片,在上下文过载中自杀”(“use the wrong blade and suicide via context overload”);Composio 的做法是只呈现少量元工具,在需要时即时发现相关动作或技能,并用代码沙箱处理10,000甚至100万条记录。“Harness 说到底就是上下文”(“Harness is nothing but context”)。
公司的 go-to-market 分化为两类:专业用户买的是配置简单,开发者买的是治理能力。 1个 MCP server 就能把 Claude Code 或类似运行时接入托管认证;生产团队则可以按模块采用发现、沙箱、认证或完整执行层。Karan 称 AWS、Zoom、Glean 和 Airtable 都是客户,动作级权限、钩子、人工审批、SOC 2 和 VPC 自托管共同构成企业信任基础。
最强的用例已经接近完整工作任务,而自治程度越高,权限设计的价值就越大。 Karan 的 Agent 会寻找开源项目贡献者、补齐联系方式、用自己的邮箱发起联系,并在1到2周内生成30–40次调用;销售 Agent 也能准备类似的定向外联草稿。他的控制模型将研究型 Agent 与执行型 Agent 分开:前者拥有广泛读取权限但不能采取行动,后者可以写入但只能访问有限数据,并接受人在环检查。
技能可能通过把判断力转移到可复用指令中,让前沿模型重新商品化。 Karan 通常让 Opus 创建技能,再让 Sonnet 执行;据他在自身体验中的判断,Haiku 表现并不理想,而测试过的 GPT 技能案例约90%可以直接运行,当前模型整体达到90–95%也并不难。关键例外是 Anthropic 更擅长轮询,而 GPT 可能停下来等待输入。Composio 正在开发指标和基准测试,也已经进行部分面向模型供应商的适配,把降低模型锁定作为核心价值主张。
AI 会强化底层基础设施,同时从界面和定制化层面对成套 SaaS 形成压力。 Karan 预计 AWS 和 Cloudflare 会受益于软件创建规模扩大;如果 Salesforce 和 Slack 能快速推出 Agent 化界面,也有机会守住阵地。Nathan 给出的更尖锐测试对象是 Intercom:两人交谈时,他称 Fin 面向数千家客户、以每次0.99美元的价格解决接近70%的工单,但 Composio 提供的133个工具可能让部分公司自行打造定制技能,潜在节省80–90%;Karan 预计企业会选择性内制,而不是全面转向自建。
Composio 的内部成本结构,已经显露出劳动力从执行转向监督 Agent 的早期信号。 一支3人团队负责构建和改进工具的流水线,公司总人数约15人;Karan 称上月该流水线的支出约为10万美元,其中 token 成本已经高于人力成本。他的概括是:“我们需要更多人来消耗更多 token”,同时公司计划推出共用1个钱包的高级工具包和 universal CLI。
1. 一把好用的瑞士军刀,不能把50,000片刀刃全部摊给模型
Nathan 的切入点是:Composio 覆盖超过1,000个应用、提供超过50,000个工具,囊括大量没有必要重复开发的常见集成。即便只是给 Google Drive 或 Slack 开通访问权限,也可能要在控制台完成一系列操作、点击大量权限设置,足以劝退普通用户。
Karan 接受“瑞士军刀”这个比喻,但不认为工具覆盖面就是终点:把所有刀片都递给模型,它可能“用错刀片,在上下文过载中自杀”。他对 Composio 的定义是“Agent 化工具执行层”。
这一执行层组合了即时发现、动态注入工具、认证、授权、范围化访问、沙箱,以及新邮件、Slack 消息或 pull request 等触发器。Composio 的 dashboard 还提供治理、可观测性和审计能力。它的任务,是只向 Agent 提供完成当前执行所必需的上下文。
Nathan 表示,过去几周他一直在使用 Composio,其中就包括处理 Google Drive 和 Slack 的权限问题。这让配置负担成为他亲自测试后的结论,而不只是对理论集成成本的描述。
2. 个人用户买便利,生产团队买控制权
对 Claude Code 或类似 Agent 产品的高级个人用户来说,价值主张是用1个 MCP server,替代分别安装、学习 Google Drive、Zoom、Datadog 及其他集成。认证既可以通过对话发起,也可以在 Composio 的 dashboard 中管理。
生产环境开发者可以通过 MCP、API 或 SDK 接入完整 harness,也可以只选用发现、Workbench 沙箱、认证或 actions 等模块。团队自有工具可以与 Composio 工具并列运行,不必被迫采用全有或全无的架构。
Karan 表示,规模化运行后,治理、可观测性和审计能力都很关键。他提到 AWS、Zoom、Glean 和 Airtable 是客户;这些客户的评估结果,也能让处理敏感公司数据的小型买家更放心。
安全体系从动作级最小权限开始:Agent 可以读取邮件,但不一定被允许发送邮件。执行前后钩子可以增加策略检查或人工审批;SOC 2 和 VPC 自托管则对应企业要求,Composio 也已经针对 AWS 的用例在 AWS 内部自托管。
3. 代码沙箱把海量任务变成可处理的工作
Nathan 问的是架构问题:Claude Code 在本地执行 Bash,部分 provider tools 在远程运行,结果还要在两种环境之间传递——那么工具执行究竟应该放在哪里?Karan 的回答是,Composio 的沙箱主要负责移除那些模型本不该重新搭建的基础设施。
Agent 会获得一套工具和抽象层,以尽量减少代码量,尤其是认证以及在程序化执行和 function call 之间转换的部分。Karan 表示,同样类似 Docker image 的环境很快也会支持本地运行,用于内部工具或机器本地工具。
当 Agent 需要处理10,000封邮件时,直接 function calling 会失效,因为不可能让每一条结果都占据上下文。放进沙箱后,Agent 可以写代码,甚至在代码中再次调用 LLM,处理10,000条、甚至潜在的100万条记录。Karan 称这“有点像 Inception”。
挂载文件夹解决了另一个不起眼但常见的故障点:放入其中的文件会被上传至 S3,并获得可分享链接。Agent 可以分析邮件或 Stripe 活动,生成报告,将报告复制到指定文件夹后分享出去,而不用另行搭建文件分享流水线。
4. Smart MCP 的关键在渐进式披露和从执行中学习
Nathan 将 Smart MCP 与第一代 API wrapper 区分开来:他要的是能够接收高阶意图、组合多次调用并逐步披露能力的一层。他认为 MCP 和 CLI 之争没有那么根本,因为两种接口最终都可以支持智能披露。
Karan 的核心约束是:“注意力绝对不是免费的。”即便上下文达到百万 token,把50,000个定义全部展示出来也可能拖垮性能,因此模型起初只看到少量工具,相关动作则在运行中动态注入。
发现机制不只适用于工具,也适用于即时调用的技能。检索到的技能可以规定应该调用哪些动作、应在 Workbench 中运行什么代码,以及此前哪条轨迹曾经得到目标结果;Agent 再将这套配方调整到当前请求上。
反复出现的失败会变成执行前的预警:将陷阱、应该做什么和不应该做什么提前放入上下文。Karan 对整个产品理念的概括是:“Harness 说到底就是上下文。你必须对上下文进行工程化。”
5. 用户请求从指令变成完整结果
Composio 通常接收不到用户的原始语言,因为 Claude Code 或其他 Agent 运行时充当了“智能中介”。如果用户要求连接 Google Drive,运行时已经知道应调用连接管理,而不是发出含义模糊的发现请求。
Karan 将12月视为信任拐点:用户开始相信,当前模型已经具备明显更高的自治能力。软件工程率先发生变化,但知识工作者也越来越多地只描述想要的结果,把工具选择和中间步骤交给 Agent。
一个直接例子是:给 Agent Gmail 访问权限,让它检查过去1个月的邮件,判断哪些已经没有价值,然后归档。Agent 会写代码,并在代码中调用 LLM 对邮件进行分类。
Karan 自己的招聘流程更接近一项完整的招聘工作:扫描 agentic、Python 和 TypeScript 项目,找出优秀贡献者,补齐所在地、邮箱、LinkedIn 和社交信息,然后用自己的邮箱联系候选人。他说,在发出数千封邮件后,系统在1到2周内生成了约30–40次调用。
6. 长上下文自治让权限成为架构原语
Nathan 描述了一个本地、规模达 GB 级的数据库,里面存有5年的邮件、Slack、私信、录音电话和播客文字稿。它能提供丰富的关系历史,但也可能包含通过邮件发送的密码、恢复码或信用卡信息,而这些内容他无法安全地逐一盘点。
Karan 的答案不是打造一个全方位受信任的助手,而是建立“多个 Agent”,分别配置不同能力。研究型 Agent 可以广泛读取信息,却没有发送消息或执行外部动作的权限,从而让庞大上下文保持在自身范围内。
可写入、能够执行动作的 Agent 则采用相反配置:拥有行动权限,但只能接触很少的敏感个人或公司信息。人在环检查可以在执行前审阅拟发送的消息或拟执行的操作,降低机密 token 被发送到外部的风险。
原则是能力隔离:广泛知识和广泛行动能力不应自动共存。Composio 的 scopes 和 hooks,旨在为不同的 Claude Code、Codex 或其他 Agent 实例表达这种隔离关系。
7. 每次失败都能改进工具,或沉淀成可复用技能
Composio 的集成本身也是通过内部 Agent 流水线构建的:Agent 获取开发者应用凭证、创建 actions、识别依赖,并测试真实场景和边缘案例。当客户的 Agent 无法理解或成功使用某个工具时,也可以调用同一套机制。
一次运行时失败可能立即生成新版本工具,并将其插入当前上下文。Karan 表示,同一个工具可能累积数万个版本;如果某次改进只服务于一个客户的特殊工作流,它也可以保持个性化。
通用修正则会广泛传播。Agent 使用 API 的方式“疯狂且各不相同”,会发现文档没有正确描述的场景;Karan 称,最近这些自治发现已经让 Composio 的 Google Calendar 工具“远好于文档建议的水平”。
Nathan 担心升级会破坏已经调校好的工作流。Karan 的回答是,工具可以广泛改进,但学习得到的技能不会频繁变化,因此能够保留用户想要的执行轨迹和可重复行为。
8. 技能让切换模型变得可行,但还不够无缝
详细技能会把路径写得足够清楚,从而降低重复执行时所需的判断。Karan 的常规工作流是让 Opus 发现并编码一套成功流程,再切换到更快、更便宜的 Sonnet 执行后续任务;他称这一做法“效果非常好”,但根据自身体验,Haiku 表现并不理想。
跨供应商移植的稳定性较弱,但整体仍然很高。Karan 表示,在他对 GPT 模型的测试中,90%的技能可以直接运行;在其他场合,他将当前模型普遍可实现的水平描述为90–95%,剩余部分则暴露出原始技能中嵌入的供应商特定行为。
他给出的最典型反例是轮询:Anthropic 模型通常会等待并持续轮询,直到任务完成;GPT 则可能停下来要求用户输入。即便两个模型都理解书面指令,这些默认行为也会改变执行结果。
被问及 Gemini Flash 时,Karan 的表述有所保留:他还没有亲自测试完全相同的技能迁移场景,尽管生产环境的体验很顺畅;他预计当前版本或下一轮迭代都能达到所需水平。
9. 技能翻译层可能成为反模型锁定层
Nathan 的重新表述是,技能可能成为一层“重新商品化层”。前沿模型在质量上的差异可能制造粘性,但穷尽式指令会把更多行为从模型默认能力中抽离出来,沉淀为可移植的产物。
因此,Composio 将自己的 harness 定位为避免模型锁定的一种方式:认证、工具和技能留在同一层,底层模型则可以从 Anthropic 切换至 OpenAI,最终也可以切换到 Karan 认为成本可能便宜约10x的开源模型。
Karan 有时将切换模型描述为保留99%的可靠性,但他也单独承认,未经处理的跨供应商技能移植率更接近90–95%。剩余工作在于进行面向供应商的转换,补偿已知的行为差异。
Composio 正在开发指标和基准测试,也已经完成部分供应商之间的适配。Karan 表示,从易于实现的90–95%提升到100%确实很难,因为技能缺乏结构,还可能悄悄编码了创建该技能的模型所特有的假设。
10. Agent 赋能层扩张,但传统系统仍掌握数据
Karan 看好专门为 Agent 打造的工具:包括 Mem0、Supermemory 和 Zep 等记忆产品,Skyfire 等支付基础设施,Shopify 等商业平台,以及 Exa、Firecrawl、Tavily 等搜索或检索服务。
Composio 选择跨品类合作,而不是押注某一个赢家,让开发者根据用例自由组合。记忆产品已经存在明显需求,而与 OpenClaw 运动相关的后台 Agent,正在提升市场对支付和商业能力的兴趣。
被问到仍然缺失的品类时,Karan 诚实地没有给出答案:开发者似乎正在攻克人类几乎所有的活动,包括 Agent 向人委派任务。他表示,还需要进一步思考,才能说出一个显眼的空白领域。
尽管 Agent 原生产品正在兴起,大多数使用场景仍然会触及 Slack、Salesforce 等传统软件,因为“那就是记录系统”。在基础设施层面,Karan 预计随着软件创建变得更容易,对底层平台的依赖会增加,AWS 和 Cloudflare 因而受益。
11. SaaS 靠界面生存,定制化推动自建与采购重新权衡
Karan 预计,现有 SaaS 的界面会发生变化,但底层记录系统仍然重要。创业公司会打造新的 Agent 化界面,而 Salesforce、Slack 等既有厂商也在快速推进,因此最终胜负取决于执行速度,而不只是既有地位。
Nathan 的反驳更看好大客户中的成熟供应商:AI-first CRM 可能很有吸引力,但企业销售周期会给 Salesforce 时间复制最强功能。更具颠覆性的威胁,可能是客户将狭窄工作流带回内部,而不是改用一整套竞争性软件。
Intercom 将这种权衡具体化。Nathan 表示,两人交谈时,Fin 已经在数千家客户中解决接近70%的客服工单,每次解决的价格为0.99美元,并提供即时、全天候响应;但 Composio 暴露了133个 Intercom 工具,这让他推测企业可以自行编码一套客服技能。
Nathan 假设,每个案例的 token 成本可能约为0.10美元,从而潜在节省80–90%。Karan 认为定制化比节省成本更重要:内部 Agent 可以调用多个应用、限制特定动作,并执行公司专属治理规则。他预计1年内不会出现大规模离开 Fin 的现象,但也同意,摩擦成本下降后,“人们会逐步从采购转向自建”(“people will inch towards build compared to buy”)。
12. Agent 委派、token 经济学和界面仍将两极分化
Karan 的委派判断标准是任务的上下文权重。如果一项任务只消耗“1–2%”的上下文,主 Agent 就应继续掌控并获得合适工具;把预约安排交给缺乏上下文的子 Agent,可能导致它与主 Agent 已经看到的重要董事会会议发生冲突。
深度研究适合交给并行子 Agent,因为探索过程会消耗大量上下文,最终可以压缩后再返回。Karan 提到 Claude 的共享任务列表模式,以及 Composio 的 Agent 化工具;后者会保留 session ID,以支持多轮自然语言交互。
在内部,一个 orchestrator 可以监督20–30个 Claude Code Agent。一支3人团队负责构建和改进 Composio 工具的流水线;相对于全公司约15名员工,公司上月在这条流水线上花费了约10万美元。Karan 表示,该项工作的 token 成本已经高于人力成本,并将这种模式概括为:“我们需要更多人来消耗更多 token。”
产品扩张也遵循同样的整合逻辑:计划中的高级工具包将让多个服务共用1个 Composio wallet,而不是继续分别管理 API key;universal CLI 则可以通过一个界面访问多个应用。Karan 预计 CLI 和 MCP 会在一个“两极化世界”中共存,token 分配和不断提升的可追踪性将共同决定二者的平衡。
Karan Vaidya, CTO at Composio, welcome to The Cognitive Revolution.
For folks who don't know what Composio is, I've been playing around with it over the last couple of weeks, and I've come to think of it essentially as a Swiss Army knife for AI agents. Obviously, we're all developing agents for a wide range of use cases, some of which are ad hoc, minute-by-minute assistance, and others that are built in much more intentionally and structurally into products. All these agents need tools, and some of the tools we have the time and luxury to build out in a really intentional, bespoke way. Then there are just a ton of other things that a lot of people have common needs for, and this is where I see Composio coming in, with a ready-to-go set of 1,000 tools that you can plug into your AI agent and give it a much broader reach than it would have if you were building out every tool one by one.
That's my takeaway from getting under the hood a little bit. How do you like the description, and what would you add to that for starters?
You're on point in understanding the problem that we solve: We provide 1,000-plus apps and 50,000-plus tools to your agents—to anybody building agents. But that's not the final solution, because at this point in the LLM journey, if you provide 1,000 tools to an agent, it will probably use the wrong blade and commit suicide via context overload.
That's where we're essentially building the whole agentic tool harness. What I call Composio is the agentic tool execution layer. The whole harness for tool execution that, while building agents, you would need to develop or would want to give to your Claude Code or Codex—that's what we provide.
That includes a few meta-tools, so you don't have to put thousands of tools into your agent. It includes managing authentication and authorization, as well as giving the right scopes to the LLM. The problem I just mentioned is that you can't give 1,000 tools to the agent, so we do just-in-time tool discovery. That's one of the tools, along with dynamic tool calling, so that only the right set of tools the agent needs for a given use case gets loaded into the context.
One of the other problems people face while building agents is that, in a bunch of cases, direct function calling is not the best way to solve the use case. For example, if I want my agent to process 10,000 emails, it will probably run into context overload after, let's say, 100 of them. That's where we provide sandboxes where the agent can do programmatic tool calling on top of our apps, so it can process 10,000 or even 1 million emails by writing code.
On top of that, I think the strongest thing we do is continual learning. All our integrations are built by our internal agentic pipeline, which goes through the process of first getting the developer app and all the required credentials, creating the actions, finding dependencies, and testing them in real-world scenarios, including a bunch of edge cases. That's the whole process it goes through.
What that gives us is that, at runtime, when the agent is using us, we can figure out that a particular tool is not usable by the agent for whatever reason. There might be an error or some other failure, or it might not be able to understand the tool. In real time, that agentic pipeline is invoked, a new version of the tool gets created, and the newer, improved tool is added into the LLM's context—the agent's context.
We also have continual learning where, when we see that an agent is taking a zigzag trace to reach an outcome, we convert that zigzag trace into a set of skills. We have the whole end-to-end agentic trace of what it is executing, what the use case was, and so on. The next time the agent does something similar, it will take a straight path, making it more reliable, robust, token-efficient, and time-efficient as well.
We also learn from failures—the do's and don'ts of using particular tools, use cases, pitfalls, and so on. That's the whole harness that we provide.
We also have a notification system, which is what we call triggers. The agent can be notified, let's say, when an email is received, a Slack message is received, or a PR is created. The whole system, or harness, around the agent communicating with knowledge-work apps—that's what Composio provides.
Okay, cool. I wanted to go one by one through all of those topics and go a level deeper on each.
For sure.
Before we do that, though, I would love to understand a little bit more about who your users are.
And maybe I’m sure this is changing because obviously we have phenomena like OpenClaw popping up and a whole new population of users coming online, and I don’t think that process has reached its endpoint by any means just yet. But as I was using the tool, I was thinking, “Okay, I see 2 ways, or 2 broad scenarios, that I might use this in.” One is that it’s become almost reflexive at this point for me to go to Claude Code first whenever I want to do almost anything on my computer.
Even if that’s something as generic as searching for an email, I’ll go to Claude Code and ask it to search for the email rather than go into Gmail and search directly there. So, there’s—I guess I would call that the hobbyist market, or the individual user who has their own assistant agent. Those folks could really benefit from something that allows them to expand their toolkits really quickly.
Just this morning, actually, I was onboarding a teammate who hasn’t used Claude Code so far, and one of the questions she had for me was, “How did you give it access to our Google Drive?” I was like, “Well, actually, that was kind of a pain in the butt.” Claude actually talked me through the steps, but the steps were pretty gnarly.
I had to go into the console, create an app, click over here, and set some permission or whatever. I never remember all the steps, so that wasn’t super easy. I can see a lot of people stumbling over that and wanting the ease of not having to go through all those steps. Slack is another one—an absolute nightmare of adding permissions and all that kind of stuff.
I see that persona. I am that persona. And then I also see that you have an SDK, which seems to be geared more toward production apps. For those folks, I’m thinking, “Hmm, that’s interesting,” because how many people want to dynamically bring tools into their app? It seems like it starts to make the app itself potentially unwieldy.
On the other hand, I do see a lot of value in managing auth, for example, for 1,000 apps. That doesn’t sound like a lot of fun. If you can make that a simple process for developers, that sounds quite interesting. But I guess I see these profiles going in somewhat different directions, or at least getting the bulk of their value from different parts of what you’ve built. So, I’m interested in how you segment the market and what you see the primary value drivers being for those different profiles.
Yeah, that’s a great question. As you rightly pointed out, we have a bifocal product with 2 focal points. One is for the prosumer market, which is people using Claude Code, OpenClaw, et cetera, and plugging in Composio Connect. That’s what we call that product: a single MCP server inside all these agentic runtimes, anything like Claude Code, et cetera, whatever they’re using.
For them, the value proposition is exactly what you pointed out: you don’t need to go to each MCP server, plug it in, and understand the instructions for Google Drive, then Zoom, then Datadog, et cetera. You just get 1 MCP server, which is connect.composio.dev/mcp. It’s as simple as that.
You put it inside Claude Code, and then you can manage your authentication directly via Claude Code. If you ask it, “I want to connect with a new app,” it will give you the link. Or, if you like the GUI experience, you can just go to Composio’s dashboard and do it all there with managed permissions and managed scopes. There, the value proposition is simplicity and getting the power of almost anything at your fingertips.
On the other hand, on the developer side, everybody is building agents at this point, from startups to bigger enterprises. One of the biggest problems people face while building agents is giving them actual power to be able to do things. They want to connect their agents with actual knowledge-work apps, and that’s where we come in. We solve it.
I think the value that we provide there, other than auth and all the integrations, is managing the scopes. You can give whatever granularity of scopes you want through us, controlling it at the action level. We provide the whole harness because, at scale, everybody who’s building agents wants to create a similar harness, and we’ve seen that pattern work really well.
The pattern works really well for a bunch of agentic paradigms, specifically chain reasoning, which everybody’s building right now. The whole harness and its bits and pieces are available in full modularity. People who want to use just our tool discovery with Composio’s tools as well as their own tools can use that.
They might just want to use Workbench, which is our sandbox where execution can happen and where auth and everything are controlled by us. They can just use that; we essentially just call it that. So, all the bits and pieces—the whole harness—are also available on the developer side.
On one hand, we have the whole harness, where you can plug in that single thing via MCP, via API, or via SDK into your agentic system. We also have modular bits and pieces: if you just want to use one component, you can use that, and if you want to use another, you can use that. If you just want to use our tools, you can also do that, while you manage the whole harness. We just provide you the auth and actions.
The idea there is that people want things like governance, observability, and auditability, and that all sits inside Composio’s dashboard. We also have some amazing enterprise customers, which obviously gives you trust, because this is very critical data that you wouldn’t want to give to any company.
At this point, we have AWS using Composio and building their core agentic product on top of us. Zoom is doing the same. Glean is doing the same. Airtable, too. A bunch of tech-first hyperscalers are trusting us. That gives you a certain level of trust because they’ve already evaluated us for all the things that you would want to evaluate us on.
What are the big things that they want to evaluate you on that maybe I should be thinking harder about? I’m a pretty prolific tester, at least of a lot of products, and I’m increasingly mindful of not giving full access to my Gmail or whatever to just anything that I happen to sign into. But I do connect a lot of accounts to a lot of things over time.
What are the biggest and most vulnerable attack surfaces, or the biggest risk factors, that the big companies have beaten you up on already to make sure you’re solid, that the rest of us can take to the bank?
Yeah. First of all, I think providing least-privilege access control is important. I think we’ve done a pretty good job. You can define what action you want the agent to have, and the agent will have access to only those actions.
To start with, if you don’t want to give the agent write-email or send-email access, you can just give it read access. The same thing applies to Slack and all the work-related apps. That access control is pretty important for having granular control. That’s one.
Then, at the second level of control, we have a bunch of ways in which you can control what the agent can take action on via hooks. Before calling the tool, you can check what the tool execution is doing and create guardrails around it, such as a human in the loop. We have prebuilt patterns for all of these.
After calling the tool, before the agent gets the response, you can have those hooks see what the agent is doing and what the agent is going to get—what kind of data the agent is going to have. If you want to have guardrails around that, you can do that. All those types of guardrails are already present in the product. That’s the second.
Third, obviously, compliance is a big thing. We have all the compliance certifications that people want, like SOC 2, which makes them somewhat more comfortable. The fourth one, which is pretty valuable for enterprises, is that we also do self-hosting.
In some cases, you wouldn’t want to use our cloud, so we also self-host in the customer’s VPC, which gives them much more of a sense of breathing room there. In the case of AWS, for example, we have self-hosted Composio inside AWS.
Gotcha. Okay, cool. It’s a good rundown.
Let's talk about these sandboxes a little bit. The paradigm there is a little confusing to me. I'm not exactly sure how to think about what execution should and does happen in different places. Obviously, if I run Claude Code on my local machine, it is mostly running things on my local machine, right? All the Bash commands and stuff are literally happening on my system.
There are also some tools, like their search tool, that's built in and happens on their side in their runtime. Those can communicate back and forth: the result of that search can get sent down to my system, and the result of commands run on my system can get sent up to the cloud to be part of inference. But I imagine that gets kind of fuzzy or weird when people have a mix of things. How do they decide, or how do you guide people on deciding, what should be run in your infrastructure and what they should run on their own infrastructure? I assume it wouldn't make sense for them to try to bring their whole app into your sandboxes, right? How should I unclutter my own mind to think about, in general, who should be running what?
Essentially, in our sandbox, we provide a ton of utilities that make it easier for an LLM to write code on top of it. That same Docker image, of sorts, we're making available locally very soon. So if you want to use your own tools, internal tools, or local tools and want them to be available inside the Composio sandbox, you can also do that. That's coming very soon.
The idea is that, in the sandbox, the agent has to write very minimal code. The auth side of things and a bunch of other things—mostly auth, and some abstractions around tool calling—are already taken care of in the primitives that the agents get. The benefit is that the agent doesn't have to deal with a bunch of things it shouldn't, and it just writes very simplified code. All the things around it, like deciding what auth to use and converting things from code to function calling, are handled by the tooling that we're providing internally.
Got you. So the biggest value driver you're highlighting there is actually making code writing easier for the language model by providing—you said mostly auth. What else falls into that category of harness? I've certainly experienced what we mentioned: it can be hard to get these things set up. It's intuitive to me to say that if you could provide solid, code-ready auth so that the LLM doesn't have to recreate that all the time, that sounds like a clear win. What else is like that, where there's enough deterministic stuff that you've built out that it makes things a lot simpler for the agent?
File sharing is the other thing. Basically, we have mounted folders where the LLM knows that everything in those folders is part of the harness, which is in the description of the function and the system prompt. Everything put in those mounted folders is, by default, uploaded to S3, and we have shareable links available for those files.
Whenever agents want to share anything externally, it's very simple. They just move or copy files to that particular folder. More utilities will be coming as we encounter new use cases. File sharing is definitely one of the biggest ones. An LLM might go through 10,000 emails and generate a report, or go through all the Stripe activity and generate a report, and then want to share it with you, the user. We make it very easy for it to do that.
An LLM can write code that uses an LLM. We've made it very simple to do that. It's kind of like inception: an LLM is writing code that uses an LLM, but it's needed for processing 10,000 emails. Otherwise, how would you do that? So, utilities around that. All these are minor things, but overall they increase the efficacy and accuracy of agents to a significant extent.
Okay, cool. On the topic of discovery, or what I think you also call smart MCPs, I've been on the lookout for a while. When this MCP phenomenon first popped up, I was thinking, “Where are the smart MCPs?” At first, we just had this massive wave of people wrapping APIs in the MCP layer. That's fine, but it seemed like where this really gets helpful is if the MCP itself is smart in some way. It could take in not a very specific command—where the MCP call could have just been an API call—but something higher-order that might involve the composition of multiple tool calls or even multiple APIs working together to do some higher-level job.
Honestly, I haven't seen a lot of that. I asked around, looked around, and looked in repos to try to figure out if anybody was doing this, and there wasn't much. Now, you guys are doing that, and it seems like it's a pretty big focus of your value. I'd love to unpack how that is working, especially getting into the progressive disclosure aspect of it.
This has also come to the fore in developer conversations recently. First, it was MCPs taking over the world, and now we've heard a little bit of the trough of disillusionment: it's a lot of context bloat, and maybe CLIs are better. I kind of always end up thinking that one of my AI mantras is, “Everything is isomorphic to everything else.” Whether it's an MCP or a CLI, you can probably do progressive disclosure and avoid crazy bloat if you think about things the right way. Those decisions don't seem as sharp as they used to be, because there's so much room to create flexibility in the context of intelligent systems. Take me through the smart MCP paradigm that you're developing. What makes it smart, and how are you building it to make things as easy on the agent as possible and do as much for the agent as possible?
For sure. I mentioned earlier that if you give 1,000 MCPs to an LLM, it's obvious that we're hitting 1 million-token context windows. Maybe at some point that will increase to 5 million or whatever, but attention is definitely not free. The less you give it, the better the performance will be, and that's where just-in-time tool discovery becomes important. Your LLM shouldn't be overwhelmed by seeing 1,000 tools; giving it the right set of tools is important.
Given that we have more than 50,000 tools and most users want to use all of them—because it's dynamic, especially in the case of developers building applications, they want to give their users the power of whatever Composio has—the idea is that the LLM doesn't see all the tools. It sees a few tools, and then new tools are dynamically added to the context. That's one part of the smartness. The other thing I mentioned was background learning of sorts.
Whenever we see that a tool is not comprehensible by the agent—which means that the LLM is not able to understand what the tool does, or it's trying a lot but always erroring out—then, in the background, a new version of the tool is automatically created. When there are issues with the tools, a new version of that tool, which we feel is very valuable for that particular use case, gets created in real time and added to the context.
That's another level of smartness: we can create multiple versions of a tool really quickly and get a new version that might be more suited for the particular purpose that the agent is picking at this point. Another thing that happens in the background is that skills have gotten really popular. The reason for that popularity is that a skill is an abstraction level above tools, where you can have instructions for particular use cases baked into skills. They inherently use tools, but there are instructions and scripts that make them more robust and repeatable compared to just providing tools to the agent.
What we do is, when we have the whole end-to-end agentic traces, we convert them into a set of skills that are reusable and provide them during just-in-time tool discovery. It's also just-in-time skills discovery, of sorts. The agent already knows how to use the tool, what tools to call to achieve a particular outcome, and what code to write in the workbench or sandbox to achieve that outcome. The use case might be a bit different, so it has to use that skill but do a bit of fine-tuning for that particular use case and reuse that code or skill for its purpose.
That's the smartness: background learning, and it also includes failures. If there are some failures that we see happen again and again, we tell the agent about them beforehand in context: these are the pitfalls, and these are the do's and don'ts of using a particular tool or achieving this type of outcome. At the end, in my opinion, the harness is nothing but context. You have to engineer the context, and that's what we're providing: effective context engineering around our tools, which makes it smarter.
So, double-clicking first on that first one—the discovery of tools—what do those requests typically look like? I can imagine, and maybe you can answer this with specific examples from specific customers, or however you want, but I'm imagining that sometimes I might just say, “I want to connect to my Google Drive,” and then it's, “Okay, great. What are we going to do? We're going to get the Google Drive tool.” Pretty straightforward, and definitely still very nice to be able to have that as a natural-language interface.
Other times, though, I might imagine that I don't even know what I want. Is there a tool for this? How would I go about doing something like that? I'm interested in the breakdown of the kinds of requests that you get and how many of those there are today.
I think there's another version of this question, too, which is: as we actually get into doing the work, how much of the work is happening in ways where the user specified it? “I want to go call Linear and get some details and then put it over here.” You might think of that as automated copying and pasting, right? Stuff exists, and we're moving it around, versus really figuring out higher-order stuff, like, “I don't really know how to do this, but this is probably what I'm trying to accomplish,” and then the system figuring out what the tools and steps are to make that happen.
I guess that exists both in discovery and at the level of execution, but the key question is: where are we in terms of people defining what they want their agent to do and the agent following those instructions, versus people describing intent and the system really figuring out how to serve that intent, even if the person doesn't have it all mapped out?
Sure. I just want to clarify one thing there. We have a pretty intelligent mediator in between, so the direct user request is not what we get. We have Claude Code sitting in between, and that sends the request. It's a very intelligent mediator in between that navigates the user request and sends the right level of intent to the tools.
In most cases, Claude Code already knows the power of what Composio can do. It figures out, “Okay, if the user asks, ‘How do I connect to Google Drive?’” it will directly call “manage connections,” which is our auth-management tool, to do that for the user. It won't give a search intent to our tool-discovery tool. That's where I think the intelligent mediator—the LLM or agentic runtime—comes in between.
To answer your question, I think it's becoming more and more intentful. We've all seen how people are using OpenClaw in the world. I think December was a big shift, where people realized that these models had gotten to a level where much more is possible and they can trust them much more than before. That has happened across all domains, in my opinion.
Engineering—obviously, software—was the first one to bite the bullet, but I think it's happening more and more across knowledge work, where people are becoming more intentful with these agents. In a lot of cases, they just give the outcome they want to drive to the agent and let the agent figure it out. We're the harness that provides all the right tools for it to be able to do that, but the agent is smart enough to send the right intent with the right tool.
Can you maybe give some examples of that—instances of people with intent that then gets mapped onto the tools? Intuitively, it seems like a lot of them would come from individual Claude Code or OpenClaw users, but I'd be really interested, too, if there are examples where that is actually happening in an app that an app developer has running in a production environment.
I don't know if we're getting there yet, or even if we want to. A lot of these things feel like, “What's the app at that point?” That's an interesting question. But give me some examples of intent that you've seen resolved into actual successful execution.
At this point, people are so confident that they give the agent access to their whole Gmail account and ask it to go through the last month of their email and archive all the emails that don't seem useful. The agent goes and writes code to do that, which uses an LLM to figure out which emails are not useful.
The other use case is that I use my OpenClaw for hiring. My OpenClaw goes through a lot of GitHub repositories, finds good commits from individuals, and creates a pipeline specifically for engineering hiring—for me to hire from good open-source agent-tech repos, Python repos, TypeScript repos, and so on. It figures out the best contributors and specifically gathers all the enriched data about them, such as whether they're based in SF or elsewhere, their email addresses, LinkedIn, and all their social data.
I've also given it its own email, so it reaches out to them on my behalf.
So it’s like end-to-end hiring—a recruiter job that I’ve literally offloaded. There have been thousands of really good folks it has emailed, and in the last week or two I’ve gotten, let’s say, 30 or 40 calls set up. That whole thing is fully done by my agent and Composio.
Yeah, I think that’s the idea, right? A lot of exploratory and actually end-to-end knowledge work is getting offloaded to these agents, if that answers your question to a certain extent.
Yeah, I mean, if it gets up to sort of job-scale things, it starts to be both pretty intuitive, I guess, what that ultimately looks like, and also potentially quite transformative for many aspects of life.
I like that. I feel a sales agent is doing something similar for sales. Obviously, he’s not emailing them because they’re very directly emailing them via the agent, but the agent has drafts ready, which they just need to press the send button.
That’s the level where we are. It kind of does the whole sales job: figuring out the people it needs to reach out to, the right people who are building, let’s say, agents in different companies, and so on. It has all their data ready—email, LinkedIn, and so on—and then he just has to press the send button.
So how do you make sure that those agents have the right contacts? This is something I’m also thinking about right now. One of my favorite things that I’ve set up—and I can’t stop talking about this, so I’ll keep it brief, because I’ve probably talked about it on a few episodes already—is that I have exported to a local database basically the last 5 years of my communications.
All email is exported out of Gmail, all of Slack, basically every DM platform that I use, and all of my calls, which I’ve been recording for the last 3 years, transcribed. Everything is organized into threads. A Gmail thread is a thread, but a single call would also be a thread, and each statement back and forth between the people is like a message, as you know, corresponding to the emails back and forth. Even the podcasts, which I break down with the transcript, I put in there in the same way.
So a lot of the communication that I’ve had—probably the majority—is now in this database, and it is extremely helpful for getting the context that is needed to understand: Who is this person? What is my relationship with them? Do we have projects that are ongoing? If I started a project, who was involved with those projects? How did that evolve over time? Whatever.
I’m a little reluctant to throw that into some random cloud container that I’m just testing out, right? So that still only lives on my local machine. This brings me back a little bit to the containers and the context, and what should sit where.
I assume, just like me as an individual, you as a company have a lot of information about what makes a good candidate—who the good candidates were that you had before, good examples, bad examples you would compare them to. It’s endless, right? How should people think about sending these agents off to do long-running things, making sure they have the context that they need, but not putting themselves too much at risk?
My personal database is already a gigabyte. It’s my whole life in there, right? So I do need to be a little mindful of where I send that around or what ports I open up to access it. I don’t even know what all is in there, right? I’m sure I’ve emailed myself credit card numbers and done all sorts of stupid things. I’ve emailed myself passwords, and recovery codes are probably in there. I’m sure I’ve done all sorts of stupid things that could come back to bite me.
How do you think about that balance between making sure the context is there so they can succeed, but managing the associated risks?
That’s where I think managed access and least-privilege access control come into the picture. The idea that we at Composio have is that, in the future, you’ll have not just a single agent, but multitudes of agents. The idea is that you’ll have different profiles and access controls for each agent.
Some agent will probably have only read-only access to your data, so that they can’t do any malicious sending or execute anything. But they need all the data, and you’ll want them to be very self-contained because they’re sort of research agents. They have a lot of data, but they don’t have the permissions to divulge that data by mistake.
You’ll have very tight access control on what they can do, but they’ll have read-only access to everything. That’s a profile that you can create inside Composio and manage—what access permissions you want.
There can be another profile where you’ve given an agent a lot of writing permissions, but that agent has very limited personal information or company-wide information, just because what if that agent emails some secret token by mistake? There, the granular access control is: you want to give a lot of write control, but you either have things like a human in the loop to guardrail what the agent is doing, or you keep it very controlled.
These are the types of different profiles that will exist in the future. You would create different OpenClaws, different Claude Codes, and different Codexes to do a mix of it or all of it.
Let’s talk about the continual-learning aspect.
Mm-hmm. I think it sounds like a huge value driver, and it sounds like probably a necessary one for Composio’s success. I don’t know if you would go that far, but what I see is that the barrier to spinning up a new tool is certainly dropping. The value of having seen a ton of uses of that tool and being able to figure out what the actual effective pattern is—that’s something that’s not going to be easy for people to recreate on their own.
If you can make a step-change difference in the results people can get, even if they spin their own tool up real quick, versus, “Okay, now we’ve seen 10,000 uses of this and know what has actually been effective,” that strikes me as the moat. We’re all searching for the increasingly elusive moats in the AI space.
I’m interested in how it works across users and how it works across apps. You described how it works in some detail already, but the philosophy part of it is what I’m wondering about. There’s obviously some sort of depersonalization or anonymization aspect to it that I’m sure is critical. But if you find an upgrade, does everybody automatically get that upgrade, or do you have to subscribe to upgrades?
Maybe the upgrades you’re doing are, in some cases, very specific to an individual or a particular app, such that you feel like you can upgrade it for them, but you don’t have to change how it is for everybody else. I feel like I both want those upgrades as a user, but when I do have something that’s working well, I’m a little bit afraid of those upgrades.
I’m afraid of that even with the model makers. Obviously, they’re leapfrogging each other all the time. If a new model comes out, it might be better, but is it going to be better on the thing that I already dialed in to my satisfaction? Maybe, maybe not. That all seems quite fraught. What’s the philosophy that guides you in figuring out who gets what upgrades?
That’s a great question. We think about it a lot internally, by the way, and that’s where we’ve designed our infrastructure to make it very easy. As I was describing, we have multitudes of versions of tools. There might be tens of thousands of versions of a particular tool. Of the same tool, you might have tens of thousands of versions.
The idea there is that there are some personalized upgrades. We see that you’re using some particular tool in a particular way, and the tool can be better specifically for your use case. We’ll have the upgrade only for you.
I don’t fully buy that fear argument, just because I think we all know the models are changing and the model behavior is changing every other day. The way you can control the behavior of the model, even when the model or some of the tools change, is by having skills. That’s where we’re very, very thorough about how we change the skills that we’ve developed for users.
That doesn’t change that often. That’s kind of the fixed level of repeatable behavior that you like—the way you like something. That’s ingrained into your skills, the personalized skills that are created in the backend by us.
The tools themselves keep getting better and better. I think you were talking about a moat, right? We have a gazillion instances where we’ve seen that the docs are totally wrong for a bunch of tools. Because we’ve gone through so many agents using our tools, agents use tools in insane, different ways.
Compared to the previous generation, when humans were using these APIs and tools, you hit a new edge case every now and then. That just makes our tools better and better because we’ve seen it across the board. Last week, we found a bunch of cases in Google Calendar where our tools are much better.
It happened autonomously. We didn’t even get to know in some cases, but our tools are much better than what the docs propose. And that’s not true for just one app; it’s true across apps. So I think that’s a moat which we have developed because we have gone through so many agents poking holes in different apps and making our tools better.
Those upgrades are available all across, right? Why would we want those upgrades to be available to a particular user? If our tool is getting better and better, that should be available across the board. So that’s how I think about it. Basically, there are some improvements which should happen all across because they are very generic, and some skills should be available across because they’re about how agents should use particular tools. Then some are very personalized to your use case.
You said something I thought was quite interesting, and I’m not sure everyone would agree with. It was that skills basically tame the models, right? I forget exactly how you said it, but you sort of said models are always changing, of course, but that’s where the skills come in. Once you’ve really defined a skill, it sounds like you’re of the opinion that you can swap out models underneath and get pretty consistent behavior, even across models.
Obviously, I know there are caveats with that, in the sense that you can’t massively downgrade to a 1B local model and expect to get frontier performance. But if we take a narrower understanding of that statement and restrict ourselves to frontier models, or whatever reference class you want to use, that’s still pretty interesting. I think a lot of people would say they don’t feel confident in that. But how confident are you in that statement?
How confident are you that the implication would be that all the frontier models are good enough at following instructions that, if your instructions are really well built out, then they become kind of interchangeable? Is that a good summary of your view?
In most cases I’ve seen, that’s kind of true because the skills are detailed enough to give a decently granular description of what you want to achieve: the path, the trajectory that the LLM or the agent should take to achieve a particular outcome. If the model is good enough and follows instructions, in most cases, the behavior of the model, or the trajectory of the model, remains consistent.
That’s a known pattern that a lot of research and industry are also seeing, where you get Opus to create a skill and then swap to Sonnet for using that skill. The first time, Opus, being smarter, is better at navigating, figuring things out, and reaching the outcome. But once you’ve done that and have the skill which Opus created, you can swap to a cheaper model and achieve a similar outcome.
Do you think that also holds true when swapping across frontier model providers?
Sometimes not. I think there are behavioral patterns across different providers which make them different. Specifically, I have some examples that I see day to day. Anthropic models are somewhat more agentic in terms of things like this: if there are some tools which require polling, then they will wait. They will write code to wait until, for example, you have to poll and then continuously poll.
I think that polling aspect of agentic polling is much better ingrained in Anthropic models somehow, and GPT just stops. It waits for user input after that. Those are some behavioral patterns that are different, which makes some of the skills—the way GPT will use those skills or would build those skills—a bit different from Anthropic’s models. But to a majority extent, except for those nuances, I think this holds true.
Interesting. Do you, in practice, advise people to do—I guess what would you say is the max-efficiency play, and do you recommend it? If I go develop a bunch of skills with Opus, what’s the cheapest model that I could trade down to that you would expect to work some large majority of the time?
For sure, Sonnet, because I do that regularly. In some cases, I write my—for example, I use skills a lot. Locally, I’ll write some skills for the first time via Opus, but then trade off for speed to Sonnet, and that works phenomenally well. I don’t do it with Haiku because I’ve seen that it doesn’t work really well.
I’ve tried some with GPTs, and 90% of the skills just work. There are 10% of the cases where there are some nuances present in the skills because of the model that wrote the skill and how it operates. They’re not exactly plug-and-play, but in 90% to 95% of the cases, it just works out of the box.
Would you expect Gemini Flash to hit that level as well?
I think so. I think Gemini Flash is decent enough in this, so if not now, maybe the next iteration should. I haven’t tried it myself, so I can’t comment directly, but I’ve used it in a production setting where it feels very smooth.
Skills as the re-commoditization layer: what I think I would say the prevailing narrative has been recently is that the models are starting to diverge—not exactly in capabilities in the macro benchmark sense, because they’re all kind of climbing the same curve, obviously—but they’re diverging more in qualitative ways that are hard to wrap your head around, thus creating some stickiness and some pricing power for companies once they get people using their stuff. I think there’s been an increasing sense that it’s harder to switch off.
But you’re making a provocative—I mean, not deliberately provocative, but it’s definitely provoking thoughts in me—that the boomerang might actually come back because all these skills are getting so thoroughly defined that you don’t necessarily need great judgment in a model to do it. You just need good instruction-following.
That’s exactly the same. That’s where we actually position Composio as one way to avoid being locked in, essentially, to a model provider. If you use Composio’s harness, you can use it with Anthropic, with OpenAI, and with open-source models.
Tomorrow, let’s say today you’re using Anthropic. OpenAI is gradually improving on a bunch of these things. If you want to switch to OpenAI, you can have all your auth and all your skills in a single place, make that switch, and get 99% reliability with the switch. After you decide that open-source models are becoming equally great and are probably 10 times cheaper, if you want to go there, you can make that switch and still continue to work with 99% reliability.
Something I’m still in the middle of doing right now is taking the article that—I think it was Thariq. I don’t know the person personally, but a member of the Anthropic technical staff put out a post just in the last 48 hours or so that was very well received. It was like, “Here’s how to use skills. We’ve learned a lot; here’s what we’ve learned.”
I didn’t even read that. Instead, I just copied and pasted the whole thing, put it into Claude Code, and said, “Here are some best practices that I just heard were popular, and people say are giving good results. Can you go apply these—or talk first, go into plan mode first, and tell me how you would think about applying these to all the various skills that we’ve worked on together?” Naturally, it has a lot of good ideas for what to do there.
I wonder if you guys would—maybe you already do this—but if you’ve already invested or might think about investing in skills specifically for translating skills from provider to provider, because I can imagine that 90% to 95% could easily become 99% if you applied another layer of transforming and compensating for the known quirks. Is this something you already do?
Yeah. We are developing a bunch of metrics and benchmarks around this stuff, and we already do some of it. Nobody, specifically in enterprises, wants lock-in because they want that optionality of moving across providers. In the current AI landscape, you actually never know who the winner is. It changes very often. Today it’s Anthropic; tomorrow it’s OpenAI, Google, Chinese models, or xAI.
We want that optionality because if you completely lock in—and these skills specifically, in my opinion, are, to a certain extent, very addictive—then on the other hand, as you said, if the skills have those behavioral patterns ingrained in how the model works, they can be harder to change because they’re not structured.
That’s where I think the 95% is very easy—90% to 95%—because most of the models are at that level now. But reaching that 100% mark with these skills, which are not structured, is actually a very hard problem, and that’s something we’re already trying to solve. We’ve made it somewhat better, but we want to solve it to probably 100%.
Yeah. Okay, that’s really interesting, and I do feel my worldview changing a little bit in real time here toward expecting a little more commoditization and less pricing power. That also brings to mind another angle of thinking about lock-in or moat or whatever, which is memory.
Although again, I think there are ways to get around this. Claude recently did this thing where they said, “Go ask ChatGPT this, then paste the answer over here,” and you’ll pick up right where you left off in terms of memory. So that, too, is not really a great moat, although it’s debatable.
But in looking at all the tools that you have and just browsing through them, I was struck by one thing. Of course, there are many tools that are relatively simple API wrappers, where APIs that have existed can now be used by agents. That’s sweet—a lot of automation power that way.
But then there’s another class of tool that is built for agents in the first place. Memory is one example, where I saw you have Mem0, Zep, and probably some others. Those are things specifically designed to unhobble the AI, or enable the agents, if you will.
There are also things that allow agents to transact, whether in fiat or cryptocurrency or whatever. Tell me, what does that landscape look like? What are the agent-enabler tools that are actually important, actually working, and maybe even strategically important?
Memory could be one. I can imagine that decoupling memory from your model provider, if it works well enough, could be another way to insulate yourself from lock-in effects. I’d love to hear your survey of this new generation of tools built specifically for AI.
Yeah, I think we’re very bullish on all of these agent-enabler tools, like you mentioned: memory-based tools, Mem0, Supermemory, Zep, and payment-based tools like Skyfire. We also have traditional e-commerce ones, like Shopify, which are almost now agent enablers in that sense. They’re all gearing toward it.
We partner with all of them, and all of them partner with us, because we want the people building on top of Composio to have access to the best-quality agent enablers, to improve the ecosystem. A bunch of good ones are already there, and more are obviously coming day to day. Our position is just to give our customers access to all the best-quality ones.
I don’t think I have any favorites there, per se. We like to give all of them to our users and let them make the choice.
Are there any classes of these things that you find to be particularly powerful? I understand that you can’t necessarily pick favorites among your partners, but are there categories that you think are especially important?
I think all of them are getting decent usage. Memory is obviously a big one; everybody wants to use that. Even on the payment side, there are tools like Skyfire for different use cases, and a bunch of these things are getting increasingly better adoption given the OpenClaw movement.
Specifically, a lot of background agents are running and want to do commerce-related tasks. Search is obviously one of the big use cases, where people use things like Exa, Firecrawl, Tavily, and so on.
It’s broad, depending on the use case and the problem statement that the builder is solving, or what people are using in their OpenClaw or Claude. All of them, or a mixture of them, are being used heavily.
Are there any missing categories or missing tools where you think, “Why has nobody built this yet?”
There are so many things coming up at this point that I might have to think more to come up with an answer here. I know people are building every sort of thing that a human does and needs. There’s agent-to-human delegation happening, along with a bunch of other things.
I don’t have an answer here out of the box. I would still say that most of the users are coming from traditional software, because that’s where all the users’ datasets are. Things like Slack and Salesforce are still the major ones, because they’re the systems of record.
Perfect transition. I’m interested in your take on those platforms and which ones are perhaps advantaged, or at least safe from major disruption by the AI wave, versus which are more likely to be under threat.
People are, of course, tired of paying their Salesforce bills and their Slack bills. In my mind, there have been some interesting but confused debates about this. One person will say, “I had my coding agent spin up a Slack clone, and look what it did.” Then another person will say, “That’s nowhere near what Slack really has to do. Think about all the complexity that Slack really has to handle.”
I always think, “Sure, but for that one user and maybe their small company, all that Slack complexity was irrelevant anyway.” It’s just a bunch of stuff they’re paying for that they’re never even using.
I’m not really sure where that settles, though. Obviously, there are a lot of different categories, from collaboration to task management to customer service platforms of all kinds. What’s your take—with the obvious disclaimer that this is not investment advice—on what you’re buying and what you’re selling based on what you’re seeing?
I think there are two sides to that. One is that, in my opinion, the core infrastructure layer is getting much, much stronger. As you rightly said, making software is getting easier, so a lot of software will get built on the core infrastructure pieces. Your dependence on software is just going to increase more and more, because now you’re chatting with your agent more than you’re chatting with a human to do some task.
Your dependence is increasing, so the core infrastructure that’s driving it is getting massively stronger. Things like AWS and Cloudflare are all base infrastructure, because it’s very hard for anybody to rebuild those infrastructure pieces.
On the SaaS side, the way I’m thinking about it is that the interface through which you use a lot of these SaaS apps is going to change. In some cases, you can build a smaller version of a bigger SaaS app for your particular niche use case. But in most cases, what will happen is that the interface through which you use a lot of these SaaS apps is going to change.
Startups will build those new interfaces and pose competition to the SaaS apps. In this particular wave, though, all these older SaaS software companies, like Salesforce and Slack, are also pretty fast to catch up, and they’re coming up with new agentic interfaces to operate through.
I think it’s about who will be fast enough and innovate. At Composio, we’re working with startups, and at the same time, we’re working with incumbents building new agentic interfaces. It’s all about who is fast enough to build the new interface for their users to operate through.
I’ve been back and forth on that myself a little bit. One paradigm I had bought into, and I think I still mostly do, is that companies like Salesforce probably will not be disrupted by startup CRM competitors. Sure, you can go build an AI-first CRM, and it might be sweet, but it will still take a long time to sell to the big customers.
By the time you can win a lot of that business, Salesforce will figure out how to clone your best features, close ranks, and prevent you from taking too much of its customer base away. So if I think about how much of the value of the AI wave accrues to incumbents versus AI-first challengers to those incumbents, I lean mostly toward incumbents.
The flip side of that is: What about people bringing things in-house? Another example I’ve been thinking about, because I actually had a chance to study this a little bit, is Intercom.
They have done—I think especially in the last week, there’s been a bunch of praise going around for Intercom—what many people see as some of the best adaptation. I’m not picking on them by any means. In fact, they provide a really strong example of what it looks like for a company to catch the wave and ride it successfully. They also had a past guest on the show, by the way.
One of the big things they’ve done, of course, is create this Fin agent, which basically has the ability to resolve, I think when we spoke to them, close to 70% of all customer service tickets across many thousands of customers. It’s 99 cents each, with flat pricing.
Okay, cool. That’s sweet.
It sounds like it’s driven a huge growth boom for them, and it’s easy to see why. If I’m paying humans to do this and the AI can respond 24/7 instantly, that’s a huge advantage.
I looked into our data, actually. My company is an Intercom user, and we’ve prided ourselves on customer service. But the speed of response that the Fin agent gives us is something we just can’t match with our staffing size.
So, okay, it has all these examples—or advantages, I should say. But then I looked at the Composio tools associated with Intercom: 133 tools just for Intercom. I'm not sure about this, but it sure looks to me like I could do literally anything I wanted or needed to do with the entire Intercom platform through those tools. And then that got me thinking: well, maybe this Fin agent is actually going to become a skill for me.
Rather than pay them 99 cents each, what I really need to do is dial in exactly what I want. I can probably do that better by owning the skill on my side versus trying to populate it on their side. Do it all through the tools. And I probably save—I don't know—90%, right? I mean, imagine it would be like 10 cents in token cost.
So what's your take on that? Do you think that people will start to cannibalize these agents? They're awesome, but it seems to me like it wouldn't necessarily be that hard for a lot of companies to say, “Fin's doing well, but they're not doing some things quite so well, and I've got all these tools. So give me the 133 Composio tools. Let's work on our own skill, and we'll get this thing better than Fin was at 10% of the cost.” Is that realistic? Why wouldn't that be realistic?
So, by the way, Fin is a great product. We used it early on, when we were just launching the prosumer side of Composio, because obviously you have a ton of support that comes when it's a prosumer product. We used it early on, and it's really easy to get started. And that's what they're solving, honestly, in my opinion: a lot of companies don't want to spend a lot of time; they want to get started and offload this to someone else. Fin is great for them.
I think where you made the right point was, if you want a lot of customizability, I think more than cost—cost is obviously a big reason—but the customizability and governing your agent while building gives you that freedom of what you want to do. You can create multiple skills. You can specifically limit your agent to particular tools. You can give it access to other apps that Composio has while building that support agent. I think that customizability is what people prefer when making that build-versus-buy decision.
And that's where I think Fin is a great product for a lot of people, by the way, and that's why it's doing great. But there will be some people who would want to customize and make the agent more powerful, and that's where I think we come in and provide those 100-plus tools for Intercom, and you can do whatever you want.
Yeah, I agree with you that they've done really well, just to reiterate that point. But I do see the world where friction keeps getting so low. I mean, in part because you've already got the 133 tools, in part because somebody can publish their skill, and in part because I can give that skill to Claude Code and say, “Hey, interview me about my use cases and how I would change this skill to suit me,” and then it's already plugged into all the 133 tools. And then I'm just like, wow, the barriers are really crumbling everywhere I look. It seems like the barriers are getting pretty easy to overcome.
Bottom line, if you're projecting—and nobody can project 5 years into the future; even 2 years is a long time—but do you think a year from now, would you expect that a lot of companies are actually trying to realize these gains and say, “Hey, we were spending a million dollars a year on a million tickets with Fin, but let's do our own thing and try to save 80% or 90% with custom skills”? Do you think that will be a movement?
I don't know if it will be a widespread movement. That will happen in bits and pieces. But for some companies, I think spending that much time is probably not worth it. For some companies, it is worth it. So that will be the idea.
But I agree: I think the friction will keep getting lower. We'll make sure that, from our side, the friction keeps getting lower and lower. I think, as you rightly pointed out, there will be skills around it. I think that definitely, as these models are getting better and better, people will inch toward build compared to buy in the future.
What's cool about agent-to-agent? So far we've talked about agent-to-tools and then agent-to-smart-tools. And then I think smart tools start to look like agents in their own way, right? Again, I think everything is a little bit fuzzy. The boundaries between a tool, a smart tool, and an agent are not always super crisp.
But maybe when I think about agents, I guess I think about representing someone's interest. If I was going to try to venture a conceptual difference between a tool and an agent, it would be that the tool is exposed for me to use to serve my own interests, and the agent is something that may represent somebody else's interests, but I can still interact with that agent. You may offer your own definition. How are you planning for agent-to-agent as part of the future of Composio?
Yeah, for sure. We think about it a lot, and I have some models for how I think about a multi-agent world, or agent-to-agent. By the way, we have a bunch of tools at Composio that are agentic. Essentially, there are a bunch of tools where you can do almost anything in natural-language format on a particular app. That's inherently, internally, an agent that does that thing for you.
I think there are some places where agent delegation works really well. In some places, it doesn't. In a lot of places, we all know the main agent has the full context about the use case and about what the user wants. It has the full set of tools to figure out anything else that it wants or needs. So that's where I think the power of giving the right set of tools to the agent, if you're not overloading the context, is generally better.
To give you an example, if, let's say, I have to book an appointment with someone, and let's say I have a sub-agent that can do that for me. But my main agent has all the right things, like my calendar, to figure out if I have a collision at that point or not. My sub-agent probably doesn't have it. It's a sub-agent of, let's say, an appointment booker or whatever, right? If I just delegate the task to the sub-agent, then it's possible that it books an appointment at a time when I have a conflict with some other important board meeting, for example. And then I'll be in a kind of “What happened?” situation.
That's where I think the main agent, because it has all the context—my calendar, this, that—will do a much better job if it is just given a tool. So the way I look at it is, if there's, in general, a 1%–2% context task—this is not a huge-context task, right? It doesn't overload your context a lot—then it's better to conveniently provide it as a tool.
And if it's a more exploratory task where a lot of context will be used, for example, deep research—we all know in most companies, parallel sub-agents run in different streams and do research on a particular topic, condense the results, and give them to the main agent. In those cases, where it's more exploratory and takes a lot of context, it's better to use sub-agents. In a lot of use cases, it's better to just bring it back to the main agent and give it the right set of smart tools.
Yeah, so there's no single answer here. It will be a mix and match based on the problem statement and what's best for that. Are you seeing anything meaningful today in the agent-to-agent space? Any examples you would highlight? It has felt largely to me so far like it's still very much theoretical, right? There's been a lot more talk of agent-to-agent than I've seen actual agent-to-agent happening.
I think Claude's team of agents is a very interesting paradigm of what they've done, right? There's a shared task list that all the sub-agents have, and they can map onto the shared task list. They can do inter-agent communication via that shared task list, where they assign someone else a task that they want another agent to do, et cetera. That's a pretty good shared-agent paradigm that I've seen in the case of Composio.
As I mentioned, we have some tools at Composio that are exposed as agents, which are somewhat like, “Okay, you can do a to-and-fro as well.” You give the task to the sub-agent tool, which is essentially natural language: “Go and find this” or “Go and do this” in pure natural language on this particular app. Then, if it requires something, it will give it in response, and then you can use that sort of session ID to control the conversation again and again, to and fro.
So those are some paradigms, but I think it's still very early. It is in production. Claude's team of agents is available. They have an Agents API; a sub-agents API is coming. We have some preview version of it available where you can manage sub-agents. The agent can manage sub-agents.
Internally, we open-sourced this thing called Agent Orchestrator. A lot of our internal agent engineers use a single orchestrator agent to manage 20 to 30 Claude Code agents. That single orchestrator is figuring out what different agents are doing, whether any action is needed from the engineer or user to control these agents.
So those are some interesting paradigms that people are using. What does your cost structure for running Composio look like in terms of human cost versus token cost? How is that shifting, or how do you expect it to shift over time?
As I mentioned, all of our integrations are actually built by agents. The engineering team is building agents to build those integrations. In that particular setting, 2 to 3 years ago, that was not the case. I think everybody had a big-ass team, specifically the tool providers, to build these agents.
We have literally a 3-member team doing it all, setting up the whole agentic pipeline. Over the last month, we probably spent $100,000 on the pipeline that builds those agents. So, to answer your question, our token cost is definitely much higher than our human cost right now.
Wow. So you have a 3-member engineering team? 3 humans?
Three people are on the agentic team. Overall, we have around 15 people. But the team that builds the whole end-to-end agentic pipeline—that builds, accesses, and improves tools over time—is just a 3-member team.
And do you see the engineering team growing substantially in the future? Is there anything that will require a lot more headcount, or is it just going to be a lot more tokens?
I think at Composio, we are definitely hiring. It's just that our bar is too high. Sometimes I feel that we're tired, and that's why we're not able to hire faster. But we definitely need humans to control the agents. I think we're still not at the point where agents work fully autonomously without any supervision.
I mean, we're getting a lot of different predictions around what the future of software, and the future of the software labor market, looks like. It sounds like you think token cost will grow faster than human costs. But do you think the size of the human team levels out at some point? Or does it—what's the sort of scaling law of humans at Composio?
Yeah, very honestly, we're a startup, right? So I think in startups, the human-versus-AI scaling laws operate a bit differently because we're already very AI-first in that sense. But we still need humans to make decisions in certain cases, and that's where, in our case, we're hiring.
But we're also seeing what's happening across the board at bigger tech companies. In our case, LLM usage is already a multiple of human capital. LLM capital, or token spend, even for internal development, is already multiple times human capital. And that's where, in our case, we need more humans to spend more tokens. That's the idea.
I see.
That's not true for incumbents, where we know it's not the case. There, I think human capital is much higher than tokens spent. To answer your question, as models improve, that ratio will definitely move toward more token usage compared to human capital.
One other question I had for you in terms of business strategy connects back to the agent concept as well, although it doesn't require the agent paradigm. Why not resell more? Why not try to control the customer relationship more? Obviously, in some cases, I have a Slack account. That's my Slack account, and I want to keep it in my Slack account. I don't want to use Slack through you. That wouldn't really make any sense.
But then there are other things, and I'm thinking of things that I tried with Composio's Brandfetch, just to get logos of companies or whatever that I need—their color schemes—or Perplexity, Brave Search API, or xAI, which you mentioned. With any of these generic utility-style APIs, as far as I could tell, the only way that I could connect those accounts was to give an API key that I already have for those accounts.
And that got me wondering: why not just take the money yourself and have your own big Perplexity bill that your users can pay their way through you? From my perspective, it seems like that would be to your advantage and might also be a nice friction reducer for users, because I thought, “Okay, well, I guess I've got to go get my Brave Search API key now.”
But if I already had a payment method on file with you and it was, “Oh, sure, I'll enable Brave Search. I think it's $5 per 1,000 calls or whatever,” you could even charge me $6 for convenience, and it seems like it would work. Is that something you think you will do, or is there a reason you're not doing it today?
Definitely, that's a place where we are moving. Right now, we do have a bunch of services bundled into our paid plans, but pretty soon we're launching something called premium toolkits. What you exactly mentioned—a single wallet with Composio—can give you access to whatever you enable, all these services, via a single place, a single dashboard where you can enable them.
That's something we'll probably be launching by the time this episode comes out, or in the next couple of weeks is the plan. You can set up your credits at Composio's end and use all these services, so that you don't get overwhelmed by maintaining so many accounts and different billing at different places.
Yeah, okay. Cool. I look forward to that. I think these are pretty much all the angles that I wanted to cover. What have I not touched on that's on your mind that I should have thought to ask about already?
No, I think one of the things that a lot of people on Twitter are talking about is MCP versus CLI. I think that's a pretty heated debate right now, specifically with GitHub CLI. I have my viewpoints there. It's interesting because it affects us a lot, and we're actually, as I mentioned, just the reliable tool-execution layer.
That's why we're launching a universal CLI next week, which is the last week of March. I don't know—it depends on when the episode comes out. But with a single CLI, you can access all the different apps. You don't need a GitHub CLI just for GitHub, a Vercel CLI for Vercel, a CLI for that, et cetera. It's a single CLI that can manage all your apps with a single point of usage.
Is that fundamental? I mean, it feels to me like everything can sort of be patched. Couldn't you create hooks on the CLI? I had a question on this in the outline, as I'm sure you originally saw, but as we were talking, I came to the conclusion that maybe this debate is much ado about nothing, in the sense that, in the end, they can both work.
They may have some relative strengths and weaknesses now, but as they mature, it's kind of two sides of the same coin. That's where I think we're headed. Would you dispute that, or what edits would you make to that outlook?
No, I agree. I think that's where I was going. It's not going to be a unipolar world. Both of them will coexist. I think one of the deciding factors will definitely be where more and more tokens are being spent, because that will go into the agentic traces and be added to them more and more, and that traceability will improve over time. But I think it will be a bipolar world.
Got you. Yeah, makes sense. Anything else we should touch on? Anything else you want to make sure people know about Composio before we break?
No, I think we're hiring in SF. People listening, if anybody is interested in building the future of agentic tool execution, I'd love to talk.
Perfect. Thanks.
Thanks, Nathan.