纽约AI:Tenex如何打造AI原生工程团队——Alex Lieberman与CJ Hess
- 最值得交易者关注的数据,是Alex Lieberman给出的企业采用漏斗:在10X接触的约100家企业中,90%已部署通用大模型Claude或Codex,约60%提供培训和使用指导,约50%已部署编程智能体,而不少AI Engineer参会者猜测的比例是70%–80%。非工程类智能体进入生产环境的比例则降至约5%。Alex认为,这是原始智能与企业实际应用之间最大的落差,也印证了10X的判断:技术扩散的最后一公里很难走。
- 10X按产出付酬的判断大体经受住了检验,但实际模式需要更细致的设计:Alex表示,团队并未遇到人们担心的虚报故事点、交付劣质代码问题,AI策略师的激励则与客户留存和增长挂钩。最大的薪酬难题,是如何计算工程师在架构设计、客户会议等非编码工作上的贡献。收入能力最高与最低的工程师之间差距约为4倍,而不是10倍。团队至少每季度重新校准一次故事点标准;一位嘉宾表示,理论上模型越强,故事点的价值就应该越低。
- CJ Hess估算,从12月“Opus 4.5时刻”到如今的Opus 5.5,工程师产出大约翻了一倍;他称Opus 5.5是“了不起的模型”。Opus速度快、成本低,覆盖了他过去需要用两三个模型才能完成的任务范围。团队经常让它与Codex搭配,由Codex审查其工作;10X的“元框架”将计划、决策和智能体日志保存在代码仓库里,因此上下文“永远不只在某个人脑子里”。
- 随着团队扩张,Token成本曾令人担忧,但这种焦虑已经缓解。CJ估算,10X的Token实际支出约为员工支出的1:1至2:1。swyx的基准数据显示,不少公司每名员工的Token支出为10至200美元;AI初创公司的Token支出则从与员工支出大致相当,到后者的5–10倍不等。CJ表示,单项任务成本下降后,想靠“把Token用到极限”来严重拖累公司,比6到8个月前难得多。Alex的表格测试是:“一个聪明10倍的模型,真的值得我们为移动表格里的几个单元格付出这笔成本吗?大概不值得。”
- 本期最尖锐的分歧在垂直AI上:Alex总体看空Harvey、OpenEdmunds等垂直AI,认为企业买家期待高度定制、独一无二的方案,迫使供应商组建FDE团队,令软件毛利率承压。swyx则反驳说,专有数据、专业功能和集成能力可以支撑垂直产品;Bloomberg的价值就不止于提供市场数据。Alex指出,人们也在为Bloomberg的社交网络付费;swyx则认为软件整体市场会扩大,企业之间仍将争夺市场份额。
- Alex认为,Salesforce和SAP很难被替换,因为替换失败可能让数亿美元乃至数十亿美元面临风险;中端SaaS则要具体问题具体分析。swyx举了“Kill My SaaS”的例子:一家供应商报价每年4万美元,替代方案只需1000美元,而且即将投入下一场大会的生产环境。重建系统的理由不只是省钱;swyx表示,客户若能以分钟而非数月为单位迭代,使用体验可能好得多。
- 企业反复遇到的两类问题,是部署后台智能体和管理数据权限。Alex说,权限管理和RBAC已让CIO和CISO“头疼不已”,10X已将自家的网关方案产品化。CJ强调前提是:“流程不存在,就谈不上自动化。”如果40个人做着大致相同的工作,却各有一套做法,就应先统一流程,再把工作交给智能体或搭建评估体系。
- 谈到岗位替代,Alex强调个人仍有选择和行动的空间;CJ则直言,把AI说成只会替代人们不喜欢做的工作,“多少有点自我安慰”。CJ援引a16z称,随着AI承担更多工作,呼叫中心岗位正在减少;10X则把部署智能体描述为人员重新分配、转向下一个问题。谈到地域,CJ认为纽约的AI泡沫比旧金山小;Alex则认为,纽约在商业上的体量可能不逊于旧金山,甚至更大,也能推动AI扩散。
1. 按产出而非工时为工程师付酬的模式经受住了考验——差距是4倍,不是10倍
- Alex再次介绍10X:它是“为全球一些最大的企业提供AI落地服务的合作伙伴”,帮助客户从AI应用走向AI原生。其基本判断是,AI代表“原始智能”,真正的难题在于扩散的最后一公里。成立18个月后,公司有75名员工,其中90%是工程师或AI策略师;客户由《财富》世界2000强企业和营收1亿至5亿美元的中型企业组成。
- CJ的创业起点浓缩了这套薪酬逻辑:作为按小时收费的顾问,他“感觉自己的产出越来越好、速度越来越快,最后拿到的钱却越来越少”。在Sonnet 3.5和Gemini 2.5 Pro的时代,他押注按产出付酬能让自己随着AI提升产出而赚得更多。
- Alex表示,常见的质疑——虚报故事点、交付只勾选完任务却几乎未经检查的代码——“完全没有成为问题”,因为AI策略师的激励与客户留存和增长挂钩;他形容这与工程师的激励逻辑“完全相反”。真正棘手的是,如何为工程师用于架构设计或客户会议的时间付酬。Alex估算,收入能力最高与最低的工程师之间相差约4倍,swyx随即追问:“哦,不是10倍?”
- 故事点标准至少每季度重新校准一次。一位嘉宾表示,理论上模型越强,故事点就越应该贬值;CJ则谈到,既要奖励产出,也要重新校准所创造价值之间的平衡。CJ认为,未来更多价值会来自“坐下来听客户尽可能完整地讲清楚需求,然后把这些内容交给模型”。
2. 元框架:上下文留在代码仓库,阶段内产出大约翻倍
- CJ介绍,10X的工程体系由结构化文档和一套供模型驱动工作的CLI组成,包含规范计划文档、智能体决策日志和可视化进度报告,所有材料都提交到Git。谈到文档与代码孰轻孰重,他说,团队确实需要知道某项决策为何做出;持久化的上下文“永远不只在某个人脑子里,而始终在代码仓库里”。
- 衡量流程是否有效的指标之一,是每名工程师每月的平均故事点数。CJ估算,从12月的“Opus 4.5时刻”到如今团队使用Opus 5.5,工程师产出大约翻了一倍。他说,模型大约从那时起开始承担长时间运行的任务;工程师构建和交付的东西发生了“惊人的”变化。
3. Opus 5.5减少了模型轮换,Token成本担忧也有所缓解
- 办公室里的“氛围检测”结果是Opus 5.5。CJ说自己不太用Fable,觉得很多人正逐渐不再默认使用它。Opus“速度快、成本低”,能力覆盖了他过去需要用两三个模型才能完成的任务。团队仍会在电脑操作任务上使用Codex,也经常让它审查Opus的成果。CJ说自己仍在最大化使用AMP,采用AMP的Orbs云端方案,以及连接Claude Code订阅的功能;swyx表示,这样能把成本降低约80%。CJ认为,AMP的跨模型功能优于他通过Claude Code、Claude和Claude Tag能实现的效果。
- 谈到Token支出与员工支出的基准数据,CJ估算10X的Token实际支出约为员工支出的1:1至2:1。swyx表示,不少公司每名员工的支出为10至200美元;AI初创公司的Token支出则从大致相当于员工支出,到后者的5–10倍不等。CJ说,随着员工人数增加,成本一度令人担忧,但如今已经下降:Opus 5.5、订阅方式和单项任务成本下降都起了作用。他说,想靠“把Token用到极限”来严重拖累公司,比6到8个月前难多了。
- CJ提出了数据公司MicroOne、Mercor和Handshake面临的问题:他说它们超过90%的收入来自实验室,但一些创始人预计未来几年企业需求会增长。这一预期取决于企业是否会微调自己的模型;如果前沿模型持续进步、Token成本持续下降,CJ认为押注企业自训模型并不容易。Alex补充说,如果Astra 6.5或Fable 5.5以高价提供最佳效果,支出可能会再度上升。他还提到Muse Wonder,称其处于前沿水平,而且“便宜得离谱”。
4. 企业采用漏斗:90%已部署大模型,只有5%把非工程类智能体投入生产
- Alex给出了企业采用进度:在10X接触的约100家企业中,他估计90%已部署Claude或Codex;约60%提供培训和使用指导;约50%已部署编程智能体,而不少AI Engineer参会者猜测的比例是70%–80%;约5%把非工程类智能体投入生产环境。Alex认为,最后这一步的落差,是原始智能与企业实际应用之间最大的扩散鸿沟。
- Alex说,推动这波浪潮的是一小群理解技术能力的人,以及规模大得多、受错失恐惧驱动的人。这种势头也可能让企业盲目追逐智能体热潮。企业常提出的需求之一是后台智能体,另一个大问题是权限管理和RBAC。员工使用Claude或Codex并需要访问公司数据,Alex说,这已让“CIO和CISO头疼不已”。10X最初为客户定制解决方案,后来将自家的网关产品化。
- CJ提醒,前提是:“流程不存在,就谈不上自动化。”如果40个人做着大致相同的工作,却各有各的做法,智能体就更难在后台接手这类工作。CJ表示,应先统一流程,再评估智能体、移交工作;最终目标是让员工更像智能体管理者。
5. 看空垂直AI,swyx认为专业化产品仍能延续
- Alex表示,他总体看空面向特定垂直行业的AI,例如为律师服务的Harvey和为医生服务的OpenEdmunds。从10X深入企业内部的经验看,实际情况往往复杂,有惯性,还需要协调大量系统和人员。买家越来越期待高度定制、独一无二的解决方案,软件公司则为此组建FDE团队;Alex质疑垂直软件如何在维持软件毛利率的同时满足这些期待,也质疑前沿模型为什么不会“把它们吞掉”。
- swyx反驳道:“到目前为止,你一直判断错了。”他指出,ChatGPT可能不具备某些数据,专业功能和界面也有价值,例如Harvey的平板端审阅和Legora。他以Bloomberg为例:即便他能从Nasdaq和纽约证券交易所获取相同信息,Bloomberg仍提供整合后的专业信息、同行用户网络和各种集成。Alex回应说,人们也在为Bloomberg的社交网络和私信功能付费。
- CJ区分了新开发的AI应用和主要靠智能体干活的垂直产品。他认为,缺少战略性合规或数据护城河的垂直智能体,会随着模型进步而陷入困境;但能改变流程、在流程之上部署智能体的应用和工具依然可能成功。swyx补充说,用户可能更愿意使用已有的智能体,再接入自己的数据;他指出,Salesforce正通过API和MCP拥抱这条路径。
- swyx举出的“Kill My SaaS”案例让中端市场的问题更加具体:供应商报价每年4万美元,替代方案只需1000美元,而且即将投入下一场大会的生产环境。Alex说,Salesforce和SAP很难被替换,因为替换失败可能让数亿美元乃至数十亿美元面临风险。中端SaaS要具体问题具体分析:关键在于机会是否值得投入工程资源,以及是否值得承担维护非核心软件的认知负担。10X自己开发了CRM和操作系统。swyx强调,重建系统也可以是进攻手段,而不只是为了省钱:如果客户能“以分钟而不是数月为单位”迭代,使用体验可能好得多。
6. 岗位替代没有简单答案,纽约是商业活动更密集的一方
- CJ说,AI会增强和改变工作,也会取代一部分工作。他援引a16z的一篇文章称,随着AI承担更多工作,呼叫中心岗位正在减少。10X把智能体部署描述为人员重新分配:解决一个任务之后,还有下一个问题要处理;领域专家也可能在不同团队间协作,或专注于更具战略性的问题。
- Alex认为,人们谈岗位替代时,常把它描述成发生在人们身上的事,却没有充分肯定他们的主动性。CJ补充说,把AI描述成只会取代人们不喜欢做的工作,“多少有点自我安慰”:很多人无法只根据喜好选择工作。他表示,雇主和员工都应承担再培训的责任;人们也应评估自己的工作是否稳定、是否容易被替代,并在可能的情况下采取行动。
- CJ认为,纽约的AI泡沫比旧金山小,但10X的团队和合作伙伴让他得以看到当地高度活跃的AI社群。他说,10X希望扩大这个社群,把纽约打造成推动AI扩散的枢纽,因为当地商业活动密集,AI也有潜力影响GDP。Alex认为,纽约在AI领域只是个小池塘,但商业价值可能与旧金山相当,甚至更大。swyx说,他把AI Engineer活动办在纽约,是为了让旧金山的人来见客户、了解现实;Alex称这是种有意思的套利。
- CJ说,他的特别项目包括搭建10X内部的元框架和平台,寻找能为模型提供工具、提升其能力的方法,以及帮助企业部署后台智能体。
完整逐字稿
I just felt my output getting better and faster, and ultimately getting paid less. I was charging hourly to my clients. I phrase it to the team as, “You can’t automate a process that does not exist.”
Okay, we’re here at AI Engineer, checking in with 10X. Welcome back, Alex.
Thanks for having us.
You brought your director of engineering, CJ. Welcome to the pod.
Special projects lead at 10X.
Okay, special projects lead. All right. You’re the SWAT team.
Yes.
You’ve been on the pod before, and people are vaguely familiar with the 10X story. Can you reintroduce it from the top? What is 10X, and how do you introduce yourself?
Very simply, 10X is an applied AI partner for some of the biggest companies in the world, helping them go from AI apps to AI-native. Our belief is that AI is incredible, but it is raw intelligence, and the hardest problem to solve is the final mile of actually diffusing it through the economy.
Our view is to act as the player that helps enterprises figure all this out. Can I curse?
Yeah.
To figure all this shit out.
Yeah, we have the explicit tag on Apple Podcasts, although I recently discovered that YouTube doesn’t like it.
Oh, really?
You can’t say the F-word, or they’ll just turn it down.
So did I just mess up your video? What are you going to do?
I don’t care.
Okay.
We’re for adults. CJ, what drew you to 10X?
I was actually the first hire. I don’t even know if Alex and Arman had the name 10X quite yet, but I have a graveyard of startups I tried to start before 10X, ultimately culminating in my own engineering consulting company. I spent some time there doing traditional engineering, then eventually moved over to AI consulting and building more and more with it. I’m a software engineer by trade, and I just felt my output getting better and faster, and ultimately getting paid less. I was charging hourly to my clients.
I really liked the ethos of 10X that Alex had first posted when I saw it on Twitter: looking to reward engineers for their output as opposed to rewarding them for their time. Fast-forward 2 years, and you can see how engineers can now parallelize a ton of work and have a lot more output. At the time, with Sonnet 3.5 and Gemini 2.5 Pro, it might not have been that obvious, but I basically took the bet that with my skill set and by leaning more into AI, I could have much higher output and ultimately a higher earning capacity. Since then, I’ve just fallen in love with that model. I think that’s what the future of engineering really looks like. Ultimately, when AI is solving all these tech problems, companies are going to pay based on outcome, and they’re not going to want to pay based on the time required to build something.
For more detail on that, people can refer back to our previous pod, where you literally pay people by story points.
Yep.
You just have to agree on what the story points are. But that also means some 10X engineers can literally make millions of dollars a year, and some others may not make as much because they’re not as productive.
Exactly. The thesis is that AI is really just a leverage bet on a human’s intellect and work ethic, so you’re going to have more variance than ever before in earning capacity. Our view is that our model isn’t for everyone, but for the person who wants to go all in themselves, who has an incredible foundation in software engineering and is constantly pushing themselves to leverage the frontier, to stretch their time as much as possible while maintaining quality output, there’s no better way to earn.
What’s the state of things? Last time I talked to you, I think you had maybe 2 or 3 people making a million a year or more. Is there an update on that?
Just to paint a picture, we started 10X 18 months ago. When we started the business, we were really an AI-powered dev shop. The original idea was that engineering is kind of like the canary in the coal mine for AI. Part of the impetus was that my co-founder, Arman, who’s technical by trade, was starting to use coding agents a lot, while some people he knew were still typing into their IDEs and not adopting the technology at all. We thought, “Why don’t we just supercharge a company’s engineering org if they aren’t adopting this?”
We started there. Then, as we moved upmarket from startups and mid-market companies, now 50% of our customers are Fortune 2000 companies, and 50% are mid-market companies doing $100 million to $500 million in revenue. They weren’t just asking for staff augmentation and AI engineering. They were asking for AI transformation as a service.
Today, the business is at 75 people. 90% of those employees are either engineers or AI strategists, who are basically technical consultants. I think the thesis has proved true: there’s a very wide dispersion in earning capacity between the highest-paid engineer at 10X and the lowest-paid engineer at 10X. It’s probably around a 4X difference.
Oh, not 10X?
Not a 10X difference.
That’s the dream.
Exactly.
Yeah.
Everything has proved right about the thesis. There have been interesting learnings around our model of paying based on story points, where there are limits to the model that we knew would be there, and we’re thinking about how to address them moving forward. But overall, the classic concerns people had about paying people based on output are that story points are subjective, people will sandbag them, or they’ll just ship a bunch of bad code.
Like, barely check the box.
Exactly.
Right? But not actually do the job.
Exactly. Those 2 concerns haven’t been concerns at all because of our AI strategists. We incentivize AI strategists, I think, in a way that’s perfectly at odds with engineers: they’re incentivized for customer retention and growth. The biggest challenge has actually been how to compensate engineers for all the time they don’t spend coding, whether it’s architecting or meeting with clients. That’s where we’ve had to find nuance in the comp model, but it’s still largely driven by output.
Anything to add?
I’ve spent a lot of time thinking about this because, if I think about my tenure at 10X, I was coding a lot, then a little less. Now we’re getting to the point where the models, when given the right context, can get you to 90% of the solution.
I do think there’s an opportunity here to think about how the value is really in sitting down with a customer, hearing everything they can brain-dump, and then providing that to the model to get closer to the solution. I imagine that in the future we’re going to look more at what the output really is, how it works, and how it looks, as opposed to the traditional story point model, while still anchoring to those as the unit.
I don’t know if we’ve talked about this before, but this idea of the meta-harness—the engineering workflows at 10X that we use—
Mm-hmm.
—that has allowed us to do really meaningful work on long-running tasks.
Okay.
Do you want to talk through our process? I feel like people would find that interesting.
We basically started trying to standardize a set of skills. I think most engineering orgs have done something similar by this point. For us, that grew into a very structured set of documentation within a repo, and then a CLI around it that the models use to drive the work being done, whether it’s creating a plan, a visualization, or a readout for a client.
Effectively, we’re building all the pieces of the process around writing code so that it happens more automatically and consistently across our team. One piece is a big canonical plan document with a similar structure, but as the models have gotten better, we’ve actually reduced the enforcement of that structure. Once you have your plan, you build it out, and we have a very opinionated way for agents to log their decisions, log what they’re doing, and keep track of what they’re up to, especially—
Everything checked into Git?
Yeah.
All this is checked in. All of this is really kind of Markdown for me now, more HTML. I like to see these picture books that the agents spit out. All that gets checked in, and then it serves as a surface where you have not only the code as the source, but also the documentation.
You know, I think there’s a debate about whether you even need docs. The code can actually be the memory of a codebase. I do think you do need to know why you made a decision and the context around the conversation that was had. Ultimately, that plus the codebase means if there’s a question, if there’s something we need to answer, or if we need to look back on why we did something, we always have it in some durable record. It’s never in someone’s head; it’s always in the repo.
That gives our agents and tools these superpowers to have all the context that I think a lot of engineering teams still just have in conversations or in their heads. Our goal is to continue to push that forward so all the context is in one place, and more and more of that work is driven by the agent, as opposed to us having to provide everything upfront.
Yeah. Amazing.
One way we measure whether any of these processes are working is: are we seeing an increasing trend in average story points per month for every engineer in the company?
Right. And what do you think it is, compared to the start of this year?
I don’t know. What do you think?
Plus 50? I don’t know.
I think it really depends, because we have some people that—I saw a picture on Twitter of a guy over a wizard orb, with these things in the sky. That’s how I think of some of our engineers when they’re whipping agents around.
Overall, I’d say it’s been about a 2× increase this year in particular. If you think of December as the Opus 4.5 moment, when the models really started to take on these long-running tasks, to today, we’re back to Opus, but Opus 5.5. That’s a magnificent model that can really do a ton of work. Over that period, I’d say the actual output of engineers has roughly doubled in terms of what they were able to do and cover in the same time span.
Is a single story point worth the same now as it was a year ago?
We’re always trying to calibrate.
Yeah.
It’s a tricky equation.
Is there inflation or deflation?
In theory, there should be deflation.
Right?
Yeah.
Yeah.
Yeah.
I don’t know.
That’s roughly how we think about it. At least once a quarter, we sit down and evaluate our criteria for these story points. How are we actually estimating these projects?
Yeah.
There’s a fine balance between still rewarding this output and, as the models get better, being able to calibrate the value we’re creating.
Yeah.
Overall, I’d say that roughly 2× increase has played a similar role in how we do these estimations. If I look at some of the features and things people are building and pushing out now versus this time last year, it’s insane what the difference has been.
Yeah. There are a couple of ways we can take this, but I’m just going to check in on the model-preference stuff, since that’s always a topic of conversation. I think you just let people choose whatever tools they like.
Yeah.
But then maybe you have some statistics on what’s preferred in the office. It seems like Opus is leading.
Yeah. What’s the vibe check right now?
Can you estimate—
The vibe—
—market share or whatever?
The vibe right now is Opus 5.5, I think.
No Fable?
I don’t particularly use Fable—
Yeah.
—and I think a lot of people right now are just defaulting away from it.
Opus is good enough. Yeah.
Opus is so good. It’s fast, it’s cheap, and it kind of hits all my checkboxes. I previously had two to three models, depending on whether they were cheap and fast, really smart, or able to solve specific problems. Now Opus is kind of covering my full range.
I think a lot of the team still reaches for tools like Codex when we need to do something more in the computer-use world. Workflow-wise, I think using both is still the best option, whichever your driver is. For most of our team, that’s Opus 5.5. Using Codex to review and having them work back and forth on your machine still produces the best results, I think. Right now, we’re pretty aligned on those two. I don’t think there’s too much—
Are you still AMP-maxing, or no?
I am AMP-maxing.
So you’re using Opus through AMP?
Yeah, they recently—
Or is that the default model for AMP?
Yeah, they recently launched the ability to hook in your Claude Code subscription, so that’s been a pretty big game changer for me.
Because that’ll really cut your cost by, like, 80%.
Totally. I love AMP because it takes everything off my computer. They have this—
Orbs.
—cloud setup, the Orbs. We actually co-hosted a happy hour with them last week, so it was cool to meet the team and Orb out.
That’s never going to be a thing.
I want to make it a thing.
This may be the first time in history that any human has said that.
Orb—we’re Orbing out. We were Orbing out.
No, they do say it’s “Orbing time.”
Yeah.
That’s it.
Yeah. But that’s my interface of choice. And the models are really between that and, if I’m leaning on Codex, Sol 6.1.
Yeah. Did you give Claude Tag a shot? Isn’t it weird that Anthropic never captured the remote-environment thing the way Orbs did?
Yeah, I think part of that is AMP being very opinionated and not trying to build a general solution. Tools like Claude Code, Claude, and Claude Tag provide value for teams that are really trying to collaborate on non-technical tasks. But part of the reason I love AMP is that it has all these cross-model functionalities. I can do that: Opus writes it, and I can even hop to another Orb and say, “Look at that Orb,” and review it. All of those things still come together to basically replicate what I’m doing on my machine. I just couldn’t quite get there with Claude Code, Claude, and Claude Tag.
Yeah, fair enough. Another thing I’m starting to track this year is the percentage of token spend to employee spend. I don’t know if you have that number. Most companies are at maybe $10 to $200 in spend per employee. The more AI-native ones would convert that to a percentage, so roughly 10%. The average AI startup employee is about 1× in terms of employee spend and token spend, and then it goes up to maybe 5–10× in token spend. Where are you guys in that range?
I actually don’t have a number off the top of my head. If we’re talking raw tokens, I’d say we’re somewhere between 1:1 and maybe—
Really?
—2:1.
That’s pretty high.
What we think about a lot is when to reach for the biggest model. I think every company is starting to think about that. But I will say our costs have recently been driven way down from—
Because of the—
—Opus 5.5.
—the subscription thing. Yeah.
We had an era when everyone was using Astra and Fable, and they just gave the best results. But ultimately, those models are expensive.
A lot of people, since the last time we talked, had this token-maxing era: token leaderboards and everything. Then they were like, “Oh, no, no, no, now we have too much cost.” That was the most predictable thing in the world. Did you ever fear about it, or were you just getting good ROI regardless?
I think about it a few ways. We were for sure worried about it as our headcount was growing. And I think even though our team—I don’t feel like anyone on our team ever really ascribed to the token-maxing zeitgeist, because—
I had a few moments.
Yeah, me too. But it’s like, if the boss is going to pay for it, I’m just going to go nuts, right?
Yeah.
Then you do the weird stuff so that you can scale back. But you have to do the weird stuff first to discover the lessons you can take with you.
Yeah, totally. This may sound crazy, but I’d be curious to hear your thoughts. I was talking to someone earlier about the debate that continues around data companies. I promise this is related. We were talking about MicroOne, Mercor, Handshake, and what the long-term future for these businesses looks like. More than 90% of their revenue is from the labs, and a number of these founders would argue that they think it’s going to flip-flop, and in the next few years it’ll be enterprise-driven.
The next thing I assume you have to believe, if enterprises are going to be the core buyers of that data, is that enterprises are going to do a lot of fine-tuning on their own models. Then the question becomes: Where is that going to make sense? Either you think there’s going to be better quality—which is a hard bet to make, because the bar is just going to keep moving on the frontier—or it has to be price. The price argument feels hard to me because we’re just seeing token costs continue to go down on these models.
Part of the reason we’re not concerned internally is that it almost feels like we’ve hit a Pareto point: costs are coming down on a per-task basis. It’s way harder today for someone to token-max and really screw the company in terms of cost than it was six to eight months ago. I almost think six to eight months ago was the point at which you’d be really concerned. Do you agree with that?
Yeah. I think there’s—
A couple of ways the future might play out there. If there’s a world where there’s an Astra 6.5 and a Fable 5.5, and they’re super expensive but get the best results, I could see us trending back in that direction. But generally, I agree. You look at something like Muse Wonder, which they launched yesterday—
Yesterday.
—today, and it’s on the frontier and dirt cheap. The competition is heating up, both in open source and closed source, and I think that will continue to drive costs down. A lot of economically valuable tasks can be done with Opus 5.5. Would a 10-times-smarter model really justify the cost of moving some cells in a spreadsheet? Probably not.
1. Enterprise AI's Diffusion Gap
Right. Coming back to your customers and what you’re seeing from the market, what’s the most common ask you guys get? And what’s something people come to you with where you’re like, “No, we don’t actually want that job yet”?
One way I think about this is that enterprises adopting AI is almost like a series of milestones in a race. Especially at AIE, when I tell people what percentage of companies have reached these milestones, they’re kind of mind-blown. The first milestone in the enterprise AI race is rolling out a general-purpose LLM to your people. Of the 100 enterprises we’re talking to today, let’s say 90% have rolled out either Claude or Codex to their employees.
The next milestone is actually teaching people how to use it—training and enablement. That never gets the love it should, but it’s arguably as important as the technology itself. I’d say that’s at about 60%.
No, that’s pretty good.
Yeah. There’s a group of companies that are like, “Our people will just figure it out.” But there’s a good chunk that say, “Okay, we need to teach our people what skills they need. We need to teach our people the features and functionality of these surfaces.”
After training and enablement within a general-purpose LLM comes coding agents. The crazy thing is, I was asking a few people here, “What percentage of enterprises do you think have rolled out coding agents to the eng/product org in their company?” Most people are like, “It has to be 70% or 80%.” The number is closer to 50%. Every other enterprise we speak to either hasn’t rolled out a coding agent, or what they’re using is closer to not having a coding agent than having one.
Then there’s the biggest diffusion gap between raw intelligence and enterprise usage: agents in production for non-engineering work. That drops to about 5%.
Wow.
I mean, you’re in these conversations all day long, but most enterprises we talk to with more than 1,000 employees do not have an agent in production doing work that isn’t building software.
Sure.
I think there are a few things going on. This whole wave is driven by a combination of a small group of people understanding what’s possible with this technology and a much larger group driven by FOMO. FOMO isn’t bad; it creates forward momentum. But it also means people latch onto the things that are most in the zeitgeist. For the last year and a half, people have been saying it’s the year of the agent—or however long it’s been.
Every year is the year of the agent.
Every year is the year of the agent. Enterprises are like, “Can you help build us an agent for some back-office task within the organization?” That’s one of the most common things, and it’s why a lot of the work we’re doing at 10X now is deploying agents into production outside the eng org for repeatable work.
The second thing is permissioning and RBAC. As every employee in the company has Claude or Codex and needs access to certain data, it’s turned into an absolute nightmare for CIOs and CISOs. They don’t have a good gateway, almost like a control panel, for data permissioning.
It literally just means what can you read and what can’t you read?
Exactly.
Yeah.
That’s a huge issue, so that’s something we spend a lot of time on.
So do you make them for people, or do you buy something off the shelf? What’s the market there?
We started by building bespoke solutions for companies—
Yeah.
—and now we’ve productized that. We have our own gateway—
Gateway platform.
—our gateway solution.
Gateway solution.
Exactly. Is there anything else that comes to mind that we’re hearing a lot from customers?
I think that covered what we’re hearing, particularly when it comes to agent deployments. One of the pains that always gets discovered—I phrase it to the team as, “You can’t automate a process that does not exist.” When you have 40 people all roughly doing the same function but doing it a little differently—
Mm-hmm.
—it’s way harder to build an agent that proactively does that in the background. You can absolutely sit down with them, give them the tools, and walk them through how they could augment their own workflow. But you really start to get leverage when you have agents surfacing work for you, you’re reviewing it and throwing it back, and you become more of an agent manager. Everyone’s been talking about becoming an agent manager, and I think we’re on the cusp of that.
When you zoom out, our recommendation to companies is often, “First, we need to build a process around this function within your company. Then here are the steps for us to actually hand off that work to an agent in the background.” If we don’t have some form of standardization, it’ll be impossible for the agent to know whether it’s doing the job right, for us to build evals around the system, and to deliver the value an agent doing this function would have.
2. Vertical AI And SaaS
One other thing I’d be curious to hear your thoughts on—we were literally talking about this right before. CJ will be okay with me sharing this. We were talking about vertical-specific AI solutions and what the future looks like for them. We were generally expressing an overall bearishness about vertical-specific AI solutions in different functions or industries. Part of it is just our vantage point: we spend all day in the guts of enterprises. There’s so much nuance and so much inertia, and so many systems and people you have to wrangle.
Oh.
And when a buyer of a vertical-specific solution is expecting truly a one-of-one experience, we’re in an age where people are expecting one-of-one software solutions.
And so if you’re expecting that, it’s no wonder every software company is basically building up a team of FDEs. There’s a convergence happening between services and software when every company wants all this nuance that wasn’t traditionally expected from software. I think it’s really interesting to think about how vertical-specific solutions will be able to do this at software margins if everyone expects a one-of-one solution. And why shouldn’t we expect frontier models to just gobble them up?
Yeah. To give some concrete examples, are you talking about Harvey for lawyers?
Yep.
OpenEdmunds for doctors.
Yep.
These are the classic examples of people who are winning their domains. And you’re bearish on them?
I’d say I’m generally bearish on them, correct.
Wow. Well, so far you’ve been wrong.
Yep. Yep.
I think the answer is very simply access to data that ChatGPT isn’t going to have, which is a pretty common moat. Obviously, Anthropic and OpenAI are also acquiring those same data sources, but they’re not as dedicated as these companies are going to be. They have innovations, too. If you’ve tried Harvey, they have tablet review. Legora also has it. We’ve had the CTO on the podcast, and they have affordances and UIs that are customized because they only serve those customers.
Yeah.
The pitch is focus, basically.
Focus, affordances, maybe.
What do you mean by affordances?
It’s about caring about the last mile for your customers. They care about their last mile in terms of the software. As long as people train in that software, it never goes away. In my finance career, I could always access the same exact information from Nasdaq and the New York Stock Exchange. But why do I pay $50,000 a year for Bloomberg? It’s what all my peers do, and it’s consolidated and specialized for me, so I don’t make mistakes. It’s tied into my trading system, my chat, my news system, or whatever.
No, I think people pay for Bloomberg because it’s the social network. I think people love the DMs in Bloomberg.
There’s that. There’s that. But as a trader, I know—
Yeah, yeah.
I actually use that for—
Yeah, yeah, yeah.
—trading. I think there are a bunch of integrations and tools. Ultimately, it’s hard to be bearish on any of them, mostly because the demand for software and tools to do your job is huge. Some of it is going to be fulfilled by horizontal, generalized tools like Claude and ChatGPT, and some of it is just going to be something better.
Mm-hmm.
I always think about the success of Superhuman as an email client. Everyone has Gmail, but here’s premium Gmail with more features for people who live in email all day long. That’s a market, right? Maybe it wasn’t the largest startup in the world, but reasonable people can disagree about how large each thing is.
Yeah, totally.
I think what’s not to be disagreed with is that the total market is going to grow—
Oh.
—and a rising tide is going to lift—
Totally agree with that.
—all boats. Then you’re just talking about percentage points of market share—
Yeah, yeah.
—at that point.
Agree with that. Totally agree.
I think there’s a differentiation between some new app built from the ground up with AI involved. That’s what comes to mind when I think about Superhuman. I love Superhuman. I don’t even spend a ton of time in email, but anything that has—
You pay $30 a month just to—
Anything that has—
—not worry about it.
—cool hotkeys, I’m into.
Yeah.
When I think about products where they’re really pushing an agent as the product that’s doing work in some specific vertical, unless there’s some very strategic compliance or data moat, I think those are going to struggle going forward. We’re going to see the ability to spin up your own really quickly, and that will continue to compound as the models get better. Or those tasks will just start to fall into the world of the models generally. But anything that has an app and tool built around it, changes the process, and has an agent on top of it—that builds on the same philosophy. I think that can still be super helpful, and those will be successful.
Yeah. Okay, there’s a related view about this that maybe isn’t quite as spicy as yours. People probably don’t want to use the agent you built. They want to use the agent they already have, and they want to plug your data into their agent, right? Everyone is sitting on some part of your data, saying, “Use my agent, please. Please pay me that $20 a month.” But you can’t just stack up all these $20-a-month subscriptions. Ultimately, probably a small set of companies are going to own the interface, and then everyone just plugs into that overall agent, which I agree with.
It’s interesting that Salesforce, one of the biggest and best-known system-of-record companies, is embracing that path: “You will not use our agents. We’ll just provide APIs and MCPs for you to plug into your stuff.”
Other stuff.
Yeah, which is kind of cool. I also wanted to walk through one example. I just released the results for my Kill My SaaS competition, where we replace a mid-tier SaaS. By “mid-tier,” I mean it’s probably not trivial, but it’s not Salesforce. It’s somewhere in the middle. There’s a lot of mid-tier SaaS out there, and people are always talking about the death of software. We were quoted $40,000 a year; we spent $1,000 to replace it. It’s going into production for the next conference.
I almost want to say we’re that customer who says, “I want the one-of-one tool, and I’m expecting all these things.” We’re building a domain-specific or vertical-specific thing for our business, and so far it’s going okay. Tell me where it’s going wrong. Also, I’m wondering how many story points that is.
Yeah. Well, CJ can size up the story points on it. My general view is that the more you move upmarket in business, the harder the proposition of ripping something out and building from scratch gets. I agree with you. When people—though only a very small group—say Salesforce or SAP is really at risk, I don’t agree with that at all. Those businesses, in my mind, just accrue value at this moment in time. Part of the reason they accrue value is that there’s so much money at stake. If a company gets it wrong ripping them out, you’re talking about potentially hundreds of millions—
Yeah.
—or billions of dollars.
You’re going to be risk-averse.
Yeah, exactly. For mid-tier, it really comes down to the size of the opportunity and whether it’s ever going to be worth rebuilding inside a company. I’m trying to think. In an enterprise, let’s say there’s a mid-tier SaaS that costs a few million bucks a year. The first question is whether, from a percentage-of-revenue or cost perspective, that will ever move the needle enough to allocate engineering resources to. I think it’s going to be case by case. For some enterprises, the answer may be yes; for others, it may be no.
I think more of it is going to come down to capability. We talk about how—I forget who created the repo that’s basically the reverse-engineering repo—
Mm-hmm.
—where—
RE, yeah.
Yeah, exactly. Everyone’s talking about that right now. Even at 10X, we built our own CRM from scratch. What else have we built? Our OS—we built that from scratch. One thing I think we realized, and every company realizes, is that one of the beauties of software isn’t just the utility you get from it. It’s all the headaches you don’t have to think about maintaining and improving it when it’s not core to your business.
I think it basically comes down to whether the size of the prize is big enough to justify the cognitive load it will take from some amount of resource in your company to nurture something that is not core to your business.
Mm-hmm.
Evidently, for your business, the difference of basically $39,000 a year—you’re implicitly saying it’s worth it to have—
It’s not just about cost. That’s defense. I also think about the offense: actually, our experience could be a lot better for literally every party concerned.
Yeah. Part of the thought there is that it’s not just that we’re being overcharged for a product. The product has been mediocre for a long period of time. What would the 10x-better version of this product look like?
Yes, because we are the customers, and we can iterate in minutes, not months.
Totally. I actually think what’s interesting is that startups and even high-growth, mid-size companies can be nimble in that way. For enterprises, where they’re so entrenched in a certain capability, rarely are they going to say, “Oh, we can create a 10x-better version of this SaaS we use, and we’re going to spin it out potentially as its own thing or whatever it is.” I just don’t think enterprises function that way.
For sure.
Yeah.
3. AI And The Future Of Work
It’s been discussed. I’d say, you know, we talk about automation. The spicier thing is: are they automating humans? Once everything is automated and smooth sailing, do we replace those jobs? I don’t know how much you can speak to that.
Yeah. I really deeply believe AI will produce good for humanity. It’s going to augment people’s jobs. It will absolutely change and displace, but I think it’s more of a movement as opposed to just cutting off work.
Yeah.
I think a16z posted something about how call centers are truly now on the decline, and AI is starting to handle more and more of that work. Ultimately, if that is a skill you’ve built up your whole career, it’s devastating to have something come in and take that function over.
That being said, part of what we do when we talk to customers about “Hey, an agent is now going to do this work” is actually about reallocation. I don’t think any business would claim that once we have this solved, there’s nothing for this group of people to work on and try to solve. Our approach is really about what we can replace, and then what the next biggest problems to look at are. I’d make an argument that in most large companies, that cycle kind of continues forever. There’s always the next problem.
If I think about how we’ve grown—
Infinite demands.
There’s always been another problem. And actually, taking those people who may be super-deep subject-matter experts and bringing them above water to start working more across teams or thinking about parts of the business—
Yeah.
—less at the level of a cell in a spreadsheet and more about how these teams are working together. Ultimately, in the long run, I think that keeps most people there and allows those businesses to become more nimble and tackle the big, meaty problems, not the drudge of most people’s day-to-day.
When you’re thinking about someone in a finance division, they’re mostly in spreadsheets. How can we elevate them to think strategically about the business, where money is allocated, and things like that?
Yeah, and I also feel like when we have these conversations around job displacement or job reallocation, we talk about it as this thing happening to people. One of the reasons I don’t like that is I feel like it doesn’t give enough credit to people to have agency in their own lives. Meaning, I think—
You can adjust.
Yeah, people really should be thinking—
Yeah.
When I think about enablement or people reskilling, I think it is the job of an employer, because they want to drive growth in their business, but it is also the job of an employee. I think when people say that AI is going to replace all of the work that people don’t enjoy doing, that’s a little bit of cope for the reality: people are going to be put out of jobs, because a lot of people don’t have the luxury of liking or not liking a job. They do a job to make money.
Yeah.
And people, to the extent that it’s possible, should have agency and say to themselves, “Okay, what is the stability or replaceability of my job over the next few years? If I am to try to skill up in a way that I’m better protected, what does that look like?” People have agency to ask themselves that and actually take action.
Yeah.
Yeah.
4. New York's AI Bubble
Yeah. I think that’s a good message for people to reflect on. The last piece I wanted to touch on before I let you guys go is New York. Greatest city in the world, according to some people. What’s the mood generally among the people that you talk to, whether it’s at work or personally? What’s the feeling? I live in San Francisco all day long. Everyone’s just in agents all day long. The discussion is Instinct, Groq, or Muse. It’s not, “Do you like AI or not?” I feel like I want to step out of that bubble for a bit, but it’s still New York. It’s still progressive.
Yeah.
It’s still forward-looking.
Yeah. I’d say there’s a bubble in New York that’s smaller than the one in SF.
Okay.
If SF is a big bubble, I think we at 10X feel like we’re in a microcosm of San Francisco because of the people who select to work at the company, the companies we’re partnered with, and the people we engage with in New York. But it’s also true that it’s a smaller universe. In SF, when you go there, it feels all-encompassing. Everywhere you look, there’s someone who—similar to what I was saying earlier when I went on stage—if people go on Twitter or X because they feel like they’re on Edge City, living in the future, AI Engineer is like that on steroids.
I feel like SF is similar to AI Engineer. The overlap between AI generally and people who have lived at some point in their life in SF is probably very large.
Oh, yeah, yeah. We’re roughly 1% of the city now.
Actually?
For World’s Fair, yeah.
That’s crazy.
Yeah.
But to that point, I just think it’s in the culture and DNA of SF, where a very big population of people live in the future. I think you can find that group in New York, and I think we at 10X have found those people. Part of our goal is to drive AI as a hub—or New York as a hub for AI in the U.S.—because there is so much commerce here that diffusion of the technology is going to have such a meaningful impact on the GDP of this country.
I think the bubble is smaller now, and we take it almost as a charge of ours to increase our bubble. I don’t know that it’s possible to get to SF size, but I do think we can have a meaningful impact on getting more people to live in the future here.
There’s a physical side to it. You see it on the billboards—
Yeah.
—it’s slowly creeping in. People used to—do you remember the Friend campaign, where people would spray all sorts of—
Yeah, yeah, yeah.
—graffiti on there? I think that was a little bit of intentional rage bait. But now you have the unironic AI billboards, and people are like, “Oh, I guess that’s part of life now.”
100%. I think there’s an opportunity to bring the edge and the living-in-the-future nature that SF has created because of the economic climate there, because of venture, because of tech companies, and because of the people who are attracted there.
But I also think of us as being in a smaller pond for AI, but I’d argue we’re in as big of a pond, if not a bigger pond, for commercial value.
Yeah, totally.
And so I actually—
The way I put it is, the reason I do AI Engineer in New York is I bring all the people from San Francisco to come here and meet their customers.
And also—
Exactly.
Get a reality check.
Totally. I actually think there’s a really interesting arbitrage there.
Yeah.
Yeah.
Yeah, totally.
What’s a special project? What is that?
A special project involves anything outside of our client work. If I think about what’s on my mind right now, it’s building that meta-harness and system internally. Basically, it’s finding the bleeding edge of what the models can do and giving them tools to augment how well they can do it. I’m also working internally on the platform that Alex mentioned.
A big goal there is to diffuse these background agents and actually get them out to enterprises in a way that’s truly, truly helpful to them, and solve a lot of those messy IT problems we hit on earlier.
Dude, when you have some solutions, we take guest posts on the LeanSpace blog. I’d love to welcome some sharing of knowledge. But otherwise, thank you so much for your time.
Sure.
And good to see you again in New York.
Yeah, likewise.
Yeah.
Appreciate it.
Once a year. Once a year.
Hell yeah.
Awesome. Thank you.