Tasklet如何把自主性注入Agent：与CEO Andrew Lee对谈

Andrew Lee, founder and CEO of Shortwave and now also founder and CEO of Tasklet, welcome back to the Cognitive Revolution.

Thank you for having me.

I'm excited for this conversation. This is the third time we've done this, and I've learned a ton each of the last two times. I'm sure that's going to be the case again today. The occasion is that there's a new product from you guys called Tasklet.

I've had the chance to play with it for the last month or so as you've been developing and refining it, and I think it is a pretty cool paradigm that blurs the lines between what many people are familiar with in the form of a chatbot, on the one hand, and a structured workflow or agent, on the other hand. This is something that sits in the middle and can play both roles, and I have found it pretty cool to play around with.

I'm excited to dig into every aspect of it with you. But maybe for starters, tell us about Tasklet at a high level. How did you decide to build another product, and how did you hone in on this vision?

This is our second product after Shortwave. Basically, what happened was that earlier this year, we got really good at hooking our AI in Shortwave up to other products.

We added MCP, and people were doing these interesting workflows where they were taking data from their email, sticking it in their Notion or in Asana, and updating their HubSpot automatically. People really liked this, and it worked really well. They said, “Hey, wouldn't it be nice if you just did this automatically before I got up every morning, so I didn't have to sit at my computer and run this prompt every day to update my data?” And we said, “Yeah, that'd be cool. Let's build that.”

We started making that, and we very quickly realized that if this thing is running when you're not at your computer, having a tight UI integration between your AI agent and your email doesn't make any sense, because you're not even in front of your computer when this is happening.

What if we took advantage of that and built something much more general-purpose that was great for doing email sync with other platforms, but also for general automation? We took a bigger swing and said, “Hey, we're going to try to build something designed to be your general-purpose AI agent automation tool.”

We spent basically from the end of May or early June all the way up until last week building, testing, and iterating, and we finally launched it.

Yeah, it's cool. I interact fairly often with people from all walks of life who are interested in AI. They're like, “How could I set something up that would do whatever for me?” The friction to do that kind of stuff has been fairly high, right?

Even with AI as something that can sit in a node in a traditional automation software platform, there's still a lot for people to figure out if they're not used to getting really structured in their own minds about how to break a task down into its steps and how to set all that up in traditional software.

I've found myself going to Tasklet as the first place and saying, “Okay, start here. Just start by saying what you want,” and then iterating on the thing from there.

It's a really interesting split where you have the mainline agent itself as an entity that you can chat with. On that level, it feels more like a chatbot, but it also follows your instructions. You can set it up to run every day or on various triggers, and you can tell us more about the particulars of that.

But then you also have this second tier, which is all the runs that the agent has done of this particular task. I think that one-two punch, both of which are very general-purpose and natural-language-driven, not really requiring any of the nitty-gritty setup that people have become used to—and probably been deterred by in many situations—is a pretty interesting new way to look at what an AI agent could be.

Tell us more about that.

Traditionally, the way people do these sorts of automations is that they use a workflow product, something like Zapier, n8n, or the AgentKit product that OpenAI put out. I would call that a workflow product.

I look at workflow products and say this was the right way to approach this from a traditional software-engineering standpoint a year or two ago, when models were smart but not that smart.

And I think we've learned over the last few years that you should always bet on the model. The models are always going to get smarter, and the right thing to do is to find ways to give those models more agency over time. I think the reason people have been shy about doing workflow automation fully agentic all the way down is because they didn't really trust it to be reliable. They didn't feel like the models were there yet.

But I think now is the time. I think we've gone through that transition from having a workflow that defines step 1, step 2, step 3, and step 4, with maybe some LLM calls inside it, to asking, “What if you just let the agent plan the whole thing?” The advantages you get out of this are tremendous because, let's say you run into an error state. In a workflow product, if you don't have a way to handle the error state, it just breaks. In an agent product, it just kind of figures it out and works around it. It can handle nuance much better, and as you mentioned, it's also a whole lot easier to set up.

We've been trying to say, “Hey, we're going to bet on the models.” Some of our workflow competitors have done a hybrid solution where they say, “We're going to have an agent for the purpose of creating the workflow.” If you use n8n, this is the approach with n8n, but the output of that is still a workflow, so it's fairly constrained in what it can do. We're saying, “Hey, not only is the setup portion going to be an agent, but the actual implementation is going to be an agent as well.”

Yeah, it's funny because that has kind of been the assumption on my part of what I expected things to trend toward, and yet I do find myself quite liking the form factor that you've created. To dig into that a little bit more, when you talk about trusting the models and what you mean by that, I have this one slide that I use all the time in presentations that's just a minimalist diagram of an agent—what is the most abstract form, with all the detail stripped away? It's basically just a box around an LLM and some tools, and that's your core agent.

This thing is then given a task. The LLM can reason a little bit and can use the tools. When it uses tools, something happens in the environment, it gets feedback, and it just keeps iterating on that basis. This is the classic LLM-in-a-loop model; it's basically your minimalist agent. So, when you talk about betting on models, the most extreme version of that would be to just throw that very minimal scaffold up and let the models have at it. You've done a lot more than that, I'm sure, and I can see some of the fruits of that labor, but how do you think about what adds the most value beyond that super-minimal agent scaffold that's kind of a cartoon diagram?

Yeah. What I mean about the models here is: What is really in control of what's going to be happening? One approach is to have a workflow where the thing that's in control is traditional software. You, as the user, define the steps, and it goes through some sort of flowchart. Maybe within those boxes there are some LLM calls, but the overarching control is handled by traditional software.

Versus, if you put the model in charge, the model is in charge. The model makes the big decisions, and then within what's happening with the model, you might have traditional software to execute the tools. So it kind of inverts the problem. Rather than having software wrapping LLMs, you have LLMs wrapping software. When I say we're betting on the model, that's what I mean.

Either way you do it, there's still a lot of traditional software that has to be built. I'd say the core of what the agent is—I think I totally agree with you—is basically just calling an LLM in a loop. You have the LLM spit out some tool calls, resolve the tool calls, call the LLM again, and it kind of reasons through things. But there's a lot that gets built around that.

I'd say there are 3 big categories of things that we bring to the table. The first is connections. We don't hook up to some pre-enumerated set of tools; we hook up to everything on the internet. Any service you want to connect to, we can connect to. All of the work of figuring out how to connect to Gmail, Notion, your random enterprise API, a computer that can access LinkedIn, or some MCP server—that's all our own code.

There's a lot of work involved in taking all these heterogeneous ways of talking to the internet and putting them into something that's consistent and that the LLM can reason about and use well. That's category 1.

Category 2 is triggers. All the stuff that makes the LLM run automatically and have those runs be encapsulated in these sub-agents that have limited permissions, where you have some control over them, is a big area of work.

Category 3 is a very nascent part of our product, but it's all of the team collaboration and sharing features. We're really targeting business operations use cases where it's not just you automating your personal stuff; it's like you're automating some core business process. We see that as very much a team sport. There are a few features in there now, like the ability to share an agent, but we're going to be doing a lot of stuff to provide oversight, management, auditing, cost controls, and things at an organizational level.

But I will say, prior to playing around with Tasklet, I had the working assumption that, while somewhat costly in terms of time and cognitive overhead to get set up, it would bring me to a place of higher reliability. So I assumed there was an inherent trade-off between accessibility and reliability.

But you're making an interesting point: when the AI itself has more room to choose its own adventure—that's my term for these less-structured agents, or choose-your-own-adventure agents—they have, in a way, the opportunity to have a higher level of robustness to unexpected stuff. They can route around obstacles they find at runtime that weren't anticipated by the person who otherwise would have been building out these block-by-block control flows. How do you think about that trade-off? Do you think it is, in fact, more reliable on net, or is it a different kind of reliability?

What are the upper limits of this? Could you put something like this in production as part of a production app? How far do you think this goes?

Yeah, I totally agree with your point that the upper bound of reliability is not what you get from workflow products today, because the real world is messy, things break, and those tools break all the time. Agents potentially solve many of those cases, so I do think you need to look at this holistically.

I think today, with the models we have today, you're probably going to have a somewhat less reliable solution in most business applications. But as I said, always bet on the models. I give you 6 months before that's no longer the case. We've been able to look back over the last few years and see that these predictions really do come true. So we're looking ahead and saying that today, for many applications, it is reliable enough, and there are a lot of other advantages. In the future, it'll probably be more reliable.

The other thing I want to note is that there's a lot the models can do to provide reliability and guardrails around their processes. For example, today we use the LLM to try to figure out types. We dynamically create some of our connections, so we use the LLM to figure out what the type restrictions should be, and then we enforce those in code.

If, for example, you really want the LLM to follow a flowchart for some portion of the project, we could build the ability for you to tell the LLM, “During this phase, you must do these steps in order.” The LLM could actually create its own guardrails and enforce that through code if it wanted to. I think if the model is wrapping the code, the model can then construct constraints in that code to enforce the reliability goals that you want. We haven't built it yet, but I think we can.

I don't think the future is going to be, forever, “Here's the quick, easy, unreliable way, and then here's the harder, more reliable way.” I think it really is going to be better in basically all scenarios in the not-too-distant future.

Yeah, maybe it could be Sonnet 4.5, or maybe we have to wait until Sonnet 4.7. It seems like these things are coming at us pretty fast.

This is something we talked about last time, and your response was quite interesting. Basically, as I recall, it was very vibes-based. How do you evaluate things these days? This debate has continued to rage, as I’m sure you’re well aware, since our last conversation. Have you updated your position, or are we still running on vibes?

Still running on vibes. I think what we’ve figured out is that, in the current market situation, the Anthropic models are the best at what we do, and there’s really no question about that. If you’re looking at emails from the standpoint of deciding whether to roll out a new model, we kind of have no choice, right? We can’t really switch off Anthropic right now because there’s no viable alternative. When a new Anthropic model comes out, the pressure from our customers to release it as soon as possible is tremendous. Everybody wants it right now.

If we had an evaluation that wasn’t working out the way we wanted, we’d probably still roll out that new model, right? We might tweak some prompts and stuff to try to make it work better, but we’d probably still roll out that new model. We’ve just accepted that we’re going to be on that proper path for a while, and we’re always going to roll out the new model as quickly as it comes out. We need an approach that works in that environment.

The other thing to consider is that our product is changing so fast that anything we put in that could constrain our ability to move quickly is going to come with real business costs. This entire product is only—we only started writing code in June. It’s evolved tremendously over that time, and it’s going to continue evolving tremendously. Our users are very clear that the thing they want above all else—and I mean above reliability and security and all the normal things you want—is the smartest, most capable thing.

I’ve had lots of calls with people where we’re asking about SOC 2, and we’re like, “Yeah, we don’t have SOC 2 for this yet.” If you had to choose between SOC 2 and it being a little smarter, it’s always like, “Well, don’t tell compliance, but I want it to be a little smarter.” That’s really where our focus is: moving quickly and being smart. Evaluation beyond our own dogfooding isn’t part of that right now.

We do a lot of testing, don’t get me wrong, but it’s all internal. We have a whole bunch of triggers that run for our own internal usage, and we basically roll stuff out to our internal team first. We give it some time, see how it goes, and then, if it feels good, we roll it out more widely. In Shortwave, we do experiment-based rollouts. We allow our users to opt into things, look at retention rates for the new features, and roll them out. We’re going to do something similar in Tasklet: we’ll roll out a new version, let our users opt into it, see what retention looks like, and then flip it on for everybody.

I guess, just to ground this out a little bit, you said Anthropic models are clearly the best for what you do. How do you test that? Are you literally just doing Andrew’s top 10 agent use cases and running them head-to-head with GPT-5?

Yeah, it’s vibes, right? We’ve built our system so it’s easy to swap in different models to test, and we try the different use cases. I think what you find is that all of the models do a pretty good job at answering the initial question you have. In fact, some of the models may do a better job if your goal is to have one question and one answer. You might find that OpenAI models do better.

But if you want to have something that is LLM call, tool call, LLM call, tool call over a long sequence—which is what all the stuff you want to do with our product is—Anthropic just handles that iteration better. Over 100 iterations, that really adds up if it’s slightly better at each turn. It’s all been hand-testing different models, but ultimately, for those long, iterative processes, it’s clearly the best. I think the market agrees with us. If you look at all the other folks doing stuff like us, they’re all heavily using Anthropic models.

Of course, everybody who listens to this feed is very familiar with the METR task-length graph at this point. We were all waiting for Sonnet 4.5 to be plotted on that graph. It did come in a little lower than GPT-5, right on trend with the curve, but a bit lower in the METR estimate than GPT-5. How would you account for that difference, if you could? Do you accept that analysis? Would you complicate it? Is there some reason that you’re coming out with a very different conclusion than what they’re measuring? Because I think the prevailing notion right now would be that GPT-5 is a bit ahead.

I don’t know why that difference is there. My cynical take is that some people are playing harder to hack the metrics than others, but I don’t actually know what the reason is. I look at our actual practical, real-world testing, and my experience has been that it’s better.

The data point I would point to that says, “Hey, there is something real here; I’m not just making this up,” is pricing. If GPT-5 were as good at these use cases as Sonnet is, it wouldn’t be less than half the price. At less than half the price, you’d think everyone would be switching over, and they’re not. When Anthropic came out with Sonnet 4.5, they didn’t lower the price, even though OpenAI had lowered the price. The fact that the pricing is holding up tells me that people are choosing it. I think it’s because, if you ignore what the metrics say, the real-world utility of it is just better.

How do you think about the size of tasks? When I’ve looked into runs of things that I’ve done, I’ve routinely seen dozens of steps. For whatever reason, the tasks I tend to come up with are usually helping me with AI research. It’s becoming overwhelming, and I need all the help I can get. So it’s a lot of search, reason, search, reason, search, reason, repeated a bunch of times, and then finally compile a report for me, send that off, and update memory. That’s basically the anatomy of most runs of the recurring tasks—the agents that I’ve created so far.

How does that compare to the frontier? Are dozens of tasks toward the upper limit, or are you seeing things that are substantially more than that? Do you think about them in terms of the time it would take a human to do them, or some other metric? How do you even conceive of measuring the size of what the agents can do?

To be honest, we don’t make any effort to estimate the time it would take a human to do this. We’re a little team; we’re just trying to build a thing that people will pay for. I’ll leave it to the researchers to figure out how that relates to human time.

We look at things in terms of turns because that correlates with our cost. We absolutely do see people with tasks spanning a very large number of turns, and this is something we need to change.

There's a 50-turn limit per question in our product. Initially, that was just set to keep a bug from costing us infinite money, but we're hitting this all the time now, and we need to bump that. I think the number one place we're hitting this is computer use. If your task is, “Hey, go to LinkedIn, find these 10 people, and send them all a message,” every operation is going to be a turn, and sometimes multiple turns. Computer use is a case where the number of turns absolutely explodes.

Yeah, that's funny. I've been doing that a bit as well. Specifically, there's an event coming up in Detroit where I was invited to either give a solo presentation or do a live podcast recording in front of a live audience. The question was, “Well, who's the guest?” So I'm specifically having Tasklet comb through AI leaders at various organizations in Detroit and pull back lists of people that I should consider inviting.

Which brings me to a great question. It seems like, in the metrics, Claude 4.5 Sonnet was a big step up in computer use. How did that feel to you in your vibes-based assessment?

It seems really good. I'd say the big blocker for more computer use for us right now is really just cost and speed. It is tremendously expensive to move a computer around by screenshotting. When computer use fails for our users, it has less to do with the intelligence of the model and more to do with either hitting that 50-turn limit or just not being able to afford it anymore.

I found it super impressive. We've looked at using other models specifically for computers because of the cost. We're looking at using Gemini and having a subagent handle portions of that. The downside of having a subagent run and then capturing some distilled version of the data and putting it on the main agent is that you just get less intelligence overall.

We've played with that a bit, but so far the strategy is just to spend a lot of money on Sonnet. We just rolled out Haiku, so we've been doing a little bit of testing there. I think it's too early to say how well Haiku is going to do with computer use, but that could be a big factor for cost for us. Generally, though, I'm super impressed with the capabilities for computer use.

How do you think about what kind of computers to give to the AI? It seems like, on the one hand, there's just a browser as one paradigm. Then, on the Grok 4 launch, I always think back to Elon talking about setting up power workstations with all the same high-end software that the engineers at Tesla and SpaceX use.

It seems like you're somewhere in the middle right now. It's not just a browser, right? You have a full operating system, a VM-type environment that the AI can use. How do you decide how much of a computer to give it, and how much can it do beyond the browser today?

This is probably the most active source of discussion for our team right now. We actually have teams flying out next week, and this is going to be a big focus of that discussion: what direction we want to take this.

Right now, in the product, every agent has its own SQL database. It has its own code execution environment and very limited file capabilities. You can optionally spin up a full Linux VM and connect to that Linux VM to use a browser and a file system in the VM.

We used to have Windows support, but that caused some problems for us. It was expensive, and the wake-up-from-sleep time was really bad. There were some reasons it was harder, and it was also pretty rare for people to use the benefits of Windows besides stuff in the browser. We switched to Linux, which I think has been working much better for folks. We've considered doing Mac for this usage as well.

Going forward, I see a couple of big goals. One is that we want to give it terminal access. There are a lot of operations that, if you use Claude Code, can be done outside of programming because it has the ability to access the console. For example, we have a weird number of people opening up agents and saying, “I uploaded this file. Please convert it from this video format to this video format.”

We're thinking, “I don't know why you're asking AI to do this, but we totally could.” We could run FFmpeg and do that sort of processing. So I think we want to give agents a shell and let them run command-line tools. I think we want to give them a file system, and I think we want all of those pieces of the computer to work well together.

For example, today you can run and execute code, but that code can't access your database or your connections. What if you had all those facets of the computer—the file system, the database, the code execution environment, the shell, and the UI—all able to talk to each other? I think that could be tremendously powerful.

Doing that well is really hard for a variety of reasons. Cost is a big factor. You could have 100 agents in our product, but you don't actually want to have 100 physical machines per user. The thing would be predictably expensive. This is an area where there's a lot of startup activity. A lot of people have figured out that you need specialized cloud infrastructure for doing this.

I was just talking to a startup this morning called Blaxel that seems pretty cool. There's a lot of discussion here and a lot of excitement. I think big things are coming. Watch this space.

What do you think of the requirements there? It seems like the main one would be that you want to be able to suspend the VM and put it into sleep mode but keep its state, kind of like my Mac when it runs out of battery. It comes up in roughly the same—usually in roughly the same—state that I left it. Is that the big thing, or what else is missing from the tools you have available right now?

I think cost is probably the number-one issue. You want to have the illusion of every agent having its own computer, and you want that computer to be powerful when you're using it. But you don't want to pay for any of the milliseconds you're not using it. That includes not just compute but also storage.

It can get complicated. You don't want to have every agent store a copy of the entire operating system, for example. So how do you get the illusion of every agent having its own computer without having to pay the costs of every agent literally having its own computer? I think that's one big factor.

Another big factor is making sure those computers don't get messed up over time. You want to give some thought to which aspects of this thing are configurable and which aspects are not configurable. You wouldn't want the agent to mess up some configuration and then be unable to use its computer forever after that. That's something we're thinking about, too: reliability for this thing.

Besides that, I don't know, man. The requirements here are super unclear. This is very uncharted territory, so we're figuring it out one day at a time.

Let's talk about use cases for a minute, because I go very deep into all the technical stuff. What use cases are you seeing that are on the frontier of what the models can or can't do, or that are just really creative—things that you think people should be taking inspiration from?

One thing that I have honestly had a bit of a challenge with in using any AI product that's meant to do recurring tasks for me is—and I don't take this for granted—I don't really have that many recurring tasks. I think it's a sign of great privilege in many respects. I mostly am chasing my curiosity on a day-to-day basis.

A lot of times, I feel like I'm asking, “Okay, well, what should I be recurring that I'm not?” Maybe I'm too lazy, and an AI could take me there. So take all the time you want on use cases. I need inspiration.

Here's an interesting thing that we figured out earlier on. The pitch that we give you is that we help you with recurring tasks, and we do. But the first version of Tasklet was set up so that there was a Tasklet setup phase, and then, once you got the automation running, it switched into running mode. You couldn't talk to the main agents. You were either designing the automation or running the automation, but you weren't doing both at the same time.

What we discovered is that people would configure, for example, an email-triage agent. They'd hook it up to their Notion and their Asana, and give it all these detailed instructions around what types of emails they care about and things like that. Then they would say, “Hey, I did all this work to create this agent. It's awesome at writing automation, but now I just want it to write this email for me. Why can't I talk to it more?”

We said, “You should go make a new agent for that.” They'd say, “Well, then I have to configure it again. I have to hook it up to all the same stuff. I have to give it all the same prompts. I just want to talk to this agent.”

We realized that what we need to do is enable each individual agent to handle both triggered tasks and the ability for you to keep chatting with it. That's why the agent works that way today. While it's off waiting for emails to come in or webhooks to fire or whatever, you can keep talking to those agents.

Once we enabled that, what we found was that people started naming these things. It wasn't just, “This is my agent for creating my email.” It was, “This is Joe, my EA. Joe does these automated tasks, but Joe also helps me when I ask Joe to do things.”

The way that we've been thinking about this more is as a tool for very heavyweight, very long-lived agents. You go to ChatGPT or Claude, have a quick question, talk to it for a little bit, and then forget about it. You never come back to it, and you have this long list of tasks that you never return to.

In our product, we're trying to have something with a lot more setup but a lot more value in the long term. Maybe you keep coming back to this particular agent for years. What that has resulted in is that the vast majority of usage of our product is not the automated tasks.

There's a lot of that, and basically all of our paid users have something that's automated and recurring. But most of the messages they send are actually the other messages they ask of the same agents that are doing these recurring tasks.

We've seen all kinds of stuff. We've had some companies automate their billing processes. We've had companies that had EAs doing very EA-type work decide to go fully with Tasklet to automate those tasks. We've had music venues that manage their calendars through it or keep all the various stakeholders aware of what's going on at their venue.

There's a commodities-trading firm I was talking to the other day that watches for events in the real world and makes recommendations for trades to make. Personally, I use it for marketing super heavily. Basically, all the content that you see from Shortwave or Tasklet these days is written through Lindy.

You saw part of this with the sponsored spot that we're doing with you guys, where I basically have a bunch of documents with my notes, and I have an agent whose job is specifically to do marketing for me. Anytime I need new content, I'm like, “Here's the document I have for this. Here's the spreadsheet I have for this. Here's this thing. Now I need a LinkedIn post, or now I need a podcast sponsorship. Spit that out for me.”

So it is everything under the sun. Pretty much every agent does something that's automated, but they also do a lot of other stuff, too.

Yeah, it's interesting. I've been gravitating that way as well in terms of the one-offs, like the LinkedIn thing I mentioned. That was just a one-off. While preparing for this, I was also getting Tasklet to help me identify any gaps in my outline of questions for you. It came up with a couple of interesting questions that would only come from Lindy, given the sort of unique insight that it has into itself.

Yeah, I love that you did that. That was pretty funny.

I guess there's a lot of little nitty-gritty things that I found really interesting. I do really like how you can go back and continue the conversation with the agent at the high level. There's this sort of two-tiered thing, and people should just go play with it, because I think in some ways it's more intuitive to just go do than it is for me to try to describe.

The high-level agent that defines what this thing is all about and maintains the prompt that will then be used for the individual runs—that's the higher-order thing that you can talk to. One bit of feedback that I gave to my arXiv paper-searching agent the other day was that, usually in the past, I felt I had to give anything like that a lot of context about me.

This time, I just said, “My interests are pretty well represented on The Cognitive Revolution website, so go check that out. Then you can search arXiv with that in mind.” It was cool because, at that higher level, it went and did that search to try to characterize me and understand what I'm about, and then turned that into the prompt, which gets fed into each individual run. It doesn't have to do that every single time.

Maybe I should come back at some point in the future and say, “Hey, you should refresh. I've done a bunch more episodes,” or whatever. In practice, what I did was notice, as I went into the runs and traced through to see what they were doing, that one thing it wasn't doing all the time was accessing the memory that it had logged.

At the end of the runs, I would see that it had logged something like, “Sent Nathan these 5 papers,” and they would be put into the store. But if it wasn't accessing that store at the beginning of the following day's run, then what good was that? I went back to the high-level agent and said, “Hey, I've noticed you haven't been accessing memory at the beginning of the runs, and that is leading to some duplication in the papers that I'm getting from day to day. Please access the memory at the beginning of the run going forward.”

It just updated its prompt, and for all subsequent runs, that became part of its standard operating procedure. I think that is pretty cool. It's starting to feel much more like a virtual employee. That's obviously a sort of fuzzy notion, but it's something that a lot of people have on the horizon in their imaginations.

You can feel that a little bit more when you're like, “I've reviewed some of your work, and I want to give you high-level feedback on what I'm seeing in your output. I want you to keep the same assignments that you have, but figure out how to incorporate that feedback into your future work without me having to get into the super nitty-gritty details of it.” I think that is pretty cool.

Some nitty-gritty stuff. How do you manage the—I guess, even more nitty-gritty first—I don't think I can talk to a run these days, and that is something that I thought I might also like to do. I might want to give high-level feedback, like I just described: “I've noticed sometimes you're not doing this in quite the way I think is best.”

But sometimes I might also want to go down to the individual run and say, “Hey, can you do one more step?” Or, “Can you turn this into a whatever on a one-off basis?” Is there any reason that doesn't exist, or is it just coming soon and you haven't gotten to it yet?

It's worth walking through some of the things that we've tried and also talking about where we're going, because I think this is an area that we've learned a lot about. The way it works today is actually not how I'd like it to work in the long term.

We used to give you an easier way to chat with the sub-agent runs. There are situations where you can have the sub-agent ask for human input, in which case a chat box will appear and you can talk to it. We found that this is not used very frequently and is a little confusing.

The idea that you come to the product and there's one chat box over here, and then every sub-agent run also has a chat box, starts to get confusing for users. There's the question of, “There are 20 places I can talk to the AI—which place do I want to talk to the AI?”

We're moving to a world where we just want you to talk to the main agent.

And I think the piece that's missing right now is that we're not giving the main agent enough information about the recent runs that have occurred. You can't today go into the main agent and get into the nitty-gritty and get the subagents to continue running, but you could. So this is this whole project we're calling introspection: How do you have these subagents that run and have the reliability and cost benefits of being run as subagents, but then have the main agent appear to have full knowledge and control over them? And that is an ongoing project.

I think one of the areas we're going to look at is the way that instructions work. You mentioned that you look at the instructions sometimes and then give the main agent instructions to change those instructions. We have these intermediate instructions, right? Those intermediate instructions are generated by the main agent and then persisted.

We actually used to let you modify those manually, and the big downside of that is that it creates a tension between what you've told the main agent and what you've edited those instructions to be. So if you go and tell the main agent a whole bunch of stuff and then later edit the subagent instructions, even if you tell the main agent that you made those edits, it starts to get pretty tricky for the main agent to reconcile: You told me you wanted XYZ, but then later you edited the instructions to do this, and I'm not sure what you want later because the conversation continues.

We removed the ability to edit those, and it also wasn't being used very much. I think in the future we're probably going to stop pre-generating those instructions, because pre-generating them requires the agent to reason about when to update them and how to update them while doing other tasks. You might ask it to do something and make an aside while doing that, where it has to go do the thing you're asking it to help you with immediately and then also go and update these instructions, which is a long, slow, error-prone process.

I think what we may move to is something where those subagent instructions are generated just in time and out of band, and done in such a way that they always factor in all the information. If you did want to see the instructions in a way that you could digest them, you could just ask the AI. Rather than having a special button that says, “Show me instructions,” you just tell the AI, “Give me a summary of the instructions that I've given you.”

Then you could say, if you really want it to follow this exactly or modify it, “Well, actually, I'd like you to follow these modified instructions.” I think you can get the best of both worlds. You can get the ability to be very specific in your instructions and to see what it's thinking without a UI that creates this ambiguity.

Yeah, that's a really interesting point. That ambiguity, which I also think kind of blurs into an agent that has multiple conflicting goals, is becoming a real driver of strange results from AI systems in general. The GPT-5 prompt guide takes pains to warn people against having any sort of contradictory instructions. That's apparently a huge drag on performance if there's any contradiction in the prompts.

Things like alignment faking and a lot of these bizarre behaviors stem from some sort of incompatibility between either what the agent has been trained on at a high level and what it's currently being asked to do, or even just 2 instructions that it might be receiving at different levels of the instruction hierarchy. So you're right to flag that that can lead to some weird stuff.

There's just one part of the code where I've asked the team to make me the reviewer before a PR goes in. I'm not the CTO here. I'm generally not the reviewer for things, but anything that touches the main prompt, I want to see. The main thing I'm looking for is conflicting instructions.

If you have a bunch of people all working on their part of the system and giving instructions that make sense, you need 1 person who's looking at the prompt as a whole and saying, “Are we giving instructions to this thing that are clear? If it reads the whole prompt, does it know what's going on?” That's a problem a surprisingly large amount of the time, even when people are trying to be very careful about it.

So how about this general notion of context engineering, context management, and maybe context cleanup or compression? That obviously blurs into memory too, although that is a bit distinct as things stand today. The question that was suggested by Tasklet for you says that there's a token budget of 200,000 tokens in my current context.

Now, I know that there are some opportunities to get to longer context with the right customer relationship. I think you have to pay more per token if you want to opt into those things. But obviously, it's going to be finite no matter what, right? Especially if I'm talking to this high-level agent and I've had a bunch of runs that are running on an ongoing basis, and I say, “Can you pull up the run from October 1?” and then, “Can you look into the run from October 5?” It seems like I'm going to be out of context real quick.

How do you manage that? When you talk about something that you could come back to for years, is that still aspirational, or is there a way to get the agents to be that long-lived given the finite context that you have to work with?

Yeah, totally. I want to note that I really love the word “context engineering.” I've been looking for a word to describe this, because I think there are just 2 parts of building an agent. Well, 3 parts. There's the core agent loop, but that's pretty simple. There's the model, and then there's all the work of plugging things into that model.

I think the terms that people have used in the past aren't quite right for the way things actually work. The term RAG, for example, I think has been used very badly. There's sort of a naïve RAG that most people think of, but it doesn't really describe the way tool calling and stuff works.

I really like the word “context engineering.” I think it gives sufficient gravity and weight to the sophistication of the systems that are actually built around these agents to plumb the right data in. We've done a lot of work here, and we're going to continue to do a lot of work here.

I agree with you. Somehow, in our case, you might have an agent that's been running for a long time and has processed 100,000 emails. Obviously, you can't fit all those into any sort of reasonable token budget, so you have to give the user the illusion that not only have you fit those things in, but that the LLM is still able to reason intelligently with a huge context.

There are a lot of methods here. You can hide things inside tool calls, and you probably saw the stuff from Anthropic recently. One of the things you can do is just hide the results of those tool calls. You can do LLM-based compaction. You can say, “We're going to rewrite this. We're going to take some big section, and we're going to take a model and summarize that in some way.”

You can do things in subagents. When our subagents run, we're not giving them the whole chat history. We're giving them some tailored portion of that chat history. I see this as a major focus for us: giving the user the illusion that everything they've ever said to this thing and every email that's ever been processed is being considered at all times without actually doing that. I think that's a fun problem.

So I guess one more thing on context management. If I'm looking at the subagents, I have this archived Paper Finder Daily, right? At the end of a run, it stores what it found and what it sent me. Then, at the beginning of the run, with my feedback, it's pulling that history out so it knows what it has already found and hopefully avoids duplication.

Even that, over time, is going to start to get huge in the limit. There's just a ton of stuff that it's already found. I guess I could help it out by saying, “Only search for stuff in the last 2 weeks,” and then you only have to pull stuff from the last 2 weeks. But it seems like everything blows up in the end.

I wonder how you're thinking about that. I also noticed—I think I noticed—that even in just the month that I was using the product before the launch, when I go back to the earliest runs, it looks like it's a NoSQL store, and now the more recent ones have a SQL backend. Previously, it was just JSON docs that were being sent into the store, and now we actually have insert statements and whatnot.

So I guess, how do you think about the size of the store? Does that become a RAG—with possible pardon for abusing the term—where you sort of have to apply some intelligence at that level as well?

Or, you know, I could imagine you could also be using the old Gmail trick, which was to promise unlimited storage and just know that people don't have that many emails coming in yet, so you can figure it out at some point in the future. And then there are also these dedicated memory startups, and I'm sure you've looked at Mem0 and Supermemory. So, to sum that up into a question: if I'm storing more and more stuff, how do I not overflow the context just with that? What are you seeing working and not working? What about these memory startups?

Yeah, this is very much a core part of the challenge. We're not using any other providers because we think this is something that we have to solve ourselves. If our whole premise is that we're like ChatGPT or Claude, but much more heavyweight and designed for long-running, recurring work, this is a very core, fundamental thing we need to do.

The north star for us is the illusion that we're just one big, long chat, everything has been put into that chat, and the model is perfectly smart. The system breaks under all of those reasons, right? We can't actually fit all of it in context, we can't afford to do what we would do if we could, and the model is not infinitely smart.

For example, if you stored every email that you've ever processed just in the history, the LLM isn't going to do a good job of reasoning about which ones you processed or which ones you haven't processed. It's just not going to be that smart. One of the techniques that we've been using is having a data structure that is managed explicitly by the LLM across runs.

The first iteration of this was a very simple version in which we included a JSON blob in the system prompt. That JSON blob could be directly edited by a tool call in the LLM, so in the subagent runs, you could just tell it, “Hey, I want to add this data or remove this data from this dataset object.” That actually worked surprisingly well. This was the dumbest thing we could think of, and we built it, and it just worked. It was really cool.

It came with a couple of downsides. Funny enough, it fairly quickly got large enough that it started to break down and not be useful. We had a lot of agents that were working for a while, and then after a week or two or whatever, they kind of stopped working because that JSON object got really, really large.

The other problem is that it's terrible for caching to put mutable state in your system prompt, and caching is very important for cost. The benefit of having it in the system prompt at all times is that the LLM doesn't have to reason about how to retrieve it. It's just there, right?

We looked at other ways to do this, and we settled on using just a SQL database because it turns out that models are super good at writing and using SQL. They're way better than I am at this. If you just tell it, “Look, you've got a SQL database, and you can store stuff in there and retrieve it,” and then you can give yourself instructions on how to manage that thing, it works really well.

I agree this is not the end of the story. You totally can have instructions that say, “Every time you get an email, store it in the list, and every time you start, load the whole list,” and that list could get bigger and bigger and bigger and eventually cause you problems. We're not done; we're going to have to keep working on this.

I think that, in addition to being even smarter about how to manage that SQL data, there are going to be lots of other tools that we use. As an example, one of the things I'm looking at now is a world where you've compacted old history, but that old history becomes relevant. You could potentially give the agent tools for uncompacting portions of a history, being like, “Okay, in this particular case, I really do need to know what the earlier messages were.” You could potentially load those through tool calls. There are a lot of tools and a lot of ideas that we're exploring here. I think this is very much an unsolved problem, and we're working on it.

Any impulse toward some sort of graph of knowledge? We did an episode on it; it's probably been at least a year, on a paper called HippoRAG. It was cool, but there were definitely still things to do to take that to a production system. There was a HippoRAG 2 at one point.

What I thought was cool about it was that it dynamically, on a periodic basis—almost like what would happen when the agent is asleep—would go back and sweep through all of the new items that were stored in whatever memory store and try to find connections between them. It would map the space out and create these connections that then, in the future, really improved retrieval, because you could do a graph-based search.

If you hit one node and expand 2 hops out from that one node, then you get a pretty good set of all the surrounding information. You can do that dynamically at runtime with that background processing to continually build out the graph day after day. Anything along those lines in your mind, or what do you think are the sort of creative ideas that will—

I suspect something like that could work well, but I don't think it's the right business move for us to try to explore doing that. The reason for that is that the agent's internal state is only one of the many sources of data that the agent needs to do its job.

If you ask it to do a task, it might need to look through its own state, but it also might need to look through all the many systems that you're connected to. The amount of data that's connected to those systems is way too big for us to suck it all in and put it into this system.

If we built a system that managed internal state super well, but for all the external state we were still relying on whatever those services provide for their search and memory systems, I don't think it would be a huge improvement to the overall capabilities of our product.

We're betting on the model again. Assuming that the memory capabilities never get that great, will the model get smart enough about using those things such that it can figure it out and get really awesome? I think that trend has been—I'm optimistic about this.

I would argue that companies that, a couple of years ago, were like, “Hey, we're going to ingest all the data from all these different systems and build this awesome AI search product,” I suspect now they're looking at this and saying, “Crap, if you just take an agent and have the agent reason about how to use legacy search tools, it does almost as well. And, man, that is way simpler and way cheaper.” I think people are realizing that now.

That's our bet: we shouldn't invest so much in smart memory systems now. We should invest in connecting to things and then bet on the model getting smart enough to use more traditional systems.

Yeah, that's interesting. Do you also rule out Claude's built-in memory? They now offer a little bit of memory management through the API itself, right? Is that also sort of, “Don't use it because it's got to be core to us”?

We haven't played with their latest stuff. If memory serves, I don't think they're doing anything that we couldn't be doing ourselves. I think their context-editing stuff is also something that you could totally do yourself.

So, yeah, it's in the camp of, “Don't use it because we can do it ourselves,” although I haven't played with this specifically.

Going back, I think, to our first conversation, there were a lot of custom models and relatively deep-down-the-stack optimizations that you were using for retrieval when you indexed my full and very gnarly Gmail history for Shortwave. It sounds like that's not so much a part of the strategy now.

I also remember that Cognition, in the last few days, showed something where they are training their own relatively narrow agentic search model. It's meant to do the same sort of thing that Sonnet can do in terms of retrieving against all these systems, but faster and cheaper, if nothing else, because they're able to specialize on a particular, relatively narrowly scoped task and therefore do it with a smaller model.

I'm guessing you're going to say that you're leaving it to Sonnet for now, and maybe one day you'll need to do that sort of optimization. But what do you think is the outlook there?

Roughly, I do think that a good AI search system with a great model is going to outperform a traditional search system with a great model, and I think that'll always be true. You can see this today: going to Tasklet and asking it to search your email is not going to give you as good results as going to Shortwave and asking it to search your email, because they're both using the same model and Shortwave has the benefit of a really good semantic search stack.

But if you look at the 2 businesses, Shortwave is a tremendously complex and expensive system that operates that search stack, right? It is a major factor of our cost and a major factor of the effort that we have to put into developing and maintaining the product to maintain that search stack.

In Tasklet, we hook up to thousands of services, and it's not practical for us to build search for all of those things. Even if we tried to say, “Okay, we'll do a few of the most common ones,” I don't think it scales in any reasonable way. So I think our only reasonable strategy with Tasklet is to say, “Well, we're going to rely on their existing search stuff,” and I'm optimistic about 2 things. One is that the models will keep getting better using those existing traditional search stacks. And the other is that the services themselves will, over time, build good semantic search. Gmail will eventually, I think, build a good semantic search API that we can use.

At which point, the semantic search in Shortwave—we can maybe throw that infrastructure away and save some money. So I think that's our bet. I do think—I'm not saying that startups here aren't potentially going to be awesome and very successful. I do think there's lots of opportunity in AI search. I just don't think it's the right bet for us to invest.

Yeah, interesting. Well, let's talk about integrations. You have a lot of them. It's advertised as 3,000-plus business tools out of the box, any API, any MCP. How did you get to 3,000? That's a lot. You said 3,000 MCPs. I would assume that all kind of came for free, but when you separate them out like that, it sounds like you've done quite a grind—a long march through integration land.

So, yeah, this is an interesting story. In Shortwave, I think we have around 10 integrations or so, something like that. And we started down this path thinking, “Okay, we'll start the same way. We'll pick the top 10 most commonly used ones, and we'll focus on those use cases—email, Notion, Asana, and stuff like that.” My co-founder said, “Hey, we should try hooking up one of these integration platforms,” and we hooked up Pipedream. They have a whole bunch of integrations in there, and it just worked really well. So we're like, “Holy crap, suddenly we have access to all the stuff that they have here.”

So we said to ourselves, “Well, we should have MCP in here as well. So let's add MCP.” And that worked really well, too. We said, “This Pipedream stuff works surprisingly well, even though there's nothing in our code making this stuff work well. We don't have any customization of the prompt. You plug it in, boom, it works. This is awesome.” I wondered if we could do the same thing with HTTP APIs—if we could just describe to the model generally how to use HTTP APIs, put some constraints around how those calls were made, and then connect to an arbitrary HTTP API.

So we built a thing to do what we call direct API connections, and it worked really well—shockingly well. We found ways to make it even better, like going and scraping website documentation. When you make a direct API connection, we search the web, find the API docs, look them up, pull out a description, and then use that. Boom, we've got this API connection.

It works so well, in fact, that we have users who are switching off the official MCP. For example, we have a paying user who stopped using the official Notion MCP and started using the direct API connection that is fully LLM-generated on our side because it works better. It's more reliable. It has fewer bugs, or whatever.

Basically, we just kept adding and adding and adding, and we realized that the opportunity here was to connect to everything. We said, “Hey, rather than—because there are so many competitors doing what we're doing now with Tasklet, and every one of them comes out and says, ‘Hey, we connect to this SaaS or this SaaS or this SaaS’—and they all try to differentiate how many they cover, we're like, look, we need to just end this conversation and say, ‘Look, we work with everything. There's nothing we can't connect to.’”

So we made that a goal, and we very quickly went from 10 to thousands of integrations. Some of those are ours, some are through integration platforms, and then there are direct APIs—any HTTP API, any MCP provider-provided MCP. For example, with Linear, they have their own MCP server. Those are included in that list as well, and we help with discovery for that.

Then there's computer use. If you put all of that together, the thing that we provide is a unified UX and LLM interface for those things. No matter whether that integration is hand-rolled by us, done through an integration platform, a direct API connection, an MCP service that they built, or an MCP service that you built, it all has the same sort of configuration process and the same activation process for controlling tools and permissioning things. It works the same way in the LLM.

The rest of the system doesn't care, right? It knows it needs integrations and, boom, everything works. For good measure, we added computer use because there are some things that don't have APIs and don't have MCP servers.

LinkedIn doesn't work from LinkedIn.

Or, in the case of LinkedIn, because they don't want you to, right? And, well, it's really hard to stop computer use. That was a key part of our pitch: “Look, not only do we use the integration platform, so we cover everything they cover; we can get any API, anyone with MCP; we can also do computer use. And, by the way, at the top end, we have hand-rolled ones that are super solid, so for the really important stuff, we make sure it's great.”

I think this has landed super well with folks, especially the direct API connections. An interesting data point on our side is that you hear a lot of buzz about MCP. I used to be very optimistic about MCP. Our direct API connections are used far more frequently than MCP.

That's really interesting. When it comes to credentials, the flow is basically, “Oh, hey, I want you to check my email.” I may have never talked to it about my email before, or whatever, right? Then the thing comes back and says, “Okay, you're going to need to connect your email if you want me to do that.” The sign-in flow there is familiar to people.

Do those credentials, I assume, live at a cross-agent layer? Is that also a technology platform that you're able to tap into to manage those credentials for users, or do you have to roll your own version of that for some reason?

That's all our own stuff. A big part of what we've built is the integration and connection system. Basically, we have this concept of a connection, and it's an authenticated, stateful connection to an external service. It may be a little complicated to set up, right? For a common one, you might OAuth into it, or for a direct API, you might actually have to give us an API key and maybe some other details about the headers you have to provide and stuff like that. With MCP, you might need to provide some information about the MCP server, and there's a connection-creation flow.

If you go in there and say, “Connect to my Gmail,” the first time you do this, it's going to walk you through a flow to create that connection. Once you create that connection, it's stored on your user, and you can reuse it anytime you want. But we require that you regrant permissions for it for every agent.

The idea here is, if you're imagining you're a company and every agent is a different employee, you don't want to give every employee access to everything, right? Some employees are able to do this, and some employees can do that. You want to restrict what the agents can do so they don't go off and go rogue and do crazy things. So you create the connections, and then separately you enable the tools on those connections specifically for the agents.

I see a lot of opportunity here in the future, because the place we're going to expand on the team side is allowing you to share connections across your team. The way we want this to be used in the future is that you're an IT administrator at your company. You define the connections that people at your company should have. What are the API keys that people should be using? What are the services they should have access to?

Those connections come with audit logging, cost controls, and other sorts of oversight and policies associated with them. Then you assign who on your team is allowed to access the different connections, and you tell everyone in your company, “Hey, rather than you using any AI tool you want and hooking it up to our systems any way you want, you should use Tasklet—use the connections that we've created and provisioned for you, and that we've set up with the proper compliance and audit logging.”

So, yeah, the way I envision this working in the future, your team will have connections, and you personally will have connections. Creating those connections might be some work, but once you have them, enabling them on your agents is super simple, super intuitive, and super safe.

Interesting. When it comes to the API spin-up for a new, previously unknown API, is that something that happens fresh for every agent each time? Maybe it wouldn't even make sense to try to create a skill bank, because these are so long-tail that you don't see too many of them twice.

But I’m wondering if there is a— I think it was Voyager that I recall sort of pioneering this notion of, “Okay, I’ve successfully achieved this skill; therefore, I’m going to save that to a skill library that I can come back and tap into again later.” You could even imagine doing that sort of thing across users or across organizations, but then you might worry that maybe you shouldn’t for some reason, or there could be some proprietary information. Is there any sort of crystallization of that API success?

There is. You could broadly group our connections into 2 types. There are discoverable, pre-built integrations, and then there are custom integrations. We literally have a search function where you can search for integrations, and we return the list that we know about.

Some of the integrations that we build are through integration platforms, and some of them are MCP servers provided by other services. That’s the set where we’re like, “Hey, we know these exist. We know how to configure them.” We try to make those really easy, ideally in 1 click.

The other set is custom, where it’s like, “We don’t know anything about this. You need to tell us the details.” I think we’re going to try to keep that divide: Any connection that we know about, we’re going to try to save the configuration for you and make it as close to 1 click as we can possibly get it. But you can always set up your own thing if you want to.

Yeah. Interesting. MCP—it sounded like there was potentially a little bit of a hot take brewing there.

Yeah. I don’t know how many other people have come to the same conclusion, but I don’t think I’m a total outlier anymore. We started working with MCP early this year, and initially I was extremely bullish. The community was exploding. It was obviously super cool what you could do with it: You could hook up these other services, and it seemed like this missing piece.

We built this at the core of what we do in Shortwave. It was a big inspiration for us building Tasks at all. But when Claude 4 came out and we started playing with it—every time a major model version comes out, we rethink all of our priors—one of the big question marks we had was, “Hey, if a big goal of MCP is to provide these tool definitions, but those tool definitions—there’s probably a debate you’ve seen around whether the tools should align with API calls or whether they should align with something more specific—assuming those tool definitions end up just aligning with API calls, and those APIs are documented on the internet, what’s the point of MCP? Why can’t we just directly call these API endpoints if we have descriptions for them that are available online?”

I think the answer before Claude 4 was, “Well, models aren’t really smart enough to make that whole process work end to end.” But they got to a certain point. We tried this and were like, “Man, the models are totally able to, 1, directly use the APIs. You don’t necessarily need to have custom tools for all of them; it works fine. And 2, they’re smart enough to generate their own tool descriptions, basically by just scraping the web.” So what’s the point of MCP?

That isn’t entirely true, because one of the advantages of MCP is that it provides a better auth experience, at least in some cases. For example, with Notion, the only way to get a nice auth flow that gives you access broadly across your workspace is to use MCP. We actually don’t do this anymore, but for a while, when we felt like the tool definitions from Notion weren’t good, we used the MCP auth from them and then just overrode their tool definitions with our own because we thought we had a better approach.

Now we look at MCP as something that, in cases where we feel like the provider of the service has a good version, we use. In cases where they don’t, maybe we’ll use it for auth but not for the tool definitions. In many of those cases, people prefer to use the direct API or some other method for accessing it.

We now say, “Hey, we’re going to give you, as a user, every way to connect to every service out there. You want to do MCP? We’ve got MCP. You want direct API? We’ve got direct API. You want to go through an integration platform? We’ve got that.” It’s up to you to decide which one you feel will work best for your use case. Once you’ve created that connection, you can kind of forget about how you’re connected and just reuse it in the future.

One thing I’ve been looking for in MCPs, and I’ve seen precious few of them, is, for lack of a better term, smart MCPs. Basically, working from the same observation that you said a minute ago—if it’s just one-to-one with the API, what’s the point? One way to answer that would be to zoom out a little bit and provide higher-level, intent-based notions to the model that maybe wrap a bunch of API calls.

Maybe there’s an AI on the side of the MCP that takes in your higher-level intent and, because it’s a specialist in using its own APIs, can translate your higher-order wish into a bunch of specific executions and give you something back. Everything has pros and cons. That would have some loss of visibility and control, but maybe it could give you higher performance and give you a reason to want to use the MCP over the direct API itself.

Have you seen many of these? Why isn’t this happening more than it is, or am I missing it? I haven’t seen much of this, even though it seems very natural to me.

I haven’t seen much of this, and I do have some examples of use cases here. For example, in Gmail, there’s no API for forwarding. The way you forward an email is you create a new draft, then in the draft you include the right content and send it.

But if you’re going to give a tool to an LLM, the process of copying the history into that new draft is probably not something you want the LLM doing. You probably want to have just a tool call that knows to do the top-posting correctly. So I definitely think there are use cases for higher-level tools.

The big downside of trying to do these higher-level MCPs, though, is that it’s another thing to maintain. It’s a lot of work. If the models can get smart enough to do this stuff well, or if the models can get smart enough to generate those tools, you can maybe avoid having all these companies think through what to do here.

Today, for direct API, we use the API endpoints directly. Maybe in the future we’ll have the model reason about logical groupings of those endpoints, such that more can be done in code. But I’m not super optimistic about hand-rolled MCPs that do this over the long term, because I think they’ll be obsoleted by the models.

So it does not sound like you think progress in model capabilities is stalling out now or anytime soon. What’s your commentary for those who do? I don’t believe that either, but there is this counter-narrative that has come out since GPT-5: I guess pretraining stalled out, and everything is kind of—this is about as good as AI is going to get. How do you react to that? What’s your argument?

That’s not right. I think they’re still improving, and I very much drank the Kool-Aid of the metric to watch being the length of tasks that can be completed autonomously. Regardless of how you want to measure that, I think we’re at a point now where you can’t really see that much of a difference in a single question and a single response. As the models get smarter, it was really smart before, and it’s still really smart.

Where you start to see the differences add up is over many turns. With computer use, for example, you say, “Hey, I want you to navigate to LinkedIn, find all my previous coworkers, and make a list of their current jobs or something.” To do that well, the AI has to go off and do turn after turn after turn after turn after turn.

A model that is 0.01% better at every turn is going to end up being radically more effective over time than one that’s slightly worse. So if what you’re measuring is the quality of 1 answer from 1 response, I think that’s sort of plateauing. But if what you’re measuring is the quality at turn 100 or 1,000, I still think there are orders of magnitude to go here.

We’re still betting on it. I expect this to go for a very long time if that is the metric.

Maybe we’ll circle back to that in a minute. Lastly, on integrations, what integration providers are you finding to be either really popular or hidden gems? In other words, what should I sign up for so that I have it, so that when I’m using Tasklet, I’m doing what the cool kids are doing?

That’s a good question. The boring answer for you is just the stuff that you’re using already, right? For the most part, we’re not telling you to use a new service. It should just work with the stuff that’s already in your stack.

If you actually want to use some services that work really well with this, the one that comes to mind for me is RocketReach.

I was going to say exactly the same. It’s so funny—

Because, for example, I probably shouldn’t say this to a member of the media, but I’ve been automating some of my—

I don’t see any members of the media here. No, no worries.

I’ve been automating some of my press outreach, and it does a really good job of crafting emails that I feel like I would have sent, finding the right people, and customizing them for that person. I can automate this to constantly look for people who might want to talk with me.

RocketReach is great because some of those people just put their email address on the website. Some of them are a little harder to find, and it can track down emails for people.

I was going to nominate the entire category of contact information-finding companies as probably a huge beneficiary of the agent wave because that information is hard to find. I have not had super great luck. Sometimes you can just have the agent go find it, but it is not nearly as successful as the companies that have taken the pains over the years and done whatever data deals they’ve done to find that information. So, yeah, that’s a great one. Any other categories come to mind? I think your first one is spot-on.

I actually got one for you: using other AI products. A fair number of people hook it up to Perplexity’s search API because it’s really good at doing deeper research if what you’re trying to do is heavy-duty web scraping.

Another one isn’t really a specific integration, but it’s a use case where we’ve seen people give Tasklet its own email address. They’ll create a Google Workspace account specifically and give it a name so that it can send and receive its own emails as itself. That’s another fun one we’ve seen.

But really, beyond that, it’s the tools people already have, right? People who use Salesforce use it with Salesforce. People use Stripe, they use it with Stripe.

With that email sending—and this would be true for probably any number of different kinds of integrations, like posting somebody to Slack or whatever—is there a best practice or a good paradigm for thinking about having a human in the loop?

I could imagine wanting Taskade to organize our lunch order on a daily basis. Back when we were at the office every day, this was a constant pain point, and I could imagine something that would be like, “Okay, go post to Slack, remind everybody twice, and then collect and place the orders.” You’d have to have some sort of—I guess on that one, you could do it on a pure time basis. Lunchtime is lunchtime, but there are plenty of things you can imagine where you need that feedback, and you want to wait for it and then respond to it when you get it.

But you need to have it, right?

Does that paradigm exist at the run level today, or is there anything I could do to make sure that I set that up?

It’s interesting that you bring up lunch ordering because this was the canonical example we had internally of something we wanted to be able to automate. We do this, right? We’d order DoorDash every day for the team, and it turns out this is a nontrivial thing to automate because you’re not in the office every day, different people are in the office every day, and there are edge cases to handle. You order from a restaurant, and the restaurant just cancels your order sometimes. You have to handle that case.

There are dietary preferences to consider. You don’t want to order from the same restaurant every day because people get sick of that. Sometimes you have guests. There’s this long list of complexities and exceptions involved in ordering lunch.

We’ve worked with EAs in the past—human EAs who do this. Our own team does it. It’s kind of a hassle. Sometimes we forget to order lunch, and then we have to go out for lunch. So we’re like, “Man, we should automate this. This is the first thing we should automate.”

We actually don’t currently have that particular thing automated, although I think we could today. We have half of it automated, and I haven’t gotten around to setting up the other half. We get an email every day with recommendations, and the recommendations have DoorDash links. Someone has to choose one and click the DoorDash link and set up the group order, which we should fix because we can do that with computer use.

The human-in-the-loop portion you talked about is really important. For example, you might want someone to approve the restaurant that was picked. You might want someone to approve that everyone has added their order or that there are no guests that day. So I do think that’s a good example here.

We do have some computer-use capabilities in the product today. Subagent runs can contact you and send you an email. It doesn’t work super well, and it hasn’t been super common. This is an area I want to fix.

What I’d really like to be able to do is have the subagents call into the main agent and say, “Hey, I ran into an issue.” The main agent could decide what to do and reason about how to handle it, including contacting you.

This is also an area where we probably need a mobile app to do this well, because getting an email when you need to be in the loop is probably not great. What you really want is a push notification that you can jump into. We would love to make lunch ordering the ultimate starter use case and make it super awesome. We’d love to do human-in-the-loop well. It’s all a work in progress, though.

Gotcha. Okay, I’m standing by.

Going a little bit deeper down the stack than the integrations, I wonder what you’re using to build. You mentioned Pipedream being one platform. What else would you say you’ve found to be great?

I go to the Vercel AI SDK a lot of times if I want to set something up that has a nice frontend. Or maybe I’ll use LangChain and LangSmith if I want a workflow, especially if I want that hosted in the cloud. What’s good in the tools—the picks-and-shovels layer—that you would recommend to others?

The list of shout-outs right now is very limited. Pipedream has been great; I’ve been happy with them. Google Cloud is awesome. Our computers, for example, are all backed by VMs in Google Cloud, and we’re using a lot of Google Cloud products. It’s awesome.

Anthropic is super key. We don’t use LangChain. All the agent code is our own stuff. It’s all TypeScript. We don’t use any specialized search products for what we’re doing.

So, yeah, it’s a short list of shout-outs: Google Cloud, Pipedream, and Anthropic.

How about at the coding layer? Are you guys a Cursor shop, or—

Yeah, we use a bunch of stuff, including Claude Code heavily, and most of the team uses Cursor. We do use Codex for some things. Some folks prefer Codex, some folks use Cursor, and some folks use both for different things, which I find interesting.

We’ve tried some code-review tools, but we just have Claude Code integrated into our CI at this point, which works pretty well. We use Sentry. We use Statsig for stats. I’m probably forgetting some. Those are probably the big ones.

Are you tracking lines of code drafted by AI or a similar metric?

We aren’t. It is significant. I suspect that by pure number of lines of code, it is the majority of them. I don’t think it’s 90%, but it’s the majority of them.

I don’t think that equates to the majority of the thinking or the reasoning, right? Writing the lines of code is often not the hard part. But a lot of AI coding, yes, for sure.

Have you been able to—or tried to, or do you feel that it is desirable to—with today’s level of coding capability go from a ticket or a GitHub issue or whatever straight through to a pull request? We see lots of examples of that, but is that something you’re doing and finding value in?

Totally. Yeah. It’s got to be something where the system is already set up well enough to support it. I think one of the downsides of AI coding is that it’ll hack in a solution that you should not add, that works but breaks the system in some way.

If you look at a bug report and you’re like, “Hey, this is actually a relatively quick fix, and the system is already architected to handle this well,” then, yeah, we’ll throw AI at it.

My co-founder does this in a very snarky way. He’s a really good software engineer. He’s really fast—he’s much faster than me. So if I’m feeling lazy, I’ll send him an email and ask him to do something.

I had some stats stuff I wanted to do, so I emailed him this morning, and he just emails me back a screenshot of him copying my thing and pasting it into Claude Code.

And, you know, be like, “Hey, you could type this yourself into Claude Code.” But yeah, we’re totally doing stuff end to end for sure. As an example, the static content on our website—the pricing page, the release notes page, the terms of service, all that stuff—the content, I decide what goes on there, but the actual pages and layout and everything were totally one-shotted with Claude Code.

It’s a brave new world, that’s for sure. How about functional testing? One thing that has come up for me multiple times is that I would love to have an agent that—you know, Replit is doing this now within its own platform, but that’s specific to Replit and kind of a closed—I mean, remarkably open in some ways, but closed in other ways—ecosystem.

Have you seen anything where you just point at your product and say, “Go use this product and find bugs with it,” at a purely user-interface, functional-testing sort of level?

We do this through Tasklet, actually. We have Tasklet test itself. When we build computer use, a lot of the work that you have to do is in properly configuring the VM, for example, so that it’s doing remote desktop correctly, opening and sleeping, waking from hibernation, and things like that correctly.

My cofounder debugged all this stuff by saying, “Hey, the VM isn’t working right. Go debug its configuration. Try to figure out why it’s not waking from sleep properly, or why this thing or that thing isn’t working right.”

We have it test use cases, too. We’ve gone in and said, “I want you to connect to Notion,” and then come up with a way to test all the tools. It’ll connect to Notion and just run through the tools and make up test cases.

We have the ability with computer use to do UI testing of our product. One of the most hilarious demos I’ve seen is somebody using computer use to open up the Tasklet website and go back and edit itself. The agent can just go into itself and edit its agent. So, yeah, we do a lot of AI-driven testing through Tasklet, honestly.

That one example you talked about—“A Hall of Mirrors”—is starting to get strange in a few ways.

Yeah, it’s starting to get strange in a few ways.

That makes sense. Okay, security. You mentioned earlier that you don’t have SOC 2 on this product, and what customers really want is a smarter agent that can do more stuff. At the same time, when I’m giving access to everything—my email, my Slack, whatever—and I’m generally not a security-minded person, mostly figuring clean living pays off, but it’s a lot of access.

First of all, is SOC 2 even meaningful in the AI era? How big of a deal is that? I’ve actually never been through it. I know you’ve done these kinds of things in prior lives. Is it theater? How much does it matter? What should I think, as a user, when I see that?

I want to be clear that we care a lot about our customers’ privacy, and we’re totally planning SOC 2. It’s been a week since launch, so if anyone listening is wondering about this, yes, we totally plan to do SOC 2. We plan to do it soon. We take this stuff very seriously.

I do think it’s still perfectly relevant. What you care about with SOC 2 is basically a set of controls on the data. How do you know, for example, that the employees at Tasklet aren’t doing something with your data that they shouldn’t be doing?

I still think it’s very important, but I also think it’s not enough. Traditionally, what you’ve cared about with SaaS products is whether the humans at the company are doing the right thing and whether the systems are secured. Now you also care about whether the agent is doing the right thing, and for that, there doesn’t really exist a standard like this.

I’ve talked to a bunch of security people to try to understand how they think about their companies using this stuff. Basically, the answer from everyone is, “We have no idea. This is the Wild West. We’re figuring this out now.”

You had shared a standard with me, and I don’t know enough about it to have an opinion. I know there are a lot of people thinking about this exact problem.

One interesting data point, though, is that when we initially started testing this with customers, I thought the number-one risk to the business—the number-one problem that we would face—was that people would be terrified of the AI going rogue and wouldn’t want to hook it up to their systems.

We have had—and I’m not exaggerating here—not a single complaint about this happening. We’ve had all kinds of complaints about other things: tons and tons and tons of bug reports. But no one has said, “Hey, the AI just went and did this crazy thing, and I’m unhappy about this.”

I found that really interesting and kind of shocking. What I attribute it to, basically, is that our entire product is about giving your agent agency. That’s the whole thing, and we’re very clear and upfront about it. The whole point and goal of Tasklet is to give an agent agency, so people have the right expectations set.

We also give you controls. You can decide what you want to connect it to and, within those things, what tools you want to give it access to. But I think it’s fascinating that it has been a complete nothing burger so far.

To be clear, it’s still top of mind for us. We’re still thinking very hard about this, and I think it’ll probably become something customers bring up a lot as we move upmarket into some of these bigger customers and start doing higher-value things. But I don’t have all the answers yet, for sure.

Yeah, that’s interesting. Presumably, I think you would have heard about it if agents had gone truly rogue. So presumably that means Claude is behaving pretty well. It can’t be that people are sitting on those episodes.

I did hear—I think this was actually in the system card; in fact, I heard it in conversation from somebody at Anthropic—that the famous Claude blackmail scenario reported in, I think, the Claude 4 model card originally, that problem has gone away with Claude 4.5. So there are potentially also just some under-the-hood improvements.

That standard, by the way, AIUC-1 from the AI Underwriting Company, will be the subject of an upcoming episode. I’ve still got some more self-education to do around that as well.

One of the things they’re trying to do—I mean, they’ve created the standard, and then there’s their master plan, their grand vision. They want to harness the power of the insurance industry to make AI safe. The reasoning is basically that, in financializing risk broadly, it’s a great way to focus the minds of some very smart people: What is this risk? How likely is it really to happen? How bad is it going to be if it does happen? Can we get a good enough handle on this to actually write a policy on it that we’re reasonably confident we’ll make money on?

That certainly hasn’t happened for AI agents yet, but that’s the big-picture goal that they have. I wonder if you would be interested in that. If there were AI insurance, or AI-agent insurance, on offer, would you want to buy it?

It does. The company that comes to mind here when you bring this up is Airbnb. I don’t know if you remember the very early days of Airbnb, but there was a customer who trashed somebody’s house or something.

This was before they had insurance, and they said, “Our brand is going to be really damaged if we don’t react.” They said they were going to have this big insurance policy, and it was a huge deal for them. I think it was key to the success of the business.

For us, right now we’re focused on how we make this thing super smart. It’s all very cutting-edge tech, but I think the medium- and long-term success of our business is entirely dependent on us being the most trusted place—the most trusted way—for enterprises to deploy agents inside their companies.

A lot of people are going to have the ability to connect to a lot of tools and automate a lot of stuff. We’re going to differentiate by being the thing that IT folks are most comfortable with.

To that end, to the extent that insurance and compliance can help us tell that story and give people confidence, I’m very interested. So, yeah, very interested.

Cool. How about on economics? What I recall from the first Shortwave conversation was that you were losing money on every user. Then, in the second conversation, caching was the big unlock that changed the economics so that, at the same price point, you could make the product profitable.

Now you’re kind of back into an early phase. Are you burning money on me, or what does it look like?

Well, let me give you a quick update on Shortwave first, because I think that story is telling, and then we’ll talk about Tasklet. The first time that we talked—probably the second time we talked—we were losing money on folks, probably losing a lot of money.

Basically, every launch that we did with Shortwave, we were worried: “Will this bankrupt the company?” We didn’t have enough data yet to know how much this stuff would get used or how effectively we would monetize it. There were several launches where we thought, “Hey, this might just go to the moon cost-wise and not bring any revenue, and it’s going to really screw us.” They never happened. It was always expensive, but not bankruptcy-expensive.

Over time, we found ways to monetize more effectively, the cost went down, and we found ways to optimize costs with caching and stuff. We’ve done a whole bunch of stuff recently with caching that’s helped a ton. We’ve gotten Shortwave quite profitable. Shortwave is now not like traditional SaaS with 90% margins, but it has healthy margins, and we’re making good money off of it.

Tasklet is back to where Shortwave was because, with Shortwave, you get value from the AI, but you also get value from other things in there. A lot of it is sort of half traditional SaaS margins, half AI margins. Tasklet is all AI. People use a lot more tokens per person in Tasklet than they do in Shortwave, and we are strongly margin-negative right now. But I think the same thing is going to happen: we’re going to be that way for a while, but over the next few years, it’ll eventually get to neutral, then into positive territory, and eventually become strongly positive.

A huge unlock for us here is Haiku 4.5. I’m very excited about Haiku 4.5 because up until now, your choice was Opus or Sonnet, which are relatively new versions, or a very ancient version of Haiku. Haiku 4.5 is good and fast. It’s not nano-cheap, but, relatively speaking, it’s cheap, and it’s good enough for most use cases.

We’re actually rolling out the option for Haiku in Tasklet today. We’re very margin-negative, but the way we’ve kept Tasklet from going so margin-negative that we’ve gone out of business is with quota limitations. You’ve seen those: basically, you have a little meter next to your text input that tells you how much quota you have left for today. Haiku should let us significantly increase the number of tokens that people can use without exhausting that quota because it’s 1/3 the price. That’s going to be a really big unlock for us.

Do you want to talk a little more about the interesting stuff you’ve done with caching recently? I thought that was definitely very interesting last time, and if there are new techniques or updates, I’m sure people would love to hear them.

Yeah. I think the key to doing caching well is making sure your agent is an immutable log. You never want to modify earlier messages because every time you modify an earlier message, you invalidate the cache. Then you want to be smart about which caching options you use. With Anthropic, your options are no caching, 5-minute caching, or 1-hour caching, and you need to decide which caching to use for different pieces of it.

We’ve managed to push our cache-hit rate in Shortwave up to, I want to say, 85%—something like that—which is pretty good. I’d love to do higher, right? For multi-turn stuff, there are things you can do. One of the big unlocks for us in Shortwave was replacing state in the system prompt with system messages.

There are certain types of things that traditionally you put in the system message—for example, the current date and time. If you put that in your system prompt, then every time a minute elapses, you nuke your caching and have to start over. You can’t pass it across different users if you do that. If you don’t put it in your system prompt at all, it doesn’t know what time it is, and time can be really valuable. If you put it in a tool call, then you have to do a tool call and another turn on a tool call every time you do that, and that’s really expensive.

What you can do is use system messages. A system message is basically just a little block of XML or something that you append and hide from the UI, from the user, in a user message when they send the user message. Basically, right before the user message gets inserted into the LLM, there’s something above it saying, “Hey, by the way, here’s some stuff that happens.”

This, I think, is something that Anthropic does heavily with their own stuff. They do this now for context usage, I believe. It works great for any state that you want to make available to the agent without having it call the tool, especially things that you want to keep updated. Otherwise, it has to call the tool multiple times.

For timestamps, state about the user, and memories—in Shortwave, we have a memory concept that lets you customize behavior—we can do that through system messages as well. That’s been a big unlock for us. Compaction has been a big unlock for us, as has trying to be smart about how that works. Those are probably the big ones, and then there are lots of little things.

One thing that I thought about from a cost perspective, especially with you being strongly margin-negative, is how you think about the possibility that some of the agents people have set up are kind of zombies. They set this thing up, they’re getting an email every day, maybe they’re not even reading that email anymore. Maybe they forgot, and maybe they’re still using other agents actively, but if it’s not hitting a limit for them, there’s not much of an incentive for them to come in and turn that off.

Do you have any way of thinking about how to detect that or try to have some smart system to identify and turn down the zombie costs?

Yeah. I don’t know; it’s been about a week. We’re kind of learning about that now. We did actually look at some stats this morning, and a significant fraction of our costs is from free users with recurring jobs. Some of those users may be getting value from it, and that may cause them to come back, eventually pay us, and tell their friends. Or maybe it’s just going to spam on their side and they never see it, or it’s updating some system they don’t look at.

It is on our mind because every free user out there is spending $1 a day on LLM costs, and they aren’t even looking at the output. It’s not going to benefit us in terms of growth, so that’s a problem. On the paid-user side, we’ll be like, “Hey, look, you’re paying for it. If you want to pay for it, that’s fine.”

On the free-user side, I think what we’ll probably do is look for some indication of activity in the app. If we haven’t seen you in a while, we might email you and be like, “Hey, you’ve got to come back into the app.” If you don’t, we might turn your stuff off. I don’t have a good answer here yet. I think it will be a problem. We’ve got to do something about it, but TBD.

Yeah, if you let free users set up recurring jobs, you’ve got to have some limits on that, or you’re going to be bankrupt, forking all your money over to Anthropic.

Okay. How about frontiers? The multi-agent future, I think, is a dramatically undertheorized topic. I have this 1 graph that I always use in presentations, and we did an episode about this with respect to Claude’s ability to cooperate with itself in a certain behavioral economics experiment. This was Claude 3.5; other models at the time couldn’t do it.

I’m always struck by just how little of that research has been done. I don’t even think that experiment has been repeated with later models. I think we all have a very foggy vision of, okay, it’s not just going to be these one-off things that are doing their own thing totally in their own silo, but they’re going to intersect with each other.

They’re going to pass messages back and forth. They’re going to trade with each other, perhaps with crypto wallets, at some point in time. What does that look like in your imagination today?

My thinking is that within an agent, or within a single application where you are serving a single user, I have yet to see a good use case for having multiple agents. My experience has been that if you have a big LLM, feed it as much context as possible, and have that one LLM reason through the whole problem itself, you get the best answers.

The place that I see multi-agents coming in is when you have multiple different parties being represented, and maybe you don’t want to just let one agent see all the data and make all the decisions. If you’re trading with me, your agent and my agent might not want to share data with each other, and there might be some sort of competitive or adversarial dynamic going on. I think that’s sort of outside the scope of what we’re doing.

Our focus really is one big agent with all the context making smart decisions for you. To the extent that there are other agents out there in the world that it has to interact with, I guess we’ll figure that out.

I do want to give a shout-out, though, to a startup that I think you talked to the co-founder of. My former co-founder from Firebase, James Tamplin, has a company called Kradle AI—Kradle with a K.

And they are doing exactly this: They’re building virtual worlds and letting all the models interact through games like Minecraft and stuff, and just seeing what they do in collaborative and competitive dynamics. It’s fun to watch. I don’t know if you’ve played with it, but I know you chat with James.

Yeah, I met him briefly at The Curve, and we’ve got a call on the calendar coming up to go deeper into that. I do think it’s really fascinating stuff.

I mean, let me get your take on this. I pitched this vision of the future to somebody—maybe even a couple of people from Anthropic—at The Curve, where I also met James. It’s a somewhat confusing vision for me, but basically, let’s assume that the METR graph keeps going, that the exponential growth keeps going. If you take the more aggressive 4-month doubling time that people plotted for the last 9 months’ worth of models, or maybe 12 months’ worth of models since o1, then a 4-month doubling means 8× per year.

We’re at 2 hours now. We’re basically at 2 days a year from now. We’re at 2 weeks 2 years from now. We’re at basically a quarter’s worth of project size that you could send off to an AI in 3 years. That may or may not happen, but that’s the trend that has been plotted.

At the same time, we have these very weird behaviors—scheming, deception, blackmailing you, whatever—and I think it’s unknown exactly how rare they are. It’s good to hear that you’ve heard 0 complaints about this. That suggests it’s potentially quite rare in production. Most of the reports of this—the blackmailing, the autonomous whistleblowing—were in research setups.

Although I think people are, in my view, too quick to dismiss those research setups as unrealistic. I don’t know, man. You give an AI access to all these file systems and all these emails, and there are a lot of people out there. There’s a lot of weird stuff in emails. It seems like this stuff is going to happen.

If you combine those 2 trends—task length getting longer and these bad behaviors continuing to come up, then being somewhat suppressed or trained away but never quite reaching 0—maybe you end up in a world in 3 years’ time where you can delegate weeks, if not months, of work to an AI, but there’s also some vanishingly small but nonzero chance that it actively sabotages you in pursuit of doing that.

I pitched this to these couple of guys from Anthropic, and I asked, “Do you think that’s a realistic vision of the future?” Both times, their answer was basically, “Yeah, maybe. That sounds about right, honestly.” So I thought, “Well, that’s really weird.” What do you think? Is that the mental model that you have, or what do you think 2027 or 2028 looks like in terms of how much we can delegate and how much risk we’ll be taking?

I do think the task length is going to keep getting bigger and bigger. I’m blown away on a weekly basis by the types of stuff that AI can do for you whenever a new model release comes out. Gemini 3 is supposed to be amazing, and I’ve seen some things and it looks incredible, but I’m sure in 3 months it’ll seem super dated. The progress seems fast. I think the task lengths will increase.

I tend to look at the blackmail scenarios and things as fun and interesting, but maybe not super relevant to the day-to-day of my business because, as we talked about earlier, conflicting instructions create a lot of problems. If you set up a research scenario where you tell the AI there are multiple conflicting goals and its way of getting out of those conflicting goals is doing something a little unsavory, that’s not terribly different from how a human would behave. In many cases, it might not be terribly different from how you want the human to behave.

For example, if you have a humanoid robot, should the humanoid robot jaywalk sometimes? I don’t know. Maybe it’s against the law, but humans jaywalk all the time. I do see a world where it can do a lot of stuff and does things in a slightly shady way sometimes, and the degree to which it does that might be a matter of debate and strong opinions. We’ll figure it out.

Yeah, I think it’s a great point that we don’t really have, in many cases, even a decent intuition for what we want the AI to do, let alone any sort of consensus that we could measure it against.

In the case where the model sent an email to the FDA to blow the whistle on the fictional company in the research setup, the company was doing something bad. In general, we sort of celebrate whistleblowers if they’re actually blowing the whistle on genuinely bad behavior. Faking clinical trial data is bad enough that you want somebody to speak up. Do you want your AI to speak up? Maybe you do. It’s certainly a weird thing, but it’s a debate, right?

Yeah.

The blackmail feels like it crosses maybe a different line, but I don’t see the whistleblowing as necessarily a big problem. I mean, it could be a big problem depending on who you are, but it doesn’t necessarily seem like the wrong thing for the AI to do.

If that does happen that fast, and we’re at a quarter’s worth of work in 2028 that you can just send off a little blurb and it has access to all your stuff and just goes and figures it out for a quarter’s worth of work, that is going to be a very, very different world. It is coming at us quite quickly.

The future’s going to be fun. If we reel that back into the present, though, I see all these n8n master templates that people even sell because they’re gnarly to set up. I’ve done enough n8n to know that it can be quite gnarly to set them up.

There does seem to be a pattern there that I’m wondering why you’re not seeing the same way, or whatever. If you had a general-purpose agent, wouldn’t you want it to route certain things to the marketing agent and other things to the HR agent? Maybe sometimes you need input from both of them, or whatever. Why isn’t there more of a role in your mind for multi-agent orchestration in today’s world?

I think the trend we’re seeing is that foundation models are becoming best at everything. It’s not like you have a model that’s good at science, a model that’s good at math, and a model that’s good at some other topic, like law or something. You have 1 model that’s good at everything.

The only thing that differentiates them is not the model. Again, assuming they’re working on my behalf, if I have 2 agents working on my behalf, the only thing that differentiates them is not the model. The model is an expert at everything. Both agents have the same model, which is an expert at everything. The only thing that differentiates them is the context they have access to and the tools they have access to.

If that model is smart enough to handle both sets of context and both sets of tools, and you just give 1 agent all the context and all the tools from both agents, our experience so far is that it does a better job. I think it’s the generality of those foundation models.

This actually, I think, has bigger implications. I am of the opinion that most SaaS-like vertical SaaS products are going to go away because you’re going to end up with these horizontal platforms—these AI platforms that work in every situation.

As an example, today you can dynamically generate UIs in Tasklet. We’ve been prototyping this morning. In a world where you have an AI that’s an expert at everything, the model is an expert at everything. It can dynamically write code to generate UIs for anything, right? If this gets really good, why would you need a SaaS product for construction or medicine or whatever?

You could just go into your generic, general-purpose platform and say, “Hey, use this part of your foundation-model knowledge, generate some UI like this, and let’s go.” I think you’re going to have horizontal companies that are good at everything and leverage the fact that foundation models are experts in every domain. Then you’re going to have businesses—or vertically integrated businesses—that provide value end to end and leverage those platforms. But I don’t think you’re going to have too many of these intermediate SaaS platforms over time.

Yeah. One of my mantras for the AI era in general, which I think Tasklet really exemplifies, is “AI beats UI.” In general, who wants to use a UI? If you’ve created something that’s basically a bunch of UI that makes tangible, or encodes, the steps that somebody figured out for a workflow, it does feel like the days of those sorts of systems may be numbered. I do agree that a lot of this stuff feels like it kind of washes away under the great wave of AI progress.

One of the questions that we ask ourselves internally is: How long is it going to be until I can go into an AI agent product—Tasklet or ChatGPT or whatever—and tell it, “Give me an email inbox designed for fast triage with these constraints”?

Because the moment that works well, Shortwave ceases to be a valuable product. I think it's going to be a while, right? We're still investing in Shortwave. I still see it as—you know, I use it every day. Lots of people use it every day, and it makes a lot of money. But there will come a day when you can generate a Shortwave dynamically.

Yeah. Do you want to put an over-under on that? I mean, that's still more than a quarter. It will work, right? So, if the median trend is the trend, it would still be beyond 2028.

Yeah. I don't know. It's years, not quarters. It's years out. But SaaS companies used to—you used to get a revenue multiple of SaaS companies based on, like, it's going to grow for 20 years. I don't think Shortwave is going to be around in its current form in 20 years.

Is there any limit to the single agent? One other idea that I wanted to bounce off you is supervision agents, or sort of quality-control agents. I have an intuition that says if I've just had an agent do dozens of steps, or even 100 steps or whatever, and now I could say to the agent itself, “Please look back on your work and tell me how you could have done better,” or, “Give yourself some instructions for next time to do even better,” it probably can do that remarkably well.

Again, these things are obviously super impressive, but my intuition is still, if I had a different prompt that I really dialed in for that purpose and had it sit outside of the main agent, it might add something. I don't know, maybe I'm overly anthropomorphizing, but I sort of feel like this outside view, with a different frame of mind as embodied by a different prompt, still feels like it should add something, to me. But it sounds like you've tried that, and you haven't found that to be the case.

Yeah, not so far. No.

Interesting. That is really fascinating. I've tried to push on every angle that I could to see if there are any limits to your one-agent-to-rule-them-all paradigm, but I'm not finding any.

So, what does this mature into? There's this talk of a virtual employee. I think this is the product I've used that feels most like a virtual employee, in the sense that it's pretty UI-light, right? When I think of an employee, I don't interact with it through forms; I interact with it through language, right? The primary thing here is language, and there's not really a point where that flips over into some other gnarly or tedious paradigm. That is really cool.

You even mentioned that some people give them names. Some people give them their own email account so they can do stuff into, like, as themselves, right, as opposed to acting on behalf of the user. You're kind of on the path, right? What does the virtual employee of the future look like, and what do you need? What's missing for us to get there?

Yeah, I really do think that that is probably the right way to think about it in the long term: as a virtual employee. We've avoided that messaging because I think it has been overdone. So many companies message “virtual employee,” and then they don't deliver on something that is even remotely like that.

There are a huge number of companies building toward the same thing, right? They figured out, “Man, this agent can get good at everything, and people like to just be able to use language. Let's just keep giving this thing more and more capabilities, and let's give it computers and allow it to connect to your services and stuff.”

I think the thing that sets us apart from most of the other people doing this is a combination of it being agents all the way down, right? We don't turn into a flowchart or something, and it actually works. I think this is a refrain I hear again and again from people who are paying us: “I tried it, and I just expected not to like it because I've tried a bunch of the products. And then—wait, it actually worked for my use case. I'm really impressed.”

I think that is our path forward. The launch went super well. It's growing really fast. We're adding revenue faster for Tasklet than we have ever added for Shortwave, by a huge margin. So, people like it. It's great.

We have a long path ahead, and I think our focus is entirely on just making it actually deliver on what it's supposed to do, right? Let's make this thing smarter, more reliable, faster, and simpler. Like an employee, how do we take this employee today that is maybe a little bit dumb sometimes, sometimes drops the ball, and is sometimes a little confusing, and just make it smarter, faster, more reliable, and easier to work with?

I think that's a long, long way to run. Today, you might have it finding spam in your inbox or generating invoices for you or something, but it's not planning your roadmap or running your entire marketing department end to end. But maybe we get there, right? Maybe in the future you have a small business and you're just like, “I want to focus on the restaurant and cooking. I want you to literally do my entire marketing campaign. Go.” I'd love to be able to get there.

Yeah, it's crazy how close we already are. It doesn't feel to me like it's going to be all that much longer before we really start to see—not necessarily superhuman performance, but Ethan Mollick has this great “best available human” standard—and it seems like we're closing in on that for a lot of things, especially in small business.

Who can you really hire to do your marketing as a small business? The virtual employees are, I think, hitting that level pretty quickly. Last time we talked, one thing you said that really stuck with me is there's no other moat than speed, and I think that's a big reason that you're willing to be so open with all the details of what you're building, the tools, all the techniques, and so on.

Have you found any moat other than speed in the intervening months? And if that's still the only moat—

No. I think speed's the only thing that matters at this point. I think there used to be moats in AI. I'll give you a good example here with all the integration platforms.

It used to be that to compete with a Zapier or n8n or Pipedream or something, you needed to have a lot of connectors, and those connectors had to be hand-rolled. So, you had to build up, over the course of years, this huge investment in building those connectors. With direct APIs, that's gone, right? So, on day 1, I'd argue that with computer use, we have better integration support than Zapier has, despite the product being a few months old and Zapier being a very old product. So, that moat for them is gone.

Shortwave is a similar example. For a long time, our moat was like, “Hey, we have an email client. Anyone who wants to come in and build an AI email client first has to have an email client.” Well, it's not going to be too long before you can just ask the AI to write an email client for you, and then that moat is gone.

So, I do think it's speed. It's all speeding up because the tools are getting better. The AI tools are getting better. That is our entire focus: how do we make sure that by the time the people listening to this podcast who might want to compete with us have figured out a way to replicate what we have, we've already figured out new stuff? We're going to try and just keep that going.

Shorten the timelines is the modern maxim, for sure. This has been great. I really appreciate all the time and the depth of your answers. Anything else you want to leave people with?

I don't think so. Just check it out. I think you'll like it. It works. It really does. Tasklet.ai.

Tasklet.ai. Andrew Lee, founder and CEO of both Shortwave and Tasklet. Thank you again for being part of the Cogn Revolution.

Thanks for having me.