[BidClub_]
Latent Space · · 121 min

[One Million AI Engineer Special] How Stanford Teaches AI-Native Engineering — Mihail Eric, Monaco

Mihail Eric

AI & SoftwareTechnicalCompany Building
YouTube ↗
TL;DR
  • AI-native hiring raises the entry bar toward senior-level judgment. Mihail Eric tells Stanford students that “every company needs you, but the floor is much higher”: implementation is increasingly delegated, while architecture, system trade-offs, and understanding remain valuable. His response is apprenticeship through real open-source contributions, where maintainers teach judgment that agents cannot confer simply by generating code.
  • Verification and maintained intent become the constraints as code generation accelerates. At “30 plus PRs a day,” traditional review makes humans the throughput ceiling, but indiscriminate merging defeats the purpose of engineering. Eric supports selective auto-approval grounded in verifiability; the hosts press the unresolved question of preserving a feature’s accumulated requirements across many individually plausible changes.
  • A simple agent loop is easy to reproduce; reliable work across a large codebase is harder. Eric teaches a useful coding harness in 200 lines, then exposes production prompts through intercepted requests. The harder problems are research before implementation, allocating context across agents, and removing instructions as capabilities move into model weights—without letting automation displace understanding.
  • Monaco’s differentiation rests on distribution, enriched data, and sales expertise alongside its agent harness. The Series B company targets seed-to-Series A customers before they become deeply attached to Salesforce, then aims to grow with them. Eric describes its “world database” of enriched companies, accounts, and contacts. The host illustrates buyer timing with a VP of sales changing jobs; Eric discusses finding the right buyer at the right time, but does not say Monaco relies on that specific signal.
  • Vertical agent companies can win users first and backfill model economics later. Monaco initially used frontier models to discover what was possible and is now exploring open-weight models, post-training, and RL environments for cost and speed. Its engineering rule is “you buy to learn and you build to keep,” although Devin’s organic adoption internally shows that useful vendors can retain their place.
  • Security automation leaves a gap between finding bugs and understanding what ships. The dependency debate cuts both ways: rebuilding components reduces outside exposure but concentrates undiscovered mistakes in private code. The hosts challenge the assumption that better models settle security, emphasizing sensitive data, billing, authentication, authorization, and the loss of awareness when generation outruns review.
  • Gateway consolidation makes architectural sense, but buyers can preserve point solutions. Eric accepts that MCP portals and LLM gateways address overlapping governance problems; Swyx’s counterexample to his own bundling thesis is OpenRouter’s success through narrow model-routing focus. “You buy your org chart”: enterprise ownership and purchasing boundaries can outweigh the appeal of one unified platform.
  • Software factories must prove themselves through products rather than orchestration activity. The conversation frames remaining human control around “intent” and “review,” while the hosts criticize systems whose principal output is more infrastructure for themselves. Their advice to students is to raise ambition toward projects such as a browser, or build rigorous benchmarks and evals; elaborate agent activity alone is a poor measure of progress.
Digest · the substance, structured for research

1. The junior engineer’s floor is moving toward senior judgment

  • Eric conceived Stanford’s CS 146S, The Modern Software Developer, around November 2024, then spent roughly nine to ten months developing the proposal and curriculum. Its first offering was fall 2025, before “Opus 4.5, 4.6 days.” The second version reflects how much autonomous coding agents have changed the work since that first class.

  • The original course surveyed agents across the software development lifecycle: specification, implementation, review, and SRE. This version emphasizes principles Eric thinks are becoming more durable, particularly context engineering, architecture, and working in production codebases. His position combines teaching with practice as Monaco’s head of AI; guest lecturers supply frontier experience beyond his own.

  • This is explicitly not a beginner programming course or a replacement for computer science. Algorithms and other foundational classes are prerequisites. Eric wants students to understand how systems work and what their trade-offs are, then delegate more implementation details. His employment message is optimistic but demanding: “Every company needs you, but the floor is much higher.”

  • An entry-level engineer can no longer assume that performing at Google’s L3 level is sufficient, Eric argues; students should approach something closer to senior-level capability. Swyx’s deliberately provocative formulation—most students are “cooked,” while some become exceptionally valuable—draws attention to the distributional problem. The hosts acknowledge that Stanford students’ prospects do not answer what happens across the broader workforce.

2. Open-source maintainers become the missing apprenticeship

  • Last year’s structure combined nine assignments with an open-ended capstone. Eric’s example was giving students a repository, asking them to use an AI review tool, and judging its output. Those exercises became too easy: students could potentially “one-shot” them. In his informal classroom survey, everyone was using coding agents; he said nobody really types code by hand anymore.

  • The replacement puts students inside substantial open-source projects, including coding agents and tools serving agents. Contributions become coursework rather than optional extracurricular activity. The important change is encountering an existing system with real constraints: students must understand unfamiliar code, make useful changes, and satisfy maintainers who have reasons for accepting or rejecting an implementation.

  • Eric sees mentorship as the bridge between experienced engineers who acquire extraordinary leverage from agents and juniors who lack the underlying mental models. Maintainers review contributions and explain “good software,” passing on design judgment through actual work. Successful contributors may also get hired. The educational and recruiting mechanisms are connected: students demonstrate capability to the people evaluating their changes.

  • Access to compute should not become the limiting variable. Eric has arranged additional credits with participants including Factory, Cognition, and OpenAI, beyond ordinary educational offers. He does not want a “$20 a month subscription” to prevent students from exploring what is possible. The objective is to let them “token max something” when that produces a useful experiment.

3. A 200-line harness makes the agent loop legible

  • Eric strips a coding agent down to a model interacting with tools in a loop. A useful implementation that edits files can fit in about 200 lines and requires no more than Stanford’s introductory algorithms course, CS 106B. It lacks features such as sub-agents and MCP, but removes the impression that the core mechanism depends on inaccessible “secret sauce.”

  • Students then move from the miniature implementation to inspecting a production system. Using mitmproxy, Eric intercepts requests and responses to reveal system, planning, and compaction prompts. He says he used “Opus 5.5,” described as two days old. The progression matters: students begin as tool consumers, build the baseline mechanism, and then inspect what the real harness adds.

  • The hosts admit underestimating Claude Code early on: one query cost approximately fifty cents, provoking “Who’s gonna use this shit?” Eric recalls Boris’s instruction to build for where the model would be six months later. The subsequent improvement in models helped unlock long-running autonomy and substantially changed what belonged in the curriculum.

  • Harness complexity is a moving target. The hosts discuss system prompts around 20,000 tokens, an 80% reduction, and subsequent expansion and contraction. Eric highlights instructions for Git and task handling moving from prompts into model capabilities. Weaker models may need examples; stronger ones may be overconstrained by them. His preferred early design principle was “fewer tools do more,” using powerful tools such as Bash and SQL.

4. Context engineering requires research and selective memory

  • Eric finds that surprisingly few practitioners advance beyond vanilla prompting: “Do me the thing.” That can work until the agent meets an unfamiliar, million-line codebase. His workflow introduces research, alternative proposals, a human decision, and then a plan or specification before implementation and testing. The research stage deliberately establishes what the agent needs to know before it starts changing things.

  • Swyx’s unresolved problem is distributing that context across multiple agents. Should a sub-agent inherit the entire conversation or start separately? How much should its findings be condensed before returning? A review agent may need broad visibility across everyone’s work. “I don’t like anything I’m doing,” he says of the current techniques; one large specification does not solve coordination.

  • More context also introduces new errors. The host describes repeatedly expanding and contracting what agents receive: first they know too little, then they overfit to an incidental comment from earlier in the conversation. “They took it as the law.” Eric calls this among the areas that remain more art than science, requiring deliberate consolidation, refreshing, and attention to context bloat.

  • The hosts recommend accessible research into system-prompt changes and downstream effects of context management, without pretending every paper is definitive. Eric’s practical prescription is smaller than keeping up with every announcement: experiment for ten minutes a day. Re-solving a familiar problem with a new tool can reveal enough to close much of the knowledge gap.

5. Skills should preserve understanding while automating procedures

  • Eric distinguishes a tool call from a procedure. MCP can expose an external function or service; a skill codifies a repeatable sequence of actions in something like SKILL.md. The educational objective is to have students formalize those procedures themselves, rather than merely discover another collection of prewritten prompts to consume.

  • His preferred teaching examples include Matt Peacock’s “Grill Me” and Jeffrey Blitz’s “Explain Diff.” The latter produces a visual explanation of a change, then quizzes the developer about it. That final interaction is the point: skimming an attractive summary is different from demonstrating that you understand what happened. The skill makes the user participate in learning rather than only receiving generated output.

  • The hosts reject the equation between output volume and productivity. A one-line request can now generate tens of thousands of lines, leaving its author disconnected from the system. Their recurring principle is that there is no good situation for “turning your brain off.” Useful skills often ask more questions and force thought precisely because producing the implementation has become easier.

6. Skills create a maintenance problem of their own

  • Swyx argues that writing skills matters more than collecting influencer recommendations. Naive versions are often bad, including those generated by default skill-creation tools. He maintains a “skill cutter” that removes recurring weaknesses and excess material. The complication is that useful instructions can sit inside otherwise verbose output: editing requires preserving what actually changes behavior, not simply shortening everything indiscriminately.

  • The worst failure may be an unexpected trigger on an unrelated project. A broadly written description activates a procedure that was meant for one earlier task. Eugene Yan’s practice of pruning unused skills every few weeks offers one response. Swyx suspects some people at OpenAI and Anthropic would prefer skills to disappear; he presents that as speculation, calling giant collections of Markdown files a sloppy endpoint.

  • Another host’s counterargument is usability: people can obtain substantial value without opening those files, asking instead, “Look at the last five times we did this, make me a skill.” Eric distinguishes a potentially more durable purpose at Monaco: encoding organizational knowledge for Devin review, including preferred API construction and service boundaries. That is different from saving an arbitrary recurring prompt.

7. Monaco starts before Salesforce becomes hard to displace

  • Monaco is an SF-based Series B company building an all-in-one, AI-native sales platform. Its initial customers are early-stage startups, primarily seed through Series A. Eric’s wedge is meeting teams while they still track pipeline in a Google spreadsheet, then growing with them. Pulling an established enterprise away from Salesforce is a much harder starting point.

  • The company spent more than a year in stealth building toward an integrated offering, rather than beginning with a narrow sales point solution. Eric says they have learned a lot about the all-in-one over the last year and a half and are now going more upmarket. He likens the distribution model to companies such as Brex: start with emerging businesses and expand alongside them.

  • Monaco’s proprietary harness powers sales workflows including TAM and ICP scoring, ICP definition, pipeline changes, meeting preparation, and summaries. Its enriched “world database” covers companies, accounts, and contacts. The host illustrates buyer timing with a VP of sales changing employers, potentially making that a better moment to propose a sales platform. Eric says demand combines identifying the right buyer and the right time, but does not say Monaco uses that specific job-change signal.

  • The company uses Parallel and Exa, which Eric declines to rank, while also doing its own enrichment. He locates additional differentiation in forward-deployed account executives, or FDAEs: salespeople who help customers onboard and build workflows. Monaco is codifying expertise from its own GTM team, including former Brex personnel. Without that domain knowledge, he concedes, the offering would be more commoditized.

8. Vertical agents can backfill their model economics

  • Eric’s starting point for vertical AI is frontier models as external services: first discover what is possible. Cost and speed become more important as the product matures, motivating open-weight models, post-training, and RL environments. Monaco is now in that phase. He references debate over Harvey’s +50% versus -50% margins as a prompt to examine economics, explicitly setting aside whether those numbers are right.

  • Swyx calls this the “agent lab playbook”: begin with users and a product, use whichever models are necessary, then “backfill the model side.” Model labs approach the market from the opposite direction, starting with pretraining and seeking users afterward. The relationship remains complicated because a foundation-model supplier may also compete with the vertical application it serves.

9. Monaco buys to learn and builds around its infrastructure

  • Monaco’s background-agent platform, Monocoder, is built on Coder at coder.com and adapted to its internal needs. Eric describes isolated environments, support for multiple agents, and long-horizon coding tasks. It fills a role similar to Ramp Inspect. His preference emerged from the fit with Monaco’s existing cloud infrastructure rather than a general insistence that every engineering tool must be proprietary.

  • Its Slack-based agent, Hal, spans AI SRE, customer success, and QA. It helps debug problems and open corrective PRs, while salespeople use it to understand product behavior. That breadth creates a practical design question: the same underlying system needs different “personalities” when answering a developer and when explaining something to a salesperson.

  • Eric corrects himself to state the governing rule: “You buy to learn and you build to keep.” External tools help a team discover what it needs; internal systems become attractive when available offerings are immature or poorly matched. He stresses that Monaco remains willing to revisit vendors. Code review is an important exception where it continues to find outside software useful.

  • The first review-tool evaluation examined real PRs, comment categories, verbosity, and utility; no vendor crossed the threshold. A later trial of Devin spread because engineers requested access themselves. The hosts challenge whether this was merely “vibes”; Eric distinguishes the earlier structured analysis from the later organic adoption. He did not want to force another rollout after the first unsuccessful evaluation.

10. Agent readiness begins with ordinary engineering foundations

  • Eric treats agent readiness as a prerequisite for parallel agents, background systems, and factories. Before launching tens of agents, a team needs a codebase they can understand and verify: sensible structure, boundaries, documentation, and feedback. He credits Factory for articulating the category early. The report or dashboard is useful, but the underlying engineering conditions are what make subsequent automation possible.

  • At some companies, the constraint is less glamorous: backed-up merge queues, slow CI/CD, or excessive time to stand up an application. “The answer is not even always just more agents.” Another SRE bot does not remove a systemic infrastructure problem.

  • AI SRE becomes more compelling with enormous dependency graphs, Kubernetes infrastructure, and scattered Helm files. Eric praises Resolve and Night Shift, illustrating enterprise complexity with Ford- or Walmart-scale organizations. The hosts describe tens of thousands of repositories, sometimes more repositories than developers, with departed authors’ code still “load bearing.” Smaller systems may remain manageable enough to build internal assistance.

  • Resolve previously supplied a class sandbox with multiple services and planted bugs, letting students act as on-call engineers investigating through its agent. The wider objective is closing the loop: agents should inspect build logs and, ideally, production analytics. The hosts puncture elaborate trigger terminology by noting that much of it ultimately runs through cron jobs, while preserving the importance of runtime feedback.

11. Thirty-plus daily PRs make review the throughput constraint

  • Once agents can produce “30 plus PRs a day,” traditional review makes the developer the bottleneck. Eric rejects both being the compulsory gate on every change and letting anything merge. The question is how to retain justified trust while allowing the generation gains to flow through the rest of the system. Otherwise, “what was the point of generating at scale?”

  • Thoughtful architecture and verifiability can make some classes of changes suitable for auto-approval, Eric argues, without claiming everything qualifies. For the remainder, teams must decide whether humans should examine every line, the specification, or another representation. He broadly agrees with Ankit from Aviator’s argument for eliminating conventional review, but identifies another choke point alongside it: deciding what should be built.

  • Monaco invests in detailed PR descriptions that function as condensed specifications, including file-level pointers and the boundaries being changed. A one-paragraph summary is insufficient. Eric distinguishes this implementation record from the issue’s earlier specification. The hosts also observe that review activity is not growing in line with merged PRs, reinforcing the pressure to change what review means.

12. A local PR specification cannot preserve the whole system’s intent

  • The hosts’ pushback is that an internally consistent PR can still violate accumulated requirements. A feature is the sum of many changes, not just the latest implementation description. A model may reverse something deliberately introduced five weeks earlier. Humans become the “spec gods” who remember why it existed; instructions to maintain that broader understanding have not reliably solved the problem for the host.

  • A host points to Basis, the accounting company, as an example of version-controlling different artifacts for different purposes. One records intended future behavior; another captures the “canonical state of the code,” including otherwise implicit knowledge. Eric acknowledges the example, while the host presents it as a potentially useful approach—not proof that the industry has settled how to preserve specifications and implementation together.

  • Swyx describes an exploratory “program database” that he may never ship. It would capture traces, make specifications persistent objects, manage attached skills, and support greater concurrency than Git’s locking patterns. His objection is representational: code alone is too narrow a source of truth when assumptions, constraints, intent, and many agents’ work also need coordination.

  • He thinks Linear could win because it already holds tasks and intent. Eric broadens the possibility: B2B products collecting customer feedback or analytics may become coding-agent companies by closing the loop into product changes. Linear, Amplitude, Decagon, or a CRM could potentially turn an observed problem into action: “Here’s what’s broken. Go close the loop.”

13. Dependency removal trades outside exposure for private mistakes

  • In the security discussion, a host highlights Semgrep, an established security company that has augmented its offering for agents, and stresses dependency tracking and supply-chain discipline. The practical instruction is basic but consequential: pin dependencies appropriately, and do not arbitrarily run npm install while assuming the entire chain is trustworthy. Autonomous actions and abundant generated code expand the surface requiring attention.

  • Eric poses the possibility that the most secure codebase has no dependencies. The host agrees and extrapolates that cheap code could make building everything internally seem safest. Eric then pushes back: that statement is dangerous, because a team could make its own mistakes instead of relying on others to discover them. A package with 100,000 users has many opportunities for someone to find a flaw, whereas a company’s own users may never identify or report the equivalent problem.

  • Eric’s qualification is that open source itself may change. He believes most OSS repositories, “if not all of them,” will become agent-run software factories, citing Warp’s terminal as an example of the direction. If fewer people inspect each contribution, the historical argument for broad human scrutiny becomes less straightforward. Whether the resulting codebase is more or less vulnerable remains unclear to him.

  • The hosts recount a Vercel proposal to port widely depended-on C and C++ software to Rust, probably for about $6 million, seeking faster and more secure implementations. They do not treat that as proof that models eliminate security problems. The host’s objection remains: what if generated rewrites introduce different bugs? Greater capability changes the failure modes without demonstrating that failures disappear.

14. Better bug finding does not guarantee awareness of what ships

  • Discussion of “Daybreak” and “Glasswing” turns to whether privileged defensive access and delayed releases can settle the security problem. The hosts doubt the timing logic: give defenders three months, and attackers might still spend six months probing afterward. Giving a cyber model to 100 priority organizations also creates 100 targets to compromise. These are objections to assuming the release sequence resolves the underlying asymmetry.

  • Another host emphasizes awareness over average bug rates: models may make fewer bugs on average, yet “you don’t even know what you’re putting out.” Developers increasingly neither write nor review everything they ship. Code scanning and security review remain relevant, but the loss of understanding is a separate problem from whether a model performs well on vulnerability detection.

  • Eric identifies data storage, billing, and authentication as especially sensitive areas and says, “Basically, don’t write code there.” A host affirms the concern, distinguishes authentication from authorization, and advises learning concepts such as AWS IAM before asking agents to implement them. As the host puts it, “Once you have the language, you can prompt for it.” Eric agrees that these conceptual foundations remain necessary despite implementation becoming easier.

15. Background agents turn nontechnical work into hidden codebases

  • The background-agent material extends beyond an IDE into asynchronous work, CI/CD, repository events, and operational triggers. Eric plans to use examples including Stripe Minions, Ramp Inspect, and Spotify’s internal system. He also praises Cloudflare’s adoption and write-ups as unusually strong for a public company of its size. The common question is how software teams make agents useful when nobody is actively prompting.

  • The hosts argue that coding agents already sit underneath apparently nontechnical products and workflows. Swyx runs SEO and AEO inside Codex because ongoing research requires scripts, stored data, and visualization. A marketer may never inspect the underlying code, yet a persistent, multistep workflow can still benefit from a codebase. The visible interface need not reveal the machinery doing the work.

  • Eric accepts that nontechnical use cases can be implemented through code, but separates that from the interface question. Does the developer enter through the consumer assistant, inspect its underlying computer, or move into a development environment? The hosts see convergence; Eric asks whether that necessarily makes one product the universal entry point. The conversation leaves the product boundary open.

16. Gateways bundle technically but sell through separate buyers

  • Eric remains positive on MCP as an existing standard, while the hosts distinguish that from confidence in MCP portals as a business. At scale, Eric sees a need for a central entry point covering governance, security, auditing, and cost optimization. His case is conditional on organizational complexity: the value becomes clearer when many tools and services require common controls.

  • Swyx argues that the portal folds into the LLM gateway, alongside authorization and billing. These are versions of the same architectural layer, the “God box” mediating between multiple users and multiple services. Eric accepts the consolidation argument, explaining that separating the terms in a curriculum is partly pedagogical. Combining related controls also lets them depend on each other in useful ways.

  • OpenRouter supplies Swyx’s counterexample to his own preferred architecture. It succeeds by focusing on model routing rather than also owning observability, operations, and MCP access. “I love being wrong like that,” he says: a narrowly defined product can sharpen engineering and marketing. Architectural overlap does not establish that one vendor should own every adjacent function.

  • Eric points to enterprise teams owning separate parts of the stack; consolidating endpoints requires agreement across those boundaries. Swyx translates that into “you buy your org chart” and “if it’s for everyone, it’s for no one.” Eric flags platform engineering as an increasingly important role, with responsibility spanning cloud infrastructure, developer experience, and security rather than a single application feature.

17. Factories still need intent, review, and a route into customers

  • The host frames the factory’s remaining human control as “intent” and “review,” with review likely to loosen before intent. Eric’s course framing similarly asks where humans remain, naming intent and verifiable systems. In his role analogy, a technical overseer understands enough constituent parts to set direction, metrics, and corrections. His 89% agent-written-code scenario is hypothetical; he says even frontier factory vendors claim “30 some percent, if even that” of fully agent-driven contributions. The hosts caution that dark-factory advocates have also publicly changed their minds.

  • Selling the whole factory is another matter. Eric thinks adoption probably has to be piecemeal: establish a coding agent, add background work or SRE, then connect the pieces. Simply dropping a complete factory onto Ford’s desk is implausible. The hosts suspect many companies want to build their own, making the boundary between infrastructure vendor and internal platform an unsettled commercial question.

  • They cite an OpenAI write-up describing five months building an internal tool with zero manually written lines. Its initial slowdown and subsequent offloading illustrate that harness engineering involves learning, not instant acceleration. Warp’s course partnership offers students another perspective: contribute to the factory and help keep it running, rather than only submit a conventional feature and leave.

  • Eric’s human-role analogy is a CEO who understands enough constituent parts to set direction, metrics, and corrections. The technical overseer similarly adjudicates product requirements and engineering constraints. The hosts resist declaring product and engineering roles immediately merged: Anthropic’s platform product and engineering leaders had recently answered “not yet” when asked about combining their jobs, despite increasingly overlapping work.

18. Orchestration becomes a trap when it produces only itself

  • The discussion of Steve Yegge’s Gas Town turns on the recalled criticism that “the only useful thing that was built at Gas Town was Gas Town.” The hosts treat this as a failure of accountability to real products. Elaborate orchestration, rising token expenditure, and apparent activity are not enough if the system mainly sustains its own development.

  • Swyx says he wants hires who have been through agent “psychosis” and emerged from it. He describes paying an employee’s salary and token costs while receiving little output and finding that feedback does not land. Eric wonders whether Gas Town was also ahead of its time or difficult to understand; the hosts counter that its manifesto and terminology helped spread the idea.

  • The Boris framework they examine distinguishes five team-adoption stages: gated, assisted, parallel, supervised autonomy, and AI-native. A host says the artifact describes Anthropic as being at stage three and trying to reach four; Eric jokes that this would make Boris himself stage four. That distinction matters: an individual power user’s workflow is not equivalent to an organization operating the same way.

  • “Dynamic workflows” offer one concrete advance beyond copying a parent agent into sub-agents. The system writes a task-specific harness on demand, then runs it to perform the work. Eric says Monaco has built a version for GTM use cases. The pattern is specialized executable orchestration, rather than assuming every delegated task needs another identical general-purpose agent.

19. The next cohort needs higher ambition and better measures

  • The hosts expect PR contribution or merge rate to look ridiculous as an engineering metric two years from now. But the replacement remains unresolved: revenue feels too distant, and customer satisfaction may measure product decisions more than code. Eric’s answer is to train toward holistic systems and product-engineering judgment, connecting a technical contribution to the wider thing being built.

  • For capstones, Eric uses YC’s requests for startups as prompts: “Go build a company.” Last year, a designer photographed notebook sketches and generated a working Vercel app; Eric thinks the work probably helped her get hired there. The hosts acknowledge that accomplishment while arguing that the bar has moved: turning a photo into an app is now much less ambitious than writing a browser.

  • Eric describes support for one-off experiments reaching a thousand-plus agents, making credits a way to test ambition and identify exceptional builders. Meanwhile, many companies remain at single ChatGPT calls or initial Cursor adoption. He is emphatic that this gap is larger than Silicon Valley assumes, leaving room for students who understand frontier methods to contribute meaningfully elsewhere.

  • The hosts’ proposed “week 11” is direct: “Make benchmarks. Make benchmarks and evals.” Rigorous engineering posts and useful evaluation work can open doors at frontier model and agent labs, and potentially become businesses. They cite SWE-bench as an undergraduate project from Princeton. The recommendation is to investigate one important problem deeply enough that other people can assess and use the result.

Full transcript
Speaker 1

Okay, this is a pretty special episode because we’re celebrating one million subscribers on the AI Engineer YouTube channel. Go over there and subscribe if you haven’t heard about it already. Our guest is Mihail Eric, who is teaching The Modern Software Developer at Stanford, which is going viral for covering AI-native development best practices, and is also Head of AI at Monaco. He’s not only a teacher, but a practitioner. Those of you coming to AIU Code Summit next week will be celebrating all the extreme progress in coding over the last year. I’m excited about the themes of fighting slop, system one models, and increasing your ambition for major rewrites of basically everything you’ve seen in code. There’s never been a more exciting time to be an AI engineer, and with one million AI engineers, I think it’s just getting started. So let’s get into the interview.

Okay, we’re in the studio with Mihail Eric, adjunct professor at Stanford and head of AI at Monaco. Welcome.

Mihail Eric

Thank you. I’m glad to be here.

Speaker 1

You caught our attention because of your course, which you’re refreshing for the third time—third year?

Mihail Eric

Second time.

Speaker 1

Second time?

Mihail Eric

Yep.

Speaker 1

CS 146S, The Modern Software Developer, at Stanford. This particular generation of junior developers and junior engineers—the people who are now entering the workforce—will, I think, be the first generation of that new shift. I looked at the syllabus and thought, “This is everything we cover.” Every now and then, I like to invite people like yourself, who cover a broad range of topics, so we can go topic by topic and give people the state of the art on each of these things, which I assume you’re doing with all your guests anyway.

Mihail Eric

For sure.

Speaker 1

Tell us the history of the course. What made you want to spin out CS 146S?

Mihail Eric

This is the second time it’s being taught, and I taught it for the first time last fall, in fall 2025. This was before the Opus 4.5 and 4.6 days. Even before fall 2025, the course had been about a year in the making. The genesis of the class was around November 2024, when I had this idea that we were seeing all this stuff happening with coding agents. They were an earlier version of coding agents, but already it felt like something was drastically changing in how developers were doing their work and how quickly we were getting work done.

At the time, there were scattered blog posts and things you could find online if you were trying to get into the methodology of how to do these things. I had the idea: “Hey, is there a way to structure a curriculum around this that’s a little more formal?” My alma mater is Stanford, and I’d always had this dream of going back to teach something and pay it forward. I’ve always been very deep into education.

I talked to Chip Hieatt, who also taught a class and is one of my good friends.

Speaker 1

Chip Hieatt?

Mihail Eric

Yeah, Chip Hieatt, exactly. I spoke to her. She had taught, I guess, Designing Machine Learning Systems, so I asked her what that process was like and took a lot of inspiration from how she designed the curriculum and thought about getting something through the school. I put it together and spent about 9 to 10 months on it, from fall 2024 through the middle to late part of 2025. I got it through the committee and made the case for the class to exist. I had to put together a proposal and everything.

That took me to fall 2025, when I taught it for the first time, and it was very well received. Students were really interested in it. There was even a Business Insider article about the class last year. This year, a lot of things changed. Coding agents—autonomous coding agents—have really picked up in a way we hadn’t seen before, so it felt like it was time to do this again, refresh things, and here we are.

Speaker 1

Meanwhile, you’re also head of AI at Monaco.

Mihail Eric

Yeah.

Speaker 1

Which is also a very buzzy startup.

Mihail Eric

Yes, yes. We’re in the airwaves right now and having a fun time doing it.

Speaker 1

I do like that you’re also a practitioner. You’re obviously implementing all this stuff at Monaco. Any principles or observations about what you’ve changed from last year to this year at a high level?

Mihail Eric

A ton. When I reflect on last year’s version of the class, coding agents had been around, but we were still in a phase where companies were emerging to tackle every part of the SDLC. There was a coding agent for code review, one for spec design, and one for SRE. Of course, some of those companies still exist.

Last year, it was a lot more about a broad tour of everywhere you could use coding agents. A lot of the curriculum was geared toward exposure—a kind of odyssey through everything you knew about the SDLC changing because of AI. This year, I think some principles are solidifying: careful context engineering, and really thinking through design and architecture as a way of handling complex tasks.

My class isn’t intended for the beginner programmer. It’s not a replacement for traditional computer science, like algorithms; a lot of those classes are prerequisites. But if you have a good foundation, applying it and using those principles in production environments and the real world is now something students can do. At Monaco, we deal with massive code bases. I also help a lot of companies think through their massive code base designs and things like that. So it’s more of a practitioner’s course for students who have graduated or are entering the workforce: how to be the most productive members of the workforce.

Speaker 1

You said you’re passionate about education. I feel like students in that phase of college are in a pretty interesting spot. Any advice or takes? Should people study computer science right now? Is Claude Code and Codex all we need? I want to hear your thoughts.

Mihail Eric

There’s a lot of anxiety from students, as you can probably imagine. A lot of what I’m trying to communicate this time around is that the anxiety is still here, and there was anxiety last year as well. But I want to make the case that a well-trained AI-native engineer—and I know that’s a buzzy word, but we can dive into what aspects of that are most interesting—can be one of the most useful assets at any company.

This is the first week of class, and in the first lecture I say, “Hey, I really believe every company needs you, but the floor is much higher.” You’re entering your first job out of school, but it’s not sufficient to be an L3, in Google terms. You generally have to be almost at the senior level going into this. So how do we get there more quickly? I’m trying to say that there’s still a core bit of foundational software engineering that’s important: understanding how software systems are built and the trade-offs of those systems. Beyond that, humans should still be valuable, and you can defer more of the implementation details.

Speaker 1

This is something I’ve said before, and it’s almost the kind of thing you say to be spicy, but I genuinely kind of believe it: most students are cooked, and some of them are going to be super valuable. I don’t think it’s a popular thing.

Mihail Eric

This is a delicate topic.

Speaker 1

Stanford’s kids are going to be fine, right? They’re from Stanford.

Mihail Eric

Sure.

Speaker 1

But what happens to the rest of America?

Mihail Eric

Right, right. That is an open question. It’s a weird situation.

Speaker 1

Yeah, because I feel like a lot of education is slow to move. I’m curious: how do you determine where to put the end date for the curriculum in a course on AI?

Mihail Eric

Oh my God.

Speaker 1

You’re teaching a Stanford course on modern software development. That’s changing every year.

Mihail Eric

Ah.

Speaker 1

You cheat. You just get people from the coding agent companies to come and speak, right?

Mihail Eric

Oh, yeah.

Speaker 1

Like, yeah, have them be the—

Mihail Eric

Yeah, that’s funny—

Speaker 1

The old guest lecturers.

Mihail Eric

It’s funny. Well, yeah. On the guest lecturers point, that was also the case last year. That was by design, because I came in saying, “I’m a practitioner,” as you pointed out, Swyx.

Mihail Eric

I’m a practitioner. I do this in my day-to-day; I design these systems. We build these things at Monaco, and I’ve seen that many times before. That being said, I don’t want to claim I have all the answers today. It would be hubris to make that claim.

The structure of the class was always: here’s one practitioner’s aggregated set of insights from working in the field and talking to a lot of people in the field. But there are people literally on the frontier, seeing things even beyond that. I invite them to put their proposals and their view of the world forward, and that’s why the guest lectures are there.

Speaker 1

No, I think it’s really impactful for students to see big names, people actually doing it, hear from them directly, and be able to ask them questions. Of course, they want to intern and get jobs. It’s good to have guest lecturers.

Yeah, no, I think what he’s saying is—let me rephrase it. We’ve also interviewed Ange, who teaches the Coachella course, right? That’s like, here are all the names. It’s only the names, right? Whereas he’s also producing things like Claude Code and 200 lines of code.

Mihail Eric

Yes.

Speaker 1

No one’s telling you to do that. No one’s going to come to you to do that. But when you want to teach fundamentals in a framework you can use, and a working understanding, this is the kind of thing you do.

Mihail Eric

That’s right.

Speaker 1

That’s right. Yeah, exactly.

Mihail Eric

But I do think there’s a balance for this kind of material, to your point. It’s not 100% solidified, by design. That’s a feature, and maybe a bug, of some kind. A big portion of the curriculum did change from last year, so there’s a dynamism to the whole development of the curriculum.

I’m trying to put forward that there are some things I believe are sticking around. Prompting coding agents, for example: they’ve gotten smarter, so what does that mean for the new ways of dealing with them? But Boris is also going to know something I don’t know. So let’s have him come to the table and share his learnings.

Speaker 1

One of the fun things I noticed this time around is that a good bit of the course is on open-source contributions. What’s the thinking behind that?

1. Open Source As Apprenticeship

Mihail Eric

This is one of the things I’m honestly most proud of and most excited about. We’re about to be in week two of the class. These are the very gracious open-source partners I spent the greater part of this summer talking to and getting excited about this offering.

Last year, when I taught the class, we had nine assignments and a capstone open-ended project. The nine assignments were pedagogical exercises I came up with, like, “Here’s a repo. Use an AI code review tool to post a code review, and then see why it’s good or not good.” They were relatively pedagogical and, honestly, pretty easy. I think a lot of folks could have one-shot them, and I thought, “Okay, this is not that helpful.”

At this point, agents are around. Every student I surveyed on Tuesday was using coding agents; not a single person really types by hand anymore. They’re all using coding agents. These are sophomores and juniors at Stanford.

Speaker 1

Do you survey them on market share? What tools they use?

Mihail Eric

What tools, yeah.

Speaker 1

Okay.

Mihail Eric

I haven’t yet, but that’s a good call.

Speaker 1

You should do a survey. It’s fun.

Mihail Eric

Yeah, yeah. I’ll do that.

Speaker 1

You could be spicy. Stanford prefers—

Mihail Eric

No, no. I think through the university, most of the providers have educational programs. They’re very generous with plans for students. By default, even before the class, there’s probably a Claude license, a Cursor license, and a Cognition license as well. I don’t know; you might know them better than I do.

What I was also able to accrue on top of that was additional credits for students. I didn’t want people to be bottlenecked by a $20-a-month subscription. If you want to token-max something just to see what’s possible, then you should. Factory is getting involved, the Cognition folks are getting involved, and OpenAI as well.

For the open-source partners, one of the things I’m most concerned about when it comes to educating the next generation of students is that if everyone is saying, to Swyx’s point, “They’re cooked,” what’s going to happen to them? There’s almost this sense of them being left behind. I think mentorship is a really important part of upleveling everyone and making sure there isn’t a massive gap between the senior people who were already in the industry before coding agents, know what’s up, adopt coding agents, and become amazing, and the juniors who are just asking, “Okay, now what do we do?”

One of the goals was to put students in situations where they’re dealing with complex, large codebases at scale, because coding agents are really good. We’re going to work with what I think are some of the best AI-native repos: projects that do AI-focused things, are coding agents themselves, or build tools in service of agents in the modern world. I want students to get involved with these projects and have part of their assignments be actual contributions to them.

In those projects, they’ll talk to the maintainers, who will review their code and help them build mental models of good software. That way, there’s a passing on of the craft of good software design in a way I don’t think most students get the opportunity to experience. And, of course, I’m sure some of them will get picked up as a result of doing well in those assignments.

Speaker 1

One of the interesting things, for what it’s worth—and I’m going to get a bunch of hate if I don’t say this—is I think you say “cooked” versus “cracked” for dramatic effect. It’s nice to state the extremes.

Mihail Eric

Sure, sure.

Speaker 1

Maybe we should go into the details of the curriculum, or is there anything else you want to explore at a high level?

Mihail Eric

Yeah, that’s all there.

Speaker 1

I have high-level questions about students using tools and best practices. Are you in favor of individual tutors? I’m assuming this will also be picked up by students, so before we dive straight into the course, do you have any guidance there?

You said you’re passionate about education. There’s also a stigma around how teachers and professors use these tools, and how students can use them best. It’s not just throwing your assignments in so a tool can one-shot them. How do you still learn? Do you have any guidance there?

Mihail Eric

I still have some assignments in there that aren’t intended to be the hardest things in the world, but are meant to be more conceptual. What does a skill mean? These are more about the tactics of some of these tools: skills, agents, and agent-ready codebases like AGENTS.md. There’s a certain amount of, “If you haven’t heard these terms, get up to speed on them.”

I think most of the learning from putting the tools to the test will happen both in the open-source projects and in the final project, which is very open-ended. They’ll have to explore something interesting.

Speaker 1

Cool. Maybe let’s go through the course from one to ten, breadth-first, and then we’ll dive in deeper.

Mihail Eric

Sure.

Speaker 1

We start with the internals of coding agents. You teach people how to write their own Claude clone. It’s very helpful that it’s now open source. And then?

2. Inside The Ten Week Curriculum

Mihail Eric

Week one is all about the internals of harnesses: building a coding agent from scratch. Week two is about how to think about context engineering at scale, beyond vanilla prompting. Coupled with that is thinking about tool design and MCP. Then people…

I think for a while MCP was getting a lot of hate. I’m still pro-MCP. I see it all the time, and at this point, it’s a standard that’s existed and I think will continue to exist. So, talking about that and effective tool design, that’s week two.

Week three is more about skills and what it means to extend beyond simple, single-purpose tools to procedures. That’s where we’ll talk about skills, which starts leading into the automation of agents.

Week four is about foundational ways to customize an agent: agent config files like AGENTS.md and CLAUDE.md, hooks, and subagents. There are a lot of mechanisms in modern harnesses that you can use for different purposes, so I want to expose students to those.

Agent-ready code bases are probably one of my favorite topics. This is especially relevant for folks going into industry and thinking about code at scale. What does it mean to have a repo that’s ready to be ingested and consumed effectively by an agent? We’ll think about the trade-offs of designing surface-level boundaries, as well as docs, verifiability, and things of that nature. That’s week five.

Week six is agentic code review. Extending from week five, which is about agent-ready code bases, week six is about how code review is becoming a bottleneck. I don’t know that students have seen this unless they’ve had a lot of experience; some of them are master’s students who’ve been in the industry, but a lot of them haven’t. Most people using code agents today are experiencing a new bottleneck in code review. How do we unblock that and ease the bottleneck? What mechanisms are available? We’ll look at verifiability and how we can extend it into the code review process.

Week seven is security. This was in last year’s class as well, and I wanted to reinforce it. With any autonomous system taking a lot of actions, we’re seeing a lot of things get hacked this year, especially. Some of the core pieces of software, from Vercel to LLM gateways, have been affected. I want to spend time talking about what new attack vectors exist and how we can mitigate some of them.

Week eight is background agents. I want to start touching on how you run agents in many different product and service areas, not just in what was traditionally an IDE. You can also have things based on triggers in your code base or repo, during CI/CD, or triggered by events, like SRE-style bots. How do you build a system like that? This past year, a lot of companies started releasing posts about things like Ramp Inspect. Spotify has its own version, and I think there’s Stripe Minions. These folks have been doing really cool write-ups about the internal infrastructure they’re building for this. I want to touch on that and expose people to what it is.

Week nine is more about what happens when you’re on a team in the modern world and the core pieces of team-level infrastructure. That includes everything from LLM gateways to MCP portals, which is MCP at scale, and how to think about evolving roles. The vanilla, maybe not-super-nuanced thing to say is, “Everyone’s a builder,” but I want to go deeper than that. What’s actually getting subsumed? What roles are evolving or merging that didn’t exist before? What should students think about in terms of a defensible skill set that maybe wasn’t true before? There are a lot more team-dynamic considerations.

Then it all builds up toward the factory, which I think is top of mind for a lot of folks today. Once we’re building factories, where do humans still exist? What are the new choke points that are intentionally there around human involvement? Things like intent and setting up verifiable systems. What happens in the limit when maybe 89% of code is fully driven by agents? We’ll get into all of that.

Speaker 1

Nice, solid 10 weeks. I was going to say, some of these are more established patterns than others, and I think people need to know that not all of this is settled science.

Mihail Eric

Sure. I’ll endorse all of these as things that people talk about and that you should know. We know the open problems; we may not know the answers.

Speaker 1

That’s absolutely right. I think even with the software factory, are we talking about dark factories or half-lights-on factories? I think there’s a spectrum there.

Mihail Eric

Yeah, and there are people on the very frontier, like Warp and Factory, building factories and selling them as services. Even they’ll only claim 30-some percent, if even that, of fully agentically driven contributions.

Speaker 1

Yeah. What does that mean in the limit? It’s hard to say. There are people who went very, very far toward a dark factory and then came out saying, “I was wrong, guys.” We covered that on AIE. Just be aware that they probably believe the things they’re saying at the time, but they have the right to change their minds, and they do. Even Boris recently tweeted, “Yeah, I am often wrong.” Everyone is treating his word as gospel because he created Claude Code, but sometimes he disagrees with himself.

Mihail Eric

Totally. And that’s okay. This isn’t politics. We’re still scientists and engineers. I don’t claim to be dogmatic, but I want people to experiment. I think that’s a principle as well—you asked what principles earlier. You just have to be ready to experiment.

Oftentimes, when I give lectures or talk to people about this stuff, they ask, “How do I stay on top of this? There’s so much going on.” It can be as simple as experimenting for 10 minutes a day. Spend 10 minutes doing something you didn’t do before. If there’s a new tool or framework you want to try, maybe do the exact same problem with that new thing. That alone will cover most of the gap.

Speaker 1

I’ll call out that Cognition is on the week where you’re talking about code review. I actually worked with Silas on Devin Review. It’s pretty good, I would say. It’s hard to benchmark these things. There is a review benchmark, but no one uses it. Qualitatively, Devin Review is pretty good.

Mihail Eric

Well, we use you guys.

Speaker 1

Oh, really?

Mihail Eric

Yeah. Monica does use Devin. Silas actually spoke last year, and he’s great—super smart. We use you guys, and among the providers of code review tools that we sampled, we converged on Devin for a lot of our use cases.

Speaker 1

Silas is actually more on the model side these days. Walden is the person you want.

Mihail Eric

Yeah, I was talking to Walden as well.

Speaker 1

I’ll also shout out Factory. They’ve been all about agent readiness, and they build signals and tooling for it. Do you guys use that?

Mihail Eric

Yeah, yeah.

Speaker 1

Any opinions?

Mihail Eric

We do use agent readiness of our own kind. Eno is awesome as well. I think they were super early on this, and when he codified it in his talk at AI Engineer Europe, it was a fantastic talk and ahead of its time. There was only one person I could see as the most appropriate person to talk about this. I really wanted to give him the spotlight, and they were super excited.

Oh, yeah. It was funny. I was on a call, and we were talking about this stuff. We said, “Okay, we need to form some opinions about it.” Then Factory shipped—

Speaker 2

Yeah.

Mihail Eric

—a full product on it that week. I was like, “Okay, you won.”

Speaker 2

Yeah. Like, obviously.

Mihail Eric

Yeah. I mean, everyone kind of agrees on the same things.

Speaker 2

Sure, sure, sure.

Mihail Eric

It’s just a question of whether you want to turn it into a dashboard.

Speaker 2

Yeah, yeah, yeah.

Mihail Eric

Yeah, because it’s a dashboard.

Speaker 2

Right, right, right. But there’s value in it, and I think they did well from a marketing perspective by jumping on the readiness report. I regularly pitch it to other people and say, “You should run readiness reports of your own kind.”

Mihail Eric

Yeah.

Speaker 2

Do you want to go piece by piece, or do you have anything else?

Mihail Eric

I can go piece by piece.

Speaker 2

Yeah. Okay. So the first one: you focus on Claude Code, but then you also have Pi as an open-source sponsor. What’s up?

Mihail Eric

Okay, this one is maybe a little bit more marketing-y, but one thing I do want to highlight—and I made this clear last year, too—is that I’m not endorsing any tool. I think that’s the wrong way to approach something like this. Tools come and go, and, transparently, when I taught the class last year, Claude Code wasn’t as popular or as readily accepted. Boris had only 4,000 Twitter followers, and now he has, whatever, a million. I don’t even know if he does, but he has a lot of followers.

Claude Code has really emerged on the scene.

Speaker 2

I think we also underestimated Claude Code when we had Boris and Cat on the podcast.

Mihail Eric

Right, right.

Speaker 2

It was two months old. I hadn’t really used it—I used it once. A query cost me 50 cents, and I was like, “Who’s gonna use this shit?”

Mihail Eric

Totally. It was still in beta for a while.

Speaker 2

Yeah.

Mihail Eric

But I think it’s good on them for how they built it before the model was ready.

Speaker 2

Yes.

Mihail Eric

You hear so much about building for where the model will go.

Speaker 2

Totally.

Mihail Eric

It was super ambitious. The models weren’t that good; you couldn’t feel it. And then we see where it came from. It’s funny, because when he gave his talk last year, I remember it was a very high-level set of slides. One of them was literally, “Built for where the model is going to be six months from now.” This was October of last year, and I can see why he would have made that point if he knew where Opus would be in six months. I’d argue that was one of the reasons the curriculum changed as much as it did: there was suddenly this huge arms race around true autonomy in long-running, agentic coding tasks.

So, back to your point, it’s less about endorsing Claude Code. I use Claude Code, of course, and other things as well. I’m more interested in what a modern coding harness looks like. I wrote a blog post earlier this year where I broke down some similar principles. The TL;DR is that with about an introductory-algorithms level of understanding, you can define an agentic loop that does the bulk of what you’d call a coding agent. There are bells and whistles, of course: no subagents, no MCP, none of that. But if you want to make something you can talk to that will edit your file, and do some pretty sophisticated things, it doesn’t take more than 200 lines.

3. Inside The Coding Harness

A lot of what I try to do in education is peel away the stuff that seems like the secret sauce and show that, at its core, it’s not that complicated. These are still students, and you want to teach conceptual principles. There’s nothing more to it than Stanford’s CS106B, an introduction to algorithms. You don’t need more than CS106B to build something like this. And there you go.

Speaker 2

Given that you’ve already taught this class, did you get any pushback or questions? What did the students not get, or what kinds of questions did they ask?

Mihail Eric

Not really. I haven’t received any pushback on this idea. I think most of them are consumers; most students end up using the tools. They don’t get much exposure in their coursework, if any, to peeling that onion back.

This is the week of the harness. Tuesday’s first lecture was logistics, but the first real bit of curriculum was the 200 lines of a harness. Yesterday, in the follow-up, we took Claude Code and used proxy tools to intercept the requests.

Speaker 2

mitmproxy?

Mihail Eric

Yes, exactly. We used that tool to intercept the requests and responses, look under the hood, and see what was happening. I said, “I did say you can build Claude Code in 200 lines, but here’s actual Claude Code.” I took Opus 5.5, a model that was 2 days old, intercepted those requests, and showed them exactly what was going on: the prompt, the system prompt, the plan prompt, and the compaction prompt. Then we went through all of it and showed it to the students. Now they have a state-of-the-art harness they can understand.

Speaker 2

It’s nice to get those 3 levels: students are very much consumers; here’s the baseline agent in a loop; and then the real one. I like that you have a video walkthrough of your blog post, too. Do you see students at the extremes, where some people are power users who consume a lot, and others might find this straightforward if they know how agents work? How do you balance that in the classroom?

Mihail Eric

I think it’s a bit self-selecting, because the curriculum is all here. It’s been available for a few weeks. I announced it a few weeks ago, but it had been in the works for at least 1 or 2 months, so if you looked for it, you could probably find it. The reality is, if students aren’t interested in this, they just won’t show up.

What I find interesting is that students do show up, but there are also a lot of people from across Stanford, including postdocs in other departments. There’s a lot of interdisciplinary interest from people in pathology, medicinal fields, and other engineering fields that aren’t pure computer science. A lot of GSB students and MBAs are also interested in learning how not to have a technical co-founder, or whatever their motivation is.

Speaker 2

By the way, if anyone hasn’t done the mitmproxy exercise, you should do it.

Mihail Eric

Yeah.

Speaker 2

You’ll be shocked. Claude Code has the longest system prompt of all the coding agents, something like 20,000 tokens.

Mihail Eric

Yeah, although that’s true—they’ve also reduced the size. They cut it by 80%.

Speaker 2

Yeah, I know, but then it grew again, and then they cut it and grew it again. It’s a seesaw. It’s interesting. You’ll be surprised what goes in there that you sometimes don’t use. It can also hint at the internal workings of some of the tools. Anything you found there that would be cool to talk about?

Mihail Eric

Yesterday, when I was talking about this, I mentioned one of the things I found most interesting about Claude Code back in the day, and I think part of why it really blew up in popularity: their focus on simplicity. In the early days, they were among the first to lean into the idea that fewer tools can do more. They leaned into SQL, Bash, and other powerful, Swiss Army knife tools, and made that a big driver of capability.

Mihail Eric

I think they did a fantastic job leaning into that. I think that’s why they were as popular as they were. When I presented something like the system prompt, it was like, “Well, there was a different system prompt 2 months ago.” They’ve reduced it in complexity, and things that were once part of the system prompt, like how to use Git or set up tasks, are now expected to be captured in the model weights themselves. They’ve baked in a lot of those learnings at the model level.

That raises interesting questions about the model versus the harness. How do you think through that trade-off? What do companies invest in? What should you invest in to push the frontier of your own capabilities? As a community, we still seesaw between the two a lot.

Speaker 1

I think you don’t want to fault them for growing a system prompt and then cutting it.

Mihail Eric

Yeah. At some level, that’s also how the model was. With weaker, older models, you need to give them examples of what’s good and what’s bad. Then, obviously, you cut those away at some point. That’s just part of learning what has been done.

Speaker 1

You can apply the same idea if you’re using a local coding agent with a less capable model. You might need to add those examples.

Mihail Eric

That’s right. Exactly.

Speaker 1

And then, of course, there are interesting AI learnings from that. As the models get better, when are you over-constraining them by adding those examples?

Mihail Eric

Yeah.

Speaker 1

And how do you pull away the over-constraining? That’s part of the learning as well.

Speaker 1

Okay, we’re going to talk about context engineering. RPI—I haven’t seen that, but there are many versions of spec-driven development. What would you highlight to a broader audience about what you’re putting in here that people should know?

Mihail Eric

This is the one-hour version of the course.

Speaker 1

Exactly. Please condense everything.

Mihail Eric

That’s fair.

Speaker 1

It’s also a trailer, right? If you want more, go to the source. If you’re signing up next year or next fall for courses, this is your one hour. It’s almost like the Quibi of the class: too long to be a trailer, too short to be the class.

Mihail Eric

Is this what people will double-click on?

Speaker 1

And you know the best part? If it’s a trailer, by the next time you teach it, it’ll change again.

Mihail Eric

That’s true. That’s true. A good callout on that, Andrew. The main thing I want to reinforce is that I see this even when I’m interviewing people: very few people in industry progress beyond vanilla prompting. There’s a vanilla form: “Hey, do me the thing.” As the models get better, that can carry you a certain distance. But inevitably, anyone I speak to who comes from Google or somewhere with a massive codebase has to be much more thoughtful about how they’re ingesting context.

Thinking through the principles and techniques for that is important. The RPI framework I’m describing is functionally similar to Dex’s RPI: you have a research phase, then a proposal or spec, then implementation, followed by testing and so on. In this framework, after the research, you propose a few alternative ways to solve the problem. That’s where I invite the human into the loop to decide on an approach. Then you move into the plan, which is the spec.

The TL;DR is that there’s a way to do this that scales to codebases that are millions of lines long. It’s not just a vanilla prompt. If you’re going to work in these big codebases, you should use something more involved.

Speaker 1

Anything to add on context engineering?

Speaker 1

The weirdest thing about how I think about context right now is when you split out to a sub-agent. Sub-agents can inherit your entire main-thread context, or they can start without it. There’s a lot of disagreement about that, and sometimes you want to condense what the sub-agent does and put its research back onto the main thread. It’s very hacky. I don’t like anything I’m doing, so I don’t have any answers.

All I know is that when you want to fan out work, it’s not just about having a big spec and doing RPI or whatever. It’s about fanning out the work and making sure all the agents have the right context and guidance. If you have a review agent reviewing everyone else’s work, you have to make sure it has everything.

You always go through this phase of expansion and contraction. You think, “Oh my God, these agents don’t have enough context,” so you give them more. Then they overfit to some random, one-off comment you made a while ago and make stupid mistakes because you didn’t mean it that way, but they took it as law. I don’t know what the word for it is. It’s multi-agent management for context.

Mihail Eric

Yeah, that’s absolutely right. There’s some aspect of this that’s more art than science. To your point, there’s the fan-out for exploration and subtasks, and then the consolidation or compaction, whatever you want to call it. You have to be thoughtful about refreshing the context and understanding when it’s getting too bloated.

Dex had this concept of the “dumb zone,” and we should be more thoughtful about it.

Speaker 1

He’s good at those little diagrams.

Mihail Eric

Oh, yeah. I think he’s—

Speaker 1

The little diagrams.

Mihail Eric

Yeah. He definitely informed a lot of the curriculum here.

Speaker 1

Why didn’t we have him on?

Mihail Eric

Well, he’s fantastic to think with on all this. The diagrams are huge; they’re useful.

Speaker 1

I’d add some breadcrumbs for people looking into this. It’s good to see best practices for context engineering, like whether your sub-agent inherits the main context. Claude-managed agents have a version where sessions are separate and the context isn’t inherited.

This used to be a big topic before reasoning models, when agents didn’t run as long and we had shorter context windows. Everyone was carefully managing that precious context window. LangChain used to write a lot of blogs about this and good context engineering. People should know all this.

In the student context, you can expect the average student taking your class to know these things, so you should know them even better. This is a fun space for research, too. One of the research papers presented at AI Engineer looked at the changes to the Claude Code system prompt, plotted a bunch of changes in performance, and tried to reverse-engineer it.

There was another paper that looked at context-window management and how it affects downstream performance. They weren’t the best papers, but the research is accessible and generates good blog posts. I recommend it. It’s a good area you can easily poke into.

Mihail Eric

Yeah, that’s a good shout-out.

Speaker 1

Agent skills, CLI.

Mihail Eric

Yeah, yeah. This is an extension of a single “function tool” call, like MCP, for interacting with third-party external services. It extends into what you might call procedures: how to automate a set of actions over and over, then formalize them in something like a SKILL.md file.

I want to give people exercises in codifying some of those procedures. I’ll also include a few fun skills that I think are good to use. Matt Peacock’s “Grill Me”—

Speaker 1

Yes.

It’s a good one. There are others I like from more of an educational perspective, like Jeffrey Blitz’s Explain Diff. I think it becomes especially interesting when you start talking about code review, because the whole point is to understand the code—

Speaker 1

Understand the code.

Mihail Eric

Understand the code, exactly. You’ve got to understand the code. I don’t know if you’ve used it, but effectively, you run it on the diff you’ve made, and at the end you get a visual representation of what’s happened, so you can skim it and understand what’s at the core of it. What I really liked was the quiz at the end, where you go through and reinforce whether you understand what’s happening. That level of interactive back-and-forth is quite nice.

There are a couple of blog posts in this domain that I really like to bring up. One I saw recently makes the point that there’s no situation in which turning your brain off is a good idea. I think these skills force you to turn your brain on. With coding, people want to turn their brains off and say, “Make me an app. Read my mind and guess everything correctly without me telling you about it.”

Speaker 1

That’s right.

Mihail Eric

A lot of these skills are in the genre of, “No, no, ask me more questions, please.”

Speaker 1

Yeah, 100%. I think there’s a lot more output than people are used to. I can type in a one-line prompt and get tens of thousands of lines of code and a whole structure. It’s like, “Look at my productivity, guys.”

Mihail Eric

More effectively, you should know what you’re working on. You can really be put out of touch, out of the loop. You should spend more time thinking about what you’re actually doing.

Speaker 1

Mm. Doing the work and getting the code written is much easier than it was.

Mihail Eric

Yeah, yeah. There’s no excuse for not understanding it.

Speaker 1

For sure. Sorry, what is a web skill?

Mihail Eric

Skills that interact with the web itself.

Speaker 1

With the web?

Mihail Eric

Yeah, with the web itself.

Speaker 1

Computer use?

Mihail Eric

Computer use, yeah, things like that.

Speaker 1

Anything else?

Mihail Eric

Mainly things that involve pulling information from the web. Are there best practices for skills that interact with the web?

Speaker 1

I think this is more of a survey or demo of things I found interesting. I’m trying to remember where it was. Was it Vercel? I’d have to look back at the reading materials. There was one site I found—maybe it was browser-based. They might have—

Mihail Eric

Ah. Do you remember this?

Speaker 1

Well, it would have been them that coined this thing.

Mihail Eric

Could be. I was just like, “Did I miss something? This is a big part of the skills community.”

Speaker 1

No. Yeah, this is good. He’s being educated.

Mihail Eric

I think they made some kind of website showing some of these interesting capabilities. They like browsers.

Speaker 1

Seems legit. CLI is a fun one. Do you have a lot of students for whom this is their first time in a terminal or a CLI, or is it the norm now?

Mihail Eric

No, I don’t think so. Most of them have interacted with Claude Code at this point.

Speaker 1

I’m curious about the split of people who are now just using the apps, right? Claude Code also has a desktop app. So does Codex. Codex is primarily an app now. You’re not in the CLI, not in the terminal. It puts a lot of average people off, but just a random note.

Mihail Eric

I’d say there’s another thing about skills. There are skills that are bundles of skills, where they cross-reference each other.

Speaker 1

Yeah, like plugins, right?

Mihail Eric

Like plugins. And then there’s also the art of writing a skill, which is something I’ve focused on for AIE.

Speaker 1

I think a lot of these discussions focus on skill consumption. You have all these influencers like Matt Pocock saying, “This skill will solve all your problems.” But no, you should write your skills yourself.

Right.

Speaker 1

And actually, when you try to write them naively, they’re usually bad. It takes a little bit of skill to write a skill.

Mihail Eric

That’s absolutely right.

Speaker 1

What are Swyx’s best tips on writing skills?

Mihail Eric

I have a skill— I have a problem with skills, people. I went through a phase of psychosis where I wanted a skill for everything.

Speaker 1

No, no, no. How do you write good skills?

Mihail Eric

Oh. I have a skill for cutting skills.

Speaker 1

Okay, okay.

Mihail Eric

Claude Code and Codex both have skill-creation skills, right? I have a skill-cutter skill that cuts them down because the default skills they create document every way those skills are bad.

Speaker 1

So you always run this after a skill is created to cut it down?

Mihail Eric

Right. It’s really annoying because sometimes there’s slop, but there’s good in the slop. You have to whittle down what’s there.

Speaker 1

Oh, interesting. So even the slop is still in the language of the agent, and it means something?

Mihail Eric

Yeah, exactly. It’s fun. Everyone has their own version. I use a simple vibe check: you call a skill, run it a few times, and distill your preferences into a skill that says, “This is what I like, this is what I don’t like.” It adjusts over time and can automate a lot.

Speaker 1

Yeah, well, the problem is that the worst part is when a skill triggers when you’re not expecting it to. You wrote it to fix one thing, and now you’re working on a completely different thing and it triggers. Why? Because it’s badly written. The trigger says, “Call me whenever blah, blah, blah happens.” It looks good, and then you start a different project and it triggers. You’re like, “What the hell is happening?”

Mihail Eric

Some people keep adding skills whenever they mess up. I know people like Eugene Yan are pretty particular about how many skills they have. They prune them and clean them out every few weeks.

Speaker 1

Yeah. This is a huge problem if you’re in the skills world. I suspect there are very smart people at OpenAI and Anthropic who want skills to die.

Mihail Eric

Hmm.

Speaker 1

They won’t say this publicly, but it just feels sloppy. I can see people in my own skills community being sloppy. It doesn’t feel AI-native. It’s not what AGI was supposed to look like. It’s just everyone having a giant bunch of Markdown files.

Mihail Eric

I mean, I think part of it is—

Speaker 1

Yeah, there’s a gardening problem. A lot of people don’t look at their Markdown files. The UX of just chatting about what you want, creating a skill in chat, using it, and seeing the output has a lot of value. Don’t overcomplicate it for yourself. You can get a lot of value by saying, “Look at the last five times we did this and make me a skill,” then iterating on it in chat.

Mihail Eric

There’s another way to look at it. A skill can just be a procedure the agent runs whenever it needs to do something like a code review. But in the case of something like Devin, we’ve codified skills as a way of representing our best practices and exposing them to the agent or the code review system. That lets us implement APIs the way we like to define them and enforce service-level boundaries the way we define them.

That’s an alternative way of representing organizational knowledge, rather than just an arbitrary thing you want the agent to do so you don’t have to write the prompt each time. I like that positioning. That purpose might exist longer than these random procedures.

Speaker 1

I just Googled “Monaco AI skills,” and it’s giving me the country.

Mihail Eric

Hey, maybe the country is— One day, one day, we’re gonna become—

Speaker 1

Well, we’re gonna beat them on AEO, that’s for sure. Can you just buy the country somehow?

Mihail Eric

Here’s the thing: we were about to buy— No, we’re not about to buy the country. But we did get a little bit of flak when we bought the monaco.com domain. I will say that. We did get sued by them.

Speaker 1

Ew. What?

Mihail Eric

Yeah, there’s a whole public thing about this. They didn’t like that, but here we are. And we won. We won.

Speaker 1

Okay, then we’re keeping it in. I was about to say this is spicy, but you won. Is that why we see the flags flying around in here? Because it’s like a victory, because you won?

Mihail Eric

It’s our victory.

Speaker 1

That’s funny. For those in SF, just look up.

Mihail Eric

Yeah, yeah—you’ll see.

Speaker 1

For those listening, we’re showing this on YouTube. You fly a plane around. I think you’re still one of the few that actually do this on a regular basis. Sam Blan came up with it: $6,000 a day. I don’t necessarily think it directly causes me to Google what Monaco is, but all of Monaco’s ads are just Monaco and a dollar sign.

Mihail Eric

Right, right. Oh, yeah, we stand for money. It’s also a dollar sign and the next number or whatever it says.

Speaker 1

Right. The whole point of the ad is, “What the hell is Monaco?”

Mihail Eric

Yeah. And you could argue that’s a debate to be had, but in some sense, what is the purpose of marketing if not to get attention? Maybe there’s one level beyond this. I’ll go to events, and people are like, “Oh, you guys are the ones with the planes. You guys are the ones with the big number on the billboard—193 million, whatever it is.” But then they’re like, “What do you do?” And maybe that’s the whole point: I just needed someone to come and use it as an icebreaker.

Speaker 1

Oh, God. I will say it’s what I said as you walked in, too. I was like, “Ah, the planes.”

Mihail Eric

Yeah. And now it’s, “What do you guys do?” “Well, let me tell you about what we do.”

Speaker 1

As someone who’s been in those meetings deciding what we do on the Devin billboards, we spend so much time on the words. Maybe we should just be like, “Devin.”

Mihail Eric

I think we do it.

Speaker 1

Which is so dumb.

Mihail Eric

Look, I think Sam had another post this past week where he was talking about it. Sam is effectively our marketing mastermind here. There’s a little bit of history behind the planes, too, but the TL;DR is his whole shtick is, “I don’t have all the answers. You just gotta try things.”

Speaker 1

It’s good, yeah.

Mihail Eric

A lot of this is just trying things, and if it sticks, great. Did we know this was going to blow up as much as it did? Not really. But now that it did, it’s—

Speaker 1

You had an idea.

Mihail Eric

Yeah, exactly.

Speaker 1

And the mental model is: start small, go big instead of start big, go small.

Mihail Eric

Right. It’s way easier to start small and go big.

Speaker 1

Exactly. Do this post.

Mihail Eric

That’s right.

Speaker 1

Also, it’s $6,000 a day. It’s very much just trying things.

Mihail Eric

Yeah. I think people—

Speaker 1

Cheaper than a billboard, for sure.

Mihail Eric

Yeah. And it’s fun. Not much harm done. But the simplicity here that’s worth mentioning is, when we’re trying to one-up the plane, we’re like, “You know what’s better than one plane? Two planes.”

Speaker 1

Ooh. Two planes. What? My mind went to a rocket.

Mihail Eric

I also had that idea.

Speaker 1

I was gonna say, I’ve seen the memes.

Mihail Eric

I’d like for that to happen at some point, too, but we’ll see.

Speaker 1

This is the next SpaceX launch. We just see Monaco—

Mihail Eric

Yeah.

Speaker 1

And they—

Mihail Eric

There you go.

Speaker 1

There you go. Gotta warm up to our SpaceX friends.

Mihail Eric

Well, I mean, we’re on Monaco. Is it worth talking a little bit about Monaco? Do we leave it there?

Speaker 1

Yeah, yeah. We mentioned Monaco, right? Let’s give a little more. I think it’s a little annoying to mention “What the hell is Monaco?” and then not answer it. Can you answer it?

Mihail Eric

Sure. Well, I hope so.

Speaker 1

Whoa. This is the country. It’s a small little, I guess—

Mihail Eric

Oh, no, dog poop.

Speaker 1

Oh, no. There you go.

Mihail Eric

There we go. There you go.

Speaker 1

Okay, this is Monaco. This is the Monaco that I know and love.

4. Monaco Builds An AI Sales Platform

Mihail Eric

Yeah, we’re a Series B company based here in SF. We’re building an all-in-one sales platform. It’s fully AI-native, with agents doing all the fun stuff—or at least the stuff humans don’t like to do. Agents do like to do it because they don’t really have opinions, and they do it in a less error-prone fashion, so salespeople and sales teams can focus on what humans are good at, which is closing deals.

Speaker 1

So presumably Marc Benioff doesn’t like you guys.

Mihail Eric

Probably not. But he’s also honestly a very AI-native person.

Speaker 1

Mm-hmm. Very aggressive with Agentforce and everything. What are you doing that he doesn’t do or can’t do?

Mihail Eric

We didn’t go after enterprise from the get-go. We said that if our goal is to build the all-in-one, we need to build it iteratively with a group of customers and grow with them. It’s much harder to pull someone away from Salesforce than to meet them when they’re still tracking their pipeline in a Google spreadsheet, show them a more AI-pilled answer to the problem, and grow with them. In a way, it follows the same model as Brex or other YC companies: you sell to your batch mates, grow with them, and then go upmarket. I’d argue we’re among the best, if not the best, startup-focused sales platforms. All our customers are early-stage startups. We’ve primarily focused on seed through Series A, and now we’re going more upmarket. We’ve learned a lot about the all-in-one over the last year and a half.

Speaker 1

And you are the head of AI?

Mihail Eric

Yeah, that’s right.

Speaker 1

What are some cool AI things you guys have built?

Mihail Eric

We’ve built our own agentic harness for sales use cases. To call back to coding harnesses, we’ve taken a lot of inspiration from systems like Codex and Claude and built bespoke tools and workflows focused on sales and geared toward GTM use cases. Our version powers pretty much everything in the Monaco platform, from scoring TAMs and ICPs, and defining ICPs, through to pipeline evolution and all the mundane things a salesperson may not want to do, like updating pre-meeting notes and summaries. We have agents powering all those workflows, and it comes out of the box. We have long-horizon agents and, in some cases, short-horizon agents for GTM use cases. It’s pretty domain-specific, built on foundation models.

Speaker 1

Would you do any of your own models for this?

Mihail Eric

That’s a good callout. So far, I think this is generally true of a lot of vertical AI companies: you start by using frontier models as third-party services to see what’s possible, because they’re at the frontier. We’ve seen this in the past week with the whole Harvey discussion about 50%-plus and 50%-minus margins. Irrespective of the numbers, what’s interesting is that at a certain point, things become interesting from a cost perspective. You start to wonder whether you can do it faster on open-weight models. A lot of the purpose of the AI pieces is then to post-train those models and build environments to post-train using RL and things of that nature. We’re kind of in that phase right now.

Speaker 1

Ooh, fun. I’ve been calling this the agent lab playbook, but that’s my coinage: agents start with the user, then use whatever models they need to build products, and then backfill the model side once they have the user base. Model labs go the other way. They start—

Mihail Eric

Yeah.

Speaker 1

—with the big pre-trains, and then they try to find users. Hopefully, it works as a split. Monaco would be on the revenue GTM side.

Mihail Eric

That’s right, yeah.

Speaker 1

I think it’s interesting that they’d want to serve you while also trying to compete with you—

Mihail Eric

Mm-hmm.

Speaker 1

—as model labs.

Mihail Eric

That is correct.

Speaker 1

One of the hard problems I’ve run into—I’ve dabbled in some of this—is enrichment. I spend a lot of my time doing that for SEO, research, and what have you, and I imagine for you it’s a core part of the business. Do you find that models are good at it? Are there vendors you like working with and want to give a shout-out to?

Mihail Eric

Yeah, absolutely. We were early adopters of Parallel, and I think they’re very strong.

Speaker 1

Okay, so just general search?

Mihail Eric

Yeah, I mean, they’re—

Speaker 1

Not like ZoomInfo or one of those guys?

Mihail Eric

No, no, no.

Speaker 1

Yeah.

Mihail Eric

We use Parallel, and we also use Exa, so we’ll give a shout-out to both of those folks.

Speaker 1

Can you compare them?

Mihail Eric

No chance, brother. They’re both friends.

Speaker 1

Okay.

Mihail Eric

They’re both friends. They’re both folks we like to work with.

Speaker 1

They’d be very interested if you liked one over the other.

Mihail Eric

Yeah. Look, I like them. I’ll keep it neutral. I’ll stay Switzerland on this today. But, of course, we do our own version of enrichment as well. One of the most valuable pieces of software we’ve built, or one of our most valuable systems, is our world database: the database of companies, enriched companies, enriched accounts, and enriched contacts. When we’re constructing ICPs for people, we have the most up-to-date information about who they should reach out to at which company, at that point in time.

I’d say where we’re world-class, hands down, is demand. We’re a very good demand company. We help companies increase the number of meetings they book.

Speaker 1

I see. So, find the buyer at the right time, right?

Mihail Eric

It’s a combination of those two pieces.

Speaker 1

What do you mean by “at the right time”? Say you’re targeting VPs of sales. The right time to reach out isn’t when they’ve been at the same place for 5 years. A good signal is when they’ve gone from one job to a new job. They’re entering a new place where they’re more likely to purchase, or be open to purchasing, a new sales platform because they’re in the middle of a job transition. So, for a VP of sales moving from Company A to Company B, a good signal that you should reach out is detecting that their LinkedIn has changed or something like that.

How much of this is your expertise put into a model and harnessed, versus models being good at generic best practices? Is it the glue, making it reliable and doing it all in one platform? It seems like there’s also domain expertise baked in that we wouldn’t know about.

Mihail Eric

Without the former, it would be fully commoditized.

Speaker 1

Yeah.

Mihail Eric

I mean, not fully commoditized, because there’s still a lot of value in the all-in-one. I think that’s something we definitely pioneered. Most companies in the sales space started as point solutions. They tried more of the YC approach: land and expand. You pick a niche and then expand from there. From the get-go, we knew we wanted to be all-in-one. That’s why we stayed in stealth for the greater part of a year and some months: we were building.

That alone is powerful and meaningful. But I think the real power is that Monaco has a world-class go-to-market team selling Monaco. The same go-to-market people who sell Monaco are also what we call forward-deployed account executives, or FDAEs. They’re effectively human salespeople, consultants of sorts, who help onboard our customers, get them building agentic workflows, and guide them through every part of their sales go-to-market lifecycle.

So, yes, it’s the all-in-one, but it’s also the fact that we have these people. We’ve brought on a lot of former Sam’s team members who were some of the best people in GTM at Brex and other places. We’re codifying all of that in the platform.

Speaker 1

Yeah, interesting. You talk very differently from a normal head of AI I might talk to, because you’re also a former founder, right?

Mihail Eric

Mm-hmm.

Speaker 1

You’re also used to thinking about these things in this way, and it’s interesting to—

Mihail Eric

Right.

Speaker 1

—it’s important to have that—

Mihail Eric

Right.

Speaker 1

—in your role.

Mihail Eric

Yeah, cool.

Speaker 1

I think you’ve introduced Monaco enough there, and as the Monaco story develops, we can revisit.

Mihail Eric

Sure, sure, sure.

Speaker 1

Maybe the bridge back is: what is coding, or agent-native coding, or modern software development like inside Monaco? When you’re teaching, you might be trying to give people the lay of the land, but maybe inside Monaco you’re more opinionated about one thing versus another.

Mihail Eric

Yeah. We made investments in coding agents basically from day one, and we’ve learned a lot. We have our own scars and learnings, and I think we’ve been getting much deeper into sharing some of them. On our blog, we’ve increasingly been posting some of our learnings. Maybe not the one at the very top—that one is more of a product—but I’d point to “When AI Gets Desperate,” “Before Scaling Contributions,” “Build or Buy,” and “SRE.”

We spent a lot of time looking at different providers and vendors, and we ended up building a lot of things in-house. For us, something like a background agent platform: we have our own version of Ramp Inspect that we call Monocoder. It’s built on the Coder platform, and it’s what we use to scale up our engineering productivity.

Speaker 1

Coder, like Coder.com?

Mihail Eric

Coder.com, yeah. We’ve adapted it to some of our use cases. It’s a really well-designed piece of software for isolating development environments, scaling out to multiple agents, and running long-horizon agentic coding tasks. So, Monocoder—shout-out to John Tracy and co. for building that out.

We’ve also built our own version of a Slack bot that we call Hal. Hal is part AI SRE, part customer success, and part QA engineer. It lives in our Slack and helps us debug things and put up PRs when things are broken. Our salespeople also use it to find out how the product works and get to the bottom of things. We’ve had interesting questions to answer around the different personalities Hal should have, depending on whether it’s talking to a salesperson or a developer. We’ve built out a lot of systems like this over the last year or so. Shout-out to Yunja for Hal, and to John Tracy as well.

Speaker 1

So, you’re building a lot in-house. You also use other tools. What’s the line? “Build or buy”—was that the idea?

Mihail Eric

Yeah, “build or buy.” It’s a good question. There’s a general principle I usually go by: you build to learn and you buy to keep. Or, sorry, you buy to learn and you build to keep.

Speaker 1

Yeah, the other way around.

Mihail Eric

So, you buy to learn and you build to keep. I think that’s generally true here. In the early days, when we were still trying to figure things out and get our feet under us a little bit, we…

Code review is one example. We spent a lot of time looking at different code review platforms. Today, we still use Devin because we’ve found a lot of value in how the system has been built, so shout-out to the Cognition team. Along the way, we tried some SRE tools and some off-the-shelf, more background-agent platforms. Given our infrastructure and how we’d set up our own cloud platform, it made more sense to build this out internally. Of course, we’re not dogmatic; we’re open to revisiting them if something comes out. I think we usually ended up building in-house when we felt there wasn’t a mature offering out there.

Code review is an interesting example because we did a whole survey of code review tools, maybe over a year ago. At the time—

Speaker 1

I’ll shout out CodeRabbit, right?

Mihail Eric

Yeah, CodeRabbit and others. There was also Graphite.

Speaker 1

Yeah.

Mihail Eric

Graphite, yeah. We used Graphite for a while before they got acquired. There was Ellipsis, which was a YC company at the time. There were a bunch of folks we tried. They were okay, but I don’t think they crossed the threshold of being immediately useful yet, so we put it aside. This was well over a year ago. Then we went back to it organically. Shout-out again to the Devin team and the Cognition folks: we tried it again. Somebody had told me, “Hey, you should try this. It seems like it’s gotten a lot better.” I’d actually been at Silas’s talk during the class last year, and it was good. It was picked organically.

Speaker 1

So purely vibes, right? There’s no way you know this.

Mihail Eric

Okay, here’s what I’ll say. When we did that first evaluation and didn’t pick a vendor because there wasn’t one we were really happy with, I analyzed our PRs. I made pie charts showing what the tools were solving and whether we were happy with the trade-off between verbosity, how often they posted, and their utility. So it was more statistically driven than just vibes. The first time around, I was pushing for us to adopt an AI code tool. The second time, I wasn’t going to push hard on it again. But then people organically started saying, “Hey, Devin is good. Can you add me to this?” Today, people are getting added organically. It’s one case where we’re still leaning toward the vendor. We’ll see what that looks like in the long term, but outside of that, we didn’t feel the maturity was there.

Speaker 1

And it’s not painful enough that you would—

Mihail Eric

Yeah, we’ve involved Devin right here.

Speaker 1

That’s right. That’s interesting. Good talk for Devin, who doesn’t focus on code review.

Mihail Eric

Yeah, I didn’t even know. For the record, I’m only an advisor. I’m not that involved anymore, but it’s interesting to hear about it externally.

Speaker 1

Mm-hmm.

Mihail Eric

Internally, there’s a certain bubble. On Slack, you only hear positive things because people only want to circulate positive things.

Speaker 1

Right. But you have no dog in the fight; you just choose the best tool. If you want to get us free credit, that’d be cool too. I don’t think anyone would say no to that.

Mihail Eric

I think Silas will hook you up.

Speaker 1

Okay, so that informs all your teaching work.

Mihail Eric

Yes, that’s right.

Speaker 1

I think that’s really good. We can bring in more of this as you like, because this lived experience is very valuable.

Mihail Eric

Sure. I’m happy to go back to agent-ready codebases or more.

Speaker 1

For sure. Weeks four and five kind of blend together. Whatever you want—whatever strong beliefs you’ve introduced in Monaco and recommend people start with.

Mihail Eric

For sure. Weeks four and five blend a little bit. Week four is more about conceptual foundations and ways to do this, because there are a lot of ways to customize agents. Week five is where I really lean into applying this in practice on a real codebase. I’ll give a shout-out to the Factory team and highlight the value of something like a readiness report. The main point I want to reinforce is that agent-ready codebases are a precursor and a building block for everything that comes afterward. People have been—and I think still are—really hyped about multi-agent systems, with agents working in parallel and tens of agents doing all kinds of things. You need to lay a foundation to do that well: how verifiable your codebase is, how well-structured it is, and how understandable it is to an agent. Agent readiness is almost the first piece of the next four or five weeks, up to around week eight or nine, when I spend more time talking broadly about AI software teams and organizational structure.

Speaker 1

That’s definitely building up the building blocks for background agents and the software factory. I think this is where it all gets interesting. One thing I noticed you talking about for Monaco, but that isn’t here, is the SRE side. When you close the loop, you want the agent to read your build logs and, ideally, your analytics from live production.

Mihail Eric

That’s right. It’s not explicit, but that’s a really good call-out. I’m going to weave it in in two places: the background agents week and the software factory week. This idea of closing the loop based on triggers or events is now one of the two or three entry points people talk about when they discuss agents kicking off tasks in the software factory. That’s absolutely right; it’ll come in there.

Speaker 1

It’s interesting that people call it trigger-based or event-based. I guess that’s the actual technical implementation, but it seems almost too low-level. The trigger is usually a cron job, right? Let’s be real. Event-based is usually a cron job that surfaces an event, and then it’s event-based. It’s all crons.

Mihail Eric

Sure, sure.

Speaker 1

Or a webhook or something like that.

Mihail Eric

That’s absolutely right. At the end of the day, being a CS person by trade, I fall back on that terminology.

Speaker 1

Have any AI SRE companies made an impression on you?

Mihail Eric

I think the Resolve AI folks are doing well there. Night Shift is also doing quite a good job with background agents. I’m a big fan of what they’re doing. They’re Greylock-backed, and I think they’re doing really well in this area. SRE is an interesting use case, and this goes back to the build-versus-buy question. Where does an AI SRE see the most value? Typically, it’s when you have a tremendous number of service dependencies, massive systems, and tons of Kubernetes and Helm files strewn everywhere. I think that’s why Resolve AI has done really well in the enterprise. You go and sell to Ford or Walmart and say, “Your service dependencies are crazy.” You see these diagrams of how everything is set up, and then they go in and say, “We’ll try to make sense of all this.” At smaller scales, it’s usually still manageable to roll your own. But for the enterprise, I think they’ve done a good job.

Speaker 1

Yeah. For people who have never worked at this scale, because it took me a while to see this, it’s on the order of—let’s call it tens of thousands of repos. More repos than developers. These are situations where you don’t know what code is running—

Mihail Eric

That’s exactly right.

Speaker 1

—anywhere in the company—

Mihail Eric

That’s right.

Speaker 1

—because probably the person who wrote it is no longer here, right?

Mihail Eric

That’s absolutely right.

Speaker 1

And, by the way, it’s still load-bearing.

Mihail Eric

That’s absolutely—

Speaker 1

That’s the scariest thing.

Mihail Eric

That’s absolutely right. It’s like—

Speaker 1

And then you’re the person on call, interacting with a system that goes down, and the person who knows it still isn’t there. There are old, out-of-date runbooks, and you’re like, “What the hell is going on here?”

Mihail Eric

It’s a scary, scary thing. And yes, at the enterprise level, it’s a whole different beast.

Speaker 1

Yeah, so the college hacker, with one person per repo, has no appreciation for this. But I think if you drop them into a simulation and say, “Okay, day one on your job, this happens. What now?”

Mihail Eric

Yeah.

Speaker 1

Yeah.

Mihail Eric

So another shout-out I’ll give here: the Resolve folks made an assignment for the class last year. They created a sandbox with many services doing various things, and there were even bugs in those services. As part of the assignment, students had to use the Resolve agent to probe and play the human on-call person, probing the system and discovering what was happening.

Speaker 1

Wow, that’s—

Mihail Eric

They’ve got an interesting—

Speaker 1

Yeah, no, it’s an interesting website. It’s been a while since I’ve seen this. It’s very interactive. Okay, they’re being a PagerDuty now.

Mihail Eric

Yeah.

Speaker 1

Yeah, that checks out. For sure. But also, this video—I was—

Mihail Eric

Well, you know, you were shouting “Adam.”

Speaker 1

Interesting. Interesting. Sure. But okay, we never get a chance to talk about this side of the world except for conversations like this. Any other segments? I think AI SRE might be too broad or even too narrow, depending on how you look at it. Too broad meaning there are several interesting subdivisions of AI SRE, or too narrow meaning that’s not even the biggest problem we have.

Do you mean specifically for Monaco or broadly, for organizations thinking about this?

Speaker 1

Broadly, because that’s our audience. Take it however you like.

Mihail Eric

I think it’s a function of how we scale. For the enterprise, there are a number of issues that aren’t always solved by more agents. Since we’re still on week five, “Agent-Ready Codebases,” when I talk to companies at a certain scale, the things holding them back are often lower-level DevOps and CI/CD issues. Their merge queues are getting backed up. It takes too long to—

Speaker 1

Okay.

Mihail Eric

—stand up an app. There are foundational things that have almost nothing to do with adding one more SRE bot. A bot wouldn’t be the answer, because there are more systemic things they need to figure out.

Speaker 1

So that brings us to week six.

Mihail Eric

Yeah, exactly right. That’s where we get into—

Speaker 1

To the merge queue and beyond.

5. Code Review Becomes The Bottleneck

Mihail Eric

Yeah, this is a love letter of sorts to modern agentic code review. I think the biggest thing that happened between last year and this year was that it suddenly became very clear that code review is a bottleneck. Once people got really AI-pilled, the velocity of generation became so high that you could, in theory, put up 30-plus PRs a day. Then what? At what point do you decide to put yourself in the middle to gate and review things the traditional way? Then you’re literally the constraint, the choke point for velocity, and that’s not good. What was the point of generating at scale?

Or you ask, “How do we get the best of both worlds?” We want enough oversight to trust what’s getting merged, but we also want to merge at scale without freewheeling and letting anything go. A lot of what I want to touch on is that middle ground. If you’ve thoughtfully designed your agent readiness and verifiability, there’s a class of problems you can feel comfortable auto-approving some percentage of the time. What needs to be true from a design or architecture perspective for you to auto-approve? When do you have to insert yourself and say, “I still want to review this carefully”? And even then, are you doing line-by-line review or something else?

Speaker 1

One of our hottest topics this year: we have a guest writer program for Latent Space, and this is from Ankit at Aviator.

Mihail Eric

Nice, nice.

Speaker 1

He has a whole thing on killing the code review. That may be the final boss, because it’s a bottleneck. He has a few gates by which he does this. I don’t know if you’ve talked to Ankit or—

Mihail Eric

Yeah, I’ve read this post.

Speaker 1

Okay. I haven’t spoken to him personally, but I’ve definitely read it. Was he right? Any criticisms?

Mihail Eric

No, I think that was broadly right. I guess you mentioned the last boss. Maybe it’s not the final boss. I think there are 2 choke points: the review and probably the intent, or the thing at the top. Like, this is the—

Speaker 1

Yeah, and there are some stats I like to pull up. Everyone’s stats are different, for sure, but clearly more people are merging. The number of reviews isn’t increasing in line with merged PRs.

Mihail Eric

Yeah, that’s right. If you look at any modern company that’s really invested in this and thought about it carefully, some people have already written posts about it, and I’m sure you’ll write about it too. Amplitude, Cloudflare, and Augment have all written good posts about the design of their AI code review systems.

Speaker 1

Mm.

Mihail Eric

These are a bunch of really good posts. If you want to share some of those links, they’re fantastically written.

Mihail Eric

Shout-out to Cloudflare, because I think that, in a sense, they’re a legacy or bigger company, but they’ve done a fantastic job with their content around the AI SDLC. The short version is, yes, this is true. We’re still finding the sweet spot: what’s auto-approvable and what isn’t? And if something isn’t auto-approvable, what do we review? Are we still doing line-by-line review, or some combination of the spec and something else?

Speaker 1

Are people keeping the spec up to date at all times? The spec is in the PR, right?

Mihail Eric

Yeah, no. There are a few different places. There’s a spec in a Linear issue, and there’s a spec in the context of a PR description. You can have the PR description explain more or less the entirety of the implementation in a condensed format. As long as it’s the most up-to-date representation of the codebase at that point in time, I think that’s sufficient. We’re thoughtful about our PR descriptions. They’re not just one paragraph; they’re effectively a spec of what happened in the PR, with file-level pointers to what changed. That gives you both the high-level view and a sense of the file structure and surface-level boundaries you’re touching. That’s where we’ve invested quite a bit.

Speaker 1

So I push back on that because, to me, of course, at bare minimum it should be internally consistent with the PR spec.

Mihail Eric

Sure, yeah. No, what I meant is the spec for the feature, the codebase, because that is the summation of all PRs. If your PR spec got it wrong or missed something, it’s going to be the delta between what the whole codebase, or the whole history of the codebase, is supposed to do and what your PR is trying to do.

Speaker 2

Mm-hmm. Yeah. I don’t expect you to have an answer. I really stress about this myself. I try to instruct models to do it, but they’re bad at it. That’s it. [laughs] The humans are the spec gods: “Five weeks ago, I did this other thing, and now you’re reversing it.”

Have you seen the post from Basis? They had a tweet article, and I thought they did a great job talking about the different file artifacts they track via Git. Some are the specs—the intent of what is to be—and others are what they call the canonical state of the code, which is a little more like what you’re describing: implicit knowledge, implicit intent that also has to be codified somewhere. That part is all version-controlled, and I think they found that works for them, so that might be another thing to share as well.

Mihail Eric

Yeah. Vibhu was trying to pull that up.

Speaker 2

This is Basis—the accounting company.

Mihail Eric

Exactly right.

Speaker 2

We’re having Mitchell at my New York conference soon.

Mihail Eric

Amazing.

Speaker 2

Talk to him about that. I like that they’re coming up with this. My solution for this is something that replaces Git in order to capture all of this. I don’t know how ready I am to talk about it, because I may never ship it, but I call it a program database. It captures all traces and reifies specs into something we can keep and refer to. It also controls the skills attached to it.

Mihail Eric

That’s cool.

Speaker 2

I think Git is only focused on code, and you need a source of truth for everything else outside the code, including the implicit assumptions, the intent, and the other stuff. It also needs to be highly concurrent, because Git, by design, introduces mutexes everywhere.

Mihail Eric

Mm. Yeah.

Speaker 2

So you need super concurrency. Have you seen GitButler?

Mihail Eric

Yes, yes.

Speaker 2

The guy who used to be CTO at GitHub—I think his name is Ian. I forget his name, but, yeah, Scott. Scott. There are a lot of people going after this, and the question is what the ideal representation of tasks is. I think the winner probably isn’t me; it’s Linear.

Mihail Eric

Really?

Speaker 2

[laughs] That’s where the tasks are.

Mihail Eric

Yeah, where we’re going.

Speaker 2

And they’ve clearly decided to become a coding agent company. They have all the source of truth there, as long as you buy into Linear.

Mihail Eric

Yeah. That is true. I have this somewhat-held belief that a lot of companies are going to evolve into some version of a coding agent company. Any company that interacts with B2B software, where you’re collecting something about the user or some product insight, can close the loop. Ultimately, the purpose of a coding agent is to act on some event or something happening where a user or customer is involved, and use that to fix some aspect of the product.

I think what Linear is doing is something Amplitude could do as well, because they’re also getting insights about user analytics and user problems. If you stretch it, even a company like Decagon or a CRM could do something similar: “Here’s what’s broken. Go close the loop.”

Speaker 2

Mm. Mm-hmm.

Mihail Eric

All right, so set all those things up for an agent-ready codebase and set up code review. Security is week seven, not week one. [laughs] Students: “What do you mean security’s not important?”

Speaker 2

No, I make fun of it because I always say that at Amazon, “Job Zero is security,” and they always say this.

Mihail Eric

Yes, yes.

Speaker 2

Anyway, security review at the end.

Mihail Eric

Okay. I was at Amazon, and I don’t remember that, but I believe you.

Speaker 2

This is a thing Andy Jassy says.

Mihail Eric

Okay, okay, okay. I see.

Speaker 2

Yeah. I was there partly during Jeff’s time and partly during the transition to Andy. This was a topic last year as well, and it’s one this year too. Isaac also spoke last year. I find him to be an incredibly thoughtful leader in this space.

Semgrep is an interesting company because they were around even before agents. Their open-source repo was around well before the traditional SAST defense mechanisms. They’ve done a good job augmenting their offering for this space. It’s an interesting time; there’s a lot going on, and these attack vectors are coming to the surface. There’s a lot more slop and AI-generated code. One of the core insights I want to drive is dependency tracking and supply management—supply chain, basically. You have to be careful about your dependencies, pin things appropriately, and not just arbitrarily hit “npm install” and expect the chain to work.

Mihail Eric

So potentially, the most secure codebase is one with no dependencies.

Speaker 2

Yes.

Mihail Eric

Right? You look at a dependency and say, “Cool implementation, bro, but I’m just going to make my own.”

Speaker 2

That’s absolutely right. It’s an interesting insight because, taken to the limit, if you believe code is now commoditized and easy and cheap, why not just build everything in-house? Of course, that would be the safest thing: you’ve seen everything and control everything, instead of depending on Axios or whatever. We’ll see how many people actually act on that, but that is the safest thing. I agree with you.

Mihail Eric

No, that statement is dangerous, because then you’d just make your own mistakes instead of relying on other people to discover mistakes. The whole point of open source is that—what is it?—sunlight is the best disinfectant. It’s like Linus Torvalds’s thing. You’ll be slow to discover mistakes in your own custom software dependencies, because you only have your users, and they won’t report it to you. But if something has 100,000 users worldwide, someone will find something, and then you can depend on it more.

Speaker 2

Yeah. This is—there’s no… I don’t want to be—I’m trying to be a voice of reason.

Mihail Eric

Right, right. You can’t be absolutist in both directions. I kind of trapped you there. I talked you into the talk track of vendoring everything.

Speaker 2

Right, right. And there’s a bug there.

Mihail Eric

Yeah, it’s an interesting one. I wonder how much that’s impacted by the fact that everything—I’ve always believed that most open-source repos, if not all of them, will just become software factories that are effectively agent-run in the future. What happens then? Even if you look at Warp, they’re also selling a software factory product, but they’re running their open-source Warp terminal as a software factory. That suggests humans will still be involved, but increasingly less so. Certainly, not every single person is reviewing every single thing that comes in at the same level we did in the past. Does that make this codebase more or less vulnerable to security? It’s hard to say. That’s where there’s still a little nuance, and even the people I know who work in open source are trying to figure out exactly what that stable point is.

Speaker 2

There are a couple of related arguments, and I’m happy to keep going. Security is a nonstop, huge surface-area problem. There’s one argument from Malte Ubl of Vercel: “Okay, guys, let’s take every piece of highly dependent software written in C or C++ and port it to Rust. We’ll be faster and more secure. It’ll probably take $6 million, but you’ll get it done.”

There’s another argument, and this also links to “Pacing the Frontier,” from the frontier model labs—

Mihail Eric

Mm-hmm.

Speaker 1

The model labs are like, “Okay, let’s slow down. We’ll do Daybreak, we’ll do Glasswing, we’ll make sure everyone’s patched, and then we can let loose on rampant acceleration.” I think there’s an inherent belief that the models will fix all security.

I think that’s unproven. What if they introduce new bugs, just different bugs?

Speaker 2

That’s fair. That’s fair.

Speaker 1

I’m looking for pushback. I’m looking for someone to say, “No, you’re an idiot. Obviously models are way better at this.”

Mihail Eric

Yeah. I guess that’s almost the question in the limit: Do we believe a model will create perfect software? If a model’s creating—

Speaker 1

Yeah.

Mihail Eric

—perfect software, then you’re done. Is there such a thing? Then we’re done here altogether. We’re just going to go home and chill on a beach for the rest of our lives.

I think there’s also the question of whether it’s good enough to use for security first, so people can patch, and then just delay the release and do it afterward.

Speaker 1

Hmm.

Mihail Eric

Or is the problem that it’ll be much better at finding vulnerabilities, so you can really test it for longer even if you’ve patched? How long, right?

Speaker 1

Yeah. You can give it 3 months. Someone can try to exploit it for 6 months. The problem’s not solved, right?

Mihail Eric

Right. Right. Yeah. And my whole problem with this—and this is a whole rant we don’t need to go into—is that you name 100 select organizations and say, “Okay, these are priority. They have access to a cyber model you’ve just created 100 attack vectors for me to use to compromise them.” And, by the way, they’re pretty easy to compromise.

Speaker 1

Yeah. Mm-hmm.

Mihail Eric

I don’t know. Anyway, it’s above our pay grade.

Speaker 1

It’s always fun to get into security.

Mihail Eric

Yeah, security is very juicy right now.

Speaker 1

Yeah. I think the security folks love it because they usually don’t get that much attention, and now they have a ton of attention. Is that true?

Mihail Eric

What do you mean?

Speaker 1

I mean, I find that, at least from a VC perspective, people love investing in security companies. A company like Wiz has done very well.

Mihail Eric

Wiz and Socket—

Speaker 1

I literally had this conversation with a VC 2 days ago. Apparently, Wiz and Socket are the only security companies in the last 5 years that have worked.

Mihail Eric

What about Abnormal?

Speaker 1

I don’t know. What do you mean by “worked,” from a VC point of view?

Mihail Eric

I see. From an economics perspective.

Speaker 1

Yeah.

Mihail Eric

Hmm. Interesting.

Speaker 1

Yeah. Interesting. Not for me to say. Obviously, there are many great players in that field, but it’s a tough industry.

Mihail Eric

I think it’s not controversial to say the best time for any security company is right after a high-profile incident has happened.

Speaker 1

Yeah. Obviously.

Mihail Eric

Then you’re going to sell a bunch of security.

Speaker 1

Totally. I think that’s fair. People should be aware; people should know the basic mitigations. I do think code scanning is important, but I want people to understand that models introduce bugs you wouldn’t make as a human. If you’re not checking those bugs, then—

Mihail Eric

It’s not even that they make bugs. It’s that you have less awareness of what’s being written, right?

Speaker 1

Yeah, because you’re not even reviewing everything.

Mihail Eric

Yeah. Forget bugs. Models are pretty good; on average, I’d say they’ll have fewer bugs. The problem is that you don’t even know what you’re putting out.

Speaker 1

Yeah. I think we have to stay on top of mitigations, and there’s also the combination of security and security as part of code review. Doing those checks is important.

Mihail Eric

Yeah. One more thing—or just check my intuition on this—is that not all parts of the codebase are created equal. The parts of the codebase where security is very important are data storage and billing. Let’s call it data, billing, and auth. Auth is obviously related to all those things. Basically, don’t write code there, and everywhere else, go nuts.

Speaker 1

That’s absolutely right. It’s really funny: if you don’t know the difference between auth and authz, please learn it. If you’ve never looked at AWS IAM, which you have, please learn it. These are basic concepts to understand, and you can prompt for them. Once you have the language, you can prompt for it. If you don’t have the language, you’re screwed.

Mihail Eric

Totally. I’d even extend that to certain cloud-platform things. There was a huge incident at one company—I forget which—where a DNS issue brought things down. Do you remember this?

Speaker 1

There have been many, many DNS issues.

Mihail Eric

You’re going to have to narrow it down.

Speaker 1

I know. There was a particularly prominent one. Was it—?

Mihail Eric

There have also been many prominent DNS issues.

Speaker 1

I don’t remember now. List them off. Let’s go down the list.

Mihail Eric

It’s always DNS.

Speaker 1

Yeah. DNS is more of a reliability and uptime issue.

Mihail Eric

Sure. Security is more like, “Did you lose money? Did you leak customer information?”

Speaker 1

Yeah. I’d probably put that above DNS, for sure. DNS is about general uptime, SLAs, and stuff like that.

Mihail Eric

Background agents—now you’ve got your buddy from Cloudflare here. But your guests aren’t necessarily correlated with that week, right?

Speaker 1

Yeah, no. Week 9/1 was a bit of a scheduling thing. I’ve thrown in Elad and Amjad. I guess Amjad is definitely week 9. I was trying to get Elad at the end, but timing-wise, it couldn’t work out.

So it’s a bit of a weird one where I’m not lecturing at all in week 9, but we’re coming back to those topics in week 10, so it’s a little staggered in a bizarre way. Week 8 is Rajesh, who’s been leading a lot of the infra stuff at Cloudflare—cloud infrastructure, DevX, and beyond. As I said earlier, I think they’re doing a great job. Their adoption, for a company of their size—a public company of their size—is fantastic, and their content is really good, too.

This is fundamentally about all the ways you can use asynchronous background agents. I’m also going to spend quite a bit of time talking about Stripe Minions, Spotify’s version of this—I forget the name—the Ramp Inspect, and things of that nature that show what it looks like to scale this out.

Mihail Eric

Cloudflare. Instacart? Grokbot? Muse?

Speaker 1

Yeah. Muse. I guess those are less about consumer use and more about software teams using them at scale.

Mihail Eric

They’re all coding agents under the hood.

Speaker 1

It’s true. Everything is a coding agent.

Mihail Eric

Grokbot is one of those interesting things that’s very much presented as consumer-y. It’s Grok under the hood. It’s Cursor under the hood.

Speaker 1

Yeah, yeah, yeah. It’s the Cursor team.

Mihail Eric

No, no, for sure. I’ve seen Cursor working in the data docs, but they don’t have a codebase integration, right?

Speaker 1

They do.

Mihail Eric

Oh, okay. I haven’t used that.

Speaker 1

I’ve only been using it for OpenTable reservations.

Mihail Eric

Exactly.

Speaker 1

I think that’s the point of what they’re trying to do: you can use it for consumer stuff, but there’s extra power if you know how to use it well, because it’s actually Cursor under the hood. I think this applies to all coding agents: Claude Code, Claude Design, and people using Codex without knowing they’re using Codex. A random code-interpreter window pops up, and you’re just asking ChatGPT a question. At some level, there’s a coding agent under the hood.

Mihail Eric

Yeah.

Speaker 1

I’ll give you another practical example. I run all my SEO and AEO inside Codex, and it has a codebase, even though a traditional marketer would just check Ahrefs or something. I write all these scripts for doing all sorts of things. I also need to store data and visualize it. Pretty soon, if you’re doing anything on an ongoing basis that has a lot of steps or a lot of data, you need a codebase.

Hmm, that’s an interesting take. I think models also like building codebases. But that could be a temporary thing, right? I guess my point was more about the interface: are you going so far as to say that a Grokbot will have a place where a power user can see the code under the hood? Is that what you’re saying?

Mihail Eric

Well, there is a computer attached, right? Maybe they’ll just jump to Cursor, I guess. Is that what you’re saying?

Speaker 1

Exactly, because otherwise you basically cannibalize Cursor. If, in the limit, what you’re saying is that Grokbot is Cursor, then you don’t even need Cursor if everything under the hood is just becoming—

Mihail Eric

It’s the same team, so they—

Speaker 1

Maybe they don’t care right now. But, yeah, it’s—

Mihail Eric

Yeah, it’s all Grokbot.

Speaker 1

Yeah, I buy that. I do accept that even things that seem outwardly like nontechnical use cases are technical under the hood, and can be written, designed, and implemented in code. That much is absolutely true. The question is more: for a developer, is that their single point of entry?

Mihail Eric

Yeah. I think it’s not even developers, right? Think of a regular person doing marketing who’s using Claude Code, or just Claude in the app. They’re working in a codebase they would never know or touch, but there’s a codebase under the hood.

Speaker 1

Mm-hmm. Yeah, Claude Code is a good example of background agents. That’s a big part of how Anthropic works. You should see how they work.

Mihail Eric

Yeah, yeah. Check it out.

Speaker 1

Okay, AI-native team. We talked a lot about these topics, like gateways. We just put a routing episode out today, and MCP portals. I’d say MCP portals are the least established.

Mihail Eric

You said you’re bullish on MCPs.

Speaker 1

I’m bullish on MCP. It’s not bullish or bearish; it just is.

Mihail Eric

Mm-hmm. It’s just part of the landscape now.

Speaker 1

That’s a statement.

Mihail Eric

It is. There was a whole “MCP is so back, MCP is dead” thing. People like extremes. But, yeah, portals are an unproven business. I thought about starting one. I actually pitched a guy at Anthropic on potentially starting one, and then I lost interest. I lose interest in a lot of things.

Speaker 1

What’s in there?

Mihail Eric

What I’m thinking is, if this is more geared toward scaling things out, and what happens when you take some of these principles, a lot of this was informed by conversations I had with people leading AI-native teams at various companies. I think something like an MCP portal becomes interesting once you’re at a certain scale, where you effectively need a single point of entry for governance, security, auditing, and cost optimization. I’d be curious to hear what you see as the future—or maybe lack thereof—of something like this. If you believe in something like OpenRouter, a single point of entry for many things to connect to and be governed, I feel like there’s a world where this could easily exist.

Speaker 1

My only tweak is that this folds into the LLM gateway. They’re the exact same platform.

Mihail Eric

I’ll give you that. I can accept that. And then, by the way, it also folds into the other thing—what did I say it was? AuthZ, permissioning, and billing. Ultimately, all infrastructure converges on the same problems: there’s a box in the middle, and then you have boxes on two sides. The God box will solve everything for the enterprise, and that’s true only if you understand the job of the God box. Separating MCP portals and LLM gateways means you don’t fully accept that they’re the same layer problem.

In a very similar way, I find that people don’t understand database querying. They separate filtering from ranking, or from general recommendation, but they’re the same problem. When you combine all these things in the same layer, you can make them depend on one another in useful ways. If they’re different layers, they can’t.

Speaker 1

Yeah, I actually won’t disagree with this. As a slightly pedagogical point, I think this breaking up into separate terms versus one is absolutely correct. The underlying architecture is the God box, where one thing goes in and there’s a bunch of other stuff—third-party, internal, and first-party—on the other side.

I have a spicy tweet about this. Where is the God box tweet? Oh, yeah. This is the M×N to M+N architecture. Let me bring it up.

Mihail Eric

Sure. Sure.

Speaker 1

Anyone working in infrastructure needs to understand this: congratulations, you now have a job in infrastructure, or you want to be an infrastructure engineer. Your entire job is making God boxes and then taking them apart. Either you don’t have a good God box, or you don’t like someone else’s God box, so you have to take it apart and make your own.

Mihail Eric

That’s funny. I like that. That’s very funny. Into the heat death of the universe. Wow.

Speaker 1

This is a restatement of the idea that there are only 2 ways to make software: you bundle things or unbundle them. But what are you bundling? The things that go together, hopefully. Over time, people’s opinions shift about what goes together and what shouldn’t. Maybe Monaco should be more of a point solution; maybe Monaco should own everything. In some ways, you’re bundling for the startup segment you serve. In other ways, you’re unbundling from Salesforce, and that’s how you win market share from Salesforce.

Mihail Eric

That’s interesting. It’s an even broader statement, connecting it to vertical AI. Bundling and unbundling—that’s the whole Craigslist and Airbnb thing.

Speaker 1

Yes, and Tinder and all those things are separate.

Mihail Eric

Literally separate. Tinder is basically a personals site.

Speaker 1

Yeah, yeah. Never thought about it that way. One of those things: we just shipped the OpenRouter episode today, right? OpenRouter is a huge success. Lots of people tried to build gateways, but none of them were as successful as OpenRouter. OpenRouter only builds gateways. They disagree with me that the router layer should also do ops.

Mihail Eric

Because Arize is doing ops and observability. They don’t have an MCP gateway; they only handle model routing. I love being wrong like that. They’re saying, “We are the point solution, and we only do this one thing really, really well.” That’s their engineering focus, which is good, but it’s also their marketing focus, which is nice.

Speaker 1

I think what you’ve described—the one-stop shop for all the things—is ideologically what makes sense. But in practice, especially in the enterprise, this is broadly under infrastructure, under platform, and there are separate teams owning different parts of the stack. For that consolidation to happen at enterprise scale, someone has to come together and say, “Now there’s a single endpoint you’re hitting for both of these things.”

Mihail Eric

And that’s fundamentally a hard thing to get everyone on board with.

Speaker 1

Yeah. So often it’s more of a political or people problem than a pure software design problem. From a software perspective, this makes a ton of sense.

Mihail Eric

Yeah. But software systems are sometimes bottlenecked by people problems more than pure software problems.

Speaker 1

Right. My framing is usually: you ship your org chart. I forget the name of the principle, but Conway’s Law says you ship your org chart. You buy your org chart. When you start a startup or make a tool, who in the company are your best friends?

Mihail Eric

Hmm. Hmm.

And if it’s for everyone, it’s for no one.

Speaker 1

That’s interesting. So a conscious part of Latent Space and AI Engineer is, “Okay, there’s this category of software engineer that really cares a lot about AI. Is it going to become their day job? Let’s only make it for them.”

Mihail Eric

Right. Not a bundling of engineering. It’s never going to be the dev tool for everyone.

Speaker 1

We talk about databases and security, but there’s an AI angle to it.

Mihail Eric

Mm-hmm.

Speaker 1

We’ll see. That makes sense. Cool. Software factory. One thing I’ll shout out, related to this point, is that I think a role of the future that’s increasingly important is platform engineer. This is a huge one.

Mihail Eric

More lore: at Temporal, before all this, we wanted to own platform engineering. If I hadn’t started AI Engineer, I would’ve become a platform engineer. That’s the role that decides all this mission-critical stuff, and then the kids over in applications can do whatever they want on top of our platform. The big boys do platform.

Speaker 1

And I think nowadays they own cloud, DevEx, security, SecOps. It’s all the things where you can actually lose the company or make or break it. It’s platform people. Product people can adjust the CSS.

Mihail Eric

That’s how you’re starting a fight here.

Speaker 1

I’m not the CSS adjuster, but I appreciate everyone.

Mihail Eric

It’s a fun time. What’s the future? I feel like all the guys can adjust all the things.

Speaker 1

Yeah. So there are a few aspects here. There’s a little bit of spillover from the previous week: coding agents and big teams, some of those concepts. But we’ll touch on the software factory. I want people to start thinking about this world of the machine that builds the machine, and where, if at all, people are still in the loop. Today, I think the answer is some combination of intent and review: intent, where you’re defining the specs and what needs to be built, and review, where you’re doing some amount of your own checking of the system. I think that bottleneck is going to loosen up before even the intent piece does. For now, those are the 2 choke points in a fully automated system. I want to give people exposure to that.

For one of the open-source partners, Warp, they’re doing this as part of their contribution to the open-source program. They’re going to make a version of Warp that runs through the factory, and students will be building and contributing to the factory and keeping it running.

Mihail Eric

Yeah.

Speaker 1

Work in our factory.

Mihail Eric

Exactly.

Speaker 1

Then they’ll find things they find interesting and pull them back in. This is one of those things where I’m not sure. Obviously, Factory itself is very keen on this. I feel like everyone wants to build their own.

Mihail Eric

Yeah.

Speaker 1

That’s a good question. They’re entitled to try to sell the products they make. I think it’s a hard system to own in its entirety. It’s a lot. “Decide everything for me, please.” It is a lot. I haven’t gone too deep in understanding how they approach their go-to-market here, but I suspect you’d have to do it piecemeal. You’d say, “Hey, we first do your coding agent. Oh, then also…” You basically land and expand. We talked about this bundling and unbundling piece. You don’t start by saying, “Here’s your whole factory.” There’s no way you could get a Ford on board with that, just plopping the whole thing on their desk. At least, that’s how I’d think about it: first be their coding agent of choice, build up some background agent for them, own some of their SRE pieces, and then say, “Now we can weave it all together if you define everything in code or configs.”

There was a fun blog post about this from OpenAI. Our buddy Ryan—we had Ryan on the podcast. They spent 5 months making an internal tool with zero lines of manually written code. They walk through it. It’s called “Harness Engineering,” so you can learn about harness engineering. One of the learnings was that there was a slump where it was slower at first, and then you offload, and you have peer review. It’s from February, but it’s a vision of a software factory: fully committing to letting agents write everything, and how to do that well.

Mihail Eric

Absolutely. Some version of this is going to exist. I do wonder about the sales piece here, the actual business of this. That’s an interesting question.

Speaker 1

From a principles perspective, I think this has been the vision of the future. Beyond that, I think there’s going to be some discussion about what it means to be a developer in the future. People talk about taste, understanding systems and architecture, and software taste. I want to go a little deeper and start putting some answers to that question. So this is your answer to where AI software engineering goes next?

Mihail Eric

Yeah. The factory is tactical, but beyond that there’s a question around the role. If the factory is running full steam ahead and the conveyor belt is churning out code, what do we do? You and I had this discussion with Jerry at the dinner a few weeks back with Dex and Jerry, around the factory. One of the points there that I’m pretty much on board with is increasing levels of abstraction, and that somehow someone, even a CEO, still has to have enough oversight and understanding of the different constituent parts to make a judgment on direction, course-correct, and set metrics.

There’s an analogy that translates to someone who oversees the technical side. We call it a manager of agents, or use different terms. They’re all buzzwordy, but in some respects, it’s someone who understands product requirements and constraints, technical constraints, loops through all of those, and knows how to adjudicate.

Speaker 1

In the chair before you, we just had Anthropic’s head of product and head of engineering for the platform. I asked, “When are your jobs going to combine?” They said, “Not yet.” If Anthropic isn’t there yet, you shouldn’t feel like it’s imminent.

Mihail Eric

Good. Did they say how long, or was it just “not yet”?

Speaker 1

Not yet, but who knows? There was definitely a lot of, “Okay, yeah, I’m doing her job. She’s doing my job.”

Mihail Eric

Yeah, I think it’s also an interesting perspective for students, right? You said there are a lot of sophomore undergrads. They’ve got 2 or 3 years to plan out what they want to be doing.

Speaker 1

Yeah, there are different ways to look at it. You’re not straight in industry. The thing I care about, and think about a lot, is the loop stuff. You start writing prompts, and then you no longer write prompts. You write prompts that write prompts.

Mihail Eric

That’s right.

Speaker 1

And then you just keep going up. I think the recursive, concentric-circle nature of going outward but also inward is something that builds the factory. I think you build up to the factory, not build down.

Mihail Eric

That’s right.

Speaker 1

But I think a lot of people will eventually start building down. You start from the top level—what is my goal?—and then figure out the systems. That currently creates a ton of slop, and you shouldn’t do that. But someday you will.

Mihail Eric

Mm-hmm.

Speaker 1

So that would be my question to prompt you: What’s the prime question that people exist to solve? That’s your thing, right? Let’s say an engineer today: you’re measured by your PR merge rate.

Mihail Eric

Right.

Speaker 1

PR contribution rate.

Speaker 1

Well, 2 years from now, that’s going to be ridiculous. Okay, what’s your main thing?

Mihail Eric

Right, I don’t know. Money?

Speaker 1

Yeah. People always go to money too quickly. It’s something in between money and—

Mihail Eric

Right, PRs.

Speaker 1

Customer satisfaction—

Mihail Eric

CSAT scores from your features. Even that’s a product decision, not really a code decision, right?

Speaker 1

Mm-hmm. I don’t know. But if engineering is evolving into more of a PM engineer, a product engineer, a hybrid, then—

Mihail Eric

Then I think that’s my answer to the question of what the role evolves into. I think it goes without saying that I would recommend students today spend more time thinking about holistic systems—not just their single part of the engineering puzzle, but how that ties in with everything else.

In practice, this is what ends up happening. For their capstone projects, I encourage students to go out and build something. In fact, the seeds of ideas I give them for final projects are the Y Combinator Requests for Startups. I’m almost like, “Go build a company.” Go build something interesting.

Speaker 1

What did people build last year?

Mihail Eric

A variety of things. One person was in the E-school, studying EE, and built a translator for parsing manuals that have memory addresses in C, then converting that into code they could interpret and visualize. Stuff like that is specific to their use case.

Someone was a designer and is now at Vercel. She had an idea where you take a picture of your design notebook and generate an app based on it. For her demo, she snapped a picture of some random scribbles she’d made, and got a fully fleshed-out Vercel app. I think that probably helped her get a job at Vercel doing some version of that.

Speaker 1

Wow, nice. That’s a high bar, actually. Vercel is very known for its design engineers.

Mihail Eric

Yeah, and I think she’s now working on Eve, from what I remember.

Speaker 1

Ah. No v0 in your—

Mihail Eric

Eve is an OSS partner. One of the ways Vercel is partnering with us is by having students build with Eve.

Speaker 1

Yeah. So this is the concept of meta-harnesses, right? Harnesses of harnesses. You can always go one level deeper.

Mihail Eric

I do think it makes sense, at least for me. In the same way that model routers wrap model APIs, you need to wrap harnesses as well if you’re going to swap between them.

Speaker 1

Yeah, that makes sense.

Mihail Eric

Agreed. I think students should also stay grounded. There’s a lot happening at the edge, but there are also a lot of companies still using a single ChatGPT call in production, just playing around and not having tried much else. There’s still time for it to work.

Your point about taste is good. Have taste, try stuff. Whether each company has its own version of a software factory or you’re building tools to help companies do it, stay on top of it, use the tools, play with them, and you’ll figure it out along the way.

Speaker 1

Yeah, and I think—100% agree with all that. First, to your point, a lot of companies are still prompting ChatGPT or some version of it. Maybe they just got a Cursor license and are now getting people onboarded. That is way more true than we believe, certainly here in the Valley. I can say that with 100% confidence.

But for students, being exposed to this is important. The fact that they know this and are on the edge will eventually let them be some of the more interesting and valuable contributors when they go to places that maybe aren’t up to snuff or on the frontier.

Mihail Eric

One progression-type talk that’s still very influential, from about a year ago, is Steve Yegge talking about the levels of psychosis. He’s all the way through level 10 now.

Speaker 1

Where he’s—

Mihail Eric

He’s given up and moved to a farm. I’m making fun of him. He built Gas Town.

Speaker 1

Yeah, I remember.

Mihail Eric

He moved to Gas City. Now he’s burned down the city and said, “I’m building my game. No more Gas Town.”

Speaker 1

I thought that— I remember talking to him at the AI Engineer World’s Fair earlier this year. Wasn’t his whole thing that the only useful thing built at Gas Town was Gas Town?

Mihail Eric

Yeah.

Speaker 1

Okay. That’s a horrible failure.

Mihail Eric

Totally. People should be aware that psychosis is real. If you’re not holding yourself accountable to delivering real products, you’re just screwing around with LLMs.

Speaker 1

I wonder how much of it is relevant to this, but I played a little with Gas Town. It’s a very involved system, maybe a little ahead of its time. I’d also wonder if it was bottlenecked a little by the marketing, because it was difficult to grok from a terminology perspective. There’s just a lot going on there.

Mihail Eric

I think the Gas Town marketing also helped propel it. What was right before Gas Town?

Speaker 1

Beads.

Mihail Eric

Yeah, Beads. There was OpenClaw, something else, and then Gas Town. The manifesto also really helped spur the concept. People who would never use Gas Town learned about it because they could read the manifesto. There’s a marketing angle to it.

Speaker 1

I think Steve isn’t necessarily someone to take literally, but directionally, he’s correct. If you’re at the idea stage, it’s time to move into the power-user stage. If you’re using a single agent, it’s probably time to move to multi-agent, and you probably should try to build your own orchestrator. That’s all part of it.

Arguably, this is already out of date. There’s more to the software factory beyond this, and this article predates the software factory. As an employer, I have this problem in my company today: I don’t want to hire anyone who hasn’t been through psychosis and come out the other end, because I’m currently paying for somebody to go through psychosis. It’s very expensive in salary and tokens, and I’m getting nothing out of it. My feedback doesn’t get through.

It’s almost like I would encourage your students to have psychosis and then come out and ask, “What was wrong with me?” That’s what Steve just went through: a very public 2-year journey.

Mihail Eric

It’s weird on the student side. There’s a lot of industry where you think you’re being productive, but you’re not. That’s probably the student version. You’re making Notion charts and all these optimizations and personal apps, and then it’s like, “Dang, I went through the psychosis, but nothing has changed.”

Speaker 1

Right. So go through it.

Mihail Eric

Go through it.

Speaker 1

I haven’t seen this. I’ve seen a version of this from Boris, actually. It was more geared toward Claude use cases, but I remember he made a Claude artifact that showed the progression, I think more at the team level. He had 4 stages, not 8, but it roughly followed the same progression, from no adoption through to—

Mihail Eric

Only 4 is good. This is too many levels.

Speaker 1

We’ll see if people can pull it up. Other than that, zooming out, have we discussed all the things—

Mihail Eric

That’s the one.

Speaker 1

Oh, that’s the one? Okay. It’s the Boris—

Mihail Eric

Yeah, it is an artifact.

Speaker 1

Okay, so it’s gated, assisted, parallel, supervised autonomy, and AI-native.

Mihail Eric

It’s zero-indexed, so it’s 5 levels. Things start at zero.

Speaker 1

Wow. Actually, it’s pretty good. I need to read through this in more depth. And use AI-native.

Mihail Eric

Yeah. I’d give a shout-out to dynamic workflows. This is a sub-agent pattern that the OpenAI guys are also exploring, and it clearly fits. Basically, you’re writing a harness on demand for the thing you’re trying to do.

A lot of the sub-agent conversation is just copying the parent agent, but you actually should be writing way more code and then running that as a sub-agent. That’s what the Anthropic people have chosen to call dynamic workflows.

Speaker 1

We built a version of this as well.

Mihail Eric

Yeah, it’s a good pattern. For Monaco?

Speaker 1

Mm-hmm.

Mihail Eric

Yeah. What does it do?

Speaker 1

On the fly, sort of like harness use cases for GTM.

Mihail Eric

Ah, yeah. That’s great.

Speaker 1

I’ll note that when Boris wrote this, it was for team adoption, for people who are listening. He also says, “Anthropic is at stage 3. We’re still trying to get to stage 4.”

Mihail Eric

Wow. He’s like, “I’m at stage 4. I’m the guy.”

Speaker 1

But it’s not as simple as Steve’s. Steve is very good at “do this, do this, do this.” This is much more for running a team and wanting to AI-pill your team, not just be a user of one.

Mihail Eric

Yeah. Anthropic is still at stage 3. Don’t be like—

Speaker 1

Yeah.

Mihail Eric

Yeah, yeah.

Speaker 1

Okay, I mean, you should teach this. This is great. This has got all this stuff—this is everything we talked about.

Mihail Eric

Yeah. I can’t believe I haven’t come across this.

Speaker 1

Good resource, yeah. Very, very good. Okay.

That was a wild tour. One semester of Stanford in 1 hour—or 2 hours. Thank you for doing this. Any parting thoughts? Any call to action?

Mihail Eric

No, thanks. I mean, this has been a ton of fun. It’s been 9 months in the making, and I’m excited. This is the end of week 1. There’s a lot to do and a lot to learn. I’m excited to see what happens, especially with the OSS partners and getting students deep in the code of some complex codebases and having them build some stuff. It’ll be fun. I’ll keep you guys posted.

Speaker 1

You’re doing another version next year? If OSS people want to get involved, they need credits. Can they reach out?

Mihail Eric

Sure. Sure, yeah.

Speaker 1

Do credits matter? Do they want them? Do they actually want the credits, like the students, or—

Mihail Eric

Yeah, they want credits. You do have a credit card. I don’t think a lot of students want to spend 5 bucks. I work in AI, but—

Speaker 1

Yeah, no, I think—

Mihail Eric

Look, I remember there was a certain number of credits for sure, but on top of that, he was like, “Look, if they want to go hog wild with this, let me know, and we’ll do these one-off experiments where they can scale up to 1,000-plus agents.” And so—

Speaker 1

Great. This is an email from Vercel.

Mihail Eric

Yeah, an email from Vercel.

Speaker 1

Yeah. Those kinds of people are valuable. So this is Wilson from Cursor, right? Who’s like, “I’m going to write a browser,” right?

Mihail Eric

Yeah. That’s right.

Speaker 1

Those people who are ambitious and can actually execute are very valuable, and that’s basically a recruiting call.

Mihail Eric

That’s right.

Speaker 1

But—

Mihail Eric

Yeah, that’s a good point. I do think students should understand how to be ambitious because snapping a photo and turning it into an app is not that ambitious in this day and age. Writing a browser is. And there are a lot more projects like that.

Speaker 1

Yeah. I think that’s right.

Mihail Eric

It’s all about raising the floor. Last year, credit’s due to the students of last year, but this year, obviously, we’re going to expect more.

Speaker 1

Yeah, that’s the floor now. We’re going to expect more.

Mihail Eric

But that’s the whole point, right? We’re leveling up. Last year it was like, “Hey, have you heard of prompting? Here’s what it is.” I spent some time talking about K-shot prompting as one of the lessons, and I don’t even think you guys think about that now.

Speaker 1

I guess there’s a version where you stick out in your cohort of classes, and then there’s also a version where you’re in early engineering roles with a bunch of people in industry, and people are still trying to get jobs. Not to call out one demo, but you can do a lot to stick out.

Mihail Eric

For sure. I’ll give students a freebie for week 11: make benchmarks. Make benchmarks and evals. This will get you a good one. It’s going to get you in the door of every frontier lab and every top agent lab.

Speaker 1

Good engineering blog posts with some rigor on anything interesting. There’s a lot of low-hanging fruit there, but engineering blog posts do really well, right? You went deep on one thing.

Mihail Eric

Yeah. There’s also a lot of money to be made in that, so it’s not just about getting in the door. If you turn that into a business, you can really cash in on it.

Speaker 1

People don’t know, SWE-bench was an undergrad project.

Mihail Eric

Yes, yes, yes. From Princeton, right?

Speaker 1

Yes.

Mihail Eric

Yeah, yeah, yeah.

Speaker 1

Great. Okay. Well—

Mihail Eric

Awesome. Fantastic. That’s it. Thank you so much.

Speaker 1

Thanks a lot.

Mihail Eric

Yeah, thank you, guys.