GitHub CEO Thomas Dohmke on Copilot and the Future of Software Development

Thanks for joining us. I guess we were just talking before we started recording about acquisitions, and you've been on both sides. Do you have any advice for me and my team post-acquisition of Weights & Biases?

1. Advice for founders post-acquisition

I would say, stay sane. Get some sleep in between the craziness of that process. As you mentioned, I've been both the seller of a startup—I was the co-founder of a company called HockeyApp, which we sold to Microsoft in late 2014—so I know what that was like with my team going through that acquisition.

Until the last moment, we were sitting there, and then everything was signed. We were sitting at dinner, and there was an emptiness surrounding us. You feel like you should be happier than you are in that moment, and then it takes some time to actually process that everything is now changing your life.

I've obviously been involved in the GitHub deal in 2018, mostly behind the scenes. After we had acquired GitHub, we bought, I think, 6 or 7 small- to medium-sized companies. I remember a number of conversations with the founder CEOs of the companies we were acquiring and the emotions they went through. I could empathize with them and understand where they were coming from, relating back to my own experience.

So it is something, for a founder CEO or founder team to sell their company, that is probably an experience in life that is hard to compare to anything else.

Well, I wonder if it was like this for you, but one thing about this transaction is that I couldn't talk about it with anyone for quite a long time. It was also very slow: It felt done, but then we were working through lots of details. There were lots of regulatory approvals, so it felt, honestly, very anticlimactic to me.

I was thinking, "I've been wanting to work on the future stuff for a long time, but I couldn't." It felt very unmomentous to me, I guess, but it probably felt like a much bigger deal to all the people around me who were just learning about it and going through that.

And why? During that phase, when you cannot tell people that this is happening, you also have to—you're almost forced into lying to a certain degree, or planning for a future that you know will no longer happen. You may commit to an offsite or a customer event, and then, by the time that actually materializes, the world will look fundamentally different.

Then you tell people, and of course you deal with a lot of uncertainty and doubt. I'm sure you've gotten the question, Lukas: "What's going to be my role in the future company? What is going to be your role?" People tend to doubt what you're saying is actually the truth, given that as soon as these deals are closed and you've moved into the new office, either virtually or in reality, you realize that the grass isn't always greener on the other side.

There are often as many problems in the buyer company as there were in the seller company. So, yeah, I hope everything works smoothly for you and your team. Next time, maybe we can talk about life after—what it's now like after everything is fully closed and integrated, and so on.

Yeah, no, I appreciate it. It's a very unfamiliar experience for me, honestly, because I've spent most of my career running companies. I didn't think I was particularly attached to the CEO title, but one big difference is that I can't really guarantee people that I can control what happens. It's a very different kind of conversation.

I feel a lot more empathy for the managers who have worked for me in the past now that I've had that experience. I think it's really making more sense to me why people want so much clarity from their leaders, along with fast decision-making and things. I actually think it'll make me a better CEO to be on the other side of this for a period of time.

Naturally, you lost a little bit of control, right? Because one way or another, now you have a manager above you, or a set of managers, and the board and whatnot. While people might trust you, they haven't gotten to the point yet where they've seen you react to instruction, direction, or command, if you will, from your management chain. That certainly is a different life.

2. What made the GitHub/Microsoft acquisition so successful

I think, in many ways, that is what is so fundamentally different between being a leader in a medium-sized to large company—including me as GitHub CEO, because obviously we're part of Microsoft and I have a management chain within that construct and deal with the Microsoft finance team and HR and all these kinds of things—and being a founder CEO.

As a founder CEO, you live with a different uncertainty: What does tomorrow look like? What does the company look like in a year? But other than your board, if you have one, there isn't really anyone telling you what to do. I always felt, as a startup founder, that this is the ultimate freedom you can have—even more so if you have a bootstrapped startup where there isn't really a board, venture capitalists, investors, or anything like that.

But it's not that you wake up every morning and everything is a happy place, even more so when you have a product in market and now you have customers and they have support issues. I'm sure you've seen that: They email you and say, "Hey, your team hasn't gotten back to me, and I'm really frustrated, and I'm paying you so much money." Then you're like, "Okay, I was planning to have a nice night with my family, and now I'm dealing with this customer escalation." So, yeah, that is the job.

I'm curious, actually—we're immediately going off script a lot here—but I feel like GitHub, from the outside, feels like one of the most successful acquisitions that I can even think of. This is such a softball question, I guess, but what do you think made the GitHub acquisition so successful? And was it a success? Was it all that you hoped, having thought about it before it happened and now running it?

Yeah, I mean, if you ask around in Microsoft leadership, the answer would clearly be yes. This was one of our most successful acquisitions. I think the other 2 that are always listed in a similar realm are LinkedIn, which has a very similar integration model where they have a certain level of independence.

Ryan Roslansky is the CEO; they have their own office and their own brand. If you go to LinkedIn.com, there isn't much Microsoft in terms of branding or anything like that. It's very much perceived as an independent platform within the Microsoft ecosystem.

The other one is Minecraft, which was a very different acquisition, also in how it started and how it went through. Similarly, the brand is very independent. For most kids playing Minecraft, that's probably their first touchpoint with the Microsoft ecosystem, unless they're playing Minecraft on a PC. But if they have it on an iPhone or Android phone, or on a tablet that their parents hand them, it's not even that.

So you have these 3 brands, and GitHub kind of sits in the middle. It is more of an enterprise brand; that's where we make most of our revenue. But at the same time, it has this recognition from consumers. My favorite example is when I go to the LEGO store in the Pacific Northwest.

Naturally, a lot of the employees there are students, and so they know what GitHub is. They're as much fans of the company that I'm running as I am a LEGO fan, and you don't get that with many other enterprise companies. Part of that is obviously that GitHub is both an open-source company hosting all these open-source projects and a company that sells enterprise products to many of the companies that want to work like an open-source project, want to emulate a lot of these practices, or are sitting somewhere in the middle.

They are both providing open source, like your company, and have part of the stack that is closed.

We’re really happy about the acquisition. Revenue has grown. We announced in July 2024 that we had surpassed a $2 billion run rate, and you can look up for yourself what the rumors were back in 2018. I don’t think there’s an officially published number, but it’s significantly up.

Acquisitions often have scorecards. The scorecard on the revenue side is more than green, and the scorecard on the growth side is more than green. With Copilot post-acquisition, and with the OpenAI investment in 2019, we materialized the best of Microsoft—with its cloud, experience, responsible AI, and all these other pieces—with the best of GitHub, including our developer-first approach and the OpenAI partnership. That gave us access first to GPT-3 and then to the Codex model to build the original GitHub Copilot.

But why is it so green? Is it the OpenAI connection? I feel like, even before that, one thing I noticed from the outside was that development seemed to accelerate after the acquisition. That was my impression as a user. I don’t know if it felt like that on the inside, but what happened there as part of the deal?

We actually defined 3 acquisition principles, or integration principles. The first one was—and I think that’s really part of our success—that above everything, we put the developer first. In many ways, you could argue GitHub has always done that since it was founded by hackers who had discovered that Git is cool.

And, Thomas, did I tell you that they worked for me? I’m actually user number 7 of GitHub, or something. I don’t know if I ever mentioned that.

I don’t think you did.

Yeah, it’s funny. I was there when they started. It was on my co-founder’s Mac, actually, that they were hacking. Chris, Tom, P.J., and then Scott as the fourth co-founder.

It’s fun to go to the Wayback Machine and look at what GitHub.com looked like in the very early days. I think many people have a much more evolved picture in their heads of what the UI and the front page looked like, but the reality was that it was almost like a commit log, with single-line updates from Chris, I think, mostly saying, “Yep, we have this new feature. Check it out.” That was all that was there on the front page.

The UI looked much more basic than it does today. I think many people have this 2015–2016 evolution in their heads, as if that was what GitHub was always like. Obviously, in the early days of this cloud-native, Web 2.0—or whatever you want to call that—wave of startups, things looked much simpler than they do today.

So we put developers first. Whether we write blog posts, build products, or create internal processes, everybody at GitHub, across every role—HR, finance, legal, and product management—is using GitHub every single day. The only other tool we’re using as much as GitHub is Slack. Those are the 2 tools that, as a remote-first company, we use very intensely. That helps us evolve the product.

The second principle—and I think this is the mistake in many acquisitions—was: How is Microsoft accelerating GitHub? In many acquisitions, it’s the other way around. You think about how to find synergies between the company that you bought and the company that’s buying you, and you look at how the startup is accelerating the big company. We thought about how Microsoft was accelerating GitHub.

That’s where you see things like GitHub Actions, which was an effort where we took existing technology from Azure DevOps called Azure Pipelines and brought that into GitHub. It looks like GitHub, and it has a format for defining these workflows. The underlying technology, and part of the team, came over in early 2019 to build what is today GitHub Actions.

You can track that progress. Part of the success was that we very quickly figured out how to do joint sales with the Microsoft sales team around the world, which gave us coverage that GitHub, with about 800 employees at the time of the acquisition, could never have had. That helped us accelerate, but Microsoft really invested in GitHub to let it grow.

I think that is crucial. In many ways, it’s almost like a Series D or Series E round of investment. The third principle was: How does GitHub accelerate Microsoft? We always said that comes later, and that we would think about it when we had figured out the other 2 pieces.

Copilot was probably the first product where we were intentional about taking our learnings, partnering with Azure, and building on top of a joint stack between GitHub Copilot and all the other Copilots that Microsoft now has. We leveraged the responsible AI components that Microsoft offered us, but we also contributed back to them. I think that describes the success of GitHub now, almost 7 years after we announced the acquisition in 2018.

3. How GitHub Actions and joint sales with Microsoft accelerated growth

Copilot is a really incredible product. For a lot of people, I think it was the first time they were really seeing the power of LLMs. I remember when I first started using it, I had this “Oh, my God” feeling of seeing how amazing this could be.

With that product, I feel like there’s so much momentum in Silicon Valley. You hear about Codeium and Cursor, and it doesn’t feel like Copilot has the same mindshare, at least among Silicon Valley companies. Do you feel like you’re behind in some ways and planning to catch up? What is Cursor doing that’s different from what Copilot offers?

We were very early when we started with Copilot in the middle of 2020, when there was almost no AI hype whatsoever. GPT-3 was available to some nerds, data scientists, and people who understood what language models do. Oftentimes, you had to explain what a transformer was to a customer before you could get to the point of why this might be changing the world.

June 2021 is when we launched a preview of Copilot. You may say we created that whole market with that one announcement. It was a very short blog post and a landing page that showed how code completion worked for different languages and examples.

If you travel back in time on Hacker News and other sites, there was an equal amount of skepticism and excitement. We started giving people access to the private preview. Some were skeptical, then turned around, and you saw that evolution happening: “After using it for a while, I actually see the significant impact.”

There were also legal challenges and discussions around fair use on the training side, replicating code, and licenses. A lot of the investment in the early days was simply scaling out Copilot. I think we had a million users on the waitlist in less than a year, and then going through the waitlist into the private preview also required us to scale the architecture in that first year.

The public launch, what we call GA, or general availability, was in June 2022. That was even before ChatGPT came out. This whole time horizon is so compressed, with so many things having happened since then, that it’s easy to forget that Copilot was not only in preview—it was a GA product before ChatGPT came out.

I was on a trip right after the ChatGPT launch, and I could see that the world had dramatically changed in every customer conversation I had, whether I was in Tokyo, Singapore, or Sydney. It went from, “AI is kind of this cool technology, but we’re far away from that future and from it actually being useful,” to, “Thomas, tell me more about how you built Copilot and how we can adopt Copilot.”

4. Copilot’s origin story and early reception

We had the first case studies showing that developers were 55% faster. Ultimately, the play for GitHub is that we have the platform. When you buy GitHub Copilot, you not only get it in VS Code, JetBrains, Xcode, Android Studio, and whatnot; you also get it in GitHub, including a code review agent to look at your pull requests.

In Microsoft’s earnings last week, we published that Copilot now has 15 million users. The Copilot code review agent, just as an example of that platform play, has reviewed over 8 million pull requests.

I think that’s where we’re seeing the future: this holistic workflow where you have Copilot for autocompletion, chat, agent mode, and MCP integration. Then it continues in your pull request to review code and fix security vulnerabilities. With Project Padawan, which we announced earlier this year as one of the early projects we’re working on to implement a software engineering agent, or SWE agent, you really have a flow where you can go from an issue to a pull request with the help of an agent.

Do you have a sense of that? You’ve thrown out different numbers, and a lot of people talk about productivity gains.

I myself wonder how you could even possibly measure that, but do you have any sense of what the speedup is, or what you’re shooting for with these projects?

We’re ultimately shooting to make every software engineering team, starting with ourselves, able to deal with the ever-growing number of lines of code and the complexity that comes with that. The problem when you compare productivity gains without Copilot and with Copilot is that you’re never working on the same thing twice. You can do case studies, and that’s where the 55% came from: We gave 50 developers a task to do without Copilot and 50 with Copilot, and everybody did the same thing. The team with Copilot was 55% faster. But that obviously is a clinical study, if you will; it isn’t reflective of the real world. I would never have 100 developers on my team do exactly the same thing.

Even when someone works on the same topic, the same subsystem or microservice or whatever, they’re not doing the same thing day in, day out. In fact, they’re adding more stuff to the existing service. And so the complexity is always rising.

When you look at productivity gains, you’ve got to ask: What’s the baseline? What are you comparing against? Ultimately, making AI part of the developer workflow had 2 impacts. One is that we actually increased the complexity for developers building these systems, because full-stack now means back end, database, and all that infrastructure; front end; and now you also have AI, which means you have to integrate models, evaluate them, fine-tune or post-train them, and operate them.

Then you have things like what OpenAI recently saw: They had a new version of the model, tuned it too far in one direction, and had to roll that back. But even if you don’t want to wait for customer feedback on that, you have to work with really good eval test suites and a really solid monitoring process. That’s not fundamentally different from monitoring your uptime and your exceptions in Sentry and all these kinds of things, but it adds another complexity layer to our development teams.

So what AI has done to almost all software engineering companies, because the expectation is now that you’re integrating AI into your stack, is that it has increased the complexity. On the other side, it has made coding much easier.

I think autocompletion is all about flow and staying in a flow state. When you’re typing code in VS Code yourself and it predicts the next 10 lines of code, it removes the obligation to go into your browser and search for how to use a key vault or how to implement an algorithm. Even with simple things—when you work in a Ruby on Rails app and have a model with a couple of columns, then go into the controller and want to instantiate the model—having it tell you what the different attributes of that model are can be useful.

Some of that already obviously works with IntelliSense or IntelliCode, just more classic autocompletion, but it really keeps you in that flow state. Oftentimes, developers can just accept code, and then it’s close enough to what you’re trying to achieve that you can modify the pieces that are broken, have syntax errors, and whatnot.

And let’s face it, that was also true when we copied Stack Overflow code from GitHub repositories or blog posts, right? That code was never actually perfect. You had to take it, paste it into your code, and then make it work. So that’s where I think autocompletion has really improved the flow state.

Now, with next-edit suggestions, you can just tap, tap, tap from one suggestion to another. The latest version of VS Code now has autocompletions with syntax highlighting, so you can parse that a bit more than the original ghost text. This brings us back to the early days of this UI concept, ghost text: basically showing you 10 lines of code within the editor that you can autocomplete into. That didn’t exist; we built that in partnership with the VS Code team for the original Copilot.

Before that, autocompletion was a dropdown where you could scroll through the different options from the documentation. In TextMate, I think it captured what else it had in that file and created kind of like a DSL out of that—not a DSL, but a menu of options.

Now, if you go into agent mode and agents, or things like Bolt.new or Lovable, or Vercel’s v0, in many ways, if you know how to prompt it, you can actually have it write all the code. v0, as an example: You give it a prompt and it renders the whole web page. Behind the scenes, it obviously uses Next.js, and then you click a single button and it deploys it to Vercel.

It was never easier to spin up a quick web page for your wedding or birthday invitation, or even to sell a product real quick with a pop-up store. Then you can use prompts to redefine that code.

5. Launching Copilot before the AI hype wave

That’s super efficient, right? You often don’t even need to hire a developer anymore. You see a bunch of examples out there where the users of these tools are not actually folks with a software engineering background. They just heard about this on your podcast or elsewhere, that this is cool, and then they’re trying it out, similar to how they’ve used ChatGPT to render an image. When the image isn’t exactly what you want, you redefine it with more prompts. They’re doing that with these web pages.

Where I think the biggest challenge lies is that the majority of actual large-scale software projects—VS Code is one, GitHub is one, Weights & Biases and others—is that the complexity of these projects is so high that today the agents cannot actually tackle them.

It’s much the same way that, so far, there still isn’t a car on the market that you can buy that has no steering wheel. The closest is Waymo in San Francisco and a bunch of other cities, but it’s ring-fenced to that problem space and has a bunch of extra sensors.

I think that comparison applies to agents as well. There are certain scenarios where that works well, but there’s a whole big complexity out there where it doesn’t work well, and where the agent—or agent mode in VS Code—takes a file, adds the 2 lines you wanted, and then removes 5,000 lines or screws them up in some form.

I think that’s where it’s becoming crucial that developers still need to know how to code and can make a very educated decision about whether they want to use the agent or just go into the file themselves and make that 1-line change, where they know exactly how to do it and where to do it. It’s almost a waste of energy and time if you’re trying to write a prompt for something that you can write much faster in code yourself.

Having this continuum, from having AI infused into the editor with code completion to asking coding questions, explaining the code, inline chat, and those kinds of things, all the way to agent mode and autonomous agents, I think those 2 things all belong together. As developers, we want to move in both directions, up and down, up and down that spectrum.

If you see a pull request that was created by SWE-agent and you see a bug in there, what you want to do is go into the command line, say, “Use the GitHub CLI to check out that pull request,” and then just keep working in VS Code or any of your other favorite IDEs.

Do you think that’s going to be true 3 or 4 years from now? How do you know that? You could look at the situation and be like, “Well, it’s getting better and better. Every month, it seems to be better.” Why won’t it get better at going into specific parts of the code and changing them? I totally acknowledge that you definitely have to do that today, especially on a larger codebase, but so many smart people are working on it. Why doesn’t that get fixed?

It’s getting better. Certainly, the models are getting better, but in some ways you could argue that now, in 2025, we’re still fighting some of the same problems we were fighting back in 2020 with GPT-3 and Codex, where the model just writes code that is a little broken or mistypes an import statement in Python. Now, with agent mode, it can use tools and fix itself by looking at the error message.

But I think fundamentally, the role of a software engineer is not just to create new code and new files. It’s actually to design an application that is unique for the mission, the OKRs, whatever is driving you. Part of that is that you want to build a successful business that grows revenue and returns profit to the owners or the shareholders.

And I think that systems thinking—how to design the application, how to decide between MySQL and Postgres, or between React and jQuery—I don’t know, maybe that is an easy decision.

Go and use React. You know what I’m saying? There are 10,000 choices to make when you’re building an application with decent complexity. Just like when you’re building a house, almost no house looks exactly the same as another house, and almost no application looks exactly the same. If all an agent can do is create code with a few prompts, that startup has no value, because everybody else could also build it, right? The complexity will significantly grow as these agents become more powerful.

I think the flip side of that is that we’re here in 2025, and many, if not all, financial services institutions—banks and insurance companies—still have some kind of mainframe COBOL legacy code, unless the bank was founded in the last 10 years and didn’t do any acquisitions. That journey of managing old code that somebody else has written, figuring out what the developer intended, asking how to refactor it, determining whether there are actually unit tests, and, if there are unit tests, whether they cover enough of the code—that is actually the bigger part of software engineering, right?

6. The rise of CopilotX, chat, agents, and enterprise adoption

If I switch sides and join your company as an engineer today, my biggest challenge is understanding how it all works. I could use an agent to achieve my goal, but how do I validate that what the agent has written is actually correct, doesn’t introduce a new security vulnerability, and doesn’t make the thing so much slower that you need 10 times the compute to run the platform? I think that back-and-forth between the agent and its output and the developer is going to stay crucial for many years to come.

Let alone that, obviously, there are also a lot of security risks in accepting code into the codebase without understanding what the code actually does. This is called the zero-trust principle. Today, most companies don’t allow any developer to push into the main branch. They force a pull-request review with a different reviewer, branch protection, all those kinds of things, and run CI/CD. Only if all these things are green can you merge it back and deploy it, right?

We’re doing that because we’re worried about attacks, and we’re worried about the model writing code that injects a security vulnerability. Whether that came from an attacker or from a built-in vulnerability in the model is effectively irrelevant. We have to deal with that risk, and that risk is mitigated by humans reviewing the code. Maybe they’re using AI for that as well, and I can see that happening. But ultimately, we have to have humans understanding code. You have to have humans designing systems and making decisions about when to use an agent, burning through GPU cycles to write code that actually generates value for the companies we work for.

I guess then you’ve written about this long arc of leveling up to higher levels of abstraction in the world of software development. I totally agree. I sort of felt bad when I was in college for not going deeper on C, because that felt like an essential software skill. But maybe, unless you’re in certain domains, it’s less and less important over time.

I guess you’re also talking here about going in and being able to work at an architectural level—almost like a manager-like software architect mindset—of figuring out what database to use and how to lay out the application. But you were also talking about going in and actually modifying some of the code, or at least understanding it, for security or reliability reasons.

It also seems like the software would be better than a human at writing tests and doing security reviews. Why wouldn’t that get automated away?

It will get automated, I think, to a high degree. If you look at having AI write documentation or unit test cases, it already works to a significant degree, although we also have to be realistic. The most popular benchmark right now is SWE-bench, which has 2,000 pairs of issues and pull requests from a dozen Python repositories. The best agent in SWE-bench is at 62% or 63%, I think.

We were at the top of it until recently, but a 65% score across a dozen Python repositories is like getting a 65% score on a math test in high school. That’s not a great result, right? Ninety percent, I would say, is where we’re getting there. But even that is a 10% gap of uncertainty in a business.

Just this week, the founders of SWE-bench built out SWE-bench Multilingual, and they also announced an agent that can generate new test data for these agents. With the multilingual version, I think the numbers go significantly down, into the 20% to 30% range. We’re far away from these agents actually being at 100%.

Will that point happen? Of course, in the same way that we’re going to see fully self-driving cars. But we don’t know when we’ll get to that point, because it’s not an easy problem to solve, and it’s not one that has been solved before.

But I wanted to come back to the fundamental part of your question: The machine we’re ultimately building software for still runs on a processor that has a built-in instruction set. That instruction set is abstracted all the way to the programming language, but not higher than that. The programming language is the last deterministic abstraction layer that basically maps directly into the instructions that the CPU or GPU executes.

Natural language is inherently nondeterministic. You can say the same sentence or describe the same user interface, and then build a completely different website with different design, functionality, databases, and so on. Having the ability to jump between those 2 abstraction layers, I think, is going to be the craft of a software engineer: making decisions about when you can just describe something in natural language.

That’s ultimately how we think in our brains and how we think when we wake up in the morning and have an idea about something we want to build. We’re thinking in English, German, or whatever language you have in your head. The challenge in the past was, how do I convert that into a programming language? That’s often more frustrating than most folks want to admit, because it takes way longer than you thought it might take.

Then it’s easy enough to pull in an open-source library until you realize, “Okay, this actually created a whole new problem in my application.” Maybe now you have a dependency that requires you to upgrade to a newer version of Rails or whatever have you. I think that conversion—being able to take my idea, describe it in a few bullets, or even use Claude 3.7 Sonnet to reason with the model about what my specification looks like, then create a Markdown file in the project and feed that into Copilot agent mode to actually implement it—is the new way of working.

It still requires an engineer to do that thought process: figure out the user interface flow, build an amazing product, and then jump down into the coding layer to find the one line that’s broken and that you need to fix to compile the application.

7. Why agentic coding still lags and what needs to improve

Do you find that, with all these people suddenly using automated systems, your user metrics have been changing a lot in the last year? I would think that automated code generation would potentially create a massive increase in the amount of code being sent to GitHub. You might get more branching because models are more comfortable just generating tons of pull requests. Are you seeing effects like that in your own data?

I think the biggest one we’re seeing is that adoption has significantly increased, and AI is now part of the lifecycle. It’s no longer a question of whether software developers should use AI as part of their day-to-day workflow, whether that’s in the inner loop—writing code on my computer, code completion, chat, and agent mode, everything that’s synchronous with me on my machine—or in the outer loop, with pull-request reviews, summarizing the pull request, and having a SWE agent build the pull request itself.

I think it’s accepted best practice now to use AI in your workflow. That doesn’t mean everybody has adopted it, but we’re way past the chasm and are now talking about the late majority, if not the laggards that are even later. That’s where you see very intense competition in the developer-tool space.

I would argue that I’ve never seen anything like that in my 30 years, since I started coding on a Commodore 64. You mentioned a few startups earlier, but it’s not only the startups; it’s also all the hyperscalers that have AI coding tools. IBM has one, and everybody is playing in that space.

The extended version of that goes from training models to fine-tuning models, having an AI stack, and going all the way to coding tools.

If you will, the biggest use case for AI today is software development. Maybe that gets us to the point where AI becomes AGI, when the AI can write its own code and improve its own code. Adoption is one thing. I think it's accepted now that AI is part of the developer workflow. The amount of code AI is writing is constantly increasing, and the models are getting better.

We've now been shipping a new code completion model every month, and we have an ongoing mid- and post-training process to improve the code completion model with offline and online testing. In those experiments, we're looking at how many accepted characters are retained. That's one of the key metrics we're looking at: when you get a completion, how many of those characters do you accept, and then how many do you retain?

There's a window where we're looking in the editor at what you're doing and whether you're modifying those characters. That's the metric we're tracking. Based on that metric from offline and online testing, we then decide whether that model is good enough to become what is now the May model, which is the code completion model that Copilot is using.

I think that's where, even though you could argue that code completion is somewhat commoditized now and everybody considers it a standard feature, there's still a lot of improvement happening based on metrics. We're making the models more efficient, improving the margins, reducing the number of GPUs you need for it, and ultimately improving the user experience by having people get more accurate code with lower latency.

Latency, by the way, is the core metric for every developer tool. If it isn't fast, you're always going to have an audience of software developers who are not happy. Even with these agents, I think there's going to be a big problem, both for the synchronous agent on your machine and the asynchronous agent that runs in the cloud. I don't want to watch an agent do its work for 15 minutes. What I do is start it; if it doesn't have any questions, it goes running, and then I go do something else.

8. The value of system design and human oversight

But we all know context switching is hard. How many of those agents can you really start and then go to another branch on your computer before this becomes more annoying and destroys your flow than actually helping? Even with that, how do you keep the developer in the flow state, and how do you keep the agents fast enough that it feels fun? I think that's going to describe a lot of the innovation that's going to happen in the next year, and that's why you hear many people in the industry saying that, with agents, what software development looks like will fundamentally change over the next 6, 12, maybe 24 months.

Yeah, it's funny. I guess from where I sit, it kind of feels like it's already changed. This is my own personal workflow, but I find myself using a lot more agentic code generation than tabs these days, which I think is a lot less fun, actually. I kind of like that you're talking about the fun of coding because the context switching is horrible, right? It goes from coding taking me into a flow state to coding meaning I have to fight my brain to stay focused while I wait for the thing to finish.

I think it's more efficient for me. You obviously have to support both uses, but do you look at how much people are doing in those 2 different ways of engaging? Am I the unusual one here, that I've moved to more of the longer pauses, or are most people just tab-completing?

Yes, most people are still mostly tab-completing. That's just the nature of things, right? You're not changing how the majority of developers work overnight. There's certainly, within the developer and AI code generation space, a whole different S-curve of early adopters of agent mode. I think that's still a smaller population than the large majority that had adopted code completion.

Let's face it, code completion was one of the magical things about the original Copilot. You didn't have to adopt anything. We could have probably announced this and not even mentioned AI, and just said, “Here's cooler code completion.” It's there, and if you don't like it, just ignore the ghost text. Or Command-Shift-P, disable Copilot is always fair game if you're annoyed by it as well, especially when you're working in a file where you might not even want the content of that file to go to model inference.

We see an adoption spectrum between code completion, which has the highest adoption; chat and inline chat; and all the way to agent mode. Agent mode, to some degree, is still something that developers have to learn, and there are scenarios where it makes sense to start vibe-coding with the agent versus just doing it yourself.

Tobi Lütke, the Shopify CEO, recently had this memo for the whole company saying AI is now a baseline expectation at Shopify. I think the most fun bullet in that memo was, “If you want more headcount, you first have to show that you can do it with AI,” and increase the productivity of the company. I think that shows you where the thought process is.

The thing with agent mode is that, if it takes too long, as a developer you're always going to have the question in your head: Could I have done that faster myself? I think that's a culture change and a mindset change that will keep going. As the models get faster and more efficient, and user behavior shows at what point you want to take over and say, “Okay, this is good enough. I can do it,” we are going to see a behavioral change across the population.

I actually think a lot of vibe coding is fun, in the sense that my hobby projects don't run any unit tests, and I don't put a pull request against myself either. What I really want is to have an idea, write some code, see it light up as an app in my dock or as a web page, and then feel good about it. I want to say, “Look, look what amazing thing I just created with just a bunch of lines of code and a computer.”

That was always the magic for me in software development. You didn't have to build a factory first, with tools and all that. You could get any computer—the earliest PCs or the Apple II—and they had a programming language. The operating system was BASIC, even on the Altair, even though that didn't have a keyboard or a monitor.

That dream is still there for many developers: I have an idea, and I want to implement something. Now, if you can do that and then spin off an agent in the backend to write you the unit tests, you don't even have to watch it. You can just write the next method, and in the meantime it has built you 15 unit tests for that one method, then runs them and fixes them itself.

9. How AI tools are changing developer workflows and culture

That happens in the background, and you don't have to watch it. You can just focus on the next method in your class. I think that's keeping you in the flow state because you're not going from the thing that is fun—which is really building features and functionality—to the thing that is usually not fun for software developers, which is writing unit tests, running them, waiting, and then seeing the last one fail and wondering, “What did I do wrong here, and why is this one not meeting the assertion?”

I think that's where we're going to see a lot of user interface innovation over the coming months: spinning off these agents, keeping the developer in the flow state, and letting them really do what they want to do.

Is that a core guiding principle for you, to make it fun? Because I would think a lot of CEOs would be like, “I don't really care if my developers are having fun. I want them to be productive here.”

Certainly, I'd say developer happiness is one of our principles. We definitely believe that if developers are not happy, they're not productive, and if they're not productive, they're not building amazing products. I think those are very strongly correlated with each other.

It goes both ways, right? If your toolchain is bad, that means your NSET or your NPS is going down, and your developers get grumpy about the toolchain and have to wait forever. We know that even at the scale of Microsoft, before I became the GitHub CEO, I ran a Microsoft engineering systems team called 1ES for some time.

One of our biggest projects was bringing down the build time for the Office suite from days down to—I think 8 hours is where we ultimately landed.

And 8 hours to build Office.

Yeah. I mean, 8 hours sounds like a lot, but if you're coming down from 3 days, you realize that 8 hours is actually good because it means you can start a build, take off for the night, and then have a fresh build the next morning. Whereas if it took days, it was so paralyzing.

So what happens if somebody breaks the build in some cross-functional way? Is it just down for a month while people figure out what went wrong, or how does that work?

Well, I don't know what the answer to that is because I was on the team.

But I mean, obviously, at the scale of an Office suite or Windows, you have many, many builds running every night, so one broken build isn't slowing everybody down. It's just slowing down that team on that branch or that feature workflow. They probably had to spend the day on it, but that's the latency problem at a really large scale, right? Your turnaround time is 8 hours.

We have that actually in model training now, right? Model-training runs for these really large frontier models take so long that if you make a mistake or there's a power outage and that interrupts the training run, you have wasted a lot of, ultimately, money in the form of energy and resources. That is now wasted time. So keeping these supercomputers or clusters up and running to do the model training is, in itself, a crucial part of training frontier models.

Coming back to this Office example, the goal there was just to bring the build time down, which means you have to parallelize the builds over multiple machines. That then raises the question: How do you do that with dependencies? Where do the object files go? What artifact storage do you have that's fast enough so that storing and uploading the file into an artifact store, and then downloading it on another machine for that next object-file compilation, isn't actually slowing down the build process? I think it's these kinds of optimization problems that we're going to see in a new form in AI, in this AI-native workflow.

The funny thing is that “AI-native” is a misnomer, right? It's AI-first or AI-infused, but nobody is actually AI-native yet. We all learned coding without AI, right? So we might be cloud native. We grew up with the cloud, although I didn't—I think you didn't either—but certainly it became a whole thing, and then you can become a cloud-native startup.

The first really AI-native startups, I think, have yet to come, where it's truly, truly AI-native. In 10 years, we'll look back and say, “Yeah, this is when things fundamentally changed, and our companies are just built in a whole different way.”

A conversation that I have all the time, and I have a feeling you might have this too, is talking to executives who want to push their teams to use more of these tools—kind of, use more code automation. It always seems really baffling to me. I feel like, if it's obviously more productive, wouldn't people adopt it? I'm not going to tell my engineers what tools to use, but I'm going to be frustrated if they're not as productive as they could be. Do you come across this? I scratch my head when an engineer wouldn't adopt a tool that would make them 50% more productive. It just seems ridiculous to me, but I'm curious what your experience is. Do you push using tools internally to try to get your team to be more AI-native?

We adopted the Shopify memo in some modified form within GitHub and made similar statements at GitHub: The usage of Copilot and AI tools is part of our culture and mandatory where it makes sense. Obviously, we don't want a salesperson to use Copilot with the same intensity as an engineer does, but there are many other AI tools that they should be using as part of their workflow.

That's where I think we're past the point where that is actually a question, in the same way that at GitHub, it is part of every employee's job to use Git and GitHub. That's not optional. You cannot say, “Okay, I store my source code on Azure Blob Storage or Dropbox or anything like that.” You're laughing, but we also know from telemetry that that is still very prevalent in software development, especially in hobby projects, student projects, or companies that are just doing something else in software development. It's somewhat of a corner case of what they do day in and day out.

10. Teaching kids to code with AI and early software memories

While Git adoption has certainly grown significantly over the lifetime of GitHub, there's still a market to capture for us just on the platform and by moving companies into Git. Nobody questions whether employees at GitHub should be using Git. I think we're at the point where that is also true for Copilot and AI tools in general.

But that doesn't mean that that needs to be 100% of the day. You still need to think about when it makes sense and when it doesn't make sense. That decision-making—do you hit Control-Space on your Mac to open ChatGPT's input pane and type in a question, or do you Google it?—feels somewhat more natural now. But it is not binary; there is a gray area. We could do both, and often it's driven by your belief system.

The question you ask—why do developers not use AI even though the metrics are clear?—is partly about a belief system and kind of like, “This is how I have always worked.” Now that I have these autocompletions or agent mode, I'm forced into a way of working that I don't actually like. Now I have to read more code and understand it than write code, and I have to write prompts, but I don't really want to write prompts.

It's this culture change and, ultimately, the change in how we do things as humanity, that we know takes time. It's just not done by forcing people into this new way of working. In fact, I think what we're going to see is a bit of a shift in the performance curve of the employee population.

When you do performance rankings or evaluations and those kinds of things, the employees who were good maybe 5 years ago in that way of working may not be at the top of their cohort or class in that new way. I'm a big F1 fan, and in Formula 1, the way this always works is that there are new regulations. For the first 2 years, 1 team has figured them out early, so they're ahead of everybody else. Red Bull, under the current regulations, had it figured out early, and then the teams get closer and closer to each other. You have really, really tight races and a fantastic season like 2021.

Then they change the regulations, and now there's a new way of driving these cars. All of a sudden, a different team and a different driver can handle this new type of car. Think about the current era's ground effect, or the V8 era, where the diffuser was connected to the exhaust, and whatnot, right? There are different drivers who handle those conditions better, and then they win the championship.

I think the same is true for developers in this new world of AI. You've got to be able to adapt, learn, and be willing to evolve your mindset into what is now the state of the art in software development.

Totally. I totally agree with that. Do you also feel like the skills that someone would need to be a good developer are changing? I feel like maybe a more product-oriented mindset would probably help in a world where the actual implementation gets easier.

The technical details have gone away, right? In the 1990s, you had to know about the CPU, the cache, the memory, and the bus, if you even had a hard drive or cassette tape. You had to really understand the intricacies of the hardware architecture you were building for, and that led to things like the 386 having a Turbo switch. The Turbo switch wasn't to make things faster; it was to slow down the software that was so optimized for the architecture that it ran way too fast and you couldn't play the games anymore, right? So you slowed things down because everything was so optimized.

Those developers who know exactly how the PC works and the instruction set of the CPU, and maybe write an assembler function to run a certain part faster, are a minority of the whole developer ecosystem. It's similar for developers who can write shaders in computer games: There's certainly demand for that, but the majority of full-stack developers don't have that skill. So I think we've always moved up the stack and left something behind intentionally, because the complexity has grown so much.

The skill that I think the future developer needs, if not the current developer already, is systems thinking: taking the problem that you're getting from your product manager or your boss, as your engineering manager, and breaking it down into a small enough chunk that you have a way of taking that abstract problem and converting it into code. That ultimately is what software developers do today.

You take a description of a feature and then, in your head or with a pen and paper, you break it down into small pieces of work that you can do during your workday. I think that thinking doesn't go away, but it moves to an even higher layer. Today, if you go into Copilot, ChatGPT, Bolt, or whatever, and you tell it to build your GitHub, it probably gives you a page that looks like GitHub.

Maybe you should try that after the podcast, but it certainly doesn't give you all the features, and it certainly doesn't give you the scale of hosting more than 500 million repos. Let alone, it doesn't give you the Git stack, or what we call Git systems, and all the pieces that are part of that application. That goes back to AGI in 5 years: at the point where that works with just AI, I doubt it, because designing a system like GitHub, Weights & Biases, CoreWeave, or Vercel still requires engineering skills and architecture. That's why I like to summarize it as systems thinking.

11. Code literacy as a fundamental skill

The other side is this notion of a full-stack builder, where you no longer have a PM, a designer, and an engineer. Testers are already gone for most companies anyway, and SRE has merged with software development as part of DevOps roles. But basically, you have a single person who can take an idea and work with one model or agent to mock out the spec and the designs, create the Figmas—probably using some form of design system. The one that we have at GitHub is called Primer, because then the AI can compose these components into a page.

Then you take it into the business logic and application logic with an agent, write all the test cases, write your GitHub Actions deploy script, deploy it to the cloud, and have an SRE agent monitor this, scale it up and down, look at exceptions, maybe take those exceptions and file a GitHub issue, and then take an agent to implement that fix. I think that full-stack builder, if it doesn't already exist in some small startups or very innovative companies, is soon going to become a role in a company.

If you're out there, I want to work with you. That sounds like an amazing role.

What you're going to have is a Venn diagram between what a PM is doing, what an engineer is doing, and what a designer is doing. Even a marketer is going to have a much higher overlap.

A really good PM who can write a really good GitHub issue describing what a feature should look like, and hit exactly the right abstraction level to then assign it to Copilot, can have Copilot spin up GitHub Actions and use tools, MCP, and all these kinds of pieces to implement that feature and create a pull request. Then you review that pull request, and that's where the engineer might come in and say, “Yep, what the PM created here is good.”

But you might have the PM basically building the whole code change—the diff—in the form of a pull request with the help of an agent. The engineer becomes much more of a conductor of an orchestra of agents, together with the coworkers in the company.

I do think—and I'm not the only one predicting this—that we're going to see very small companies that have very high valuations. In fact, with WhatsApp, Instagram, and others, we've already seen examples of that before AI, but we're certainly going to see more of that. I think that's super exciting. You can build a business in your garage all over again, at a much bigger scale.

Yeah, because not everything is—I mean, I think, first of all, making something as advanced as GitHub is kind of on the outer edge of what you need to do, engineering-wise, to make something useful. These systems are getting better, right? I don't know about you, but I think we're about the same age. I remember programming really basic computers and just being excited that I could make a circle grow in size or something.

12. Small teams, big products: what the future of dev might look like

It's funny to watch my daughter at age 5. We vibe-code together, doing the same thing. It's the same mindset I had as a little kid doing it with my dad. She could make a real web app, and kids have such weird, crazy, creative ideas. I feel like when teenagers, and even younger kids, can start deploying and really realizing their own ideas, we're going to get a real Cambrian explosion of applications.

Yeah, I mean, absolutely. By the way, I was born in 1978, so, yeah, similar. I learned coding first in East Germany on a Robotron computer in the geography lab in the late ’80s. It was a clone with a Z80 chip, I think, and a cassette tape and all that. In my head, I think it merges with my early experience on a Commodore 64, which I bought, I think, in 1991. That's where I really learned coding, because I had it at home connected to a TV.

When you say “drawing a circle,” in my head I'm thinking, well, even the Commodore 64 didn't come with a graphical user interface. It was all text-based, and then you had to switch into pixel mode. I don't think there was a predefined method to draw a circle, and you couldn't just download an open-source library either.

So I bought magazines and books. The listings in the magazines—I will not forget this—weren't actually listings; they were checksums. You had to mail stamps to the publisher to get a disk back with the program that decoded those checksums back into the code, because you could print the checksums almost like a QR code of the 1990s. You could print them much more densely than you would print code, right? So you would type those checksums into the Commodore 64 to decode them into the code, and then you had a library that you could use to draw a circle or do a side-scroller and those kinds of things.

My kids are 12, almost 13, and 10. Same thing: I actually bought them Copilot before we had the free tier that we have now. They would always come to my desk to show me their MacBook with their Python—mostly Pygame—stuff and say, “Here, I have a problem. Go and fix my bug.” I would say, “I'm talking to Lukas right now. Can we do that later?” Obviously, for them, the answer is no; this is way higher priority. Then you give them something like Copilot, and now they can highlight the code and ask questions or use agent mode.

Kids are so much faster at adopting that new way of thinking because it aligns so closely with how they discover the world and learn: by asking questions and asking more questions, right? The first time my kids used AI on their own, I think it was Adobe Firefly. They had seen it somewhere in school and came home and said, “Look, we have these puppy images.” I asked, “How did you get these puppy images?” They said, “Oh, we signed up for an Adobe account. You type a prompt and you get a puppy image, and you type another prompt and you get a better puppy image.”

They very quickly learned that if they didn't like the answer they got, they could ask another question, because that's exactly how they interact with their families, their parents, and their teachers. They keep asking questions, and if they don't like the answer you're giving them, they ask you another question.

But you have to combine that with getting coding and computer science into every school. On Monday, May 5, in partnership with Code.org, more than 250 CEOs signed an open letter to bring computer science education into more schools. You can have an agent that does powerful things, but if you don't understand what it actually created, if you cannot read the code, validate it, or fix bugs, you're actually giving up something that makes us human.

In the same way, we're teaching all kids physics and chemistry, and obviously art and science, literacy, and math. That doesn't mean you just become a physicist because you had physics in school. You're still picking your career path based on your interests, economic value, and those kinds of things. But I think it's fundamental for everyone on this planet to understand code and computer science, as these devices and agents will only play a stronger and more important role in our lives than they already do.

Well, that seems like a nice place to end, Thomas. I appreciate your time. This is a lot of fun.

Thank you. Thank you so much for having me. I agree, it was super fun and a great conversation.