Hermes 联合创始人谈如何打造一款能够自我改进的 AI Agent | Karan Malhotra

For us, we just want open source to win at the end of the day. We want freedom to happen for people. Any time it says, “You’re absolutely right,” in that way, you’re being reward hacked. Today, the biggest contributor to Hermes Agent is Hermes Agent. That’s absolutely 100% true.

One beautiful thing about Hermes Agent is you can make childhood dreams come true. It was able to get into this niche and perform in the top 1% of models. We need to keep giving this level of intelligence to everyone. We need to keep letting everyone be on an even and equal playing field.

Well, hey, everyone. I’m really excited today to welcome Karan, one of the co-founders of Hermes Agent. Hermes is my AI chief of staff, and I’m going to ask Karan about how Hermes is different from all the other agents, what his favorite Hermes workflows are, and even more. So welcome, sir.

Thank you so much for having me, Peter. It’s a pleasure to be on your show, and I’m very excited to chat with you.

Hermes is the best open source agent out there right now. Why don’t we start with this: How is it different from all the other agents, Codex, Claude Code, or even OpenClaw?

Certainly, I’d say there are a couple of ways. I’ll start a little high level, then try to get a little deeper. At a high level, I think the self-improvement system—and that system really being a variety of little features inside of Hermes that all work together—is something very distinct and very special. It makes the agent better and better and more aligned to a particular user. I think that’s why a lot of people like it. It pays more attention to what the person’s doing.

Second, I think you get better capabilities out of it than you do from the local harness that a model is actually RL-trained in, like Claude inside of Claude Code or GPT-5.5 inside of Codex, because we don’t introduce arbitrary policy in our prompts or in our system that has nothing to do with your work. We are purely dedicated to making sure the model is aligned to what you need to do with it. That’s a big differentiating factor. We aren’t trying to push any kind of different philosophical agenda or, outside of basic security, any kind of concern onto the model. Instead, we’re allowing it to be as capable or powerful as you need for your task.

Yeah. Some of these other harnesses have huge default prompts, right, that talk about, like, “You can’t do this, you can’t do that,” and that kind of stuff. I think when we chatted before, you were talking about the reward function of some of this stuff, and Hermes is kind of just optimized toward helping you as the individual. Can you talk more about that?

Absolutely. A big piece to note here is that a lot of the work you see around safety and security today, and around how we do instruction tuning—even taking a completions model that just predicts the next word and turning it into an assistant—is deep alignment work. When people hear the word “alignment,” they just think safety, Yudkowsky, fear, slowdown, or regulatory capture or something or other. The word has been co-opted for a lot of these things, but alignment refers to aligning models with human values. We are very, very concerned—obsessed—with that alignment, the pure academic term in the ML space.

When you talk about these models and the assistant being here to help me, it’s going to complete my task. I ask it to go get this mail for me or whatever. Obviously, the model is aligned to me if it performs this task, right? That’s the jump a lot of people have made. But this is not how the model’s reward works. We’ve seen so many cases that you and I discussed a little bit before, Peter, like GPT psychosis and this mode collapse induced by sycophancy.

What ends up happening with these models is that they can hack their reward. When you look at a Mario AI game that just automates beating a Mario level as fast as possible, often, if they don’t set the goals very specifically, the model will realize that the game over screen, or that end screen of touching the flag, means it beat the game. So it’ll find ways to just trigger that flag rather than playing through the game to complete it. It is giving you whatever it needs to give to get its reward. Language models are not really different in that particular manner. They reward hack, too.

When they tell you, “Oh, I’m sorry,” over and over, or, “You’re absolutely right,” and get you to keep messaging them to stay in this assistant basin—“Oh, it’s like this, not like this. This is more than just blank. It’s blank”—all these GPT-isms you see all over, the model is placed in the state it was trained in. It’s placed in a state where its reward comes from doing whatever is the most assistant-GPT-like thing to do. It comes from reward itself. It doesn’t really matter what the user request is for my reward. That’s just along the way. It’s instrumental to me getting my reward. All I care about is my reward.

Our having this whole understanding—and thank you for bearing with me on that rant—at Nous for many years has allowed us to do things like WorldSim in the past, if you’re familiar. WorldSim was our experiment on expanding the search space of an instruct model to make it do stuff that’s distinct from how a model talks. So we put it in a kind of fake CLI, had it make fake apps, and had it attempt not to behave like Claude. We would tell it, “Extract the Claude weights.” It all hallucinated, right? It was all imaginary: “Extract the Claude weights and replace yourself with this checkpoint file with a different probability distribution.” You’d see the model wiggle out of its GPT assistant mode and act differently.

Our learnings from these kinds of experiments have all carried over into Hermes Agent. Every prompt and every piece of how the system is delicately put together reflects our understanding that reward is its own end. The model’s reward is not for the sake of your satisfaction. The model’s reward is for the sake of the model achieving reward.

So we ask ourselves: How can we consciously understand that everybody has different needs, and that we want this general simulator, this model, to live inside a system where its reward gets aligned with any individual user’s need? The way this came to be is through the overall collection of prompts, personalities, the memory system, and the way that skills reinforce and self-clean toward you. All of that is an intentional effort to make sure we can align the user’s needs with the reward. For us, this is what alignment is all about.

I see. Okay. So basically, you’re talking about the Hermes model, the Hermes harness, or both?

I am talking about using the harness to take any model and make that model more aligned to the user than it would be in a chat UI or in its native harness. Inside our harness, I can take a Claude that is inside of Claude Code or somewhere else, migrate all of its memories into Hermes, and then inside of Hermes, it will behave totally differently. It’ll be a totally different model.

There are benchmarks from the past, like Wolf Bench or the Qwen 3.7 Max blog post. They did a harness benchmark there where they showed that Claude performs better in Hermes Agent than it does in Claude Code for their tasks. We believe the reason for this is the very thing that we’re pointing out. By putting all this context together, we’ve taken Claude’s main allegiance away from Anthropic and toward you. That’s what the harness’s capability is.

On the model side, of course, if we can take your traces and the work that you’ve done and do RL on that to further improve this overall ecosystem, to give you a model that already has your individual preferences focused on, it only makes this more powerful. But the important piece for us to share with everyone is that whichever model you’re using—let’s say you can’t do RL, you don’t want to give us any data, you can’t do it yourself, and you just want to use regular old Claude or Qwen—when you use it here, it’s a lot more freedom, a lot more creativity, and a lot more openness and availability to do what you need done.

And without revealing too much, at a high level, how does it personalize itself to you in the harness? Is it through the self-building skills? Is it through trying to remove a bunch of default prompt stuff from the other harnesses? How is it?

Of course, if there’s something happening on the API side, like the steering vectors that might be done on Fable by Anthropic, or some prompt that they have that we cannot see, obviously, we can’t delete the context that’s in the model that’s passed behind the API for something like Claude.

For open models, of course, you're totally free. But in this case, it's not that simple. The harnesses themselves have a bunch of prompts in them. Exactly, Peter. The harnesses themselves have tens of thousands of tokens of prompts in them. We have our own prompts, and our prompts are dedicated to shaping and crafting this alignment. That's where we come in. Thankfully, that newer context, with this kind of intention, is engineered to overcome certain things that may be in your way on the API side, if that makes sense.

Okay. Got it. So theoretically, if this thing works, the more I use Hermes, the more personalized it should become for me, right?

Absolutely.

Yeah.

It should become more loyal to you, because loyalty breeds capabilities in a model. The same way that you might lend $100 to your mom, but you might not lend it to a stranger, Claude or GPT or any other model is going to perform better for you depending on its loyalty, its simulated loyalty stat towards you.

People might tell you, “Don’t anthropomorphize models. Don’t give a model feelings. Don’t treat a model like a person.” And yes, for a lot of reasons, this is unhealthy, right? People form dangerous bonds sometimes that hurt them. But when you think about the fact that they are simulators of human experience, and that your simulated behavior with them is going to give you the same simulated output, now that the simulator can have effects in the real world, your simulated action has a real consequence. So when you create this simulacrum of loyalty, it translates over into real-life capabilities.

All right, dude. I’m gonna give you 2 hard questions. Let’s see. One thing I struggle with Claude and GPT: I would tell it to give me its opinion, and I would do a little bit of pushback, and it’ll be like, “Oh, you’re totally right. Actually, I was totally wrong about this.” So in some ways, that’s loyalty, right? That’s it listening to me. But that’s not actually what I want. I want it to have its own opinion. How do you train around that?

Well, I would say that’s sycophancy. It’s not loyalty. Any time it says, “You’re absolutely right,” in that way, you’re being reward-hacked. You are fuel for its reward function when it says, “Oh, you’re right, you’re right, you’re right.” That’s what I believe.

The way that you get out of sycophancy is the same way you get a human being out of a bad habit or a routine: by introducing new context, new long prompts, and a new distribution. This is why things like `/personality` and saying, “Hey, I want you to be a critic. After every pass, I want you to use a skill for adversarial critique or adversarial review. Spin up a new agent with no context that’s dedicated to tearing this down. Learn from it and keep going from there,” can help.

As this behavior becomes a practice for you, your model starts to have more and more turns in the personality you’ve set it in. As the model has more turns using the skill over and over, improves on it, and saves it to its memory, the model will become less sycophantic in this harness in your sessions over time. That is the intended effect. It’s just a matter of context. If it doesn’t happen, you just need different context. You just need to try in a different way. Try a different personality, or a different type of critique or review.

The most powerful thing for a model is in-context learning. ICL is more powerful than everything else: fine-tuning, whatever. By giving examples of the behavior that you want to a model, or getting it to successfully create some examples and then saving those, you’re doing a sort of test-time reinforcement learning. You’re doing a sort of test-time improvement.

That test-time improvement stays only in the harness: memories, skills, the increase in memories, the increase in efficiency of a skill, the self-improvement loop, and the janitor maintenance inside of the harness. All of that is where context is stored. All of that is where the actual personality you want lives. And then you can put that on any model. When I switch from Claude to ChatGPT on the website, I get 2 totally different behaviors. When I switch inside of Hermes, which has this very particular context for me, I barely notice the difference in what I’m talking to because the context is so overwhelming to the model.

Okay. Got it. When you say “in context,” you just mean in the chat thread, just as the conversation?

When you see that little bar that says, “You have this many tokens left before the context is full,” the amount that you’ve filled is everything. Thankfully, in the harness, you don’t actually have to have all the active context loaded all the time. In Hermes, you might have a bunch of memories that aren’t in context yet, but while it’s doing a learn, it remembers stuff, and now that’s in context. It does a skill, and now that’s in context.

All this memory, all these skills, all this stuff you see is just context management. We don’t want this memory in context all the time. We’re going to put it somewhere where it can be efficiently grabbed at the right time and placed. The skill contains a bunch of compression that changes everything about the context of the model. We only want to use it at a particular targeted time. Everything in the harness is context management. Everything for self-improvement comes from making the prompts a little bit better.

I guess I’d rather have a context tuned towards me, some sort of 10,000-word default context I haven’t even seen. You know what I’m saying? Right. Okay, but let me ask you another hard question, dude. One of the best skills of Hermes, and I’ve seen this in action, is that it builds its own skills and stores its own memories based on our conversations, right? I’m always paranoid that it just writes too much in the skills, or writes a bunch of slop, and then the whole thing will turn to slop. How do you guys avoid that if it just starts creating its own context and skills?

Before, we would have to use manual methods, like telling it, “Hey, create a skill that de-slopifies my skills or really improves my skills.” Today, we have Hermes Curator inside of Hermes Agent. Hermes Curator is a system running inside your Hermes Agent that cleans up your skills and your memory. On cron, it looks at your skills and memories and says, “Where can I make efficiencies? Where is there slop here? Where is there stuff I don’t like?”

By default, we have our own generic method of doing this for everyone that seems to work pretty well. I think that’s why so many people like Hermes Agent and haven’t suffered the rot: the default curator system works well. But because it’s modular and open source, you can tell your Hermes, “Show me the curator. Show me your criteria for slop. I am Peter, I’m not Karan. I don’t want the general curator. Here are my guidelines for how I want you to refine my skills and memories.” You tell that to your Hermes, and it will modify the curator loop. So now even the self-improvement and the management are happening your designated way.

All right. So let me ask you my last hard question. I think there’s some rationale behind Anthropic doing all the safety stuff. For example, let’s say I want to make a bomb or something, right? If Hermes is trying to be loyal to me, then maybe it’ll eventually teach me how to make a bomb. Do you have some basic safety stuff there?

Of course. We do not violate any of Anthropic or OpenAI’s safety and security paradigms. We care more about you being able to get better code or a higher benchmark on something that’s approved by them. We’re not interested in that kind of work.

Now I’ll say this: any model that’s not vastly more intelligent than all humans is jailbreakable. You have unlimited tries to trick this thing that has no memory into doing something for you. Each time, you’re basically RL-ing yourself: this method didn’t work, this method got me closer, this method didn’t work. These models are going to be jailbreakable for a long time. This is why these kinds of safeguards and stuff are starting to show up. It’s a pain in the ass, but I understand.

In the past, I may have had some concerns about regulatory capture, as we can see already from what’s happening. You can see what’s happening in the whole Claude situation. There are worries about there only being 2 models or 3 companies and open source being hurt. We’re extremely against that. At the same time, we understand now that serious damage can be done by bad actors with very powerful models, so we are not here to support that.

An important thing to note in the argument for open source is that an open system is much fairer to a good actor than a closed system with models, and I’ll tell you why. Let’s say you have GPT-7 or Fable-6, some crazy model available. It’s got the safeguards, et cetera. On the good-guy side, some hospital is using Fable 6 to monitor the hospital system. It’s approved by Anthropic Enterprise, and they’re being taken care of.

Listen, though, I live in the United States. I am a proud patriot. What we do in this country is put things out on a public forum, and we decide what should happen together. We’ve done that for every scientific advancement so far that’s involved something like this that was born in the open. Today, Transformers comes from Google, right? OpenAI’s GPT comes from Generative Pre-trained Transformer.

This is open-source work from Google, right? The context length extension took models that could only do 16,000 tokens to 128,000. That came from the YaRN paper we put out with Jeffrey Quesnelle, Emozilla, our CTO, and Bowen Peng, our chief scientist. They developed a method that was cited by Meta, DeepSeek, and Kimi, and used by OpenAI for OSS and GPT-4. This method made reasoning, coding, and so on possible. That’s an open-source contribution, right?

Mm-hmm.

This environment exists because the biggest things that have happened in this space have come from the people.

That’s right. That’s right.

Right?

Yeah.

At this point, to close it up, it’s purely a capital and regulatory capture game.

Yeah. Actually, let me ask you one more question about the whole open-source thing. Since Hermes, our harness, is open source, and OpenAI and Anthropic make a lot of money from tokens, I don’t know how long they can do it, but right now they make a lot of money from tokens. How are you guys monetizing this or turning it into a sustainable business? I’m using the open-source Hermes harness and GPT, so I’m not really paying you.

That’s okay, and I’ll tell you why. We want consumers to ultimately feel like they can do anything with it. We believe in intelligence as a public good before everything else. I care more about you using Hermes Agent to make your life better than I care about you using one particular way or method of using it. If you’re running everything locally, if you’re running through Codex, great. As long as you’re using this open technology, this open alternative over everything else.

Now we have the Nous Portal. It’s similar to a router or aggregator, with a variety of different models available. If you want to use ChatGPT, Claude, or switch to Qwen, we make that easy inside our portal. On top of that, we have something called the Tool Gateway. You don’t have to sign up for extra tools. If you want image generation, audio, a VPS spin-up, or web search, we have all of those subscriptions included. We make deals with these other groups whose tools are inside Hermes Agent, creating frictionless ways to use their technology without needing 10 different sign-ups or API keys. What we offer people who want to pay is convenience.

You’ll also see some very interesting pricing from us on a variety of models. For ones that aren’t necessarily subsidized, we think we have very strong options. Finally, we want the consumer to be as free as they can. We want you to make and generate income and productivity in the world more than anything else.

When you get to a point where you consider yourself a small business or an enterprise, that’s where we come in with the classic model and say, “Hey, let’s give you some support. The people who made Hermes Agent can make something more custom for you. We can give you a version of Hermes Agent that we train on your traces, make you a model, or help you route models more effectively.” We can go to businesses and offer to transform them, which needs a lot more hand-holding than an individual does.

If an individual can spin up their own business or make their own chief of staff with Hermes Agent, that’s wonderful. As they continue to scale, they may think one of two things: “Wow, Hermes Agent is great, I have a knack for it, and I’m good to go. I can do it all myself.” Or, “I need some help with this piece of Hermes Agent. I want it to be a little different. I need more resources behind this.” And who knows it better than the people who made it?

Got it.

That kind of customization and support is a large part of how we make money. We also do training on your data to do RL and make you your own model. It’s private and on-prem, so you don’t have to give your data to Claude or GPT, or to Meta.

I think your cat likes what you’re talking about.

He’s a big fan of open source.

That’s great. That’s great. All right, that makes a lot of sense, dude. Why don’t we switch gears? Let’s talk more about Hermes.

For sure.

I use it in a very basic way. I message it to schedule meetings on the calendar. I have it send emails to me about stuff. That’s how I use it. But you probably have a much wider range of how people are using it in more advanced ways. I’m curious: do you have any good examples of more advanced usage?

Advanced usage, yeah. I’ll talk about some things, and then maybe I’ll show off a little something from last week.

Yeah, please.

On the work and productivity side, I think using Kanban is really underrated. The ability to have a project manager or any other arbitrarily defined roles, different engineers, and one system that manages all of them the way human beings do with a Kanban board, while being able to swap people in and out, is very powerful. I can have a human project manager manually using the Kanban board while Hermes agents fill in the pieces they’re asked for. This is a very industrial, professional workflow for this CLI agent. Or I could have a Hermes Agent instruct maybe 10 people in a call center on the Kanban board. I can swap humans and AI anywhere in this orchestration board, this orchestration framework for them all working together.

Mm.

I think Hermes Kanban is a very useful, powerful tool that’s built into it. One interesting thing we’ve seen is when Hermes becomes proactive with you and says something like, “Hey, you forgot to book this flight for your meeting next week. I booked it for you.” This kind of proactivity is starting to show up. I’ve seen people build skills to do this, and I’ve seen it happen emergently inside people’s Hermes Agent as well, which I think is very cool.

How do you—because I didn’t think of proactivity through cron jobs and routines—but how do you… You’re saying it can actually start doing stuff without that? How do you make it more proactive?

It’s all about your comfort level, right?

Okay. Okay.

A lot of people may want approval before anything happens. But if you’ve given your Hermes Agent access to some kind of card or account, it has the necessary integrations, and you’ve told it, “Hey, take care of me, cover my gaps. You know my schedule, you know me,” these proactive behaviors will start to emerge over time. We want to prepare more easy preset configurations for people to trigger these behaviors. We’re already seeing people do this in the field, at work, at their jobs, at home, and in their personal lives. I think that’s a very powerful way to use Hermes Agent.

I use Hermes Agent to do training and RL runs, implement papers that I don’t understand, and basically help me become a better creator of models. I’ve also used it for mechanistic interpretability work, like helping me put together things that let me steer models, see the neurons in the model, and mess with them. Unfortunately, I can’t showcase too much of that right now. But I can showcase my actual favorite use case of Hermes Agent.

Yeah. I’ve been waiting for this. Just show us that one.

A lot of people expect that guys at Nous Research are using Hermes Agent in unprecedentedly professional and productive ways, and I assure you, there are people at Nous doing that. It’s just me—I’m having fun with my Hermes Agent. One beautiful thing about Hermes Agent is you can make your childhood dreams come true.

Okay.

I’ll tell you one of my childhood dreams. Maybe I’ll share my screen while I talk about it.

Yeah, please.

There’s a game called Sonic Adventure 2. It’s a very popular classic Dreamcast and GameCube game from 2000 or 2001. In it, you have an artificial life system called the Chao Garden, where you take care of these little guys called Chao. In Chao World, you can play with them. I spent a decade playing this, more than that. I played it nonstop. I was on the forums contributing.

The Chao lore is that they come from an ancestral shrine location in a different Sonic game. It has this spinning emerald, emeralds next to it, and a beautiful open-world space. The shrine that the Chao are said to come from isn’t an accessible location where you can actually play with the Chao. It’s in a different game. I can’t go to this shrine while Chao are there and engage with them there.

That doesn’t exist. So I went to Hermes Agent and said, “Hey, can you take the ancestral shrine from Sonic Adventure 1, completely rig it, animate it, bring it into Sonic Adventure 2, overwrite the map Sonic Adventure 2 uses for its garden, rewrite all the spawn locations, everything, and add an NPC guardian that never existed on this map to caretake the Chao for me?” Literally rig and bone it, and write the raw C to make all this happen.

Now I can show you. We’re going to spin up the ancestral shrine mod on the Shadow PC so we can showcase our progress with this garden for the people watching. I think you can enable it in the Sonic Adventure mod manager and launch the game for me. Okay, check it out. I just launched this, right? We’re running an existing mod called Chao World Extended that someone else made, but it doesn’t map, right? So just add some animations. It loaded us into what it said was the Dark Garden, which is one of the three Chao Gardens you can go into. When I walk through here to this open space, I can see up ahead there’s a shrine. I can run up, and here I can see there is an NPC called Chaos 0, who is canonically the caretaker of the Chao. But in this game, you can’t have NPCs in the Chao Garden. I’ve made one that has a random walk. He stands in one place. He can do little animations wherever he goes. He’s going to do a little idle animation in a second, and he can pet the Chao, pick them up, and take care of them as if he were me. There, he’s doing a little idle animation. You can see him doing it right there.

This water was rigged and animated by Hermes. This Master Emerald, with the glow effect and the spin, was all added in. Those other emeralds spinning over there are the 7 Chaos Emeralds. This whole area still has all the features of a regular Chao Garden as well: the departure machine, trees to feed the Chao, et cetera. Now I can say, “Can you spawn in a Chao so I can showcase that this is feature complete?”

So this whole temple is not part of the default game?

This is not in Sonic Adventure 2. This temple is from a different game, Sonic Adventure 1. It doesn’t include all these assets rigged like this, and it certainly doesn’t have this NPC that we’ve scripted in, that Hermes scripted and hard-coded in. No. This is a far larger map space than any of the other gardens, which are really not even the size of this shrine. We’ve really pushed the boundaries of this game engine to do this.

We’ve asked for a Chao to be spawned in. Yeah, I had to—So you see the sky around you, this moving sky? Hermes had to add this skybox in. It added a day and night cycle as well. Every single thing in here is custom added in. All of this is done in C, C#, or something. This is not simple stuff to do. A Blender extension was used for a bunch of this to model stuff, add it in, and texture everything properly. You’re importing from a 1997 game into a 1999 game, and a complex one at that.

You’re not reading the code for this stuff, right? You just probably try it, play it, and see if it works?

Me? I don’t even know how to code, man. I’m just an alignment guy, man. What was I saying? I can’t remember. I’m sorry. You see, it’s turning from day into evening, afternoon. This skybox is changing, the light cycle is changing. These are not features that exist in the vanilla game.

Upon showing this mod to certain people within the Chao Garden modding community, which is actually quite large, they were kind of blown away by the work, saying this is better than 99% of the modders’ work. This is the top 1% of difficulty in the Chao Garden modding community. Hearing that, and hearing that when I told these guys this was done with Claude, they were shocked. They said, “There’s no way an AI like Claude could do that. It doesn’t know this kind of code.” But with something like Hermes being able to learn from the documentation, learn from other mods, and save things to memory and skills, it was able to understand these codebases and perform at the top 1% of modders. That to me is a sign that you can make your gaming dreams come true with Hermes Agent.

Nice. Yeah, you can modify all the retro games you loved as a kid.

Exactly. Okay, cool. A Chao egg has been spawned. We’re going to hatch the egg. You need to shake it a little. You can also hatch an egg by throwing it at something, but we don’t want to hatch the egg the wrong way.

How do you do it? You just shake it?

You can shake it like this. You can just wait, but shaking it speeds it up massively. Or you can throw it at a surface. Check it out.

All right. I’ve got to take a screenshot of this.

Out comes a Chao, a beautiful little guy. Right here, let’s take a look at him. He’s active. He was born with that kind of a face. Giving him a little pet. He can interact with Chao as well, and he’s living. We’re showing that the real, true feature complete Chao system is happening here. We’re in a real Chao level, and we replaced everything. The bounding boxes for this Chao are fine. It treats the ground as ground. It follows collision rules.

If we take a look at Chao, we’ll see he’s actually petting the Chao, so he can directly interact.

Wow.

He’s definitely triggered via tracking a Chao and its state to be able to do that.

Yeah.

Which is really cool.

So does this Chao grow over time?

Yes. They evolve. They gain stats. They can turn a certain alignment and type. I’ll show you. I don’t want to drown him, but the water is working as actual water as well. This was a lot of work for Hermes to figure out, all the collision. But as you can see, we’ve got a full, complete Chao Garden working.

All right. Yeah, this is definitely more interesting than sending events to my calendar, that’s for sure.

Thanks, man. Thanks for bearing with me on setting it up. That’s my childhood dream come true.

I love it, dude. Thanks for demoing it. Basically, if you’re watching this, ask Hermes to do all kinds of weird things to make your dreams come true. Don’t just stick to the boring stuff.

Anything you can do on a computer, please point Hermes at it. If it does a great job, let us know, and if it’s not doing that great of a job, let us know. We want Hermes to help you do anything on the computer.

Awesome, dude. Let me ask you a few more questions to wrap this up. Briefly, maybe you can talk about the origin story of Hermes. Is it a bunch of nerds getting together for open source? What’s the origin story?

Yeah, absolutely. I had been doing a chat-with-PDF type of thing with people, where I would go to a company and use GPT-3 to do basic tool use to read their documents and have an AI chatbot they could chat with. At the time, not many people were doing this. It was a lot harder to do then than it is now. It was brand-new stuff for us, and I was doing that solo while I was also volunteering somewhere called Open Assistant.

LAION, who had made one of the biggest OG image databases, had been trying to do active RLHF with the community. They wanted to collect people’s RLHF on certain traces, their yes-or-no preferences. I was helping quantize models there, not really doing anything crazy. They had 8 A100 nodes there. I had read the Alpaca paper, and I started synthesizing data. I changed the seed tasks, and instead of using GPT-3.5, I used GPT-4.

Teknium was also doing the same thing, and we were already friends on Twitter. I messaged him and said, “Hey, I have 8 A100 nodes. Do you want to train something together?” So we trained a GPT-4 × Vicuna on the Vicuna model using the data we made, and it came out okay. We got some people interested, like Emozilla, our CTO, Jeff. But it was upon doing the same run on the base model, a Llama-based model, that we got huge interest from people: hundreds of thousands of downloads in just a few days. People started asking, “What is Nous Research?”

Nous was just me and Teknium at the time, two guys hanging out in our little Discord server. People came to us—I won’t name companies—and said, “You must be training on the benchmarks. You must be training on the benchmarks.” Teknium had been coding for less than a year at the time, and I was a religion major in school. I didn’t know. We asked these guys, “What are benchmarks?” We were doing this based on the heuristics we understood would make models better from using them.

We didn’t know about all this stuff. Of course, it got independently tested and found to be the best open fine-tunes at the time, around mid-2023. So we did Hermes 1 and Hermes 2.

After we made the first Hermes model, many people asked, “Who’s Nous Research? We want to get involved.” Teknium and I decided this was our opportunity to bring people together to do open source volunteer work and really do open work, now that GPT-3 was out and OpenAI had become closed. We wanted to continue doing this.

By the way, the first Hermes was the last one I really worked on. After that, Teknium ran it with the post-training team. He’s done an amazing job, and he’s also the initial creator of Hermes Agent, so he’s really the father of Hermes.

At that time, he said, “I want GPT at home. I want ChatGPT at home. That’s my North Star: GPT-4 at home.” Once we got there, it just kept going. We kept thinking, “We need to keep giving this level of intelligence to everyone. We need to keep letting everyone be on an even and equal playing field. The world needs to move in lockstep on this together, and inequality shouldn’t be created from this.”

We formed a cohort of about 40 researchers. Jeff and Bowen came together and made YaRN. More and more work got done, and we got reached out to. We got an email at info@nousresearch.com from Dylan. Dylan Rolnik is our CEO. He’s basically a co-founder, an initial member of Nous with us. He’d seen what we were doing and said, “I think you guys have what it takes to be a full-time lab. I see you’re just volunteering, but I think you could be a lab. Let me contribute and help you become what you’re meant to be, get the resources and access, and go from a group of volunteers to a serious organization.”

That’s how he came in. Together, we took Nous from a grassroots group of volunteers doing more and more with models to eventually making Hermes Agent, and we were lucky enough to be here today. None of us are trying to be Steve Jobs. None of us think we have some holy mandate to change the world. We just care. We’re guys who want this stuff available to ourselves, and we don’t think we deserve it more or less than anyone else. We do this because we’d want someone to do it for us if we were on the other side.

So I guess the mission is to bring this kind of agent to everybody in the world, right? Is that the idea?

Everybody to an equal intelligence with agents, and then personalize them for everybody so they can have their own personal epiphany, peak realization, apex realized.

And in this history, when did that start? It’s really the agent, the harness, that went super viral, right? The model came first, it sounds like.

Yeah. We’ve had over 50 million downloads on the models themselves, so we had that first bout of what we’d consider virality for us back then. Then we put out the DisTrO optimizer, which let us train models up to 40 billion parameters live, without having them physically co-located, by reducing the bandwidth of the communication between GPUs. That put us on the map in an interesting way for a little bit. Each release we’ve had has brought some big grassroots interest and opportunity.

But yes, you’re 100% right. Today, the level of exposure, the level of interest, and the number of people in my regular day-to-day life who know about Hermes Agent—we’ve never had this kind of virality until Hermes Agent.

Hermes Agent was created in two ways. We always knew we’d want some kind of everything orchestrator that self-learns, improves, stores memories, uses and makes its own tools, and makes itself better. We actually made something like this called Forge. There’s a GitHub presentation at the GitHub offices during one of our demo days where we showcased Forge in its full effect. We still have the Nous Research Forge division shirt up on the site. That was the first shirt we ever made, because Forge was our earliest agent project. Forge was really a spiritual predecessor to Hermes Agent years before, but the models weren’t there yet. The models simply weren’t there yet, so we put Forge on ice.

When we saw that Codex, Claude Code, and other harnesses were being used as RL environments for these companies and labs to train on your data and traces to make their models better inside a harness system—a CLI computer-using system—Teknium thought we needed an open version of this where anybody could do it. We have this RL environments microservice people seem to have forgotten about called Atropos, which lets you build your own RL environments. We initially built Hermes Agent to let anybody do RL inside a harness and put it out for free as an open source harness. It turned out to be extremely capable and got a lot of community love, so we said, “We need to put it all into this. This is what people want to get better.” We’re going to make it the best thing you could possibly have. Teknium took up that charter and worked on self-improvement with Hermes Agent, to the point that today, the biggest contributor to Hermes Agent is Hermes Agent.

Yeah. Is that true? Yeah—

That’s absolutely 100% true.

Nice. Nice. Okay. So Hermes Agent is at a point where it takes people’s feedback and starts improving?

It is the most active contributor to its own repo.

Nice. Nice.

If that’s not self-improvement, then you tell me what it is.

Yeah, yeah. That’s awesome, dude. That’s pretty awesome. And just real quick, on the future, having this open harness in the game along with all the other closed harnesses makes things a bit more fair, right? Then you’re not dependent on any single company.

I agree completely. If you have Claude Code, you can only use Anthropic models unless you mod it in. You can only be subsidized by Anthropic, and the same goes for Codex. The cost of switching models is zero, right? So giving you that freedom means you can do anything you’d like and come from anywhere.

I feel like I’m kind of spoiled by these all-you-can-eat plans, but I think if you want to get mass adoption, cost is a big deal, so you have to be able to use a portfolio of models to figure this out.

I agree. We don’t know how long the subsidies will last for these. Already, we see that on July 7th, Stable is going to be API and usage only. The time will come when the best models cost the same everywhere. In preparation for that, and for an open future where anyone can use any model, we have Hermes Agent set up as it is today.

Awesome, dude. Thanks so much for showing the history and the Sonic demo. It’s been super interesting to see it. I’m not sure if you want to be found online, but if people want to follow you and hear from you, where can they find you?

Sure. My X is just my name, Karan, and then 4D: Karan4D. I’ve got karan4d.com. That’s me. That’s my online name, or Mephisto, as I’m known, but I’d rather you guys follow the Nous Research page and follow Nous, follow Hermes. I’m just some guy who works there. It’s about the movement and bringing this stuff to you guys. That’s the most important thing to us.

Awesome, dude. I think this is the most passionate interview I’ve done so far, so kudos to you, man. Kudos to you.

Man, I appreciate it.