This Hermes Update Changes Everything...

summarized

TLDR

Hermes agent's latest update introduces a Ministry of Experts feature that uses a panel of models orchestrated by a lead model, leveraging caching to reduce costs while improving answer quality. Other major updates include computer control for GUI automation, a /learn skill that turns interactions into reusable skills, voice support via whisper and 11 Labs, and new integration options like iMessage and WhatsApp. The update also brings memory reliability improvements, background agents, native desktop app, and automation blueprints.

Key points

  • The Ministry of Experts (mixture of agents) uses an orchestrator model (e.g., Opus 4.8) with multiple reference models (e.g., DeepSeek V4, GLM 5.2, GPT 5.5) that each answer the question, and the orchestrator fuses their responses for higher accuracy.
  • Hermes uses a cache to avoid re-reading the full context for each reference model, making the committee approach much cheaper than running models individually.
  • Hermes can now control a computer (Mac, Windows, Linux) by opening browsers and pushing buttons, enabling automation for tasks without APIs or MCP.
  • The /learn skill allows Hermes to turn a conversation, folder, URL, or text into a persistent skill that triggers automatically when relevant, eliminating the need to re-explain tasks.
  • Voice interaction is available via whisper model or 11 Labs, allowing users to have dynamic spoken conversations with Hermes running in the background.
  • Hermes now integrates with iMessage and WhatsApp, and supports background agents that continue working after you kick off a task.
  • Memory reliability improved with atomic add/edit/remove operations, preventing half-saved states.
  • New automation blueprints simplify scheduling without needing to write cron expressions.

Tools mentioned

  • Hermes agent
  • Opus 4.8
  • DeepSeek V4 Pro
  • GLM 5.2
  • GPT 5.5
  • Open Router
  • ChatGPT subscription
  • 11 Labs
  • Whisper
  • Vanta

Techniques

  • Mixture of agents (Ministry of Experts)
  • Caching to reduce cost in multi-model inference
  • /learn skill creation from interactions
  • Computer control (GUI automation)
  • Voice interaction with speech-to-text and text-to-speech
  • Memory management with atomic operations

Takeaways

  • The Ministry of Experts dramatically improves answer quality by fusing multiple model perspectives at minimal extra cost thanks to caching.
  • Hermes can now automate any computer task visually, not just through APIs, expanding its practical use.
  • The /learn skill makes your agent permanently smarter without manual skill writing—just show it once.
  • Voice, iMessage, and WhatsApp integrations let you interact with Hermes wherever you work.
Transcript (captions)
Homie's agent just released its biggest update so far and it's going to improve the performance of your agent whilst also costing less and make it more accurate. And in this video, I'm going to cover the big four updates that you need to know about right now so you don't fall behind and you can take your Hermes agent to a completely new level. And if you're new, I'm Jack. I build and let tech startup with a gazillion customers now my own AI startups and I share here the stuff that actually works. So, if you haven't already, grab a beautiful coffee and let's dive straight in. Now, the first thing I'm going to talk about here is what we call the Ministry of Experts. What is it? You can see exactly the idea that we have an orchestrating agent and we have three experts that we're going to call upon. Let me show you exactly what's going on there. So, it can also be described as the fact that Hermes has grown a council and you can use this council inside Hermes agent. Here's a sneak pe preview of the stuff that we're going to cover. We got a lot. I hope you buckle in. I hope the coffee is ready. It's going to be a fun time. you're going to leave this video with a radically more powerful Hermes agent. It is freaking really cool. So, what is the idea of the mixture of agents? It's a framework that's actually existed, but Hermes does something unique with this that actually makes it slightly more costefficient than not using it anywhere else and it makes your final answers incredibly more powerful, which is why this is so important. So the idea here is that a few reference models each take a crack at your question and an aggregator reads all their answers and then writes the real response and runs the tool. So you could for example have Opus 4.8 as your aggregator and then you have these reference models underneath that all have a quotequote go at it and then Opus 4.8 will give you the final answer. And the reason why this is so important is that different models will make different mistakes. So then the final model C basically several different takes catches the blind spots that you might otherwise make by just using one model. Now why this is genuinely important apart from the fact that our quality goes up is that it actually uses something called the cache. So the idea here is that running a committee of models always used to be expensive homies fixed this one thing but it made it costly if you didn't do this. Now the cache helps us understand why this is so cool. So previously the cache essentially was all of the context that you would read for any message that you've sent. Now, Hermes uses this cash to make it cheaper. The TLDDR of it is that your whole context is read once and then it's reused for free, which means that previously when you would do this exercise yourself, everyone would have to reread the entirety of the context, which just which basically just costs more money. But what Permes has done, they've pinned this panel to the tail, so the cash never breaks. But TLDA is is basically a lot cheaper than doing it individually, which is why this is so cool and it makes your answers way better. So, as you can see down here, if you just use opus, this is the cost, so to speak. Mixture of agents and then spawning two agents by doing it yourself. The red is the old way. The blue is a current and the orange is opus. But what I'm going to show you in this video is this one here, which is if you use a subscription model and one, it is pennies. It is fractionally more expensive for what they claim is actually better quality than just using 4.8 by itself. Now, news have actually got their own benchmark they're releasing which is called Hermes Bench. And they're claiming that this system can get 8% over Opus 4.8, which is interesting and kind of does make sense if you're using Opus 4.8 as the orchestrator and you're just tagging in different brains to catch its errors. But let's see exactly how this works. So, I'm going to put my Hermes Agentic operating system and I'm going to come down, of course, to the Pantheon and I've got this Ministry of Expert section here. So, I'm going to simply select on that. And as you can see, I've got all of my different models here. And what I'm going to do is pick the one that I want to be the orchestrator. So, let's grab 4.8 and drop him in here. So, he can be orchestrated. And effectively, you can move these around. Now, crucially, the ones that I'm going to use here, and I think this is the best system that I found doing it. Of course, you can check out all the stats down here based on the model. So, you can see where they are in terms of speed, cost, um, the agent itself. And guys, I've also ranked them on Arena. Arena is a platform where basically the models blindly compete. So you basically rank them out of like do you think it was good or not and they compete against each other and it's a blind test. So number one is usually the model everyone prefers. So you can pick that data. Now I'm going to be going with Opus 4.8 is the orchestrator and then I'm going to tag in DeepSeek V4 Pro which you'll see down here. Let me go ahead and grab him. We got the flash down here and we got this guy. Look the cost on Deep Seek V4 is unbelievably low. It's 87 cents per million tokens. It's very quick and it's it's very very powerful. We got GLM 5.2 2, which is exceptional as well. Some say that rivals Opus 4.8. I had an amazing time with it. And then we've got chat GBT 5.5. So all you're going to do is going to come down and copy this. I'll also put a link down below so you know exactly the root you need to be using to get this done. And then we head over to Hermes agent. And then once we copy that, we can come over here and drop it in the Hermes agent chat or we can also go to Telegram if we prefer. Now, if you do want this to work, what I highly recommend you do is give it your open router API key. This basically lets it connect to every model GPT 5.2, Opus 4.8, Hey, you name it. Everything in one simple connection. The other one that you must have is your chat GPT subscription. For $20 a month, you get to code with GPT 5.5 and you get a ridiculous amount of usage. I don't know how someone is doing it to be honest with you. I don't know what the hell is going on there, but it's crazy. So, you want those two. And that's why I like to use 5.5 because it just makes it way more cost effective, especially if it's mirroring everything that Opus 4.8 is doing. Beautiful. So, now we are ready to rock and roll. You can also adjust your tokens if you want to. So, for example, it can be from around 4,000 to 16K. You'd want it to be less if you wanted a more succinct and direct answer and longer obviously if you're looking at different things. Now, these agents cannot spin up sub agents. It's just one model. So, let's give it a test by asking a question. Cool. I would like you to use my Ministry of Experts to answer the following question. Um, how does Hermes agent memory work? And just limit the tokens on this one to 4,000, please. Cool. And when that's done, I'd also just like to know a breakdown of how you derived that answer and if you took anything from any of the other models. Cool. I'm going to send that one off. And by the way, if this sounds like I'm speaking Spanish, I'm going to put a link down below for my brand new literal Hermes agent master class coming. It's going to include basically content I have never included YouTube. We start foundations. We go to setting up Hermes, powering Hermes, uh giving Hermes a memory that you've never seen before but never forgets, connecting Hermes to everything, power features, you name it. the operating system, all the stuff as well as the full Aentic operating system, all included. I'll put a link down below if you want to get access to that. The Claude Code one is already fully out. You have to check one out. It's pretty wild. So, I'm going to come back over now and as you can see, it's going ahead and actually doing that for us, giving us um some updates on the tokens. And now it's going to run all three models together at the same time. Beautiful. And our Hermes has come back with a full answer. And look at that. It's given us a full detail. So, here's how the answer was derived. The prompt found out parallel same prompt node tools and went to GBT 5.5 GLM and deepseek each proposal appended to tell the prompt are private context which is great and then opus 4.8 and eight wrote the final answer and honestly a lot three signals I use here's what they contributed and you can see the I mean you wouldn't ordinarily ask them to do that I've done it so that you can see just how they're all contributing like a kind of like a team right everyone's on a team everyone's helping out GLM chat GBT everybody hey everybody bring something to the herogenic table okay everybody's invited and this is really cool and look at this I did not invent the memory structure I synthesized it and this is really freaking cool this is exactly how you use this in a token efficient way to get the best answers. And the really cool thing is it doesn't even matter what the best model is. Why do I say that? Well, I say that because if you really think about it, right, if I come down, let's just have a look at the Ministry of Experts real quick. All we would do is this list updates dynamically. We would just say, "Cool. Hey, Opus 4.8, we've had a great run. We appreciate you, but it's time to move on. It's time to bring in someone new." And then we drop 4.3. And then if I just say, "Hey, do it." It runs the exact same thing. Obviously, there's just like a fraction of all the stuff we can do in the OS, but you get the idea. That's how we're going to leverage this Ministry of Experts. I mean GLM would be fine as an orchestrator cuz that again is insane and crushing it. It is roughly 16th the price for the same performance. So we want these two in the mix for sure. So this is a key to bear in mind but this is very different than some of the other models like Sakana Fugu because it's a committee not a router. So diminishive expert runs the same fixed panel every turn. It never looks your task picks. You choose the models by hand. Okay. Runs a fixed panel that we configure. Fuses every take. So it's quality optimal and it's insurance against model iron spots. Then the other cool update that we got fairly recently with this is the fact that Hermes can actually control your computer. This is really important because not everything that you use has an API or an MCP. So effectively anything we're doing on our computer as you can see this kind of bionic arm with a wing Hermes can do that for us. So we leave our laptop on if it's running at home if it's running virtually. This is one of the reasons why I really like running it on my own metal. I like having it in my computer AI lab. I think it's exciting. I think it's a future heading. But let's say that I go out and I forget something. Maybe there's a application or a form that you fulfill. Maybe it's a reservation for I don't know a meal you want to go with your fiance or something and they don't have the ability to do that. Well, we can use Hermes to open up the browser, push the buttons and do various different things. It's very very cool. It runs in the background so it doesn't actually take control of the computer runs different basically different operating systems like Mac OS, Windows, Linux, and it's very very cool. It can drag Windows. It can do lots of cool stuff. But this takes us onto the third skill that I really want to double click on because this one is really cool and it's the forward slash learn skill. So the idea here is that you can let's say that you and Hermes agent did something huge like let's say you cracked some ridiculous integration or you finally figured out how to do something. You can say hey Hermes go ahead and learn what I just told you. And then what Hermes will do is take everything that you did and turn that into a skill. Likewise we can give it URLs for things and do that. We can give it folders, URLs, it can be conversations, it can be text, anything we show it once and Hermes will commit this to it long-term memory create a skill for that and then in the future if we ever want to we basically if we ever ask it a question that triggers it. Hermes says, "Hey, this sounds a lot like a skill I learned recently. Kind of like cramming for a math test, but he just never forgets." And it will just pull it back up and do it for you. So you don't ever have to reexplain yourself. Meaning you get to say what is known as precious time. Okay. So, we're going to point the for/ learn at a folder URL of this conversation or pasted notes and it writes the skill itself. You never hand it a skill.md again. How freaking cool is it? It will source material with its own normal tools. Read it, search it, web extract. So, it works identically in the CLI, the iMessage gateway, the TUI and the dashboard, and we can go ahead and show you exactly how this works. So, let's take a random example of this get to reaper here, which is trending, which is about creating PowerPoint presentations. Who's creating PowerPoints in 2026? honestly don't know cuz we're all using HTML these days and can we come down here copy this code now check this out let's pull up Hermes agent and what we can do is do for slasharn if you don't see this just make sure you fully updated to the latest version of Hermes you'll need that for all the features I'm talking about here and say hey there I want you to go over to this repo here and I want you to learn it please all right and then we're going to go ahead and we're just going to paste that in and then Hermes will go ahead and learn a skill from what I've described honestly one of the things that I found is that more so than proactive I think the reactive is better especially when you say hey you know okay here's a good example Let's say that you are building a presentation for something or maybe you're trying to write an introduction for your Instagram video or something like that and you go back and forth. You often find yourself, don't you? We're like, man, that took a good effort, but I'm really happy with the with the outcome. Well, we can just say, hey, dude, learn it, create a skill from that. And I've been doing that in Claude code for ages. So, it's cool that Hermes is just like actually codified what that is. So, we never actually physically need to go ahead and ask it the same thing. Now, it goes ahead, it will ingest the source. We can do this proactively and reactively. super cool freaking skill. Now, the fourth skill I find amazing is the ability to actually talk to Hermes agent and it's so freaking cool. There's so many ways we can do this. We can connect to 11 Labs. One of the ways that I did this in my operating system actually is I went down and I used the whisper model. Now, the reason I used the whisp model is I I really like it. I think it's like very quick. As you can see, I pull this up. I've got my talk to her. But what we're going to do is full screen like so. And we can begin the conversation. And obviously, we can pick the any chat interface that we would like to. I could come down and just say, let's come down, begin a conversation. Hey, the Hermes, how's it going today? >> Hello there. I'm doing great, thank you. How about you? >> Okay, but you get the idea. It's coming from my It's coming from my Mac Mini. I've just recently moved my setup, so my speakers all over the place. It's absolute carnage, but it's great. We can have these dynamic conversations. We can play around with the different interfaces to make sure it's responsive. I actually had this change color based on whether it was thinking, speaking, listening. It's very, very cool setup. But the fact that we can take it beyond just simply typing out is amazing. And you can have this running 24/7 in the background or maybe not 24/7 cuz we got to sleep, right? But you can have this running whilst you're doing other tasks, right? And how cool is that? It can just be in chill mode so that I can just tag it in whenever I want to begin a conversation. Let's say that we're working on a project together, right? Let's say that we're doing something random. Well, we can actually just ask Homies questions whenever we want to, when it pops up. It's that kind of convenient. That's how helpful I think it is to have this voice feature. Or maybe you just want to work through something and you just want to yap and talk to it. And when you're building apps like this, I did have some questions from people in the community about, well, how do I get this to share with corporations? How do I get to like big business and enterprise? So, with our startup Glider, for example, um obviously it's super important that you do this is the speech to text thing that I've been using. We absolutely are focusing massively on the GDPR thing. Now, anyone who ask me for recommendations, the company that I use is Vanta. They are super cool. Basically they try to automate your compliance side of things. So whether you want to do GDPR HIPPA compliance if you are looking at like working with an agentic operating system with Hermes or you're working with Hermes in businesses. I've actually quite a lot of enterprise people and businesses reach out to me to help with Hermes. I can tell you right now you need to have this stuff on lockdown. Like whether it's GDPR, HIPPA compliance, you name it, you want to make sure that you've got it. So I actually spoke to the Vant team. I said guys we freaking love you. We want to work with you. They agreed to sponsor this this section of the video. So, big thanks Savant for that. So, if you're looking to use kind of compliance, get us locked in. They work with many companies that you probably recognize. They've agreed to give you I think it's like um $1,000 off when you use it. I'll put a link down below for that. I didn't get $1,000 off by the way. I just paid full price. I'll just tell the stuff that I use. I think they're really freaking cool. And if you are doing enterprise stuff or for our business cloud, for example, when we are working with lots of like, you know, big companies and stuff, it's really important. Like they won't even like have the conversation unless you're fully locked in. And it's super important that you have that stuff. So, just wanted to share that. if you are building it, it put it on your radar. It's super duper important. And then that leads us nicely onto the next section, which is some quickfire updates that you might find helpful. So let's go over here. iMessage and WhatsApp. You can now chat to Hermes in iMessage. Very freaking cool. It's also very quick as well, which is decent. The memory has gotten more reliable. So adds, edits, and removes memories in one atomic, right? No more half-saved inconsistent state. I do also want to point out to you the fact that the idea of Hermes is it's meeting you where work happens, right? iMessage, WhatsApp, that's the direction of travel and they're moving pretty quickly with it. We got background agents. So, kick off a long research or build task and they will keep working for you consistently in the background. It turns weight in the background. I could have done examples for all this stuff, but I just out of respect for your time. I don't think all these need like five or 10 minutes on it. So, I just want to put it on your radar just so you know that it exists whilst giving you, you know, keeping you fully up to date with them. Then we've got native desktop app. As you know, you can get like a Homie's got like a desktop app. We can do a few things. It's basically like um a slightly more userfriendly version of the terminal. Uh you can build profiles now which is great. They've even got pets which is very freaking cool. And you can schedule without the cron with the auto automation blueprints which basically demystifies this idea of cron like cron's such like a nerd nerd term. Sorry if anyone like loves cron but it just is. People don't know what that means. It's got a very kind of like funny background. So you can just say hey automate this thing and Hermes understands that. So leveling up your Hermes agent, look, it's good, but if you don't know every level of Hermes agent and what the biggest limitation at your level is, you're leaving too much value on the table. So, the next thing that you and I need to do is learn those levels and how to get to the next level by watching this video right

Jobs for this video

Jobs for this video
Stage Status Attempts Last error Updated
summarize done 0 2026-06-30 22:05:25.688870+00:00
transcript done 0 2026-06-30 22:04:39.514162+00:00
metadata done 0 2026-06-30 22:04:21.440951+00:00

Frontier Notes · by Hyperjump Technology