MiniCPM5 - The 1B Cognitive Core?

Okay, so for a while now, Andre Karpathy has been arguing that the ideal small model isn't just something that's like a shrunken chatbot. What he believes we need is a cognitive core, and you can think of this as a very small model, ideally something around 1B, certainly not over a few billion parameters, that has stripped out most of the encyclopedic knowledge, keep the reasoning, the tool use, the ability to look things up instead of memorizing everything. Now, this is an idea that I agree with a lot. I've been working on language modeling since 2018, and even back in the early days, many of us thought it was kind of stupid to try and get a model to have all of the knowledge inside its weights. It was much better to just use external tools in some way. And generally, that's how things have played out. If we look at all the big proprietary models now, they all support tool use in a really big way. The challenge has been though, that the smaller models just aren't that good at being able to use tools and to do agentic applications. And while I don't think we're totally there yet for the cognitive core model, we're certainly seeing things go in that direction. A lot of the phone manufacturers are putting smaller models on their devices, and then customizing them for really specific use cases with Laura fine-tunes and just being able to swap out the different Laura weights on the same model. So, in this video, I want to look at one of the models that is a leading candidate for this kind of idea, a 1B model that can do tool use, that can run on device, and probably is just a small step in the direction that this organization is taking with their new series of models. All right, but before we get into the model and looking at using it with a mini harness, let me just tell you a little bit about today's sponsor. Okay, so today's sponsor is Evo Map. So, they've got a whole bunch of different features in here. They've got a marketplace where you can buy and sell skills. They've got a bounty board. Currently at the moment, they're running an open-source contributors challenge. If you just click up here, you can see that they've got this event in progress. And what this basically is is an open-source API grant program. So, to be part of it, you basically just need to verify with your GitHub account and add in a repo of skills. And you can see, depending on how many stars your repo has will determine how much credit you get. So, once you've added your repo to the actual system, you then become part of the leaderboard. And you get access to credits, which you can basically use to get tokens for proprietary models like Claude Opus 4.8, Gemini 3.1, and even OpenAI models like GPT-5.5. You also then have the benefit of if people start using skills that you share, you can actually earn more credits as well. So, if you want to find out the whole details, come and check out the guides and tutorials that they've got on their site, and be sure to check out their API grant program while it's still open. All right. So, the model that I'm talking about is MiniCPM-5. And this is from OpenBMB, the Tsinghua NLP Lab. And I talked about them a couple months ago when I did a video about their 4.6 model, which has vision capabilities as well. Now, this model doesn't have that, but it's gone in the kind of direction where it's really focusing in on the agentic stuff. And they're also focusing in on locking in this 1B size, which I think is really cool if you're going for this idea of the cognitive core and you want it to run not only on today's latest smartphones, but on smartphones from 2, 3, maybe even 4, 5 years ago. A 1B model is something that would be able to run on those devices. Now, this kind of size of model I've covered before in the past. Going back a long, long time, we had things like TinyLlama, which was a tiny 1, 1.5B model. We also had Meta release least of the 3.1 or 3.2 models as a really tiny version as well. And on top of that, we've seen people like Qwen who have recently released the Qwen 3.52B reasoning model. And interestingly, the MiniCPM-5 1B has actually beating that as a non-reasoning model. All right, so if we look at the specs in here. Now, of course, there are going to be other models out there that are better. You've got your things like your Gemma 4 82B, 84B. You've got the LFM models. Again, quite a bit bigger for those. And in some future videos, I will look at using those on device. But I thought it's just kind of interesting to see where we're at with a 1B dense model. What can you actually get done? Because this is the kind of model that can not only sit on a mobile phone, it can also sit in your browser. It can sit in tiny little apps that you run on your CPU, etc. And it certainly is a step in the right direction of the cognitive core model. All right, so if we break down the specs here, we've got a 1B dense model. It's actually a Llama style architecture, which is kind of interesting. It can go out to 128K context window, which is really kind of wild at this size, right? Now, it's Apache 2 license, and they've actually released multiple versions of the model as well. So, the first version up on Hugging Face is the base model. This is the raw base model just pre-trained only. It's been pre-trained on web data. And the cool thing there is they're actually telling us what the data sets are. If you want to go in and actually see what the ultra-fine web data set looks like, they've actually released that. You can go and look at that. If you want to go and see what their pre-training for math data looks like, you can go in and look at that as well. And that's just not something we're getting from a lot of the other open model providers out there, let alone the proprietary models. So, you have to point there, if you wanted to recreate this, a lot of the actual training recipe and data is all there for you to do this. Now, this brings me to the second model that they released, the 1B SFT, right? This is their supervised fine-tuned model. It's tuned on about 400 billion tokens, 200 billion tokens of deep thinking SFT, and 200 billion tokens of this hybrid SFT. Oh, and if you want to see what that kind of data looks like, you can come in here and have a look at that as well. And then lastly, they released the main version of this, which is the fully trained one. So, this is basically trained with supervised fine-tuning, also trained with reinforcement learning, and also trained with on-policy distillation. And I think the whole concept around on-policy distillation deserves a whole video of itself. So, I'm looking at doing that in depth in the near future. But, that whole RL and the on-policy distillation here, not only boosts the average scores that they're getting for math, code, instruction following, etc., but it's also actually cutting back on overly long responses. And that's something we've seen with some of the other small models is that while they might actually have reasoning and long chain of thought, it becomes excessively long, which really doesn't add to the quality of the final output. So, it seems what Open BMB here is going for is something that has the ability to think and to reason with via long chain of thought, but not just to get stuck in those kind of things. Now, if we look at their benchmarks in here, we can see that they're doing quite well against other models of their size. But, where I think some of the most interesting results for this really are coming from the artificial analysis side. So, if we look at their results, they talk about that not only is this doing really well for the number of parameters, etc. For me, the really interesting thing here is that when they point out that MiniCPM 5 is more token efficient than larger reasoning peers. And the way that they measured that is basically counting the number of tokens for a particular set of benchmarks that they have. So, this model's actually using 31 times fewer than Qwen 3.52B for the reasoning version, and even eight times fewer than the non-reasoning version. Now, it is using more than the old 4.6 model, but it's also scoring a lot more than that model as well. Another thing that looks really cool about this model is that when it's actually tested on the AA Omniscience benchmark. So, this is a benchmark that basically punishes any sort of hallucination. And it actually scores negatively versus if you just say you don't know, you just score zero. This model's scoring way better than models like that Qwen 0.8B model and the old model, which tend to score sort of very much in the negative range, meaning that they got a lot of things wrong. This model's only got a score of -1, which basically means if it hasn't been trained on that data set that the model is pretty good at knowing when it doesn't know the answer and that it shouldn't just make up an answer. That's also a thing that helps a lot with the model deciding to use different tool calls, function calling, etc. Now, the thing that I've been really interested in with this model is not just the model itself, but the kind of sort of mini harnesses that you can build around this model. That you don't want something huge and blowty. You want something that's small. You want something can run in a very small app, perhaps on device, etc. And there are already some good examples of this out there. So, you can see that OpenBMB actually call out themselves one particular developer who built an edge home harness with this in Rust. And you can see this is not trying to basically be the most intelligent thing out there. It's something that's taking this small 1B model and incorporating it into a smart home scenario. And I kind of feel this idea of basically using these small models in things before that just didn't have intelligence is what we're going to see a lot going forward. And while the obvious one is phones and apps on phones and stuff like that, my guess is that going forward you're going to see a lot of hardware have these kind of tiny little models in them. And for a lot of hardware it's going to need to get a lot smaller than 1B in here. But it allows you to basically take things and give them a layer of intelligence that they just didn't have before. Another nice little example of sort of a mini harness that's actually released by Open BNB themselves is this mini CPM desk pet. All the code for that is up on GitHub. We're going to have a look at that in here and have a play with it. Unfortunately, the license on this one is perhaps not as good as we would like, but at least we can get a sense of what it can do and how it could be used if you wanted to make something like this. So let's jump in and run some tests on the model and see how it all performs. Okay, so here is the desk pet actually running. You can see that I can drag it around the screen. It basically just lives on the screen. And then in the menu bar I can do chats with it. I can do set the settings, etc. I'm not sure what it could be really useful for in this case. It is more sort of like an amusement thing. But this is a full electron app and it's actually downloaded and running the GGUF version of the model in here. And you can then select different Laura models. If you want to basically have the base model, if you want to have a Laura fine-tune of it. I'm guessing this is to give it the personality in here. So while this is a fun novelty thing, you could imagine these things kind of taking off. Okay, so if we just use the standard sort of tests that I run a lot, you'll see things like it can turn out content one shot reasonably well for sort of standard stuff, though it really hasn't come close to doing a 5,000 word essay. And even when I run this multiple times, even if we get long chains of thought in the thinking, we're still only hitting sort of two, 3,000 tokens out for this kind of problem. The same is true for the puzzles or for some of the things like that. It will tend to use a lot more thinking in here, which is a good sign for that. But, you're definitely not guaranteed that you're going to get the right answer out for this kind of thing. Something like the SVG test, look, these ones I really don't expect it to do well. It's This requires a lot of generalization. It requires a lot of sort of understanding about putting SVGs. You can see it's done 5,000 tokens of thinking there, but if we preview this, we've kind of maybe got the sort of wheels of the bicycle, but we've got nothing else. Not expected that that's going to do well on a 1B model. The same is true for asking it to do an HTML page. It will The thinking is actually quite succinct here, and it will generate HTML. It looks like it's generating out reasonably decent HTML, but sort of basic. You can see that, okay, if we're looking at Dario's retreat here, we've kind of got a 1997 style website coming out. Again, totally not fair to expect a 1B model to be able to do that if it hasn't been trained for those kinds of tasks. So, let's jump into the agentic tasks and the evals just to sort of see how the model responds overall. Okay, so if we come into check this model out, you'll see a few things in here. So here I'm basically just doing a simple chat with it, and you can see even if I just ask it "Hi, how are you?" We've got a really long chain of thought, right? This is quite long for it to basically come to the conclusion that it should just assist me, etc. Okay, and you can see that while it's super fast, again we still got this reasonably long chain of thought. It doesn't seem to be responding to the system prompt as well. I can check that, but even if we try to basically set it in the actual conversation, it's not responsive like that. And you can kind of see this in here that okay, I've basically given it the name Jennifer three or four times in here now. It acknowledges that it could have a fake name, so again I gave it the name Jennifer, and then it will reason over that. But it doesn't just sort of adopt that. When I ask it "What's your name?" it says "I don't have a name like Jennifer. That's just a playful reference for the conversation." So it's not super great at instruction following for things like that. Where it becomes a lot more interesting though is if we start to look at the agentic or the tool use stuff. Now remember if we sort of looked at the spider graph that they had, they claim that this is quite good at agentic things. And I really have to continually preface that all of this we're talking about a 1B model in here. So what I'm going to do is I'm going to run through a bunch of tool call agentic tests in here. Let's go through and see how they go. We can see that okay, single tool calls, no problem at all. It understood that okay, that it should use the function get weather here. It's using that with the right argument being location. Again for calculator, looks like it's done that really well. It's then able to refer to the tool in its thinking, etc. Doing sort of repeated tool calls, it does quite well at that as well in in here. It looks like it's done a pretty good job at being able to look up multiple places, etc. And then if we want to do things like multi-step chain reasoning, it's done that one pretty well. The currency conversion one is pretty simple. Even for things like this searching and response, this is kind of like in some ways you could think of this as being sort of a mini rag where it's basically getting a query and a bunch of data back. It's doing that and then it's able to then fetch a page to actually go and do that. So even for these sort of basic tasks, it's doing quite well. Where it can get hit and miss in here are for things like the long-running jobs. So in this particular use case, it's gone through and it's passed for this including this doing out to sort of 12 tool calls in this case. But I'll run it again in a sec cuz you often find that these two are the ones that it will fail. As you see here, it basically passed the first long-running one, but the second one it has failed on this. And I found even if we test it with the long-running sort of agentic trajectories, this is where it will sort of fall into having problems with this kind of thing of where sometimes it will work and sometimes it won't. And this second one in particular is very common to fail. If I change the model to be the GLM 5.2, it's going to be a lot because we're not running it locally, but you'll see that the quality here is just going to be a lot better. It can go out for a lot longer with tool calls going through it and it has no problem with passing these. Of course, it's not a fair comparison to compare a 1B model against GLM 5.2. Just lastly, going back to the mini CPM 5 and running the no tool restraint. So, this is basically where we're just prompting it even though we've got tools there, we're telling it answer directly without calling a tool. And the model seems pretty good at that, right? That's a test that it seems to pass very well. So, if we come in and look at some of the evals and run some of the evals, we'll see things like this where GSM8K, the model appears to not do well, but usually that's because of really long chain of thought going through this. So, when you look at it, it's just that it's running out even with 16,000 tokens, it's basically either getting into loops or just getting stuck in this really long sorts of chains of thought and not actually getting to the final answer. So, of course, there are some where it actually gets things wrong, but if you look at it, most of the ones that it's failing on usually are things related to length in here. And then, the ones where it doesn't think for a long time, it tends to get to the right answer quite nicely in here. When we look at something like MMLU, we wouldn't expect the model to do really well, right? We're talking about a 1B model. We find the same issue that sometimes it will just get it wrong quite often just because it's running out of tokens that it gets into these sort of thought loops etc. where the reasoning just goes on and on and on probably way too much or you'll see where it gets into loops. So, here for example, you can see that it's getting into a loop of certain number of tokens and it just ends up repeating those forever. So, that's one of the kind of problems that you will see. That said, when it actually doesn't go overboard on token usage for the long chain of thought, it actually does quite well and you can see here this one is a good example of where it goes through and ends up getting the right answer. So, I should point out that the limiting of long chain of thought is not a problem that's super easy to fix even for the GPT models. Don't forget that OpenAI doesn't actually give you the real chain of thought for you to see. You're seeing a summarized version of that. And between sort of 5.0 to 5.4, even 5.5, one of the main goals that they've been focused on is being able to still get to the right answer or to get to a useful output but reducing the amount of chain of thought to actually get there. So, I'm not totally surprised that you're going to see this kind of thing in this small model. All right, so just to finish up, even though I've pointed out a bunch of things perhaps where this model is not perfect, for a 1B model, this is really amazing that we're starting to get these results where it can consistently act in an intelligent way and be able to do agentic tool calls and reasoning, etc. So, my guess is that Open BNB is not going to stop here. Hopefully, we will see a vision version of this model in the not too distant future. And we can already see that people are doing lots of fine-tunes of this, both full fine-tunes but also just LoRA fine-tune adapters like this where they're taking it and using it for a particular language, for a particular skill like stock analyst, etc. And because the model's so small, it's not going to take you forever to do fine-tunes of this. So, overall, if you're looking at doing something on-device where you need text-only, this really is a model that's worth checking out. Like I talked about earlier in the video, I'm going to go and look at some of the other models that are competing with this in future videos. And as always, if you found the video useful, please click like and subscribe and I will talk to you in the next video. Bye for now.