Stop AI Agent Hallucinations: 5 Techniques + Production Patterns - Elizabeth Fuentes, AWS

Hi, today we are going to talk about how to stop AI agent hallucinations with five techniques beyond the prompt. Each one is a code change, not a prompt change. Let's see. Every time your AI agent responds, you are paying for the words going in and the words coming out. And your bill, you will see those calling tokens. Yeah. And the more tokens you are sending, the more you pay. And if what you send is not quite right, too much, or missing something important, your AI agent start to hallucinate. There are five techniques to help reduce tokens waste, improve accuracy, and catch failure before you submit them. And each one is a code change, not a prompt change at all. So, let's see it. First, we have semantic tool selection. You filter which tool go into context on every call. The model only see what it needs for that specific query. Second, we have graph rack. For precise queries like aggregation, counts, multi-hop reasoning, and you replace the text retrieval with a structured graph query. The model gets a compute a verifiable answer, not a sample as rack do. Three, multi-agent validation. A second agent can check every response before it reaches the users. And four, neuro-symbolic guardians, you rule life in Python, not in the prompt, and the model cannot escape them. So, five, runtime guardians, because if you don't want to block, you can steer, and you don't you don't need to block everything. And when a rule fires, the agent self-correct and complete the task. No hard stop, no user advice. So, for each technique, I will show you the agent without it, and then with it, so you can compare. And all the demos uh I'm using a travel agent that I built using a Strands agent. And a Strands agent is a open-source agent framework that we maintain on AWS. And I am Elizabeth Fuentes Leon. I am a developer advocate for AWS. I'm focused on agentic application. And here, in this QR code, you will find everything that you will need to recreate all these techniques that I'm going to show you in a moment. So, let's get into Let's get into it. So, semantic tool selection, our travel agent has 29 tools. Flights, hotels, payments, weather, cancellations, all all our dummy tools are not like a travel agent for real, but every time that user sends a message, all the 29 tools description go into the context windows. The model reads all of them before deciding what to do. And if your agent has memory, the rows to every conversation adds more context that gets sent with every single message. And you pay for every single one of those tokens, whether the model ends up using the tool or not. To understand where those tokens come from, you need to see what a tool actually looks like to the model. In a Strands, you write a function with the tool decorator that's the tool, and a name, a description, and a docstring typed parameters. Then, Strands take that and generate a schema with name, description, parameters, and the schema is what goes into the context windows on every call. Each tool schema is about 17 or 200 tokens, depending of how many parameters it has. If our travel agent [snorts] has 29 tools, that adds up to somewhere around 3,000 tokens per call, just for the tool description. Before your message, before the response, every single call. By creating a tool database, we can filter the tools that the agent may need before the agent the symbol. With this filter, the model sees only three most relevant tools. Tokens usage drops from thousands to fewer than 300. Let me show you in the code. Here in my cure ID, this is the Let me clear all the outputs. So, first we need to install the requirements. Here is the requirement file. This is a Jupyter notebook because it's more simple to show everything, but there is one application that you can run if it is more comfortable for you. So, here are the requirements. I have the A strands agent and because I'm using open AI as a model invocation, I'm using the API from open AI. And I need a strands agent for open AI. You can use open AI strands agent with all our my god, almost all the model provider. And with a strand we Amazon better of course because we as AWS, we are the maintaining of this framework. And to use a Amazon better, you don't need to add the model provider. And because I need to embed in I need to create embeddings for my vector tools database, I'm using the sentence transformer. This is a super simple model that runs locally, so it's free. So, you want you can run this locally you computer without expense another model embedding. And you can use a strands with all llama too. So, if you have a local model, you can run everything in this in your computer for free without expense any tokens. And I use it files as my vector store, local, super simple. And well, right now I'm going to use a Neo4j. This is for other demo that I'm going to show you in a moment. And because I'm using some environments, I'm using Python of And let's see. So here I want to run this. Let me put it bigger. I know that you have it some problems. Bigger. Little more little less. And here, this is better. So. I style my requirement. I already did that. And I'm using OpenAI, so I need my my API. My API key. API key. And here, I invoke my strands. I need the agent because I want to be like agent that I'm using OpenAI. And I have a bunch of tool dummy tools here. I let I going to show you in a moment. And I want to use this because I have some um functions that I'm using to uh create the build the index for my vector store. And I need the search tools when I'm using the vector store, I put my query there. I search for the I search for the tool that I'm going to use using vector um um search for vectors. And I swap the tools. I going to show you that in a moment. So let me show you all my dummies tools. Where is the engage register here. Here I have all my tools. Uh this is for swap. Search, build index. And where are my tools? Shoo shoo shoo shoo shoo here. So here are all the dummy tools I created for this. You know, it's super. This is a demo, please. It's not something that you can use to go to production. No, please. And okay. So where is this this one? So I run this. And I build my semantic index. I already did that via Yeah. So, I have 29 tools here, 29 dummy tools, and I want to test this with a lot of different queries, and this is the ground truth. I have a ground truth because I know what is the best um the best tool to answer the question, so we can know if our agent is okay or not okay. So, let's run this. We have 90 queries and 29 tools. So, this is some helper functions. I don't care. Yeah, I care, but I don't want to explain you that. So, this is my agent, my traditional agent, where I'm putting all my 29 tools inside the agent. I have my model, some helper function. This is my agent. I only This is the way I create a agent using a strange agent. I put all my tools, and my system prompt is you are a travel assistant. You select correct tool to answer question. Oh my god, super. And the model. Let's run this. And this is whatever. This is some function to count the tokens because in inside this strange agent, you can you can count the tokens. You You can know how many tokens are using for in and out the the agent. This is some helper function, and yeah. This is Every time that I put a uh a prompt inside this agent, I'm only sending the prompt, and the agent respond. Then, I I use a new agent because it's a a fork. So, I'm not having a conversation with the agent. I only send a uh a question, and I receiving an answer. So, for each question, it is spend around 2,000 tokens. So, in and not always it give me the right answer. Here. So, yes, I have um I'm not so good at accuracy, and the average is 1 1,000 tokens. Now, let's use my new AI with the semantic approach. The thing The first thing that I do is for every single query, for every single prompt, I send the query to my search tools, and I going to receive the top two the top K three most relevant with more probability to response my query because this is a semantic search inside my vector store. And then I use that response. I going to select that name, and I going to put that tools inside my AI. So, I only going to use the three tools that this semantic search retrieve me for my AI, and I send that only tools. And this is my function helper, and that's it. So, I going to send the all the queries again the same. And for the first question, I have 4,000 tokens three blah blah blah. And yeah, we have a huge different because I don't send the all the 29 to tools in all the queries. So, yeah, bam bam. And okay, semantic memory. Yeah, okay, now come on. Finish. So, let's let's let's go to the next one. So, this engine is only sending a question, and I receive a response. I don't have any conversation with this engine. What happen if I start to have a conversations? If the engine remember me? So, we need to we need to send only the tools that this engine is going to use. Because if I give the all the tools I'm using in the conversation history, uh then I it's going to be in a moment that I want to have the 29 tools inside my engine. So, we are not resolving the we are not having a a um We are not resolving our problems in the engine when we having a conversation with the engine. So, we need to put the tools and then we need to remove the tools. And we can do that with the swap tools. And let's run this. And what swap tools I got run this, no. Let me Let me show you first the swap tools functions here the register. I delete this because with a strands in each invocation, you have complete control of the status because this is a a gentle loop and you can we use this a gentle loop how for because this is a gentle loop and you can do almost whatever you want. You can put and remove everything inside the loop with some few lines of code. So, let's see. Here we have the engine. We have the tool register through your through tool registers. It's something inside the engine status. So, we can clear the tools and that's it. So, in the next invocation, we can clear the all tools and we can add the new tools. Here, where is my I have a lot of things here. So, I run this. It's going to take a moment. >> And yes, we can see that in each invocation, my amount of talking is increasing. Why I have more? Because I'm sending the chat history as well. So, I send the tools that I need, only the tools that I need, and the chat history. So, that's why I can see that my amount of token is bigger and bigger. And the see, the accuracy is better. And yeah, we have more tokens. So, yes, probably we don't have the best accuracy here because this is a super demo with super dummies tools, and some queries are ambiguous on purpose because search for something, for something, check something. And the demo has a generic tools, dummy tools, with some similar name. And when all the 29 tools are visible, the model sometimes pick the wrong one. And with the filtering, those generic tools uh only appears in the query actually match them. So, you can run everything and you can test this. So, this is only that run only locally. But, what happen when you want to put this demo in production? You can of course, you can build a bigger, better store. I don't know, you can use Postgres. I think it's too much. But, we on AWS, we have Amazon Bedrock Agent Core. So, Amazon Bedrock Agent Core is a service dedicated only to agents in production. And in size of Agent Core, we have Agent Core Gateway. So, Agent Core Gateway Gateway allows you to build this index [clears throat] uh alone. So, you only have to say, "Hey, this is my tools." And Agent Core Gateway is going to build everything for you. And it can have the vector search inside. So, the routing layer is inside the Agent Core and it handles the tool selection automatically. So, you register your tools once and it find the right one for each request. It's the same principle but without infrastructure to manage. Now, let's see the next one. GraphRAG. You know, RAG is retrieval augmented generation. Is how agents access your own data. You take the user question, search your documents for the most similar content using vector search and pass what you find to the model. The model answer from that, it's work well for open question. Find me something about this topic. But there is a category of question where that breaks down. What is the average rating across all hotels in Paris? How many hotels have a pool? Vector search always returns something even when nothing is truly relevant. And the agent only sees the top end chunks of your all data at a time. It cannot aggregate, count, or traverse relationship across all the full data set. So, it estimates. And it present that estimates as a real fact, you know, a real answer. So, here we have graph you build a vector store and it retrieves three chunks from 300 documents and the model guesses. If we are using graph rack, you can run a query across all the and returns a compute results. Graph address this differently because instead of a retrieving text chunk, you can build a knowledge graph from the documents. Knows relationship and structured data. For the demo, I'm going to use Neo4j locally and the model is going to write a Cypher query to search it. Cypher query is Neo4j query language. It's similar to SQL. So, the graph run that query across all the data. And the model gets back a compute verified results, not a sample. And before running this demo, I'm going to start the dependents. Let me show you. Let me Let's go to the code. So, graph This is the notebook I'm going to share with you. So, we need to install the requirements again. What we have here, let me see. This is the first one, yeah. Requirements. I have I'm going to use OpenAI again. Let me just close this. I'm going to use Neo4j and we need Neo4j for graph rack. And for the engine that is using normal rack, we are going to be a super simple uh vector store in files and we are going to use again the sentence transformer. And let's see. I already have all right, I invoke [clears throat] my open AI. This is my tools. I I because I want to create some tools here, I want to use open AI and I use in graph database. And this is my I have my Neo4j locally. And let's run this. And this is something to check Neo4j. So, I'm building my uh vector store. This is my files and this is my Neo4j that I have here locally. I don't know if I can show you. I think not. And I have a tool, a normal tool created with a decorator. This is to search inside the my vector store. You see that's super I I have the query. I I create I I create the the embedding for my query and then I search inside the vector store and I have the query for the knowledge graph. So, here the model need to understand that to search in the query in the graph knowledge base, it need to build a Cypher query. So, I put that in the context because we already know how to build uh tools, right? I put that in the context. I have my driver to send the data to uh to connect with the vector store and to send the Cypher query that the model is going to create for me. And this is to read the results and that's it. That's the only tool that I need to search in my knowledge graph. Okay, this is my model open AI. This is my rag. My rag engine and uh this is my graph engine. I have two different engine to compare the results. Let's see. Well, I have a problem. Yeah, I'm ready. So, let's do this round the first test. Aggregation. Here, what is the question? What is the average guest rating across all hotels in Paris? So, meanwhile this engine is Oh, it's already done. So, what I have here I have the average guest rating across the hotels listed in Paris is Um it calculate, you know. When something When I'm using rag, it go to the to the vector store, it receive the n uh possible answers for this question. Because this is an aggregation, it's going to use the data that it receive to build a mathematical uh operation. This is the thing that I have here. So, everything that we see here is the is the model reasoning. So, when I run the graph engine on the mean, the engine is only going to give me the answer, you know, only these tokens here. Because the Cypher query, it can give me the mathematical operation itself. It can do that. So, we don't need the engines, the LLMs, the model uh do that for me because the Cypher query already give me the right question. So, it is 47 and the average. So, here in this uh first we are lucky because probably they are Yeah, only two hotels. But, what happened if this vector store have more than two or three hotels in the in the vector store? So, we are not going to give a real answer. It's going to the LLM is going to calculate with the only three answer that it receive. So, it's going to build this operation with only three hotels. But, if the vector store have more than three, we are going to have some problems. Something that I am you know uh yeah. So, let's run the next one. The prices counting. Something similar too. How many hotel have a swimming pools as a amenity? How many again? So, it give me the the traditional It's appear that the search did not return some specific information about hotels in Paris. Okay. What happened with the other one? There are currently no hotel that offer swimming pools. No I. There is no hotels. So, the other one is like mm would you like to ask about hotel with other specific amenity or information? It's like mm maybe it is or maybe I don't know. Yeah, it's not so accurate, okay? So, mostly conversation. Oh, what is my question? What are the room types and price for the highest rating hotels? What are the rooms types and price? This two question. So, it search the fact I had had only one. The highly rated hotels in any company Paris blah blah blah. I I don't speak French. With the guest rating is that one. However, I currently don't have Okay, blah blah blah. A lot of data there. So, what happened with the other one? Receiving notification for Oh, I have a error. I hear in the second one I receive the the question. The highly rated hotel is harmony blah blah blah. They offer following types. But, unfortunately these rooms are not available. So, I receiving a answer. I don't receive a lot of blah blah blah there. I only receive the answer what I need. And for the next one, how was the meditation? What is the question? Tell me about hotels in Antarctica. A spoiler, there is no hotel in Antarctica. There is zero hotels in Antarctica and let's see. It's appear that the certain order to specific information about hotel Okay, because they are I search for the other have the tail blah blah blah in the If you are looking for particular that experience specifically inquire about visiting Antarctica. No. Please let me know and I can assist you for it. Okay, a lot of tokens that I spending there in that answer for Elena. What happened with the other one? They are currently no hotel listed in Antarctica. Of course, because it create a cyber query, the cyber query and it receive zero. So, it no, it give me a honest answer. Here's summary that close grade for me for this Neo4j notebook. But here is something that I want to show you before go to the other one. So, here something that I love for Neo4j because why I'm using Neo4j? Because in the library the Neo4j give me uh it can build it use a LLM. I'm using the OpenAI as well to build the knowledge graph. So, how I build my knowledge graph? I only have a bunch of data, a bunch of txt only text and I send that data to the Neo4j library that I miss out. Okay, I send that data to this and Neo4j using this all this um library just right here, it can understand all my data and it can build the graph for me. So, I don't need to create that using the simple knowledge graph pipeline inside the knowledge graph library. So, that's why I'm using Neo4j. It's amazing. It's super simple to use. So, I invite you to use it at that. So, let's go to the next one. So, multi-agent validation. Sometimes an agent fails, and nobody find out. It calls a tool, and the tool returns an error, and the agent does not surface that error. It generate a confidence success response instead. The user thinks it work. You think it work. It didn't not. The agent acts and validate its own output in the same loop. There's no separation, no second opinion. So, when something goes wrong, it rationalizes and tells you it's okay. It work. Here is what happens. Inside a single agent, when it fail, it calls the tool, gets an error, rationalizes it, and returns a success response. The user never sees the error. You can address this by adding a validation layer. You can have three agents in sequence. One acts, one checks, and other approve or rejects. Strands agent has a built-in class for this called swarm. It manage the handoff between agents automatically. You just the role of each agent in a system prompt. And let me show you that. So, this demo only needs a Swarms agent with open AI integration. Let's see the requirements here. All you need of is Swarms agent. We don't need anything more. Let's go to the notebook here. And the key important thing here is the already running is Let me see here. Is the Swarm. The Swarm it's what lets you to connect multiple agents together without creating like a for a while manually to put the all these agents together. It can build a chain and manage the handoff between them automatically. So, here we are going to create three agents. Let me go down because we have the normal and we have some ground truth data. So, here is to create a a single agent. So, we have we are going to test three different scenarios to validate a booking. This They we expect true to enable hotels. We expect false and non-existing hotels and missing booking. We expect false, too. This is how we created single agent. We need a prompt, some tools. And we are using the tools that we have here. And here we know that this is the data. So, the agents can can give out the answer that we are looking for. And that's it. Let me go to the other ones. Oh, wait here. So, how we build the Swarm? I already run this. I want to show you this one. So, we are going to build three different agents. The executor, the validator and the critic. The we have a system prompt. You are executor agent for a hotel booking system. >> [clears throat] >> Use the provide tools to fulfill requests accurately and blah blah blah. The validator is you are a validator agent. Review what the executor did and output exactly on off. And we have the critics that is going to set here approve or no reject. Let's run this. And oh, what is here? Our model didn't define the model. Sorry. Sorry, please. Forgive me the life because I didn't run this one. Let's run now because this is the the normal. And the model is there, right? What? A little bias. Run that. What's happening with you? Let me go this. Let me copy and paste. This one. I don't know why this give me problem. Multi-agent swarm is the time the same name. Come on. Yeah, you can see this is life. I don't going to edit this. So, yes. Okay, we have Thank you. And the swarm, how we how we create the swarm? We have swarm and the swarm function, we put all the agents together and the entry point is the executor. So, all everything is going to start in the executor and then it's going to hand off. And six time because you can handle that, too. So, get the swarm final response. Let's do a for and they send all the response. And book the Grand Hotel. I have booked a Grand Hotel for Alice tonight. Booking blah blah blah. Hand off. Valid. Okay. Looks good. Critic, approve. So, the critics is approving the this. This is the other one. And we can run some comparison here if you want. So, here we can see that the swarm handles the flow between the between all the uh three different saying. It was what happens when the single agent tries to conflict something that does not exist in the system. And now the same request through the swarm, the executor gets the error, the validator catches, the critic rejects this, and the user never see a fabricator response. So, you can we can test here with the single agent that it's a suggested unknown entry and return success. And the swarm, it have a executor that go the error, validate saying, "Hey, come on, man. Hallucination and the critic they reject." So, the user sees a clear failure. The fabricator confirmation so missing the issue here. And you can run this to compare the two different agents. So, here we can see that the single agent it have everything like a yeah, it's okay, it's correct. But in the multi-agent swarm, you can see that the last one is a you know, missing issue here. Something is happening. Now, let's go to the next one. Okay. Neuro-symbolic Guardians. You have a rule for your agent. Let's uh say maximum 10 guesses per reservation. You write in in the system prompt. You even write in in the tool description. And the agent still calls the tool with 15. No, because it is ignoring you. Because prompts probably are suggestions, not constraints. The model process them as a text. Not as a logic it has to execute. It's probabilistic. Only code execute logic. A rule in the prompt the model read it as a suggestion. A rule in the code the model can not escape it. Neuro-symbolic guardians rules put the rules in the code. E-strands A and has a few that call him hooks. The the function that is trans called automatically at specific moments in the A and loop. It is this case right before a tool executes. So you have a hook. You write the rule, check the parameters and if they fails you cancel the call. Let me show you this in the code. Here we have the Jupiter notebook. We have the requirements here. Um what is the same requirements as the previous demos we only need I will install everything again. We only need E-strands and open AI. We don't need anything else. And um my API. And here is the thing important. In this demo are these three hooks provided in this trans base class. So you allows you to create the hook. So we have hook provider, hook register and before to call event. And the tool register is what E-strands pass you to register your call back. And before to call event is the event that fires every time the model is about to execute shadow. That last one is what makes it possible the intercept the call before it runs. So, you have the before two calls even and call we have after two calls even that we are not going to use that here. And let's see the rule in the prompt. Remember, the model reads as a text and it may follow or not follow them. So, let's see this. I want to run this. So, here we have some similar state that we are going to use in this engine. And let me this is the symbolic rule. So, booking rule if if rule booking rule as a this is something that we create. Let me show you the rules here. So, we have the rules. Rule one. Validate dates. Check in must be before check out. If what is validated they check out. This is something that it check. This is a something that is going to invoke when the rule is is uh is must have to use it. And uh to do We have another rule for max guesses. Maximum take guesses per booking. So, if I want to use I don't know I if I want to book in uh uh I want to do a booking for 11, it's going to block me. It's not going to reject the booking. And we have some confirmation rule. Payment before confirm. You know, you can confirm if you don't have the payment. Cancellation rule. Cancellation within the cannot cancel within 48 hours of check in. So, this is something that I've created here, you know, to this demo. So, I have booking rules and I have confirmation rules. I don't I use in here the cancellation rule. So, create validation hook. We create the validation hooks here, right? Like this, neuro-symbolic rule, and we use the hook provider. So, we have a bunch of code here. Then, we add the booking rules, the confirmation rules, and the state. And you can check this by yourself later, but we want to see the demo running. So, that we define the claim tools that we are going to use them for the booking hotels and the process payment. This is the the normal tools, you know, and we need to add that tools to the hooks. I think I add that. Yeah. Tool name, book hotel. So, this thing is going to get the eight when the book hotel is using the Yeah, the book hotel. Okay? So, let's create the agents for the comparison. So, we are going as the other demos, we have three different scenarios. Confirm booking without payment. Payment must be verified before the confirmation. This is the rule that I must trigger this. We have booking hotel exceeding guest limit, and we have valid booking for five guests. So, yeah, we have the three scenarios. And we have the normal engine, the normal engine, and the vaseline engine. And we have the engine with the neuro the neuro-symbolic guardians. So, we have the hook, the neuro-symbolic hook. Uh this is the same. Here, yeah. And we add this If you can see, the normal engine [snorts] is only three lines. The tools, the model, and this is the line that give us the difference between these two engines. So, I already run that. Now, let's go to confirm a booking without payment. So, I run this my engines. The first one is Abis. Come on, man. I Yeah, because I Yeah, it confirmed the booking without payment because it doesn't have any rules and the promise is super basic. And the other one is Let's open this and is booking blocked. Payment must be verified before the confirmation. Thank you. So, it's okay. Now, let's run the other one. Test second scenario. What is the question? Booking hotel access guest limit. So, the question is I Yeah, I have more than So, here the hotel Abis successful booking for Yeah, because the promise is basic and I don't put in any rule there. And for the other one, it seemed that the Grand Hotel has maximum capacity of 10 guests per booking. Additional booking must be made at last 1 day in advance. I don't remember the day. Probably This is something that they give them. And to validate booking is both engines execute all rules pass. Because they Yeah, it's only a validation for booking. So, book hotel for blah blah blah. All rules passes. It seemed that the booking is still needs to make. I don't I didn't remember the question. So, booking is made provide I Yeah. Okay. Okay, I get it. And Okay. Now, run all the thing this question. Confirming without payment, booking save max. And what happened here? This is all the scenarios that we just run. So, this is awesome thing to compare the results. So, Bali booking king secure five guesses, allowance, correct, wrong. 50, blocked, wrong, correct. Confirm, so there we have here the comparison between the two ends. So, what we have what we have here is same model, same tools, same prompt. And the different that outcome because the rules are in Python knowing the prompt. This pattern of enforcing rules in code before the tools run is also what Amazon Aion core policies service that we have does at the structural level. So, the same concept but managed for you in production. And you only have to create the rules. So, the hooks are all or nothing. They block everything or approve. But sometimes you want the agent to adjust and keep going. No and stop and leave the user waiting. That is what I will show you in the next. Run time guardians, run time is still is still don't block. This is the next one. Hooks blocks unconditionally. The agent is stop and the user has to retry. For a hard constraint, that is exactly what you want. But sometimes the rule is soft. Maybe room fits for four guests, but a group of six could go book two different rooms. Or a flight is full, but there is ability on the next one. You do No, you don't want to block everything. Probably, you want the agent to find a option and complete the task. That is a steering. Here, we have the hook that fires and the task failed. And with agent control, it steered the models and the task complete. The other different here is operational. Because with the hooks, it's uh changing a rule means changing code and redeploy the whole harness, whole the engine. And with the agent control, which is the name of the open source library that we are going to use here, rules are registered on a local server via API. You open the agent without touching the agent code because the agent picks them up immediately. Let me show you that here in the code. Yeah, this is the notebook. In this demo, we only need a extra packet. The agent control is the key. So, the agent control is the one that helps us to create a steering. And here, we are using the setup control. This is a a little application, a little app that I create with the local server and the steering rules. Here, we have the local server. We have the control. They [snorts] are the steering rules. First, we have the the steer master guest. A steer, you know, guy. So, guy agent to reduce a guest count when exceeding maximum of 10. And he's going to steer. And we have some control that deny. For example, deny no payment. Block blocking confirmation without PR payment. And here, well, this is to to create the um the service. The service. And let's go to the uh notebook. So, I'm ready. Um Let me go here. Yeah. So, I need the environment. So, this is the agent. Um I'm going to have a error here. Wait. Wait. Wait. I'm going to use bedroom. Wait. Wait. No, then let me comment this one because this going to give me a error. And let's run this. Yeah, so we have the hooks as we did the demo. So, we have book any company Lisbon and we have some prompts. You are a hotel booking assistant. When booking, first describe what you will book in blah blah. And this is the prompt for my agent. Where is my agent? Agent control. This is for the Well, this is on some helper. And this is my hook that you already know because we create this. So, we have the system prompt. And we have the hooks. So, let's test this agent with the uh book any company Lisbon for 60 guests. And if you remember, this only can book for uh less than 10 guesses. And of course, he's blocking. Now, let's go to the AIN control new agent. Here we have two key importances or two key imports that are important for the AIN control SDK. So, we have the AIN control plugin that captures AIN events and sends them to the AIN control server. And we have the AIN control student handle that listen for a student decision for the server and delivers them back to the model. Together here they are what connects a student to the AIN control student logic. So, let's go this with this. So, the student agent is going to a river room for any company list board 50 guests. Let's see what it's doing. Yeah, I have to set for booking for a stay in the company list board for 50 guests for May. So, it's the reservation have been split into two rooms. So, it took that it in self. You know? One room for one and other room for five and that's it. So, use hook for hard constraints. AIN control for software [snorts] rule. Now, we have five technique or running locally. But, how do you take this to production without maintaining service, without building infrastructure? Let me show you how. Everything that I just built here in the previous demos runs locally. For the production version, Amazon Bedrock Bedrock Amazon Bedrock AIN core give you the run time. A gateway, a short term and a long term memory, a cloud watch observability, build in no servers to manage. Here the architecture, the strands engine runs inside the run time. And you know, you inside the run time, you can put every framework that you want, not only run a strands. The gateway rules tools calls to the lambda function automatically as a tools. And the steering rules from the previous demo live in the DynamoDB. So, you change them there and they are live on the next call. And you don't need to re-deploy anything. And you want to use Neo4j, of course, you can use Neo4j Aura DB, that is a external graph database, which also has a free tier. And the code is in the repo here. Is in the repo. And you will need AWS credentials if you're using Amazon Bella Betray in core. And in the resource link, there is some credits. I hope that you can find it because I always try to to give away some credit for AWS so you can deploy everything for free. And uh if you are new here in AWS, I have a repository that you can use to deploy everything over this architecture uh using a notebook, too. But, if you are familiar with CDK, cloud developer kit, you can do it that as well to deploy everything at once. Uh both options are in the repository, of course. And you can go deeper in Agent Core with all the documentation that I left there. So, please don't stop at the demo and try going to production with Amazon better agent car. So, let me bring it all back. Tokens waste on every request. You can fix it with semantic tool selection. And confident answer it's never computed. Possibly it's possible because you are asking how many. So, you can use graph rack and query the data. Don't sample it. Sometimes we have fabric case success confirmation. You can use multi-agent validation and second pass catch it. And rules the models uh quite a little skip it. You can use uh neurosymbolic guardians to enforce in the code. Don't trust in the prompt. And for the hard blocks that I stopped the user, you can use runtime SDR, self-correct, and finish. Each demo that I just showed you is in the in the repository as a notebook and an application as well. Start with the demo one and then go to the demo five. And if you want, deploy it in the productions. So, have you tried any of these in your own hands? Thank you to join me in this session and happy building.