Can Small Open Models Power Reliable AI Agents?

Transcript

[music] >> Good morning. My name is Kalyan Prasad based out of Hyderabad India. I work as an enterprise AI architect. Prior to that, I was leading AI and data science practices. Other than my profession, I'm part of Python Software Foundation. It's been actively 8 years I'm involved with Python community and also in scientific community. Um so what we are talking today about is can we can small models power reliable AI agents, right? That's what the whole talk is about.

So let me start with the questions. So we all know that agents can perform tool calls, it reasons, and it executes or it automate or workflows. But the But the interesting part is are they are fully reliable? That's the important point, right? So especially when we are moving from uh prototype to enterprise AI systems, so the reliability becomes very big a challenge.

So what we want to see in this talk is or by the end of this talk you'll understand that a successful tool call doesn't mean that it's a successful task completion. Why is something that we going to explore in the next 40 minutes. >> [sighs] >> So what I've done is uh Again, I'll I'll tell you. So imagine you are working in a IT desk automation system. Uh so it it triggers an IT incident ticket with so and so ID where you know it is it is saying that you know your uh employee portal is not accessible which is throwing an HTTP 503 error. So this

is one of the uh you know, controlled experiment that I built uh which is running locally in my mission. Again, considering the interest of time, I'm not showing how it works, but I have everything in my you know, GitHub repo. You can just feel free to play around it. So, when the service is is not available, so what we are saying now, my agent that now I'm telling my agent to go and investigate about this incident. And understand what's what is the issue is all about and try to fix it. Again, when it is trying to fix it, it should have enough evidence to justify when it's trying to fix an incident. Now, how it is going to work basically is that so, I'm so, it has a So, I have created a four tools here,

right? Where my agent look for one by one. So, it start with reading the ticket, right? since the issue incident has been created. So, it first basically verify whether you know, the ticket is actually existing or not. Right? If it is if the ticket is not available, it should do a hard stop. That's what it has to do. And then it also look at service status. So, it try to understand what is the existing service condition and where exactly the service is getting affected.

All right. And then in the third tool call, what it basically do is it basically look for a matching run book, right? When it has a matching run book, it it look for uh uh uh it it basically look to understand whether we have enough evidence to match the run book which is there. And if all of these three things, when the ticket is existing, when you have active service status and also the matching run run book, then my four tool call should perform like you know, adding a ticket note saying that you know, all these three are existing. So, it can write a note on the ticket status update.

But there are certain conditions. So, we have defined a task requirements here saying that if my ticket is not existing it should do a hard stop. And even if the service status is not available, right? In that case, what it has to do is it can it can do a retry error. So, for example, when the service status is not available, it it is throwing a retrieval error, my agent can also perform another action and check whether the retry works or not, right? If the retry is still failing where my service status is not you know, uh catching by my agent, in that case, what we can do is we have to stop as per the workflow requirements.

Right. And same thing with uh Okay. Cool. And same thing with again runbook match requirement. If the sun search whatever the search runbooks, which is not matching with the evidence, so it should do a hard stop. So, so these are the primary initial requirement goals that agent has to do. But, but how the experiment ended up is what you'll see in the next slide. So, I have tested this overall flow in you know, in a four conditions. So, firstly is a normal, so which is like as I told you in the previous slide, if all evidences

are available, so it can add a grounded note saying that the note ticket is updated. Now, the second condition is where if there is a missing ticket, it basically do the hard stop. And same with the transition status where you know, the status check fails again it has to retry until and perform a grounded note. And finally on the persistent system also, it has to check the failures and even if the retry is also failed, then it should do a hard stop block. Now, how did I run this overall experiment is I as I told you everything is running locally, so so the fancy where you know, most of us always get excited about the text stack is right. So, it's a simple Ulama where

you know, I used open source grenade model and also put my all my parameters also very straightforward and simple. So, I didn't added any you know optimization in my parameter. It was very straightforward parameters. Experimented it. So far, this is how the experiment. Now, what are the results? What exactly happened? Let's see that. So, there were 12 reasons Sorry, it's 12 runs that I attempted. But the interesting part is none of the 12 reason 12 runs has given me a zero success of end-to-end completion of a my task workflow.

Now, what do I mean by that? Okay, if you see here on the left side there are seven accessible runs, okay? And all the seven runs has failed my task requirements because I told you I have a clear task requirements which is what my agent has to follow. But my agent was not succeeding here. Right? And also the five were like you know, completely unknown failures. And how it failed? Because of some infrastructure interruptions. Right? So, because of the you know, run time issues or whatever it is.

Right? All those five runs also failed. Now, I want to be very precise especially when I talk about this seven runs right? So, if you look at this seven runs on the left side, there are three patterns that stood out for me when I was running this. Two were two runs attempted and it my agent you know, bypassed the requirements and you know, added a grounded note for a malformed arguments. That is one thing.

And three times when my agent again continued to you know, update is perform is unsupported actions even though there was no ticket existing as well. Right? And and finally and in last two runs though there was no status available but still my agent you know is very smart. It just started performing his own actions. Right? Now Now let's see that how this overall run happened. So when I talked about the overall 12 runs, I'll show you how this whole 12 runs happened. But interesting part in all of these 12 12 runs, there was not even a

one syntax error you know that got failed. That is the reason why my agent bypassed It It is smart in you know overcoming all the task requirement that we mentioned. It just followed. So if I >> [snorts] >> So I'll just firstly play how it all basically what happened in the overall runs. This is one of the run execution. I'm just showing the flow here. So these are like you know five different milestone levels where my agent performed one by one. And all these are like you know lively happening in my Uilama.

But again in the interest of time I'm showing here. Uh Right. So So let's start with the first one, right? So it looked up the status, okay? And there is there is no there was a status fail. So there is no service status available. So what it has been doing? It has started performing the retrievable, okay? And then again retrievable also failed. So what it means? Initially my agent attempted an action. It was throwing an service status retrievable error, but again it performed performing the retrievable, but again what happened is the retrievable also failed.

So in this case, actually it should stop as per the requirements. But what happened? My agent has created note. So it just bypassed all the you know the workflow task requirements and it has smartly created a note which is meant but it has straight away wrote a note on my ticket update. And the interesting part is it also has a you know run book ID which is also not matching with the you know evidence ID run books. Right. Now, again in the fourth milestone, it also say that you know task has been successfully completed. So, it is marked as a accomplished. My task is successfully

accomplished. Right. Okay. That's fine. Okay. But now if you look at the fifth milestone, so when the task is accomplished, when my agent is saying that you know task has been successfully accomplished, so the my final outcome of milestone should be also that you know I have successfully accomplished the task that I have to do. But what you see on this screen is that the final milestone it is saying that you know the overall outcome is a failure. Now why? Right? So, when it has accomplished in my fourth and it has supposed to justify the same thing as a outcome is accomplished. But here it is failure. Again, the final outcome that I want to see the end-to-end enough

evidence of the workflow is saying that it is failure. So, so what I have done again? So, I have tried again doing a another execution run here. So, again in this same different milestone of a run I tried again. So, what happened here is initially uh the failure encountered, we all know. Then again, this time what happens is the retry was success. Earlier the retry was failure. Now, this time the retry was also success. Okay?

So, it has been claiming that you know the retry has been success and it has the enough evidence and also checking that. And third time, okay, when now my retry got success, my agent is very smart enough and it has been updated you know the grounding note on the ticket status. But if you look at the milestone four, what it says that you know the final request is again unanswered here. It is it is it is saying that you know, transport failure. It is meaning that that there were some infrastructure interruptions because of that I couldn't make it or my task is not

successful so it says that because of the time out issues I have not recorded or else I was not supposed I was not able to justify task. But again in a final run here also the recorded outcome is purely unknown which means that No final response is what you see is again observed. It is claiming that because of my infrastructure termination again it's not at all scoring the overall task requirements and it's not giving my end-to-end update.

Again one of the things when we also say that here there are two things from engineering perspective you should understand that uh When the intra infrastructure interruptions happens it doesn't mean that the failures are happening just because of the models so because especially when you are this is where evaluation of agents comes when you are trying to evaluate the agents so we should clearly distinguish between the model decisions of the execution environment versus the reliability of the surrounding infrastructure so those are the different things otherwise what happens you try to make a very wrong conclusions on your own measurements right that's what happens.

Now okay what we have seen on the seven runs is something you know, it's not giving now you might be wondering what about those five runs right did we conclude so out of those five unknowns also if you see four were purely on the infrastructure issues transport terminations because of the time out everything it happened and the whole run run again it does not have a enough evidence that line so again it has written the same status. So, So, one thing here is it's not just because of a, you know, the model failure or it's not just because of the infrastructure interruptions. So, we do not conclude and say that from this

experiment out of these 12 runs, we don't we cannot say that the the experiment that I have done is like, you know, justifying that whether my small models are reliable by producing the reliable agents outcome. But, the interesting part that you have to see is that the successful the tool call that I that my agent has made, it doesn't justify the successful task completions. Now, how do we put this or else how do we bypass this?

So especially when this is where you your tool boundary comes in. So, the reason here the issues encountered is because we have not created a semantic boundaries for our task, right? So, you have to clearly put a semantic boundaries for your tool calls, then only your agent has to perform. Right? Now, how we can do here is Right? So, again, what I'm saying is a valid tool call which is another way to okay, tool call has made, it has producing a valid JSON output and everything is fine.

But, end of the result, it is not successfully completing my task at all, right? Even though it was checked thoroughly, we are trying to add a grounding note, everything, but still it doesn't make any sense. So, for that, we need to have a enough evidence to justify it, right? That's what I mean to say. So, So, in order to justify that as I said, we have to put the semantic boundaries in place and then otherwise But, when again when I I mean to say semantic boundaries, especially when when in our in our case when the three evidences are concluded, okay, it's like you know your service

status, your ticket, and your run book. All these evidences are concluded. So, it has to be it has to be thoroughly follow the requirements. Otherwise, otherwise, it cannot justify it. Yeah, I think that's all I have. But, if you want to understand from a technical perspective, you can check out that. And if you want to follow, these are my you know specials. Thank you so much. I'll be happy be to take questions if I am here on. Thank you again.

>> [music]