KamiwazaAI Field CTO James Urquhart on Orchestration Frameworks for Managing AI Agents

Hello, and welcome to the latest edition of Techstrong AI Leadership Insight series. Today we're with James ICU Horne, who's CTO for Kae ai. And we're having a chat about, well, what's it gonna take to orchestrate all these AI agents and other things that are floating around out there?

James, welcome to the show. Thanks very much, Mike. Appreciate being here.

So I think we can all see this coming, right? There's gonna be thousands, who knows, maybe even millions of AI agents eventually, and they're all gonna be doing various tasks, and they're gonna be floating in and out of our workflows. So how do we manage this and how do we orchestrate all this stuff?

Because I got a feeling we're gonna be quickly overwhelmed. Yeah, I think, you know, this is one of the, the key things that I, I think a lot about is, um, you know, there's a number of things out there that'll help you, like kind of build an, uh, an agent and maybe help you deploy that agent somewhere tied into workflow. But how do you do that at an enterprise scale?

Um, is a really fundamental question when you have to your point, thousands of agents, tens of thousands of agents running within your organization. And the key thing is, especially we don't know what the a the final architecture is going to be. There's no sort of winning architecture for running these agents at scale yet.

So how are you gonna be able to adapt to, um, the differences of different ways of running, say swarm environments where you have, um, you just have masses of agents that are given sort of a common outcome that they're trying to approach, or hive environments where you have a central sort of coordinator who's starting and stopping other agents to fulfill a task over time, um, or even workflow. You know, the base ones, the ones that are most common today, where you've got a, you've got a fixed workflow, you've got an engine of some sort, um, and you have agents that are doing different tasks in that workflow. The key thing is, is we, we, we know some things about the fact, like AI agents are likely to be very ephemeral.

They'll, you'll, you'll start one when you need it. You'll, you won't keep it running all the time. Um, they, they may be crated on the fly by other AI agents so that the actual definition of the agent kind of pops up as it's the need is discovered and goes away if it doesn't work out, or, or if it, once it completes its task.

So you start to ask questions, well, what, what do we need to survive in that environment? And one thing that we need is we need to make sure that we have access to data that's necessary to complete the task that these agents are doing, regardless of where they're running. So we need, and we want, ideally, access to the systems of record rather than having to replicate and duplicate data all over the place.

Um, a second thing that we need is we need the ability for our prompts to maybe go and use, um, you know, a model, uh, in one location tied to one data source to answer one part of a question and, and have a different part of that inference running in a different place to answer a different part of the question and be able to bring that back as a common response. So one prompt running on ultimately many models and coming back, uh, to into play. Um, we need our agents to be able to sort of just ask a question and know that ultimately it's gonna get the, that proper response.

We need agents to be able to communicate with each other and we need them. Um, we need a, a good mechanism to make sure that that's secure. Um, all of this, by the way, has a security factor to it, access control, um, the, you know, the appropriate, uh, network protections and so on.

So those are some of the things that we think about, um, a lot in terms of orchestration, is how can we provide that substrate that makes all of those things there and available so that as you build an experiment with IT agents and try to figure out what are the ultimate architectures that make sense, you know, that you always have that access, you always have, uh, that coordination and, um, that knowledge of, uh, of location awareness as well, which is another big point in this, um, that, you know, that things are gonna run where they need to and data's not gonna leave the boundaries that it shouldn't leave. Alright. To your point, I think people are talking about this with things like, you know, the agent to agent protocol, but that's just a protocol.

It sounds like we actually need something that feels like a platform that's gonna execute those protocols in a way that provides the governance, the security, and the understanding of what's happening between these agents. Is that a fair assessment? Yeah, I think, I think every one of those, uh, protocols have been defined in a way that, um, uh, implementations have some freedom in terms of how they implement the protocol.

So, for instance, data a is a good example. They don't require any form of queuing, um, in order for you to have that communication between agents. But of course, if you're gonna do that at scale, you almost certainly need some form of queuing to make sure that you avoid, uh, race conditions or overwhelming systems, um, incorrectly.

So, um, yes, there's a need for, uh, for platform, um, to deliver sort of, and I use the term capacity, and it's, it's, it made a lot of sense where we're just talking about compute, network and storage, but now the capacity than AA agent needs is the capacity to, to get knowledge and the capacity to communicate more. So, I mean, underlying is still compute, network and storage, but, but it's the way that the a you design an agent and think about an agent, you're thinking less about, um, how the computing works, and you're thinking more about, if I tell a model to do X, is it gonna be able to figure out and go find the things that needs to do to complete that task? And that's really kind of the fundamental difference, is you still need to deploy things.

You still need to make sure that you're consuming the right resources underneath, but the developer or the agent should not have to be thinking about, how am I gonna get to A-A-V-P-U that has enough power on it for me to process this video, right? That, that that should not be what that agent designer should have to worry about. You know, when I talk to some folks, um, we're all obsessed about how we get the agent to actually go do something, but, um, it turns out that convincing the agent to stop doing something might not so be easy either.

So we need to have a framework or something that just says, you know, when you reach a certain level of parameters, you know, stop, otherwise you're just creating useless data for the sake of data. Right? Yeah.

I think that, um, there's still an agent design component that, um, that a, a platform like ours isn't necessarily going to take care of, but there, the ability to observe what's happening with the agent and to understand and react to the behavior of the system as a whole and of individual agents is really important. Um, one of the things that we're doing here by creating all these things that are smart and adapt and learn and whatever, um, and having them come in and out of existence, is we're creating almost the very definition of a complex adaptive system as, as that, that area of science knows it. And that means there are going to be emergent behaviors that come from unexpectedly from the way different agents end up working together that we didn't plan for.

So that's one of the things that we very much take serious, is how do we help you understand the emergent behaviors, uh, that are happening in, in, in your agent environment, uh, and help communicate with the other parts of your systems that are maybe handling, uh, you know, different specific aspects like workflow engines and things like that. Mm-hmm. Does that include negotiations between the AI agents?

'cause I'm asking the question. 'cause it seems to me there will be instances where AI agents will be giving, uh, conflicting missions and just like humans have where, you know, I may be in the marketing department and you may be in the sales department, and we may have a conflicting mission that we need to negotiate some middle ground on. Is that something that AI agents are gonna be able to do?

Or is that where the humans come in? Well, I, so I, I just actually, uh, I haven't published anything yet, but I, I actually did just write a blog post recently for myself, um, on this very topic, right? So the politics and the economics of these systems are gonna be real as well.

Um, do we do anything explicitly today on that? Um, uh, we don't, I'm not aware of any system out there that really does, um, a, a a, you know, phenomenal job, uh, at scale of doing those kinds of things. But there certainly are people experimenting to find, um, figure out how to do that.

AI will be involved in doing that. So there's checks and balances systems that are already in existence in large scale, uh, agent environments. Those checks and balances systems ought to be, um, you know, very, uh, just kind of very normal and very easy to have out there.

Um, uh, but it's also something, uh, it's one of the reasons why I say, you know, the jevons paradox of the AI age is everybody talks about how all the developers are gonna go away and how those jobs go away. But I don't believe that, I think the nature of engineering changes, I think it goes less from code writing and goes more to how do you curate these complex environments and the way that these things work together and handle the politics and handle the economic clashes and, and, uh, make tweaks as you need to, to help make sure that the system provides an ultimate outcome that's important to the business. But those things will exist.

And, uh, and we are, you know, in for a decade of learning a lot about what that looks like and how we can manage it. Mm-hmm. Another thing I don't see people talking a lot about, but I think we'll be needed is audit trails for what, who, what agent did what to whom when, and there'll probably be some sort of, uh, regulatory body that wants to know that information, but right now it feels like everything's kind of a black box.

But is that where we need to get to? Yeah, I think, uh, you know, at the very least what we do is, uh, we log the heck outta everything. Um, you know, we keep track of, like with the inference engine, as we, as we, um, do distribute, um, portions of inferencing out, we keep track of what, what decisions were made by ai, uh, to do that and what the responses were.

Um, we, we keep track of, uh, you know, of, of interactions, um, between, uh, between agents and, and the Kaza platform underneath for, uh, for data access and other things. And also, um, you know, we can help keep track of communication between agents, um, that are, are trying to complete tasks together. Um, it, it is absolutely true that there is no way for humans to manage systems like this unless they can see what the system is doing and see it at some detail and catch, um, catch what's going on.

That may be important behaviors, uh, you know, bad feedback loops or good feedback loops. Um, and so, um, so the, yes, this is part of what a, a genic orchestration environment needs to do is very much provide, uh, documentation of what the or the agents are doing in the orchestration environment. And one level of scale is always gonna happen.

And I'm also asking this question because we seem to talk about using AI agents internally to manage workflows, but eventually they'll be expanding multiple organizations and multiple companies, and will that create another level of complexity on top of all this? Yeah. Um, yeah, I wrote a book for a rally in 2021 called, uh, flow Architectures, uh, the Future of Streaming and Event Driven Integration.

And, and the, the thesis of that book was as we come up with standards for moving near real-time data, um, between, across, basically across organization boundaries, across domain boundaries, that's going to, uh, uh, just create a flood of demand for those kinds of connections. And I believe that AI is proving to be the killer app to drive that forward. And you see it with MCP usage, um, already, um, you see some, um, some examples of companies attempting to use agents to automate their, their interactions with other companies on a b, b basis.

Um, it's very early, yet it's not a standard thing that everybody is doing, but, um, but I believe that, you know, the reason MCP and A to a and some of the other standards are so important at this early stages, they're beginning to find the way that we are gonna communicate not only within our organizations, uh, among a agents, but also across organization boundaries. And that brings questions about, um, security and monetization and, um, and, uh, you know, and auditing the, the origins of, uh, where data comes from and all those kinds of questions. Um, so, uh, we do see that, um, uh, we do see that there's a interest in that.

We, most of what we see today in development with agents though, is sort of within organization boundaries for now. Um, but I, I definitely see experimentation happening with, with beginning across those boundaries. Hmm.

Also, as one wa one's put it, um, it's one thing to be wrong, it's another thing to be wrong at scale, but how do we roll things back if they're happening in near instant real time? And, um, and, and what is the implications for being able to do that? Yeah, I mean, there's lessons learned from past architectures that we can, uh, we can take advantage of, right?

So we want to have, uh, uh, we, we want to have a strong understanding of where we need to log things in the sense of, um, uh, of, uh, of something that we can roll back step by step, um, or recreate, uh, actions that were taken. Um, so things like what, what people have done with eventing and Kafka, um, you know, those, those kinds of patterns will also work in this environment where you can begin to, uh, to put some transactional logic around different types of interactions. Um, remember that a lot of what agents are doing is actually interacting with tools that exist already today.

So there's already mechanisms and those tools to help, um, with things, but those checks and balances things that, that I was talking about earlier is a very important part of this picture. We need agents that, in part, are verifying that the outcomes we want to have and happen are actually happening. And that can trigger, um, a, a, a known action in response to something not working out the way we want it to, not, uh, not validating correctly.

Um, so, you know, it, it becomes almost like a, um, you know, I, I, I don't know exactly what that pattern's gonna look like at scale. Um, but I believe that it, um, if you look at some of the people that are experimenting with swarms and, and other of those kind of large scale environments, they are explicitly defining prompts as a part of the definition of those things that say there must be, uh, agents that are verifying and validating that the following things are true. Mm-hmm.

So the agents will essentially police the agents. We, yeah, I, I, I believe that's the only thing that's fast enough to actually police them. I, I think humans will police outcomes at sort of large growing scale.

You know, are you getting the outcomes you want from your system? Is a, fundamentally a human question. But, um, and then I believe that some of the, the sort of training and psychology and, and, and all the things that in terms of, of how helping move agents in the right direction to get the outcomes you want is gonna be human involvement.

But in terms of reacting in the minute to a transaction that's happening, I think, um, you know, agents or possibly other types of software services, um, are gonna have to play that role. But I, you know, it, it seems like agents are the logical way to go about it. So what's your best advice to folks?

'cause at the end of the day, I kind of feel like the agents are being built first, and then someday we're gonna s slap something that looks like an orchestration framework on top of them after the fact, but maybe the cart's before the horse, which way should we go? Yeah. You know, it's a chicken and egg problem to a certain extent.

Um, absolutely validate that. Here's the way that I put it is if you, or there's a lot of organizations that know, that have done enough experimentation that they know that, um, they're looking for the agentic environment that's gonna change, that it's going to positively affect their business. They already know that they've gotta figure out how to put the pieces and parts together, and they're, they're running experiments today.

I would say if you're gonna start running experiments at large scale, that's the first sign that you probably need some sort of orchestration engine to make, uh, to lower the cost of that experimentation, to make sure the data access and, um, and inferencing and, and communication are, um, are straightforward for all your developers across the organization, across organization, boundaries were necessary. Um, and then I would say, you know, um, there are some organizations, uh, out there as well that are beginning to, um, uh, to implement, uh, small experiments at larger scale. And I would say as soon as you get to a point where you're really running, you know, hundreds of agents at any given time, maybe thousands of agents at any given time, um, there's a lot that an agentic orchestration system can do to remove the toil of managing all of those agents to make it, um, uh, less effort and less risk for your organization from security perspective, from a performance perspective, and so on.

And so, at that point in time, if you're like, we're, we're beginning to scale up what we're doing here to a point where a, you know, one human or a few humans can't manage it very well, um, it's time to look at, uh, an orchestration environment for the same reasons that when you were doing containers, you eventually looked at Kubernetes because you, you had gotten to a scale where managing all those containers became very, very difficult. All right, well, folks, you heard it here. Hey, it's quite literally just like going to the orchestra, right?

I mean, there's musicians, they all know the music, but they still need a conductor. That's gonna be you. Hey, James, thanks for being on the show.

Thanks very much, mark. Really appreciate it. All Right.

Thank you all for watching the latest episode of the Textron AI Leadership Insights series. You can find this episode and others on our website. We invite you, check all those out.

Until then, we'll see you next time.