Why Agentic AI Success Depends on Data Governance

ai Leadership Insight Series. I'm your host, Mike Vizard. Today we're with Sanjay Kureja, who is head of global technology for eClerx, and we're having a little chat about, well, the trouble with AI and AI agents might not have anything to do with AI agents or the AI models, and it might have everything to do with governance and data.

Sanjay, welcome to the show. Thank you, Mike. Thanks for having me on the show.

So let me talk to you here for one minute about this, but we're all obsessed with AI agents and hallucinations and all the things that can go wrong, but correct me if I'm wrong, but I feel like the AI agents are just exposing existing cracks in our data management and data governance systems as it is, and it's maybe not even their fault at the end of the day. No, you're absolutely right, Mike. And the more and more we talk with our clients, the more and more we discuss with them on how do we go about building this entire agentic AI roadmap with them, we realize it's less of a model problem because most of the models today are almost at par.

It's mostly an underlying governance, infrastructure, and data. That's the fundamental problem. So if the data quality is bad, what gets processed out is also bad.

And same goes from a governance perspective. So if you do not have the right set of controls around the processes, around the ingestion of data, around the consumption of the output, then also you have a challenge. The model will do its job.

Whatever you ask the model with the prompts that go into it, the model is going to do the job. But the surround is more where the focus is. Most of our clients are trying to drive those discussion, moving out of model efficiency to how to make sure that the entire process and data do not have a debt.

So there should not be a process debt. There should not be a data debt. I think the model performance is more or less given from our clients.

Is it your sense that maybe we are ahead of our skis, and there's a tendency where everybody wants to go build the next great AI agent, but nobody wants to necessarily address the data management work up front, or they're forgetting about that, and then they're only discovering when the pilot project goes wrong. But is there a smarter way to think about this before we get started, or are we just going to learn from the school of hard knocks here? No, you're absolutely right.

And Mike, if I had to relate it back, it goes back to the RPA days, six to eight years back when the entire RPA wave started. The reason why people wanted to do an RPA was because it was taking a lot of time to fix the broken IT system. So that's why RPA came into the forefront.

It's the same wave which is going now, we feel. More and more people want to jump in to run AI agents because they don't want to miss out on the bus. But they do not realize you can do a very successful pilot on a limited data set.

You can take a small process, small use case, take a limited data set, run a pilot. But when it has to go into production, it's going to process real-time static data. It also is going to ingest a lot of data.

So it's not just a proof of concept static data which is going to sit there. It would also have the pipelines for ingestion of data. So it will be processing a lot of real-time data.

And similarly, it will be kind of spinning out a lot of results which are going to get consumed by the downstream systems. That's where people do not realize if the input and the governance is a problem, the output is going to create a lot of mess. And I think everybody's talking about this.

Gartner published results saying that only 70% of the projects are failing. Every other research firm is publishing results that data debt is going to be a big thing that one should kind of watch out for. So that's true.

I think the difference in running a small pilot and being successful at the pilot vis-à-vis kind of moving it into full-time into production on a scale, that's where the gap comes in. Who knows this? Do the IT people know this and the folks who are building the AI agents are just blissfully unaware?

Or do we just have too much faith in the quality of the data that we have in our systems, and now we're starting to realize, well, maybe it wasn't all that what it was cracked up to be in the first place? I think it's a combination of a lot of things, Mike. One, it's a huge FOMO problem, and the fear of missing out, that I'm not running an AI project.

The world is running an AI project, so I should be doing something. So because of that, a lot of people pick AI projects. And then when they have to kick off an AI project, the easiest way is to make it run on a smaller data set, pick up a smaller process, just to try to prove that the project would be successful.

Now, moving that into full-fledged production requires everybody coming on board. You need the CDO to be on board because most of the data would be validated. The data governance would sit with the chief data officer.

The CIO, CTO needs to be on board because most of the technology aroundagentic AI, what people now calling hardness engineering. So it's not just the model, it's the orchestration there, it's the infrastructure, it's the underlying data sets. So the CIO, CTO should be involved, and then the business should be involved because that's where the use case sits.

So the business has to decide where they want to deploy it into production. I think all three need to work together to make sure all the processes are in place, the governance is in place, and the underlying data is in place. Where people miss out, which we kind of see in today as well, is everybody wants to get a couple of pilots out, but moving that pilot into production is where the gap kind of hits a roadblock.

Right. And are there going to be a lot of dependencies between all these AI agents that we're about to deploy? And it would seem to me it only takes for one of them to be off on the data, and then they start passing that through to the other AI agents, and then everybody starts making decisions based on faulty AI output that is traced back to maybe just one AI agent went awry, but then they basically poison the well for everybody else.

Is that a fair assessment? Absolutely, that's a fair assessment. And that's a concept a lot of clients do talk about.

So most of the implementations today, Mike, are not a single agent implementation. It's not that I have to pick data from one or process it and then push it out to another place. Most of them are multi-agent orchestration.

So you have a large process that you want to agentify, and you would have multiple agents. One probably reading datas from mails, the second one processing something, third one pushing it out to a Salesforce, fourth one you're operating something into SAP, and the fifth one, let's say mailing, sending an email out via Outlook to a customer. So there is a handoff happening across multiple agents.

What gets even tricky is if you're using your agentic framework with a third-party agentic framework. So let's say I have a Salesforce deployed in my shop, and Salesforce has its own agent force. So there's a handoff happening between my set of agents built on one particular platform with another set of agents built on another platform.

There are A2A protocols which exist, but you're absolutely right. So given the complexity of these deployments and given the fact every implementation ends up being a multi-agent orchestration, if there is a point of failure which happens at any of the process stage, it's going to have a ripple down effect. So that's where, to your point, data, absolutely, that's critical, but the governance part is also very critical.

How are the handoffs happening across agents? How are the credentials getting passed on? How are the risk and controls in place?

So all those absolutely need to be defined for the project to be successful. So what's to be done about all this? Do we need to basically go back in and have a massive cleanup of all the data that we have out there before we can really proceed with all this?

And how big a effort might that entail? That's a million-dollar question, Mike. Absolutely, the data needs to be cleaned up, and there should be one source of truth for the data also.

But if you start cleaning up everything, it's going to take some amount of time. So you need to figure out which data sets are critical, which are the ones you want to run your AI agents first, and then start working on those. Things like master data, they're super critical, whether it's customer master data, product master data, they're very critical.

So making sure that data set is clean becomes the first starting point. Obviously, if the master data is clean, hopefully the transaction data is clean. Then it's not only the data that you are generating, it might be data you're sourcing externally.

We've seen organizations which kind of source data third party from Dun & Bradstreet, LexisNexis. There's a lot of third party every organization needs. So that data needs to be integrated with your organization data.

So you're right, absolutely. You need to have a very solid data strategy. There needs to be somebody who's working very religiously on the data strategy, the data governance, the data modernization, the data cleaning.

And then you have to align your agentic roadmap along the lines where you have these clean silos of data sitting and have something which kind of works on top of that. And it does not need to be centralized. I think most of the organizations do realize having federated data sources is also good.

You don't have to make sure all the data is cleaned up and all of them is pushed into one single data source. But as long as even the federated data sources are clean, that's critical. You will need to ensure that data cleanliness is part of the entire roadmap that you are looking at.

So at the end of this, do I need to then go back through, say, my data sets and basically certify them as AI ready, and maybe that's how I approach this, where I'm going to create some sort of, I don't know, label that signals to the AI agents that this data is ready for use? Absolutely. You're right.

I think that entire data cleansing, data labeling, metadata management is an important part of it. So if you're picking upIf any organization picks up a project, you need to make sure that that's part of the roadmap. And that's what we've also seen, Mike, with most of the clients where we work.

" I'll give you an example. We have a very large client looking for supply chain efficiency, and they were very keen on using agentic AI for supply chain, especially around goods return, which are damaged goods or goods which come back from the customer. But when we started the project, we realized the process was less of a problem.

Having clean data on customer masters, on the addresses, on those transactions was more critical. So absolutely, the vision might be to automate a lot, automate all the returns that you process as part of the supply chain and logistics. But I think when you scope it out, you need to take a look, as I said, across everything, data governance, everything needs to go into it.

Yeah. As we kind of play this all the way through, though, might there someday be a set of AI tools that will help me clean up my data to make it more usable by other AI tools, if that makes sense? Yeah, absolutely.

There is a lot of talk on tools which can help you clean metadata. There are tools which are now talking about faster implementations, faster migrations across platforms. So yeah, as an organization, you should keep on evaluating all the toolkits that are available.

How will organizations therefore need to be organized in the age of AI? And I ask the question because, well, a few years ago, we saw the rise of the chief data officer, and now we're seeing the rise of chief AI officers. And of course, there are CIOs in this mix, and there's no shortage of C titles.

But is there some way of thinking about this that allows an organization to unify all this? Because otherwise, I feel like we're going to have a lot of teams that are doing a lot of redundant work. No, absolutely, Mike.

And I can share with you the way I think it, this is primarily my own perspective. So the chief data officers when they came in, the places where we found them to be successful was where they were not bottled into trying to do data cleansing. They were more around setting up the governance, setting up the process for the entire data management.

That's where the CDOs were very successful. And we are seeing the same on the chief AI officer. So all of them, and you're absolutely right, the chief data officer, the CIO, CTO, and the chief AI officer have to work together because we feel that this chief data officer and the chief AI officer play a very pivotal role in defining the governance, defining the overall model of ingestion, and the output.

The chief AI officer, our experience has been, works very closely with the business to identify the use cases, set up the entire governance structure. The technology or the hardness engineering still sits with the CIO, CTO. So, if I have to deploy a tech stack to get an agentic AI implementation done, we've seen that still sits with the CIO.

But most of the governance, the process, the guardrails, the success criteria, and similarly on the data side, everything from a data cleansing, data governance, data management sits with the CDO and CIO. So as long as the roles are clear and people do realize that they have to work together, we've seen all three working very well for any organization to be successful. Short of maybe taking all those people and throwing them in a room and locking the door until everybody comes to an agreement, what's your best advice to folks about how to approach all this in a way that maybe will bring some rational thought?

So Mike, where we've seen agentic AI being successful, couple of things. One, people should not just rely on proof of concepts. Proof of concepts are easy to execute, but they do not tell you the problem that you'll hit on scale.

So the amount of effort you spend in proof of concept should also include the amount of effort you spend in doing the assessments, trying to build all the governance structure, trying to do the entire risk and risk mitigation strategy if you go ahead with the project. Then the CIO, CTO organization is very critical on putting this stack together. So, it's not just as you rightly said, it's not just the model, it's the underlying data, the underlying infrastructure.

It is the entire orchestration there. It's a multi-agent scenario, and it's not just touching one single system. You have backend ERPs, you have front-end CRM systems, you would have HRMS systems.

So it's going to touch across various systems. So making sure you kind of assessed the processes, you've understood what is the success criteria for the processes. You've scoped it out, you've built all the guardrails, you've done the risk assessment of going out with the implementation.

Do the preparation that is required, as you rightly said, from a data perspective, from a controls perspective, from a business benefit perspective, and then to the actual deployment. So if we take care of all these steps, we've seen projects being very successful. So, it's not that a lot of projects are failing as long as you can plan, execute, think through, assess, put guardrails around, projects are successfully getting deliverables.

All right, folks. You heard it here. Hey, the building of the proof of concept or the POC for AI agents, that's the easy part.

The hard work begins once you start getting that thing up and you start accessing all the data because suddenly you realize, boy, there's a lot of stuff from the last three decades or so that we never really addressed in the first place. Hey, Sanjay, thanks for being on the show. Thank you, Mike.

Thanks for your time. Thanks a lot. AI Leadership Insight Series.

You can find this episode and others on our website. We invite you to check them all out. Until then, we'll see you next time.