Agentic AI ROI Starts With Trusted Data

ai Leadership Insight. I'm your host, Mike Vizard. Today, we're with Ahmed Bashir, who's the CTO for DevRev, and we're talking a little bit about, well, what's going on with all this agentic effort that we're seeing out there, but it's maybe not paying off as well as we initially had hoped or thought, or could be just one giant work in progress, but let's see what's going on here.

Ahmed, welcome to the show. Good to be here. There has been now report after report after report about organizations struggling to get a return on their agentic AI investments.

And the part that I can't quite figure out is whether or not it's the technology itself is maybe more complicated than we thought, or is it the simple fact that our processes are just kind of broken to begin with, and the AI is just exposing weaknesses in our existing systems? Yeah, I think it's a mix of a couple of things, really. We've looked at a number of customers that got quite excited when they saw that AI could cluster, classify, detect really, really quickly.

These are things that were typically hard for humans to make assessments at scale, to do classification at scale, to do summarization at scale. And so whether it was support ops or finance ops or sales ops, didn't really matter. In all of those cases, the classification summarization assessment was being done really well by AI, and that got everybody excited.

And then what happens invariably is you have to take that assessment and then get to some level of remediation. Ultimately, you have to put a status update on your website, or you got to escalate a prospect, or you got to notify a customer. And that's where it seems to me that we started to realize that we need a greater degree of precision, a greater degree of safety and efficiency, more so than we ever considered for humans.

Because notifying a customer was something that was very easy for a human because at the very end, you had human intuition to lean on. But here, what was hard for the human is easy for the machine, and what's easy for the human is hard for the machine, and that's the piece that I think a lot of people forgot in this entire conversation, and we're beginning to see that now. " And it comes down to precision, safety, and efficiency.

So what do I got to do to get that level of precision? Because I think part of the things that we're dealing with is a lot of the data that we have stored historically is kind of messy, and if the data's messy, well, the outcome's likely to be messy. Yeah.

I think this is part of the challenge with a lot of the latest benchmarks is that, depending on the nature of the benchmarks, a lot of these benchmarks that you're seeing AI companies talk about, they're getting increasingly complex in terms of the task at hand. So you're starting to see people look for incredibly difficult tasks that you would maybe find on a physics Olympiad or some sort of a chemistry exam, and maybe even very difficult programming competitions. And the reality is, while it's important for frontier labs to test and build new models based on that, most knowledge workers in the enterprise, that's not their challenge.

The challenge is the organizational complexity, the complexity of how data is organized, and maybe the data sanitization problems that come with that, maybe the redundancy problems that come with that, the permissioning challenges that come with that. And I think what people are starting to really understand and respect is the fact that it's not going to be models alone that are going to provide the solution. It's going to be a combination of models and memory.

And the memory has to solve for the governance. It has to solve for the semantic layer. It has to solve for the orchestration.

In other words, it has to deal with the fact that there's real complexity in organizations because they are, by definition, they're relational, and they have complexity associated with that aspect of organizational structure. So does the AI agent have enough memory to really maintain those relationships? Because one of the things you see over and over again is that the context window is limited.

And as such, then the AI agent starts to forget things, or it doesn't remember things it knew once upon a time, and it has no concept of past, present, or future sometimes. Yeah. It's a great question.

I think it kind of reminds me of the difference between getting to a particular destination using an atlas versus using GPS. And we're going to have to learn this all over again with large language models. In other words, you could send all the information to the large language model and ask it to contextualize and pay attention to the right things, because that's effectively what LLMs do is they figure out what to pay attention to.

But if you send it too much information, you're going to spend a lot of money, and you're probably going to confuse the LLM as well, no matter how much better they get. And I think you're starting to see a surge in people saying: Why are we trying to send the most amount of information into the context? Let's try to send the least amount of information into the context, because at least at that point, there's less chance of speculation, right?

So let's figure out what data we could actually filter, join, negate, relate before we send the prepared context to the LLM. And that's more or less what companies like ours are doing. We're figuring out that indeed there's a place for databases, there's a place for SQL that lands before you actually touch the LLM, and that SQL is declarative, is deterministic, it's exhaustive, it's precise, and most of all, CPUs are a lot more efficient than GPUs in terms of doing an SQL statement and executing it versus effectively sending a ton of tokens into a GPU to process them over and over again.

So I think this is a realization that's becoming more and more clear to more and more people. To your point about that, we've had relational database systems forever, and I think I talk to people now, and some of them are trying to use graphs, some of them have created indexes of data. Is there a right mix of things that I'm supposed to do to create the context for the AI agent before it gets loaded into memory so that the AI agent is essentially more efficient, right?

Yeah, it's a great question. I think what makes it confusing to people is because a lot of technology that's come together over the last 20, 30 years, and let's just be clear what's come out. We've had warehouses for analytics, data lakes as well.

com and others. And there's, of course, transactions that are hitting relational databases like Oracle, right? So these are sort of the paradigms over the last 30, 35 years.

And every one of those paradigms has been very verticalized. What's interesting about AI is that the interface is so universal because it's conversations, that it can be a shape-shifting experience. Sometimes the conversation is analytical.

Sometimes it's a directive to perform a task. It's more a workflow-oriented conversation. Sometimes it's really about finding information, so it's truly a search-related conversation.

And sometimes it's a transactional conversation. You want to update a priority, or you want to reassign something to somebody else. And so because the interface that AI has brought is just so universal, we have to think about how to bring these technologies together so that the memory layer that we're using can be, in equal parts, analytical and transactional.

It can be orchestrated towards action, but it can also just simply be a search result. And so you kind of need a technology that can honor all of those relationships that you have with your data because they all exist, and AI has to allow for all of them to coexist. Do you think we're going to experience something that feels like a trough of disillusionment after all this hype, and then once we get our heads around all this stuff, we'll come back and say, "Hey, we figured out how to make it work"?

I think it'll be more like what iTunes and maybe Napster figured out with music is that if you wanted to really experience music in the age of high-speed internet, the point wasn't to build a robot that could flip the cassette tape or change the CD. You effectively had to come up with a format that could then allow for the new experiences, the iPhone to happen, the shuffle to happen, for playlists to happen, right? So you had to move from one format to another in order to really gain the benefits of the new technologies, whether that's iTunes or iPhone altogether.

And I think we're going to see that. We're going to see people really understand that their data is going to have to evolve once again in order to maximize the use of AI responsibly with precision and efficiency and safety. And I think everybody believes that they can make do with what they have, but invariably, data evolves and every generation of technology has led to some level of evolution of data, and this one's no different.

Mm-hmm. Do you think part of the issue too is that historically, every time there's a new innovation, we use it to do the same thing we've always done slightly faster? But the thing that we're trying to do today is very sequential.

And I look at AI agents, and I think one of the things that may be underappreciated is these things might be able to do things more in parallel, but we don't think in parallel. So do we need to change the way that we humans think in order to maximize the value of the AI agents? Yeah, it's a good observation.

" And I think many of us have known that. The reality is we got to keep it super simple. When we talk about what AI offers us, we keep it super simple.

We say it's trusted answers and safe actions, and I think that's what everybody's looking for. Everybody goes on journeys and tells stories, and that's what answers and actions are. And so I think at the most human level, and I think that's what we need to do, is figure out how do we become more human as we bring in more AI.

As we like to say, how do we work softer? Effectively, how do we become more human by allowing AI to do the things that were quite artificial for humans to do, like clicking on computer keyboards and many other things that we've gotten used to over the last 20 years, but our wrists are paying for it, our backs are paying for it. " And that means we have to get better at asking the right questions and believing and insisting on trusted answers and then, of course, insisting on safe actions.

I think if we do that, we're going to capitalize on this wave to a large degree. Mm-hmm. What's your best advice then to all the IT leaders and business execs out there that are somewhat frustrated at the moment, I think, and maybe casting about, and they're trying to figure out what the issues are, and they're pointing fingers at various things.

But it sounds like to me, ultimately, there's just some misunderstanding of the core technology. So how should they wrap their heads around all this? I think it's important to start with tangible goals.

That's why a company like ours is finding so much success in solving problems for lines of business, because lines of business actually have real pain. And whether it's trying to get a higher deflection or better resolution rates or trying to close deals faster or ramp AEs faster, whatever the deal may be, when you have real success criteria and you have real goals, you can start to work backwards from that and say, "You know what? How would a safe action that was autonomous help me?

" And that's really how we're able to help get adoption, and adoption is what gets traction and ultimately helps change behaviors and henceforth culture. And I think that's where it begins. I think sometimes we get caught up with people like CIOs and chief AI officers trying to circumnavigate the world, trying to figure out all the good and bad of AI, and ultimately what you need to do is get with a line of business that has real problems to solve, and then you solve them one at a time.

Mm-hmm. Do I need these massive large language models, or might I be better off with something that feels like a smaller language model that's more targeted to a specific set of data? Yeah, I think, again, it's going to be a both and.

I think the large models are incredible, and we will continue to use them or distilled versions of them. And I think the world needs the best 3D printer we've ever created, which is what the Frontier Labs are going to produce again and again. And there's going to be fierce competition because there's tremendous money to be made in that space.

So I do think we need to create the absolute best 3D printers we can. On the other hand, I feel like offline mode is a real thing. Going off the grid is a real thing.

Being able to go into draft mode and have your work be personal and private until you're ready to share it with the rest of your team, the rest of your organization, that's a real thing as well. And I think a lot of those things can be carried out with smaller models. I feel like a lot of protections, whether they're on the client side or the server side, can be carried out with smaller models, much smaller models.

Whereas there's going to be a lot of reasoning, a lot of planning that will continue to leverage the absolute best models that we can create. Mm-hmm. Will there be more attention paid to cost going forward?

Because we do hear people are kind of eating up tokens pretty rapidly, and the more advanced LLMs now have more advanced reasoning capabilities that are more expensive to invoke. But ultimately, I think there's a correlation between what you were talking about in terms of how much is in the context window and how much it costs to process, and we need some math to figure that out. Yeah, look, we've done livestreams in the past where we've showed that the same question can be answered by our product compared to, let's say, Claude, with 95% fewer tokens.

That's a 20x reduction, and it's not magic. It's figuring out how to use SQL before you go to the LLM. And let's be honest, most enterprises have a trillion tokens or more.

That's between all the opportunities, the contracts, the conversations, the incidents, the tickets, the customers. There's just so much information. If you were to actually think about it in terms of tokens, because after all, your relationship with an LLM is a token relationship, tokens in, tokens out.

So if you think about trying to convey the peculiarities of a trillion token enterprise to an LLM, if you could just put a database or an index or an embedding store in front of it to turn those trillion tokens into 20,000 tokens or 40,000 tokens, that's exactly what we need more of, and that's what we do. So SQL is not going to go away. Your local CPU is not going to go away.

All those CPUs running in Amazon and GCP are not going to go away, and I think we just have to figure out how to keep those as part of the platform and part of the pipeline, and we'll be fine. But 95% fewer tokens, we're just going to need more people that are made aware of that and start leveraging that, and that's where the memory comes in. It's not just models alone.

" I think the best customers for me are the ones that are curious. That means these are customers that are actually very pro-Claude and others like that. I think customers that started 2026 with talking about Claude Code and talking about Claude Cowork, that's exactly where we want to be because while that's a really interesting experience if you're a developer or maybe a sort of an engineering-leaning individual on the product side or elsewhere, I think for the vast majority of the knowledge workers, you're going to need precision, you're going to need safety, and really, you're going to need to be in the surfaces that you value.

You can't abandon all your surfaces. So being able to embrace multi-surface, multi-model, multi-player, these things are going to matter a lot, and I think knowledge workers who respect their craft and respect the diligence that goes into doing their job right, they're starting to see that. So it's really the person who is curious about technologies like Claude are the ones that begin to love our product more than anybody else because they already understand that there's a world where they can work softer, and they just need the right tools.

And when they start to define what it means to say right tool, it starts to come down to the things that I'm talking about, precision and safety and efficiency. Do you think also that maybe we should revisit how some of our organizations are structured around the data that we collect and store? Because I feel like there's a lot of silos, and yet if we want AI to do some sort of end-to-end process, we need to find some way to span those silos in a way that maybe is a little more cohesive.

So we always like to say culture is an output, and let's just say organizational structure is part of culture as well. So it's an output, it's not an input. So companies like ours, we do aspire to change culture, but we have to meet people where they are.

That means we need to understand respective SaaS tools, where people store data, how they interact, where they interact, what surfaces matter to them, where they meet their coworkers, where they meet their customers, and we have to basically figure out where the watering holes are and coexist. And so that's going to be priority one for any company like ours that wants to succeed in today's world is to meet people where they are. We're not here to displace large behemoth tools that have been around for 15, 20, 25 years.

If that happens, and if part of old SaaS goes away as part of this cultural wave, that could easily happen. But I think for us, we have to figure out how to create immense value in a world where we coexist, and I think it's largely possible. So we're not too worried about it, but like I said, if we can be cultural ambassadors here, that's all the better, but that's something that happens over time.

It's an output, not an input for us. All right, folks. You heard it here.

Even in the age of AI, it's still all about the data. Hey, Amit, thanks for being on the show. I appreciate it.

Thank you. All right. ai Leadership series.

You can find this episode and others on our website. We invite you to check them all out. Until then, we'll see you next time.