AI Data Management – Techstrong AI Podcast EP58

Hello, and welcome to the Techstrong AI Podcast. I'm Amanda Ani, and with me today is Rajan Goyle. He is the CEO of data pgo.

How are you doing? Very good, Amanda, nice to be here. Yes.

Can you share a little bit about your company? What services do you provide? Well, Amanda of absolutely, uh, we are a software only company building a data platform for, uh, modern infrastructure, which means that we are modern data center infrastructure, where we are transitioning from ho homogeneous computing platform to heterogeneous computing platform, or what I call domain specific computing, uh, which require, that means that the servers that you get in the data centers today are not only CPU only servers, but you have CPU plus something else, primarily GPUs, FPGAs, TPUs, and other XPOs.

So what we are building is a data platform for this modern infrastructure to take advantage of this, uh, newer breed of accelerators or crossing elements. And in the process, make the, uh, uh, platforms efficient and scale, uh, and also handle the complex of complexity of the data, which is, uh, growing from not only volume point of view, but also from structure to tructure and semi-structured data. So as the market is shifting from, um, uh, in the data space and the infrastructure space, uh, those are the two kind of key irreversible trends.

So Data Pego is positioned well to take advantage of those two trends and, and offer a product, uh, which will help customers reduce their cost, PCO, uh, run their workloads faster and more efficient and scale. So that's what we are building, Amanda here. Fantastic.

So our topic of the day is AI data management. So you seem like, uh, you have a lot you can share with us today, and we're gonna start with, from your experience, what are enterprises struggling with when it comes to AI data management? What, what are the issues they're coming to you With?

So if I were to say there are two or three different problems, Amanda, that we, when we talk to customer, they highlight one is obviously the cost of the data platforms is untenable right now, right? So in terms of, because the data has grown so much, application demands are, uh, kind of growing, and as a result, um, both the business users and IT departments are struggling to keep up with that and while keeping within the budget. So that's the TCO is one number one problem in everybody's mind while they want to now go into or grow into more ai, uh, related problem, because now everybody has brought or bought this argument that, um, you could have a single unified data lake with structured, unstructured, semi-structured data stored into one place, but in order to extract insights from that, you need a lot more energy, a lot more computation power, and that requires more, more budget and more dollars, right?

So that is the, uh, the balance there. How do we keep your current bread and butter, uh, going and scale and so on, and why, how do you venture into the new things? So that's kind of more business point of view.

The technical point of problems are obviously data is huge in the, in all these enterprises, but which one is the right data to influence your, um, uh, in influence the models and your, uh, or, or, or the workloads? So the context specific data, how do you clean it? How do you extract that and prepare the right data for the right, uh, um, your jobs is a another problem.

So a lot of, uh, customer that we talk to, they spend or struggle lot of time, uh, making the data prepare for their AI workloads. And the third aspect, if I may add, Amanda, is the availability or non-availability of a unified platform. So the way the industry is right now is that you have, people have solved the problem or they know how to solve it, and it is now becoming almost easy to use.

Like how can you do more easily? But that's when you worry about efficiency, the scale, and so on. So we are just entering that phase in the industry where people want to worry about, Hey, do I have a unified platform which is built to scale and efficient and so on to handle my newer workloads and new kind of data?

So those are the two, three problems that I highlighted. If I may summarize the business cost point of view, how do I extract data context specific data for my AI workloads? And third, what kind of tools are available for me to bid that job?

Right? So those are the three problems that I hear customers facing, uh, going forward. Yeah.

So let's talk about, let's talk about those one by one. So with that first, um, issue, the return on investment, the budget issue, what advice do you have for business leaders, um, to, to help them out, um, to hit that return on investment, um, during this change process By data pego? No, no, I'm just joking here.

Yes, no. So it is a, the reason for the cost, uh, uh, is one is fundamental that the problem is becoming bigger and bigger, right? So there is no escape from that, but some of the practices have been, or the platforms, the tools available are still designed for older, uh, generation of that technology.

So the modern technology, so you net net we are leaving the gains and efficiency on the table, so upgrade or refresh your data platforms, which are designed for modern technology. So that's kinda one way. And also over there, um, you, the enterprises have to be careful that pick the technology which are less disruptive.

So that, uh, uh, is there any product which still allows them to use the modern infrastructure without disrupting the ecosystem, without changing their workloads, without changing the data format. So without moving the data, and that's what, uh, as an example, data pego, uh, does that kind of product offering where no change in application reduce costs, but in, if you talk in general, yes, so there are, uh, techniques available now or technologies available, we just need to keep up with that. That will save a lot of, uh, TCO make it efficient and so on.

And the same budget can be applied to the, uh, to your newer, uh, uh, AI driven workloads as well. And so that is how, that is one answer, Amanda. Let me pause if this answer your question.

Yeah. Okay. And so moving forward then, managing that data making, pulling in clean, organized, uh, data, do you notice that they seem to have a, a problem with the unstructured data?

And what are, what are some of the things causing those challenges? It is true that, uh, you know, that, uh, at surface you feel like more data you have, uh, uh, and more. It should be useful so that you can intuitively think about a direct relationship with that.

But most importantly, what's important is that, uh, can you extract the context specific data from that, even from the unstructured data you have, which components, which fields or what portion of the document or the image or the PDF table or whatever is more relevant for your job, right? So that you have to, uh, uh, apply the, use the right tools to extract that information so that you're not confusing, if I may use that word, your models or, or, or the systems downstream, uh, from the data which you don't know. So that is a, that is a art and science because you need to know the, the, the domain specific knowledge, what, what the application you're solving and what data do.

So that is where the people have to be able to extract it. And that covers the third part, which is that you, that means you need to all have it, right tools to, to, in order to do efficiently. And it's an itrate process.

You can't just say, I, I did it once and then I get the right model or right, ai, uh, workload done. So you may want to learn an itrate. So if you make the cost of data processing cheaper or economical, then people would not doing a shooting in the dark, they will be able to do a lot more, uh, I would say informed decisions in the IT process and only the user right data to do that.

So that is the, just covering both, uh, the second and the third part that giving a context specific data requires processing of it. And if the processing has the right tools which are economical and efficient, then you can have a loop, i a feedback loop there and then, uh, and, and, and have a better data process. Your AI workloads, let me pause here, Amanda, if this makes sense.

So What, what parts of traditional data architecture aren't holding up and and what advice do you have for, for companies? I think, uh, two parts to that. One goes back to I come from my, in my previous life from like building infrastructure or a processors and storage and networking system.

So meaning that I always look for what, uh, obviously you can just, if I pause my thought there, you can, you can push the problem to the practitioner that hey, be more efficient and, uh, use best practices and, you know, things like that, you can pass the problem to the user itself. But to be fair, uh, it is, um, it is more or less, excuse me, for that, uh, it is more or less on the, uh, on the even an infrastructure provider, uh, that are the new platforms designed to take advantage of this shifting infrastructure, right? So hence you would want to have a platform which is designed for taking advantage of GPUs or, uh, other accelerated processing elements available, which makes it easier and economical to use.

So that's kind of how I look at it. Um, then, then makes it easier, uh, uh, for, for users rather than passing a problem them. Yeah.

So I feel like I'm reading more and more about new data centers popping up all over the place. Mm-hmm. Um, so in your opinion, um, do we have an issue that we need to consider about energy use, uh, and the amount of energy available for, for processing all this data that's needed everywhere?

Yeah, absolutely. I think those are related problems in the sense that, uh, energy for power and for cooling and the space all are related. And, uh, this is a, a, a very important problem at scale.

Uh, and the data processing, if I, if you hear, uh, other, um, uh, leaders, even like Jensen, Huang has mentioned from Nvidia, CEO, that bulk of the cloud spend, uh, or the compute spend is happening on the data processing. Uh, so this translates to how much power, how much cooling, how much space the data center is used for data processing. So the, if there is a direct correlation with that, so if you make the, uh, data processing efficient, uh, and you can reduce the footprint of the number of servers or racks required to do data processing, that translates to your cooling and power budget, whatever.

So there is a direct relationship with that. And as the data is becoming more and more complex, I would say it requires more energy. I mean, to, to give a simple example, if you have table data and you're sorting it or extracting some filter, doing some filter on that, versus you are doing on a PDF or in images, you did lot more energy, uh, and order of magnitude more energy to do that.

Uh, so as the data is becoming complex, it requires mongy. So that means, uh, the systems designed in previous decade, uh, on structured data are not suitable to do that. They will not be able to keep up with that, right?

So you, you do need to use the power of GPUs or other specialized engines to be able to process this modern data, right? And there is a clear cut example of GPUs being deployed for training and, and now inference workload and without GPUs, world will not be here where we are because the amount of data and the processing power required to train large models is not even practical to do it on CP only system. So, so it's a similar pro, uh, argument for me to be made for data processing as well, Amanda.

Alright. Well, if there was one key takeaway you could leave our audience with today, what would that be? I think, I mean, one is that it is an exciting field.

Um, it is a data is the new oil, uh, even though we have heard that cliche many, many years back. But now data is indeed important. Uh, companies or enterprises can stay more competitive, become more efficient, business friendly if they rely, make decisions based on data.

That means if you are gonna make your operation data centric, if I may use that word, you need to have the right tools and right infrastructure to extract information out of data so that you can, you're not frustrated by by that. And in order to do so, you have to choose wisely your next generation data architectures, uh, how you store data, how you organize it, how do you process it, what kind of tools you use, and what kind of teams you build to to, to do that, right? So they'd say, uh, you don't have want to disrupt it, um, in day one, but if you think systematically and apply the right infrastructure tools as data pillow, if I may use that, uh, sentence, we are building, providing those, uh, platform for the, uh, modern workloads and modern infrastructure.

So those kind of tools can enable, uh, the enterprises to stay up to date with this, uh, with this modern world now. Yeah. All right.

Well, thank you so much for coming on the show and sharing your insights with us today. It was my pleasure. Thanks for having me, Amanda.

And thank you to our audience. Stay tuned. There's more.