UT08x09: Building Data-Driven AI Applications with Metrum AI
More episodes and seasons of Utilizing Tech: https://utilizingtech.com/
As enterprises roll out production applications using AI model inferencing, they are finding that they are limited by the amount of memory that can be addressed by a GPU. This episode of Utilizing Tech features Steen Graham, founder of Metrum AI, discussing modern RAG and agentic AI applications with Ace Stryker and Stephen Foskett. Achieving the promise of AI requires access to data, and the memory required to deliver this is increasingly a focus of AI infrastructure providers. Technologies like DiskANN allow workloads to be offloaded to solid-state drives rather than system memory, and this surprisingly results in better performance. Another idea is to offload a large AI model to SSDs and deploy larger models on lower-cost GPUs, and this is showing a great deal of promise. Agentic AI in particular can be run in an asynchronous model, enabling them to take advantage of lower-spec hardware including older GPUs and accelerators, reduced RAM capacity and performance, and even all-CPU infrastructure. All of this suggests that AI can be run with less financial and power resources than generally assumed.
Guest:
Steen Graham, CEO and Founder, Metrum AI
LinkedIn: https://www.linkedin.com/in/steen-graham-0724557/
Hosts:
Stephen Foskett, President of the Tech Field Day Business Unit at The Futurum Group and Organizer of the Tech Field Day Event Series
LinkedIn: https://www.linkedin.com/in/sfoskett/
X/Twitter: https://x.com/SFoskett
Bluesky: https://bsky.app/profile/stephen.fosketts.net
Mastodon: https://techfieldday.net/@sfoskett
Jeniece Wnorowski, Datacenter Product Marketing Manager and Head of Influencer Marketing at Solidigm
LinkedIn: https://www.linkedin.com/in/jeniecewnorowski/
Scott Shadley, Leadership Narrative Director and Technology Evangelist at Solidigm and Director on the Board of Directors at SNIA
LinkedIn: https://www.linkedin.com/in/scottshadley/
Learn more about Solidigm: https://solidigm.com/
Learn more about Solidigm’s AI efforts: https://solidigm.com/ai
Follow Solidigm
LinkedIn: https://www.linkedin.com/company/solidigmtechnology/
X/Twitter: https://x.com/solidigm
#UtilizingTech #AIattheEdge #AIInfrastructure #Sponsored
Transcript
As enterprises roll out production applications using AI model inferencing, they're finding that they are limited by the amount of memory that can be addressed by A GPU and a lot of the other architectural considerations. This episode of utilizing tech features Steam Graham, founder of Metro AI, discussing Modern rag and agentic AI applications with Ace, Stryker and myself. Welcome to Utilizing Tech, the podcast about emerging technology from Tech Field Day part of the Futurum Group.
This season is presented by soy and focuses on AI at the edge and other advanced enterprise IT topics. I'm your host, Stephen Foskett, organizer of the Tech Field Day event series. And joining me today as my co-host from Soy is Ace Stryker, somebody you may recognize from our last season of utilizing Tech.
Welcome to the show, ACE. Thank you, Stephen. I'm very excited to be back and, uh, we're here in beautiful Sunnyvale, California today.
Thanks for having me. Yeah. It's, uh, pretty cool that we were able to get together in person to record this episode.
We're actually here for our AI infrastructure Field Day event. And, uh, Heim is gonna be presenting, uh, this afternoon. And so we thought, thought that it would be fun to record an episode of utilizing Tech right here on the show.
Uh, talk to us a little bit about, uh, what we're gonna be thinking about today. Sure thing. Yeah.
Well, the, the topic du jour is AI, as it has been for, uh, our last, uh, several conversations. Uh, what we're getting into today is really around, uh, the inference side of the AI equation. So we've been spending a lot of, uh, calories there are lately, uh, talking with partners, customers, understanding emerging use cases.
Uh, as we've said, uh, there is no AI without data. There is no data without infrastructure, right? And that's where soddy comes in.
Um, what we're learning is that the, the sheer magnitude of data, uh, on the inference side of things, uh, is just blowing up. There's, um, uh, uh, that's not just our point of view. I mean, there's, there's analyst reports from McKinsey and Tech Insights and others that'll kind of reinforce that notion.
But what we're seeing is people use these models so much and they, they, there's so much, uh, data involved in going in and outta these models during inference, uh, that it's really putting a strain on infrastructure and it's driving requirements higher and higher at a very fast rate. And so I'm, I'm looking forward to getting into that topic a little bit today with our guest. Yeah.
And, and, and I, importantly, uh, memory constraints pay a huge, or, uh, play a huge part. So one of the things you're gonna be talking about today on both the podcast and as well as at the field day event, is how that can be, uh, reduced, uh, through some clever engineering. And speaking of clever engineering, that's why we've got Steen Graham here.
Uh, Steen, welcome to the show. It's nice to have you. Well, Thanks for having me.
And Ste Graham, CEO of Metro ai, and we do a lot of work around building AI agents and, and also model and hardware performance evaluation as well. So talk to us a little bit about, uh, well, I guess let's, let's just go right into it. Um, many companies are trying to deploy AI based applications.
Many companies are trying to build applications that incorporate, uh, enterprise data. Uh, retrieval, augmented generation or RAG has been a huge topic, but what we're finding is that that can place some pretty extreme, uh, stresses on infrastructure and require quite a lot of memory in order to implement in the real world. Right?
Absolutely. And I think, you know, that the modern kind of transition from your historical CPU first data center to A GPU First Data Center has caught a lot of enterprises in this chasm of that making that transition. Meanwhile, the pace of change in AI is presenting all these opportunities for them to move to these modern software stacks, but their existing infrastructure doesn't quite work for it yet today.
And, you know, I think what we've been working on with the team at soddy is how do we kind of look at those scenarios and pave a pathway instead of going, you know, full GPU centric data center with all your infrastructure, how do we kind of give them a pathway, an affordable TCO optimized pathway into deploying those, those latest AI agents and agentic rags stacks for their their use cases? Hey, Steve, can we, just to start, uh, no doubt some of our audience, uh, is familiar with retrieval, augmented generation, what that is, but can you give us a quick summary, you know, how is that different from just selecting a foundation model off the shelf and, and plugging it in and starting to feed inputs into it? What, what is Rag Bayou and why are folks so interested in it?
Yeah, so I think the, the, a simplified view of things is, you know, several years ago with the advent of transformer based large language models, we, you know, you would just use a model, serve it in a chat bot, and then the model would hallucinate. And so obviously for, you know, enterprise quality needs, that doesn't work. And so the, the next kind of iteration of, of innovation was how do we actually put your data close to that large language model?
Um, and notably what sometimes we use a, a vector database or, or a graph database in some scenarios as well. And graph graph could be super useful for, for different different use cases. Um, but you just basically take all your existing data and usually it would be like, you know, for a particular use case and the, the associated tribal knowledge associated with how that use case is solved.
And you're pairing that, that vector database with the language model. So now the language model before it generates an answer is querying, um, you know, high fidelity accurate domain specific information. And that's really kind of the simplified view, you know, of, of the rag, you know, environment today.
Now, what you'll hear in the year 2025 is everybody will say, this is the year of AI agents. Um, and, you know, I've, I've definitely been hearing that. Yes.
Yeah. Yeah. And I think AI agents are kind of, you know, an extension of that evolution where we're actually now allowing the AI to query like, and use tools from the company API calls, you know, to the company's internal CRM system or their HR system, or your supply chain system, or your Jira tickets.
And so now we can, we can do things like take that domain specific information that RAG already has, add an AGENTIC framework on top of it, and then extensively do that where you can actually create a digital worker that can get the job done while a human's not in the loop. And that's where, you know, people are really trying to look for, where's my 10 x, uh, ROI with ai? Turns out it doesn't happen when a person's in, in a chat bot, co chatty, and even a rag based environment, traditional rag environments, have had that kind of chat bot type interface with your own data.
So you're chatting with your own company's data for higher fidelity outcomes, no hallucinations, but you're still not getting the scalability of a digital worker that the AI agents will provide. Yeah. And, and that's, I mean, I guess if you wanna talk metaphorically, I mean, RAG is a great idea, but essentially, I mean, it's, it's metaphorically, it's a librarian with a really great card catalog who can look up things and make sure that things are contained within the dataset and that they're the right things and so on.
You can validate data. Uh, there's a lot of things to love about it, but the problem is it needs to have that really big card catalog or vector database, and it needs to have this huge data set, and that can take up a lot of space. And that's been, uh, something that's, I think, held this back, even though it sounds great.
How do you have that encompass, you know, your, your company's entire corpus of data? How do you Yeah, Well, I think, I mean, most companies, you know, sit on, you know, terabytes or, or petabytes of data. So like step one is, is just basically organizing that data and the high fidelity data.
And there's the, like the last 10 years we've went on a data journey. So for the companies that have transitioned to leadership data lakes, you know, they're in a good position to be able to, you know, start vectorizing that data. And most of the data base companies and data lake platforms are now offering the opportunity to kind of vectorize data as well.
So there's a lot of opportunity and pre-work that's already been done to put the AI models in a position to succeed. But we're still kind of probably in the position where you do want to nail a particular use case. So, you know, curating that data set for the pro the domain specific problem you're trying to solve, um, you know, who's the business unit owner that wants to solve that particular problem is still incredibly important.
So you don't want to just dump everything, um, you know, into a vector DB and start querying right away. You're probably not gonna get the highest fidelity results. And a lot of corporate data is, is greatly outdated, um, as well.
So there's, there's obviously some curation you wanna do to set yourself up for success, but there's a lot of pre-work already already being done today to, to make that possible. But that is the most important part of the journey, having your data organized and ready to go. There's no question there's a lot of data involved in the, in the stuff we're talking about.
Um, I'm curious, when you look at the actual, uh, architecture that, that these models are running on, um, and that this rag data is sitting on, can you give us a sense of, um, you know, is this stuff typically done, uh, in, in memory? Is there a lot of, uh, storage involvement in, in real time as, for example, an enterprise is running, uh, you know, an inter uh, an inference workload and consulting, some kind of rag connected external data source? Mm-hmm.
Yeah, I think, I mean, maybe like just looking at kind of the, the models themselves like, and where, where people are battling right now with GPUs and because we're centering the data center and all our applications around GPUs, 'cause they're the bottleneck, you know, so any rational kind of throughput analysis always says the most expensive component is where you want to have the bottleneck. And so there's a lot of pressure on the GPUs right now. The trade-offs that the, you know, the companies making GPUs have is it's really, really costly to put a bunch of memory in the GPUs simultaneously.
The larger the model, roughly, the better the performance, the more state-of-the-art model, um, that occurs. So there's this really big challenge, you know, and in the GPU memory around fitting big, big large models in the GPU memory. Um, and so that's kind of the number one bottleneck that, that we all, we all face, um, in the market today.
Now, when you start pairing that with, you know, a rag based architecture, which usually we're running the Vector DB on the CPU, and then, you know, we're, we're bringing in all that, that vectorized data, the memory hierarchy kind of levels that a little bit more, you know, more like a traditional, uh, memory hierarchy that you would an anticipate as well. Um, but yeah, the GPU constraints are happening and then you're pushing the workload now into more of a full application systematic software workload that's, that's driving more of the traditional, you know, storage memory as well. When you shift to a rag based architecture.
And AI agents is more, extens is similarly extensible to that where, you know, you're, you're running a lot of application logic for that domain specific use case. API calls that are all happening in more like traditional compute infrastructure that doesn't need to happen on the G-P-U-G-P-U is just focusing on serving that model performantly. And when you say GPU, I'll just point out too that, uh, when it comes to edge especially, but even, um, increasingly in data center AI and cloud ai, uh, it's other types of accelerators too.
I mean, you know, there are, there are definitely acceleration engines out there that, uh, in, in many cases can provide, uh, better service than just a standard GPU, but they have the same constraints that you're talking about. In fact, in many cases, those accelerators have even greater memory constraints. Yeah, you, you absolutely nailed it.
If you look at the companies, um, that are doing a lot of innovation in AI accelerators mm-hmm. And I use JPS as kind of to, to cover the AI accelerator world almost interchangeably. Um, but the, those companies in many cases made probably decisions to, to save on bomb costs Yeah.
And not have memory, um, you know, in their systems that have, they've driven them to make decisions on how they serve models, and then they have to paralyze model to serving these large models mm-hmm. As well. And some of 'em great wrote great systematic software to do just that.
But this is like a big challenge in the world today, especially as you, you apply larger models, but also you apply a chain of thought reasoning. We start to like massively increase the amount of inference calls we're doing, um, by giving the model the ability to kind of think through things more. Um, you know, now you're, now you're really driving a, a significant workload and ultimately a high memory footprint too.
Not to mention people are announcing like nearly unlimited context windows, like 1 million token context windows at all. And, and wanting to kind of like, make sure we sustain that over time, which, you know, the context window versus rag scenario is, is an, is another, another trade off too ace is because with these massive context windows, you're almost getting rag in that mm-hmm. LM application as well.
Um, but the kind of the fidelity of an enterprise application, I think still still likes the separation of a rag environment. A lot of those context windows are mm-hmm. Teed up more for consumer based applications at this point in time.
Right. But definitely muddies the waters quite a bit. Yeah.
One of the things we've been hearing about a little more often, and call it the last nine months, is, um, approaches for, for grappling with the increasing amount of data involved in inference, we've seen, uh, storage vendors come to market and talk about approaches, for example, for offloading some of your rag data and, and accessing that directly from storage. Uh, NVIDIA is just a GTC talking about the key value cash and, and approaches for, um, you know, placing that in storage as opposed to in memory, especially as you, uh, involve more complex models or longer interactions between models and that just grows and grows and grows. Is that a, uh, is that a feasible approach?
Is that, is that something that folks should be thinking about as a, as a realistic solution to the problem of memory constraints as more and more data gets pulled into the pipeline? Yeah, absolutely. I think, um, you know, kind of one of the, the one of the gifts that we have that I think is under underutilized today in the AI world, probably 'cause we're all focused on like this GP memory constraint is, um, you know, disc a NN.
And what that allows us to do with the, the disc, a NN based optimizations we're allowed to offload workloads onto solid state drives that traditionally would be run in memory. And, um, you know, while the indexing time it takes a bit longer to index it, the net results is your queries per second and performance improved dramatically. So that's, I think, a little hidden, you know, hack that you can use to lower the, the memory footprint.
Um, That's a little, I, I wanna, I wanna dig into that one. 'cause that seems a little counterintuitive, right? When you tell someone they can read some data from, from storage as opposed to from memory and you're actually seeing higher queries per second when you take that approach.
Is that right? Yeah. And that's, Yeah.
That sounds wild. Yeah. Yeah.
Well, I mean that's, I mean the, I think the work that's been done, um, within the dis a n working group, and obviously we're spending a little time on the indexing side doing some pre-processing and some optimizations there. But once you've kind of made that, that trade off, when you index that kind of one time-ish trade off when you're indexing, then you know, you've got a little bit of the algorithm based optimization that will give you that, uh, queries per second performance. And it's not like a two x type differentiator, but it's like same level, uh, you know, level performance, you know, plus or minus.
Um, it can even in some dataset be dramatically more. But, um, and then you're looking at same, same level recall accuracy. So just at a trade off of indexing time.
Wow. And so it's indexing time, not even capacity. 'cause I was thinking that it would be a trade off with capacity as well.
'cause the nice thing about storage is that you can have a lot more capacity than you can have with memory. Yeah, that's, that's a fair point. You're definitely using more capacity with that implementation as well.
Uh, but you know, that's capacity that, you know, as long as you're using like the, the state of the art, you know, like PCI based drives, like that's capacity usually have, you know, in the system, uh, and initially as well they're using for other applications. Yeah. And that, and that's where I want to go to too.
So, um, the side effect of making things, and again, even if it wasn't faster, even if it was just not slower, that's still groundbreaking. Right? And the side effect is that you have much more capacity, and so you can deploy applications with much, much more data to support them than you could in memory, even if it was not the same level of performance.
Right. I mean, even if memory blew it away, you'd still run outta memory pretty quickly. And I mean, so, you know, we've talked about soy, uh, you know, you guys have, you know, very big drives, you know?
Mm-hmm. I mean, I remember the announcement of the 60 terabyte drives and the 120 terabyte drives. Uh, I don't think we're talking about having 120 terabytes available to a rag application right now, but are we?
Well, I think it's definitely, you know, in the scope, it really, I think it really depends on how much high value data and enterprise hal, and if they've got, you know, a hundred terabytes of high value data that's for a domain specific application that improves the quality of the output. You know, it's definitely, you know, in the scope of deployability today. That's wild.
Um, and of course, it doesn't just have to be one drive. I'm a storage nerd. I mean, absolutely.
I mean, most, most storage systems use multiple drives, but just the fact that we have that kind of capacity that could be made available to these applications is really, really shattering because there's just no situation in which you could have that kind of ram at an affordable price point if you really wanted to deploy an enterprise application with many terabytes of, of data, you just couldn't affordably. Yeah. Yeah.
So speaking of affordability, one other cool thing we've been, um, having fun with recently is because that kind of core problem that we've, that we've always seen about GPU memory footprint, we thought it would be really interesting to see if we can offload the actual large language model onto the SSD. So this is actually very unique and, um, you know, what, what We've done, and I've heard about people investigating that, that's a really cool idea. Yeah.
And there's some, there's some tools and technologies. Um, in this case we're using a, a feature in deep speed, which, you know, in many cases we use for training applications, but deep, deep speed has some capabilities around model offloading. And so, uh, what we've done recently is we've taken a 70 billion parameter model.
Mm-hmm. Um, which doesn't fit in like a L 40 S based Nvidia, GPU. Um, and we've actually offloaded the model into solid state drives.
And while you don't get the same performance, you know, you couldn't deploy that model at all, you know, and so it gives you kind of model capability based on offloading. So for people that haven't refreshed all their infrastructure mm-hmm. Or they're waiting to get the latest and greatest GPUs, you can actually use this technique to use a bigger model, um, on a lower cost GPU by SSD offloading.
So that's kind of a, a cool innovation. And I think just like disc a and n has evolved over time and performance has improved over time, I think we'll see a level of innovation and performance improvement in SSD offloading as well, that it's gonna be, it's gonna warrant paying attention to, especially as we're, we're increasing the number of, you know, chain of thought reasoning and applications, and all these inference calls are exploding right now. So at some point you have to look at affordability.
Um, and that's, that's a great way to hit a different, totally different level entry point, point on pricing. Hmm. The, um, the model off way offload thing is, is really compelling to me.
It's a really interesting, uh, idea. And I wonder, like, my, my, my gut sense is that that may be interesting to folks particularly, uh, who have interest or, or needs to deploy, uh, AI solutions at the edge, because in a lot of cases we have, uh, more severe power constraints, space constraints. You may not be able to put the latest and greatest GPUs in your Edge servers.
Right. Um, do you see that as a, as a potential play for this where, hey, you can now run a 70 billion parameter model on A GPU running at potentially much less power than the GP would've otherwise needed, and now we can, now we can take that AI to new edge environments? Yeah, it, it definitely meets that, that criteria that you look, when you look at the edge, you think about, okay, we're, we're constrained, you know, from power footprint.
Um, usually there's a big latency requirement at the edge, um, but the existing infrastructure at the edge that that's lit legacy, typically Edge has a little bit more legacy infrastructure, so it kind of checks all those boxes as far as trade offs you'd wanna make at the edge. Now, it might not be a 70 billion parameter. That might be a technique you use on a 7 billion parameter, or if you're deploying on some really legacy infrastructure at the edge, it might be a 700 million parameter model.
So I think that it scales down to kind of the, the right footprint, uh, for the edge as well. I wouldn't, um, ignore kind of the enterprise, uh, cloud applications here as well, because what's happening with, um, with the transition from chatbots to rag to AI agents over time is AI agents are running autonomous of human intervention. Um, now you can always, you know, human in the loop it, but what we want our AI agents to do is they want to, we want them to be our digital workers that are working for us while we're asleep, we're hanging out with their family and enjoying life.
And then we wanna come back in the morning the next day and see the output, all the reports and documents that, you know, the AA agent conducted for us while we were enjoying some great sleep and some great family time. And that can be done on a batch based processing node. So we don't need to like, you know, get the most high performance, uh, GPU in that scenario.
Um, we can kind of use what MacGyver, whatever we have available today, and, and leverage that and then, and deploy it as well for batch based workloads. So I think AI agents offer us a great opportunity to do some trade offs in, in latency. So like real time tokens per second, little less important for AI agents, um, depending on the particular workload.
Yeah, we've been hearing that as well. And, uh, with RUM and some of the research that we're doing, in fact, we're starting to see people talk about using, um, CPUs for, especially for agent ai, for the same reason, because it's sort of an asynchronous workload. Um, also because there's a proliferation of CPU course.
The CPU cores have a lot of specialized functions. In many cases, they're actually getting specialized AI instructions, and because they have greater addressable, um, memory in many cases than GPUs or accelerators do. So CPUs can look increasingly attractive for this.
And especially in, in, and with many of the things that you're talking about. I could see a lot of that going hand in hand with this CPU trend as well, uh, wanting to use more storage instead of memory to reduce the overall bill of materials to deploy some of these agentic applications. Because essentially, um, you know, you kind of take this to its logical conclusion.
We could see sys systems running, um, agentic applications on conventional servers with, you know, a reasonable amount of memory and a reasonable CPU and a reasonable amount of storage, thanks to the fact that we now have capability to use that. Is this a vision that you would share? I think yeah, absolutely.
I think there's a lot of opportunity to take that historical data center architecture and run AI agents on it, whether it's CPU, and we actually have a, a number of AI agents that run a hundred percent on CPU U mm-hmm. New GPU required. Now, that being said, I think some of the, you know, older GPUs, fabulous performance still, you know, for those type of workloads as well.
So I wouldn't, I wouldn't try start transitioning a hundred percent to CPU in all case scenarios. But for those batch based workloads where you're, where you're fine with a little bit more latency, um, it definitely works. So those existing data centers, I don't think need to be totally retrofitted today in all scenarios, you know, for a GPU centric architecture, um, we can make use of them for, for deploying AI and AI agents.
Uh, one, one more question from Mete. Um, I'm curious, since you're our expert on agents here, and we haven't had one before on the podcast, I want to pick your brain on this. Um, let's say you've got, you've got a model, you've connected it to some rag data, right?
And, and what it can do without identifying is that the right term without giving IT agency, is it can, it can give you insights and advice, right? And then, and then when you give IT agency, you're now connecting it to other tools and systems and allowing it to, to take actions on your behalf. Does the, does the act of giving that model agency have significant, um, uh, repercussions in terms of the amount of data generated or used?
Like you, there's a lot of data clearly involved in training a model. Mm-hmm. There's a lot of data in RAG potentially, and then whatever systems you're connecting the model to have their own dataset, which presumably existed before the connection, but are, is there, is there a big impact to incremental data simply by virtue of making a model agentic?
Yeah. Is that something you guys have looked at? Yeah, I mean, It absolutely, I mean, we see this in, you know, our AI agents that have, you know, chain of thought reasoning.
Um, and the more autonomy you give these agents, I mean, the, the human is the bottleneck in the scenario. So if you, the, the better you design the workflow, the more API calls that that agent can do, the more tools it can do, the more, you know, autonomy you can give it and problems it can solve, it's just massively explodes the level of data, uh, being created. I don't wanna like characterize that data as synthetic data, but I would say, you know, kind of non-human generated data footprint mm-hmm.
You know, is, is massive. And I think, you know, obviously I think we're, we're kind of at, at the era where the non-human data generated footprint is, is gonna be, you know, much greater than the human generated data footprint, uh, based on giving the AI autonomy as well. And also kind of the advent of modern robotic tools are very data intensive.
So I think, I think that that transition point's gonna be a very interesting transition point. It's great for the storage business, so ASU should be happy. Um, and it's also valuable synthetic data, right?
And, and this is kind of what we're struggling with with AI right now, is, uh, you know, there's, you know, the open kind of web data we've certainly ran out of as far as, you know, training models. So then there's a path for synthetic data. There's more human labeling, more reinforcement learning, and now we've got this whole category of, you know, AI agents doing chain of thought reasoning with their data footprint.
And so, you know, all, all of that kind of non-human generated data is gonna be really important for the fidelity and the future of AI models. Because if that's high quality data, if if we're generating high quality workflows, then that'll be super helpful. Obviously, if the workflows don't work or the, um, model's failing, then maybe that will, you know, devalue that mm-hmm.
That type of data relative to human generated data at this time. Right. So it's kind of a really exciting to watch that play out.
Cool. Yeah. Yeah, it really is pretty interesting.
What's, what's happening here. And I, and I'm actually excited because, you know, generally people, uh, there's sort of a, a thought that, uh, AI applications require absolutely cutting edge, high-end hardware that's really expensive, that consumes tons and tons of power that just, you know, basically there's a lot of negatives around ai and, and many of these negatives are true, but the industry is absolutely working to address those challenges and those criticisms. And in many cases, we're going to see applications being deployed on much more, um, restricted or, uh, modest hardware with, at, at a much lower price point.
You know, and, and I think that all of this means that this, this technology can have a bigger impact than we might've assumed, simply because it doesn't necessarily require the, the, the biggest, baddest, hottest, uh, hardware to run on. You can run it, um, more approachable on, on more modest hardware. So all of these things, I think, go in that direction, and I think that that's a, that's a positive for all of us.
So thank you so much for this conversation. I guess, um, what last thing would you wanna leave our audience with? What's your summary of, of this message?
I think, uh, for me, I think there's, there's a lot of ways to deploy ai. And I think the, the innovations we've seen in the last year are, you know, on affordable deployment of AI are probably 10 x, um, what we saw in the last 10 years. I mean, it's just an incredible pace of change on driving affordable models.
And I think, um, what's most important is, you know, designing the right business workflow, you know, for these autonomous workers or AI agents. And then, you know, figuring out the deployment methodology. There's so much innovation happening on, on affordable off the shelf hardware, or even affordably deploying a state-of-the-art hardware, um, that I wouldn't let that get in the way of, you know, your company's innovation.
Excellent. Well, thank you so much for joining us. It's been great having you.
Um, before we go, where can people continue the conversation with you? ai. Excellent.
And, uh, ACE, it's been nice seeing you again. As I said, you were one of the co-hosts last season. Um, so check out utilizing Tech season seven.
Where else can people catch up with you? Where have you presented recently? Or, or where are you gonna be?
Uh, boy, oh boy. There's, there's, uh, a lot going on at Solid I these days. Certainly.
com/ai, where, uh, we hope to be featuring some of Metro AI's excellent work in the near future. Um, we'll be at, at conferences all summer long, so, so keep an eye out at, at all the big ones. Uh, for now my head is spinning, uh, with all the implications of what scene is talking about.
And so I need to go have a, have a lay down and kind of chew on some of this stuff. But really, really appreciate you being here, Ste. Uh, uh, I've learned a lot and, and thank you.
Yeah. And, uh, me as well. And, uh, I will point out that, uh, by the time you watch this episode, the solid IME and, uh, Metro AI presentation will be published on YouTube.
Just go to YouTube and, and search for solid IME and Tech Field Day. And you'll find that, uh, that was part of our AI infrastructure Field Day event, which happened in April. Uh, we're also gonna be doing an AI Field Day event, a cloud Field Day event, and we just announced another AI infrastructure event later in the year as well.
So check out the Tech Field Day website for more information about that. So thank you very much for listening to this episode of Utilizing Tech. Uh, you can find this podcast in your favorite podcast application, just search for utilizing tech, or you can find us on YouTube.
If you enjoyed the discussion, please do give us a rating or review or a comment we'd love to hear from you. Uh, this podcast was brought to you by Solid I, as well as Tech Field Day, which is part of the Futureum Group. com, and we are present on the socials.
You'll find us on x, Twitter, blue sky, and Mastodon. Uh, just search for utilizing tech. Thanks for listening, and we will see you next week.