Business Moves, AI Pros and Cons and GPU Optimization – Techstrong AI Podcast EP28

Hello and welcome to the Techstrong AI podcast. I'm Amanda Ani, and with me today is Mike Vard. How are you doing today?

I'm doing great. Always happy to be here. Same, we have a long lineup of topics today.

So let's start with the open AI exodus. We have yet another person defecting from open AI This time it's John Schulman. He said he's leaving for Archrival Anthropic, and as we've said, lots of people been leaving.

There's been so much craziness in this area. So can you share what's going on here? Yeah, I'm not a huge believer in the the great man theory of history, so I don't really buy into this whole notion that, you know, one individual kind of drives everything and you, you know, the way you might read and say the business press.

I think sometimes we forget that these companies are made up of many people who step up over time and um, so, you know, we'll see. But it's pretty clear also that there's just a lot of, uh, dissension floating around opening ai and I'm not quite clear how that ultimately impacts their progress. And it seems like arch rivals, such as philanthropic are gaining ground and you know, the folks are voting with their feet.

I guess in some instances it's hard to read the tea leaves here, but I would just kind of make the note here that says, um, I wouldn't bet on any one of them right now over the others, I would kind of say, Hey, maybe we should just keep track all of this and um, start to treat these things as interchangeable. I mean, a lot of times they're exposing an API that you're just calling. And from a certain level, a lot of applications, that's all they need.

I mean, if you want to go build and customize your own LLM using their stuff and put it in, um, some sort of enterprise app, great. Uh, that may require a little more lock in. But, um, I feel like, uh, maybe we're just too much wrapped up on the L LMS and not enough on the, shall we say, the use cases or, or how we're building the apps in a way that we're not so dependent upon any one LLM.

So maybe the drama doesn't need to be nearly as high. Yes. And and I would like to point out too, it's not like this is the only company this happens to.

This happens all the time. I think, you know, I'll say it's just business people leave companies and go to other companies all the time for better offers or perhaps they like what's happening in that company a little bit more, like you said. Yeah, there's a lot of burnout in startup companies too, and people get tired of the, the stress and the strain because a lot of the times early on before anybody heard of you, you were working with a limited number of resources and you're trying to make something happen and that takes a toll too.

Right? Absolutely. com that shares a survey and it says that AI is rapidly expanding and that it's imperative to adopt AI for the future if you wanna succeed.

So what are your thoughts here? Well, we've talked about this in the past where there's a lot of, shall we say, uh, chagrin right now or about the amount of revenue that's being generated by all these AI startups. And yet if we go look at the average enterprise, I mean they may not be consuming massive amounts of compute cycles yet, but they will be and they're clearly, uh, investing and they're clearly doing the research and looking for the use cases that make the most sense.

Um, the issue is it's not always readily apparent how something that is probabilistic and you know, makes the best guess at something can be inserted into a business process that always has to be deterministic. And this is gonna take a little bit of time and maybe we'll come up with some additional use cases for things that are best guess and that'll be helpful. But, um, I think we got it in our head somehow.

Rather this is gonna be a magic switch that we're just turn on and be know revenue dollars would be falling like crazy. And, um, it's difficult to say that that's gonna happen at this point. I think it's gonna have a profound impact.

I'm just not entirely sold on the notion that this changes our way of life tomorrow. Yeah, absolutely. I mean, AI is just one tool and certainly it's a great one if you have a use case for it.

And I will say if you have a use case for it, these articles do explain why it's imperative to go ahead and implement. However, again, if you don't, don't just try to jump on the AI bandwagon Mm-Hmm. And be certain somebody out there is experimenting with something that's gonna impact your business model and they're probably gonna be using AI to drive some portion of that.

And I will say if you're not in the game today, uh, you know, try to get in the game a year from now will be a lot harder. Uh, you need to develop the muscle mass and the, the skills and the expertise required to play. Um, so I don't think you can afford to kind of sit on your hands and just go, uh, we'll wait and see how all this turns out.

I you gotta be in it to win it, as they say. Yeah. But there is something to be said, I guess from the companies who are sitting back to see how it plays out for a little bit and to develop those use cases to determine what solutions they need to solve before they implement.

Right. Look before you lead, for sure. Yes.

All right. So then we're gonna talk though about existential threats because while we talk about the amazingness of AI at the same time we have so many articles, one on tech strong AI and one on Security Boulevard that uh, ask is AI a friend or a foe? It seems to be the debate since it came on the market.

Some people say it's a big threat, other people say it's the best thing since slice of bread. I think it's a little bit of both. What are your thoughts?

Well, that knife we used to uh, cut that friend you just referred to probably has two sides to it. So yes, it's definitely gonna be a threat and it's definitely gonna be a help. Um, it will be a threat because the bad guys are gonna use ai, they're gonna create more malware faster, it's gonna be better malware, it's probably gonna be more targeted malware and, and they're gonna try to hijack our LLMs and steal our sensitive data if not the entire LLM.

These are all true facts and probably already happening somewhere. This one friend of mine once said, the thing about cybersecurity is if you can imagine it, it means somebody else is training. Um, but it's also a friend.

Um, people who are in the cybersecurity space especially will benefit the, I would rather be in cybersecurity with AI than without it. That's for sure. And I think most job functions are gonna be similar.

I think at some point we're all gonna be like, well yeah, I mean, why I wouldn't take a job that wouldn't give me access to AI tools to do it. That would be crazy, right? I mean, why, why make my life harder?

I'm hoping that, you know, we always think about these things within the context of our jobs per se, but theoretically there could be more companies in a given space that the cost of producing something is lower. And uh, we might see all kinds of entrepreneurial efforts from, uh, 'cause things that used to require, you know, a hundred people to do might be done by five. Well that's the case then maybe there'll be instead of 20 companies doing something, there'll be a hundred.

'cause the cost of delivering that service profitably will be different. But we can all kind of play in that space in a way that's more aggressive. Um, so I kind of think about it as, um, it's most definitely gonna be a friend to everybody on some level.

It's just a question of how are you gonna use it and where, and um, how you're doing your job today might be different and where you're doing your job tomorrow will most likely be different and that's okay. Yeah, absolutely. And I think with any new technology is a potential new threat using that technology.

But as they say, fight the AI with the ai. There you go. All right, so now we're gonna talk about AI at the edge.

Um, we have an article on tech Strong AI says, taking the brains to the brink running AI on IO OT devices with edge computing. Combining AI with edge computing is beneficial. Uh, and then it goes on to explain all the different ways that it can help with those edge devices.

So can you share a little bit more? Yeah. This is happening and we're already seeing in the mobile space, right?

Um, we're seeing Apple in particular talk about running algorithms right on the hand l so I don't have to do a round trip to the cloud and the IOT device is the same idea. Um, do we need GPUs everywhere to do that? Probably not.

We need to training the AI model somewhere and then we need to have some sort of, uh, processor unit capable of running the inference engines out of the edge and we need to update those inference engines. Do we need data scientists to run all those inference engines at the edge? Probably not.

Um, I think we're gonna see all this stuff move closer to the edge simply because we need the model to be where the data is and where it's being analyzed. And all this round trip behavior to the cloud is probably, you know, the laws of physics say that that's just gonna be too slow. So as I kind of think it through at the end of the day, um, that AI of inference engines, you know, they're not like the AI models in the cloud that are, you know, patent bytes of data.

They're much smaller. They can be run in terabytes in some instances and they can fit on the latest and greatest processors optimized for the edge and might argue that um, AI might have a bigger impact at the edge than anywhere else. Yeah, absolutely.

We've seen quite a few articles on this topic and uh, it does seem like a solution more business leaders are looking to I, yeah, sometimes we get it in our heads that something is like massive when it's not and we have our little prejudices about how things are gonna be built and structured turns out, but they're usually not the case. And over time, the amount of compute engines that we need, I mean going back to the mainframe, everything that's more efficient over time, just because something requires a massive amount of GPUs and CPUs to accomplish today does not mean it's always gonna be the case. And it certainly doesn't mean the inference engine's gonna require it.

I think we just need to open our minds a little bit. Absolutely. Well, with that, we're gonna talk about the last article which says, um, uh, this is actually an interview that you had and it's about enhancing GPUs with, is it fission technology, Fon technology.

Tell me a little bit more about this interview you had. Well, this is, uh, about a technology that essentially puts a, a cache of memory between the uh, application and the GPUs. And the idea is to make the usage of the GPUs more efficient.

And in some instances that can even be outside of the cloud. It could even be theoretically closer to the network edge. And, and the point here is that we're obsessed these days about the fact that we can't find GPUs and they cost a fortune, but there's other ways to skin this cat and we can put different technologies in place, whether it, before we mentioned um, uh, inference engines, well those things don't always need to run on GPUs.

They can run happily on a variety of different classes of processors. And um, so I think, you know, that's just one way to get after that. But the other instance is, well, okay, so let's say I do need GPUs and I'm using GPUs because, well, they're a little bit better at paralyzation than other processors.

Okay, that's true. But I still need to maximize the utilization rates for those GPUs and putting a lot of more efficient memory architectures is one way to go after that. So that's what this fellow is pointing out in that interview.

And I think, um, we have forgotten hardware over the years. We kind of didn't went into the cloud and we thought about it more as well. Yeah, that's something the cloud service providers are doing and I get this instance and I don't have to think about hardware anymore.

I think AI is about to make us think about hardware a lot more because we're realizing how expensive it is. And this is just one example of how maybe to reduce those costs. Yeah.

And at the foundation, at the ground level is the hardware. So it is important. Yeah.

And I just don't think, um, the economics of AI kind of work with just putting GPUs everywhere, just not gonna happen. So we need to get a little more innovative in our thinking and understand, uh, you know, as the saying goes the right horses for the right courses. Absolutely.

Well that brings us to the end of our podcast articles to share today. So what are your final thoughts? Um, it's easy to get discouraged sometimes with AI because we all walked in with this sense of irrational exuberance, shall we say it?

But now we're on the trough of disillusionment. Okay, great. Probably gonna be on multiple cycles of that over the next few months.

We're gonna go from, you know, woo, we can do this now, this will be great. And then we'll be like, oh, this is hard work. None of this stuff is easy.

It just doesn't magically happen and it requires a village of folks. Really, when you think about it. I need not only data scientists, I gotta have data engineers, I need some developers, I probably need a DevOps team, I also need the cybersecurity people.

All this has to come together in a way that, um, operationalizes this stuff. And I think that's where we are. Time, patience, and perseverance.

Exactly. Alright, well thank you to our audience for tuning in. If you didn't cash last week, sits there for you.

And then again, let us know what are you most interested in. Go check out these articles that we shared and uh, send us your feedback. Have a great day.

See you up.