The Battle of the AI Models ChatGPT 4.5 and Claude 3.7 – Infrastructure Matters EP73

This is Textron tv. Welcome everyone. We are back again for Infrastructure Matters, episode number 73 with my buddies here, um, Don Iff, and of course Keith Townsend, who is coming in from Tennessee.

You got your, um, your, your jet stream or whatever that thing is called. Your, The, the Airstream is, uh, Airstream parked in the middle of Forest. I, I, I posted earlier this week that if you wanna learn anything about site re reliability engineering, try living off grid for any period of time and you'll, you'll, you'll have a crash course master's level and keeping a website up.

There you go. Are you on a 5G or are you starlink? I am starlink.

So our platform will cool it, it'll make up for it. I'll break in and out in our real time recording of this, but it records locally and uploads it, so you know, it is much better than nothing. Yeah.

Whole lot better The wandering man out in the wilderness. Okay. So guys, we've got quite a bit to go through this morning, so I'm gonna jump into it.

Um, first a short little piece on some earnings of reflections. We've had lots of earnings this week. Um, I just spent the time this morning going through NetApp, pure and Nutanix.

I haven't gotten to Dell yet, so we're not gonna talk about that one. They may know, um, and kind of where they were at. All in all, um, everybody's had good quarters.

Pure has an outstanding quarter. Um, pure storage. Um, they've just kind of blown out their money.

The, the revenue, 12% year to year growth. Um, the subscription business is up 21%, just all kinds of really good numbers that are coming from them. And, um, but, and then NetApp had a slight miss, um, because of some delayed of some transactions, but they were still, they were 2% year to year, um, up in terms of where they're looking at and, you know, still decent quarters.

Um, and tonic, of course is up as well. Couple of common themes that went through all of them. Um, they're experiencing the softness in the market over in Europe.

So there is some softness going on there because there's certain uncertainty in the US is probably carrying it all. Um, the second piece, they all commented or had questions to them about, um, VMware, um, trans, trans transitions. All of them have had, of course, Nutanix is double downs on that.

That's kind of a key piece of their market. Um, and that is happening, but it's slow. I mean, it's is happening.

It's slower as you would expect because you've got licensing issues, you have hardware issues to migrate. Um, although they're coming about how fast you can move up in the cloud, like with an AWS, um, just because there's no hardware stuff there. And the third one is an ai.

And we're gonna get into that. We're, since these guys are all storage people, and they're primarily storage, that has to do with the enterprise. That's a slow moving ship, ship right now, or train, or whatever you wanna call it.

Um, because most of that work is right, still going into the hyperscalers, um, the maybe AI factory kind of companies that are putting up CSPs, et cetera. But they are seeing, you know, definitely, um, NetApp has seen a couple of big data lakes being put into place that, you know, people are taking where they've already got this, got their environment and expanding it. Um, a little bit less conversation about this from Pure.

So that's kind of a bit on the transactional kind of piece. And I will stop there, and then we're gonna go on to some other things that are even more interesting. So, okay.

Um, let me go on to my, my, my partners in crime here because we have a bunch of things going on with chat. 5, Claude pre seven, and, um, Google giving away Gemini codes. So, um, Keith, why don't you take it off first and then you, and then Diane can, uh, come into it.

Yeah, it's been a busy week for ai, uh, product and model development. 5 hit, uh, yesterday of this recording. Mm-hmm.

5 for quite some time. And I have to tell you that some of the early assessment from the AI experts is disappointment. 0 to four.

Oh. So, uh, Diane, I'd love to hear your thoughts on both of these. Yeah.

Well, it's, um, it, it's the, the last big release between the, uh, uh, uh, before the much expected GPT, uh, five and, uh, uh, opening IC and they, they put a lot more pre-training work into this to find more connections, uh, between all the data. 5 than it does in DC car one. Um, that's two orders of magnitude.

Uh, and, and, you know, can they be profitable on this? Uh, we'll see. Uh, right now, uh, only GPT, um, pro users can use this.

Uh, it's available today, unlike Alexa Plus, which we have no idea when it's actually gonna ship. Uh, you have to get on a wait list for that and some weeks in the future. Uh, but, uh, plus users, which is, uh, you know, the bulk of our subscribers will have it next week, so we'll get more insight into it.

Uh, it's just, it's, it's an important bump. Uh, it still keeps them, uh, at the top of the leaderboards. So it is a highly capable model.

It's not that it is, uh, that it isn't. Uh, and, uh, it has somehow, somehow they found a lot more training data to throw at it. So they must be licensing, they must be spending, uh, to get access to data sets that, uh, that are not openly available.

Uh, so, but we'll see a lot of testing now that it's in people's hands as of, uh, yesterday morning. So it's gonna be interesting to watch. Well, okay, so how does that work?

If I'm gonna license training data because it's not publicly available, and now I've trained my, my model on it, it's now publicly available, isn't it? But through the license, right? So they, they've paid to make it available as the argument, right?

So, uh, uh, hopefully the licensees know what they're, they're, uh, they're up against. 'cause you can, uh, you know, the techniques are emerging to extract almost the original dataset if you know how to query, uh, the, the, the model to get that out. So, we'll, we'll see how that, how that proceeds.

It kind of feels like once it's in the wild, it's in the wild. I mean, yeah, that you've put it out there and, you know, that's the big issue about why, Well, this is what called what's, what model distillation is all about. You can actually get all the information out of a model, uh, in a way that you couldn't, like out of a search engine.

No, it's very interesting. Okay. Okay.

Well, what about the Google giving away Gemini. Gemini? Yeah, so the, a lot, a lot of these models are, uh, kind of put into action, probably the biggest areas in AI coding.

I'm looking forward to having a really great discussion with Brian Lau, who's a principal, senior principal architect, or, uh, developer at Amazon on, not on the AI side, but he's been a big proponent of using AI in development. 0 code assistant. So I can, uh, just light up my favorite IDE and this code assistant is fully free.

There's, I think, some ridiculous token limit that the average individual developer probably shouldn't reach. The enterprise version is still need, still needs to be licensed for multiple users and work groups. 0, uh, uh, code assistant is actually pretty good.

So, uh, it's an amazing, it, It's top leader port, right? Yeah. 0 is a super capable model.

Uh, it's pretty brave of Google to give a, a huge number of completions away. Uh, so you can do a lot of work for free. Uh, and, and I think it's really smart to go after developers, 'cause they're the king makers.

They made AWS what they are. 0, get in the hands of lots and lots of developers, uh, 'cause it's gonna be the cheapest option for, for most to be able to get their hands on a really powerful coding ai. So the assumptions that if I'm gonna code with Gemini, that code is gonna run on Google Cloud.

Yeah. And I, I think that's a strong correlation, Cameron. We've had Google at Cloud Field Day a bunch of times, and the delegates were always surprised at how well integrated the platform is with the model.

So you can use Google's runtime, Google, GKE, it's various serverless platforms to actually call Gemini. And it's really easy to do. com or Google whatever the website it is and use it similar to how we will use chat.

TP Google has made it much a much better developer experience than maybe a end user experience, if that makes sense. I think that comes from, they're being, they were kinda like third to cloud, but, uh, and that means they learned a lot of the lessons from everybody. And they have the, I think, the best cloud architecture, um, you know, product architecture of them all.

Um, uh, uh, you know, it's, it's the most modern, just there's, it's just not as used as the other two because it, you know, they, they, again, they, they came a little bit later, later to the party. So yeah, this, this will, I think, key to help them get an extra level of adoption both in AI and in cloud. Yeah.

And these announcements don't surprise me, because as we're coming into, um, GTC, which is the, the big GPU conference that's put on, um, by Nvidia, and that was up the 21st or something, like the third week of the, of March. Um, you know, we we're now, right now getting briefed on announcements that are gonna be, you know, piling in over the next three weeks. Um, and it's the, I'm not gonna be at the conference, but I think you, you guys are gonna be, that thing is gonna be crazy.

Absolutely crazy, No doubt. Yeah. It's become the defacto AI conference of the years.

Yeah. It's going, it's going to be up there with, uh, super compute as a, as a as just noise. Yeah.

Um, and getting into that, so Di Diane, you raised an, uh, conversation for us today. Um, a little, a debate around whether or not ai, the technology of AI is a layer on top of the current infrastructure, IT infrastructure or whether or not it is a, what I see people talking about, which is this thing called the AI factory. So, and Dell coined that term last June when, when they, or DTC at their big conference, um, the ai, the, they called it the Dell AI factory, and then they've had the, the 18 wheeler right rep rolling around all over the United States talking about the, the AI factory.

And then I've seen that term picked up by multiple companies. So it's not no longer a Dell term, it's a actual term that the market seems to be using for some reason. Um, and that's just recently happened.

Um, and it's part of, actually in the briefings, couple of briefings that I've had just recently coming into GTC. So let's talk about that. You know, why would it be part, or why would it be separate?

Well, and there's, you know, there, and there's arguments being made for both, but I think this is, what's, what's coming up is, is we've, you know, traditionally kept our, our data in, you know, SQL databases. Um, we then, you know, moved the no SQL and graph databases and document oriented databases and so on. Um, then vector databases arrive that says, all right, no, you really need to understand, you know, much, you know, a mu much more unstructured information needs to be actually more deeply understood and e more easily accessed to retrievable.

Um, and then of course, now we have foundation models and large language models, uh, diffusion models, and they store data. There's no, there was no question about it. We were just talking about how you, you know, you can extract some of the, the, the data that's under the covers.

And so is it just part of the data layer that we've always had in our infrastructure? Is it something new? Because, um, uh, you know, AI does things that these technologies didn't do before.

Uh, uh, you know, if we look at AI ages, you know, they actually take autonomous action, um, in a way that we didn't predetermine. They, they determine how things will happen. So there's an argument that we now have an AI layer, uh, uh, yeah, a new AI layer on top of our infrastructure layer.

So I was just wondering if you guys want, you know, what your take was on that, Pete? Yeah, so I'm going to say AI is just another version of compute. I think it's a, I think it does blur the, the lines in between data and compute a little bit.

But if we look at kind of the vectorization of data, if we look at how models are trained, models don't keep all of their knowledge within the model for, you don't have the same level of clarity around your data. Uh, when you're talking about the model itself. Now, when you're, uh, when you're, uh, using your own data to, uh, an adjacent to a model, that's a different layer.

But again, that's compute that you're just saying, I'm going to apply this compute this application layer against my data. So I don't see this as yet a new layer. I just see it as, you know, an advancement in compute.

Yeah. I, I, I, I view it as, as it's a new, uh, spike through the layer, right? It has both compute implications and data implications, I think, uh, and, and it's, it's an, a new, uh, you know, column in the, in the, the infrastructure layer.

So, and I'm gonna go back to some of the earnings calls that I was sitting through, because these are, these are all data, data people. I'm not necessarily the vector people, but, um, there, if you looked at like NetApp and, uh, George Curry and talked about, you know, a couple very big wins of people creating a data lake. They were already currently a NetApp customer.

They're broadening that base to be a, uh, creating a data lake capability. Um, and then talking about, you know, your, how you're bringing in both your file, your block, which your databases, um, and then, you know, also the next piece is a multimodal kind of piece, the videos, et cetera, that have to go into the training. So they're creating the separate system here.

However, once you train that, that training data, depending upon what the application is, is gonna go against potentially a transactional system, right? So if I'm going to use AI to present information on my website based upon maybe a retail transaction or whatever that, you know, or if I'm gonna use ai, let's say I'm gonna use AI on insurance, you know, kind of submissions, that kind of thing. So due to that analysis, so you have this connection between the transactional traditional systems, processing systems with this AI kind of analysis that goes through, um, think also customer service, right?

Customer service that's part of that application within that application. So I asked to be integrated with that layer. Um, you guys are known better than I do because you're, you're better.

You're coders and that kind, or you've coded and that kind of stuff, and I haven't done that. Then I'm thinking about kind of like VAs just came out and added block to their file and object capability. So that's recognizing the data layer.

And, and why they're doing that is because they're recognizing the transactional data has to come into that training piece of it and do the training as part of it. So they're expanding that piece of it. Um, so, and, and, and, and you have, okay, so Vast and NetApp have talked about bringing in, you know, their, they're building vector databases within their data data management system.

Um, Dell has chosen a different war way that what they're doing is they're just integrating with other vector databases not incorporating into their data management. That's a strategy difference. But you still see this, you know, this integration of these pieces here, pieces, um, pieces.

So it'll be interesting to where this turns out, I get your comments, blow a hole in what I just said or whatever. No, I think it, it, I mean, quite frankly, yeah, thank you. And I think it's, that's representing a huge shift in the market.

Just, uh, a few years ago I'd be in briefings with HPE Dell NetApp and asking about the data layer, not the storage bits, the zero, the zeros and ones and deduplication and all the ser uh, uh, the, all of the storage level services they offered, but the actual data and helping to make data easier to process. And none of those players wanted anything to touch with the data. They said that was left to up to ISVs database providers, and, um, basically sis and they wanted to focus on the bits and bobs.

Now, the conversation has really changed because we're seeing, again, to the, uh, earlier comment, the, to Diane's earlier comment, this, this, this, this explosion beyond a single layer and this blurring of what's needed. If I need to retrain my model on my latest data, data or my latest transactional data, what's the fastest and easiest way to get to that? If, um, if my AI model and my data exists on the same storage system, isn't it best to do it at the storage layer?

Mm-hmm. We'll see. Mm-hmm.

Yeah. That's the, and that's the data management layer that we've been hearing them all talk about. Um, you know, for, uh, NetApp it would be blue, blue xp for pure, it would be fusion.

Um, I'm not sure the name of what Vast is calling it. It's probably just vast, vast capabilities, uh, environment that they're doing. But bringing that out to be able to manage, you know, um, and then having a separate, you know, actual storage, storage plane that has all the traditional capabilities that you're down there.

So, interesting. Thanks for bringing that up. It's a good conversation.

So, we'll, we'll see where that pans out over the long haul. And, and, uh, and then you have to also think about how this connects with some of the people that have got their, their data in the cloud, so that, that's All. Well, and you know, I try to look at what's, what's different in, in AI that, you know, wouldn't normally be found at, uh, at the data layer.

And the, the only example I can really come up with is what we're seeing is the safety layer that it's appeared, uh, in so many, uh, AI infrastructure that, you know, does and make sure that, you know, there's no inappropriate information being generated. No, no private information is being revealed, uh, that the, the results are accurate, uh, reduces hallucinations. Uh, and that's not something we've ever seen before in a data layer, um, at, uh, to that degree.

So that's something new, but I, I still think it's just, it's something that we actually probably need in our data layer. So I still, it still goes back to, for now, um, AI is a new element in our compute and data, uh, layer in the infrastructure stack. And that, um, we haven't seen anything quite yet that rises to something that would, that would require us to, to create an entirely new layer because we have some new third, or, you know, fourth entity in the stack.

So Well, And we're really early stages into the enterprise architecting this. I mean, we heard that very much so from, from the, the calls that we're on, you know, the earnings calls about, you know, this is still, we're, we're looking at 20 25, 20 26 in terms of this really rolling out to the point that it's, it's in, in application. Much of the money that's going out right now is still into the big foundation models, the people that are building, um, cloud service providers that are building GPU service, um, those or, uh, the other ones that are already research labs of some sort.

Maybe it's, uh, like Harvard, you know, medical, uni medical that, you know, was cited by Vast or it's, you know, some, some other, you know, pharma that's already has that, but they're expanding that environment to not be an HPC, but a, you know, east west, uh, architecture to drive, you know, you know, looking at new drugs and sort of things. So anyway, all all interested in me going. So our next topic, um, HashiCorp big acquisition by IBM.

Um, who wants to take that one? I I kick it off real quick. I'd love to hear what, what Keith has to say though.

4 billion for, for HashiCorp, uh, infrastructure as code security firm. Uh, a ling with developers as well. Uh, but also at the, you know, during the acquisition around the same time, HashiCorp made a major change, licensing change to Terraform, which is their, their main product, um, that really left a bad taste with a lot of developers mouths, uh, uh, developers really value the, the attributes of open source.

Um, and licensing is kind of a religious topic. Um, and so, um, you is a question of what, you know, uh, is that, is that being gonna turn this into another Red Hat acquisition? Or, or, you know, how's it gonna work out?

So for our listeners, just let's briefly say what is Terraform? Terraform is the open source, HashiCorp is the distribution. Yeah.

So let's, uh, I, I guess it's important to understand where, why is Hashi Corp getting acquired? Like, you know, why is this unicorn six point something billion dollar now publicly traded company in a position where they can't grow organically? So Hashi Corp has a series of applications, Terraform being its most popular, then followed by console and a bunch of other developer Kubernetes new web type applications.

Terraform is by far the most popular of all of their, uh, offerings and projects. It is, its role is for you to, uh, programmatically, uh, describe and deploy infrastructure. So whether you're talking about AWS Google Cloud, on-prem infrastructure, VMware, vSphere, you can orchestrate your, uh, infrastructure as code.

So I can say consistently and Good across multiple clouds. I mean, I think that's one of the big biggest attractions, right? It abstracts cloud, uh, cloud infrastructure.

Yeah. So, uh, and they went the open source route and frankly, it grew it, we saw what the same thing happens, but most open source, it grew to a point and they couldn't grow it beyond that, and they couldn't really tell a great cohesive story around platform. I said two or three years ago that Hashi Corp needed to sell itself to A IBM or VMware.

Did it make sense for one of those two companies to buy them? Yeah, we'll soon. See, because IBM has made the purchase the they need, this is a, in order to, to help CTOs and CIOs understand the value of HashiCorp you, your white glove service, you need the sales force, you need the account penetration.

Terraform was one of those things. Either you bought it or you wanted it free. There was no in-between.

So a lot of folks that are angry are the folks that wanted it free. Some, uh, Terraform and the Terraform product team would tell you it's mainly the competitors that are complaining, uh, that it's no longer, uh, open source and, and, and open source in a traditional manner, and free in the traditional manner. But, uh, developers, some developers, especially you Diane mentioned it, that this is a religious debate for a lot of developers.

It's either open or it's not. Okay. So am am I understand their primary competitors would be people like a puppet or an Ansible or something like that?

Correct. Yeah. So I wouldn't, I, I don't even know if they're competitors as a mul, uh, as much as complementary, looking at the same problem in a different way.

Uh, cloud Formation, we lost piece here. Sorry. Well, the, uh, starlink is, uh, is probably switching to it new starlink that way.

So The old reliable starlink, so I'm back Star, the competitive question. Okay. So I think Terraform or competes with things like Puppet or Ansible and, or maybe they're just more complimentary, uh, Keith, yeah.

So IBM is going to have, you know, some, some work on their hands, kind of rationalizing Ansible version. Okay. We're losing him Again.

And Terraform the, I I wonder if we're actually losing him. Uh, it, it might, the upload might be work just fine, right? Diane, I'll just let you answer that question because my, the, the, I go through these periods and it's Alright.

All. So then Terraform just is positioning kind of competes with I Ansible and Puff puppet di Diane is that? Well, I think, oh, first some of what HashiCorp does, that's true.

Um, the, um, yeah, it, it really start, it started out really as secret management. Uh, so if you look at like CyberArk or a Azure Key Vault or, uh, BeyondTrust, those are often considered the more kinda the original competitors, uh, with HashiCorp. But Terraforms become super popular as, uh, as uh, for, for infrastructure management.

Um, and so HashiCorp now does multiple things like so many cloud vendors do. So, you know, there's, they have I think an array of competitors at different level, at different product levels. Yeah.

So I mean, many years ago, IBM and, um, you know, the, the current CEO did this one, he wasn't the CEO, which is, uh, drove the purchase of Red Hat. At that time, the at evaluator group, we were all scratching our head at $34 billion and like going, you know, doing the back of the napkin about how long it would take to return the investment on this thing is like, it's never gonna happen. I think it was 34 billion, maybe it was 43, I can't remember one of those two.

It Was a large number. It was a, it was a huge number. Yeah, it was a big number.

And this is not that big. But then again, it's not, you know, complete platform play, but it is the platform play. If you're, what you're talking about is managing across clouds infrastructure code and, you know, they gave a lot of cred to, um, red Hat.

You know, they brought 'em into the, you know, when you have client executives that walk into the CEO and CIO kind of capability, you're, you're kinda walking in Red Hat, right? Um, and, and so there's, I think there would be some similarities, you know, in terms of the go to market on this one, or is this just gonna get rolled into the IBM stuff as opposed to what happened? What?

Well, I think a lot of people are hoping it doesn't get rolled, rolled into, you know, uh, an IBM only story. Um, it is really valuable for IBM to be part of a, of a bigger story. Uh, the CIOs really want their IT to work with all the re the rest of their it, they don't want these silos in their organization.

So it's from a standpoint that, uh, HashiCorp provides, uh, IBM with credibility across clouds, that's, that's a great story and that's one that they should keep. They shouldn't mess with that. Uh, and I hope that they don't.

Um, but the IBM of old would've, you know, this would, is something they, they may not have really focused on preserving, but the current IBM, uh, and they're, and they're very much on the upw. People have almost written IBM off, they are back, there's no question about it. And if they can do with HashiCorp what they did with Red Hat, this is gonna be, uh, gonna do really well for them even at this prize.

So we won't have Red Hat Summit now will have the Red Hat plus HashiCorp Ben or something. Yeah, I'm, I'm really excited to see what the IBM cloud folks do with this, because, you know, we don't talk about IBM Cloud enough. The IBM Cloud does an amazing job working with some of the other hyperscalers, the augment your capabilities of running some of your traditional workloads in public clouds in a way that enterprises accept.

And that's one of HashiCorp's original stories was how do I take my mainframe app, modernize it, and have a connector, this is what console did have a connector from my new world applications into my mainframe applications, and then run that in the cloud-like operating model. So this is a, you know, the IBM will be able to expose some of the more interesting capabilities on the enterprise side of a Hashi Corp. Well, we'll, we'll see where this plan pans out.

So thank you very much guys. And thank you for, uh, listening in. I think we got everything covered here that we're gonna do today.

Did I miss anything? No, no. We, we, that was a sweep.

Okay. There it is. That's a wrap, guys, and we will see you next week.

Don't forget to like, follow, share all that stuff because you know what, even the guy that does is in the background, that's doing all the video work is now listening to our infrastructure matters. And that's really cool. Have a good day.

Thanks everyone. One.