CIO / CTO AI Insights Plus Show and Earnings Coverage – Infrastructure Matters EP63

All right. Action packed infrastructure matters. We don't have a whole lot of time for banter.

We are talking about Microsoft Ignite. Kpe Con AWS Reinvent is coming at NetApp. Earnings Super Compute 24.

And this is the one year anniversary of VMware by Barcom. It's, it's, someone told me that it went by fast, and I'm like, oh, I don't know how fast it felt felt for many people. You know, Everything else is so much more positive.

Do we really wanna drag that one out? No. We'll, we'll save it.

We'll, you know what? We'll, let's talk about that first. I have with me Diane Siff and LY base.

Uh, I'm Keith Townsend, and you're listening to Infrastructure Matters. Let's start out with, like, on the low note, one year, VMware, uh, being acquired by Broadcom. How has it gone?

Well, I think it's safe to say that, uh, in the quote I heard, um, just a, uh, a week ago, is that customers are miserable. Um, but Broadcom is declaring success. Revenue is, is up there.

They think that the deal has been good for them, uh, despite price rate, uh, uh, increases of 200 to 500%, and you can no longer pick and choose the support services. You gotta buy the whole thing. At and t has dragged them in the court.

Um, but Broadcom thinks it's been good overall for the, for them as a business. So, you know, there's at least that to say for it. Yeah, I think I concur the mi uh, miserably miserable results.

From a end user perception perspective, there has been some innovation. If you are in the wheelhouse of beware Cloud Foundation, then it has been a good, you are the rare customer. This has been a good transaction because Broadcom has narrowed VMware's focus reduced a ton of skews.

Uh, if you were, uh, aria, uh, uh, cloud-based Aria using which user, which is their observability and configuration management and IT management platform, you'll, you'll be a little bit disappointed if you use VMware cloud on AWS, you'll be a little bit dis disappointed. But if you're buying into HOKs, uh, vision of private cloud, displacing some of your public cloud spend, you, you're probably a little encouraged. It's a interesting situ.

I think we've got another six months of this gnashing of teeth to go on as people finalize their strategy around this and the numbers, and they go through the final licensing, et cetera. But, um, I think hopefully by June we're not talking about this anymore because, you know, between AI, cybersecurity and VMware, that's, I think that's all I talked about for the last year. Yeah.

The two topics have been AI and v and, and migrating from VMware vpe. So, Diane, you put this one in the rundown Nvidia day happy Nvidia Day. The, well, we'll post video day, it's kinda like Nvidia boxing day now, but, uh, Nvidia earnings, Well, it was a, a beat on the top and bottom line, like everybody expected.

I mean, the, the, the, the demands that this, uh, that, that being put on the stock are just tremendous. Uh, 'cause the, the street is, uh, you know, expecting Nvidia to blow the doors off every single quarter. But they did.

Uh, so, you know, earnings were up 94% year over year, um, there, you know, to 35 billion. That's astounding. Uh, all on the back of AI chips, you know, this is a, a company that, you know, until recently was making most of its money off of crypto and gaming, and now ai, you know, they, they're now the AI company and the, they're now the world's largest company of any kind.

Um, you know, this is an, uh, an amazing result. Uh, um, you know, and it's all because of GPUs. Uh, now we're hearing persistent, you know, rumors that Blackwell has problems.

First they had the, the chip mask issue. Uh, now there's a, there rumors are, there's a cooling issue, but there, there, there they can sell every Blackwell they make, um, and then some for a long time to come. Uh, the demand for, uh, you know, the infrastructure around AI is just insatiable.

And Nvidia is the, is the company, you know, uh, you know, with 90% of the market, uh, because of the Cuda stack. And so, uh, you know, they're, uh, the stock was down a little bit, uh, you know, after the, you know, they didn't blow the doors off, uh, and, and then some. Um, but now they're, they're, you know, they're, they're doing well again.

And it's, uh, it's, it's a great story to see. And, and it's, it's so good to see that Jensen do do as well as he, he is done, you know, he is been with the company since the very beginning. Um, and, uh, it's one of the really bright stories in the industry.

The one item that I, um, kind of pulled out out of the earnings was the data center revenue. And I know this is ridiculous, but it was only 112% year to year up as opposed to 154% the quarter before, or 426% the, the quarter before that. Okay.

So it's, it's, the quarter to quarter change is seven. It was about the same, um, from quarter to quarter. Um, I'm not sure if that really means anything at this point.

Um, I know that there's, you know, we're, I think part of it was that there's this huge purchase that was done last year on the GPUs, and maybe they, they sat there kind of, and not completely been deployed, um, on those numbers. And when they say data center revenues, I'm assuming, you know, that's all those GPUs about whether or not it's going to AWS Me MSP or the enterprise. So they're not slicing it by that.

So that was just one, not item that came out. And then when I was reading some of the, um, financial guys, uh, reviews on that, that seemed to be a piece that they kind of want. What's going on here?

Uh, yeah. Well, you know, interesting is, uh, people forget how much it budgets really matter. You know, we talked about whether we're talking about, uh, Broadcom, you know, it takes a, it takes a year.

Uh, there's a yearly cycle that it makes the decision in one year for what they're gonna spend the next year. And it takes a lot of work to change that. And, and, uh, this time of year is always the end of last year's budget where, and they probably already spent most of their yearly budget by this time.

So I wonder how much that has to do with it. Yeah. And then, uh, I think I'm going to send a recommendation to Nvidia that they just take off all the doors.

The, the blowing off the doors no longer impresses the street. So, uh, keeping on this earnings trend, NetApp earnings, Yes. Uh, NetApp is continuing to, um, build on their success, um, you know, as a core data protect data storage company that is kind of like, is data storage at interest you right now?

Yes, it is for when we'll talk about supercomputing, but, um, they've, uh, you know, delivered some really strong revenues. Um, you know, they're up about, I think it was 9% of the revenue, but more importantly, there was two pieces, two numbers that came out. Um, and I haven't gone all the way through the, the transcript and read all the numbers, but they're up 19% year to year on, and they're all flash.

And part of that is because they have a very large entity, um, set of customers that are, are hybrid, um, and they're just still moving over. And, and George is, George Curry has been talking about this for quite some time, is that we have a very large install base that is still there that hasn't moved over. Um, and I think at some point in time it was like 60 or 70%.

So they're, they're pulling those people out and they're making, doing success, I'm sure, with their C-series, which is the QLC. The second thing that was really good to see is they, um, for their cloud first party, um, storage offering, they were up 43%. Um, it's still not a huge, massive number.

I mean, it's not, you know, they're on their way to building and that sort in that envir for an annual number. Um, but they have, are finally seeing that uptick that we had expected to see over the years in term of being, uh, when I say cloud first party, it's sold, um, by AWS, it's sold by a Azure, and it's sold by Google, not marketplace, but it's sold core to their, their products. So as we go into this next stage of ai, expecting to see some of the, that their, uh, file system capability be implemented, uh, along with a core, uh, operating a core block system, core file system for core type of, um, transactional processing.

Yeah. NetApp has actually done a really exceptional job, I have to say, of transitioning from a storage only company to a data company. And pervasiveness of their solution throughout the three cloud providers is honestly leading the market in how you transition from a on-premises company to a cloud-based company with, uh, without sacrificing a lot of that on-prem revenue.

Speaking of, on-prem, Diane and I spent a good eight hours, or collectively between dinner and, uh, our focus group about 10 hours with nine CIOs and CTOs. Mm-Hmm. Uh, in Seattle.

And Diane, let's reflect back on some of the higher level trends that you saw amongst that group. We had folks from biopharm to manufacturing, farming, high tech, enterprise financial Services. Yeah.

Uh, financial services. Yeah. A good group of folks.

Yeah. It was, it was, uh, really excellent to kind of take the pulse of what's going on, uh, these days in it. And we had the opportunity in this focus group, uh, to kinda ask them, what are the biggest challenges they're facing today?

What's really top mind? Uh, and, uh, you know, they, uh, there was, there was a lot of things. It was, uh, it was very interesting, but, uh, topping them all was making AI safe.

Uh, they were, uh, universal saying that that was, that was the most important thing, uh, that they're focused on in terms of, uh, the challenges that they have. Uh, but the next one is really, uh, um, really interesting too, is, uh, a value as a service. They really want an easy way to be able to tap into delivering value to their customers, uh, in a turnkey fashion.

Uh, you know, it is just as hard as it's ever been these days. It's, it's the, the level of complexity and sophistication as a, at a high watermark, uh, is only gonna get worse. Uh, and so they're really looking for ways that they, they can say how, you know, how do I get by to my customers, uh, very quickly, uh, very straightforward.

That's really important for me. And then I was really encouraged to hear it. Now, I think we do, did have really enlightened IT folks, um, uh, in general in, in the, in the session.

And that was by design, uh, but tracking success of it after rollout, um, most of the time it declares, uh, you know, once they, they, they've delivered a system to the, the internal, uh, end customer, uh, that, you know, they, they declare victory. And, uh, you know, and that's usually, that's usually step one. And the success of an IT system is rolling it out.

You then you is a long customer adoption and customer success, uh, patterns is what we've, what we've seen on the, the, the external side with SaaS, and now that's finally moving into it. And so they really want to track success of it after rollout, making it as success really. And then rounding out the, you know, the, the strongest signals was the changing workforce.

We see a second generation of Gen Z arriving. Um, gen Z's been the dominant, uh, new arrival in the workforce. And, and they're, uh, the first generation that doesn't look at, uh, their paycheck as the number one thing they wanna do good in the world.

Uh, but they're, they're, they very much see that, just trying to attract that, that talent, making sure that they feel like they're, they're getting their other needs, not just, um, um, a good paycheck, a good compensation as being the number one, um, benefit that they've received. It has to make the IT department a very appealing place to work if they expect this new generation. And these guys are definitely feeling that.

So that, that kind of was some of the top, uh, the, the most, the, these strongest signals we saw. So as you were talking about it as a service or as a value, a value service being delivered. And, um, is there then a, a trend at all that you picked up from that group about buying systems that are fully integrated to more rapidly deliver?

I mean, we've talked about, we see a lot of this coming out for the AI systems because it's so difficult to deploy quickly and they need to move fast. And much of what we're seeing the client, the, uh, vendors do is how do they integrate with all the partners all the way up the stack? Is that something they're looking, do you see, did they talk about looking at those type of offerings in order to bring more value to the market?

Well, We, we talked about a little bit, uh, one of the, the, the, the signals we, we got, um, was this real concern about IT fragmentation. So you could, you could flip that and say that, you can argue that that's the real challenge, is that they do have all these siloed systems. They're being required increasingly to deliver any given solution.

You gotta pull data from all over the place, not just your own, uh, uh, data sources. But now you have your, your, your data's increasingly being, being spread across all these cloud systems, all these cloud platforms and SaaS, uh, offerings. You have to, that, that fragmentation is a real problem.

And, and keeping it all secure while you're accessing all this federated data that you have across all these applications. It did have half of 'em reported that that was a, a significant challenge. Yeah.

And Diane, I, I thought what was really interesting, because you did have a good mix of really enlightened IT leaders, and we found some of the data points that they shared very contrary to some of the data points that you've, uh, collected over your larger set of CIOs. And this was a, uh, I think, uh, outlier that these folks who are thinking much beyond, well beyond the average CIO Yeah, no, I, I absolutely agree with that. Uh, and, and that's what we, we tend to do, you know, when we do broader, uh, CIO surveys, it, you know, teases up, uh, you know, automation, uh, cybersecurity and ransomware, uh, as, as being the top concerns.

And, and these folks are really more concerned about having positive impact with their internal customers, um, trying to create a, you know, appealing workforce so they can continue to attract the best talent. Uh, those are talents also, you know, appears in our broader survey. But, but, uh, these folks put it, you know, uh, really head and shoulders above other things.

And I love what you said about, you know, delivery AI in a secure fashion. Yeah. Um, because that's why I think one of the things that I've been talking about is how that, you know, as we get into the enterprise, as they start delivering these applications, it's gonna slow down as they diligently look at, have we trained this right?

Do we have this right? Are we assured that we're not gonna have some really weird thing happen out there, that it's gonna hurt the company, hurt somebody, hurt the brand? You know, And this is a classic IT pattern that they often, uh, hurt someone in the short term is to go fast.

Uh, sometimes means you have to start slow. Yeah. A lot of people forget that.

Yep. All right. So let's move from having real discussions with real customers to having real discussions with real customers and vendors at several shows that we've been to over the past three weeks.

KubeCon Super Compute and Microsoft Ignite three really major shows, uh, with growing enterprise interests, I think when it comes to KubeCon and Super Compute, but Microsoft Ignite really in the wheelhouse of most enterprises and where most enterprises are not just doing AI work, but also work, uh, on around keeping the lights on. Let's start with Kku Con. Uh, Kimberly has been a, uh, it is been a blur of the past two weeks.

Remind me, were you at Kku Con? No, I was in Scottsdale. I was, I was, I was humming it with my friends at Veeam and, and enjoying the, the wonderful place down there as they were brought in all their executives.

You went to CubeCon? Yes, I went to CubeCon. I was also, I I, I spent a couple of days with Veeam, which by the way, Veeam nice work on the sneakers that that was, that was a surprise.

If you, if you're not sure what I mean, check out the socials. Uh, KU Con was, obviously there's going to be conversation around ai, but there was plenty of conversation around VMware alternatives. But I wanna call out, uh, a growing segment of this, which is observability Kon has always been a really great conference for observability in general.

And when you think observability monitoring, et cetera, you're going to stink. Think of Splunk actually, technically you say stink. I mean, a lot of companies will say their investment in Splunk is a, a, a a never ending drain of hardware.

But Cripple was a big point of conversation. We don't have it in the notes, but Crile, uh, if you're not familiar with it, is the open source alternative to Splunk. And they, I just saw they got a few hundred million dollars in funding just recently.

So, uh, there was a lot of mature conversations around not just how you, we've gone be gone beyond how to deploy Kubernetes and talking more how do you, uh, take Kubernetes cloud native infrastructure and make it invisible infrastructure? How do you make it something that's just there and you count on and it becomes that API to the data center. And as a result, the API to ai, there was obviously a lot of conversation around AI and how do you manage AI in a DevOps type of methodology.

And for what it's worth, we're gonna talk about Supercomputing in a bit. There was no observability and no security guys there. The, the, the, you know what, let's get into super compute.

'cause Kimberly, this is my, I didn't get as much time on the show floor, but there had to been at least 400, uh, vendors on the show floor of Super Compute. It was a massive show. Close to 20,000 people now.

Yeah. Yeah. So Super Compute has been the domain of the professors, the PhDs, and the graduate students along with all the research labs that come out of the major universities worldwide, and the labs, the research labs that we have, like Lawrence Livermore, JPL, Fermi Lab, et cetera.

Um, in the past maybe 5,000 people, you saw a lot of posters being presented, which are papers, you know, that are being presented by the graduate students. Um, that was the kind of vibe that you had there. Very, very, very much so technical.

I'd go to a session and my glaze completely over because I'm listening to these HPC, high performance computing people talk about whatever, and it's, you know, I'm, I'm not that smart, Amy. Nobody hired me to Bridge did National Labs. So this has fundamentally completely changed.

Um, and this is ai. This is called going into the enterprise and the where the market is going with generative ai. And we have a mix the labs, it's not that we're going beyond the labs.

The labs are changing as well. If you look at, listen to some of the people that presented, they are going from an HPC, pure HPC environment, which is a lot of bash processing, et cetera, to an environment that they're doing more data simulation and gen ai, and it's completely different. Um, IO configuration data infrastructure, um, CPU systems, GPUs, et cetera.

And they're having to build for the next generation of research as well, because there are researchers coming in and saying, we're gonna be using this. We're gonna be going into the next direction. So when you start to, and when you look at this entire, the, the show floor, um, number one, the biggest thing I felt like was there was all about liquid cooling, because we're burning up the, all the power.

I mean, liquid cooling is front and center, and I mean, there was a zillion guys out there with something to do. Everything from the piping to the oil or the glycol and the water cooling and the design of it and everything else, how they would do it, plus the rack systems, which you had. And then of course, all the, you know, the processors from all the different vendors, the data file systems, you name it, it was there.

And, and, and I called it the world class petting zoo for technology, because you really got to see a whole lot of gear, including some quantum computers that were on the floor. So really cool. Yeah.

If it had to do with cooling or power, it was at super compute. Uh, the immersion, uh, water cooling Lenovo talked about how they're, uh, using and redistributing, uh, 45 c water to cool and, uh, heat and recycling that water. And this is specifically water cooling.

Somebody asked me, uh, you know, they said, you know, quote unquote liquid instead of, do you mean liquid Keith and not water, no water in Submersive. I saw they had an intel board. Some, uh, some first in, uh, liquid cooling.

The numbers are unmistakable. The, we're starting to go from the typical 15 kilowatt rack in the data center to 60 kilowatt racks being common to 120 kilowatt racks. Now I'm becoming the, uh, the little bit of future proofing and the most advanced research, uh, happening is happening at that 500 kilowatt rack, uh, level.

Can you imagine 500 kilowatts of power going into rack? Now remember, most traditional data center racks two use full with, uh, two use full with dual core servers all the way up to the top of a 42 U rack. It's about 15 kilowatts.

You're looking at, you know, uh, just one, uh, one AI server taking up all that power at a minimum. So it's, it's, it's an amazingly double check those halon, uh, fire control systems, you know? Yeah, you, you, you're, you're definitely going to put them to test.

And then, uh, we also saw the drive storage rate rates, uh, solid dime debuting, uh, last week. There are 122 terabyte SSDI got to hold it. They would not let me keep it, no matter how much I asked, they just would not, you said It was $15,000 for that thing.

Yeah, it was 15, $20,000. 2 petabytes of storage and two use of, uh, of, of, of data center Rackspace. This is an amazing pace of innovation that's been driven by the thirst of, for AI storage and processing.

Huge amount of data there, data storage companies there. Um, everything from just your typical file system companies to parallel file systems. I had a chance to talk to Sam Wenner from IBM, and we were talking about GPF or also known as scale now, um, and how it's just the, the numbers on there are doing terrific.

Um, and then URA Rein, you know, which is Panasas relaunched with their new name, um, you had pure storage there with their blade, you know, their blade. And, you know, while it's not a parallel file system, they're definitely rolling in. And then Hammer space, you had their announcement with tier zero capability, which is what what they're doing is enabling the storage that's in the servers, um, and enabling that to be the storage and, and do be the feeding of the GPUs, um, and then right out to the network stores that's attached to it, attached to those environments.

So the numbers that came out from them and perf storage, um, benchmark were very, very substantial. Yeah. The, the, your GPUs can be as fast as you want 'em to be, but if you can't get a IO to them, it really doesn't matter.

So, Diane, I heard you went to a small conference here in Chicago. It's is I, every time there's a major tech conference in Chicago, I don't get to go to it, but Microsoft Ignite or did you go or were you remote? Uh, no, I was there.

Um, I, I didn't wanna miss this one because this was really the, the, the big new, uh, big announcements that, um, Microsoft is gonna make around the hot topic in ai, which is agents, uh, agents are, are unlike LLMs, which produce content for you agents take action for you using generative ai. Uh, they can actually talk to a system using, uh, APIs, uh, to carry out tasks. And Bo and Microsoft had a lot of news, uh, uh, and, and the, it's most basic, they, uh, announced copilot actions, and that allows anyone to automate, uh, pretty much any task they have with simple fill in the blank prompts.

Um, they, uh, announced agents for SharePoint, which of course they did. Uh, I'm not sure that people were lining up for that to begin with, but, uh, they, they had to do that. That's still one of their biggest platforms.

Uh, but they have, uh, they announced a new employee, self-service agent, uh, a language interpreter that, that operates live, uh, during calls. And those were nice. Um, but what was really interesting was the infrastructure announcements around agents that IT departments will really care about, uh, most notably the, uh, the copilot control system.

And it's a, it's a control plane designed for it to consistently, um, and declaratively, uh, uh, drive, um, business value out of copilot, uh, agents. And it dictates how they're, how they're situated in front of users, what they can and cannot do. So yeah, it gives you all the knobs to control agents across the enterprise.

Uh, but I think most notably was their data protection, uh, news around agents. Uh, they will provide a guaranteed intelligent grounding on enterprise data, uh, while respecting any underlying permissions and controls that are, that are in place so that people can't make requests of agents to do things with data under the covers that that user maybe doesn't have access to. Uh, and then it ensures that what the agent does is accurate, done accurately, so that that's the grounding piece.

Uh, and so that you'll have agents generating new data under the covers and then putting it back into your IT systems and it's not right. Uh, this are, these are the, the, the new risks that you have by unleashing AI to directly interface with your IT systems. Uh, you, they're gonna consume data and generate new data, uh, and you have to make sure that, that it's not gonna corrupt your systems and, and, you know, put hallucinations in your database.

That's, that's a concern, right? So they announced data protection around that. So I, I really liked that as well as a slew of controls around, uh, how copay copilots and what agents, uh, can and cannot do in your organization.

Yeah. So this is penetrating the third wall a little bit. We talked to our CIO and CTO cohort about this very topic, not just, uh, Microsoft copilot, but specifically about agents and safe ai.

This is, this is one of those subtopics within safe ai. And one of the considerations that these forward thinking CIOs and CTOs were considering is where to put that business logic. Do we trust that with a Microsoft a Service Now or ASAP, or do we build it ourselves?

That was a really interesting conversation debate. So speed to value versus this, you know, desire to not be locked into a proprietary business process and for a third party co, uh, company to own such a critical part of your capabilities. Wow, I wish I'd been there.

I'm like going, okay, I love super computing, but Wow. Yeah, these are the topics on the front line and, and it has to deliver on all this because there, there is voracious demand for, for ai, even though a knowledge, it seems to have plateaued with knowledge workers. Um, but we're, we're gonna see the, the business wants to put AI into everything to reduce cost, increase productivity, uh, future, you know, uh, ready organizations and, and so that's, yeah, really exciting times.

And we look at the whole thing, the whole stack from Nvidia at the bottom and Microsoft at the top. It's, uh, we, we live in exciting times, folks, It is exciting times. We'll continue to digest all of the information that we're finding.

com. You can follow all of our musings there. As well as, uh, some housekeeping.

We will not record a episode the week of Thanksgiving, but don't worry, we'll make up for it because AWS reinvent is that same weekend, and we will come back with plenty of discussion around AI infrastructure and why all of this matters for my co-host, Kaley Base Bates and Diane Hinchcliffe. Have a wonderful holiday break and we'll talk to you soon.