$100 Billion for OpenAI? – Infrastructure Matters EP71

All right. You're watching or listening to, depending on your preference, episode 71 of infrastructure Matters. Diane, I think we're in trouble 'cause Kimberly isn't with us this week.

So we're going to touch some third rail topics that she would typically reign us in on. We're going to be as, as apolitical as possible when we talk about some of the, uh, happenings around ai, ai ethics. We're gonna talk Intel, DataRobot, SAP, and Databricks in the same room.

That's, uh, that's gonna be an interesting one. Uh, of course, AI news and, uh, some internal, we've done some testing around Intel's Gotti three Pro, uh, accelerator. So a lot of great news.

First off, well, let's, uh, start out with, uh, oh, I think I might have sold my Intel stock too soon. Yeah, I think so. Uh, well, uh, Intel stock surged this week unexpectedly.

Um, and there was a lot of speculation why, um, the, uh, you know, there's been a lot of criticism over, uh, Intel's leadership in the chip industry for the last decade. Uh, and it's gotten increasingly questionable. Uh, and, and the stock has really reflected that.

Uh, but the, the now the story is that, um, Intel is getting help or, or may get help from TSMC, uh, to modernize and catch their fabs up. Now, why would a competitor like TSMC do this? The speculation is that, uh, is trying to head off tariffs.

Um, the, uh, the Trump administration has, uh, expressed that, that the, you know, and na and named Intel that, um, uh, they would like to see, uh, uh, our chip companies do better, and they may put tariffs on, uh, Taiwan in order to, to bring that about. And this may be a proactive maneuver, uh, to head that off. So that, that's a speculation.

Anyway. Wow. That is, uh, interesting.

Bad fellows. And we'll get into, uh, uh, more interesting bad fellows in a second. What do you think this means for the AI accelerators mobile?

What's the downstream? Well, I, I think we're, we're headed for interesting times. Um, the, there's, there's only so much you can do to catch up, uh, a fad in the United States.

Frankly. We've got more, you know, advanced laws around, uh, environmental protection, uh, um, you know, building codes. Uh, it, it takes longer to do things here.

There's just some things, there's a limit to how much anyone can, uh, can aid, um, a or accelerated company in the United States. And there's a reason why, you know, people don't wanna plunk six to $12 billion down in a fab Europe. They can't get return out of it.

They, you know, they can't do two, two nanometer anytime soon because of all the permitting and, and red tape and bureaucracy. And so the question I think is yeah, uh, is, is, you know, TSMC is probably hoping for something outta this, uh, try, they're trying to demonstrate something, but I'm not sure they're gonna be able to really help Intel in a, in a, in a dramatic way. But they may not need to.

They may just get points for trying. Yeah, and I'm thinking that similar to how there's always boutique manufacturers for, believe it or not, still seven nanometer, nanometer, uh, processes, stuff that's using automotive all today, all time. 5 nanometer processes Or talk About other people, which is insane.

So, uh, I think there's plenty of capacity. TSMC has all of the work I think, that they can handle right now, and why not win some points with, uh, helping Intel out too big to fail. So DataRobot acquires open source AI startup agnostic.

Yeah. Spelled with the Q agnostic. We're running out, we're running out, uh, uh, domain names.

Yeah, it's, um, uh, terms were not announced. Um, the, uh, the agnostic, uh, has a, a, uh, AI agent framework, uh, and application, um, a portfolio that, that aligns well with DataRobot, which is a, you know, a place where you can put all your data and, and run all your AI apps. Uh, and you know, there, there's gonna be a lot of interest in creating a more consistent environment for AI instead of having all these patchwork, uh, you know, uh, AI models, um, and data lakes that, you know, you're doing rag against and all of these things that platforms like DataRobot potentially allow you to bring a little bit of order to the chaos.

Um, uh, but what they didn't have was anything to do with agents. And if you're not doing agent based AI this, this year, if you're not putting that in your product platform, you are behind. And this allows DataRobot to, to get in the AI agent space.

Um, it's, uh, it's unclear, uh, uh, the, their agnostic has, I think they said, um, 5,000 customers. Um, if you consider most of those are probably small, medium sized businesses, it, it probably, probably needed this to get the r and d money to keep their AI agent portfolio going. Uh, because if you haven't, if you're not inquiring or, uh, acquiring your ai, uh, agent, um, company this year, you won't be able to afford it next year.

And Yeah. And this interesting how this market continues to move forward. We haven't covered it yet, but, uh, it seems like people are moving away from this idea of independent agents to agent workflows.

So, uh, agents not necessarily doing, you know, random chores like you would think a chat GTP would allow you to do, but workflows specific to, uh, let's say cash flow or, uh, you order the cash or some, uh, some business process in which agents are specialized and have some type of chain of action, uh, programmability versus, you know, just generically taking the agent. So this is moving much faster than most of us can follow It. It is.

Uh, and, uh, I've actually spent the last, um, month and a half or so doing a full survey. I, the AI agent based space in a report that will come out with at, at heat trim here in the next couple weeks. Uh, and I talked to a lot of, uh, companies and a few CIOs that are, that are, uh, uh, doing something with agents and it's all around service desk of some kind.

It could be internal service desk, it could be external, uh, but it's about orchestrating some outcome for a stakeholder, uh, you know, an internal customer, an external customer, um, attach a really big, um, uh, road service network, uh, um, their CIO and they say people only log into their website two, three times a a year, and so they forget everything about it. And so they don't know how to change their address or update their payment system or anything like that. Now, they can just log in as long as they can log in, they can talk to about, say, change my address.

Okay, great. What is your new address? And I'll update it in the system.

Right. You know, that kind of thing. So we're still seeing these, these, these, the crawl, walk, run, uh, thing where most people are at crawl with agents and, uh, but that's the kind of thing that they're Doing.

Yeah. You know, it's, uh, it seems like just yesterday we were talking about AI assistance being kind of the future of ai, and that is quickly falling by the wayside. Like AI assistant becomes kind of a given in our own email.

Uh, uh, Gemini has been activated mixed results and, uh, and, and in our email system, I think it all Get back and, you Know, we're moving much faster than most enterprises can adopt this technology. Speaking of data and data lakes and cleaning up data and organizing data, a surprise to me, at least both of us cover SAP and Databricks. SAP and Databricks announced a partnership in which SAP can resell Databricks into their customer with allowing SAP, uh, SAP customers to use Databricks platform to analyze their data.

I use the term hell is frozen over, because I never thought something like this would happen. Basically, this idea that, uh, you wanna do large data analysis, you wanna across your entire data landscape, your SAP, uh, transaction data along with your, uh, flat data within your file system, other databases, et cetera, and come out with business insights, kind of the magic of these huge data analytic platforms. SAP has held this capability close to itself with its own cloud-based analytics platform.

I'm, I'm shocked. I think a lot of people were, uh, and, and confused because SAP has so many, uh, pre-existing offerings in this space. If you look at SAP, data Intelligence Cloud, uh, SAP, critical Data Cloud, SAP, customer Data Cloud, SAP, datasphere, and SAP Analytics Cloud.

And that's just some of them, um, you know, adding yet another data cloud, which is what they're calling us, um, uh, you know, powered by Datasphere is, um, makes SAP look like they don't have a clear strategy. They're not explaining how this fits into all those other clouds, which one should be up at the top, or how do, do they set off at the side in each one form a specific function. And it seems much more partnership based, uh, in terms of strategy as opposed to actually, you know, real organized product strategy.

So I think a lot of people are scratching their head, but clearly SAP customers and Databricks customers want this, you know? Uh, and and how Much of you do you think is waring off the eventual, I think it's gonna be a commoditization of analytics by ai. This seems like a, uh, I, I don't know if I've ever thought of this, but analytics as a, just a feature versus a standalone product.

Well, I mean, I think you make a good point. The, uh, the commoditization is gonna happen because everything's gonna go into ai, and you're gonna be able to pull out most advanced analytics, you know, build me a 3D visualization of all of our sales processes over the last year, and boom, out, out it come, it, it, it comes in. So you need access to the data, you need it all connected.

And where SAP, um, you know, didn't have, uh, a clear story was how does SAP and Databricks coexist? This answers that question. The question is, is it a, is it a major problem?

Is this, uh, you know, uh, obviously SEP thinks there's a revenue stream to be had there, which is really important because we're having a hard time getting a lot of the world to, to move off of, uh, perpetual licenses to subscriptions. So we're seeing, are now seeing a number of these partnerships like they did with, uh, data Steer before and now Databricks today. Um, I, you know, I, I think it's a race to the bottom.

Uh, for the most part. This is something you'll be able to, to, to be able to do relatively easily and straightforward across the enterprise, uh, quite soon. Yeah.

And SAP has a critical problem ticking with this, uh, end of life of their legacy ECC product. Uh, by 2030 SAP customers are notoriously difficult to get to upgrade. Uh, I think, uh, the latest number, I saw that by 2025, up to 25% of their customers still will be on legacy EC by 2030 and getting third party support from somewhere else.

So SAP is in essence competing against itself with this deadline. So, uh, you know, these are companies that have serious challenges. That's, uh, some of it is self-inflicted, uh, and some of it is the, uh, competition against AI and the agility AI is bringing to organizations or the expected agility, which we'll skip down into the rundown to a big competitor and friending me up.

SAP sometimes, because a lot of SAP runs on Oracle, Larry Ellison, uh, wants all of our data. Yes, he does. Uh, and so it's hard, hard to read, uh, much into what Larry says, uh, given that he doesn't necessarily drive, uh, the entire industry like, you know, some of the other, you know, tech billionaires do.

Um, but yeah, he, he is suggesting, and he may be right about this, uh, that unless, um, unless we put all of our, uh, available data at a national level, uh, into a master database, and, and he is, is suggesting all available genomics too, so that we can marshal all of our data and organize as a nation to be a competitive against China, that we're gonna be at a disadvantage. 'cause China will do it. China's definitely gonna do that.

Um, and, and of course, uh, it's, is so self-serving because he, he thinks Oracle is the best place to do that. Um, and I'm, you know, I've done work with some of the other hyperscalers, like, you know, like, um, Bezos, same thing. Has, he thinks the only AWS is, uh, reliable and global enough to run a lot of important gov government services.

And he thinks the, these hyperscalers have built these global infrastructures, uh, that will allow nations to compete on AI nations to compete, um, globally at a, um, and on digital services, digital currencies, digital assets. Those are all big conversations that, um, that haven't really been, uh, come to the forefront, but with big changes. And, you know, the, our our presidential administration, these things now seem much more likely suddenly than they have been in the past.

Yeah. Much of it seemed like big pie in the sky. Me and you were at the AWS reinvent just this past year, and we're talking about, you know, this idea of putting, um, digital currency on the blockchain.

Now you're thinking that, I mean, I've seen, uh, with Doge this idea of putting all government transactions on the blockchain and, and, um, all for transparency, but we end up with these really interesting, uh, dilemmas from a have and have nots perspective. And one of the things I wanted to highlight from this Ellison, a subset of the Ellison comment, was, uh, vice President Vance in Paris announcing that, you know, he, the, this administration wants to take the shackles off of ai, uh, and, uh, ethics. And, uh, I think his, uh, term was somewhere around something to the effect, we need to stop hand wrangling around ethical AI and just advance AI as fast as possible to beat China.

All the VCs celebrated the comment. They were like, yes, this is a, uh, this is the way to beat China and ai, the consensus is that China is level with us when it comes to models, if not ahead of us, when it comes to models. The u when I say us, I mean Western, uh, yes, powers and, and, uh, companies.

And then, uh, as far as hardware, uh, the US and Western allies are ahead of China, and then people, the argument is that China's ahead, when it comes to pe people, the, uh, people, process and technologies, I think this, it rises a interesting dilemma for the enterprise. Usually US regulations have gated our access to some technologies when they take that, those shackles off. Of course, there's opportunity, but there's also challenges in a global environment.

The EU seems to be going in a completely different direction than the us. Well, and if you look at, um, you know, responsible ai, I think is, is really the word more than ethical ai. 'cause there's more than just being ethical that you have to have to do with the ai.

You have, you know, you talk about first principles with great power comes great responsibility, and AI gives us, is the, is the most powerful technology we have. And sooner we'll be able to do things humans can't. As I've talked about super intelligence, uh, you know, in the last show, um, what, you know, who's gonna, who's gonna have access to the best super intelligence, uh, that's gonna be a competitive global race.

Europe, on the other hand, is saying, well, we'd rather be responsible more than we would want, want to be number one. And they may win that bet. If, if, uh, we end up having, uh, which I think is likely some, some accidents with ai, where it's used in a place that causes, uh, significant, uh, harms somehow, um, as people put it in more and more critical places, uh, and makes decisions without human intervention or human oversight and bad things happen, then Europe will be able to say, well, you can come here and use our AI because it's, it's safer and more responsible.

Uh, we actually have regulations. So they may, they may win that bet, who knows? Uh, and they're trying because, um, uh, the EU just announced that they're, they're, uh, they're trying to organize it at a, uh, at a national level across the eu, uh, because individually the, the countries can't compete with US or China, but collectively, uh, the eus got 450 million people.

Uh, they're bigger than the United States if they could get or get organized. So they've just committed 200 billion, uh, to, uh, their AI efforts at an EE level, uh, to compete with us and China. And, you know, of course this story's gonna be, they're gonna do it be they're gonna do it best in terms of what's right for people.

And, you know, it is, it's the, it's nice to have another horse in the race to see what, where that goes. Yeah. Diversity of ideas is just as important as competition, which actually brings me to a point we did not put in the rundown.

I don't know how we missed this as big news Elon Musk offered and, and, and some other investors. A group offered 97 point something billion dollars for open ai. And this is a little complicated.

Open AI is technically a non-for-profit. So I'm not quite sure how you offer to buy a, not this is beyond my, my, my financial expertise, but essentially he's looking to take control of the, uh, uh, of the not-for-profit that manages the for-profit open AI company for 97 billion. 7 billion.

Yeah, I saw that. Yeah. Well, it's interesting to see, um, the, the question is, the rumor is that, uh, the third run of grok, so GR three didn't go well.

And so, um, it, it's, they don't have a model. They, it has spent an enormous amount of money and they don't have enough to, to retrain it. That's the rumor.

Can't confirm whether it's true or not. Uh, but, uh, that's supposedly the reason where why, um, uh, Musk is making some kind of move to at least get something out out of it. I don't think, uh, I think, uh, open AI is actually worth more than that offer.

But I think that there's, there's there, I think there's secret covenants or stock or something. I mean, supposedly Musk dives, uh, divested himself from open ai, but he has some in on the offer is my understanding, um, that make, that gives him special privileges to make that. Um, and, but I don't think it's gonna happen.

Uh, but there is some question about whether GU remain competitive and, and some must have to do something. Yeah, AI is moving much faster than we can keep up with both on the software side and the hardware side. Long, internally speaking, our sister company, signal 65, has published the results of their GDI three testing.

I see. com to download the full report. And if you want to really geek out on numbers, our colleague Russ fellows wrote the report after all the testing, and we've been testing it for quite some time.

Basically, it's on par with the H 100, which if you, you know, kinda look back and say, well, you know, H one hundreds h what, what's the big deal? Well, deeps seq was trained, allegedly on H one hundreds, uh, rock, uh, Xi uh, built their data center around H one hundreds, their super compute off H one hundreds while the B two hundreds. And the, uh, new chip sets from, uh, Nvidia all the rage, H one hundreds and Gaudi three is absolutely, uh, data center level quality for doing infra, seeing and doing enough AI for the vast majority of the enterprises.

Any thoughts in general around just the non Nvidia market around AI chips and AI chip acceleration? Yes, and the big question is, uh, in ai, it's what are developers gonna use? And, um, if it's not Cuda, there's a big question about, you know, do I make it, do I commit to something that may not last?

It doesn't have the heritage, uh, it doesn't have the, you know, the staying power has is Cuda iss not gonna go anywhere. Every, everyone feels completely confident making these very, very large investments on top of that stack. Uh, and the question is, is, you know, chips like gdi are new kids on the block with unproven track record, um, assure they're doing great today, but can they sustain the, the competitive races betting on gdi good?

Is it gonna pull ahead? Uh, is it, is it cost competitive? Um, and but most importantly, can they get developers to use it?

Um, and developers don't dunno anything about it. Uh, and so, um, that's, that's the real question. There's a, is a huge headwind to break out in AI chips right now.

It's just, you know, NVIDIA's got a lock on the market. Yeah. I will say, if this was Intel of two or three years ago, it, you know, we would say, well, it's Intel now.

We're saying, well, it's Intel. You know, it's, uh, we, we could put, you know, a lot of faith in Intel two or three years ago that they would just spend their way to, uh, market parody, if not dominance. Now, this Intel has way more problems than just catching up in ai, AI development, AI platforms.

Uh, they did a lot to normalize the access to, got to, I'm sorry, to Cuda and Cuda Court. Uh, so they could hire, they, they can hide, hide away the complexity, but you're absolutely right, folks like deep seat, deep seeks, uh, show what's possible when you, uh, program to a machine level, uh, and, uh, bypass. Well, That, that's right there is, that's the intelligent advantage.

They need to tell that story that, you know, some of the most competitive ais are not using, um, a cuda. They're able, they're getting the best performance by going bare metal. And so that, that's very interesting.

Yeah. And the, the, I think someone should commission our Signal 65 folks to do that comparison to say, oh, if I went straight machine level on Gotti three, what does that, what does that get me relative to a B 100, 200, or whatever the latest Blackwell chip set is from Nvidia? And is it worth the investment?

What does it mean for my power and cooling? One of the things that we haven't talked about when it comes to deep seed and being able to basically run, uh, the entire model on a single Blackwell chip on like a digits, which is just air cooled, what does that mean for the power and cooling race specifically in the enterprise data center? Yeah.

Well, and I think that this whole conversation, uh, neglects the way that CIOs think about sourcing the job of the CIO is business continuity. At the end of the day, that's the only time they ever really get a phone call from anybody was when something doesn't work. Um, you know, OO other than that, they, you know, occasionally they, you know, they get a request, well, we'd like to do this or that, or use ai, but, um, uh, so they're looking for, uh, a, a stable, reliable partner.

And they're, the question is, is, uh, the, you know, the Chinese government has been running these drills around Taiwan that raised a lot of supply chain questions about whether Taiwan something's gonna happen, um, within the next 24 months. That seems to be the analysis. Um, they're, they're, they're, they're basically dry running, you know, a blockade of, of of the, you know, it's a tiny island at the end of the day.

Um, and with everything changing in all eyes, uh, elsewhere in the world on all these other geopolitical issues, there's a, there's a real concern that, that China may make their move. And so what I'm hearing from CIOs, uh, in my last CIO round table, uh, that, that I participated in with a bunch of Fortune five that are like Fortune 100 CIOs, is that they're looking, uh, to hedge their bets so that they don't have, uh, operational, uh, discontinuity when they can't get the ships they need to run their business. So they're looking at the, at that level, and this is not the developer level, um, but developers of the king makers, but the CIOs now increasingly their eyes looking, what are my other choices?

And this is about gaudi's really their, their biggest chance they can tap into that demand to hedge bets about what's gonna happen in the world that may cut, uh, you know, uh, US enterprises or everyone off from Nvidia chips. Uh, that's a big deal. Yeah.

Between tariffs, geopolitical challenges, when, when it comes to borders, we as, uh, we as a industry are appreciative that we don't do that. I don't do logistics. I'm, I'm just happy that I'm not in to the business of figuring out where I'm going to get supplies.

The big OEMs, O world, Dell, HPE, Lenovo, Cisco, the people who have to put these systems together in the end point, really have to think through how do they one not, uh, get on the wrong side of, of, of each one of these nations from a political perspective, but also able to mitigate some of this cost and access risk. It is a interesting conversation and, uh, we're, we're here for it. Diane, let's put in a, a plug since, uh, Kaly isn't here, we can put in a plug for our CXO round tables.

Oh, yes. You dunno this, but we just signed on yet another client as of I think, uh, a few minutes ago to do another CXO round table. You wanna fill us in on what is a cs?

We call 'em CXO salons. Yes, exactly. What Do, what do we do at these things?

Well, the, these salons are for organizations that, that want to directly tap into the zeitgeist or, you know, what are CIOs? Where CTOs, chief Digital Officers, if, if you prefer, uh, any of those digital CXO roles, uh, that you have, uh, you know, we've built up a community of such practitioners over the years. Um, we can bring them together, uh, to think about and, and discuss the, the challenges of the day, evaluate new product offerings.

So, uh, and their, their motivation is, you know, increasingly they're not feeling like they, they don't have a, a voice in a lot of product development in, in the enterprise, in enterprise products. 'cause it's moving so fast and being driven by Silicon Valley innovation, uh, that the, that a lot of their specific unique enterprise needs aren't being represented. This gives them a chance to get with leading product companies, uh, who are trying to build the future and get feedback from them.

But it can be used for many different things, uh, just as a, as a sounding board, as a, a forum to understand what the major issues of the day are around a specific topic that affects that, that given, um, a product offering. Uh, so there's a variety of ways, but if you want digital CXOs in a room, uh, we, we have gotten good at facilitating that, bringing them together and creating productive, useful, actionable outcomes. Concrete stuff comes outta these and everyone really seems to enjoy the process on both sides.

I know. I enjoy the process of hosting these, uh, on bo as you said, on both sides. The CXOs, the various executives from actual end customers, and the vendor, the IT enterprise vendor gets get both get wonderful insights and we get the pleasure of just hosting these things and, and, uh, watching these conversations unfold.

There's really not much that we have to do other than throw in a question and watch the magic happen and keep everyone on track. com. Uh, Diane, as usual, I've had a wonderful time.

If you're thinking you've learned so much more, so much about the tech industry from these infrastructure matter episodes, you should share them with your grandmother. That's Right. It'll help her make more friends, whether she's at home enjoying the, the sooth soothing sounds of Keith and Dion, or if she has a social group that she can, uh, listen to this while paying bid with it's a gift, or it's a, we're recording this on Valentine's Day, so it's a great Valentine's Day gift for any of your grandmothers in your network.

Until then, we'll see. Next episode of Infrastructure Matters will be B 72.