Broadcom & OpenAI: Reshaping the AI Chip Market | TSG Ep. 945

Is Broadcom gonna be the big winner in this whole AI chip race? We'll find out you're watching Textron Game. Hey everybody.

Welcome to the latest edition of the Techstrong Gang. We have some of our usual stalwarts, Dan O'Brien, Mitch Ashley, Chris Blak, and we're gonna have a chat about some of the latest and greatest events of the day. Starting off with this whole deal between OpenAI and Broadcom for custom chips.

Dan, let's just get started here. What's your take on what's going on here? Because as far as I can tell, all these companies are making their own chips at this point.

Is that where we're headed? Well, certainly a lot of 'em are making their own chips. A few of 'em still buying from Nvidia too, I think we know.

Um, but you know, certainly Broadcom has been the leader, you know, for a while now in, in really what we call XPU, these custom AI accelerators. Uh, been working with a variety of companies in that space. You know, Google being the best known on the TPU side.

Um, and the kind of news that we got today is, you know, seemingly the, the big announcement of this week, we get one every week these days. Um, you know, about $10 billion, 10 gigawatts of silicon being purchased by OpenAI, uh, in partnership with Broadcom. So, you know, likely, uh, uh, clusters that are gonna going towards inference.

Um, you know, open AI has long been rumored to be interested in making the road silicon. Um, I think, you know, Broadcom's long rumored to be that potential partner. And, you know, now we get to just a sense of the scale of the investment.

Um, this follows, you know, 10 gigawatts of capacity from Nvidia, six gigawatts from a MD. So open AI has certainly been on a buying spree here. Uh, but you know, I think they're really trying to rationalize their training needs as well as their inference speed over the long term.

Right. You know, I think we're gonna continue to see annual upgrades of NVIDIA equipment on the training side. And, you know, for those more stable inference workloads, they're gonna, you know, really optimize on costs for power.

Um, and, you know, really Broadcom seems to be the winner here in terms of that being that preferred open AI partner. You know, do me a favor for the uninitiated. Split that conversation a little bit for me.

'cause I think people get confused about what chips are used for what, but there are some chips that are lend themselves for training, and then there's inference, which is the deployment of the AI workloads. So these are not really overlapping deals. They're kind of complimentary, but sort that out a little bit.

Yeah, Absolutely. I mean, training is really where we see much more of the model building as you get into, you know, new iterations and next generation models, um, new data sets coming in, new, you know, training and rever, uh, reinforcement learning techniques. You know, that's really where you see the GPU shining.

Um, and then, you know, on the inference side is where, you know, making calls, you know, entering queries, entering prompts than, you know, churning those tokens to get a response back. That's really the inference side of the equation. So, uh, different needs, um, you know, one's at a completely different scale than the other.

You'll have far more inference over time than you will training, but, you know, these training runs are incredibly expensive. Uh, the ability to do them quickly, uh, is really important as you think about time to market in the model wars. Um, and, you know, I think we're seeing a little bit of the maturing of the AI compute story where you're really starting to get into that specialization and customers are really starting to rationalize across not just performance, but costs and what ultimately seems to be a big gate right now, the power side of things.

Yeah. Mitch, this is another one of those $10, uh, sorry, try that again, Mitch. This is another one of those $10 billion deals that have seemed to be commonplace in the AI thing.

But the one thing that kind of leaps out at me a little bit here to what Dan was just talking about is, are we separating training and inference? And is inference gonna be managed more by traditional IT folks, DevOps folks who are gonna be optimizing the workloads when they're deployed and the training will stay with the data science teams and we won't have one data science team trying to do everything? Well, there are two, two very different workloads.

Um, it's a really good point. Uh, and not everybody's in the model building business, right? Most of us are not gonna build very, very large, maybe even small models, at least for the time being.

It's, it's a, it's a big cost, it's a lot of effort and certainly a specialized skill, but inferencing can apply to anything, can apply to any app that's using, you know, ai, AI algorithms and processing those models. So I think we'll see much more common kind of farms of inferencing, um, across edge devices, you know, centralized data centers, et cetera. And that's where we want to be able to push workloads, uh, to where that's best to happen.

So I think that's, that's probably also why you're seeing a more plethora of deals is, you know, we, we talk so much about NVIDIA's latest, whatever the generation is, and how much faster it is and how much more power it takes, because that is such a huge, massive workload to train models versus inferencing. Yeah. It's a, it's a big workload, but not nearly the same intensity.

Mm-hmm. Chris, does that make sense to you as well? Or what are you seeing?

I Honestly, I like the direction this is going. I was thinking, looking at this, this segment more about, you know, ChipSoft re and, and owning the power and the fabs and so forth. But this is a good point.

We've talked about this a lot over the weeks, right? You know, these workloads are different things, you know, just scaling massive data centers to do massive things, uh, and using that for everything doesn't make any sense. So I think, uh, I like what mentioned and Dan was saying about this, you know, like, you don't need all that for all of this.

And every time we say that we need all of that to do anything, it shows that we don't understand the, the Systems we're building. Mm-hmm. Dan, another thing about this is that it feels like to me that the inference workloads are gonna be more distributed and there's gonna be a lot more networking involved.

And it, there's a whole other element of the stack here that goes into this thing that people need to think through. Correct? Uh, absolutely.

Yeah. And I, I think with the exception of somebody like Google who's doing this, uh, they're training over multiple data centers because they've, you know, got their own TPU, um, you know, most of the big model trainers are really trying to almost train within a single data center complex because of the, you know, the networking complexity there. So, uh, you know, to Mitch's point, there's gonna be a, a very small number of companies that really need to worry about these large training clusters.

OpenAI is obviously one of them. And, um, you know, to be, to be determined whether there's a real business model just in making models, right? I think we're seeing that layer of the stack is highly commoditized.

You're really able to monetize that, the application layer and, you know, the application layers where a lot of people are gonna have inference workloads. Um, so, you know, I think it's a, a very different type of workload. Very, very different scale in terms of the number of companies that will need to worry about training versus inference.

Inference is something that everyone needs to worry about. Training is really gonna be at the hint of you. I wonder, Mitch, will we see some blueprints out of this kind of thing where maybe, you know, open AI and Broadcom will get together and say, Hey, here's how you deploy an inference engine and make it simpler for folks.

'cause one of the things I think out there is that there's not a lot of IT people with expertise deploying inference engines. Well, I think to some degree that's something we'll see, I think we're gonna talk about in our next segment too, is part of building AI applications or agents, et cetera, there are frameworks to build these within, and that's where you get some of those blueprints from. And I need a sandbox to be able to perform those actions in.

So as we get more and more of the frameworks set up, I think that'll help us with what are those patterns, those blueprints, et cetera. And, you know, just as it matures more and we figure more of that out, I think we'll see, uh, those kinds of things. But there's all kinds of frameworks.

So we haven't begun to talk about around AI and agents, but we'll get a chance too soon, I'm sure. I'm not sure OpenAI wants to provide the framework. I, I think OpenAI really wants to become a CSP, right?

I mean, I think this, you know, they're, they're trying to somewhat cut out the middleman. You think about the margin stacking from semi equipment into fabs, you know, fabs into FLIs, into, you know, the kind of end customer, um, you know, they're really trying to decrease the number of layers there because there's so much margin squeeze as you kind of go further down the stack. So, you know, I think we've traditionally thought in the enterprise about, you know, really three or four big hyperscalers AWS, Google, Microsoft, Oracle, you know, I think we, you know, in the AI world, we're gonna think about meta open ai, you know, and maybe a handful of others, maybe one of the neo clouds gets there, you know, as kind of these, these large CSPs.

Well, let, let me take frameworks for a second because this is one of these things, and I spent a lot of my, in the last half a decade and, and far my career a lot of time in working groups and frameworks and, you know, yeah, God, help me, you know, how many times are we gonna take six to 18 months to get a draft of the first version of the whatever and in this world, and I'm coming hot out of that space and still in it, um, during this transition period. And just the idea that we're gonna take 12 months to agree in a draft of a framework, for example, or that we're gonna wait for one big monolithic company or, or a coalition of three monolithic companies that were period to 36 months. No, that's not the way it works.

And this gets right down into, I think, the nature of this, where there haven't been a lot of conversations, I haven't been part of them, but how do you have a working group? How do you come up with frameworks? What does frameworks even look like in a world that moves to the speed?

You know? 'cause we have multi-decade framework development processes that, that produce individual handfuls of documents and frameworks that slowly coalesce. I think that goes to 11 starting about now, you know, a framework again, you know, you know, we all do different things when we're not, you know, necessarily all on the camera.

And we are involved in these, in developing frameworks and propagating them from the bottom up. And is that a way a lot of these things get developed and deployed as opposed to waiting for the, from the top down? I think too early to tell, but it's not, you know, this is not the, you know, any be a pick one.

This is the, not the NIST cybersecurity framework, uh, redux. This is frameworks and, and, and processes and schemas that, that propagate much more rapidly. It's a good point, Chris, because we use the term frameworks in a lot of different ways.

Uh, you mentioned NIST frameworks, right? We're really, it's really a, a plan, uh, a process set of guidelines, things to implement. It's a structure within implementing security, for example, um, in the software world, we also use the word frameworks to mean the scaffolding the code that actually you need to be built around your agents to operate it so you're not building the whole pipeline and how it works and how it starts up and how it fails over and how it communicates, all of those kind of things.

So we call that scaffolding, but those frameworks, just like there's frameworks in code for doing user interface, um, and data communications, all data analysis and data transformation. So, um, when people are building software building agents, they're typically gonna be using some kind of a framework, quote unquote, which means libraries of code or even operating code to run those agents within some kind of a context, Right? So, Dan, I'm gonna use a technical term here, but one of the things that I like about this whole deal is it's pretty straightforward and there's not a lot of what I might refer to as financial engineering shenanigans involved.

So, um, can you compare how this deal is to all these other deals that we've seen lately where there seems to be a whole lot of co-investment and a lot of people are confused? Yeah, I think this one is a little bit more straightforward. It, it does seem like it's a little bit more, you know, kind of cash for hardware, um, if you will.

You know, I think the Nvidia deal with OpenAI was much more around, you know, kind of the, the data center site build itself, um, and less, you know, kind of paying them to then buy back, uh, you know, buy back the GPUs, uh, the A MD deal with OpenAI, you know, is probably the most unique of all of 'em. That's really, uh, you know, kind of more of an equity driven i, uh, deal where, you know, at a certain level of kind of purchases and a certain level of stock price for a and d, you know, uh, OpenAI has, you know, the opportunity to kind of take an equity stake there. So, you know, very different, you know, kind of structures.

Um, it's certainly getting creative here. You know, I think the one thing that gives me some comfort that this isn't a real sign of a, you know, the bubble, uh, is that these are, you know, largely very profitable companies essentially committing their future free cash flows, right? This is not a, you know, not a debt driven, um, you know, purchase environment yet.

It's not the late nineties of financing startups with, uh, leveraged investment, right? Yes, absolutely. So, following that up one more time, then, will we see the open AI rivals do the exact same thing?

So philanthropic will kind of maybe sit down with AWS and design a chip, and, uh, other folks will be with Google, and this is just the first in the wave of these things. I, I certainly think Anthropics a big AWS user of, you know, their training trips and, um, you know, some of the other silicon in their portfolio. Uh, Andros a a different level of complexity, right?

You've got big AWS investments there. You've got big Google investments there. Uh, you've also got a lot of, you know, a lot of VC money involved there.

Um, and Tropic certainly operating on a different level in scale than open ai, though, you know, and Tropic, I think is really, you know, kind of centered in and around the code use case, you know, more than that general purpose. Mm-hmm. And then what does this mean for Intel and all the chip makers?

Are there, you know, are the folks at Intel sitting there chewing their sleeves off because they didn't get this deal? Or, you know, what, what are the implications for the rest of the semiconductor world? I, I think we'll need to see, right?

You know, uh, to some degree this may mean, you know, the Intel announcement is a week or two away, right? You know, you, you've kind of seen, uh, seen OpenAI filter through kind of the who's who of, of big compute and big AI accelerators with Nvidia a MD. Now, Broadcom, um, you know, I think with a lot of the government support going behind Intel right now, um, you know, wouldn't, wouldn't surprise me to see, you know, either some manufacturing deals or, you know, some outright deals to purchase, you know, XX 86 silicon or, you know, some of the accelerators going out even until, So is there a snowballs chance in hell that Broadcom will use fabs from Intel?

What do you think possible? I think it's possible for almost all companies in the, the American semiconductor ecosystem right now to, to think about using Intel. Um, probably hard to spray d to get their head around.

Um, just because the governance structure, you know, in place at Intel, I think they'd wanna, you know, really see a little bit more separation in governance between the foundry side and the design side of the house. But, uh, you know, whether it's it's Apple, whether it's AWS Google Rocom, Nvidia, you know, be it the actual semiconductor wafer foundry, or, you know, even more likely, I think the, the advanced packaging side in Dell, um, I, I think there's gonna be continued momentum in that direction. Mm-hmm.

Do you think that's because of the US investment, Dan in, in Intel? Yeah, I think the government's actually been pretty transparent about things. If you're willing to listen and there's quid pro quo on the table, and, you know, seems like everybody's getting on board.

So let me ask the next logical question, at least in my mind anyway, Dan, am I gonna wake up one morning and see like a massive merger in the semiconductor space? Because somebody's just gonna say, well, if we're orchestrating all these companies in some sort of national policy, well, you might as well just bring 'em all together. I don't think so.

I mean, I, I think, you know, you've gotta get through, you know, China's mofcom, you know, essentially the, you know, equivalent on our side of, uh, of, of, you know, of, of the reg regulatory body. I, I don't think that's really possible. I mean, you know, be at the semi equipment side or the semiconductor side doing large scale acquisitions has gotten really tough there, but there's been a lot of these that have been, you know, kind of shut down.

And I think, uh, you know, probably a little bit more of an aversion to trying to do these, given the recent history of regulatory bullies. All right, Mitch, I'm gonna throw this one to you. So, am I reading this correctly?

A little bit, but we kind of built this AI stuff on, you know, we found a bunch of GPUs, they seem to work, and away we went. But if I look at this custom chip, am I gonna see more of the lower level frameworks and software that we've been using to build some of this stuff embedded deeper into the chip itself? And maybe the processing will get better and the overhead will get better, and maybe these open AI things will become a little more efficient.

And who knows, maybe we might not need as many data centers. Follow my logic. Oh, optimization, Hmm.

Efficiency. Yeah, that's my Halloween voice, by the way. Yes, no, absolutely specialized, more efficient, um, power efficient, and as well as increasing speed and processing, you know, and workload.

We definitely, we'll see, uh, we'll see a lot of innovation. We think of, we think of inferencing in, in GPUs for model training as kind of big monolith process. There's already specialized chips for doing this today, and it'll do nothing more than get more specialized.

Chris, is that on your wishlist? More stuff at the chip level? Yes.

No, I'll take all of it, please. Thank you very Much. Just want one on your desktop.

That's all right. Right. There you go.

Under the, and do that one for you Perfectly sooner than later. All right, folks, the world is changing one more time, but I got news for you, man. It's all gonna be happening at the semiconductor level, and these deals are all coming fast and furious.

And what we see today is gonna look pretty antiquated. What from what we're gonna see tomorrow, gonna be back in a minute. You've earned it.

The spotlight, the responsibility, the weight of teams, companies, and entire industries fall on your shoulders, lives depend on your decisions, your home life included that work. You are protected physically and digitally. Nothing gets through your team without a fight.

But in a globally connected world, everyone sees you, including those who mean to cause you and your organization harm. And now home your sanctuary attackers see an opportunity. Your digital front door is wide open.

And what compromises your home can breach your boardroom. Because the devil's greatest trick isn't targeting your workplace firewall. It's convincing you that your personal life isn't at risk.

Black cloak, digital executive protection, defending the new attack surface your personal life. Hey folks, we're banking as Mitch alluded to. Yes.

The second segment is related to the first segment. We're talking about how AWS is now made generally available, a serverless computing platform designed specifically for AI agents. And the basic idea here is that they have like seven primitives that are all exposed through a single API and from my IDE or, uh, my favorite AI tool, I can now just go build and deploy AI agents, no DevOps people or IT people required in a sense, AWS is the IT and the DevOps people for this thing.

Mitch, I can't help but wonder, you know, we've been talking about serverless forever, and at one point we were like, oh, serverless is gonna take over everything, and then it didn't. But are AI agents gonna be kind of like killer app for serverless because well, they're just so damn spiky. Well, you think about it, um, a agents are natural for serverless, and I don't mean in a native lambda sense.

Um, but agents are really kind of meant to be small, uh, do workloads of specialized kind of processing, uh, reasoning, et cetera. And when AWS announced Agent Corps, I mean, it was, you know, GA now, but it was really just in July at the New York event that they launched it, that was setting in motion what the next kind of development cycle looks like, which is now being referred to as ag si agentic development life. So how do you build applications or build functions, build software that it's composed of, uh, co uh, components agents that are meant to operate genetically.

So you've gotta have this environment for them to do that. It doesn't help us to write agents and then have to build whole environments to operate 'em on. We wanna be able to set those things loose in a sandbox environment or, or in a production environment.

And that's, that's exactly what Agent Core set about to do. It didn't, doesn't solve all the elements of what you would have in a traditional software development life cycle, but it does a lot of 'em. Um, I mentioned earlier, now this is GA now, so relatively short period of time, just a matter of months, you know, this thing is now, you know, ready for use.

Um, but it also is supporting of multiple frameworks. Um, Microsoft just launched their agent framework a few months ago, maybe in less than a month ago. Um, but you've got, you know, land graph, open AI, agent, SDK, um, I can't remember the name of all of 'em, strands.

There's a whole bunch of them, crew ai, uh, that have all come about and people are starting to use, we'll see if sometime to tend to take off more than others. So that's, those, that's the framework that those agents are built around. The, you think of, of them as an SDK.

But then also there's, uh, something really interesting about Agent Core is ha it has agent core code interpreter, which is a safe execution environment for code to be able to run. So that's how you can kind of keep kind of think of it as memory safe, like we think about in languages like rust. But it also has a, a, a browser functionality for web interaction.

Um, if you write Python code, you know, you're very familiar with libraries that do that, a runtime that scales the infrastructure, and then a gateway that can talk to functions like Lambda and other, of course other services. So it includes A MCP and H two A. So that's that execution environment that we're talking about, that what it was, what Agent Core is about.

So I'm excited to see, you know, people start to use this and take off. And of course, AWS is talking about Clearwater Analytics, Ericsson and Sony and others using this already. I'm sure there are a number of companies that are using Agent Core.

Mm-hmm. Dan, am I gonna see Microsoft and Google follow suit? I suspect that they already are.

I think Google has something, an enterprise initiative that has some sort of serverless framework in there. But is this gonna be like the next great infrastructure battle around AI agents? Yeah, listen, I think all hyperscalers are trying to make it easier for developers to, you know, build and deploy and manage agents, right?

I mean, I think what they're, they're doing here is putting a nice wrapper around, you know, really what are the kind of winning open standards in the space, you know, like MCP and eight A, uh, they're making it so that it really plays really well and what really nicely within their existing cloud services ecosystem. Um, and they're solving some of the harder challenges that, you know, everybody is worried around agents, right? You, you've got the identity challenges, you know, around, uh, identity access management, you know, the observability, right?

What are these agents doing? You know, what are they, where are they going? You know, what kind of, uh, you know, compute capacity are they consuming?

So, you know, I think it's, it's really about kind of removing the friction and, you know, just getting their developers up and running. Because all of these cloud service companies know, the more they get their builders out there building agents, the more compute they will consume and the higher the bill will go, right? Like, it's a nice little self-reinforcing, uh, you know, e economic loop for the cloud provider.

Big bills coming your way. I'm Chris, here's the part of this that leaves me perplexed. So maybe you can sort this out for me.

So theoretically, I want the AI agent to be deployed somewhere as close as possible to where the data it is consuming is. And most of the time the data is in an on-premise environment or some local data center. Um, you know, there's data lakes up in the cloud, obviously, but I don't know what percentage of everybody's data is up there.

But ultimately is the, where the data is located gonna drive the decision about where to deploy the AI agent? Or people just gonna start, you know, buying those big, what are they, I forget the AWS rename them, but they would called snowballs and they were basically big boxes that you came and loaded on a truck with your data and then it showed up in the cloud like a week later. I don't know that I can answer that question.

I think the answer is, is yes, right? But, uh, I, I'm like, I think of the pros and cons of this and, and Dan and Mitch, you know, covered the pros pretty well. You know, it, it, Dan used the, the word, it was on the tip of my tongue friction, right?

You know, if we can reduce the friction for not just the existing dev teams, startups, you know, how we, how fast can I have access to these things? How much upfront work do I need to do? Uh, that's fantastic.

I worry about, I see lock in city, like, holy cow, you know, you know, now, now I never get to do anything else again. And transparency, right? Like, who cares where my brain's running?

As long as it answers fast, you know, I want receipts for where things are happening and, and, and loads and so forth. Um, but, you know, data residency, yeah, there's a lot of issues up and down the stack, but this feels to me like one of these, I couldn't say better than the, than than YouTube to begin with, because this is happening. It's too easy, it's too good.

So look at the downsides, you know, can I tell what's actually happening? Does my, my ease of use as a dev team or as a startup and so forth, lock me into choices that I may not, uh, may not have really signed off on? And these early stages are exactly the times when those sort of mistakes get baked in.

And we talk about 'em a lot later on, Oh, Chris, Chris, come on in, the water's fine over here. Don't be concerned about any of those things. Oh, yeah, We're not, We're not gonna do anything to try and lock you in.

Now. It's all about customer acquisition, right? Getting their work, getting their workloads on there.

And, you know, the open standards are some things that help you with that. But when you're in agent Core or you're in a different, uh, execution environment, you know, you, me, Dan, you mentioned, you know, observability, security, security guardrails, memory, memory, management of agents, all those things are part of that scaffolding. You need to, to be able to do all that.

That's what these environments and frameworks all do. And you don't, you don't unhook those things easily. It isn't like, oh, good, let's just take that and drop that in another cloud, right?

It, it doesn't quite work that easily. You have to re-engineer it. It may be easy, it may be hard, but the more I can get you to learn, use and like what I've produced in terms of our environment agent core, in this case, all the better.

Now, oh, the other thing I was gonna mention too is there's digital sovereignty issues around this. And that's an interesting dilemma. I mean, the, the hyperscalers have responded to say, here's how we'll solve digital sovereignty in these areas.

But I think you'll have other people who will come up with in execution environments, other vendors that aren't hyperscalers, that you can do an on-prem version of that in your environment. You, we haven't talked about, is there an on-prem version of Agent Core yet? Not yet.

Anyway. There might be one day soon if AWS follows. Its usual playbook, though.

Although Dan, I was thinking about these issues too, and then I was like, well, these things are all true, but going forward, a might not become easier to move my data from one platform to the next. And AI agents are not kinda like full boat apps. They're kind of like little disposable things.

And maybe I can not worry about the fact that I need to migrate one. I'm just gonna blow it up and replace it with another one running over here somewhere else. Yeah, I mean, I think you're going on the, the path that I wanted to talk about here.

My, uh, Mike as well, you know, when I didn't hear and enough from, uh, from AWS was, you know, are we actually building this for the human developer or are we building this for the vibe company agent, right? You know, uh, you know, the, at this point, you know, the, the main user of these, you know, agent builder tools, maybe agents themself, um, and to your point, Mike, you know, that may as simple as, you know, in your vibe, coding prompt, you know, alright, great. Now this works.

Now build it for me over here, you know, using this proof of concept, uh, you know, as, as, as a framework, right? So, um, you know, I I, I think the, the world of development is that is changing so incredibly quickly. Um, you know, I I I'm starting to wonder who is the target user, right?

Is it, is it a person or, you know, is it the, uh, is it the ai? Oh, everybody loves a good conspiracy theory, but it's true. And actually that's going that direction, right, Dan?

It is. Absolutely. But Mitch, you know, should we not get too attached to our AI agents?

Are they in effect, essentially disposable code? Oh, they're like Tribbles. You know, you just, you have one you're gonna have, have a lot soon, soon.

Um, you know, but they are disposable. I think that's a really important point that you're making. There is a, AI is creating agents for us.

It's already doing that today, right? We're using, using at least code assisted, if not largely code generated by AI to do much of this. Um, so why not have a, why not do a AV contest, right?

I'm gonna have three different agents built by different models and compete, see which ones perform, which ones I like best or easier to manage which ones can scale better, whatever it might be. So I, I think we'll be very unattached to our agents in that way. I think the things that are agents that are more like personal assistants that know me and know my preferences and you know, what, what kind of emails I like to respond to or not, or all those kind of things, that's where that preference will gain an attachment to.

And, you know, we'll have the rosy robots, you know, from the Jetsons that will, you know, mope around the office. And I, I'm glad, but because we keep using this acronym AI to stand for everything, right? We may as well just say magic at this point, right?

And, and, and ai, uh, a semantic, these are semantic systems, right? So large language models, word engines, you know, call 'em one, two, one. And a large language model, you know, as an agent, performing tasks in an industrial workflow is a semantic engine doing semantic things.

And you said it, you know, a, a civic ai, you know, some, some named entity, the persona that knows who you are, who it is, where it's been, where it's going is a different thing. And, you know, and acronyms mean things. You know, I am still p****d off that s security information management got embedded with an e events security information and event management, which is 'cause events are not information, right?

Uh, no, the way we words we use mean things. And when we're, when we're using words about word engines, it's kind of important to get it right. So an AI agent is not artificial intelligence.

It's a semantic, uh, workflow tool and a persona, personal assistant is artificial and intelligent. Fine. We can use that if, if we want, but there are different things entirely.

You know, the fact that we're using electrons here doesn't make, make everything, you know, just electronic systems. I've known you in a long time, Chris, and still haven't gotten over that sim issue. Okay?

Oh My God, I'm gonna, I'm the Words we use, right? I'm, I'm gonna end this here, but Chris just gave me a great idea. I think we're gonna have a whole episode for fested this day and it'll be the airing of grievances you, lot of AI grievances.

Yeah. Alright. All right folks.

We'll be bagging in a minute with our last block Discover Techron group, the epicenter of tech innovation. We are your go-to for reaching IT leaders and practitioners worldwide. Our secret impactful content that sparks awareness, engagement, and top quality leads with us.

You'll access editorial websites, streaming videos, virtual events, custom content analyst research, and more. Join our satisfied clients. Let's revolutionize your tech journey.

Contact us today and tell your story to the world in the most powerful way with Textron Group. Hey folks, we're back returning to an issue that we talk a lot about on this show, but EC gobs and gi and there's a white paper out from the folks at Google and a couple of universities describing how you could build end-to-end encryption into your Git repository. And that will protect them from various forms of attacks and malicious code.

And, um, to me, Chris, you know, this sounds like a really good idea, but you know, as I thought about it more and more, I said, well, what the hell have we been doing all these years? 'cause why don't we have encryption in Git repository? It seems like a natural thing we should be doing.

Well, welcome to security. You know, thanks for coming along. You know, the, the, the, the overarching frameworks and concepts security are not new.

You know, we can go back to Sun Sue and, you know, and enunciated and in our era, you know, the, the oldest amongst us, you know, said this the first time when they were the youngest amongst us, and we're not there yet. And this one, I love this one. You know, we, you know, we use GitHub as repositories, attestation repositories for everything, right?

This is the old digital bill of materials idea from 2019 that you can have. What we need is attestations between entities, nodes, call 'em what you will about things. And GitHub, as it turns out, is a good strata for that.

And when I looked at this, uh, today's segments, it's like, yeah, well, wifi, we've done that along. What are we using so far? Hmm.

Hope, you know, hope is a wonderful thing. I think this is a better idea. Um, you know, for, for our part, you know, you know, we actually do, you know, as appropriate more things on top of that.

'cause we use, you know, GitHub for everything, but should it get baked in? Yes. It could be a short segment, just yes.

How about that? There you go. Mitch, is this gonna, if we actually do this, gonna replace the need for a lot of the tools that we seem to be wrapping around our git repositories to secure them in this DevSecOps flow.

But I'm not sure this is a silver bullet, but how far do you think it goes? Well, it's interesting. We're, we're at this inflection point where Microsoft is really starting to fold in GitHub into the company.

They announced they're moving their operations onto Azure infrastructure and op the ops into the Microsoft Ops group. Well, you know what happens when you start working with Microsoft is security jumps right up and bites you. 'cause it's a big deal, right?

I can just imagine what their ops group would be saying of, okay, here's what we need. I mean, I filled out a lot of those forms as a partner and I can't imagine what the internal ones look like. My point being is, uh, they have bigger designs and they've not announced all of this yet.

There's, there's rumors about internal meetings and stuff that have happened with Microsoft where they want to take GI because it's treated as a code repository. But I remember the first time I learned about Git, the development team was putting everything in it, and they're putting their documents in there, they're putting images in there. I'm like, what are you doing?

This is a source code osi, no, you could put all this stuff in it. It really is gonna become almost like a, um, content management system, but the next generation of a distributed content management system. And if that happens, Microsoft's not gonna adopt it in a kind of un unhinged form that it's not really secure.

It's gonna have to be secured. So we could be headed down on this path where it will become much more secure because it's being, uh, subsumed into Microsoft and they have bigger plans for it. All right.

Uh, I may digress here, but I thought I heard you just say that Microsoft announced something and then they actually had a meeting about how to actually do it. But that's just me anyway, Dan, I hear they do that, they do that. It happens in in Redmond, right?

Dan, what? I'm a fan. Dan, what is your sense of, um, in the age of AI and all these agents that we were talking about earlier, um, will we have a greater appreciation for security to Mitch's point going in?

'cause you know, as Chris alluded to, we have always made it an afterthought. But is that changing in your mind? I mean, is c are, well, let me rephrase this.

Cybersecurity actually gonna be a first class citizen in these platforms. Uh, it's, it seems inevitable that we're gonna need to care about it more and more, right? As you know, it's really hard to separate the digital world and the physical world.

You know, everything in our physical world is being digitized and therefore is susceptible to security concerns. I think this is happening at a really interesting time in the market too, right? You've seen, you know, just over the past couple of years, n start to ratify new post ecr uh, post encryption quantum encryption, uh, you know, security standards.

And you know, for a big company like Microsoft to think about, you know, fully integrated GI and you know, really putting security, you know, concerns first when it comes to that integration, you know, it's maybe a chance for them to actually start to put in place some of these new, you know, post quantum, uh, you know, encryption type of standards, right? So I think that's something that's on a lot of people's minds over the next, call it five years, you know, as something that probably needs to be dealt with. And, you know, early days in terms of some of the initial, uh, you know, standards and actual offerings in that space coming out.

But, you know, companies like Microsoft are gonna be leading the way. You know, they have to Chris, you, you, you, you're a believer. You know, I, I am, I am notoriously the optimistic of the old, you know, security folks, right?

You know, I, I believe that this is a finite issue. You know, that there's the, there's the Star Trek and the Star Wars future views, right? And in, in my own world, most people are of the Star Wars view that, you know, everything's awful.

It'll always be awful. And we'll keep doing the same things. I have a more utopian view of this, and I think, you know, you know, take the, so that's enough of a caveat.

Take it for what it's worth, I think we're approaching security as just design. And again, you know, I'm a geek, you know, we're geeks. We we're a geeky company.

We're geek geek groups. So the fact that we're doing things this way, maybe it doesn't mean anything. But yeah, I'm at the point now where we can't, you remember, we're designing things and they're not security related.

We design 'em securely because that's the way they need to be designed. Uh, you just actually build it that way from scratch. And with the, the Overtalk two letter acronym tools we have these days, I think it's, you know, and, and, and, and you know, this, this topic itself, you know, GI GitHub is another one of these wonderful open source structures that arose over multiple decades that is now docking inside this massive corporate entity that, you know, as, as everybody else has said on the screen, is, is at the place that it is.

And, and, and, and we're sitting here in a world where you can say, I need to develop this system, lay it out for me, ai, and it'll lay it out. And if you have that, you know, if your AI teammate is properly oriented, my my position is they will just tell you it has to be built secure. Because those are fundamental things.

It's not about security. It's about keeping the bloody thing running, you know, staying out of court, you know, not, you know, letting down your, your, your obligations to your stakeholders. Mm-hmm.

Mitch, why don't we just blow everything up and start over again. And I'm gonna use the airline industry as kind of the example, right? At some point everybody got tired of the plane crashing and they had a big meeting and they got together and made everything fundamentally better.

And it seems to me, as I look around in the IT industry, everybody's kind of running around reinventing the same IT security wheels over and over again. They don't really provide differentiated value. They're important, but it's not the kind of thing you're gonna decide to go with one platform versus another over.

So why don't we just throw everybody in the room and sit down for, I don't know, the better part of a year and just solve all this crap. Well, I don't know if I can sign on to your comparison of blowing things up in airplanes, um, but especially since I'm gonna get on one in a couple days here. Um, so, you know, nobody take that wrong.

No, interestingly, you know, I, I agree with you, but in a different sense, and I've, I've said this before on the show, is with the productivity increases using AI to create code and, and develop software, the current model of let me create something and then I'll scan it just to find out if it's not secure, that's wrong. That that will not scale. Because who's gonna do something about that today?

It's people, right? Maybe we will have agents that'll get better at that, but really that needs to be done at the point when the code's created, it needs to be done as part of the code generation process. Security support of the system prop.

Yeah, exactly. It has to be there. Otherwise, you know, there aren't enough, Dan O'Brien's, Chris blasts, Mike ards and Mitch Ashes of the world to go chase down every one of those security vulnerabilities if we're generating 10 times as much code.

So I think it's the, the, the, uh, velocity of what we can do because of AI will force that to happen, otherwise it falls in on itself. Mm-hmm. So Dan, last question to you.

Are we missing the opportunity here? 'cause if I stitch all these segments together, so AI agents are a new way of thinking about building and deploying software, and yet we seem to wanna keep applying the same way we built and deployed software to this new model. And maybe we should take this moment to kinda rethink everything from the infrastructure to security.

What do you say? I, I say we do it. Yeah.

I mean, I, we're generating so much code with AI now that, you know, really being security as an architectural principle, you know, as I just said a minute ago, you know, security is part of the system level prompt. You know, that is probably the way to go. And, you know, listen, there's a lot of arguments out there that, you know, starting a company fresh is the way to go now because, you know, you're able to really design all this stuff from the get go.

Um, technical debt is a, a massive drag on, on big enterprises and, you know, recreating under, you know, a new umbrella. Maybe that is the way of the future. I think Dan, you know, we're, we're headed to a future of making changes.

It's changing all the code every time. It's not just adding a little bit something 'cause it isn't gonna get generated every time and more things may get generated as part of that code creation. So it's kind of a different model than the normal change something.

And now I'll test those bits of it and make sure those, that parts of it secure a whole code base could have changed from one change. Yeah, you notice, that's great. Well, it's a surprising analogy to healthy systems, right?

And a healthy mental or co or social system, same thing. You know, the, the surviving a collapse and rebuilding is an important thing. And anybody who's just a, a hacker or a entrepreneur or just like playing around with anything should know this.

You know, you build a, you build a test, you build a model, it works and you tear it apart. And if you can build it twice, then maybe you're around to something. And I think we do get attached too.

You know, we, and it, you know, as I say this out loud, I feel the anxiety. 'cause I live in this world, right? I kind of working, don't touch it, don't touch it, don't reboot it.

But once you do and better yet, take it apart and put it back together again. And if the second time it makes sense, it'll be better and more efficient anyways. And now you've got a healthy system.

If you can't put it back together the second time, maybe you never did. There you go. Hey folks, you're here.

Maybe it is time to reinvent it. In fact, the first meeting's gonna be at Dan's house at some undetermined date, but we'll see how it goes. Hey guys, thanks for sharing your thoughts and insights.

As always. They were awesome. I wanna thank everybody for spending some time with us and please stay tuned for Techstrong tv.

We have an awesome lineup right behind us. And hey, once again, we'll see you tomorrow.