AI Is Now a Weapon: Supply-Chain Attacks, Rogue Agents & the EU’s AI Act Crackdown
Today on Techstrong Gang, Mike Vizard hosts Sid Nag (Founder, CEO and Chief Research Officer at Tekonyx), Kate Scarcella (Cybersecurity Architect and Chair of the Continuous Delivery Foundation’s Cybersecurity Special Interest Group), and Chris Blask (Co-Founder and CEO of QuietWire). The panel breaks down three stories reshaping AI security and regulation.
AI Security Under Attack: How Fast AI Threats Move
Attackers now exploit AI systems within 48 hours of a vulnerability going public. That is the sobering finding in “AI Has Become Both Weapon and Target in Modern Cyberattacks.” Security teams must now rethink their entire patch cycle. Amazon adds another data point: a North Korea-linked group is behind multiple open source supply-chain attacks. The group exploits the trust developers place in open source packages. It slips malicious code deep into enterprise environments.
Can AI Agents Be Trusted?
The panel tackles a hard question next: can AI agents act on their own safely? The “AI Proving Grounds Consortium Tests Whether Security Agents Are Ready to Act” piece covers a new industry effort. The consortium stress-tests autonomous security agents before granting them real authority. A companion piece, “The Model Wasn’t Rogue. The Control Plane Was,” makes a sharp argument. When AI agents misbehave, a poorly governed control plane is usually the real cause, not a rogue model. This distinction matters for how enterprises deploy agentic AI.
Regulators Step In
Regulation finally catches up. The panel discusses how the EU begins enforcing its AI Act in “EU Begins Enforcing AI Act, Expanding Oversight of OpenAI, Anthropic and Google.” The move marks a major test. It shows how far governments will go to police frontier AI companies that operate across borders.
Together, these three stories show an AI ecosystem under pressure from every direction. Attackers exploit it. Builders race to give it more autonomy. Regulators move to rein it in.
Transcript
Hey everybody, it's Tuesday and welcome to the latest edition of the Techstrong Gang. And we're going to have a big chat today about a lot of things related to security and maybe a small dollop of AI tossed in on top. But let me welcome the gang members today.
Kate Scarcella, how are you? Good to see you as always. I can see by the bookcases you must be home in Maine.
Yes. Excellent. All right.
Chris Blask, how are you doing, my friend? How are you? Defending the borders of Canada as always?
As always, yep. Loving life. All right.
And Sid Nag, how you doing, my friend? How are you? Where are you?
Are you in New York? I'm in New York, yeah. Yeah.
All right. One of these days I got to come find you. Yeah.
We should meet up for coffee. I, of course, am hiding in upstate New York in the Adirondacks, but I'm only going to be here for another week or so, but we'll see how that goes. That said, there's this conference going on, small little event in Las Vegas called Black Hat USA, and there's folks talking about various things, one of which is a report from CrowdStrike talking about how, well, within now 48 hours of a vulnerability being surfaced, a proof of concept is starting to emerge, and that means that probably other folks can create that same exploit.
Maybe, let's say for the benefit of the doubt, give them 72 hours, maybe an extra day or so. But Kate, is this starting to scare you yet? At what point is this going to become something that becomes our new everyday reality?
Yeah, I think it already is our everyday reality, and as much as there's exploitation, there's also defenses against it. So AI is dramatically shrinking the time between a vulnerability being disclosed and attackers exploiting it. So this new research showing many proof of concepts exploits are weaponized within 48 hours.
While they're technically not zero days, and as a person who worked with a zero day type of mind, once they're public, they're increasingly behaving like them because defenders have so little time to respond. At the same time, nation state groups are continuing to exploit trusted open source software supply chains. This is huge, everyone.
So making speed and trust two of the biggest challenges in modern cybersecurity. So not only do we have to focus on patching the vulnerability, but reducing the time from disclosure to remediation. I love open source and supply chain and everything that this encompasses.
So as a person who had to think about getting that patch out for this vulnerability, this is our new reality. And we should, as much as it is used against us, we should also use it to help us. And this is something that I continually talk about.
And I think while AI is compressing this time of this exploitation is less than 48 hours, security teams need to prioritize the way we think about risk. And I think it will really help us if we start to look at assets and risk and putting them into this equation. All right.
So adding a little detail to some of what Kate was talking about, Amazon has a report talking about how the North Koreans are now finding that they can exploit various vulnerabilities, and they're using that to drive their economy. And they're probably not the only nation state with similar ambitions. And so I guess, Chris, I'm going to come to you on this next bit, but aren't we just running out of time here?
Ultimately, all these organizations that are nefarious out there, whether they're for profit or for some sort of espionage organization, they seem to be weaponizing the crap out of these vulnerabilities, and most of the IT teams that I know are not prepared to fix anything within 48 hours, maybe even not even within two to three weeks. I think you can look at it as evolutionary forces, right? So Kate, like you're saying, we have built these systems and they've been working quite fine to date.
And you and I, and we've all been involved with them in different positions. And as a vendor, I can say that there is a known, someone has disclosed to me that my product has a vulnerability, but they haven't disclosed to the outside world. Now I have this little window of time to deal with them, come up with a solution that I get to my customers, and let's just assume that's gone, right?
That by the time I hear about a vulnerability, the window is over. But let's go back a little bit further because a disclosed vulnerability is not a new vulnerability. The vulnerability existed up until that point.
Every second, every moment that whatever it is has suddenly been known, disclosed to someone, discovered by someone as vulnerable, has been vulnerable. So I think the evolutionary, the environmental conditions for all of us have been that huge waves of attacks, huge known sweeping waves of attacks are going to happen when a vulnerability becomes widely known. Individual attacks that succeed every single day are already happening and have been happening all along.
And because they're happening by people who aren't just spam kitty, finding a vulnerability, trying to break a million sites, there are people who are breaking individual sites. And are perfectly happy exploiting that vulnerability over time and maintaining presence in your network, and they've always been there. So I think the evolutionary forces are such that those who want to rely on the assumption, Kate, open source and supply chain, I spent a lot of the last decade in supply chain.
Yeah. We have the ability to enunciate the visibility in your supply chain so you can know what it is. And I think that's a little bit new, but it's not that new for the people who've been involved.
And I think it's the kind of time where over the next couple of years, those who do not adopt those sort of techniques, evolution will deselect them. Yeah. Yeah, seriously.
When you think about S bombs, and I know we have AI bombs, but we haven't taken S bombs seriously at all. And I don't know if this is something that will change, whatever we're going to call the SBOM, the AI bomb or whatever. But people should really...
It's something that I always continually come across. We have the ability to do better. We choose not to, whether it's because of time, whether it's because we're lazy, whether...
I don't know why. It's something that I would love help in understanding humanity. When we have the ability to actually do things right, and is it just money?
Is this is what it comes down to? Are we... I have the same question about hurricanes.
Who gets killed in a hurricane? It's not like you didn't know this thing was coming for the better part of a week, but you decided to deliberately stay there and not get out of the way. So- Right.
Yeah. What is wrong with us? And so, yeah.
Golly. You're absolutely right. Go ahead, Sid.
No, I was going to say, attackers are no longer focused on breaching individual enterprise networks, right? They're going after the supply chain or software- Supply chain ... software.
Yeah Compromising trusted upstream software components that are downloaded millions of times across multiple organizations. So instead of doing one attack at a time, they're taking a bulk attack approach. So that's very interesting and different, I think, attacking the software supply chain.
And the other thing, I think AI has fundamentally changed cyber operations. AI models themselves have become high-value targets through prompt injection, model theft, data poisoning, and all of that. So I think there is an AI component to this conversation as well.
I know you said, Mike, that there's not exactly an AI story here, but I think that's big as well. Yeah. Right?
No, AI models are essentially part of the software supply chain, and I think it was- Right ... a report I saw from Barracuda that said something to the effect that a good 20% of the attacks that they're seeing are now aimed at those AI models. So, it's clearly- Yeah.
CrowdStrike is reporting, I think, and the report says that the link that you sent, CrowdStrike is reporting an 89% increase in AI-enabled adversary activities. That's a pretty large number. That's huge.
Yeah. Yeah. AI's all over this, but I'll just stick with my evolutionary epoch approach to this.
In my era, there were firewalls or not firewalls. There's SIM or not SIM. There was threat intelligence or not threat intelligence.
And before the SIM era, saying, "Oh, we don't monitor security on our network at all," doesn't make you any higher risk because nobody did. Well, once you start doing that, and you have monitoring on your network, you stand some chance of actually seeing something. There was no detect, right?
Yeah. In the first drafts of the- Absolutely ... what is it?
The cybersecurity framework, this cybersecurity framework, detect in the middle wasn't even there, because when that was being drafted, detection wasn't a thing. Yep. Right?
Absolutely. That was it. No one built.
And we're at this point now, I think, in supply chain, to your point, Sid, that if you just want to go on record and say, "We do not have an ability to understand our software supply chain," then do that- Right ... and stay there, because pretty soon, because it is possible today to have a good idea of your software supply chain, to be able to say, "Here's what we know. Here's what we don't know.
" You can enunciate that today. And I think that is a defensible position in court. It might be a defensible position with your board to get- Mm-hmm ...
to get funding budgeting, because I think in a couple of years, that's just you may as well not have an insurance policy because they're not going to pay. Right. Because you're not watching it.
That raises an interesting question. So if software supply chains are being used to bulk attack multiple organization in one go, what prevents North Korean threat actors- Mm-hmm ... to go after multiple agents and agentic entities that are developing these code bases, software.
So they can attack all these agents at one time. Yeah. Right?
So I was wondering- I- ... how that's going to play out. Maybe you guys can add some color to that, but it's an interesting question, I think.
Yeah. Being part of the chairperson for the CD Foundation, this is right up my alley, and I've put forth a project because I also think it has something to do with software bloating and reachability. Is there going to be a time where we finally actually deal with software bloating, with reachability?
This is something that we can handle. There's so many things that we can't handle, but this is something we can handle. So this is the software supply chain.
These attacks have demonstrated serious, yes, reachability, but also reachability as far as being able to get to many different companies. Think about Log4j. That hit a lot of different companies.
" Because identity and access management, as any of you know, or if anybody's listening on this today who are from the identity and access management control mitigation, what you will see is that those take a year plus. But something like this, this would be so fast. This is something where you could see measurable difference, and we choose not to.
And then- But I also think there's asymmetric-- Sorry to interrupt you. But I also think there's asymmetric warfare going on here, right? Mm-hmm.
Look at the traditional security tooling, which is designed around humans, by humans, operating at human speed, and suddenly we are caught in this middle of asymmetric warfare. All of these attacks are being done with AI tools and AI agents and AI attackers, and the tools are just not there to respond to them, right? It's like you have Patriot missiles, but if you do a drone attack, you don't know how to respond to a drone attack, right?
I think we're getting caught in the middle of that analogy. Yeah. Yeah.
No, asymmetric is-- If we don't start to understand that as well, asymmetric, and really start to change the way that we're doing business, we're really... I don't know how many times we need to sound the alarm, but Mike, take note. " This is the warning, this is the siren.
This is my next question, and I want to throw it to Chris because he kind of touched on it lightly. But will people get fired over this? And that's kind of the thing that maybe wakes people up, because at the end of the day, there's plenty of warning signals that says that this is a thing, it's coming, and if you're going to ignore it and there's tools to go deal with it, but you didn't prevent it , is somebody going to wake up one morning and the board's going to be looking for somebody's head because the shareholders will be looking for somebody to blame?
Or is it going to be the other way, where everybody's going to go, "Well, shucks, man. You were just unlucky, but we all sucked at this, so sorry, bro. " Yes.
The answer to your first question and the hard part of this is that people will get fired for spending money on these sort of things when it turns out not to be justified or too early, and not spending money on these things when it turns out to have been justified and not too late. And it's really, really hard to know exactly when it is, because- Yeah ... the reality of budget cycles and competitive...
If you are the CISO who says, "We need to make an emergency spend in this area, therefore do less marketing," and you lose market share next year and your company go to that business, and being right doesn't solve it. So I want to be really clear that there's no sim-- It's kind of my whole point in everything in the internet these days. No one can sit in your seat, and there is no blanket answer.
I think blanket answers are the problem. I'll go back to, I know we're at the end of this segment, so Sid, you mentioned North Korea, huge asymmetrical attack. The answer, if there's one code base out there, if everybody's doing a patch Tuesday, and my entire enterprise and all enterprises like me are all using the exact same code, then yes.
We are in deep doo-doo. I don't think that's the way systems should work in the future. Again, cost and that's the way we deployed in the past, maybe that's all we can afford, but it's silly and it's fragile.
And you give me a monolithic deployed code base and a supply chain, and yeah, hold my coffee. Anyone could have broken it before, and they can certainly do it now. There you go.
So, Kate, you and I talked about this when we were in Minneapolis, and I think maybe a lot of people don't fully appreciate the extent to which nefarious actors are not disclosing the fact that they discovered a vulnerability. And so there's a lot of these vulnerabilities that are probably out there and being exploited that nobody knows about until maybe far too late. So, on a certain level, is this a good thing because maybe we're shining a light on a darker corner of this thing, where the vulnerabilities that are known that are being discovered by people who aren't disclosing them and they're weaponizing them, and maybe we have a chance to just-- It's like walking into that room and throwing on the light, and all the cockroaches start running, right?
You've been in Florida too. You were in Florida too long. Yeah.
Yes, I agree with that. And we often talk about the three Rs, and now I think we're at the three T stage, where it's time, targets, and trust. Mm-hmm.
And I think that, so bottom line, yes, and we just need to take this seriously. And I really do think that go for the low-hanging fruit, go for the easiest. A lot of the vulnerabilities are simple that are taken advantage of.
And if you could do something, go for that. Sid, let me ask you this just to Chris's point. Where would I get the budget to go deal with all this stuff?
Because most companies, their budgets are allocated and we're on a certain cycle, and maybe there's an emergency fund somewhere, but that typically doesn't exist. And if it does exist, it's probably not very deep. So, will people just wait till, I don't know, 2027 to go deal with this because they simply have an allocated budget for it in 2026?
I think the era of steady-state budgeting is over, right? There was a time when you'd budget things for pilots, experimentation, and then move it to the production mode, and then expect it to run forever without any further changes to that operational model. But clearly, this is an example of where once you move things into production, you're going to have to go back to experimentations, and trials, and investigations, and exploits, and all of that.
And you need a whole different pot of money for that, right? So I almost think enterprises have to think differently in terms of allocating budgets for their IT programs and IT operations, where it's just not enough to allocate upfront budget for sandboxing, and pilots, and- Yeah ... DevOps, and those kinds of things.
And once it moves into production, you don't have to ever go back to that for that particular software in this case. And I think that's going to have to change, right? All right.
So zero sum this for me, and maybe I'll toss this back- That also, sorry to make one more point. That is also going to have to factor into the tokenomics model that CFOs are going to demand sooner or later, because you can't hide that anymore, right? Right.
So suddenly, your budget's going to explode or at least balloon. So then one is to wonder, are you really going to get the benefits of deploying AI? I think you are, and you have to absorb that, but planning for that becomes important, right?
Right. This is the core point. So Chris, walk me through this a little bit.
So theoretically, I have AI reduce the cost of building software and, I may or may not need fewer developers, but at the end of the day, I'm creating more software than ever on the base of a set number of engineers and developers. And yet, I'm going to have to spend more money to secure all this stuff. So is the savings that I'm going to have from my AI coding just going to get dropped into application security and it's an even sum game if I'm lucky?
Well, again, as prognosticators, I could forecast the average curve will go this way or that way, but for an individual organization, it depends massively. But the primitives that are going to be true for everyone, it's not that you can create more code. You will exponentially make more code faster than your systems could even dreamed of.
And what do you do? So you can not do that, and maybe that works. Maybe that is the competitive advantage is your competitors will spend all their time, and to your point, not save a dime, and you just plod along and you're fine.
I think that'll actually be the case for some enterprises. Too early adoption could be its own risk, but where the competitive advantage is is to lean in. The old ways of doing this, like the volume issue just comes up.
Volume and time. And I can produce more code, I can write more code in human language documents than I can read. That's not even a logical sentence, but it's true, right?
And you can write more code, but Mitch and I had a great exchange on this months ago on this show, right? The human in the loop. We keep thinking, well, the human in the loop, they're going to be right here.
It's like, no, no, no, they're not. They need to be so far away from where we think the human's going to be- Mm-hmm ... that it's a structural architectural change.
And if they can't get there, then maybe none of this AI stuff works. If they can get there, those who put those systems in place will have such an economic advantage they'll wipe out entire segments who don't. All right.
So folks, I'm hoping that everybody's making more revenue to cover the cost of these things, but there will be cost and there will be allocation, and there will be shift. I would love to tell you that this is your last warning on this subject, but it's probably not. But it's getting damn close, so wake up.
All right. I'm going to shift the gear here for a second and we're going to talk a little bit about AI agents. And there's a new consortium out there that has created something that looks like an AI proving ground, where you can go in and test whether or not your autonomous AI agent is safe.
At least that's the theory. And, this comes in the wake of some recent reports about how various AI agents were turned loose and broke into various websites that were out there, and OpenAI was testing stuff, and Anthropic was testing stuff, and it sure woke everybody up for this conversation. I'm not sure that anybody changed their behavior as a result of all that, but everybody at least heard about it.
But Chris, when you look at this, is this a viable approach? Because a proving ground is a metaphor from the military, right? We used to have proving grounds for artillery.
I seem to remember there was one in Maryland somewhere that people used to go to. But, is this a way to think about how we're going to go do this? You have to actually show that your AI agent is at least somewhat safe?
Evidence. I like evidence. There's all sorts of great ideas, but nothing proves it until you build it and slam it into a wall, right?
And it works or it doesn't, right? So I have my reservations. Is this French says dummies for AI agents?
I like that. Oh, yeah. Oh, yeah.
Literally. Yes. I love it.
How you do this, right? I see the headline and the title, and I haven't, to be clear, for the people involved, if you're watching the show, and you should every day at noon, so I haven't looked into the details of how they're proposing to do this, but I'm in favor of it. I think if it's a reflection of the industry, they're missing some layers of evidentiary capture that I might be really passionate about.
But no, yes. Put your stuff together, show the evidence, see how it works, make it up for debate. If it flies, it flies.
If it slams into the wall, we all learn a lesson. So, sure. All right.
Well, Kate, let's take this one step further. Should I have ratings and labels for my AI agents based on how well they did in a proving ground? What do you think?
I think that would be really cool, almost like a crowdsourcing type of thing. Okay. " We think that this is new, but this is actually old.
If you guys remember, this is what we used to do. Working with IBM, there was really old mainframes. We didn't know, we would just shut them down.
" So it's almost sort of the same idea, we're just sort of throwing stuff out there and hoping for the best, and I actually think it's a good thing. Do you remember they used to do that even when they first brought out maps? And yeah, there were some accidents, but it's along the same line, I think.
Yeah, I think there should be ratings at the end of the day, and I don't think that this is new, and I think the only thing with ratings is we actually have to know that they're humans that are rating it and not agents themselves patting themselves on the back and saying, "I did a great job today, Agent 86. " But yeah, I think ratings would be a good idea. You're always coming up with good ideas, Mike.
Well, there we go. Cindy, I mentioned this Barracuda report earlier, and one of the things that was in there as well, it's up on Security Boulevard, is they were detailing how an AI agent could be used to launch a business email compromise, and you get into somebody's email, and before you know it, you're into the CEO's email and so on and so forth. But one of the things that was interesting in there, at the end of it, it said, and the AI agent, if programmed, or maybe even if it does it on its own accord, will go back in and wipe any evidence that it was ever there or anything ever happened in the first place.
So where will we get the trail from to actually figure out what an AI agent did? Because the AI agent may just go back and wipe all the evidence. I think that's why we need validation, like the idea of validation, right?
You really can't have agents just simply summarize alerts and recommend actions anymore. These agents should be responsible and should be certified to do things like isolate endpoints, revoke credentials into actionable things. But they also need to be validated on realistic, advanced sort of conditions.
And I think that's the key, because without a validation mechanism, you're going to have rogue agents, runaway agents, and do all kinds of crazy things that this Barracuda report alludes to. But I think there's one other aspect that is related that I would like to talk about, which is essentially, I think one lesson we learned from this whole sort of topic on this block is that the security incidents that we're witnessing in the context of this conversation is not about the AI model anymore, right? The real security challenge is about what I call the control plane.
The model is not the security boundary anymore, it is the control plane that is a boundary, right? And the control plane is what governs things like identity authorization, tool access, memory management, network connectivity, all of that, right? So unless that control plane is absolutely secure, and the control plane is not running by itself, because we know control plane is not going to be operated by humans, it's going to be operated by agents, and those agents, going back to my earlier points, point has to be those validated agents.
So it's all a web of complexity, if you will, right? Well, and let me riff on that if you don't mind, because control plane, I agree 1,000%. When I say control plane, I don't mean dashboard.
No, I know. Control plane means your whole architecture you're using to control things, right? And back to your question, Mike, it's a perfect example.
So if I set an AI system out to do a bunch of things and to keep records someplace where the AI agent itself has the authority to erase those records, then by technical industry terms, I'm an idiot, right? That's bad. It was about a month or two ago, there was that story we covered.
There was this company, their AI wiped out their production system and their backups. And I can't help saying it, those were not backups. Those were not backups.
You can't delete backups live from an operating system- Until it's done ... or they're not backups. Yeah.
If I can RM space minus RF in the wrong place and wipe out my disaster recovery, then I am the disaster. So, if you design a bad control plane, then you can do all sorts of stupid things today. And with AI, you can do them at speed.
It's awesome. Yeah. So Kate, here's the thing that leaves me scratching my head about all this, and we're all talking about guardrails for the AI agents, and yet it seems to me the AI agents are programmed to aggressively complete a task by any and all means necessary.
So if I tell it not to do something, well, it'll just go find some other way to go do that thing, and so here we are. And so does that mean, I guess, I don't know, can we reduce the level of aggressiveness of the AI agent, or is that a forlorn hope? Or we go back, we need a control plane, or do we just need better controls on all the data sources, and these are the things that we've been ignoring all these years anyway, or check the next box, which is all the above?
Yeah. Well, check the next box to all the above. But wouldn't it be nice, if it's true that agents are very much governed by words, would there be a way to control them?
And is that just like a pipe dream? " And I can't tell you that I have, and I think this comes along Chris' line of work when we talk about canons and everything else, but I do think that we could govern agents. Whenever we read about incidences, they always seem to be because they have found some sort of loophole.
But I wonder if we could just do a test on this idea. Like, don't go outside of this parameter, regardless of any other eye candy out there. Don't go outside.
And I wonder if we could test it. I don't know, Chris. If we put to Mike's first idea, would there be a way to control it, like really control it?
Or would it continue to find an outside path? I think there's a really binary answer, and the answer's got to be yes. Because if the answer is no, to be clear, we can't have AI.
Maybe we can't have the internet. Mm. Because cats and dogs will rain from the skies, and the whole thing will burn down.
And the HuggingFace incident, I find it fascinating. I find it instructive that the good people out there with the big budgets in these big companies, this is how they do things, right? Without setting up a test like this.
Again, it was just simple human errors, nothing rocket science about it. But nobody thought to put in a system that said, "If this AI system actually starts accessing the internet, flag me. " Yeah.
I don't know. Yeah. Let's find out afterwards, because the entire environment that it was set up was, and nobody gave any thought to it at all, right?
Mm. There was no visibility. I mentioned SIM and that sort of thing.
Look, the SIM market is interesting. Network awareness and monitoring is this whole bag of cats. But to be clear, you're OpenAI.
You've got the budget, you've got the time, you're doing something high consequence, and the state of the industry is that putting any sort of monitoring around it just didn't even make it across the planning level. So yes, we can do this and sort of do it all the time. It's not rocket science.
All right. " And they go poking around with the thing that I told them not to do in the first place. I think AI agents are very similar.
So Yeah. Except they're less polite, yeah. Right.
All right. So, final topic. The EU is getting ready to finally put some regulatory stuff and oversight in on content generated with AI, and they have rules that are, I guess, nominally went into effect, but I think you have a couple of months yet to comply.
But the idea here is that anything that is synthetic-created is going to get a label on it so people will know that it is AI-generated. It sounds like a reasonable approach. I don't know.
Maybe. But Sid, as you look at all this stuff, the other side of this thing that comes to mind, too, it's like, well, at what point is there more AI-generated content than human-generated content? So maybe we should just label the stuff that's human-generated because it'll be easier.
I don't know. Well welcome to AI. I think the big story here is AI is not software or models anymore.
It's become critical infrastructure, right? That's the first big story. And the second big story is the AI industry has entered its regulatory era.
Now, I know that's been sort of toyed around for a while, but I think this is serious, especially when the European Union clamps down on it. And they have a history of doing this, for good or for worse. I'm encouraged they're doing it because I think it's the right thing to do, and they've done some good work with some of the EU regulation in the past, like GDPR and things of that nature.
So that's the good part. I am concerned, though, in how it actually manifests itself. For example, if you look at the history of European Union trying to control or regulate the cloud market, the thing, the Gaia-X and all of that was a disaster, right?
It was like, "We don't trust the North American cloud providers, and we want to do Gaia-X," but nothing really materialized. Hey, if you really don't trust the North America cloud providers, go build a cloud of your own. But they couldn't, right?
So I think you've got a little bit of yin and the yang going on here. But I think the other big story is for the first time in a model builders like OpenAI, Anthropic, and Google, even Google, are operating under a regulatory framework, will have to operate, I should say, that goes beyond voluntary compliance, right? Because obviously voluntary is not working.
In many instances, at least the EU believes that it is not working. And then I think the EU AI Act, which is referenced in the article that you sent, it moves governance from principles to actual enforceable obligations, right? So I think you're going to see that.
And then I also think it has an interesting twist, which is now if you're compliant and you're kind of going along with the governance mandates, then you may have a premium label. We talked about labeling earlier. You may have a premium label next to your model where you can charge more money saying, "I'm EU compliant," right?
So you may see some tiering of pricing based on that kind of labeling and USDA certified kind of thing, right? So who knows? That's also possible.
And then finally, I think compliance is not really a legal function anymore. It requires significant investment in infrastructure for monitoring, evaluation, logging, all the orchestration stuff that we talked about, the control plane stuff. So I think all of that is really relevant in the story.
I don't know. Chris, what's your take on it? And the other part of this too that I'd love to get your opinion on is, I'm not quite sure.
Let's say I am a human and I create some piece of content, but it references some other content that was created by somebody else who used AI. Am I obligated now to slap an AI label on that? Or how many dependencies and how much nuance is there going to be here?
This is one of these things where it's really hard to be kind, right? Because all the people behind it, I'm sure, mean well, but it's a terrible idea. Right?
Mm-hmm. It reminds me of parents of a certain age remember the Wikipedia wars, right? Wikipedia got really popular, and my kids were just the right age.
The entire education system made absolutely sure that no one ever opened Wikipedia. And to be clear, to all those parents, teachers, and principals out there, I was in the back scene saying, "No, no, go to Wikipedia first, obviously, because that'll save you 80% of the time on this project. " Mm-hmm.
"So let me show you how research works," right? So saying, you know, hey, AI, you know, you said, and Mike, you both touched on it, everything has AI content. Everything.
Yeah. So why don't we just say everything has AI content and put the label on everything? Because what does that even mean anymore?
And by the time we have the 18-month public conversation on those labels, that definition will have moved on anyways. Yeah. Mm-hmm.
So I think it's more usefully saying, if I want to go out in public and say, "I have no idea the origins of anything that my systems is, and I generate things completely in the blind because I'm an idiot," that's kind of where everybody is right now. Right? Because mostly we don't have these systems in place so that you can say, and again, AI, there's a large language model that does this in this context at this point in this process.
Can you say that or can you not say that? If you can't say that, you probably also can't say where in the CI/CD pipeline this library was inserted and various other things. So, labels from the top down on things that end up being almost impossible to define.
Good effort? Is that what you say on the golf green when somebody does really poorly, right? Golf applause.
Yeah. But now flip it upside down. Do you want to have systems where you don't know- Mm-hmm ...
what created what, and are you okay with that? Yeah. I am reminded with a laugh, since I was talking about my children earlier, but back when we had ratings and labels on movies, they would take them to the store, and they were looking for all the stuff that was rated R.
" But there's one other, before you move on, Mike, I think there's one other aspect to this conversation, which is doing all of this is going to require additional resources, right? Yeah. It's going to require more compute.
It's going to require more storage, engineering resources, operational tooling. Who is going to absorb those costs? Other AI providers, other model builders, OpenAI and Anthropic are going to absorb this cost, or are they going to pass it on to the buyer?
Right? That's an interesting conversation. What does that do to the whole cost model and tokenomics and FinOps that we keep talking about repeatedly to the buyer in adopting AI?
Right? So it's not that straightforward. So I just want to- Mm-hmm ...
put it that way. All right. It will get better, though.
" What do you say? I say you probably just made a US policy as you just spoke. So thanks for that.
Thanks for putting us in that position and giving that idea out there. Yeah. I was just watching something recently about how our president has just really messed...
Oh, it was coming around, and it's along the same idea of helping these AI companies and where that is going. But along the same lines, I'm sure that if there is a way to put a tariff on this, because our tech bros who are in this field will be like, "Oh, no, we can't have this happen. " I'm sure it will be a tariff.
Yeah. I don't know. To Sid's point, though, I'm pretty sure we're past the days of where the AI companies are absorbing these costs, right?
They're just going to pass it along to the end customer, and it will be up to various publishers and everybody else to put these labels and manage that process. Sid, is that about right? I think so, but I think the more important point here is nobody's talking about it, right?
It's like the best-kept secret. And the question is, when does it manifest itself, right? Here we are, the vendors and the AI model builders building $500 million or $1 billion data centers.
Great. Capacity is being built up the wazoo. " Trying to figure out how this is going to work, and this is yet another wrench in the works that'll slow down their AI journey, right?
Because if these things don't become transparent and these little things get into the mix, which drives up the cost over time, and no one's tracking them, no one's being transparent about them. I almost feel like we need a presidential executive order to be transparent about AI costs that are creeping into the cost of running and operating AI, right So I think that's really the concern I have, right? Well, I will override presidential executive orders with Darwin.
Because as I look at the end of this episode today- Ah, yes ... the term that I can think of is show your work. That's it.
Yeah. Because the pros and cons, the risks and opportunities in the economic space, global economic space, the AI space right now are bigger than tariffs and so forth. If you make sure that you're recording, that you have the ability to be transparent, at least to yourselves, your shareholders, your board, and so forth, your internal operations, I think you'll find economic advantages to that by itself.
And then if there's some spasm of international trade because some nation state does a thing, that is likely to be a rounding error compared to the risks and opportunities you have of just being out of control. But you can keep the records so that you can show your work to yourself, to your boss, to the regulators, to your customers, and I think that's a better bet than trying to second guess international standards bodies or nation states or anything else. All right, folks.
You heard it here. We got to end the show. But to Sid's point, hey, if there's something out there that we should be talking about that nobody's talking about it, you're going to hear about it on the Techstrong gang.
I want to thank our guests for sharing their knowledge and insights. As always, I want to thank you all for spending some time and watching the latest episode. Please stay tuned for the rerun and the rest of the Techstrong TV lineup, and we'll be back tomorrow with more news, probably from Black Hat USA and elsewhere because, well, the hits just keep coming.
See you next time.