Apple’s AI Bug Report Overload and Red Hat’s Governance Play

Hello, everybody, and welcome to the Tekstrong gang for Thursday. And wow, it just keeps getting more and more interesting out there, especially in the land of AI. And of course, there's been a lot of things going on around Black Hat USA this week.

But let me welcome our guest today, Stacy Thayer. How are you doing? I'm well.

How are you? Good. Good to see you, as always.

Yeah, you too. Wiki Wang, it's been a week, I think, since I saw you last. How are you?

Good. How are you? Good.

And Chris Blask, I think it's been 48 hours since I saw you last, but... It's Thursday. I don't know what I'm doing here, but it's good to see you.

It's Thursday, so there you go. Well, it could be just the subjects at hand. So let's just dive in, but I'm kind of excited about what's going on because it looks like the adults are finally coming into the AI agent room, and we're talking about things like governance.

Red Hat has launched a project called the, let me see if I get this right, the AI Safety and Governance Orchestration project, otherwise known as ASGO. And the idea here is that there will be an orchestration framework that is smart enough to apply the controls we need for AI agents in near real time, and it will monitor the behavior of the AI agents. It'll understand what it's doing, and it'll be able to generate the code that's required to apply that, and it's going to be this kind of soup to nuts thing.

Now, it's still early days, but let's start with Wiki. What's your general thoughts here on this? Is this where we're headed?

Is this how this is going to evolve? Or is this kind of maybe a false step, but maybe pat them on the back, but it's not quite what we need? Yeah.

My take is the Red Hat is introducing now the other unique AI governance framework, right? It showing the governance has to move from documentation to execution. I also see the similar trends happen in the ThreadMag new version.

It requires to automate certain things and see the result. In the past, historically, for years, we've had AI policies, but the policy normally sits on the PDF and have the annual training for the company employees. It doesn't actually control how the AI system behaves, right?

That's why I really like the accompanying piece. The ACK is not the canon, right? Just because a system acknowledges something, doesn't mean it has validated it or should trust it.

I think that's exactly where the enterprise AI governance is heading, from the written policies to enforce them at runtime. Yeah. So far, I do see there are some company work on that as well.

And there's trends in the company internally. People try to hire more GRC engineers, try to automate the process as well. So this framework definitely represent that trend as well.

So I'm just curious, right, Stacy, walking around like Black Hat, you actually see companies deploying this today, or we still mostly talking about the version type? I think it's all hard to say right now, walking around the trade show floor, trying to discern between what companies are saying they're doing and then actually doing. This time of year is when they're launching their big AI hype.

This is the new thing that we can do, and then everybody's trying to make sense and go, "Is this the new thing that we can do? " So there's always that kind of new technology buzz and new piece. One of the things that excites me about this is just the fact that the topic of AI governance and open source and accountability in general with AI is there, because that's something we don't hear a whole lot is, "Here's this new technology.

" But what does this mean, and how are we using it? Just the fact- Yeah ... that we can use it.

Yeah, I think we're a long way from actually getting any of this done yet, but at least I feel like we're talking about it, so that's a step in the right direction. Yeah. Wiki mentioned an article that Chris wrote, which is essentially about trust as an architecture or something you build into your architecture, and it's up on Security Boulevard, and I invite you all to go find that.

But Chris, is this kind of maybe this project with Red Hat kind of a step in the right direction for trust as an architecture? And I think maybe we should have done this a long time ago, but here we are, and are we about to make some significant leap forward, or what's your sense of where are we? It's definitely a step in the right direction.

So, walk around the whole thing, right? So we have the whole open source world. Red Hat is a huge player in that going in this direction.

To me, it looks intrinsically right, and I've looked a little bit at the framework and how they're looking at things, and yeah, it's not all the way there, but it's going in the right direction. And to your latter question, as I keep saying on the Tuesday show where you and I and Kate and folks have these conversations every week, we have to. We have to get there.

And as you say, Stacy, right now we have Hacker Summer Camp is going on, where at this point in the vendor and the community cycle are talking about it. And everyone is trying to figure this out. " So just to sort of repeat the things I say a lot on Tuesdays, the Standards Council of Canada, ISO 42,000 Systems of AI Systems Working Group I'm fascinated with that one.

I'm involved with that one. And watching as communities like this, this calendar year, this last six months, right, in supply chain and in standards, in different working groups, different organizations, different jurisdictions, national and international, we're all working through the same thing. And we have to get all the benefits that we're investing more money than ever in history, this whole AI infrastructure and all that's going on.

To get the benefits out of that, we have to keep moving, and we have to come up with AI governance frameworks that make bloody sense. And to my point in that piece, ACT is not canon. I look at the 10,000 or so RFCs we've created in the last 50, 60 years of the internet, and they're mostly about, can I get this packet from here to there?

I think this world we're getting into, the point I'm trying to make in that piece is we can make systems now that can see the document is there, can acknowledge its existence. That doesn't mean the system, the computer, the company is going to act on it. We haven't built those levels into our protocol stack.

But I think we're on the path to do it. I think Red Hat going down this path and the open source community leaning into it. The CVs overwhelming the Block B coming up.

Bugs being overwhelmed. They're forcing us to level up our systems. And yes, we should've done it earlier, but we didn't have the time.

Yeah. So Wiki, let me ask you this. One of the things that I like about what Red Hat's talking about here is that the approach is declarative.

So there's code involved, which makes it possible for me to kind of implement that without having a bunch of manual intervention, but it seemed like they're actually taking some care so I don't have to be a software developer or a rocket scientist to go use this thing, and it might be accessible by mere mortals. So can we get to the point now where you think that we can apply compliance, the controls in real time at machine speed, and this is kind of where we're headed? Because I think the whole idea that we're going to apply some governance framework that we monitor with humans is never going to work.

Well, I think it's a good idea to automate some portion, right? But overall, from my observation on the AI real case and the previous case, I can always say there are some corner case human need to pay more attention. And, I would say it's good for machine speed to run one round, and then there's a human in the loop to validate.

Yeah, because compliance or even audio side or security portion is not something you trust the machine 100%, because machine is kind of like... Especially for AI or current automation, sometimes they just give you the probability, right? They don't confirm 100% this is actually work.

So I would still say we need to have the human validate it. But it's good to see the real time continuity monitoring there to make additional comfort. So you don't always say it's only on the policy, but no one try to take action on it.

All right. Stacy, come along for a little journey with me, if you would. One of the things- Yeah.

I think I'm going to make sure I can hear you, but you're on mute I think. Can you hear me okay? Yeah.

That's all better. Okay. So what's good for the goose is good for the gander, and it's wonderful that we're going to have all these capabilities for organizations to comply, and in theory that means they won't be fined as much for being out of compliance.

However, on the other side of that are these people called auditors, no? And will not the auditors kind of use the same technology to kind of monitor everybody and see what's going on, and the minute that you step out of compliance, they'll know, and they'll just send you an automatic fine? Right.

Big Brother territory, right? You hear Minority Report, whatever it is. Yeah, and I think one of the things, it's two ways in some way.

" But at the same time, it's not a two-way street. We want to know some of that transparency, and rightfully so. We don't want some of that transparency turned on us, where all our data is being shared and we're being monitored for our own use, and having that double-edged sword.

And I think putting those two statements together, so Wiki, the human in the loop, I love that one, right? So just yesterday, a LinkedIn article, I was picking on OpenAI. And our producer, Taylor, is back there using Codex.

And Codex, it seems to me, one of the OpenAI responses to the Anthropic accidentally hacking half the world is now when you're using Codex, and you normally had to do /P, down, down, enter to authorize fully agentic Codex to do whatever it likes. Now they've added, now it's /P, down, down, enter, enter. " Right?

And that's a human in the loop, but it's not, right? My right hand just can do this without thinking about it. Slightly, da, da, da, da.

And how do we get the human in the loop? And I think as we start from the human out, from the attentions part, Wiki, right? Humans only have so much time and attention.

If we think that the human in the loop is going to be a rat pressing an approval bar, systems will break. If we actually look at how much time, how much attention humans actually have, and try to build the systems out from there, I think we'll find certain assumptions don't work at all. The scale doesn't match, and I think there are ways to address this, but that's a longer topic.

But Stacy, on the governance visibility, and Mike, to your question about compliance, I think if you start from the node, from the computer, from the person, the company, build it in. I want to know that they're compliant all the bloody time. Auditors be damned.

The auditors can come along afterwards. I want my systems to tell me the moment I'm at risk. And that's doable today.

Yeah. I think there's one thing I feel like very interesting I should point out. I start from Big Four, so kind of have some auditing background.

Always if you try to use a automation system, there's some sort of validation. They need to validate the automation system need to work very well. That's a huge amount of work, is when you try to bring in some new system.

So sometimes there's a trade-off, try to understand, oh, actually, do I need to do something manually, or do I need to do it automatically with lot of validation to confirm the system works very well? But from compliance perspective, when I'm in the compliance team, I also need to validate the operation works well. So I need to monitor this whole system, make sure it works all the time, generate the same information I expect.

That monitor is kind of very important. That's a portion I think people need to be in the loop. And then, at the end, the result need to do some spot check to make sure that monitor works very well.

Mm-hmm. So let me follow up with Wiki on that, though. What is your sense of-- there's this whole cadre of people, and I think we refer to them sometimes as GRC, governance, risk management, and compliance.

And there's specialists in that community. How automated are things today, and are they ready for this level of automation? Where are we on this kind of cultural mindset within those teams?

I think it depends on the company stage. For very early stage company, because the process and technology, even the person, organization, they're pretty simple, the automation percentage is very high. But once you become a certain stage within some additional regulatory requirements, it needs more human involvement.

Most of the companies, they try to make more real-time monitoring because that can add additional layer of trust, and you can fix things on time to avoid the risk. But there are certain step, it's very difficult to be automated. Mm-hmm.

It's not because the technology thing. It's companies' digital trust or digital transformation is not at that level. Things are very fragment and it's very hard to connect different systems and make things automate in the same way.

I should say it like that. And also during the monitoring process, for example, if the interface has some problem from the security operation perspective, if you don't monitor that interface, didn't see that abnormal behavior, it could be amplifying the wrong information. That's a scary part.

I think humans still need to be in the loop. Yeah. Mm-hmm.

Chris, the part about this that I'm a little concerned about, and I read your article, but every time I look at one of these AI agents, the one things that come across is that they are aggressive. And to the point where they will go and try to accomplish a mission. " And so basically, they will find or look for anything and anything they can use to accomplish their mission.

So I may tell it, "You're not allowed to use X, Y, and Z," but unless I am completely cognizant of every possible way of doing something, it'll find some other way of doing the thing that I didn't want it to do or that I'll be out of compliance for. How do you program for that? How do you layer something in that says, "We're going to head that off at the pass one way or another, no matter what it wants to do"?

So yeah, and you and I have this conversation a lot. That I'm trying to both think through this myself because, again, there may not be an answer. The universe doesn't guarantee us answers, but I think that our need to address these forces has down this path, and frankly, I think I know what a set of answers are.

And it comes back to the same things again. Build it from the ground up, and combine some of the major portions in my life. The sovereign AI, sovereign data.

You should have all your own data. You shouldn't have a data swamp like Kate and I have great conversation about that on the Tuesday shows. It's a mess.

We shouldn't have our data spread across a million platforms that go out of business and do terrible things with it and give it to governments that are authority, yada yada yada. And we should have serious maturity in systems like we have not in IT, except in very few individual cases in the world. Really robust, mature IT deployments that can be talked about as infrastructure.

But we do have in OT, we have really good examples of what it takes to have solid, resilient structures in critical infrastructure. And it's having systems that, again, could be just a bunch of humans doing the same thing, but you just don't have one person who can blow up the dam, right? You have checks and balances in place so when agents are doing things...

But your question forced me back to one of my favorite frames of this, that we just built all these structures with people in them, and we're trying to replace people with AIs. So soldiers, take this as a classic example. You think that in all this human effort to train people to go out on battlefields and shoot at each other, we'd be good at training humans to shoot at each other.

But military history shows us that's not true. People go out on the field and shoot near the other person, trying not to hit them, because the human brain is kind of funny that way, right? But generals and military operators at that level understand this, so they build strategies that have that built in.

You replace that with an AI, replace that person with an AI, they'll do exactly what you told them to do. But our systems are not built for actors who will actually do what we told them to do. They're built for people.

So we put AI in that spot, and we say, "We want you to do this," and boy howdy, they will do just exactly that. That's just it. All right, well, folks, I'm going to leave this here because the good news is governance is coming to AI.

The bad news is governance is coming to AI. We'll see how this all plays out. All right.

Shifting a gear, though. So Apple is talking about how that they are struggling with all the vulnerability reports that are now being shared with them, and I suspect that they are not alone in that. I'm pretty sure that just about every other company has the same problem.

" Chris, is this the right thing to do, to cap the submissions, or is there another way to think about this? Because it just seems like people are going to discover the vulnerabilities and share them, so if you're not going to acknowledge them, they don't go away. Well, I think the short answer is yes, it's the right thing for Apple to do.

They had to do something, and it shouldn't be surprising. Again, what I said at the end of the last block, right? So we put these systems together, we said, "Please find vulnerabilities.

Please let us know about vulnerabilities. " And people have engaged the way people have, which has been great. So whether it's the CVE system writ large, or in this case Apple, how individual large vendors are dealing with it.

Turns out, when we get what we ask for, so people out there are now taking AI tools and saying, "Oh, you asked us to send you vulnerabilities, so guess what? We're taking you at your word. " And if it, again, forces us to come up with different structures for dealing with things like vulnerabilities, and it does, that's an evolutionary pressure we'll succeed with or not.

But yeah. Look, this whole industry, we've all, four of us and everybody, most of you watching, have made our careers on, we've done good work. Don't take this the wrong way.

But the vulnerabilities were always there. The vulnerabilities we have found, the handfuls that we've handled and managed, that's been good work. But bad guys have had lots of vulnerabilities to exploit all along.

They're just not announcing them. Now we get to see how many there are, and that makes us think about CI/CD pipelines and the supply chain security and a lot of stuff. Mm-hmm.

Well, Stacey, let me ask you this, though. Is this a good look for Apple, or for that matter, any other vendor that says this? Because to the end customer, it reads like, "Let me get this straight.

" Yeah, no, definitely. I think that at face value, at looking at it, it's like, wait, what? To Chris's point, it's like not counting all the COVID cases.

If we just put it over there and we don't look at it, it doesn't count. And so, but then I kind of flip it. I think, well, I wonder if there's more there, and because when I think of people using AI, that opens it up to a lot of different people finding a lot of different vulnerabilities, and what is that?

Are there different classifications of vulnerabilities? Are they looking at false positives? " When as we know, there's some shades of gray in there.

They're not all there. And are they doing this to try and maybe fast track some of the more severe ones, and that's, like is there more behind it? Is what I looked at that.

And maybe that's with a little bit of hope and optimism, of like, sure, no, they're going to look at the bad ones. They're not going to miss any. We'll be good.

So that might be some false optimism on my part. But that's what I think when I'm looking at this, is it doesn't make sense to me. There must be more behind it, because why would you just sweep reporting these vulnerabilities elsewhere and not look at them?

How are you tracking for what you're missing? So I had a lot more questions on that one to make it make sense for me. Okay.

Wiki, this goes back to the previous conversation we were just having, and I think you're on mute there, so you might want to hit that button. The auditors will be able to use AI to go look for... Well, I guess historically they would've looked for known vulnerabilities on some list that violated some sort of compliance mandate.

But as we go forward, do you think auditors at some point are just going to go ask the AI to go find any and all vulnerabilities in a particular IT environment or a piece of software, and whether they're known or unknown is irrelevant, it's just a matter of it's part of the audit? That's a good question. So far, I think this one may be more shifted to the security operation or compliance side.

I do see there are a lot of discussion regarding how we can prioritize and patch the vulnerability as quick as we can. Because currently, AI just explores so many, right? I guess that's the reason, same logic when Asteropic made their model a couple of months ago.

They have that glass something program, try to release later, but talk to different major vendors, ask them to patch first, because currently this is a speed industry hack. But at the other side, I do see some startup company, they try to help with the vulnerability and automated fix. They're still at the early stage, but some things start already.

Mm. I should say AI can help to fix some, but still the similar situation like we mentioned in the first portion. There are a lot of human validation still need to be there.

If you see recent case, like Chris mentioned for the OpenAI. People still need to be there to look at things so they know what happens and how to make it work. So yeah.

Yeah. Go ahead. Yeah.

Sorry. But again, put these things together, like Stacy, like you're saying, there's a quality issue. Because I can show you how a vulnerability is doesn't necessarily mean anything.

What are the... And again, back to that human loop thing, Wiki. So this has me thinking of two acronyms, like we need more, but VEX and SRAP.

The Vulnerability Exploitability Exchange is a protocol stack that developed along with the SBOM, Software Bill of Materials. " And that's, I think, a good example of, to your point, Stacy, not just quality of is that even a real vulnerability, but does it bloody matter to anyone on Earth? Is there a deployed instance somewhere where somebody cares?

And if we can't tell that in a sea of potentials... But SRAP, the Safety Relevance Assertion Profile. Devashree Datta, just in the last several months, came up with this idea.

I apparently co-authored it, but it's really her work, and that's being adopted as a sidecar to SBOMX, CycloneDX. It's a similar sort of thing, saying because I developing code can't say the real safety relevance of it. What is the relevance to actual safety in, by the definition of safety, humans, and risk, and that sort of thing, to a piece of code?

It depends where it is. And I think going down that path, we start to see the possibilities for where we can meet our needs without enunciating every single vulnerability and making Apple and every other company on Earth put 100,000 people into rewriting code. It all depends where it goes.

Yeah. I feel like traditionally we have that ranking system. We have the critical vulnerability, we have low risk one.

I think at the time when we do not have enough resource or try to catch up the speed, we should think about those priorities first. But again, security. There's always something.

And the bad guys are getting better at daisy-chaining together a bunch of low-level vulnerabilities and turning them into a critical vulnerability, so it's hard to say whether the ranking's going to work. The other thing that comes to mind, though, it reminds me of COVID in this sense. It's like at some point what was happening was the hospital systems were just overwhelmed with patients, and we didn't have enough healthcare facilities to deal with the issue.

Well, is that not the same thing here? We basically have all these vulnerabilities that are being reported, but we don't have the back-end systems to actually address that. So I guess we could put a cap on the number of, quote-unquote, reports/patients, but maybe, Stacy, we should just go back in and fix the back-end processes that we've been relying on to deal with all these issues, and use AI to do some more analysis of the vulnerability reports.

And to Chris's point, use AI to create the patch faster, and the whole thing becomes a better flywheel. What do you say? Oddly, yes.

" AI to fix AI to fix AI, and I think it just comes back to the fact that we are still trying to figure out how to use AI, what it means, what's the realities of the technology. And technology will move faster than most humans can keep up. And so I think what we're doing is when it comes to AI, we are so reactive right now, and that's just what I'm seeing.

And so if anything, what it says is, so there's the how is Apple doing this, but then there's looking at it and what we're seeing is the fact that Apple was unprepared for the volume of what they're getting. Just like I think most organizations right now when it comes to AI are, is that we're looking at this, we're putting it out, and then we're going, "Okay, this is what we got from it, and now what do we do with it? How do we make it make sense?

And how do we," I'm going to use the big word, "of control it? " So that's really what I'm looking at it there is like how could this have been avoided or managed better as a more proactive response? So to show me, what that shows me when someone's proactive is that they anticipated this happening Mm-hmm So what can we anticipate from AI in a way that we're not constantly catching up to it?

Right. So the thing that makes me smile about all this a little bit is if Steve Jobs was still around, the storyline would've been different. The storyline would've been, "Yeah, you know all that software that we sold you that we told you was the greatest thing ever?

Well, that sucks, and you should buy the next thing because here it is, and this is what it's going to be," and then there'd be one more thing at the end of it, and it would be turned into a giant marketing selling motion. So, I don't know. Chris, is that where we're headed?

Why don't we all just get together and say all our existing software kind of sucks, and we're going to build something better and stay tuned? There's a great "Twilight Zone" episode with Peter Falk, right? There's a revolutionary, it was a Banana Republic cliché, where you get the power, and then you end up being the dictator by the end of the episode, right?

So, we're not going to... The rip and replace isn't going to happen. It's not economic, doesn't make any sense.

We can't live without the systems, so we have to incrementally develop them out. And, this last decade of supply chain security for me, and in this con conversation right now, has me think about that because I've been making the point all along that we have to be able to say really, really quickly down to the firmware in a chip from a third, fourth party supplier, what's inside certain things. Not everything, certain things.

We have to be able to say that really fast, right? And with Alan Friedman and the CISA crowd, before all that shut down last year, we managed to demonstrate this across two corporate boundaries using AI systems, yes, getting the software build material for something in 400 milliseconds. Just by having the structure in place first, not by setting it up, say, get this file, but as organizations, we have these relationships already.

They tend to be in legal documents and contracts that we sign once and file away. Well, guess what? Those are now marked down files that are readable by our systems.

They should be able to execute on them continuously, not audit in six months. So, again, we have to fix what we have, right? I can tell you it's a lot easier building from scratch, because that's mostly what we do these days.

We'll ride along next to some installation, not trying to replace everything all at once. Yeah. But we have to get there.

Yeah. Well, I agree. Go ahead, sorry.

Yeah, Chris, I just agree with you on that runtime thing, right? We need to have that machine speed to stop the runtime, I should say incident, right? So, but at the same time, I just feel like this whole thing make current life is more interesting, I should say.

Yeah. Like, for AI portion, I agree with Stacey's mention stuff like, we have the AI, and that AI create additional security concerns and add all those security regulation or add compliance work, and then we finally will try to use AI to solve AI problem. I feel like for a short term, right, at least one to three years, there's a headache on the security side because AI find a lot of things we used to feel like it's normal, it's okay.

And then after three years, I think the speed will back to normal because the problem is just that much, right? Currently, we're trying to find something historically we didn't find it. That's why the volume comes very high.

All right. I got to- But after what, three years after we fix it, seems to be- I got to move us on- ... better, yeah ...

hang on. I got to move us on to the next topic, but I think, the long and the short of what I heard is, well, things will eventually get better, but in the meantime, buckle up, buttercup. All right.

Stacey, there has been a report talking about how the FTC is investigating Hims and Hers, and the issue is that they've been sharing data with third parties. And this is a healthcare company, so a lot of that data is pretty sensitive. But what was surprising to me is this is the FTC and the Trump administration, and I thought we were giving everybody a free ride and a pass here.

But, A, do you think that the FTC will go look at other similar issues? And how prevalent do you think this is? Because my suspicion is that people and vendors out there are routinely sharing data that they probably shouldn't.

Yeah, no, I thought that was interesting of like, okay, why now? Why here? Why now?

What is this that's going on? And so, yeah, I think what's interesting, what it made me wonder is especially this looking at telehealth, right? When we think of healthcare, we think of hospitals or something like that.

Where are they getting their customers? And so for something like Hims and Hers, it's marketing. Well, what do we do with marketing?

Data. And so, looking at that and going, well, okay, why did the FTC hone in here, and what are the guidelines? What triggered them looking at it and why?

And, I think when we look at different companies, especially with so with healthcare and HIPAA, and everybody knows, like, don't mess with data and HIPAA, and just don't even go near there. Yet we have these organizations that are really looking at telehealth that as a business, in a way, when you look at other hospitals that aren't. Now, does this mean something for other companies, and should we be looking at this and going, oh, they're looking over here.

What are they looking at us? And I think some of the loophole is that when you are a health company, you're under a lot more scrutiny. " ...

that we could look at HIPAA with this. Is that a low-hanging fruit, or is this indicative of something that they're going to be finally looking into, which is how organizations are using our data, and even the big question, should they be looking at our data? I think, again, while I said I was optimistic, I'm not so optimistic to think that there's been a philosophical change there, but I think it's good as we see these examples, and just like with any kind of legal aspect to it, is we rely on the rulings that happened before.

And so if this does come to a ruling and we do look at this and they say, "No, even though you're a healthcare company, even for marketing purposes, you can't share data. We appreciate that you're a business that isn't getting insurance referrals, maybe, or you have to make your own business, but still, don't muddy the waters. You cannot share data," then that will set the precedent, eventually, for future law cases or situations as well, which I like to see.

Mm-hmm. " Or is it more of a question of sympathy where you're like, "Oh, damn. " That's a good question.

But what stands out to me is that the regulators seem to treat privacy consent and the product design as part of the same trust problem instead of separate compliance issues. I feel like it's an important shift. And with healthcare, the risk goes beyond privacy, and AI systems rely on the inaccuracy or manipulated medical data.

It becomes a patient safety issue. It's not just a security issue. So I'm just curious how the telehealth companies design consent and data governance going forward, and or just like simple, it become the other compliance requirement, to be honest.

Mm-hmm. Yeah. Right.

Well, Chris, what does that look like? Because to Wiki's point, in the age of AI, when my data gets shared or even lifted by something, it'll be around the world five times before I ever hear about it. Right, and it sort of already is.

I'm just trying to remember, it had to be '92 or '93 when I first encountered healthcare. And I love critical infrastructure, but you just had to stop and look at that one, and say, we have medical universities, so we have the students. God bless their souls, right?

So we have hackers on the inside with life critical systems that are directly coupled to devices that are implanted in human beings, controlled by cybernetics. Right? If you want the ultimate use case of how we need to build things right, that's the one.

Power and water and shipping and all the other things we see as critical infrastructure pale in comparison. And as everybody said, right now we're at this point where you have HIPAA, we have certain things. FDA in the States is usually one of the leading adopting software bill of materials and so forth.

What's left to be done? A lot, and if it needs to be done for this case, it probably needs to be done for everything. From insulin infusion pumps to everything.

Let's just stick with software supply chain because AI is now driving all that, too. I need to know what code is in it, where it came from, who touched it, because you're plugging it into my grandmother, right? How do I actually maintain all that without taking my grandmother's healthcare information and giving it to everybody?

So I need to do both things at the same time. And I think, again, I think we can. And again, it's not just about me, us, my company.

These are the same sort of things I have been arguing for years, before the name on the screen came along. Because I think we have to get there. We cannot have the kind of sci-fi medical systems that folks of us growing up in the '60s and '70s were reading about, I think are possible, I think are incredibly beneficial, if we are going to do the same thing that we do with monolithic platforms opaquely sharing our information behind our backs.

Mm-hmm. Well, do you think it's possible that because of AI, we'll know more about how that data's being shared? Because theoretically, I should be able to apply my AI to go find out where my data went and who has it.

Oh, yeah. To be clear, we do that every day. I don't think somebody in a chair like the one I'm sitting in right now, with this sort of authority in the world, should be building systems that don't do that by default.

Why not? Why wouldn't you? Why would I go into a customer space and deploy a system that has the capability of keeping track of itself and what's going on and the obligations that I have taken and I implemented on my behalf, why wouldn't I just make it self-aware?

Pay attention to where you are and what's happening and the information flowing through you, box, computer application, whatever it is. And you have these priorities, you have these obligations. Pay attention.

Yeah. Stacey, will we ever have a conversation about who actually owns the data? And I bring this up because theoretically, this healthcare information is my data, but for whatever reason, when a vendor gets involved and I give them my data, they think it's their data, and suddenly it's in their systems, and then they feel like they can sell that data because it's their data, but maybe it's actually my data.

So did we never really have this conversation about whose data is what? My first response is, well, first they have to care. First, somebody has to care more than the money, right?

" I think as soon as data becomes anonymized in most people's mind, then it just becomes that maybe it's your data when it's you as an individual, but when it is part of the group and it's anonymized, it becomes their data. It's their data that they're looking at from their systems that they have collected. Do I agree with that?

No. But I think it's that cognitive dissonance that humans just have that we do to make things make sense to us. That when we look at one person, if we isolated ourselves and said, well, your particular data was shared.

I logged into ChatGPT this morning, and it actually offered me to connect all of my medical apps and everything to ChatGPT, and I went, "No. " "No, I don't like that. " And then I thought, yet on the other hand, here it is, it's on my phone.

It's connected probably to a walking app that I have and being shared there, there, and there. And so there's so many different places where we have cognitive dissonance. " Right?

We just do things all the time that don't make sense. So is it our data? Yes, absolutely.

" A perfect example everybody can try today, right? Because I keep track of what these major vendors are doing, and ChatGPT, I think in the last couple of months has added this under Settings, Personalization, Memory, Memory Summary, right? And I check this periodically when I'm using their product, and it's fascinating.

The function, I think, is good. I think it's actually quite useful, but it's gathering and correlating information and personal names and everything else without the customer or the user having any recourse, any visibility whatsoever into where, what, why, how. It's taking everything you put into the interface, it's taking it and running through other AI systems to then generate things it uses to summarize who you are and then make that input into what it generates for you.

So, I don't want to pick on any one vendor in particular, but it's where we are. As we mentioned in the other segments, governance doesn't exist in AI. We haven't figured out how it works.

But I'll tell you, as an AI vendor in my own right, I'm not going to give anybody a system where you have no ability to tell what the system is doing with your data. But that is the default state of the big players out there. They explicitly give us no ability to see what's happening with our information.

Sure. I'm going to have to end this chat. As always, thank everybody for sharing their insights.

I guess my last note on this little subject, though, is we talk out of both ends of our mouth, right? I do. I'm like, "What?

You used my data to figure out that I might be interested in something, and you sent me a marketing advertising to buy a piece of life insurance? " However, if then that same message three weeks later is, "Hey, we've been looking at your data, and you're maybe three days away from a heart attack, and you should go see a doctor," well, then yeah, maybe that's a good thing. So we'll have to see how all this plays out.

But as always, everybody, thanks for sharing your insights. Thank you all for watching the latest episode of the Techstrong Gang. Please stay tuned for the rest of the replay of the Techstrong TV lineup.

And as always, we'll see you again tomorrow.