AI Leadership Insights: Navigating AI Governance Challenges with IBM’s Phaedra Boinodiris

Hello, I'm Mike Vara and welcome to the latest edition of the Techstrong AI video series. We're here with Ra Re, who's global lead for trustworthy AI for IBM consulting. And we're gonna be talking about, well, just what does it mean to have trustworthy AI and how do you get started phage, or welcome to cho.

Oh, happy to be here. Thanks for having me on. A lot of folks are generally familiar with the concept and they understand governance, but I think a lot of folks also just nod their head and kind of think that they're agreeing to something, but no one needs to know exactly.

We're in to get started with all this. So what are you hearing from folks and, and what does it take to kinda have actual trustworthy ai? I, I actually, uh, offer, uh, uh, an edit to your opening statement, which is, I, I don't think people do understand in, in general AI or GORD governance or what it takes the nature of the actual work to do this well.

Uh, I think there's a lot of misconceptions, a lot of myths, a lot of lack of understanding on, on this subject. Earning trust in AI is so critically important in order to be able to get the kind of outcomes that we ultimately want from the use of technologies like this. But it's not strictly a technical problem at all.

It's not a technical problem with a technical solution, but one that is socio technical. And it, it's so interesting, Mike, when I, when I ask large technical audiences the question, who in your organization is actually accountable for outcomes from ai who, who's accountable for those outcomes? The top three answers I get are pretty bad.

They're pretty bad. I mean, the, the first one is no one overtly bad. The second common answer I get is we don't use ai, which is absolutely laughable because of course their employees are using ai, whether they're formally keeping track of it in an inventory or not.

I mean, several of the licenses that this company or organization has already procured, likely has AI embedded in it and in its most recent version. And then another common answer that I get that is concerning is everyone, and the reason why everyone is concerning is because if everyone is being held accountable, is anyone actually being held accountable? Because I, ID opine and in our last institute for business value study, ID opine, you have to have enough power to do the work of, of governance.

You have to have a funded mandate to do the work. And it's, it's a lot of work and it's actually expanding. It's growing.

I wonder also if we get enamored with the whole idea of ai. And a lot of the folks that I talk to don't really understand that the output for, especially from these gen AI platforms is probabilistic, which means it's a best guess and it probably won't show up the same way twice. And we are trying to insert that into business processes that have to be this done the same way 100% of the time and audited and they're deterministic.

And I just wonder, in your experience, do people kind of get that or is there a mismatch in our thinking? I think there's definitely a mismatch. 'cause there's a complete lack of understanding there.

There's a, a massive gap on the subject of AI literacy, massive gap. Uh, and you mentioned AI governance. People sometimes think that it's only generative AI that needs governance if they understand what the risks are.

And of course, all forms of artificial intelligence, uh, require the appropriate s guardrails and the, the appropriate considerations to ensure that these models behave in the way that they are intended to behave. So I, I think there's, there's a, uh, a tremendous lack of understanding, which is why so much of the work that we're doing right now is really introducing AI literacy in a holistic way. I think also people have it in their head that they already have some sort of governance framework and that it'll just be extensible to ai.

But what are people not thinking through entirely? Well, I mentioned that the, the work of governance is, is expanding. So for example, like you have to get value alignment with across your entire organization so that everybody who has a role to play on the subject of artificial intelligence recognizing, recognizes the importance of getting it right and, and responsibly curating it.

That's one. The second is to be able to actually capture the AI model information and the metadata about these models in an inventory system. Then you have to keep track of regulations and there's a changing regulatory landscape, but then also there's a recognition that you can have AI models be lawful but awful, which means you have to push into ethics.

And anytime anyone pushes into ethics, you have to be a really, really good teacher. Have people, like how would you even recognize what are the functional and non-functional requirements of an AI model that reflects an organization's ethics? Has it even been detailed what an organization's ethics is and how you expect that to be reflected in technolo technologies like ai?

Which to your point, like yes, the definitely there's data governance, data privacy has been around for a while, but AI is another level in, in terms of the expansiveness of, of all that needs to be done when it comes to making sure these, these models are behaving appropriately and can earn trust. And it seems like also the bad guys have figured out that they can poison these models and they can do it in a way that is relatively simple. They just figure out where you're pulling data from and start inserting some data that will get that model to either completely misbehave or behave in a way you don't want.

Well, it, it, that is indeed concerning. And what worries me from a cybersecurity perspective is so often CISOs who are responsible for cybersecurity within an organization are oftentimes not even invited to the meetings on AI investments. Like they're not even being invited to the meetings to be able to understand like, what is this organization's AI strategy?

What are the concerns? What are the considerations? So that they'd be able to have some kind of an input on what is being procured or built with an understanding about what you're describing.

But then even so, and this is I think really important for your audience to hear, even organizations that truly have the best of intentions with respect to how they wanna use AI can end up inadvertently causing harm due to a lack of those safeguard rails. And due to that lack of AI literacy and understanding. So it's extremely important that there's the right multidisciplinary approach to the work.

What's the level of sophistication and understanding among the auditors these days of how these models work? Are they starting to ask some more difficult questions and is it just a matter of time before, uh, you know, a levy gets passed to somebody that's going to make everybody pay attention? I think that, um, it's slowly, there's beginning to ask better questions, but again, I, I think it's still early days.

And I think when we start to see more lawsuits as an example of, uh, again, organ from, with, with respect to organizations who had good intent but ended up inadvertently causing harm, that's when we're going to see organizations paying more attention because I, I suspect the regulatory landscape, at least within some parts of the world, are not going to be strengthened within the next year or two in, in fact, I think more parts of the world are looking at rolling back regulations in order to be able to have more investments or be perceived as being more AI friendly to businesses. One of the subtler issues in my mind is that a lot of times the data that we're using to train the model, uh, reflects a bias that's hidden in the system somewhere. And it's not just like a bias where, um, it's about race, creed, or color.

It may just be as simple as, uh, the data suggests that this area in a real estate transaction is undervalued, but it may turn out that that was, uh, something systematic in terms of redlining of that area, and now we're gonna put that into our a model, into the model, and it'll just make things even worse. A hundred percent. A hundred percent.

And it, it's why, again, when I say AI literacy, we desperately need a holistic approach to AI literacy, meaning there ha this, this is actually is an opportunity, I think for a, uh, a rejuvenation of the liberal arts, let's say. Because if you're lucky enough to be able to take a class in AI or data ethics or AI ethics, like you're likely in a school of engineering and you've sub-categorized as a coder or machine learning scientist or data scientists, but literally not everyone else, we need to have far more interdisciplinary cross-disciplinary programs on the subject of ai. So those individuals who will be, for example, as you said, determining uh, creating AI models to predict interest rates on home loans actually knows what the history of redlining is.

Because if they don't, they will end up calcifying, systemic and, uh, systemic biases, uh, in order to produce yet more inequitable outcomes. So it's, it's, and again, it's not because they're evil, it's not because they're nefarious, it's simply because they don't know. Do you think we might see a spate of lawsuits on this topic?

Uh, I think I've seen a handful so far already, but it, it feels like it's only a matter of time before some lawyers start delving into some discovery process for the data to figure out that the model was flawed. Yes. And we, like you said, we've been seeing more and more, there's an AI incidents database, in fact on, on the internet that, uh, that details, uh, you know, lawsuits or times where audits were made publicly available and uh, reputations were lost, et cetera, et cetera.

But I think again, it goes back to literacy. You know, if, if we don't have more individuals being able to be critical consumers of the tech and to ask the questions, who's accountable for this model? What's the level of accuracy of this output?

Where did this training data come from? Was it gathered with consent? Is, is this data even representative of all the communities that we need to serve?

Like if we don't have people trained to be asking these kinds of questions, then we're not gonna to, to get the kind of trusted models that ultimately we as a society are looking for. I think when I look at this, I often see two extremes. One is some people are overly trusting in the data and not doing enough critical thinking.

Other folks know where the data came from and don't trust the output whatsoever. So do you think that in the age of ai, we might get to something in the middle where, uh, people will be savvier about the data in general, but those that have been suspicious might be become more believers because the process could eventually be more vetted? I, I think we're going to slowly get to a point where people are saying, give me the evidence, give me the evidence of this output.

Where did this training data come from to be asking those kinds of questions and really forcing organizations to be transparent about their model and to be held accountable for those models. But again, in order to be able to get there, we've, we've gotta change how we're teaching the subject in schools, Do we, not just in school, but I wonder, um, well, you know, folks that have been outta school for 20 years need to go back and get some courses and some lessons and things that, you know, will make them, uh, eligible to work in the future. I was asked at a summit last week, you know, what are the top three skill sets or competencies that you would want to see, uh, students double down on in the coming years with respect to ai?

And I said, I don't need to give you three, I'll give you one. And that is knowhow to be a lifelong learner, because this space is going to constantly, constantly, constantly be evolving. So making sure you're respective of what you wanna be when you grow up, that you, you're focused on learning how this kind of technology can augment your intelligence and how to be a critical consumer of it, because this space is gonna be consi constantly changing like this, AI literacy can never end.

So what's your best advice to folks? And the flip side of that question is always the same, which is, you know, what are you seeing out there that makes you roll your eyes? Well, I would say the rolling of the eyes is, uh, as I mentioned, you know, even organizations that have the best of intentions end up causing harm.

And there's plenty, plenty, plenty of stories in the news. And it's not just generative ai predictive models too of, of, uh, uh, organizations getting it wrong and end up ending up causing disparate harm or exacerbating existing, um, biases or unfair biases. But then also, what has me roll my eyes?

'cause I, I've been preaching a lot about holistic approaches to AI literacy is, you know, the, the school systems today and the culture of continued siloed approaches to curriculum because we desperately need to have more holistic AI literacy programs that includes like, yes, you need to have school of engineering and computer science so people can explain the nature of how the sausage is made. But you need linguistics, you need philosophy, you need government, you need, right, you, you need to have all these disciplines in order to be able to teach this appropriately. And I, at opine, we need to bring it much earlier in people's academic careers and teach this subject in high school and middle school and not in computer science class.

We need to teach this in social studies class. Because if you think about it, the real nature of data, like my favorite definition of the word data is that it's an artifact of the human experience. We humans, we generate the data or we make the machines that generate the data, but we have 188 biases and counting.

And there's many good reasons why we as human beings have biases, but we have to know, like ai, it's like a mirror that reflects our biases back towards us, but we have to be introspective enough to look into the mirror and decide, does this actually align with my organization's values? Because very quickly we are moving from do I trust this AI model to, does the worldview being represented in this AI model actually align with my own? And that's why we've gotta be better at teaching the subject, uh, to, to the next generation, if not this generation as well.

All right, folks, you heard it here. Even in the age of ai, critical thinking is crucial because well, garbage in is still garbage out. Hey Phia, thanks for being on the show.

My pleasure. Thanks for having me. This was fun.

All right. And thank you for all watching the latest episode of the Textron AI series. You can catch this on our website.

We invite you to check them all out. Until then, we'll see you next time.