The AI Dilemma: 7 Principles for Responsible Technology | Digital CxO Summit

Hi everybody. My name is Art Kleiner. I'm really gratified to be here in an audience of decision makers, people who are using technological systems, digital transformation, and in particular, ai.

I'm a advisor to companies on, uh, technology organizations, people leadership, and the how the behavior of all of them affects each other. And I've just co-authored a book with Juliette Powell called the AI Dilemma. Seven Principles for Responsible Technology.

The dilemma is pretty simple. The technology in the wrong hands is dangerous, and in the right hands, it's beneficial to all. And both of those hands belong to us.

And how we frame the technological use and development in the early stages makes a big difference in what the outcomes are and, and how they affect people, and how that rebounds back to affect us. We're in a time when the responsibility related to AI is becoming more and more obvious. So, you know, there are deep fakes.

They're probably gonna have some impact on the next set of elections. There are robo taxis on the streets in San Francisco. There are intellectual property issues that are coming up.

There are a lot of people who are prominent in the field saying, AI is moving so quickly that we need to put a pause on it. We're not putting a pause on it, but there are more and more people saying that we should think about it. And here's a little diagram of the number of incidents and controversies that come to light each year.

Uh, this is logged by a group called the AI Algorithmic and Automation Incidents and Controversies Repository. And as you can see, the number is up to 364 this year. And we're still only in September.

Back in 2017, it was only 59. So these issues around our use of this leading edge technology, and by extension, all digital technologies are going to require some thought and some change. And in the next half hour, I'm gonna talk about what those changes are looking like.

And we're going to hopefully have a couple minutes for questions at the end. So let's start with a problem that, uh, hopefully none of you are in the position of, but maybe somebody here is the manufacturer of a self-driving car, and therefore it's up to you to program the machine learning interface that determines what it does. And so you have to decide, for instance, like the trolley problem.

If the brakes go out of control and the car is careening towards an intersection and it has to go one way or the other, there's no time to avoid hitting someone. Is it gonna hit a pregnant woman or is it gonna kill the dog? Which one?

How do you program that in advance? Or suppose it's coming to the intersection and it's got a choice between someone crossing on the green light at the right, or crossing on the red light at the left. Which one does it strike?

Suppose it's a choice between a young family and a group of older, you know, senior citizens. Suppose it's a choice between women and men. Suppose it's a choice between athletic people and sedentary people.

Who do we sacrifice? This is a pretty tough question, and there's an online exercise put out by the m i t media lab that is still there, that asks questions of this sort. And they asked, so far, they've asked, uh, they've gotten responses from millions of people around the world.

233 countries in 10 languages, about 40 million decisions in all. And you know what? Everybody agrees on nothing.

Well, they agree on three things. Everybody wants to save more lives. So given the choice between hitting five people on one side and two people on another side, just about everyone will go for let's only hit two people.

They will favor saving human beings over pets. So yes, we, we, we strike the dog and they will favor saving children over adults. And there's nothing else.

And if you play this game, you know, your response will vary. You know, some people kind of play it like a video game. That's so I did.

And some people, like my co-author, Julia Powell, Powell really, you know, found herself empathizing with every one of these decisions. But either way, the decisions are going to have to be made, maybe not around self-driving cars, but certainly around many of the things that AI and, uh, other similar automated and algorithm technologies do. So in the seven principles that we developed, we looked at how people are coping with these issues now, how companies are making decisions, what companies are doing to self-regulate.

And honestly, the track record with self-regulation is not terrific. It's mixed at best. And we distilled from that seven principles that if you keep them in mind, in the development of your approach to ai, including generative ai, you'll be doing better in the long run.

Less liability, more intention, and, you know, more realization of what you're trying to do and less harm to people and fewer hours. Looking into the mirror at 3:00 AM the first principle is called be intentional about risk to humans. This is a picture of Stanislav Petrov, one of the heroes of the Cold War.

He was in the Soviet military in 1983, and he was in charge of looking at incoming missiles. If there was a sign from the satellite system that there were incoming missiles, he was supposed to, uh, you know, sort of communicate that to the command and then they would retaliate. One day.

There were a sign that there were five incoming missiles from the United States, their worst enemy. But he had a feeling in his gut that this was a system malfunction. This wasn't an accurate reading.

And after a minute of doubt, he just decided, he sent a note to high command saying, there's an error with the system. There is a false, uh, message about incoming missiles. It was a false message.

The Soviet Union did not retaliate. We are here. And when Juliet and I heard that story, we thought, what if there had been an AI available at that time to intervene?

Would any country be reckless enough to put an AI system in place that would, you know, uh, make the decision faster than a human being could? And I said, no, it's not possible. And then we asked a nuclear physicist, Marcus Sabol at Columbia University, and he said, actually, we, you know, the community of nuclear physicists are concerned about it because in a changing, chaotic and globally competitive world being first gives a country an advantage.

And some leaders may see it that way. And ai, automated weapons of mass destruction are a terribly scary eventuality. Bummer.

Well, this kind of thing is why governments are now talking more and more about regulation. They can't regulate the, um, governments necessarily, but they can regulate the companies that are the contractors who supply these systems and who work on these systems. And, uh, in the, uh, front, uh, lines of regulation, the most advanced proposal is the European Union AI Act, which European Union Parliament is now considering it's likely to be passed.

It could go into effect as early as 20 24, 20 25. And it does a remarkable, well, it does several remarkable things. One is that it, um, it, it includes every company that does business or gathers data in the European Union, which right now that means just about every company doing AI or using AI worldwide, it can find those companies up to 6% of their global annual revenues.

And it divides all AI activity into four categories. Minimal risk are, you know, a lot of the operational things we've been hearing about all morning or minimal risk, they don't really affect people. Uh, games, spam filters, you know, basically anything that cannot do harm to people will be basically under, you know, unregulated under the, uh, eus AI Act.

Limited risk systems are systems that communicate with people. Your virtual therapists, your virtual friends, as long as they don't pretend to be human. If you know their ai, then they're okay too.

High risk systems are self-driving cars and predictive analytics and lots of things in involving medical research and other forms of research, things that could have an effect on people. These are going to be audited. They're going to be audited, like privately held.

Companies are audited now with independent accounting auditors who are, you know, not necessarily beholden to the companies themselves. And this is a first technology. Companies have not been audited by outsiders before except in cases of litigation.

Now this will be happening regularly, and there are a lot of changes involved in that. Very interesting to see how that unrolls accept risk. Systems are systems that single out groups that, uh, you know, predict, uh, that do biometric scans of people or facial recognition that, uh, manipulate children into risky behavior.

Those are going to be banned. And that too is a big change. Italy banned chat G p T for about a month, and then they rescinded it.

These bans will be indefinite. So the first principle, be intentional about risk. We're gonna be seeing a lot of conversations about that, and companies are going to have to decide what their intentionality is and how they muster the activity necessary to, um, to keep on track and to choose the risks and choose their approach.

Principle two is related to this. It's open the closed box, you know, chat, G P T or any system. Dali doesn't know it, can't tell you how it recognizes a cat any more than a toddler can tell you how that person recognizes a cat.

And in fact, the toddler has a better accuracy rate. 98%, uh, systems are about 85%. Here's what happens when you feed images of a cat into a generative AI program.

It processes the cat, it gets closer and closer to understanding what the cat looks like. And then when it's asked to create an image, it starts at the upper left with basically random pixels and iterates closer and closer and closer and closer until finally it gets something pretty close to a cat. And then that's the image it, uh, feeds back to you.

There is no theory of cat mis, there is no dividing line between what's a cat and what's not a cat. So what if it's a system, a predictive analytic system, and it falsely accuses somebody of fraud, which as we'll see, hap shortly does happen. Or if something else, how does that person query the system?

They can't ask the program to explain itself. It can't. They have to ask the company for information about how it programmed the system and what data it shows and what model it chose and what, uh, why it created this software system in the first place.

And there's a name for that type of information. It's called a trade secret. So increasingly we're going to be seeing companies balancing their legitimate need to keep private, their innovation versus the legitimate need of people who have been affected by AI systems to question and query those systems.

Another big change and a lot of uncertainty about how that will play out as well. The third principle is confront and question bias. This is from a project called Gender Shades, uh, at m i t by Dr.

Joy Wallum Weenie. And, uh, that team tracked, uh, programs set up by Microsoft and Facebook and I B m as you can see, to tell from a photograph whether that's an image of a male or a female. And notice under the light skinned male that it has, these programs had a 99% accuracy rate under lighter skinned females and darker skinned males.

The accuracy rate was not as good, but hovering at least near to or above 90%, and then under darker skinned females, they were all below 80%, and two of them couldn't even get a third of the answers, right? So that suggests that there's a problem with the data set and that there was a bias built into the data set of, um, you know, used to used by these programs, the number of pictures we heard earlier, that AI programs are less emotional than people, and that's true, but they are just as biased. They reflect and amplify and automate our existing biases.

And honestly, we're all biased. So it falls to us to the companies using this and working with this and developing this to confront and question that bias. Another big change because we'll be called upon to do that.

Uh, a fourth principle is reclaim data rights for people. Each one of these strands is a bit of personal data that Apple has the right to gather about you when you click okay to use one of its portals. Other companies have about similarly dense strands.

There is going to be more and more of a drive for us, for us people, us individuals to manage our own data rights. You know, to choose who can see our data, how often they see it, how long those rights last for. If there's money involved, we may start to see people asking to be recompensated, to be compensated and systems evolving to help them do that.

Why is this happening? Because we are coming to realize, we, people everywhere are coming to realize that we are our personal data. It's the first thing people see about us.

Another huge change that, uh, we, the developers of these systems are going to be thinking about and recalibrate. Another big change is our fifth principle holds state care accountable. That man is Mark Ruta.

He was the prime minister of the Netherlands. I know at least a couple people from the Netherlands are here. So you may remember that there was a child welfare benefits scandal that started about in 2007, a predictive analytic system run by the government to determine who was likely to commit fraud.

And they falsely accused 26,000 families of child welfare fraud. They forced those families to return the money that they didn't have. Some families went bankrupt, some families split up, some families, um, went to jail, and about 1500 children were separated from their parents.

The system was biased against immigrants, particularly from Morocco and Turkey. It was biased against families with, uh, you know, single parent families or where one person held two jobs. And the accountability part is that this situation only came to light after several years, and the families are still waiting to be compensated in made whole.

There's an agreement to compensate them, but again, we need better systems of accountability. So I have a personal way of thinking about this. Um, I don't know how many people here are parents, but if you're a parent and your experience was like mine, I have three daughters.

When the oldest was an infant, I thought, I've got this. She's sleeping. I know just what to do.

And then she got a little older and her daughter, her sisters came along and suddenly I knew there was a lot more that I had to learn. And that's where humanity is right now with artificial intelligence. It was so quiet and now it's growing so fast and it's getting into all kinds of trouble and it's doing great things, and we're gonna be responsible for it when we're working on the AI dilemma.

I sometimes thought, you know, this is like Dr. Spock for leaders of artificial intelligence. We need guidance because the consequences of what we do are often unintended and it's hard to keep track, and we're learning as we go.

And that's where accountability is going to come in. Our sixth principle is favor loosely coupled systems. This is Charles Perot, a sociologist who studied large complex systems like ai.

This was before ai, and he was studying nuclear power plants and nuclear power plant disasters like three Mile Island. And he discovered that there was always a case where a company that was really tight knit with just a few people making decisions, part of the same system, part of the same organizational structure, without much input from outside, without much training from outside, something would go wrong and this would happen. We need organizational structures where the dominoes are laid out so that if a couple of them go down, they don't take the whole domino set with them.

That's called loosely coupled. And then finally, the fifth, the seventh principle is called, uh, embrace creative friction. If you work with computers or with computer systems or with it in any way, you know that friction is usually considered bad.

We want to have frictionless, seamless interactions. But it turns out, and you know, here's a book, one of the most popular books on user interface design called Don't Make Me Think, okay, please don't make us think about it. Just give us what we want.

But it turns out that what we want is not in the short run, is not always what we want in the long run. And the most successful, you know, uh, least the, the companies with the poorest with the best consequences. And yes, the most profitable cons companies are those that embrace and employ some form of creative friction.

Uh, sociologist David Stark also at Columbia is one of the people who discovered that there is a, you know, there is an association between deliberate processes to step back and say, how is this going to affect people with what we do? And ultimate, uh, profitability and viability. It's a lot of research on that.

What do you talk about when you're in one of these creative friction conversations? Well, you talk about there are at least four logics involved with, uh, developing the future of AI systems and automated systems. There's the engineering logic where, you know, people really want to do it well.

And if there's ancillary stuff like responsibility, often they wanna develop, you know, they wanna delegate it to the specialists. But here, this is a whole system and the specialists don't have all the answers. We need to bring some human logic in with our engineering logic.

There's corporate logic going down. You know, business people want to compete, they want to be profitable. They, their business survival depends on it.

And so therefore, they're pressured to cut corners when they see their competitors doing so. And they have to bring into their, the longer term, um, more farsighted business logic, the strategic logic that allows them to operate AI systems and develop AI systems more effectively. There are social justice advocates increasingly involved with ai, increasingly confronting com companies, their goal is to protect the vulnerable, which often means everybody, but sometimes means specific groups, and they're holding companies to account and they're not always doing it as effectively as they could.

And so that becomes a part of the conversation. And then there's the government logic, which is, as I said, stepping into regulate. And they're concerned with authority and security, and they too can overreach.

And companies are aware of that, and companies need to innovate and government needs to allow companies to innovate. So there's a lot of conversations to do. And when the, when the, at the end of the day, you know, who's going to watch the watch robots, these are going to be questions that all of us will play a role in answering.

As Winston Churchill said, the price of greatness is responsibility. Thank you very much for your time with me today. And we have about eight minutes, uh, to tackle questions if there are.

So, uh, art, we have a, uh, something from the audience here. It says, uh, my thought, if AI is being used, shouldn't it prevent a possible safety issue in the first place? Just thinking, what's your thought on that?

So can we use the technology that we are overseeing to help us oversee it? Of course we can. And there will be a lot of innovation on how to do that effectively.

But at the end of the day, the, uh, we have to make sure that human beings are in the loop. And there's a problem with automation, which is known as automation complacency. When people are assigned to oversee the technology, they get bored, they get complacent.

We are often willing to delegate control. It turns out that there's an illusion of control, which makes us think that we're in control when we're actually less in control. For instance, when you use a, uh, when you use a chat system to write a a paper, you may think, well, this is great.

I'm in huge control. I've gotten this done much faster. But have you checked it for accuracy?

That may take just as much time and effort. So we need to learn to use the technologies, and we need to learn to stay aware, to be aware of our own thinking and to apply critical thinking and to instill that in our staff who are gonna be operating these programs, developing these programs on our behalf. Alright, uh, the someone else from the audience, uh, was talking about how they're relating to the Strange love and Dr.

Spock story and, and, and the, and war games. So, uh, if you wanna talk a little bit about that. Uh, well, you left out Minority Reports and The Matrix.

Um, we have an amazing, um, ability to imagine the worst and, uh, somewhat pretty good ability to imagine the best. The hard part is imagining the real future, which is gonna contain both. I do a lot of work with scenarios and have done that for a long time.

And, you know, there's no really great or awful scenario. The important thing is, does a scenario have something to tell you? So when you're in a company and a scenario says, you know, you could be held liable for this, that doesn't mean you're going to be, and it doesn't mean calculate the probabilities.

How likely is it? It means think about solutions now that will be robust no matter what future comes to pass. So that if we end up in, you know, minority report or, um, or, uh, 2001, we know how to deal with it when it comes time for it.

And if we end up in a great future where AI continues to generate more and more practical, valuable forms of itself, and we learn how to work with it effectively and people, you know, jobs are not lost, jobs are created doing this, that we're prepared for that future. We want to make robust decisions now that will allow us to do whatever we need to do going forward. Let me say one or two things then in the minute that we have left.

First of all, I really appreciate where, you know, people in industry now are, we have been very used to tools expanding and developing, and we have become more and more specialized. AI is like a forcing function. It's forcing people to act more as generalists, and that's not a bad thing.

It's going to turn out to have immense effects on the way people, you know, enter the workforce and go through leadership development. It's gonna have immense effect on, uh, the choices we make. And, uh, and it's going to really, uh, the way in which we approach this, the way in which we organize our conversations.

It may seem like small things, but they are going to really impact the future that we're creating together. Thank you. com and look forward to, uh, learning more about what happens at, uh, at this group and at other groups to come.