What to Expect When You’re Expecting a GenAI Baby | RSAC Virtual 2024

I am sun ou and thanks for joining my presentation on anticipating the next stages of the AI revolution, or, uh, my preferred subtitle, which is What to Expect When You're Expecting Your Generative AI Baby. So, a little bit about me. Um, I am, uh, one of the co-founders at a company called Agnostic, which focuses on, uh, uh, need to know controls for a large language models.

But previously I was the chief scientist at Bank of America. I've been in security for a long time, and the had number of different, um, accolades. But, uh, when it comes to ai, I, I'll tell you, um, there's a lot of people who claim to be experts.

Let me, let me just declare upfront, I am not one of them. I think I've just gotten past the peak of Mount Stupid and just, uh, getting past the Valley of Despair. Um, but it's one, uh, have a full disclaimer that says, look, if you're looking for an AI expert, um, who's at this, at the upper right car part of the curve here, that is not me.

However, I've been thinking about the problem in a way that hopefully will help those who are, um, following behind just a few steps behind me, can, uh, quickly understand where we are, where are we today, and how do we anticipate what's coming so that we can be prepared for the future. And one of the first ways that we can think about, um, understanding the capabilities of AI is to be able to benchmark it with some sort of age. And that's kind of, uh, when, when we think about what to expect and when you're expecting, we wanna understand what age, um, these capabilities are at, because then it gives us, uh, a way to benchmark what our expectations are.

And, uh, at the danger of anthrop AI too much, uh, it just helps us understand and the level sets so that we can then, uh, calibrate our thinking expectations and, uh, what, what to, uh, how to handle governance and other kind of controls. So the goal here is to be able to understand, uh, what the first part of the goal is to understand, uh, how old is my generative AI baby? How old should I, uh, consider it?

What's the age of its development? And, uh, one, there are a couple of different ways to think about it. And what I'm gonna share is a couple of different models to think through the problem space.

The first model is offered by darpa and darpa, um, is been around AI for a long time, and so I would, uh, count them as a true authority in the AI space. And they've come up with this notional intelligence scale that, uh, is comprised of four factors, perceiving, learning, reasoning, and abstracting. So perceiving it's about the, this ability to, um, to consume lots of data and to be able to make sense of the world.

Um, the second is, uh, to be able to take all that and start adapting to it, um, to be able to, um, again, make sense and then to, to, uh, adjust to those situations. The third is reasoning, which is how well a machine can decide on what to do next, and to be able to explain why it made that decision. Um, note that the DARPA calls out decision, uh, this sort of decision making and understanding why as a, as a key criteria here.

And then last one is, um, abstracting. This is how well a machine can take what has learned and, uh, decided and apply it to a completely different domain. And the goal here is to be able to understand, uh, where we are at the, with these different technologies, uh, in relative to the human level.

So at the far right of the spectrum is what we consider to be human level intelligence. And the perspective is how far along are we when it comes to, um, our, our current level machine level capabilities in each of these four areas. So if we look back in history, then what DARPA has provided is three different ways of ai.

The first wave is, uh, what they call handcrafted knowledge. Um, and this is, this has a little bit of perceiving a lot of reasoning, um, and we've seen a lot of these technologies we rely upon these technologies. Um, today, if you've filed taxes recently, um, uh, then the, uh, this is, uh, uh, TurboTax is an, is a great example of first wave AI that allows us to, uh, have a system reason, but it doesn't do any learning and it can't really be used for anything else outside of that particular purpose.

The second wave is statistical learning. And this is where we are today, um, where there's, uh, a really strong ability to perceive there's a strong ability to learn, but reasoning and abstracting capabilities are not there. Um, the, the, the, uh, this is statistical based learning, and, um, it provides really great results, uh, in aggregate, but individually, some of the results can be pretty, uh, pretty bad and catastrophically bad as well.

So that's the second wave. And this is where we are today. The, um, the third wave is what we call contextual adaptation.

And, uh, in the third wave, what we are able to do is to have models that, or have, uh, systems that can provide exp explanations as to why things are happening. Um, it gives an understanding of why and why not. And ultimately, the, the reason why this is important is because it allows us to understand when the machine has failed and why the machine has failed, that ultimately allows us to increase trust in the machine.

And what we see is with second wave systems, it fails catastrophically, um, in these individually unrealistic sort of states. Again, statistically it's pretty strong, but, um, there are these edge cases where it may classify, for example, a stop sign as being, um, a speed limit. And you can see how that can actually end up with catastrophic results.

But with the third wave, what we're doing is we're coming up with a world model, and that allows us to then explain why this sign is a stop sign, and perhaps why I shouldn't actually even observe it as a stop sign, because, well, it's red and it's agonal, but it's not at an intersection, so maybe I can ignore it because it's not at an intersection. And so being able to create these world models is part of, uh, what we expect in the third wave. But the key thing here is, um, the ability to reason and to be able to understand why or why not, and what, um, what's interesting is, uh, what we have seen with the second wave systems is that the reasoning is not quite there, right?

What's interesting is that, uh, when Chad GPT first rolled out, or or early on when it rolled out, they made a claim that reasoning was actually quite strong. In fact, if you look at the, the dashes here, uh, back in the early 2023, you see that chat GPT has reasoning at, at all five bars. And a lot of people ask, oh, hold on, does this mean that we're a third wave?

And I would say, well, no, not exactly chat. G pt, uh, or OpenAI later on, uh, would say, Hey, we have advanced reasoning, but they got rid of the depiction here because I think they got some, uh, flack for that. And even if you were to ask chat GT itself, Hey, does it qualify for the third wave?

They would say, Nope, uh, it doesn't. So, uh, it doesn't understand context, it doesn't have the ability to explain. And so as a result, um, GPT-4 wouldn't qualify as a third wave ai.

So this whole notion of ability to explain and to explain why or not is a re really key aspect of then understanding how to benchmark the age of these systems and another model that we can use. So this is the DARPA model to then explain, um, the, the need for reasoning as a key factor in, in the, the age of a system. But let's look at a different model now to, to calibrate, to triangulate the age here.

So, um, um, um, my family, we, we used, we used to homeschool, and one of the things that we learned in homeschooling is that, uh, is the theory of learning and how, um, it's broken up into these three phases, the grammar stage, the dialectic stage, and the rhetoric stage. And in each of these stages, we learn different things and we have different learning styles. But you'll note at the dialectic stage, uh, we are able to be able to explain why and how.

Okay, so it's interesting because what we're, according to the DARPA framework, again, this emerges in the third wave, and we're not quite there yet. And if you're, uh, if you've noticed, there's three, these three different stages, grammar stage, dialectic stage, and rhetoric stage. And this actually maps to our school systems.

Our school systems are broke down into these three stages as well, where in elementary school it's primarily focused on the grammar stage, the junior high school, it's in the dialectic, and then the high school is on the rhetoric stage. So, going back to this for a moment, if we haven't left this third stage yet, or sorry if we haven't left, uh, if we, if we haven't been able to explain why and how, then the question is, where are these foundation models? Where do we have current AI systems today?

And I would argue that they're still in elementary school. So that helps us then calibrate our understanding of, um, of what age, what is the age of ai. Um, alright, so now the next thing to understand when it comes to anticipating, uh, what's coming next and to, to expect what to expect and when we're expecting, is to look at hereditary traits.

And one way to look at hereditary traits is through another model. And this model is called the DIKW pyramid. And what this model helps us understand is this progression of technologies that have allowed us to move up this stack.

Technologies that have allowed us, allowed us to take advantage of data, take advantage of information, and now tools like chat, GPT, which allow, uh, us to take advantage of knowledge. You could think of, um, this as, uh, tools like chat, GPT allowing us to become, uh, a knowledge driven organization. But one of the challenges that we all face in, um, in, uh, in AI is just defining this problem space.

How, how do, how should we think about this problem space? And if you take a blank sheet approach, it seems very hairy, but this pyramid allows us to anchor our understanding, uh, because we can say, what is it that we've done in the past with data? What is it that we've done in the past with information?

Well, in the past with data, we've had challenges like data privacy, data quality, data security. Well, those same suffixes can be applied to information as well. Information privacy, information, quality information security.

And these words, pairings, when you apply to knowledge, allow us to then understand the problem space of the future. It allows us to systematically think through what are the challenges that we're gonna see when it comes to ai. Because if it can replace the word AI with the word knowledge, it, we can see this template, uh, unfold before us.

This is the shadow of the future. And to give you some perspectives of how we can apply this, let me, let me give you some examples. So most of you, I'm sure, know what a data quality, uh, or, or, or know what data quality is and have, uh, suffered examples of, uh, data quality failures.

Well, what's a knowledge quality failure? Or what, what's a knowledge quality issue? Well, might I suggest hallucinations is an example of a knowledge quality issue?

Well, if you look at hallucinations as this brand new thing, you may struggle to understand how to deal with it, but if you see it as a quality issue, then you can look at the data quality governance processes, data quality roles and responsibilities, and maybe even the type of controls and say, okay, what though applies or what can be rolled forward to tackle a knowledge quality issue like hallucinations? And so this model helps us think through that problem space very systematically. But we also have to recognize that there are a whole new problems that emerge for which we may not necessarily have controls yet.

Um, for example, let's say knowledge privacy, um, well, you know what a data privacy issue is, what's, what's a knowledge privacy issue? Um, if you go back to, um, about 15 years ago, Facebook rolled out something called social, uh, graph search. And what was really cool about graph search was it enabled us to, uh, ask questions about or find answers about what our friends liked and common interests, like what our favorite movies were, what favorite pizza cuppings you had, uh, those sort of things.

But it also helped disclose things like what the sexual orientation was and what the political voting patterns were. Well, Cambridge Analytica abused that and abused our knowledge privacy, not our data privacy, but our knowledge privacy. There was no specific data element in fair in Facebook that says, here's your political party.

Um, but it was inferred, and the inference allowed, uh, us to tap this knowledge, age, or knowledge level, and, uh, was again, a violation of knowledge privacy. And now we have to think about, okay, what are the type of controls to help us address that? Um, and it's not restricting PII because that's not where, that's not the, that's not the type of knowledge privacy, um, that, that's more of a data privacy concern.

Knowledge privacy is something that trends in something like that. So anyway, my point is that, uh, this helps us understand, um, the shape of the future and what, what the future is gonna build, um, look like as we look at the hereditary traits from each of the layers below it. Now, I should say, uh, you should, I'm sure you noticed, uh, at the top is wisdom.

And I think that's still in the domain of humans. And, um, as, as parents of this new generative AI baby, we certainly want to exercise a good amount of wi wisdom or discernment because, um, knowledge does not equal wisdom here. And we oftentimes see, um, a huge amount of confidence coming out of tools like chat GT and we can confuse competence for confidence for competence.

And, uh, for those who are familiar with the Dunning Kruger curve, uh, the, uh, this, what we think we, what I think we've seen in the past, um, when it first rolled out was the system, uh, seeming to be very competent and asking, answering questions. But it really, it's more, um, questions like what you see here around, uh, how quickly you can make a baby if you had nine women, um, that you see how confidently tragedy PDs able to answer this question, but it's more reflective of it being at the peak of mountain stupid. But that said, um, the, uh, this is still a pretty young baby, right?

And, um, we're starting to see it grow and develop, and it seems like it's taking a long time. The days along for how these technologies are, are progressing. And over the past many, many years, um, we've seen this steady and slow progression of technologies or steady and slow progression of capability of these models.

What you're seeing here is a, is a graph that shows on the y axis, the accuracy of these models to perform various, uh, tasks and the size of the model as it goes, um, uh, over time. And what we have seen for in, in the early days is, uh, this slow steady progression of accuracy over time. But over the past 5, 6, 7 years, what we started seeing is this, this curve start spiking up.

Um, and what this portends or this suggests is this rapidly increasing capability that is emergent. Um, there are traits that we're seeing that, uh, were entirely unexpected. And to me it's like hitting puberty.

All of a sudden now we're seeing, um, unexpected things happen from these, these, uh, tools that, that we've rolled out. What we expected before was this steady growth where as the model gets larger and larger, it, uh, we still have this trade off between, uh, it being general purpose and being highly performant. And that was our expectation that this system will continue to grow in this steady, uh, predictable way.

But what's happening is more something like this where all of a sudden now that curve is bowing outward and it's causing us, uh, some this, um, it's disconcerting to some degree because we don't know what to expect, um, at this point when, when, uh, these models are starting to hit puberty and it's doing things that we didn't realize it could do. And so as a result, we're starting to see, um, these tools start gaining competence. We're starting to feed in world models that help, uh, tragedy PT realize that you can't, you can't accelerate baby development.

It still takes nine months for, um, a woman to make a baby. Um, and so it's starting to get better, but at the same time, I think there are still challenges that we're seeing. Um, because, uh, uh, as recently as a couple months ago, I asked this question, what country's flag has a colors green, yellow, white, and blue, and only those colors.

And the chat gt thinks it's St. Vincent's and the Grenadines, and it's missing, like, um, white, I think here it is, even though it says it has all four colors. Um, this is what Gemini says, that they think it's she shells and it has red.

So I'm only asking for those four colors, and it adds another fifth color. Um, and then, and Claude thinks it's aia. And here, I, I don't see green and I do see red, and, uh, I don't see any yellow.

So there's, there's, it's clearly there's still some opportunity for growth. And, uh, by the way, for anyone's who interested, the, the flag is Brazil, but that's it. I don't know if, uh, one, one thing that I should, uh, be mindful of.

Well, one thing that we should be mindful of as we look at these models is, um, an LLM, we should think about that as a really, really good, like, what their job is gonna be in the future is like an English teacher or a creative writing teacher. They're probably not gonna be a good math teacher or a history teacher, or, uh, pick another discipline. Uh, rather, they're gonna need help, right?

They may, they're gonna be really good at writing English, they're gonna be really good at creative writing. Um, but when it comes to something like math, we may wanna have it defer to a different teacher, something like, uh, a python, right? Uh, please.

And that's what we are seeing happen now, where you ask a question of chat, GPTA math question, and it says, sorry, I'm really just an English teacher. Let me ask, uh, the math teacher, and here's Python, and let Python do the do its trick. Um, and so just recognizing, um, what it, what its primary skill is, it will help us then calibrate our under, uh, our expectations as well.

Uh, if you ask it to do things that are outside of just the core English, it's, it's not, it may pass the bar, but it's not a lawyer, okay? So don't ask. If you ask it for case law, then just be very careful about the answers that you get back.

All right? And then one another problem that we've all seen, um, is hallucinations. I talked about that as a knowledge quality problem.

And, um, what's interesting about the, this, this challenges that, uh, um, this is really not too different than what our brain already does. Uh, if you think about dreaming or if you close your eyes, you're essentially hallucinating because it's, you're not, uh, there's nothing that you're anchoring your sensory inputs on. And that's what, uh, happened when, uh, there was a, uh, surgery that was being done on this girl where the, uh, the girl had epilepsy and they were trying to figure out, uh, how to address it.

And, um, the doctors were stimulating a certain part of her brain. And every time that they stimulated that, she would laugh, but she of course couldn't see them stimulating her brain. So it wasn't anchored in, um, what, uh, what her visual input was, was providing.

What she would do, do, do instead is when she was asked, why did you laugh? Whatever she was seeing at that point in time became the cause for why she left. So she would say, oh, it's because of that picture on the wall, or, oh, it's because of what you're wearing is funny.

In other words, she hallucinated, she hallucinated because it wasn't anchored on, it wasn't grounded on some, um, some fact or something that she could directly see, uh, that's causing the, that's doing the cause and effect there. So effectively, yes, it is normal actually for these systems to hallucinate. It's, we're modeling our, our brains.

Um, artificial intelligence is a model of how we, uh, have developed intelligence and our, and our brains hallucinate as well. Um, so I guess it's, in some respects, it's not, not a surprise that these systems can also hallucinate. And we also have, uh, other issues too.

Um, one of the interesting things that have emerged is the ability for these, um, uh, uh, for us to turn words into mathematical vectors and to then do math on them. And so here's an exercise that you can follow along with me on. Um, if I gave you this math equation of Tokyo minus Japan plus France, what does that equal?

What if you think about it, you'll see that it equals Paris king minus man, plus woman equals queen. Doctor minus man, plus woman equals, well, some of you may think nurse, others may say doctor or physician. Um, but the point here is that our biases are baked into these models, and then they're being reinforced, uh, when these models keep getting trained over and over with our biased inputs.

This is really our own, uh, biases. This is not a machine bias as much as it's how we reflect our own biases in the words that we use. And what you're seeing here on the right is, um, gender differences for certain wards.

is from that middle line, the more strongly is affiliated or associated with that particular gender. So you can see where nurses on the upper, on the top, it's pretty far away from the line, which means that, um, it's generally, uh, associated with a, with a female, whereas, uh, a word like captain is gonna be more generally associated with a male. And we all recognize that, um, this shouldn't be like we should say when doctor minus man plus woman.

It could be, it should be doctor as well, it should be like right in the middle, uh, because that should be something that's genderless. But something like the word nurse is somewhat challenging because what about nurse as a verb? Well, I don't know about you, but I don't know too many men that can nurse, like, in the way that a woman could.

And so the, the language that we use is so full of nuance that, uh, removing bias like, like this can be pretty hard. Um, and so that's, that's the real challenge for us. We're, we're, we're trying to raise this responsible ai, but we don't necessarily also have a clear universal definition of what we, what do we mean by responsible or safe or ethical?

And then even if we could define them right now, just recognize that it can change tomorrow. And, and one way to represent this is, uh, think about the, um, the story that everyone knows around the genie and, and, uh, the magic lamp. What, let's suppose you rob a genie, uh, rob a lamp, and you get a genie pop out and you get three, three, uh, wishes.

And everyone sees these large language models as a way to, um, to, um, to make lots of money. So we say, Hey, let's turn everything we touch in the gold. And of course, we quickly realized it's a curse.

So the genie pops up a second time and says, Hey, um, you need to gimme some guardrails. And so we say, okay, don't, uh, tell anyone how to hot wire cars. Don't tell anyone how to build napalm bombs and a whole bunch of other things.

And the genie, uh, grants our wish and great, but then we realize, oh, shoot, I forgot, uh, I don't want people to learn how to build biological weapons. Well, we with a third wish, um, what do we actually wish for? We can't keep adding to that list because it's an infinite list.

And ultimately we need to be able to have the system recognize what might be considered harmful and to be able to adjust, um, on its own. And that's really one of the big challenges that we're seeing in the near future to be able to raise, raise responsible ai, we need to align it with our value systems and to be able to give it, um, have it understand what our values are and what's responsible and so on and so forth. And there's a, a number of, uh, books that I can point to that, that talk about this problem.

Um, but, uh, that's, it's still a challenge and it's, it's pretty well manifested in the whole understanding of what's responsible is, uh, is difficult to cha uh, to, to sometimes decipher because we see problems like this. What, what clearly Google was trying to do the right thing by, um, uh, providing a way to represent people, diverse people in, you know, in different circumstances. But we clearly over rotate, uh, in this case where Google is starting to represent things that are just historically incorrect.

Um, or, uh, other cases where Google thinks that, uh, just because you're under 18, uh, I can't help you do certain things because it's for your own safety. So it, it, you can see how it can get absurd because, um, what one people, one person thinks is responsible may actually end up being, uh, wholly irresponsible to another. So anyway, that's our challenge.

And, um, uh, being able to raise responsible AI is something that, uh, I hope we understand and come to agreement on, because right now there isn't, and I'm not sure if there's ever gonna be a universal definition, but let's at least start the process of understanding what that might look like. And with that, thank you very much and, uh, hope you have a good day.