Bridging the Growing AI Gap Between Business and Consumers with Joe Bradley at AIE 2024

Hi everybody, and, uh, thank you for the opportunity to speak here today, and thanks for your attention. My name's Joe Bradley. I'm live person's chief data scientist and the head of our AI and machine learning products as well.

Um, I've got an interesting talk for you here today, I hope, uh, we're looking at going from kind of like hope to reality in a way about generative ai. Um, so how do we get the, the basic tenant of what we're trying to accomplish here is talk about how do we get from, you know, the promise of generative AI into something that gets into business' hands and in consumer hands in a way that feels great to everyone and solves the business problems that you wanna solve. Um, so to help us understand and kind of catalyze that discussion, we're gonna, we're gonna dive into in just a minute, uh, some research that LivePerson has done, uh, where we've surveyed about 2000 people, 500 of them, uh, business executives, and about 1500 of them consumers to try and understand, you know, how people on both sides of the enterprise equation are thinking about AI technology today as the landscape is changing so quickly.

Uh, but before we do that, just in case you don't know who LivePerson is or what we do, 'cause we are a B2B company, so not everybody knows. Uh, we are one of the leading, if not the leading messaging, messaging and digital conversation provider in the world. Um, so some, some quick stats here just to, just to kind of show you that we work with the world's biggest brands, uh, in a broad range of marketplace verticals.

Um, and we do that not only, you know, allowing human agent to human consumer conversations, but we also do that, uh, with conversational ai. Um, and that part of our business has been growing more and more, uh, every year since we rolled this out in around 2018. Um, you know, something like 70% of the conversations on the live person platform, and there's over half a billion of those every year, uh, are touched by a chat bot or conversational AI system.

Uh, and that slice of the pie that's really only a human to human conversation with, whether it's customer service or sales, uh, both of which we support other things as well, uh, is growing smaller and smaller every year. Um, so I, in, you know, when I talk to people aren't familiar in the B2B world, sometimes, uh, the quip I'll make is we're the biggest conversational AI platform you've never heard of. Um, since we have some business leaders on here, I I'm betting more of you have heard of us than the, you know, typical consumer sets.

But, uh, that's who we are. That's what we do. And, and increasingly we've been building more and more generative AI systems with these big brands, uh, and driving more and more conversations into this new technology.

Um, so excited to talk about that today. All right, let's get into it. So we all kind of know, I think that AI is a pretty big deal right now.

There's a, you know, the, the language of inflection point has been used, things, things really changed a couple of years ago, about 18 months ago, when, when the world became aware that these large language models, um, had the efficacy that they did or that they do, uh, that creates a an interesting business problem for all of us. And, and there's kind of like different scenarios for what could happen in, you know, as we, as we move into this technology, you know, it could look, uh, as quick and easy as the maybe the migration into, um, you know, digital travel booking and away from, uh, travel agents, if anybody remembers that period of time online. Or it could be maybe as painful and strife ridden, you know, as, uh, as cable to streaming, which is a transition we're all still kind of going through.

So, so there's, I guess the point is there's no real, um, guarantee that with great technology, it will easily get into consumer hands and easily be made valuable, uh, eventually I think the tides of history take you there. But, uh, for, you know, in the short term, how, how these transition goes, you know, can vary quite a lot. So what we've done is we've tried to understand how customers are thinking, and we try to understand how businesses are thinking.

We've tried to put it all together for you. Um, so as part of our survey, we're really looking at how sentiment's evolved on the business side and on the consumer side. And we've seen some relatively surprising results, perhaps unsurprisingly, about a year ago, early 2023, consumer sentiment towards AI was about as high as it's ever been.

Um, 62% of consumers feeling positive about it then, um, that's really dropped in the last year, uh, considerably down to about half of consumers feeling positive about engaging with ai, particularly in a business context. I mean, so there's a lot of excitement still in the consumer space about, you know, them, them using AI for their own purposes, uh, but some trepidation that's creeping in about, uh, it being a positive experience part of the business landscape. Um, that said, on the other side of the equation, business leaders are as bullish as ever, uh, 91% of business leaders build positive about using this technology to engage with their customers.

Um, and that, that probably is warranted on the long tide of history. The question is, again, can we get there? Can you get there in a way, um, that's gonna feel great to your customers, solve their problems and save or make you money?

Uh, so that's the gap as we talk about it here. There's just, you know, there's lots of consumers that are excited, but there's, there's still some real concern in the market, uh, and or rather on the consumer side. And as you can see, it's pretty stratified by, you know, age as you would expect.

Uh, so here's just a comparison between younger and older consumers. You can see a pretty radical disparity in terms of how they're, how they felt over the last year. Um, the, the, the recent events have brought more comfort to the young and have brought less comfort to older consumers.

Uh, and this is, you know, not inconsistent with general technology adoption, uh, but it's probably particularly poignant with AI where people, you know, really don't have a broad range of understanding of what it really is and what it really does, what it can, and what it can't do. The dangers it does pose today versus the dangers. It, it doesn't and may pose or may pose, you know, near or far in the future.

Uh, that's probably something that's harder for your older customers to have as strong a grip on, just on average. Um, and so as you're engaging with your customers, this is something very important for you to be aware of anyway. Let's talk about the details, though.

What is it that they, that consumers want with this technology versus what is it that they're concerned about? Um, and, and I think the answers are, are a little bit unsurprising here, but it's always good to know what we've got our hands on and, and make sure, you know, usually the right answer is unsurprising in retrospect, but sometimes you couldn't sit down and write down all the bullets, uh, before you start. But what we found is, is I think clear, um, the ease of use and the convenience and the responsiveness of these systems is a big value to customers.

They want a seamless, easy, good experience. Uh, the challenges that they're worried about, or safety, you know, is my data gonna be used in a way that I don't expect? Is that, you know, AI gonna track things about me, I don't want it to track.

Is it gonna be wrong? Um, is it gonna hallucinate answers and, and confuse me? And we'll see that that does really happen.

Um, and does it feel wooden or mechanical or robotic, uh, as an experience? Is this a good experience for me? Are the, the big questions here?

Um, so this is what you've gotta kind of prepare and frame your consumers for, and these are the problems you have to solve when you use this technology. Uh, anyway, that's kinda the problem space. Let's talk about the, the return on investment that's available here and, and what we kind of know about that, or learned about that.

Um, and as a way of understanding that, if you want to get the most out of any technology, AI or not, or any customer connection, AI or not, um, one thing is resoundingly clear now, um, your customers are hoping and expecting that you're gonna greet them on a range of channels, and the digital messaging channels are not optional anymore. Uh, a lot of brands, I think, still think that, well, if I've just got a great contact center and, you know, if I kind of manage that, okay, um, that's gonna be enough for my customers, and it, it just is, is no longer true. Consumers is, are speaking loudly and clearly that they want digital channels as part of a standard, uh, customer experience for them.

Uh, they also want something that is very difficult for a lot of brands to provide, and I'll talk more about this in a little bit. They want truly connected interactions. Uh, they want what you and I want, which is when you are talking to a brand and you share some information with them, or you share a problem with them, that the next time you talk to that brand on whatever channel it is, that they remember the problem you had and, and they respond accordingly.

So if we still haven't solved your connection problem, you know, please don't try and sell me a new phone right now. Right? If we, if we did a great job, uh, you know, managing your order and we got something sp sped up for you on time, uh, and, and got it to you when you needed it, you know, maybe that's a great time to deepen your relationship with a customer.

So on the customer side, you know, or rather on the business side, there's two benefits here. Um, increasing loyalty when customers feel like they're treated well. And then actually growth too.

When, when customers feel well treated in a customer service context, when they feel like a business is listening and connected to them and having an ongoing conversation with them, they want to grow their relationship with that business. That's the secret of many e-commerce businesses, uh, through the two thousands, in fact. Uh, but then there's kind of like a, a step up from that, what customers want.

And what we, again, as business leaders want as well, is real great personalization. Uh, personalization is a buzzword that's thrown around a lot, and it means anything to people from, you know, targeted shopping recommendations on an e-commerce website to, um, you know, all the way out to a, a personalized and fully connected conversation where, you know, I remember deeper things about you as a brand that can help you solve bigger problems. And the, the sort of further end of that spectrum around, you know, targeted personalized product rep recommendations around web cookies, you know, using web cookies to get all that and, and targeted ads and things.

That level of personalization, uh, is really something that consumers, you know, have never felt wholly at peace with. Um, and today feels like it's, uh, falling far short of the real opportunity in front of us. There's always an inherent little bit of achiness about being tracked around the internet and having ads and things displayed, um, you know, for you based on your browsing behavior and non-consensual information, you know, sort of usage.

And the really nice thing about the conversational paradigm is that there's a, there's a very clear kind of inherent reciprocity and, and consensuality to, uh, talking about your needs. So I see a lot of brand conversations every day as part of my job, as I work with our brands. And, you know, it's very common to see consumers coming in, in a conversational context and, and starting with, Hey, I'm getting in shape for my wedding, therefore, I, I'm working on, you know, I need to find new running shoes.

I wanna find a running club, right? They start with a problem they have in a conversational context in a way that you don't online, in a way that cookie doesn't capture. Uh, and so the, the conversational setup now and what's possible with AI to scale, that creates a radically new and better form of personalization as an opportunity for the enterprise.

Um, so that's something where there's gonna be a real return on investment here, and consumers are loud and clear that they want that. Uh, okay, let's, uh, let's talk about what the problems might be or what issues you, you need to, um, you need to solve in order to make this work. And one of them is, uh, you, you really, customers don't a lot of patience right now.

Um, if you put a customer on hold for 10 minutes, you, they are just, they're just about equally likely to go and, um, you know, find you on another channel. And also even more, which, which make you even more nervous, is that they're thinking about a competitor. Uh, it is a huge loyalty killer to have a bad, uh, experience when you have a problem you're trying to solve with a brand.

And it's particularly difficult to be stuck on hold waiting. Nobody really uses the phone in a personal context in the way that we used to 20 years ago. Uh, and so it always, it, it consistently feels like a holdover to consumers when they have to sit and wait on a phone hold, uh, with a company that they want to work with.

So be careful with that. Uh, and in general, consumers demands for, uh, brands and the expectation of the consumer experience that they want, how much they expect the brands to come to them, uh, and how easy they expect those experiences to be, to manage for them are just growing higher and higher and higher every year. So the bar is going up again, set, really starting that bar was starting to be set up in the age of e-commerce and, and just continues to grow.

So your, your experience, your consumers expect you to provide great experiences for them. They expect you to do it across multiple channels. That's, that's really kind of table stakes.

And then on top of that, you can personalize and you can grow, uh, even more. Uh, what do consumers think that AI technology can do to help them? Um, again, a lot around making my life easier, simpler, making the experience like less painful to deal with.

Uh, and, and that matches here with what you're looking at around business, what business leaders are saying as well. They believe that this technology can provide faster answers, better information can reduce wait times, you know, and can create, can provide some of the consistency across the consumer experience for them. Um, so that stuff's pretty well right on.

It needs to be managed. You need to deploy this technology well to do it, but that matches quite well with what consumers want. That said, there's some risk as we use ai.

Uh, those of you who are paying attention to the media may have seen, uh, a recent airline trouble where a bereaved passenger, uh, was quoted a fair by an airline, and the airline was, uh, forced in small claims court to uphold the fair given by the chatbot that the airline employed to do this, which was a generative AI based chat bot. Uh, this is a, a very interesting precedent, and if you actually read the ruling, there's an important fact that comes out that you all should be aware of. The reason that the brand in question was forced to hold to the chatbot's fair quote was not simply because the chat bot quoted it, it was because the chatbot quoted it, and the brand was unable to provide, to show, uh, sufficient controls on the system to mitigate problems like that.

So what regulators and courts, and governmental bodies will be looking for in the coming months, we believe, uh, in coming years, is not perfection from this technology exactly, but it is controls and management capabilities that brands can demonstrate so that it can be seen and it can be clear that they're trying to mitigate and solve for mistakes like this, you know, and measurements of how likely mistakes like this are. Um, so an important thing to be a little bit nervous about and also a little bit thoughtful about as you, as you work with generative AI partners, are they providing me the tools that I need to mitigate hallucinations? Are they allowing me to see the level of hallucinations in my platform?

Are they allowing me to look for biased responses? And, and, and then do, do I have a lever or a dial I can turn to handle that? There is one thing I think that has very little contention that came out of our survey, uh, which is pretty much everyone agrees you should not claim that your AI is a human.

This is something that some brands have done over the last 10 to 15 years, uh, and it's, there's still some brands out there I think that, that occasionally think this is a, a good idea, it'll be simpler and, you know, make their customers feel great. They have a human that can talk to them. Uh, you're running a foul.

If you do this as a brand, you're running a foul of the EEU AI act, and, and you'll probably run a foul of other, uh, global regulations as well over time. This is this, this seems to be centralizing as a tenant, uh, around how, how AI should be handled. So our strong, my strong recommendation is do not do this.

Always have your ai, um, self-identify, make, make it clear to your customer what kinda experience they're having. That's how we're all gonna get through this and, and find great experiences. Anyway, Uh, I guess one quick last point before we kind of, kind of back up and look at some takeaways is, you know, our, our early slides are really about consumer feeling about ai.

And so you might have a quibble that, hey, well, they probably, maybe they feel weird about ai, but they feel great about generative ai. Um, and the, uh, the jury really comes back with the same response here. There's a, you know, about a 50 50 split amongst consumers that are excited about generative AI for the customer service space in particular.

Um, and almost all business leaders are so, so the gaps a little bit pervasive here. Uh, and it's, it's not that this space or this type of technology, you know, suddenly makes people feel different. They have the same kind of ness, um, here and with this tag.

And, and I think, uh, you know, I, I guess one last point, uh, consumers wanna have a little bit of security around what the AI is doing for them and how it's being managed on your side. So go back to the Air Canada example from before. Um, they want to know that there's some supervision for these systems.

They want to know that, uh, these systems have ways where you're trying to limit bias. They wanna know your responsible technology users. They're less concerned about the emotional, emotional connection when you ask them, though, we do see with chatbots that are live now, um, a strong interest or, or a strong response to the, you know, the natural empathy and, and sort of humanity of the conversations that these systems create.

Um, so, so that's kind of an added benefit. But, but on the face of it, consumers really wanna make sure that you've got your hands around this technology in a way that is keeping them safe. Okay, so let's talk about some high level takeaways.

When, when we read this information, we think about what's going on in the marketplace today. I, I think in a nutshell, we see an opportunity here, uh, that there we, we could be entering into and, and kind of are entering into the age of digital conversation for enterprise. Uh, I think, I think, you know, very clearly make the argument that we've entered into the age of digital conversation for, you know, uh, private folks and for people in general, for the public, uh, with each other, you know, really over the last 10 to 15 years with the advent of social media, um, all the chat apps and, and now the AI chat apps as well.

Uh, but what's really coming for all of us in the enterprise now is an opportunity to manifest that in the commercial space with the, you know, at the scale of the earth. To put it bluntly, uh, consumers want a connected conversation. Uh, they want personalization, they want the ease of this experience.

Uh, they want all of that as a primary want. Uh, they want businesses that feel that way and work that way to talk to. And, and I think importantly, the other takeaway is it's not exactly about the AI itself.

Consumers are, are interested in AI because it can talk to them. They're not interested in conversations because there's ai, right? Like the primary element here is that they want to have that interaction.

They want to be able to solve problems with brands. They wanna be able to, to solve their problems in life with, you know, purchasing and growing their relationship with brands. They wanna do all that conversationally, easily on the same digital channels that they use every day.

And they want AI to help them, because AI can help them with that, not just inherently because they think AI is cool, though. AI is pretty cool. Um, so, so that's kinda what we mean here when we get into not quite the ai, it's really can you get the conversation right as a business?

Uh, and doing that is non-trivial, like getting the great connected, meaningful, personalized conversations that really solve problems with your customers. Not easy to do. You've gotta connect a host of systems on one side that contain all the data, some of which are highly siloed, talk to airlines all the time.

Some of them literally don't have CRMs. Uh, they have a bunch of systems that are pulled as needed and, and don't have a common connection point. And it's not just airlines.

There are other, other brands that are still like that, but even brands that do have big CRMs, the integration of all that data, the connections between it, the availability of it in a conversational context, and then the ability to push that out through all of these channels, uh, is, is the way you make this work. And that's a non-trivial real, um, you know, engineering and product task for your company to take on. AI helps in a lot of ways, but it doesn't take that problem away.

It can help you solve it, you know, in some ways more easily. Uh, there's kind of anatomy of the digital conversation or the conversational enterprise here, and all the things you need to get, you know, get this right. Uh, and then the benefits if you do, uh, you know, we all, we all talk about AI chatbots, and we know that's a, a big piece of this puzzle, but, um, you know, you've really gotta, you really gotta think about the humans in this equation and how you can bring the data to help all the people involved.

You have human agents who might need to help them. You have human customers on the other end, who, who, who need help. And then you got, you know, systems and data to bring to bear on this.

So, um, so fundamentally that's the problem. It's, it's, it's making, you know, all these connection points work. And again, AI is an enablement technology to help you do all that.

Okay? But, but then when we put large language models and generative AI and all of this, you know, I think we all see the promise here. Um, and, uh, I hope you enjoy the firework gif in front of you, and I hope you can see it in the recording, I guess.

I don't know, but hopefully it'll come through. So I think we see the promise, uh, and in the contact center in the CX space, um, it kind of falls into three main categories. First of all, you've got, um, the, what we, what we most, I think mostly go to first when we think about the promise of gen ai.

You can, you can be talking to the computer and it can help you solve your problems. It can help route you to the right person. They can answer questions that you have.

Um, and as a business, you can use it to warm customers up, gather leads. Talk about an example that we've done with that in a minute. Um, but also you can make your workforce kind of superhuman, right?

The, I I think it's important here that you look at your contact center employees in the same way that you look at probably many of your employees, your developers are using GitHub copilot, right? You wanna make sure that you've got the right tools for them to do their job better. And Jenny and I can help them as an assistant for them.

Um, and then finally, uh, as the world moves more and more into conversations in the enterprise, it's more and more important that you understand what's going on in those conversations and can aggregate across them and can, um, and can derive insights from them, both in terms of how your operations are working, but also in terms of what your customers are saying about your product and their needs. Uh, and so conversational intelligence and a way to really make it radically easier to analyze all that unstructured text and make sense out of it. Um, really important piece of the puzzle for, from an analytics perspective.

Also, from a real time interrupt perspective, I've got a problem, you know, conversation going off the rails. Can I stop that in time? Uh, and then I guess to just make maybe one more point here before, um, before moving on to how do you actually do some of this stuff?

Uh, you know, this is, this is what we see in the marketplace as far as general trends. Um, people want to drive operational efficiency. They wanna make a better customer experience.

They wanna do it, they wanna bring value into the equation much faster. Um, so there's, there's a lot of, of, uh, opportunity here that everyone kind of grows. I think these are all right.

Um, this is, this is sort of what AI offers now. Uh, but then there's also some nerves around this and, and some concern around this. And those nerves and concern are well founded.

Uh, the opportunity comes with risk, as we've discussed, hallucination, data privacy problems, uh, bias that's harmful and unintended. Is this technology enterprise ready? Are you bothering your customers with it?

Does it feel good to them? Could create reputational damage if things go wrong. Uh, that's the risk on the other side of the opportunity that we've been discussing here, and there's a whole new set of problems.

This, this slide a bit on eye chart. I don't wanna, you know, won't go through all of it, but there's a whole new type of problem with generative AI that we've never seen before. Model hallucination, we've talked about consumer behavior is different too.

People are trying to jailbreak these models and get them to say things they're not supposed to. There's an opportunity if a chatbot has the capability to take action on your behalf, say with your bank or with your credit card company, or with your, you know, medical information, that, that, that action can be abused, intentionally abused by customers. They can try and hack the bot to do things that it shouldn't be able to do.

Um, and so let's talk about how do you get by those problems and how do you actually deploy this stuff successfully? 'cause we've done quite a few of these. Now, um, at a high level, you really need to think about your control mechanisms that you have, and you need to use all these different control mechanisms to control these systems.

We talk a lot about LLM training and fine tuning, but that's, that's the tip of the iceberg here. It's in some ways the weakest form of control. You should start from the bottom.

Do I have the application and network controls that I need to make sure my gen AI chat bot or system can't take the actions just never could take the actions that I don't want it to take? Like, uh, you know, moving bank balance, or sorry, moving money from one account to another when the authenticated user on the other side is not in either one of those accounts, right? Let's make that impossible at a network level.

Do I have sensors, interrupts and response mechanisms that can pick up the ball when the gen AI seems to be dropping it? Um, do I have good, do have good ways to sense that with machine learn models, with whatever else I use? And do I have good mechanisms to articulate that?

Then you get to these tip of the iceberg tuning techniques that most people think about when they think about making gen AI safer, tuning up the prompt and fine tuning the model. Those are very valuable and very important to do, and they do provide a lot of that last mile of a great customer experience, particularly prompt engineering. Um, and LM training for targeted use cases can work well.

Uh, but they are looser, weaker, less, uh, firm controls, and you should treat them as such. And, and you should hierarchize your risks, let's say, uh, against a framework like this. And make sure you're solving the right level of problem with the right level of control to actually build an architecture that makes all this possible.

Uh, this isn't it, right? This is maybe what a lot of us thought the future would look like when LMS first came to light. We're just gonna talk to the model and it was gonna be so smart, and it wait, its weights is gonna answer everything for us truthfully and factually.

Not only is that not realistic, but it's, it's an uncontrolled system. You're just talking to a model waits. They, they may be trained three years ago, they may not be relevant for what you're trying to talk about today.

And, and, and there's no way to exert control around that. So pretty quickly the world glommed on to retrieval augmented generation, which is a good start. Uh, a way to, uh, retrieve, you know, this is a way, a way to prompt models where you retrieve information, you put that information into the prompt based on whatever the customer query is, and then, uh, you, you hope that the prompt having the right information in it will yield a truthful and factual answer for your customer.

Sometimes it does, sometimes it doesn't. It's a lot better than the last model, but still, uh, not perfect today. Some more realistic enterprise architecture for that same use case Q and a, uh, you know, bring in more types of data to prompt the models, have separate models looking for hallucinations, uh, have protections on the system looking for prompt abuse.

Understand whether or not the customer's problem is being resolved. Much of this can be done with LLMs, uh, but they can be done. But oftentimes it's beneficial to do separate pieces of this puzzle, uh, separately.

So rather than depend on one large language model, main reason being I can control those separate systems, I can measure those separate systems, I can tell you how accurate they are. I can provide the level of oversight clarity that's needed for a case like the Air Canada case where with bereavement, so I can show you, yes, this did happen, but it only happens one 10th of 1% of the time, and then I can go win that court case. Going beyond retrieval generation, though, the way that we look at this problem at LivePerson is we look at this as a problem where, uh, you've got, you know, specific AI agents, and we use this term a little differently here than is used elsewhere that are solving for specific types of use cases.

So the way we work this problem is we, we tune up and, and, and set up, uh, a smart agent that can solve specific business use cases. Then we weave those agents together. So then we're not asking the generative AI to context switch too broadly.

Uh, we're able to control what information goes into those prompts. We're able to tightly constrain what those models are allowed to do, and then we can orchestrate that, uh, you know, those constrained systems together through standard, um, or, or sort of let's say, uh, ex preexisting conversational AI technology and orchestration layers. This works, we think this works really well because it gives you a, a finer degree of control as an enterprise and allows you to, um, make sure that the each piece of the experience is working the way that you want it to and tune them up independently from one another, uh, rather than having them interfere.

Uh, we have some examples that we're excited to talk about, so I'll briefly mention them just to let you guys know, um, what's possible here. So, a very simple example with one of the world's largest professional communities, uh, PMI, uh, simply adding, taking a traditional chat bot and adding an NLU layer that's done with LLMs, uh, drops unmatched phrases dramatically. So, so we're not really doing a gen AI chat bot here for this brand in a way that it's, it's having that much of an interaction with the customer.

It's asking some disambiguating questions, but it's kind of sitting within an existing conversational AI flow. Uh, but it's really doing a, a quick job of radically improving that experience and improving the NLE with very little effort. And then finally, one of my favorites, uh, is Open Universities Australia, which is a, a large educational, uh, provider in, in Australia, but isn't necessarily known that broadly throughout the world.

Uh, they did some work with students or prospective students who are trying to learn about higher ed on their website and, and learn what's right for them. Can they go into a, a, you know, a higher education program? Do they have enough time in their lives to do it?

What would they study? And this is a, a, a discussion and a product guidance chat bot for them. For the consumer side, it's a lead qualification chat bot from OU a's side.

Uh, and it's, it serves really well for everyone. There's a lot of reciprocity in this experience. People come to these conversations with lots of questions about what's possible in higher ed.

They don't know what they don't know, and so they're kind of nervous to answer or to ask. I mean, and those nerves are really allayed nicely by the chat bot. The chat bot answers quickly.

It has a friendly tone. It feels like it understood. It does in fact seem to understand what they're asking and what they need.

Uh, and it, and it essentially gets them into a state that when they then say yes to having a conversation with a human about setting up, um, a, uh, you know, a higher education program together, the lead qualification, uh, rate is radically improved, right? So, you know, more than doubled compared to just starting the conversation with a human alone and, and, uh, more than tripled compared, oh, sorry, more than tripled compared to just starting a conversation with a human cold alone. And more than double compared to starting with a, you know, sort of legacy standard AI chat bott, uh, this is something, there's a lot more examples I could talk about.

I, I wish I had a chance to, but, but just to put it in the air. So you guys, as businesses understand, this is really the year that this technology is starting to come to life for major brands. And we've seen a bunch, this, this is, I I have a bunch of other examples that are not even on here, but we've seen in the last four or five months, uh, one after the other, after the other of these major companies, uh, begin to turn this technology on and begin to use it with their customers.

So I think in 2023, there was a lot of concern, or, or there were a lot of big questions around, is this technology usable? Is it gonna be five years before anybody really puts it in front of the customer? And, and there was probably some, a lot of toe dipping in the water.

Uh, what I would say is 2024 is a time where people are getting in the water. Uh, we've seen that change pretty dramatically in the last few months, and we see it, you know, continuing to surge as the year goes on. We're not, we're able to control these systems well enough that they can be great experiences for customers, and we're able to put the, put the guardrails around them and the framing around them that consumers understand what they can do to avoid the existential problems.

And, and, and we have the platform to do it. And there are, you know, um, potentially others out there as well. Uh, and that's making us see now this promise begin to truly come to life.

And, uh, I think that's probably as good a place as any to leave it. So, uh, we'll be taking questions after this, I believe. And I guess, uh, thank you so much for your time today, everybody.

I hope that was helpful and, uh, uh, look forward to hearing from you if you have questions or, or any follow-ups.