Implementing AI Technology – Brian Land, Lucidworks

This is Textron tv. Hello, I'm Amanda Ani and I'm excited to be here today with Brian Land. He is the Vice President of Sales Engineering for Lucid Works.

How are you? Great, Amanda. Great.

Thanks for having me. Well, can you explain to our audience what is lucidworks and what services do you provide? Sure.

Yeah. Here at lucidworks we essentially help the search and discovery part of the journey. Um, and you can think about that as three different markets, customer experience, customer service, and employee experience.

So, um, on the customer experience, think about commerce, Lululemon, helping customers find things faster, using machine learning, customer service, helping customers, you know, solve their problem, get their question asked faster as well. So delight the customers and also help deflect those incoming chats or cases into the call center. And finally, employee experience, making employees more productive.

You know, a lot of, uh, big organizations have silos of data, sometimes petabytes, so they have a hard time finding documents and information, so making them more productive and finding the that information faster. Wonderful. So we're gonna talk a little bit about implementing technology, especially AI and business today.

So what are some of the challenges faced by businesses when it comes to trying to leverage ai? Sure. I think, um, you know, a lot of our customers, especially commerce on that customer experience, they have been experimenting with ai, right?

Google, Amazon, and Apple. They were early to market. Now most retailers have some type of AI on their website, so finding things faster, even at the type ahead.

So I'm aggregating user behavior and clicks. And so, um, I see the biggest challenge more on the employee experience. So some customers, big enterprises are still having challenges of enabling ai, consolidating their data.

But I will say a current challenge over the last couple of months is how do we, they're all asking about across all three of those markets about generative ai. You know, how do I leverage generative AI and large language models, you know, for the customers, customer service employees and all of that. So, do you have some solutions and can you give some use case examples of implementing generative AI into companies?

Sure. We do have examples. So I lead our solutions engineering team.

We do demos, POCs, and so we talked to customers and prospects a lot and partners. And so that's what we've been hearing over the past six months. I would say six months was like once a week, and then it was like every day now.

Now it's like every day, every meeting customers are asking about this. And so our, our product fusion, it's always been open platform. So you can roll your own machine learning model.

So Advanced Enterprises, they may have a data science team, all right, I wanna roll my own machine learning model. But now with large language models and generative ai, I mean, it's amazing what's out there, right? With uh, open ai, Chachi, BT, and Bard now with Palm two that came out from Google and, uh, and yeah.

And so, so Fusion is very, uh, you know, it's a perfect solution for that since, so to roll into Chachi, bt in fact, I get these questions over the past months and weeks, you know, it was, uh, hey, my C-suites asking why do we need a search vendor when there's Chachi, BT I can get an answer. So, but, but, but there were challenges around that, right? You don't now fast forward, you know, some of these enterprises are locking down Chachi, bt they don't want to put their enterprise data in the consumer app.

And so that's where Fusion comes on. We were an enterprise platform. We provide guardrails and, and control around their data and the, the input and the output.

So I think that's the biggest benefit is like you need those really guardrails and control and still get the full value of these large language models. There's so much can be done. We're still learning stuff every week when we can do these p especially my team and the POCs, cuz customers ask, this is what I want to do.

And they're like, oh, well this is, this is a way to do it. So we solve a problem and show it to 'em in a poc. Now we'll have pilots are running on websites, so pretty interesting, pretty fun days actually.

I've been a longtime search engineer for a very long time. So it's, it seems like a, like a new revolution over the past few months. So do you think that this is gonna be a required technology that businesses will need to harness or is it not always a good solution for every company?

For example, do they need to look at the cost benefit and how, how do they weigh if it's beneficial and if they're seeing success by implementing this technology? Yes, co Well, cost is a factor. So, and I see our cost consumption on open ai, it gets very expensive very quickly.

And, uh, there's a few ways of controlling costs. One is instead of executing a query, uh, uh, an API hit to OpenAI for every single query, think about all the employees, all the consumer shopping on these sites, that can rack up very expensive, but there's a lot you can do at behind the scenes at index time. You know, what are the top dresses in summer, you know, that, that have flora in it, and then, uh, floral prints.

And so you can get that information from a large language model, store it in a product like fusion or an index, and use it for future use. So it's a, it's a one-time hit. And we have a great wine demo as well.

So our, our merchandisers, you know, you have merchandisers at these commerce sites and they're doing things like, all right, what wine pairs well, or what food pairs well, with this wine, it's a one-time, that's a prompt, really think about these CHATT CHIP D prompts. And so that's a prompt you put in our engine, and then it populates the data one time, and then customers can see that output, that really valuable data, you know, millions of times. Um, yeah.

And so, uh, it, so it's not really, it's not one size fits all. It's like all the different use cases. What do you wanna prove?

Um, you know, some of the other questions I heard from, from customers and prospects who are like, I, I don't want you to show my competitors' data on the output, because you can get that in chat g bt like, what are the top, you know, what are the top cruises? What are the flight destinations, what are the hotels? And so being able to control, that's also, you know, you can do a lot of configuration with the prompting of a large language model.

And so you have these series of prompts, and then you get the data and you refine it, you control it, enrich it, and then present it back to the user. So that's what I mean by the guardrails and control. Same thing with employee experience.

One of the questions I, I got, uh, probably about a month ago, I was like, I don't want, uh, an employee to ask who's getting rift next week, right? So it's, uh, so, um, you know, you, you don't want to, and you don't want to push all of your documents to a large language model. All of your data, of course, uh, be very sensitive to that.

So, um, we are guiding our customers to this, you know, some of the data that can be pushed to a large language model, enriched and extracted, and then what cannot, but there's also level of what we call security training. So there's a sensitive document that employee a may not be able to see, filter that out, so they never see it. So we're having to index not just the data, but also employee document entitlements and things like that, the role in the organization, geo location.

So, you know, without guardrails and control, you really can't do that. So it's, it's another, another great thing we're working with customers. Wonderful.

So you address the security issue, but how do companies ensure the quality of the data being generated? Because I know that there are some concerns about the hallucinations and, um, bias and things like this. So how do they ensure quality in their implementation?

Right, right, absolutely. So the quality is a big thing. Hallucinations is a big thing.

And uh, with, with a lot of search engines, you can do what's called ground, uh, grounding in the truth. And so, you know, I've been a search engineer nerd for a long time, and, um, so you can enter a search query, this is the old days before large language models get a set of results. And that is, you know, it's always scored and parameterized.

So it's like, this is good quality relevancy and it is relevant to what the user's searching on. So what we're finding, this is, again, like I said, like every week we're finding something new. My engineers in the field, so, you know, in the past, uh, we were, we were sending prompts to GPT four, getting results, enriching it, and then, uh, a query to the index.

That's the ground, you know, it's like grounding at the end. But now we're able, we're showing where we can do truth grounding at the very top of the pipeline query pipeline. It's a technical term, but ground the truth in the results and then constrain the large language model to that.

And then you can even do stuff after that. You can enrich the data, add new metadata, user intents and things like that. It's pretty, pretty interesting.

Awesome. And I wanted to rewind a little bit. You talked about the staff and employee experience.

Um, how do companies handle when they're trying to implement this technology, how do they handle kind of the buck back from employees who are not happy with the technology or not comfortable with it, or not seeing the benefits? Well, it seems like the other way around, you know, well, since I've been working here at Loose Works, it was, it was, uh, hey, I want the Google experience because I'm trying to find an internal document and I have to go to Salesforce, confluence, windows Share, SharePoint, I can't find it. So they're always looking for that Google experience and even Siri, Alexa experience.

I like, I want to ask a question and get it in the first response. Um, so there's been a big push for that o over the years, and, um, you know, those technologies like Siri and Alexa, they are neural networks, very large. You know, they've been trained and trained, and so we've already been working with, uh, with that for about three years.

We have our own solution around that. But you know, for example, the power chatbot and, uh, now added a large language model on top, like chat GPT or GPT four, it, it, even the, the, the employees trust the data even more now. So it, it is always been, uh, you know, we're rioting, we need faster access to documents and data, and now they see it can be done.

So it's, uh, we're kind of seeing that, that kind of trend. So it's mostly positive feedback and, um, really helping with efficiency and the searches. Right?

But, and it, but you know, the first challenge was consolidating all that data from the silos. That's been a long, um, age old problem in search technologies. So I have 12 data sources, like I mentioned, confluence and Salesforce, Microsoft, all that.

So, you know, we have connectors and just the data, you know, so get it all into one index to make it findable across, you know, with a single search box or something like that. But then once it's in there, add the machine learning, the neural network, now the larger language bot on top to enrich it and even power a chat bot. So, you know, you have employees ask things from a chat bot and, uh, and using their data, get those ask in a natural language type of question, get really relevant results.

So we're, we're seeing a lot of positive, uh, results there. Wonderful. So looking toward the future and this technology is advancing so quickly, so when I say future, I mean a lot could happen in the next five years, but where do you see this technology going five years from now?

Yeah, five years. I mean, there's, uh, it, it, you know, I've used these, these new models now, these large language models, there's automation on top of it, so it's model on top of model, it's, it's improving itself. So that's where I'm seeing that level of automation.

I mean, I think Morgan Stanley, I believe said, you know, 20% of jobs may be impacted within five years in different sectors. And, um, you know, if, if you have models acting upon models in this automation, just keep growing and growing and doing our jobs faster and, and, and easier and more automated fashion. I mean, that's where I kind of see it, you know, right now it's a big, big model, large language model.

We're asking natural language questions, we're refining our prompts, and now we're getting better at asking two or three or five level prompts. But I'm still a human asking that. And, uh, and now we're mo we're, we're moving that methodology to our query pipeline so that now the pipeline takes care of it.

But still, you know, I define those prompts or questions in a pipeline. That's step two. Now, you know, in three to five years, you're not gonna need a human or engineer or Right.

Team. This is gonna keep improving, improving over time, all automated. So I know.

Um, so that brings up one more question then that I have, is there is a concern about this loss of jobs, but do you think that, um, likewise there's gonna be some jobs gained in other areas, and can you go into that a little bit? Sure. I think so.

I think everyone's, every enterprise is gonna be more efficient and, you know, with efficiency comes, uh, you know, uh, more equity and so to grow the company. So, you know, it kind of floats all boats. That's my perception.

That's my, um, you know, that, that, you know, that's my opinion is that, you know, all these companies are gonna be a lot more efficient and they may have to constrain certain departments. I'm not sure if it's HR, marketing or what, and that'll all come out in play, but they're gonna keep growing and grow in other areas. So it's, I I think it's gonna be good for the economy as a whole.

Well, thank you, Brian, for coming on today and sharing your insights about this technology that is just making waves across the globe, I feel like, and look forward to speaking with you again in the future. Awesome. Thank you, Amanda.

Thanks for having me. It was a great conversation. Thank you.