AI and Automated Workflows with Jeremy Barnes at ServiceNow Knowledge 2024

This is Textron tv. Hello and welcome to the Knowledge 24 Conference hosted by ServiceNow. This is the first in a series of interviews we did at the show, and our first person here is Jeremy Barnes, who's vice president of Product AI for the Now Platform at ServiceNow.

And we're gonna be talking about it initially at least, where what happened in the last year, it seems like LLMs went from, my God, they're gonna transform the world. So now it almost feels like they're commodities. I can bring my own in certain announcements that here at the show, there's multiple ones that you guys are working with.

It seems like each one might over specialize at times. So how it's been a year since we last chatted, look back and say, how did we get here? And where are we, Well, where we are.

Is it a place where it's no longer a brand new technology, but it's something where people are using it for, you know, for real outcomes. And so like any technology, there's a lot of excitement at the beginning. And then after that, there's a lot of hard work.

And I'd say we are a lot further through the hard work part than we were when we began. And we've also discovered, as people get to know the technology, their ambitions are increasing and there's more stuff they want to do. So the one size all fits all model doesn't work anymore.

And so we are meeting our customers where they are saying, we wanna do these extra things that we hadn't thought of at the beginning. And that's why our strategy is expanding too. As we operationalize ai, it seems to me that we've gone from thinking about all these techniques that would be required to expose our data to the LLM and how we were gonna manage that.

And there was a lot of manual thoughts around that. Now it seems like the whole thing is becoming much more automated. And so how do we manage it?

The data that we have in the enterprise versus what's in the LLM, we hear more about knowledge graphs these days. How is that all coming together? Well, we took a approach, uh, on that where the, the promise of the ServiceNow platform is that we, uh, don't own our customers data.

And so, uh, the way in which we work is it's much more about grounding the data, the LLMs in our customer data, that it is about, you know, training these models o over everything. Uh, one of the reasons you're hearing less about all the, the data part these days is because the, the LMS are getting to the point where even if they're not trained on the specific data, they're now good enough to solve problems. We know how to prompt them.

We know how to bring in grounding information. We're beginning to learn how to make 'em take action and, you know, do things with a platform as well. And those are things that, from my perspective, that has not gone away.

We've just had so many people working on it. We are taking those problems away from our customers and bringing them into the platform, been solving them, uh, here as we go. To what degree do you think people are gonna have to become, quote unquote prompt engineers?

'cause it seems like we're making the platform more transparent to the average end user. Yeah, so our, our goal and really the, what, what we deliver is that our customers won't need to be prompt engineers in order to get value from the platform. Now, some of our customers want to do kind of new and exciting things.

And so what we wanna do is meet them where they are. If they want to do a small amount of customization, we wanna do it in a, in a platform way. We don't want 'em to learn a new skillset, right?

This, this concept of prompt engineering, everyone wants to, wants to become one. It's probably not, uh, something that everyone actually wants to invest the time to do well. So make it so it works outta the box for most customers.

Make it easy for our customers to, you know, dial it into their use case. And then we're a platform. So if they want to build something where they go full on from engineering and you know, make their own LM and things like that, that's fine.

They can do do that on the platform as well. But not all customers need to do that. ServiceNow has a long history of providing the automation frameworks that a lot of folks depend on.

The LLMs themselves have some automation capabilities. Where is the line between what does ServiceNow platform does for automation and what the LLMs are gonna wind up doing, especially as they get smarter? Yeah, so I, my personal perspective is that the LLMs actually aren't good at automation.

So what they can do is they can, uh, they can give you an idea of what you would do to automate something. They can't do the automation themselves. Alright?

You need a platform. And you look at, uh, you look at a simple process, you know, like make a recipe or something. Okay, it can tell you the steps to make an egg sandwich or something like that.

You look at something complex, which is, you know, doing some kind of a workflow that involves, you know, three or four departments and you know, you might need five or six human beings in that workflow to either approve things, sign off, bring in their own perspective on it. You can't have that happen in an LLM. It can't talk to those people.

It can't coordinate them. It can't ask for approvals. It can't track the work as it's being done.

And so what you need really is our true workflow automation platform. The LMS help you leverage that platform. They help you get more value, they help you to interact with it, but the actual work is done as part of a workflow, not by the LMS themselves.

Since our LMS are just a, a small part of the overall picture here, I feel like we're kind of on the cusp of a Cambrian explosion of applications at the show. You guys are talking about a no-code tool that lets the average end user go build an app. Theoretically we could have hundreds, thousands of applications that organizations are building.

How do we bring some order to what could easily become some potential chaos here? Maybe too much of a good thing? Yeah, well, I think first thing to point out is that there are already hundreds or thousands of applications in our customers built on top of the platform, and they often have to spend large amounts of resources in building, in maintaining those applications like that.

So taking away some of that work and making it easier for, to say, this is what I need and you have it, uh, come out, that's a, that's a net win for our customers. Now, once you've done that, then you've built it, then the question is how do you deploy it, maintain it, uh, and like that there's advantages in using large language models for part of it. But that's part of what, uh, our customers do anyway.

They have their teams, they, they build their KPIs. You can measure and track and taking venture of things. So long as you're doing it inside the platform, uh, you have the full visibility, uh, in, uh, in terms of what you're doing.

You can discover things that are already being done like that. And so the important thing there is to make sure that you lean into the platform. Uh, it's designed to have hundreds or thousands of business applications for each of our customers.

Works just fine, uh, for that already. We're just making it easier and taking away some of the work and the pain in, in setting that up. How do you think automation's gonna evolve from here with all this?

And I asked the question because we have seen islands of automation for years now, and they're slightly disconnected and, um, you know, the handoffs are disjointed and they're not as tight as they are. Is that friction gonna drop? And for that matter, you know, will the silos that we have artificially created around sales, marketing, whatever it is, start to kind of collapse a little bit as things become more interwoven?

Yeah, I, I think that's gonna happen. Uh, but there's, there's multiple things that at play here. So there's firstly the technical systems where, you know, they speak different API protocols or things like that, and they need to be bridged.

And AI is starting to be good at that kind of thing. Doesn't really matter what the data looks like. It can figure out what it is and, you know, use data to solve a problem.

There's a second problem though, which is disjointed IT systems and a third problem, which is disjointed organizations where different departments are the only, no thanks for the second one. You need a common platform. And that's the, that's what we provide.

And with the integrations we help bridge the other platforms around the human part. AI is not gonna solve that, that human problem. I think what people are gonna discover is that there's a huge opportunity out there, and it's gonna be worthwhile.

We're seeing at the C level now, more and more conversations about, okay, this is owned in the whole organization, not just in the department here and there. So we're gonna see more enablement on the, at the corporate, uh, level of that. But that's something that, uh, you know, anyone who wants to lean into generative AI is gonna have to work out in the context of their own organization.

I think one of the things we have seen in the last year is a greater appreciation for data. Um, the, the adage still exists. Garbage in, garbage out.

AI models kind of highlight that in ways that are, uh, complex and shall we say, lead to hallucinations. Do you think that the rise of AI is forcing us to revisit data management and as such, making us rethink our platform strategists? Certainly the, uh, the traditional platform strategy, which was that you take best of breed for an individual application area and combine 'em together, uh, that is starting to be rethought.

Uh, because often the, you know, if you want AI digital, good job, there's garbage, uh, you know, garbage in, garbage out. But there's also, you need to have the context, right? The, you know, uh, a particular claim.

If you don't have the full context, how are you gonna evaluate it properly? Like that full set of information. And so there's really a question of, it's not just the quality of the data, it's having the right data available in the right place as part of the workflow.

And again, it needs to be segmented. There's regulation around things. It needs to be governed and needs to be maintained.

And so what we're seeing here is data is an asset for the entire organization, but it needs to flow through the workflow in a natural way so that the right data is available in the right place at the right time. And because artificial intelligence allows you to do cross funding, horizontal workflows, not just the the vertical ones we've seen before, that's creating a different, uh, way of thinking about data, which is how data flows, uh, within your business processes rather than where it's stored and sits statically. We have seen everybody coming up with something that feels like an AI assistant.

There's a lot of different flavors of it. Um, how will these AI assistants know about each other and kinda work together? 'cause a lot of times I'm asking it to create ch handle some task in an asynchronous fashion, but now I've got all these different ones doing these things.

How do we kind of like bring them together in a, in a team, Right? Yeah. So I think there's two aspects to that.

There is the experience aspect, which is where do you go when you want to, you know, talk to or work with your, your assistant or your team of assistants as it may be. And here we're seeing a, a bunch of solutions, like some solutions where, you know, one AI you can call and you tag it and it'll hand it off to a different ai, other places where it will kind of, you'll tell it what you want, it'll send you to a different system to, to complete the request there. So as an industry, we're still trying to figure that out.

Now, what's important is that the, when you're trying to actually take action and solve, uh, the, the, you know, solve the issue or the request, you need the intelligence to be in the place, in the right place. You can't have this intelligence, which is essential intelligence, which knows about everything. 'cause it's gonna be like a jack of all trades with a nut master of nut.

You need to make sure that the intelligence is embedded in the platform, which has all the data, which has the ability to take action, has the ability to follow up, uh, there. And so what we are seeing is that's, that's crucial. And so how we design around that is, uh, something we are still working out as an industry.

How do you think, um, the way we work is gonna evolve? You've been doing this now for a couple of years, but you've been working closely with customers in the last year. There's a bunch here at the show.

What have you seen that's kind of surprised you or, you know, what are you seeing or hearing from your customers that you went, Hmm, I didn't think we'd hit that. Or maybe they're telling you things that are, uh, things they wish they knew a year ago before they started. Right.

I think the, um, for me, the, the most surprising thing that I've heard from our customers, uh, you know, apart from some of them are just crazy ambitious. Okay, that's the like, absolutely crazy ambitious, you know, this is something that they're gonna take and they're gonna run with it. And this is going to, you know, make this is gonna change their business and everything there.

But the other thing I heard is that it's very, very easy to, to, uh, you know, just listen to the hype to see, I saw this demo and this demo is something that's, you know, someone like some startup did it or this, you know, the thing is proof of concept. And they say, wow, this is gonna transform my business. And so then they go with the idea that the after proof of concept, there's 95% of the work done.

It's not, it's 5% of the work done. The other 95% is in the future. And so what we're seeing with a lot of our customers is taking these proof of concepts and bring them into reality.

They're discovering all the gaps that, that you have. And this is the, this is the stuff which some people would consider boring. Like how do you set up your security policy properly or how you deal with compliance?

Or how do you make sure that the data flows in real time so that you're not waiting for the answers or, or like that. And it's all these kind of little things which don't make the demo, but make the actual application where people are getting, uh, stuck on. And so what we're seeing from our customers is this, uh, you know, this realization that the path to get the value is not about just cobbling things together.

It's about having a way where those things are already brought together in one place so they can build with confidence, uh, on top of a solid foundation. Mm-Hmm. It's a come back full circle.

Seems like every day I wake up and there's some new advance in the realm of lms. Um, they're getting more expensive, they get bigger, they get smarter. But not everything requires these complicated LLMs.

Some things can be handled by smaller LLMs. So do we just need to get smarter about orchestrating what LLMs we're using for what purposes? And, uh, where does that manifest in the platform and there, and, and do I need to care or will you handle that?

Yeah. So in that response to the last question, you can care if you want to if you're a customer, but, uh, we do care about and we think about it all day every day. And so yeah, we'll have you covered whether you care about it or not.

Uh, what we're seeing though is, you know, a year ago everyone was talking about these lms, what they can, what can they do as i how many billions of parameters and that more billions are better, right? That was the, the idea you want just most billions possible. Well, you know what, these gigantic lms, they, they can do really cool stuff, but you've gotta be able to do prompt engineering.

It's easy to get wrong. And then these lms, they can go off the rails 'cause they've seen so much data, they've seen stuff that maybe they shouldn't have in the, in the, uh, kind edge of an enterprise. And these LMS also, they can be slow and they can be expensive like that.

And so we are discovering that it's not just about the, the size of the LOM, there's a lot of other characteristics are important. And so that's what we observed, that that's why we, uh, started down our domain specific LLM strategy is for some tasks we want it to be much smaller. We are interested in what it won't do, not just what it will.

And we want it to be, you know, really fast, uh, really, uh, you know, great in terms of the economics and much more robust. And so by designing an LLM for a use case and saying one size of not to me, uh, does not fit all, you're able to solve things much better. Yes, it could be solved by a large LLM, but you have a much better solution when you can kind of open up a black box and say, there's not just one LLM for everything.

And so our approach from the ServiceNow platform is to say, you know, the right LLM for the job might be a general purpose, one might be one you trained and bought yourself might be an open source, one might be running on the platform, might be running off whatever the best solution is. That's something which you'll get in the Service Now platform so that you can build with confidence knowing that you don't have to keep up with reading the scientific papers about the new advances like that you can build your application and know you're solving a real problem and it'll stay solved. Do you think there'll also be more awareness that, you know, the cost of all these LLMs is different over time based on how big they are?

Is is the ROI that you mentioned, the economics becoming a bigger conversation these days? I, I would say that for most of our customers, the, uh, it's more related to their budgeting cycle and the way in which they've thought about their, you know, their 24 budgets. Uh, there are some who are at a, at a large scale in terms of their generic gen AI efforts.

You know, our customers who use now assist, you know, it's very, very, uh, it's very, very predictable. They know kind of exactly what they're gonna pay, and we make sure we take care of the economics. So for our customers, it's not such an issue.

But for those who are building, you know, different kinds of applications, I'd say most, a lot of the scale yet where it's really, uh, important, you know, you hear it more in the really early adopters or the really giant companies who've been doing it for, for quite a long time. Mm-Hmm. When you talk to customers, are they kind of doing let's let a thousand flowers bloom approach, or are they maybe going after one or two use cases that are, that they're gonna do well first before they try to boil the ocean?

Yeah. Also, last year our customers had small budgets and they did proof SOCOM six. And so that was definitely a thousand flowers blooming.

We saw, you know, all kinds of sometimes amazing, sometimes, you know, crazy ideas bought by our customer saying, Hey, this is what generative AI can do for my business this year. It's become much more a year of, okay, we understand it can do a lot of things. We are gonna have to rationalize it because the, you know, they're not gonna build, uh, teams where they can run a thousand year independent use cases each with the own team like that they're being forced to Yeah.

As it moves into the, uh, closer to the heart of the, the business, they're having to put, uh, everything around it to support it, to maintain it, and to, to keep it running well. And so those concerns are starting to become, uh, you know, more important than they they were before. Uh, as you know, as they move from, it's a brand new technology to this is something which we need to make real in the context of our business.

Do you think generative AI is also, um, shined the spotlight on the other forms of AI that we've had for a while, predictive, causal, whatever it may be, but, you know, are people starting to understand that this is only one form of AI and we need to mix these things together to get to the right outcome? Yeah. From a purely technical perspective, there's not that much difference between generative AI or the, the line is not nearly as clear as people would think.

Yeah. 0, which is it's, you know, it's machine learning, but doing what they'd hoped it did before. Now it actually doesn't.

Right? And so it, yeah, it's more flexible. It solved the, the kinds of problems that you are, uh, that you'd hoped it had before.

Uh, so from that perspective, yes, I mean, you want to use the right tool for the job. If you are making, if you can solve something with a simpler model, which you know is not gonna go off the rails and, you know, cost orders of a magnitude less, why wouldn't you solve it with that kind of a model? And so generative AI will become part of the toolbox, and for some applications it'll be absolutely vital for other applications.

It won't be used at all. At some point it's not gonna be really important, whether it's generated AI or not, it's easy to solving the, the business problem you're, you're trying to, uh, solve, Man, we're at this show, and I know that you've been around here and there any use cases at this show that really pop out at you where you go, wow, that's truly innovative, or something that just kind of, you looked at and you went, wow, that just blew my mind. Yeah, so if we look, um, if we look a little bit, uh, further into the future there, we're starting to see people dream in, in, in terms of, uh, generative ai.

And so, you know, what we are seeing from, uh, most of this is coming from customers to be honest. Uh, you know, the stuff that we show, uh, the, the generative AI functionality we show, uh, you know, that's my team, uh, who's involved in building a lot. So it's not really new for me, but when we hear our customers dream, you know, they are really, uh, thinking about, you know, where does their long-term competitive advantage come from?

And so it seems like it, I've seen customers who are using generative AI in order to test out designs in order to understand, you know, if they're usable or not, and that that kind of thing there, which is like, this is an issue for them, you know, in their business they need to have the best usability. And so, you know, thinking how can we use generative AI for that? You know, that that's one example.

But you know, there's almost as many ideas as there are customers out there. And, you know, that's, that's what I find really inspiring about it is, you know, the, the, it, it really allows people to be creative and imagine that something where they don't see the exact solution. I think, well, maybe just, maybe, maybe not now, but maybe very soon I'll be able to solve that.

And so, yeah, we're seeing a lot of stuff that's from our customers. Are you seeing people kinda reinvent themselves in their jobs as a result? I mean, we hear a lot of people talking about, you know, concerns about loss of jobs, but it seems to me like it's almost like a blank check to go rewrite my job and create a whole new job.

So are people getting creative and are they having that level of imagination? Yeah, there, there are. I think there are multiple ways that people react to it.

You know, for us, I mean, our interest is in the future of work, right? But, you know, work, work is done by people. I mean, it may be assisted by, uh, by generative ai, by that work is done by people.

And so, you know, those stories that we are hearing about, you know, how people are mapping out what their job is gonna be like in the future, that's really core to, to what we think about and the, the, the future of our business. Uh, when we design our generative AI functionality, uh, you know, we are solving problems that are really concrete problems. Uh, we observe people, uh, you know, through statistically as what they're getting stuck on when they're doing their day-to-day work, like where their outcomes are not matching the KPIs and things like that.

And so they're kind of, the near term horizon of it is all, it just makes it easier for people to be successful in the way that they define success in their job. There's a whole lot of, uh, other things and people are reinventing themselves, and some people are very, very excited. You know, some people are scared, but I think a lot of people, uh, are very excited about it too.

And what we are seeing is how that plays out in real time across all of our customer base. 'cause in the end, it's our customers who, uh, choose how to deploy it and, and what they wanna work on. And, you know, we kind of partner with them and we get to see and learn, you know, as through their eyes and their employees as they're doing it.

All right, folks. You heard it here. It's not so much about who or what you were, it's what you wanna be.

Hey, thanks for being on the show. Bye. And we'll be back in a minute.