Jeremy Barnes, ServiceNow and Jeff Boudier, Hugging Face | ServiceNow Knowledge 23

This is Techstrong tv. Hello, and welcome back to ServiceNow Knowledge 2023. We're here with Jeremy Barnes from ServiceNow and Jeff Burier from Hugging Face, and we're talking about all things related to ai.

Jeremy, there was a raft of announcements at the show. They cover everything from using, uh, more advanced forms of existing AI machine learning algorithms to now generative ai. So walk us through a little bit of how all this comes together in the platform.

Yeah, well, this is a really exciting moment for us because we've working on this for month and months and months and months and even years, some of it. Uh, but what we're announcing today is, uh, or we're releasing, uh, is what we're releasing in our Vancouver release and some kind of preview of what's coming afterwards. And that is both, uh, what we call the generative AI controller, which is a way to connect into generative AI providers and bring the power of generative AI directly into the platform, uh, accessible from workflows, scripts, everything like that, as well as now assist for AI search, which is something which brings generative AI into search to create a much more consumer friendly, uh, interface.

And, uh, you know, really heightens the experience there of people so that they can self solve better agents, can get some right information, uh, things like that, leveraging generative AI behind the scenes. And now not everybody knows who Hugging Face is. I know in the world of ai, everybody knows, but we have some folks who don't.

So, describe a little bit about what Hugging Face is all about and what the relationship with ServiceNow is. Of course. Now, our, uh, mission as a company is to democratize good mission learning, and we do that through open source, uh, community, uh, driven, uh, AI and ethics first, uh, ai, uh, AI built from, uh, ethics, first principles.

And these three things are what brought us together, uh, with ServiceNow research, uh, to partner around a project called Big Code. And Big Code, uh, is an up science project that just released, uh, the best, uh, code large language models available today, uh, to work and generate, uh, code. So, super excited about this release.

It's called the Star Coder Model, uh, out of the big code project. All right, cool. How will people consume large language models?

I mean, and I don't think most enterprises are gonna go build those. It seems like there's other folks who are building them, like hugging face, and, um, so will I therefore take my data that I have and try to blend it with the data that's in those models to create some sort of outcome? How is this gonna manifest itself as we go forward?

Chairman? Well, if you just take the perspective of, uh, large language models by themself, uh, it's a very narrow perspective, then there's only so much you can do. And then there's question of do you train them?

What do you do with them? If you look in how they're embedded inside a platform like ServiceNow, suddenly there are much more interesting options. And so we expect that most people will consume generative ai, uh, in a couple of ways, both you built in or access through a platform that adds on top of it all the, uh, usability features, things like that.

But also something which heightens the experience and the productivity across all personas, be it, uh, you know, requesters, be it agents, be it, uh, admins, be it developers, uh, you know, bought to life, uh, within a platform and an experience. And so we expect most people will not, uh, train their own models, but almost all end users will benefit from them, uh, delivered via platform. Who is building the models for you then, if it's not gonna be the enterprises, or maybe it is, but who's actually working on this project that you described, and how big is this whole effort?

So our whole goal is to enable the community, the research community, the AI community, uh, to, uh, build and offer open source models. Um, and so we do this ourselves. We do this in partnership, uh, with other entities, like for the big code project.

Uh, we do this with the community. There is a huge community contribution within big code and its predecessor project, the big science project. Um, and then we offer a platform, uh, with the largest open source platform for ai.

So we offer a platform tools, uh, for the community to, uh, share, uh, their models, uh, with everybody. We have, we just passed, I think it was yesterday, uh, 200,000 free and publicly accessible models on the hiking phase platform. So the, the scale is massive.

Now, this is just one of several that you seem to be invoking as a partnership in Microsoft for different, um, use cases. So will the language model that you use differ by use case and should people expect ServiceNow to kind of partner with all kinds of folks? And might you build your own large language model for something?

Yeah, So I mean, our, our vision is that whatever the technology is that we bring it to life in the ServiceNow platform. And so, you know, we will partner with, with all of the partners, uh, that we need to bring the value to our customers, uh, through the ServiceNow platform. There are some use cases that, uh, maybe there aren't language models that are available that are ideally suited for them.

You know, an example, uh, is the, the big code language model, which, uh, you know, it emulates something was already available, but ads this layer of governance on top of it, which just didn't exist before. And so where we see a gap in the market, uh, we will absolutely, uh, fill that gap, uh, for the benefit of our customers. And again, our goal is not to, uh, create language models for the fun of creating them, although are just kind of fun.

Uh, the goal there is to make sure that the problems that our customers need solved, that we have the solution for them, and that's accessible from within the platform. We go from irrational exuberance about AI to fear and loathing and a flick of a switch. So what is your sense of what's real and not real about ai and what should people kinda, you know, realistically expect versus all the hype and noise that they're currently seeing?

There's a, there's a bit of a, a cult of AGI out there. So agi, artificial general intelligence, um, from, from our perspective, there is both so many use cases that are already out there, uh, for ai, right? I talked about 200,000 models.

Uh, that means like every single language, every single task for working with text, with odio, with images that is already available. And there's also a lot of, uh, potential risks that are already here today without having to think about like what AGI could, uh, bring upon us, right? So we're trying to maximize the positive outcomes of ai, um, through ethical guidelines.

I think the BCO project is a great example of that, right? Because there was so much effort, uh, into, uh, putting together a data asset that's open, it's publicly accessible. It's called the stack, it's a trillion tokens, and it's all based on code that is permiss that has a permissible license.

So you have no risk of getting stuff that you shouldn't be allowed to have in there. There's also a mechanism so that developers can say, no, I don't want, I do not want my code to be part of this training and this model, and they can remove that through an OPTOUT process. There was a massive effort around d duplicating the data to improve its quality and to remove any information that was personal.

And all of this process is being documented. I think we had a great bo uh, blog post, uh, with your colleagues, uh, harm re uh, I, I, I have to give him a shout out, uh, from ServiceNow research, uh, around the, the, the, our work, uh, on the data. And I think one of the best outcomes of big code is that education, uh, we're providing setting not just an example, but giving the recipes to the AI community so they can build, um, uh, more responsible, uh, data assets and models.

So much of the value around ServiceNow was built around applications constructed with a low-code tool. Now we have a natural language interface. Do I need the low-code gooey anymore, or is that just hidden behind the natural language interface?

How will this all kind of play out? Yeah, so it's a question really of user trust and the, everyone imagines the, uh, process of writing a new app and then it's done. It's not, uh, there is a huge amount of maintenance and things like that, and, uh, you people need to be able to understand the state of things as well.

And so we see language models being something which will augment what's available in those, uh, kinds of gooeys. There's nothing like the visual cortex for being able to see how something's laid out and, and understand it. And so, uh, we don't imagine that that will all go away.

We think it'll just become much more productive and maybe there'll be slightly less icons, uh, to get lost in, to click on, because you'll be able to do where things directly, uh, by, by language instead. But overall, we see it enhancing the user experience, uh, whilst we're remaining familiar enough so that people can get up to speed with it really, really quickly. The quality of a large language model is tied to the quality of the data that goes into training, frankly, in the enterprise.

We haven't always been so good at data management and data qualities. So do you think as we shift to ai, this is gonna force people, uh, maybe have some better discipline and some better hygiene about the way they manage data? Um, possibly.

Possibly. I think the, the, the new sort of paradigm is the idea of transfer learning. And so the idea that you can reuse a pre-trained model that has been built upon publicly available data and then reuse very, uh, efficiently your private company data to improve that model specifically for your use case, your brand, uh, et cetera.

So yes, there's going to be lots of, uh, best practices to adopt so that you can take the, your company data and then transform it and format it and maintain it in a way, um, that you can, uh, uh, build models that have great, uh, uh, data lineage, that have great data versioning. Uh, so you can have, uh, all the audits that you're going to need to make sure that, uh, models are, are built, uh, in the best way. Can you even envision an application going forward that wouldn't have AI infused into it?

It seems to me it's gonna be pervasive. Uh, we can imagine some scenarios in which you, you wouldn't, and you, there, there are gonna be questions around, uh, you know, things that need to be ultra reliable, for example, kind of a fail. We, we all know that generative AI models sometimes make mistakes.

You're not gonna put them in a place where no mistake is possible. So absolutely, I think it's a question of where it's appropriate, uh, and the context that it's in. For a lot of consumer things, I think that generative AI is gonna be quite pervasive, and we're gonna see, uh, because a goal there is to heighten the experience as much as possible.

And, uh, in places where productivity is important, I think we're gonna see a lot of generative AI too. But other places, uh, you know, it's gonna have to be something, it's gonna be based on principles of governance and those principles, they have to have the answer no be possible somewhere in order to be effective. And so we, as that develops, we will see some things here falling on one side.

A lot of things, I think, and a few things fall on the other side, whereas say general AI is not the right solution for that right now, We have seen some folks call for a pause in development. We have seen, uh, bureaucrats and politicians get involved. How do you think this is all gonna play out from a regulatory perspective or we we're currently gonna see some more regulations, but, you know, what's a reasonable approach?

Um, yes. So I've seen the, the, the call for our moratorium. I don't think it's a good idea.

Um, yes, politicians are getting involved and it's a great thing they need to get involved. Um, yes, regulation is coming and, uh, there's a lot of conversations right now, uh, between AI researchers, uh, companies, uh, and, uh, regulators. Uh, there was a senate hearing, uh, today, uh, I know the EU commission is also, um, um, asking for inputs.

Our team, our ethics and society team was meeting with the EU commission today. So it's a very active debate. I think for me personally, um, the most crucial thing we need to get right is the availability of the source data for the models so that we can build all these accountability so we can build all these auditing so we can build all these regulatory framework based upon the training data.

Mm-hmm. Now, why is person once said, it's one thing to be wrong, it's quite another thing to be wrong at scale. What's your best advice to organizations about how to get started with AI and not have that chaos that he just described?

Yeah, I absolutely, uh, I think the most important thing is that they think through, uh, the governance around their, their use of ai, whether people like it or not, uh, it is coming and each company needs to be able to think, you know, to, to put in place, uh, the right safeguards around it to make sure that they can answer in the affirmative when, you know, not only legislators, but also their customers and users are gonna be asking for some accountability around it. So I'd say, uh, and that governance extends to the understanding the full supply chain for the, uh, models or the software that comes in, or you are asking questions of the vendors around it. It involves, uh, understanding what are the situations in which it's appropriate to use it and, and what you situations, uh, it's not.

And also, you know, bringing on board enough of the, there is a, um, you know, a fair bit of foundational, uh, work which has been done in the AI research community about, uh, you know, putting in place some structures that you can use to think about these things. And so you're working with, in openness and transparency with the, uh, research community, because in the end, they're the ones who understand these models the best. And so we need their help, uh, in order to make the right decisions for, you know, not just for the, uh, community in general, but each company and, and, uh, situation also needs to, to be able to look at things in that way too.

In many cases, AI researchers have been working on this stuff for years, maybe decades. A lot of folks just discovered this stuff six, seven months ago. What's coming over the horizon that you're looking at from where you sit that people might wanna be thinking about?

What's, what's the next big thing? Yeah, six, seven months ago, I think six or seven months ago, what we've seen from our vantage point is that, um, ai, which, um, has been very much a scientific driven field, uh, and science works through open publication and collaboration. And six or seven months ago, we went sort of from this to a situation where AI was kind of, um, locked behind, closed doors with the latest models not really being published to the scientific community, which makes it hard, uh, to understand how they work.

So that's been a transition. Um, I think it's only been, uh, six months, but, uh, you are, uh, telling me about, uh, AI research in its early days. So we started this company hugging phase seven years ago, and at the time, AI wasn't able to do a lot.

Like you could recognize a cat in the picture, right? Um, and today, uh, today can do so many things from generating understanding texts, uh, transcribing what we're saying, and to subtitles and turning these subtitles into an image, an image into a video. It can do a lot of things.

And at Hacking phase, the reason we started this company seven years ago is because we think that ai, uh, is going to be the default way to build technology, but that doesn't mean that, uh, you're gonna hear people say, Hey, I'm gonna use, I'm, I'm using AI to do this. No, people are going to be doing what they do. It's just that AI is gonna make it a little bit faster, a little bit easier, it's gonna make people more productive.

Um, and it's going to blend into every product that we use every day. What comes next for service now? Kind of cool stuff that you showed today, but, you know, like they said, you know, what did you do for me tomorrow?

What's coming next? Yeah. So I, I think there's, uh, you know, the possibilities for productivity, for experience, for the ability to, uh, you really understand, uh, situations and understand people at a different level, uh, that's all coming, you know, these tools are available now.

So for us, it's, you know, most, uh, of what I've seen in the market at the moment are small proof of concept and things like that. So what's coming next for us is how we just bring this into all the places that can create value at scale with the right safeguards in place, of course, so that people can confidently go ahead and, and make use of it. And, you know, it can, it can play a positive role in, uh, the business of our customers and, and the, the users that they serve.

All right, folks, you're heard it here. If you don't understand it, they call it magic. If you do understand it, they call it science.

Try to understand it cuz you won't be afraid of it. Gentlemen, thank you for being on the show. Thanks.

Thank you so much. All right. And we'll be back in a minute.