Deel’s Dougal Martin on The Emerging Role of AI Librarians

Hello, I'm Mike Vizard. ai video series. We're here today with Dougal Martin, who's head of Knowledge for Deal, and we're talking about, well, the need for AI librarians.

And I'm gonna let Dougle explain what it is an AI librarian does. Dougle, welcome Michelle. How you doing?

Great to be here. So start us off here. Is, is this one of these new jobs that are being created by ai or what's going on here?

Um, I mean, I think you could call it a new job. I think there's always been a role at any company that maintains large amounts of information. Somebody has to be accountable for that information.

Uh, but that's taken on a new importance in an AI enabled environment, right? Uh, AI is great at, uh, accelerating the sharing of information, information discovery. And that means it's gonna scale bad information just as quickly as it scales.

Good information. And so, you know, in the context of deal, we're, uh, you know, all in one global HR and payroll platform, compliance plays a huge role in the services and features we build for our clients. And so my job and the job of my team is to capture, uh, and document the collected expertise of, you know, deals, lawyers, all of our local HR teams, our payroll experts, make sure that information is accurate, well organized, and to curate that information so that we can use it to build new tools, build new features for our clients, and also share that information with our clients to support their own compliance.

You know, I talk to a lot of people about ai and one of the issues they seem to be having, and it, and it seems to come up over and over again is, well, a lot of the processes we have today are deterministic, right? They're supposed to be done the same way each time, every time. And the LLMs are probabilistic, right?

They're taken a best guess and they hardly ever do something the same way twice. So how do we kind of marry those two things together For a company like Deal? And I think for the types of things that we're talking about, it's about the way you structure your information, right?

And so we have a 26,000 article knowledge base, which is growing by about 2000 articles a month. And that's where we capture important context about things and, and language, large language models are great at that, right? And they can synthesize that information.

They will never give you the same answer twice. But because we have a well-structured knowledge base that provides rails for the AI to follow when it's asked about certain things, I can make sure that the information that is important gets into that answer. But we don't just use an LLM, right?

We have API queries that can access our own database of information where I hold variable information like rates or numbers or, you know, feature availability or different types of systems that I wanna index. So there's a lot of tables there too. And that's highly structured information.

And I have a lot more control in how that information gets organized into an output. So it's not just LLM, right? LLM is a tool, um, in a large suite of AI tools that create, you know, an interactive AI chat thought that we use to deliver, deliver expertise to clients.

Some folks I talked to are betting that there will be AI models that track what the output of other AI models to ensure and validate, and that provides some governance and essentially what you might call adult supervision. Is that the way to go? Or, um, is there some other way to think about this?

I can say with almost absolute certainty that that is going to happen. Um, we talk about that in the context of, of AI in the loop. And what we mean by that is, you know, different models are good at different things, right?

And you want to build a model for the output that you want, for the purpose that it serves. And so we have models that are good for, you know, calculating costs. And we have models that are really good at delivering, you know, contextual answers about different types of entitlements and leaves.

But we also wanna build a model, you know, did that conversation with the agent go well, what, what was the substance of that conversation? And was there information, information missing from that conversation so that we can then use that to drive another round of content creation to make sure that we've got editing our clients are looking for. So it's, it's a big system, right?

We're not gonna have one model that does it all. So this AI librarian, how do you train to get that job? What skills do you need?

I mean, you know, who, who fits the criteria? Um, I think I'm still training to get that job. How I came to this place is a bit of an accident is I took a roundabout, a roundabout, you know, education and career path.

I was trained as an anthropologist I really didn't like. Um, and anthropology is basically a degree in writing people and people, people-centric systems, right? How do people interact with each other?

It's about writing culture. Um, I assumed for a long time that I would have a career in academia. I did try that for a little bit.

It wasn't for me. Um, but was very gratifying to discover when I gave up on academia that there's a large contingent of anthropologies anthropologists down in Silicon Valley. Um, I got my start, uh, in technical writing at Facebook, uh, basically because Facebook realized quite early on, you know, you've got all these engineers who are spending a lot of their time writing for the finance team or writing for another team, and they don't want to do it.

And the finance team doesn't understand the output. Well, we should get writers to do that, right? We should let the engineers be engineers.

We should let the writers write. Um, and so I think, you know, AI librarians are emerging from that documentation space, which is now we've got this new tool that can support documentation creation and documentation discovery. And so I think you're gonna see a lot of anthropologists in that space.

I think you're gonna see a lot of creative writers in that space. Sociologists, psychologists, lawyers, anyone who, um, has an affinity for content creation. Anyone who likes organizing information, uh, is gonna find a role I think in the, in the AI revolution.

Do you think that part of this exercise is that in order to ensure that the AI model generates the right output, the librarian is gonna play a role in what content is actually exposed to the model to ensure that the output is relevant? Yes. Yes.

Um, I wouldn't go so far to say that I'm the arbiter of truth, uh, but there is a need for truth, right? Um, and, and you know, at a company like Deal where compliance is at the core of everything we do, I'm not just the knowledge manager for deal. I'm the knowledge manager, you know, for tens of thousands of our clients.

Um, and so we have to establish what the ground truth is and we have to have a rigorous approach to what makes it into the, into the model, what makes it into the knowledge base that our model has access to. So as we go forward here, is this gonna be something that every company needs their own AI librarian, or do you think at some point maybe there's a, a set of services we can all count on that somebody might provide us? 'cause we all have the same basic problems.

I mean, the services exist now, right? I am, I am the AI librarian for all of our clients. I think it depends on the organization, uh, the organization and the types of information that they have.

If you're dealing with large data sets, you're dealing with large complex documentation, especially longitudinal documentation. If you're looking for, um, predictive value, someone has to be accountable at an organizational level for the integrity of that information. Whether it's an AI librarian or a chief information officer, somebody has to be accountable for that information and establish the processes through which it's maintained.

Um, AI is only as good as its inputs and its outputs are really important. People are gonna start relying on output more and more, and we're gonna take those outputs that face value. So whether or not it's internal, because you're a large multinational organization and you wanna control all your own data, or whether you're a company that's using, you know, a company like Deal, uh, where we have a lot of the data that you're relying on, um, someone somewhere on that, on that chain has to be accountable for that.

So do you think at some point we're gonna have, uh, this job become a higher demand following some sort of lawsuit involving somebody saying, you know, you, you, you did me wrong. 'cause this AI thing that you exposed to me wound up surfacing something that was off kilter and, you know, destroyed my business workflow and cost us millions of dollars. And, you know, is that the, is that the wake up call that we need here?

Or can we be more proactive about this? I, I mean, I think a lot of organizations are being proactive about it. I'm not gonna make any, any predictions about, you know, litigation and product liability around ai.

That's a space that's gonna evolve continuously like it does and it's gonna be regulated and managed in different jurisdictions differently. But I do think that, um, most organizations are waking up to this reality, which is that if you're gonna depend on something, you have to have some way of ensuring the output. And that if you don't, responsibility for that is gonna fall on you, right?

If you, if you rely on an AI output and you lose your business, does having recourse, you know, to to liability on behalf of someone else save your business. It's too late. There you go.

So what's your best advice to folks about how to kinda set all this up? 'cause I think a lot of times, you know, we deal with organizational behavior issues and either somebody says, it's not my job, or everybody says it is their job and then it winds up being nobody's job. So how do we get in our arms around this?

It's a really, really good question. Um, I think a lot of it depends on your, your, your own organizational culture. Um, but for deal, we made the decision very early on that, um, a robust knowledge function and a large knowledge base that had strong, you know, uh, strong controls, strong quality controls, that we could rely on the integrity of that information, that that was gonna be a differentiator for us.

And so we set up processes really early. You, you start by going, you know, what kind of information do we have? Where is it gonna be managed?

Who's gonna be accountable for that information? And then how are we gonna index that information to other information that's relevant, right? One of the big dangers of large knowledge bases is that you have a lot of information that isn't connected to other information.

And I think your, you know, your listeners would know or your viewers would know that, um, an AI doesn't read 40,000 articles every time you ask it a question, right? It has different models that tell it where it thinks that information is gonna be located. So you need strong connections between what are really informational silos so that if I surface a process, so someone says, how do I do X and I surface a process that that information is gonna appear alongside, um, these records of decisions that have been made about that process or relevant compliance information that is connected to that process so that you get a whole picture and not just a, a piece of the picture.

And so, you know, you've gotta look to your, um, information maintenance processes. Everyone has to be responsible for a piece of information. Every piece of information needs an accountable owner or a subject matter expert, but someone's gotta own the architecture of that.

How are you gonna put all those pieces together and make sure that all of those pieces are discoverable for an ai? All right, folks, you heard it here. When you're standing in front of a customer and things are going wrong, the customer does not want to hear from you that says, you know, your very elaborate AI model went wrong.

'cause they won't care. They're just gonna blame you no matter what. So you might as well figure out how to get in front of this now.

Hey, Google, thanks for being on the show. Thanks so much for having me. All right.

And thank you for all watching the latest episode of the Textron AI series. You can find this episode and others on our website. We invite you to check them all out.

Until then, we'll see you next time.