How to Be a No-Code AI Power User with Andrew Zigler at AIE 2024

Hi, my name is Andrew. I'm a developer advocate at Mattermost, and today we're gonna be talking about how to be a no code AI power user, and to figure out how we get there over our time together today. Just wanna show you the agenda briefly, and we will get glimpses of this again as we go.

We're just gonna start with a quick introduction to myself and, and, and my perspective to the conversation, and then we're going to talk about ChatOps, because you can't become a no code power user of AI without understanding the mechanism by which you do so. And then we're gonna look at two examples, uh, once we kind of compare them with, uh, u utilizing AI in that context. And then we're gonna look at some ways that you can apply that moving forward.

But before we dive in, I just wanted to take a moment to introduce myself. I find it helpful when I'm listening to a presentation to understand a little bit about the speaker. Um, like I said, my name is Andrew and I work at Mattermost.

And before I was a developer advocate, um, I actually was a teacher, so I taught in Japan for two years after college. Um, and I worked with kindergarten and middle school students. Um, and you'll be really surprised by how transferable the skills are of working in a kindergarten classroom to working in software engineering.

And since coming back from to the states, I've worked in private education helping build online courses for students to, to, to learn, uh, virtually, um, especially during the pandemic. And now I work in as a developer advocate for Mattermost. And Mattermost is a, uh, collaboration platform for mission critical teams, for developers, for folks that are doing work that needs them to be highly connected in highly secured environments.

So we're talking about specialized chat in places like healthcare, finance, the military sensitive sectors that need to control their collaboration data. Mattermost is their collaboration tool of choice. Uh, just a little bit about us.

We're an open source project, and the examples we're gonna run through today are built on Mattermost, but they're gonna be built off of principles that you can take and apply to any of the collaboration tools that you're using today. And if you're interested in learning more about Mattermost, we have a really deep integration platform you can also check out in terms of integrating, uh, our collaboration tool with your own backend. So now that we've done that, I just want to, uh, move into what is ChatOps?

What is the vehicle by which somebody is able to become a low code or even no code power user of ai? So ChatOps simply put our conversations put to work. ChatOps is a concept.

It's an idea of taking people and tools, the processes that they work within and automation and putting them into a transparent place where they can all work together. This collaboration environment is called chat ops. When you are able to communicate with people and your tools in one shared place, get alerts and notifications on things that are happening in your environment, as well as obviously messages from your coworkers, as well as being able to enact, uh, workflows and tools.

Reach out to either your own backend tools or your providers. The third party sources that you use every day in your job chat ops is about making those tools easier to use and putting them at your fingertips. So achieving chat ops can be done in a variety of different ways.

You can use tools and integrations within your chat environment. You can create custom integrations like slash commands or even web hooks that respond to events happening within your, your channels, maybe responding to keywords or requests or happening in response to things, uh, in your actual, uh, infrastructure. Maybe you have Grafana monitoring your services and there's an outage.

That outage could send information into your collaboration stack into Mattermost, and then you could take automated or procedural actions from there to respond to the problem. When you're doing that kind of action, you're participating in chat ops. So an example of how chat ops is used and how this is a tool that that folks are able to use across the board to get more done.

Uh, you know, matmos is used within the Air Force to share, uh, documents related to planes getting in the sky. Um, so this is actually part of a case study we did with the, uh, US Air Force Air Mobility Command, um, which is partly responsible or largely responsible for getting planes in the sky, uh, for the Air Force. So before Mattermost came into the picture, there was a really long and complicated process because you had the mechanics that would prepare the plane.

You had the pilot that actually fly it. You had the mission operator that put all the information together for them and all of the support personnel across the way, and they're in maybe, uh, some of them are on the base, some of them are remote on different bases or other locations, and some of them are traveling or in the field. And so, um, being able to collaborate all of those folks and all of those stakeholders within one shared place was really difficult for them.

But by using a tool like Mattermost, they were able to share information and resources more securely, and they were able to do that with chat. And this is through chat ops, right? They're able to share documents into a secured environment.

They're able to take actions based upon their flights, have automations that track their planes when they go in and out of the sky. And in doing this, they become more efficient. These planes that would maybe originally sit on a tarmac and burn fuel or weight or cause traffic congestion are now more, um, able to get in the sky quicker.

And so you're saving time and, uh, consumption on fuel, you're saving, uh, labor costs, you're saving, um, actual like churn and and thrash between the folks trying to actually enact these operations every day. So ChatOps is a way that allows them to utilize their tools into the actual conversations that they utilize. And it's a philosophy that, um, any kind of company, big or small mission critical or every day can really benefit and learn from.

And so today we're gonna be covering some examples of that and to really unlock the potential of this, especially in the last year, there's so many ways that you can combine this ChatOps philosophy with new AI tools. So up until now or until recently, really when you think about utilizing, um, ai, you think about going to chat GPT, right? You think about opening that tool that that has direct access to the LLM and asking your question.

So you engage with the LLM, you get your response, maybe you send a follow up back to them, you engage in a conversation with the tool to get your information. This is what we would maybe call a traditional AI user experience. It's one player, notably, it's just you and the LLM talking together.

However, when you are talking about integrating AI with chat ops in your Mattermost environment, you're now talking about a collaborative AI user experience. We're not talking about this single player experience, but instead we're talking about a multiplayer AI user experience within Mattermost or wherever you folks are collaborating with ai. So if you have a channel and you have maybe threads of conversations that you and your coworkers are, are contributing to this channel, then you're creating a shared context, just like how before you'd go to chat GPT and have your conversation and, and figure out what you would need to know, you could utilize your channel in that same way and utilize the LLM with your coworkers in a transparent workflow.

This is starting to sound like chat ops, and this is where you get the opportunity to utilize the LLM to summarize threads that are happening within your chat channels to tell you about your messages that you've missed since you last went into a channel, or you were last online. Maybe you were on vacation, or you recently transferred into this team, or you're just trying to get your bearings after. Uh, you know, y yesterday's shift ended and today's began for organizations of huge sizes and of mission critical needs, um, getting caught up when that information is vital.

And the, the any tool that you can utilize to, to lift those pieces of information and help you act on them more quickly, um, is treasured. So being able to summarize the new messages and figure out what you missed as one thing, but then you can go a step further and see what action items were discussed in those conversations, or were there questions that people didn't answer. And because the LLM is a tool utilized by you within this environment, it knows things about you, it knows your role, it knows your name, it knows what channels you're in.

So it's able to tailor its response to you based upon what you are contributing to the conversation or the organization. You can also utilize it to summarize actual meetings and calls that you have. Maybe you have a voice call on Mattermost, which supports, uh, voice calling and screen share.

Afterwards, you could turn that recording into a helpful summary to share back to the channel. This helps your team work more in an async way. If you miss an, uh, meeting or you're not able to make it for a reason, you can catch up on the recording and the expectation becomes that you can find this recording and it's gonna provide you the information you want.

And so, um, it, because of this kind of recording feature, it's easier for teams to collaborate no matter where, where or when they're located. And as you can see, here's an example of it getting shared back to the channel. And then as well as using the AI within this shared kind of channel environment, you get shared contextual interrogation.

And contextual interrogation is a big scary word that really just means asking questions based upon what is said. So you might provide it, um, uh, an initial premise or an initial question. It might provide you a response that's helpful.

Um, and then you could follow up with it, right? You can ask it to maybe change the format of its question or apply this feedback that you're giving it to make the response more tailored to you. And you get the added bonus of being able to do this in a transparent and open way, maybe within a, a channel that you and your team are using on this particular project, or a, a thread that's specifically tailored for the, the topic at hand that allows the LLM to gather and utilize the information that you and your colleagues are already so diligently producing and discussing, and use that to tailor a response for you.

And this is a really effective way of utilizing this kind of tool. And the things that we discussed are really active, right? You're clicking buttons, you're getting summaries, you're asking questions and getting those responses.

And there's also a whole other layer underneath that that, that we haven't even really tapped into. That's really just the top, the tip of the iceberg. But underneath, there's all of the passive things that the LLM can do for you in this kind of environment.

Um, it could summarize things on a schedule or, or manage tasks for you automatically create tasks or to-dos or reminders and third party apps or applications, um, if you give it the tools to do so. And that's something that we're gonna be looking out later in this presentation. Um, it can also help you engage with, uh, folks who reach out to you on your behalf, um, as well as handling events, uh, that happen on the server or otherwise, and, and being able to surface information for you.

Um, the LLM is really great at understanding what matters to you if you tell it, and then seeking that information out passively and, and surfacing it to your attention. Um, so you can train it on something or, or point it on something that, uh, is really important to you, um, and then set it to work. So in everything we've talked about so far, we've really talked about the functionality with buttons and interfaces and ways that the user can interact with the LLM, but what about the LLM itself?

What, what is this model that we're talking about that's working? You know, Mattermost is all about choice. And, and choice will also give you the choice to choose what model works for you.

So you can see here I'm running through a list of, of some model providers. And really the key with, um, using Mattermost and the AI in this example is that we don't dictate, or or, or tell you what LLM you have to use or can use. Uh, we have an open, uh, and no lock-in philosophy with utilizing AI within your collaboration environment because for us, we're trying to give, uh, customers and organizations the ability to choose, uh, what works best for them.

Um, they have different levels of tolerance to risk. Um, they have different levels of tolerance to, uh, security, and they need the flexibility to choose when and how and on what terms something within their environment is deployed and used. The same goes for their LLM.

Some organizations are, are, are not allowed to be utilizing AI yet. And so they don't have access to the L LMS that would plug into the system, but others do. And of them that do they maybe have chat GPT or Microsoft Azure's Open ai, they might have anthropics Cloud, they might even be making their own models.

And so within this kind of tool, you're able to utilize different LLMs just to, based upon the needs of your team. All of them are going to plug in whether they're a local or third party. And if they're local, which is in the case of like local AI or leapfrog ai, then you're going to have the ability to self-host your LLM, which means you have matter most self-hosted your controlling all of your communication data, and then you have your LLM self-hosted and you're controlling all of your artificial intelligence data, and then they are connected to each other and nothing's even leaving your network.

This is the future of how companies will be thinking about and utilizing and integrating AI into their tools and into their actual, um, cloud architecture into the things that they provide for their employees. Um, so now that we've kind of run through all that, I want to touch on some examples that we can learn from in order to, uh, actually create a chat ops tool. So what we've covered so far are things like summarization, um, and being able to, uh, uh, quickly access the LLM within your chat environment, which is very handy for collaboration and for getting caught up on information.

But what if you want to create a very specialized workflow or a what if you want to provide the, um, LLM with access to specific, uh, information, maybe that's from your own backend, or maybe you want it to look up information from, uh, a website or another tool and use that in its response to you. Really what we're describing is we need to give the LLMA tool and, uh, tools are ways that we get things done as humans, but the LLM can also use tools to achieve certain tasks, and it's going to choose to use those tools based upon the context of what's being asked of it. What I'm describing now is called function calling, um, which is a feature that's available for many models, including chat, GPT.

And how this works is it allows you to provide a, a list or a library of, of basically functions to the LLM, you provide the, the function name and the description of, of what it does, if it's invoked as long along with the, uh, the schema for the kind of data it's expecting. When you put those tools together and you provide them to the LLM, it will choose at its own discretion when handling your responses, what tools to utilize. And so one example of a tool that you can access and, and actually build, um, is, is through a repository I put together that has a matter most, uh, plus the, um, uh, our AI plugin.

You can put in an ai, um, uh, API key and then utilize it with a tool called Nation. Um, so nation is a tool that allows you to build automation workflows. These automation workflows can also be tools that the LLM is using.

So in order to really understand what we're going to build, I wanna fast forward for a moment to, uh, let's see this slide just to look at, uh, an example, nation workflow. So this is an example of just a drag and drop interface where someone has created step-by-step instructions for a tool. This is a lot like Zapier.

If you're familiar with buildings, um, zaps for your backend. Um, this is basically a, a, a drag and drop interface where you have inputs like time or web hooks or, um, uh, other events that trigger then a sequence of other actions and nation being drag and drop. And it has a really, uh, really diverse library of tools and integrations that you can choose from.

You can build these really easily, even if you are not a coder, uh, you are not familiar with, with, um, with writing code. You don't have to be to create these tools. And then once you create them within this interface and enable them in the way I'm going to, uh, show you in a moment, your LLM can then call them and utilize them.

So really what we're doing now is we're designing specialized tools for the LLM to solve specialized problems for us. So going back to the beginning of my example here, um, the actual demo that I'm providing in this repo, and you can grab the, the, the, the, the, the repository here with this QR code. Um, and this is actually built on GI pod.

GI pod is a, uh, cloud, uh, developer workspace that allows you to basically work like you're n vs code or using a, like a docker server or a virtual server. Uh, but in the browser it's a really easy way to test out new ideas. Um, and it's very flexible.

So here I've set up a demo. Uh, so once you go through the repository, um, and click on the button to get started, um, it will load this environment and provide easy links for you to access your own Mattermost deployment. So we now have a Mattermost deployment in the cloud for the sake of testing this workflow, um, as well as, um, access to nation that drag and drop automation workflow builder tool.

So it's gonna spin both of them up, which you can access on the ports tab here. Uh, you'll be able to get the URLs. So this is an example screenshot of nation.

This is a drag and drop, uh, builder that I did for this presentation, uh, that you can also find within the repo itself and, and upload to your own nation. Uh, how it works is it, it intakes a slash command from Mattermost and it fetches, uh, a list of events from developers events, and then it filters those events based upon whatever keyword term I provided to it. So if I did, like, for example, like slash find events, um, uh, DevOps days, for example, it's going to produce a list of DevOps days events, and then in this case it's going to post them to a channel.

So this is a low code example because some parts of this particular workflow do have code blocks. So for example, we're, we're intaking the mattermost slash command. This is no code we're fetching the, uh, developers event, JSON.

This is also no code, notably, I'm only just putting in the URL and nation is doing all the rest. This is, this is not code I've written. And then it fetches the information from developers that events to share it.

It splits those results into individual items so that we can filter them, and then it's going to filter them. This is the, the low code. This is the only rule of code that I wrote as part of this particular tool, uh, where it takes that, um, that that actual, uh, keyword that you're looking for.

In my case, DevOps stays and it's gonna filter your results to get the ones that are left. And in this case, it found DevOps stays Raleigh. And, and that's the one that it's going to post to the, to the channel.

So this is one example of a low code, um, AI tool that I built with a drag and drop builder called nation in order to, um, actually a allow you to, uh, look up this information on the fly and utilize the LLM to return that to you. This is a response given back from the LLM in this case. So there are lots of ways that you can improve this workflow.

Um, my example had past events, you could filter those out just based upon that field. Um, you could also use, um, message attachments to make them a little more pretty, um, and engageable within Mattermost. And you can really do anything with that data, like make a, make a task or take an action.

Uh, but there's a lot of ways that we could actually turn this into a better tool for the LLM to utilize. This is really the initial step of what we're trying to achieve, right? If we, when we think about ChatOps, we want to step back for a moment and think, what am I solving in this case?

I want to have access to, uh, at my fingertips, um, the information for developer events that are happening. Maybe I want to use that information to, uh, uh, go check out those talks that are being given there or figure out some, uh, cool topics that people are talking about right now at developer conferences. So there could be lots of different ways I want to utilize this information.

And it sounds like it's well equipped to be a tool or a function for the LLM to call. So now let's look at an example of taking this low code example and turning it in effect into a no code example that then leverages the LLM to do those things. So taking the same idea in the, the, the repository I provide to you, there's a, there's a, there's a fork that we have right now.

This is like a, a fresh off the branch, just cut the other day experimental AI copilot plugin version from one of our engineers that allows this type of functionality I'm describing. So you just are able to grab that file and upload and enable it. Once it's enabled, you're gonna create a API key for nation.

The reason we do this step is because we need to provide it as an external tool provider to the Mattermost AI co-pilot in this particular demo. When we do this, what we're really doing all this, all that step is doing is allowing your Mattermost, um, AI copilot to ask the, um, ask a nation like, Hey, what's the list of tools that you have available for me? Uh, what are some things that I could do with the stuff that you have available?

So in this case, we're gonna take that workflow, uh, from before the, the one that I created and turn it into one of those tools. Um, and, and in doing so, we're going to give it a description, uh, in, in the first node. We're going, going to put in a bit of text that tells the LLM what that does.

You might recall from me earlier, I said that the LLM needs a few things of information. One of them is, uh, obviously the description of what the function will do. So the LLM can make the decision to call it.

Um, as well as that in the second node of the workflow, you want to add the, the JSON schema for any input or data that's gonna be utilized without within the workflow. So in our case, for this tool, that's just really one variable. That's the search query.

You know, we we're, we're looking up developer events, so we're gonna provide it a search query. That's the only input that the LLM needs to provide to this tool in order to get the information out of it. And then in the final step, we're just returning the actual information within this, uh, very easy, uh, format.

This is technically code in that you're returning JSON, uh, but it is really JSON that you're writing with, uh, English fields, right? So you're describing what this tool gave it. It's returning a list of objects that are matching what they asked for, and most importantly, it's telling the LOM to utilize those, uh, that that bit of information to actually return its response.

So if your LLM is configured to, uh, uh, do things in a specific way, it's gonna continue to do that, right? It's just going to utilize this information from the tool in whatever way is most useful for it. But if your LLM, you know, talks like a wizard or a pirate, it's gonna talk like a wizard or a pirate when it's giving you this information.

'cause we're just giving it a tool for that LLM or that AI to utilize, um, within that particular moment. So I hope that encourages you to do some mad scientists experimenting of your own. Um, there are opportunities for you to do so via the repository that I have linked to on this QR code.

And on the, uh, at the end of this presentation as well, I have another QR code that's going to, uh, have access to all of the slides I covered today. So you can go back through those step-by-step instructions. We're building those tools for your LLM.

You can easily get this repo and get started from there. So if you missed anything or want to go in further detail, it's really easy to do so. And so just to wrap things up, I wanted to just take a moment to conclude what we talked about and think about next steps.

Um, the most open one ended way to think about what we learned today is we figured out what that term is for the things that we do every day within chat channels with our tools. And that term is chat ops and, and we figured out how chat ops and AI interact with each other. What are the different ways that you can use, um, AI enhanced tools within your collaboration workspace in order to get more done with your colleagues?

And at the end, we went through some examples of creating really easy drag and drop tools that utilize whatever's in your backend or whatever third party system that you want to figure out. It can ping any server. It can also even just be fully replaced with a server of your choosing that's returning information based upon the tool you want.

But in our example, this is a drag and drop workflow. It's easy for anybody, even someone who doesn't have coding experience to actually create those tools with nation. And then they become exposed to the LLM.

And it allows you to utilize those tools in order to get better responses from your AI and help you and your teams do the work that's most important to you. So you can take the steps that I've provided here today to model that own change within your OR organization, get started with Mattermost and AI copilot, for example, or AI tools within the collaboration, uh, pro platforms you're already using, and figure out how you can give them the tools, the functions that they need in order to help you get your work done. So thanks for joining me today and I look forward to seeing any questions in the chat as well as following up with y'all in order to get in contact with me.

I have my info here, um, and as I promised, there's the QR code where you can grab my slides, uh, and dig in deeper on any of the information that I went through today. So thank you for joining me.