Laying the Groundwork AI as an Advisor – BMC Software

Hi, and welcome, welcome to our conversation about AI readiness in the mainframe environment. My name is Mitch Ashley, and I lead the software lifecycle engineering practice with the Futur Group. Today I'm joined by Anthony Dero.

Anthony is Senior Director of Architecture for AI with BMC software. Welcome, Anthony, Mitch. Thanks for having me.

You bet. Great to have you. Now, this is a three part series.

Our first part is talking about AI readiness, and the series is, uh, sponsored by BMC software. We appreciate the folks at BMC, uh, putting this on and putting this together. So, Anthony, let, let's jump right in.

So we hear a lot about organizations needing to be AI ready, especially for the mainframe environment. At the earliest stage, what does AI readiness really mean? Yeah, Mitch, this question, I can't tell you how many times I get this, whether it's I'm speaking at a conference or customer visit.

This always comes up, you know, how do we get going? How do we, we get started with that, and it's so foundational into a successful journey with ai, but yet it's a step that you'd be surprised how many organ organizations just kind of ignore or are not even aware there is a readiness, uh, you know, playbook that they, that they should be, uh, following. So it all boils down to, uh, from an organization perspective, you know, how do we roll in AI technology?

How do we use AI technology safely within our organization? How do we put guardrails around AI for, uh, you know, for protection against data? Uh, for example, you know, uh, from a, from a legal perspective, you know, uh, what policies and governance that we need to have in place.

Uh, we bring AI into our organization and there's all kinds of challenges around that. But at the end of the day, you know, that's one part of the organization's gotta deal with that. And then it comes down to the individual, you know, groups and, uh, departments within an organization and how they want to utilize ai.

So the first really good step in that journey is looking at AI as an advisor. Mitch, really look at it as like you would bring in a human into your organization, you know, based on their experiences and, and their background to have an a dialogue exchange with them about whatever challenges that you may have. And you're gonna lean on that person for their insights and guidance based on their experiences.

Ai, that's a great first step with AI image. Look at AI as an advisor. It's there to explain, it's there to guide, it's there to recommend, et cetera.

It's there to provide knowledge and insights that you may otherwise miss or not know how to surface. So from that perspective, that is a safe AI journey to start moving your organization to. But then the other side of that is the skills of your staff itself.

When you bring AI into an organization, you wanna make sure that your SA staff is skilled in AI usage. You want to make sure your staff is skilled and understand on where they should be applying AI within the organization. So there's some education and training that need to be done for your staff.

There's guidelines, uh, uh, and policies that you need to be putting in place, guardrails that you need to be putting in place. And that's all very, very, um, very focused on individual organizations and what that means. But that's the first step, um, to get that, those foundational aspects of AI in place.

That's a really good point about having that kind of direction you want to take with AI versus it's so accessible. We can use it, try it out, but how are we gonna focus and leverage it for the organization. And you mentioned the concept of AI as an advisor, using that as your first entree into ai.

Talk about how that is different than maybe automation, autonomous ai, agenda, ai, all the terms that we hear about, uh, doing things with ai. Yeah, so what, you know, when you do hear about autonomous AI and agents, that's all around actionability and the AI take, you know, perceiving a situation, making a decision, and taking it in action, jumping into the deep end of the pool when it comes to AI in that regard, that, that, that's concerning to a lot of, a lot, a lot of folks. So when we talk about the advise the advisor part of that, the advisor takes no action, right?

Again, the advisor is there just to guide you, nurture you, and move you along. But it's up to you, the human to actually take those actions. It's up to the team who's using AI to infuse AI with the right pieces of information to get the right types of guidance that they want from that AI system.

But that AI system is benign, right? Again, the AI system is not going to take any actions on, on your, your behalf. It's all back to you and what you want to get out of that, that AI system.

So if you're a developer, I'm gonna use AI as an advisor to maybe gimme code, recommendations, code, explain, um, maybe to do a best practices analysis on my code, et cetera. That's, that, that's really good. And maybe from the AI ops space, Mitch, we're gonna use AI as an advisor to oversee my, my dashboard and maybe surface insights to me out of that dashboard that I would otherwise miss.

But there's no actionability to it in that regard. It's just providing the insights and information so that that is, that is a part that fits very naturally into the advisor part of it, as opposed to the autonomy part of ai. It's good you mentioned that.

'cause it is a much more comfortable way to kind of enter into the AI space and start to use it, but you don't have to jump right into automation and agents and, you know, doing more of the, you know, advance things. If you wanna think of it that way. You'll build trust, you'll learn about AI by using it.

And we, and we've done that ourselves, right? You know, look over the last 18 months, whoever your chat provider of choice may be. But that's how we, we all got into the game of ai.

When, when, when, when, uh, you know, chat cheap PT was released as an example. We all went out there and, and started having conversation with AI at that point, whether it was professionally or personally, that experience was an advisor type experience. You know, we sent it a bunch of questions and we got responses back and we had a conversation and a dialogue with it, but nothing happened.

There was no actionability to it. So that was all of our entries into the AI world. And for organizations, for enterprises, that's a great first step also in the, in the start of their AI journey To that point, there are plenty of ways to engage with AI and query, use it as a tool, but what do you need to have in place to be an effective advisor role in, in the environment we're talking about?

Yeah. So one of the things that we've learned in our journey with AI so far, and I think as an industry, we've all learned just bringing a large language model into the organization, not enough, right? It's like it's, that's just, that's the bare minimum entry that you could do.

But the problem with just bringing a large language model into your organization is it doesn't have any context. Those large language models were trained on huge corp of information. They were targeting the masses of users, where once you get into an organization and you bring AI into an or into an organization, you're, you're in a particular domain.

You're in a particular realm. So now how do you, how do you utilize this lodge language model that's general purpose for a specific domain that you may be in? Well, the way you do that, and what we've learned o over the past, you know, 12 to 18 months, is you have to augment that large language model.

You have to augment it with realtime product data or whatever data, uh, realtime data that your, your organization is playing in. You also have to augment the language model with additional knowledge, whether that's workflow, knowledge, processes knowledge, best practices, knowledge. It's, it's your enterprise knowledge.

Whatever that means to you in your organization, you want to infuse that into your AI system. So then you have the large language model with your enterprise knowledge, with your real time data access, uh, knowledge. It's a combination of all three of those that brings relevance to AI with an organization because it brings relevant context into your organization and the AI perspective.

And when we're using AI advisors, and I agree with you very much about the point of, you know, contextualizing it with information about your organization. Where do you see the fastest value that can be delivered by using, uh, AI advisor in the mainframe teams today? It's definitely in the DevOps space by far that it, it's the DevOps community that has really opened their arms and embraced ai.

And the mainframe environment is no different, whether, you know, from the cloud environment to a distributed environment in that realm, the developers have accepted AI in the mainframe space. There's a, you see a lot of interest, a lot of adoption AI in the, uh, mainframe space. So that is, to me, has progressed us as an industry in the a those working in the AI space, the work that the development com community has done over the past year, 18 months has really accelerated our journey, uh, with ai.

Now, you also starting to see other areas starting to get really interested in that. The AI ops space, as an example, is getting, getting a lot of traction now when it comes to, uh, to ai. And we're heavily looking into that within our portfolio, in our AI ops, uh, part of it.

But it's the knowledge capture that is what's gonna play the biggest game here, why we're in this massive transition within the mainframe community. We have a lot of folks heading out towards retirement on the tail end of their careers. How do we capture that knowledge and how do we infuse that into our AI system so that next generation coming in has that experience?

They can lean on that they otherwise would not have that person they would go to, you know, Bob, Bob is not here anymore. But if we were able to capture Bob's knowledge in some way, shape, or form, and put that and infuse that into the AI system so that next generation can lean on the AI system and get access to the information that Bob had, that is game changer in our mainframe space. It's really, it's not only helps get that next generation up to speed, Mitch, but here, he, I I just had a conversation yesterday with someone about this AI on the mainframe is making the mainframe sexy and attractive to that next generation coming outta colleges and universities.

We're in the conversation, just like the cloud space in the distributed space when it comes to AI and technology advancements in general, that is really cool. It very much is the sense of excitement in the mainstream environment, particularly with ai. And I, and, and you have a really good point about that knowledge loss, you know, as folks retire, move on, whatever it might be.

So the next generation of people work in a mainframe, have got that information contextually available to them in ai. I can't think of a better application of ai. Yeah, absolutely.

And we hear that from our customers. Our customers are like, you know, we got decades worth of white papers. We got years and years worth of, uh, video recordings, training material, et cetera.

How do we capture that? How do we, how do we get that into an AI system? And that's something with B-M-C-A-E, uh, assistant that we, we, we took very, very serious, right?

So it's like, well, how do we do this? How do we allow our customers to capture this knowledge that they have and get it infused into B-M-C-A-E assistant? And we're delivering to our customers a tool that makes that really easy to do, uh, where they can, uh, manage documents, they can manage videos and build out their own knowledge base that B-M-C-M-E assistant would be totally aware of.

Now, when we ship our solution, we have the large language model. We have an AMY knowledge base that we ship, the customer can build their knowledge base, and then we have access to all of our product data. So we got all this information that's available to BMC AMY Assistant, that goes back to what we talked about before about what's relevant context to a customer.

Yeah. We can't talk about AI without talking about trust, and I've heard you discuss the importance of explainability. Yeah.

Talk more about that. Love to hear your thoughts about why that's so important. Oh, Yeah, yeah, yeah.

So with, with ai, of course, you know, trust always comes up in the conversation from the very beginning. We all started working with generative ai. That was the, you know, everybody was talking about trust in that regard.

It's multiple ways to answer this. You know, we have some responsibility in the solutions that, um, that we provide our customers. We gotta give the customers insights into what our AI system is doing.

We have to connect our AI system into their workflows and processes around auditing, logging, tracing, et cetera, observability in their organization. So how do we do that? So as an architect, from the very beginning, foundational, we have to be able to capture everything that is happening through our, uh, our AI system through BMC Amy Assistant.

From a user typing a prompt to us formulating a response, not only did it has to be auditable, but as much insight as we can provide on why we came about a response has to be clearly articulated. And some of that is clearly articulated back in the product experience. So when we give a response back, we may cite in that response where we, why we came to this conclusion and what pieces of information led us to the, to this conclusion.

But it also has to be totally, uh, traceable and auditable behind the curtain so that the administrators of the AI system have full optics into everything that is happening in that system. It cannot be treated as a closed door system. So it, it, it's the optic optics into the AI system.

It's the auditability, traceability, logging, everything has to be done. So if you go into the system, Mitch, and you are working with BMC Amy Assistant day in and day out, the system administrator has, you know, full trans full transparency into all the things that you've done with the AI system. And when, and, and customers have asked us for that from the very beginning, we started working with our customers in this journey that was foremost right at the top of the list.

They need to understand what's happening in the system and why. And we've done that. That's foundational for us.

That was something we had to put in at the lowest level of the architecture. That's not an afterthought. If, if, if you go with that approach as an afterthought, you'll miss things.

It has to be done at the ground level of the system. Yeah. That explainability of transparency is fundamental, that that builds that experience that you start to build that trust with very much so.

And it's that trust that's gonna lead us to, to, to the next part of the AI journey beyond the advisor where you look at AI as a true partner in your daily journey. You look at AI agents and agentic AI as a digital workforce doing work, and, but we gotta take those steps and build that trust. Speaking of taking those steps for organizations that maybe just starting out, thinking about AI readiness, what do you think are the smartest first steps to take?

We went through this journey ourselves. So, so we have a pretty wide and deep portfolio, which we within our BMC Amy, uh, product area. So we had to go through this exercise.

Where do we find true immediate value that we can deliver to our customers? The AI journey was new for us too. We had to be very careful, very systematic on how we approached it.

So the, the way we approached it was, let's just start looking at the low risk, but high value returns that we can give our customers with our AI infusion within our products, within our portfolio. And we've been very, very successful at that. But one of the key things, even though it's, you know, it may be a, a low risk, high reward type, um, AI enhancement, we want to be able to also capture and measure that.

You have to be able to measure and capture that to make sure you're truly getting your return on your AI investment. This model worked very well. I, I, I, I, I spoke to other architects about this model.

I spoke to customers about this model, and this is a really good entry point model. Start small. Don't try to drink the ocean, as they say.

Start small. Identify those low risk impacts. You don't want anything that's gonna disrupt your business, uh, on a day to day.

But then just start taking those steps. And before you know it, when your organization gets more and more comfortable with AI and you start building the trust with AI, and you start to get a good feel of what you can and cannot do with ai, before you know it, you're starting to take on bigger and bigger and bigger challenges with ai. And when you look in the mirror, you'll see yourself progressing pretty far pretty quickly with AI when you start that way.

Those are some great insights and very sage advice, I think. Anthony, thanks for joining us today. Thanks for being part of this.

Thank you. We really appreciate the BMC software team for sponsoring this kind of event where we can share this information, share some of our experiences, and bring up some of these important questions. So this concludes our first segment that we're doing in this three part series covering AI readiness.

In our second segment, we're gonna be talking about infusing intelligence with ai, using AI as a partner, using generative AI in the mainframe environment. Thanks for joining us. We look forward to seeing you on our next segment.