The Hidden Risks Associated With AI Use – Techstrong AI Podcast EP50

Hello and welcome to the Techstrong AI podcast. I'm Amanda Razani. I'm excited to be here today with Rob Juncker.

He is the Senior Vice President product and engineering at Mimecast. How are you doing? I am well Amanda, and thanks for having me today.

Thanks for being on our show. Can you share a little bit about Mimecast with our audience? You bet.

And first and foremost, it's been an exciting time here for Mimecast because for a long time we've been known as email security yet last year we spent a lot of time in the industry looking at some of these advanced cyber threats that were hitting our user base. And as an organization, we've shifted from email security now into not just doing email security, but focusing much broadly or on this whole human risk management problem and how do we make sure that humans don't fall victim to either attacks that are coming their way or alternatively human error that causes them to put an organization's data as well as, um, reputation at risk, if you will. Um, so it's been an exciting year for us, a lot of year to change.

And also around the AI spectrum, you've gotta admit there's been a lot of, uh, change here as way in which people collaborate and use this technology as well. Absolutely. And it's advancing so rapidly entering into so many different use cases, which brings us to our topic of the day, which is AI generated content, which has revolutionized productivity, but comes with hidden risks.

So, uh, you know, this is a great week to talk about this. Uh, we have quite a few newsworthy events this week. Can you share what you're seeing from your experience about the enhanced risks that are associated with ai?

Yeah, you bet. I mean, when you think about ai, it really marched onto the scene here just, you know, years ago at this point, but really hit the mainstream here in this last year. And what we saw was organizations across the board to start figuring out how do I take AI and bring it into my products, but how do I also bring it into my organization in a way that massively increases my productivity, right?

For shorter multiplications on productivity, make my users, um, you know, have less errors, but then also, um, tune it as well. And you know, even this week as we talk about the, the little bit of news that came out around deep seek, um, what we are finding is that most organizations now are rapidly adopting AI technologies. And to be honest with you, that's fantastic.

We all want more productivity and many of the, the things that AI allows us to do allows us to focus on the more strategic portions of our job, as opposed to some of the things that we're all asked to do that are very, you know, basic repetitive tasks that, that are there. And I think as we, what we've seen in organizations now in this rush to move to AI technologies is that they're all being faced with the question of how do we begin to adopt this technology in a safe and secure way into our organization? And Amanda, you gotta admit, even from like every chair right now as you look at organizations, it started off with one particular use case.

Maybe an organization said, how do I use this to write copy? And it's actually evolved into almost every single position organization having a different series of use cases for AI that allow them to achieve that productivity. But also if it comes to challenges too, Absolutely.

I use it for quick summarizations and advice to learn about a topic real quick. It's great. I mean, the use cases are unlimited, I feel like.

So, and Amanda, isn't it funny too, like as you bring up those use cases around summaries, in some cases we can't even get out of AI's way now. Like we'll join a Zoom and it'll say, Hey, your AI companion has joined as well. And it's like, it's naturally becoming something that's invading us, but also being super helpful in that regard too.

Oh, yes. It's so helpful when in regard to transcripts too, not having to type those out. It's great.

So with that though, do come these risks, what advice do you have for business leaders to sort of avoid those risks? Yeah, yeah, but let's talk about the hidden risks first because I think that's where things get interesting, right? And as you start thinking about the personas and the roles that we all perform in an organization, as an example for a product and engineering leader, I'm constantly dealing with proprietary roadmaps, timelines for deliveries, product messaging, engineering documents, intellectual property that comes from patents and other things that we're looking at.

And, and first blush, it might make a lot of sense for me to fire one of those things off to an AI engine, say, can you help me improve this? Or what things have you thought are you thinking of, you know, that might be able to extend our, my ideas or improve upon them? The reality is though, is that so many people today have no policies, no controls, no kind of documented procedures for AI in their organization.

That very quickly, if I'm not careful, and if I'm doing the easy thing to accomplish my job, I would choose an AI model out there that might use my documents to learn, um, some of the new proprietary things that are coming out and I expose my organization to risk. And at the same time too, and heaven forbid, and we see this all the time right now in some of the risks that we manage here at Mimecast, people might actually take a long document and say, read this, summarize it for me. But in the process of that long document, what they don't see is that there's all this hidden intellectual property around customer data deep into this a hundred page document that also puts customer data at risk, right?

And all of these factors are those hidden risks that, you know, organizations need to be aware of. Now, as you talk about best practices and, and some of the things that we need to be focused on right now with ai, um, is that we need to really ask our CISO, ask our security leaders, ask our CIOs, what is that privacy policy? What is that AI policy?

And what controls do we have in place as an organization to adopt AI both responsibly, but then at the same time ensure that we're following best practices where data protection is also being honored for our customers, um, as well as our roadmaps and any other intellectual property that we have in, in that organization. And I think that that's really where the rubber meets the road and the crossroads now, um, is coming together that these leaders in security need to figure out what AI models do they need access to, how do they get private models to them that safeguards those data privacy concerns that they have? Um, and then make sure that the organization is following those paved roads where you've said, this is our AI choice that we're making and we're staying true to 'em.

And you know, I I will say this, Amanda, it's funny because here at Mimecast we've got a couple different models and a couple different products that we use, and it really becomes easy for a user to say, if one of those doesn't meet my demand, like how do I go out and just grab the next one? Because is anyone looking? And this really gets around to the second challenge.

Not only are there hidden risks in the way in which we're operating with these models, right? The second bit of this is like, how do we police users and put controls in place to make sure that they're not going off course and potentially exposing your organization to data risk by using an unapproved mechanism as well, right? And I think all of those come together.

If, if we identify the fact that AI is important to our organizations, and by the way, for everybody on the line, you're not adopting AI right now, figure out how right figure, um, because you're gonna be left in the dust. But then the second bit of this is how do you responsibly adopt AI into that organization? Um, and make sure that you're not putting your data at risk and the vital data of your customers that you're managing as well.

So in your opinion, is it safer to use AI products that are developed specifically for a company rather than open source public AI tools? Well, I think what's important for organizations is if you're going to use something that's open source, right? Just realize the risks that you're running into.

There's plenty of ways that you can take those, open those open models and bring them into private usage for you so that you're not actually exposing your data into a global model that anyone could else tap into. And a lot of those commercial agreements that you can reach with those vendors allow you to be able to keep, um, uh, that that level of privacy, uh, associated with it. But the second thing I will say is this, is that for all of those models that are out there, some of those models are tuned and designed for very specific pur purposes, whether it be, you know, a general purpose, you know, GPT that you've got out there that you can ask generic questions to.

It's ones that are designed around marketing best practices. And I've seen, you know, AI bots out there today and some of the things that we've looked at where we talk about low fidelity versus high fidelity, right? And the more and more organizations have to build models that are for everyone, the lower the fidelity of that model becomes.

But the more and more you can train a model on who you are, what you need, what your purpose is, and it's a purpose built model, those have a tendency to go really far in your productivity. But ironically, the higher fidelity you get, the more you're going to expose things like, tell me how to develop a customer communication plan for this specific customer who has these kind of environments in place. And the higher fidelity you go, the more there's likelihood that you're gonna expose vital information out there, um, to those models as well.

So with, with, with more capabilities comes more risk from a user perspective as they go deeper into that. Um, but we even see that, you know, across the board today as we look at all of our emails that we're, we're looking through here at Mimecast, I mean, we're readily easy, readily able to add, uh, identify that more than 8% of those emails now are completely AI generated. Which if you think about the 180 billion of data points that we're looking on on a daily perspective, that's a lot that people have turned towards those models to get hyper fidelity out of them as well.

Do you think as AI becomes integrated in everything and we're using it both professionally and personally, that we're taking, um, a lot for granted and um, and uh, we're becoming more and more and more at risk and um, there I'm seeing a lot more, um, scams via ai. Okay. Uh, so what are your thoughts on this?

Yeah, you know, and it's great because I think one of the most important things we've learned through security is that security comes from a defense in depth approach. And there's going to be times, especially with some of these AI tools where Amanda, if I wanted to target you with very specific email and I know enough about you through your social media profile, I could probably create a phishing email through AI that is almost indiscernible to you from being phishing versus actual content that could be coming with something from, you know, your hobbies or things along those lines. And this is where the defense in depth gets really important, right?

And I guess as you bring about this whole human risk management vision, part of what we're focused on is not not just protecting Amanda from receiving that phishing and malware email on the front side, but we're also putting controls in place so that if for any reason that you may be on your personal email, click something that we're able to identify that you've been put at risk, right? Or you've actually been compromised, and then take actions to secure your data at that point. And AI is no different, right?

I think all of us now, and we just went through a massive ISO certification here at Mimecast to prove that we're using AI responsibly. But as we go forward from here, what's important is that those, that everybody who adopts AI is getting those certifications as well as those compliance controls in place to ensure that as they embed AI into their products and, and their offerings, that we're securing our customer's data as best as we can to the compliance. But I think as long as people stick to the, to those standards, stick to the compliance controls, focus around defense in depth, where we're gonna protect Amanda from email all the way through every action that you're doing at that point, you know, I think we're gonna be able to keep this under wraps in control and also create a more productive workforce out there.

Absolutely. Well, if there was one key takeaway you could leave our audience with today, what would that be? I would tell every organization right now, like AI adoption is happening, right?

And I still see some people and I talk to some people who are dragging feet on that one, right? And this is the time that you need to get out ahead of it. Establish that AI steering committee for your organization, determine what your baseline is for that assessment, and what does your organization need to have that hyper productivity through AI to pull that together, develop a company-wide policy for AI to make sure that you're actually covering those controls, set those cybersecurity standards, and then implement the compliance controls to keep the human safe from possibly exposing your organization a data risk.

And if you do those four things right there, um, I think an organization not only will responsibly be able to adopt AI while also protecting their customer data, but I think there your organizations, we're gonna see great innovations coming out through using this technology to help us all better, not only ourselves, but the work that we do at work. Wonderful. Well, thank you so much for coming on the show today and sharing your insights.

My pleasure, Amanda. Thanks again for having me. All right.

And thank you to our audience. Stay tuned. There's more.