Planning for Resilience with Splunk’s Patrick Coughlin | AWS re:Invent 2023

This is Textron tv. Hey everyone. Welcome back here.

We are live in Las Vegas for our EWS Reinvent coverage. We are actually up in our studio in the Wynn Hotel, which is next door to the main exhibit, uh, area in the, uh, Venetia Palazzo. Really happy to be joined right now by Patrick Coughlin co Coghlan.

You got it. Okay. From Splunk.

And, and Patrick, welcome. Thanks for joining us. Thanks for Great to be here.

Great to have you here. And, and joining Patrick and I, actually it would, well, I'm joining them is our CTO and, and principal analyst, Mitch Ashley from Textron Mitchell. Absolutely.

Thanks for being here. Good to be here. Again.

You could be talking with Patrick. We've like connected here and there. Yeah.

A few Times. Now we get to have like a real reimbursement conversation. Have you techron tv Patrick, we should start with what, what's your title at Splunk?

Yeah, so I, I lead global technical sales at Splunk, which the, the easiest way to think about it is, is sales engineering. Mm-Hmm. The technical field.

Um, but also what makes it exciting at Splunk is, um, uh, we do a lot of our new product introduction through our technical field organization. So, mm-Hmm. When we have new capabilities, new use cases, new features, new products coming out, um, we bring them out through the technical field, which means I have a relatively loud voice in, in, in helping to guide, um, the product direction.

And also, uh, being clear about what the field readiness is and what the customer feedback is around new products. You know, kinda the tip of the spear. I was just gonna say, in many ways, the tip of the spear and getting things out there, but also getting the feedback.

Yeah. Mm-Hmm. Super important.

It's super important to Splunk to, to ensure that we have, you know, a customer centric view of what we're putting out. We release a lot of new capabilities all the time across security, it and observability, but if you don't have that feedback loop, you can be putting a bunch of crap out there that nobody wants to use. Mm-Hmm.

Agreed. And too big, too long to find out. Right?

Yeah. You can react more quickly. Talk a little bit about, um, the relationship with AWS.

Obviously not, not you, you've been a partner with AWS for quite some time. Tell us a little bit about that and how that's working And Absolutely. Uh, AWS has been a, a partner Splunk, probably for the better part of a decade.

Um, uh, one of our largest partners, um, a a lot of our Splunk cloud infrastructure originally was built on, um, Amazon web services, the first cloud that we went to, of course. Um, and now we've got a couple of other cloud partners, but AWS is still, of course, our biggest, um, but we're doing other things too. Um, we're innovating on, on new capabilities together.

Um, one key area we've been spending a lot of time talking with customers about is, is around this concept of, of federated search. Um, you know, the, the days are numbered where, uh, customers would, would pick one single product and put all their data into that tool. Mm-Hmm.

Or product or platform. Customers today want choice. Uh, they want to be able to store data in the most cost effective place.

Uh, and so what that means for Splunk is, is not all the data has to reside in Splunk, but we can still provide the analytical value on top of your data regardless of where it lives. And a great example of that is the partnership that we're building with AWS around federated capabilities around edge processor and integration with their Amazon Security Lake. Um, mm-Hmm.

We're also working with AWS on the open cybersecurity schema framework. Yeah. OCSF Mm-Hmm.

We love an acronym in security. Always. Don't we?

Always. So, uh, I gotta have one there. Um, and you know, we've got hundreds of companies that have, have joined in the partnership of that.

Um, I think we've, we've spoken about that in the past. Um, but it's, it's really creating a new standard for how to, how to represent, um, event data, um, that is, that is adopted across the cybersecurity industry. Um, and, and it's super helpful because one of the biggest challenges in cybersecurity is everybody sort of spits out their data and different structures and different formats with different ontologies and data models.

And that makes integration a huge challenge and a huge headache. Um, and when I talk to customers here and, and around the world, that's one of the things they want. They want choice about where the data should stay, should live, but they also want it just to work.

Yeah. They want their tools and their vendors and their partners to just work together. So those are a couple things we're working on with AWS.

Absolutely. You know, you used the term data lake a few times, and I know that's a big word here. I've seen it on the billboards, the flesh in our suite at night.

But, um, in my mind, data lake is almost an old word, an old way of thinking about it. It's, you know, like, I've got Hoover Dam and behind the dam is this huge lake of data. But what we really have is a tributary system.

Like right. Where data is, is spread out all over the place, and we need to be able to access it. And part and parcel of that is of course, that not all data's created equal.

Yeah. Right. Some data.

I'm going to keep writing Splunk because I need that data as fast as, and right there, as you know, close to the heartbeat. Other data I may not access as much, but I, I need it to be, you know, in my background, if you will, and I can afford to or at the edge. Or it's the edge, which is easier on.

Right. So it, I I think we need a new word rather than lake. Yeah.

And, and I'm not sure we'll get it in a single word, you know? No. Like we, we, at Splunk, obviously, we spend a lot of time thinking about data.

And if you think about the observability side of our portfolio Mm-Hmm. Where it's all about logs, metrics, and traces. Right.

And, and there's different tools even within the Splunk portfolio that you should use for different types of data. If you work, if you care about metrics and traces, our observability cloud is the right place to live. If you care about log data, you should be working in Splunk Cloud, Splunk enterprise.

Right. Right. Um, and so, you know, I, I agree.

I don't, I don't know if we'll find a single word, but one thing I've been thinking a lot about is, you know, for, for decades we've talked about people, process and technology. Technology. We talked about cloud transformation in the sense of people, process and technology.

Where's data live in that? Is it, is it sort of a subset of technology or do we need to have a fourth thing, a fourth pillar of how we think about adopting new frameworks in the enterprise? And and do we need to specifically, is it time to call out data as one of those pillars that we have to design around?

So I, I have a slightly different view on it. To me, data sits on top of all of those things. 'cause data's primary, you know, one of the, one of the good things, like there's ever a good thing about a pandemic, but one of the good things about Covid, silver lining.

A silver lining Good, good phrase. One of the silver linings for me about data was about covid, was that we got off the Schneider around AppSec. Everything was about the app.

Yeah. App, development App, security app, app app. And realized that the app is no more than a window right.

Into the data. And that, you know, to quote or semi quote Bill Clinton, it's all about the data. Stupid.

Right. It's about, I don't care how pretty your app is and what platforms it runs on, it has to be able to give me access and analysis of the data. Right.

'cause data's primary. Right. And I think coming out of COVID, that's where we are.

Data is primary. And you know, we've gone, I don't know, two and a half minutes. I don't think we've mentioned AI yet.

Yeah, we will. We'll, I'm gonna mention it. Go for it.

It's teed up because really what AI's useless without that, it's all about the data. Data. Yeah.

Right. That's right. And so, to me that is, that's what, so it it's almost not on an equal with people process and technology.

It sits above them. It's more important. 'cause without the data, you can have all the people processing technology we want.

That's right. It's the data and, but There's different types of data in there. Yeah.

You know, like there's the data that you use to sort of, uh, power the application. There's the data that you use to, to train the model and ai, and then there's the data that helps you be more resilient. How do you know how that app is actually performing for you?

Right? Mm-Hmm. An app is also useless if you can't predict predictably serve it up to your customers.

Right. So how do you, how do you ensure that you have the right telemetry coming off of that application, whether that is logs metrics or traces to, to ensure that it is resilient? And is it withstanding, you know, new kinds of attacks versus the things that we might know or new failure conditions Exactly.

That we haven't done failure testing On. That's exactly, I mean, when we say digital resilience, to me, what, what I get excited about is, is it encompasses both those things for the first time, like you just said, you know, attacks and, and failures in the same sentence, right? Where, where digital resilience says it doesn't matter so much anymore.

Whether it's a malicious compromise coming from an external threat actor or an insider threat, or if it's a, uh, a failure at the infrastructure layer or a service, um, or performance degradation. Your application, the outcome is the same, which is something isn't working and it's not being served up to your customers in the way that you designed it. Uh, and digital resilience is all about how do you sort of bring together the security, the it, and the observability capabilities that you need to provide more secure and more reliable delivery to your customers.

So I have to jump in and put a plug in because we have a, a webinar coming up. We collaborated on an ebook with Textron Research and Splunk all about resilience from an organizational standpoint, kind of talking about the people process technology as well as security and, and the operations. And a couple of great people that are gonna be on the webinar that come from both of those worlds Yeah.

And kind of have that conversation. So that's happening on the 29th, the 1:00 PM Eastern go-to November 29th. November 29th.

Yes. So if you're watching this live, You'll know That today, right? Today is the 27th.

Yes. So two days. But if you're watching it next week, go to see do the magic of the internet.

Yeah. You can watch it on demand And text on Exactly. com and either watch it live or, or see the on-demand version.

You can check it out there. You know, so I listen to all of this. And here's, here's what I hear though, too, and I think a lot of our people out there, here at too, Patrick, is it seems no matter what we do, we just generate more and more data.

Oh, yeah, yeah. Absolutely. Oodles and oodles of data.

Can they, going back to the AI thing, can AI help us with this? Yeah. I mean, are we gonna drown in our data one day?

Yeah. Yeah. Well, I mean, I think it's, if you ask a lot of people here, they probably already are, you know?

Mm-Hmm mm-Hmm mm-Hmm. And so, um, there's, there's definitely an, a challenge in keeping up with the pace of data generation, but the, where AI comes in, and, and I'm, maybe I'm a bit controversial on this. Um, you know, there's so much hype around artificial intelligence.

You can't swing a dead cat down there without hitting somebody who's talking about it right now. Mm-Hmm. Um, what I think is exciting about artificial intelligence is, and, and what makes it a little bit different from tectonic technology shifts of the past, whether that was the move to the cloud that started this whole conference Mm-Hmm.

Or mobile or virtualization, or going back further, the internet itself. Um, you know, there's a few things that happen with these tectonic shifts, kind of regardless of the hype, which is one of them is it creates new attack surface area. Um, you know, the move to the cloud created this whole new sort of sense of mm-Hmm.

What, what's the attack surface area there with ai? What I, what I find interesting is it's creating this new attack surface area around training data. You know, 18 months ago, nobody was talking about protecting their data, their training data as a real differentiator for enterprise value.

But when you think about AI roadmaps today, it's all about who has access to what data and how are they protecting it and securing it and ensuring that there's, um, uh, protects from privacy or whatever it may be. Um, so I think it opens up this conversation around what new attack surface area has actually been to open with ai. And then the second thing that happens inevitably is what's gonna happen with compliance and regulation, right.

Whether it's cloud, mobile, internet. Sure. AI will be the same.

And, and so as much as we want to get hyped up about how this AI thing is totally different, and it's going to either save the world or end the world depending on who you talk to, the reality is there's some things that are the same. And maybe we should talk a little bit more about what can we learn from the move to the cloud that is AP applicable to this new crazy wave that we're seeing with ai. Yeah.

It isn't that all jobs are going away, right? I mean, still have a lot of people still putting, installing servers and data centers. Everybody thought that job was going away.

Absolutely. We thought that job was going away with the cloud. Yeah, Exactly.

And it hasn't, but, you know, one thing I wondered, because you mentioned about data is, and That job may have gone away in some places, In some places, but it still performed in Other ways. But, but then the people necessarily didn't just sort of fade into the background. Right.

Those people it your SREs of the future, right, Exactly. Data center ops. Yeah.

Data center Is massive and data ops, et cetera. Yeah. You know, one thing I'm wondering, you talked about, about data in the training model, one of the things that seems like potentially happens is that with domain specific models where we're putting our own, not generally kinda scarf off the internet, but our corporate data, our IP it seems like that greatly increases the value of the trained model, right.

And something that we have to protect. Yeah. Yeah.

Because now if, if you, that gets out from outside of the enterprise, that's not just your data, It's, that's intellectual and It's challenging. That's data set that can be done things with Right. With.

Well, so I, I think, you know, it's funny, I was reading an article just before we came on here, so Google, don't shoot me. Um, Google, they're still around there. Here.

Yeah. Who, who, oh, I'm kidding. No.

So, you know, Google came out with the standalone bot, spider, whatever you want to call it that, um, can help block your data from ai, LLM mm-Hmm. You know, Borg Borgen Right. Or whatever, what they call it from being in it from assimilated.

Yeah. It can like shield it from being Assimilated so it be used. Yeah.

Something like 180,000 sites have already signed up. Yeah. Oh, yeah.

To have their data shielded from the board. Yeah. Um, I don't mean that seriously, it's not the board, but shielded from, from these LLMs.

And I think Mitchell, that's gonna be a major thing. com for the last 10 years, I have Yeah. 30,000 articles.

Yeah. Make that an LLM that runs on top of a larger LLM. Yeah.

And use that just for my paying customers or registered customers to, you know, about DevOps. Yeah, that's great. I certainly don't want to make that available to everyone, because that's my ip.

That's right. Yeah. So I think that's, I think Mitchell, I think that's what you are really the talking about.

Exactly. And we need vendors like Splunk who are gonna help us do that. Well, and what Do you need to secure those things?

It's not terribly different from the things we've needed to secure cloud assets or to secure data centers. You need visibility. You need predictive detections and anomaly detections that where things deviate from a baseline.

We need remediation and response playbooks. And ideally with more automation infused into that. I mean, this, this is the portfolio of Splunk, but, but it's also, we don't need to pretend like we need a whole new framework No.

To, to, it's not start all Over, bring this innovation into the enterprise at Scale. Absolutely not. But then there's a second use case I think, which is to this Google standalone bot, which is, Hey, just because it's on my website doesn't, and, and this goes back to the be and I was here when the internet first went commercial.

I remember fighting this fight, just because it's on my website doesn't mean you could reproduce it, use it. Yeah, that's right. Abuse it and, and do whatever you want with it.

It, it's still my ip. It's kinda like that open source model, right? Yeah.

It's my ip. You could look at it, you could maybe see the source code, but I own it. That's right.

Yeah. And, and I think as a society and as an industry, we haven't quite figured out where that border is about. Just because you could see it on my website doesn't mean you can synthesize it where it's in a repo and GitHub or it's anywhere on anything.

Right. Well, you could use it it to spit out when some kid from college is looking to write a report on, on something. Right.

That's right. It's my IP still. I it needs to be at attributed back.

It need, you know, but, But we're in the early innings here. I mean, so, so Google, Google came out with this capability, which is fantastic, but isn't it amazing how well the market works sometimes? Like this is a concern we're now seeing innovation around it.

Right? Well, and that is the beauty of action response. Yeah.

Action response. And like, this is this, I mean, I'm a cybersecurity professional by trade. It's all about action and response.

Mm-Hmm. Attacker makes one move. We try to make another one.

Yep. So I, I have, and call me naive, but you know, in the same breath where I say, keep calm, we can learn from this. I also say, you know, there will be innovation.

The pace of innovation on the AI front is, is terrifying in some ways at how fast it's moving. But at the same time, the market is a pretty incredible thing. And, and we can sort of do this action and response innovation where we can step to the challenges.

And I think the next front for that, where we're already seeing some really interesting innovation is around explainability of how these, how these AI models work. Mm-Hmm. You know, there was this concern for the last, you know, year that, oh my God, these are all black boxes.

We'll never even be able to say how they work. How are we supposed to scale a technology into the modern enterprise? How are we supposed to regulate a technology that we don't even know how it works?

Well, there's breakthroughs just in the last couple of weeks around how we can actually measure and, and articulate explainability in models. And I think we'll see more innovation come around that It's kind of a, a, something we can lift or, or apply from open source, which is one of the big values in the trust in open source is the visibility. Yeah.

Transparency. Exactly. And whether we're gonna open up all models and be as transparent parent as the source code and in a repo, I don't know.

But, but even that transparency of what's going into it, how, maybe how about how it's being trained and knowing what's in the model. It's supposed to a black box. 'cause that's, people don't trust that.

That's right. That's right. But even that, like, like right now, CIOs, CTOs who are bringing AI products to market are, are having this debate of do I use an open source model or do I use a closed source model?

Well, boy, that's a debate we've had for decades around other technologies. So let's not forget that we know what the trade-offs are in those things. Mm-Hmm.

And sometimes it is visibility, it's community leveraging a community versus, you know, potentially control and, and, and perception of security around closed versus open. And, and so let's take these things that we've learned from the past and apply them to the future for ai. Absolutely.

You know, it, it, so I've been a, a student and a fan and consumer, an advocate for open source And author. An author for a great, yeah. For A long time.

Yeah. And I, it, I, to me, there's a pendulum that swings here where, look, three years ago I really felt we were in like a golden age when it comes to open source. 'cause open source was the, the Shazam.

Right. Everybody OpenTelemetry was the, you know, the model of How it auction OpenTelemetry. It's still the second biggest.

It still is Project in C and Sunk Splunk is the single largest contributor Attributor. Absolutely. But, but we're starting to see companies pull back.

Right. We're seeing companies change their licensing. Mm-Hmm.

Red Hat and IBM. Yeah. Red Hat is IBM.

Yeah. Mm-Hmm. Um, red Hat and IBM right?

Yeah. Changed their licensing, uh, cars Hashi. Yeah.

Yeah. Corp changed their licensing. People are recognizing that there's a, there's a time and a season for all of these things, but they don't, does it just open source has a lot of advantages, but it's not for everything for everyone at every time.

And that's The same way all the time. I think maybe the pendulum swung a little bit too far. Right.

And now it's coming back and now It's coming back to where it should be. But things so Equilibrium, things Like OpenTelemetry to me, they, they solve such a foundational problem for the modern enterprise of today, which is I'm tired of all this agent creep. I don't like to lock in.

Right. We're All doing the same thing. There's no real differentiation.

That's very Simple use Cases around that, that, that make that such an obvious future proof decision. From where I sit, there's other open source, like, I'm not sure I would be building on top of an open source AI model right now, when you look at some of the, the vulnerabilities that are coming out around that, I, when you look at who's behind some of the contributions to those, those repos. And so you do have to be careful about it.

You don't want to, you don't want to go in blindly. Don't Bet the farm quite Open source all the time. Only I think that's a mistake.

And we were, we were swinging there, but there are obvious places where open source makes such sense for the enterprise, and Tel is one of 'em. Yeah. I, I don't disagree at all.

And I think actually the observability space in general is a great example. Yeah. Of, look, the table stakes can be open source, right?

Everybody anties up the table stakes, but what what differentiates companies from each other is what you do on top Of them. Well, so that's exactly it. And like, as a security person who's somewhat new to the observability space and has always felt like these worlds are gonna converge, uh, hoping they converge faster than they actually are, you know, it's something that I admire about the observability market that the security market didn't have was, you know, we would all compete over our different endpoint agents.

Mm-Hmm. You know, and one's better than Or agent list. No, no.

On agent list list. But what is not code, it's not an agent or scanner list. It's not an agent.

Just some code. It's a lightweight scanning lightweight agent. Right.

So we would try to hardly eats anything, but that's It. We would try to compete over what was really the commodity technology instead of, instead of going to the abstracted layers where outcomes and value really, really was delivered. We, and yet observability kind of leapt, leapt over that and said, look, there's some stuff here we can all benefit from.

Let's use it and let's, let's get busy going to work for the customer. You know what we, I I think we see this with the open store with the, uh, software supply chain security and the SBOs SBOs as well stuff. Yeah.

Right. Because, you know, the concept of the SBO seems simple enough. Yeah.

Implementing it is a B***h. Implementation is a b***h. Because you know this, we all talk different languages.

That's right. We all, you know, our, the dependencies and using it is what's really, you know, even more difficult. Have you seen anybody doing cool stuff there?

Um, yeah. I, I, I will tell you some good friends with, I call 'em the Chrises, Chris Weal and Chris Eng. Okay.

Over at Veracode. Yeah. Veracode.

Um, they, they got, so what their, their attitude is to me, what they're doing with Sbam is with a friend of mine. Uh, it's actually I saw Security Bloggers Network. Uh, it was a company called Feed Burner hundred years ago.

No, I know Burner. Yes. Was the CEO Feed Gator was the original origin of it.

No. Feed Burner was original. Like RSS ingest Because back then Yeah, I remember you had RSS, you had R Ss one oh RSS two.

Oh yeah. Simple little RSS that RSS this RSS it was a tower Babel RSS where no one, just because you had an R Ss feed didn't necessarily mean I was able to read it in Feed Gator or something. No, that's, that was the readers.

Those were the readers. Feed Burner Normalized. It normalized them all into like one Super thing.

That's what made security bladders. And That's how I got to Dick Coastal who went on to become a CEO of Twitter started Feed Burner and they Yeah. Came to me.

And so my friend Brad Feld, the friend of Mitchell and I, Hey Brad Feld. Sure. So Brad said, Hey, my friend, he was an investor.

Yeah. Brad was an investor in all those, but he said, Hey, my friend Dick wants to put together blogs of a certain, you know, of a common theme so that he can inject advertising. 'cause he's gotta make money from this feed burner thing he's doing.

Yeah. He said, I'm gonna do one in the VC world, would you mind doing one in security? I said, sure, I'll put it together for you.

And then, um, like three months, five months later, Dick sells feed burner zero revenue, huh. For I think 80 or a hundred million Really? To Google.

Yeah. And then he went off to help them do Twitter. Google came back to me about six months later and says, Hey, Dick said you helped him with this security bloggers thing.

We don't know what to do with this. Yeah. Would you mind taking it back?

I said, yeah, take it. You know, and I, that was 2005. Wow.

Something like that. And I've been running it ever since. Um, but it's that same sort of, we need that normalization layer that someone's going to do it, Somebody's gonna do it.

But, but like the fact that we have to do it is part of the problem. Yes. Like, and so, so that, that, that is why, you know, initiatives like Open Cybersecurity Schema Framework and OCSF, which is, which is, you know, big partners and big players in this space, like AWS and I like the Think Splunk and Sure.

And, and Salesforce and Cisco and others have come together and said, Hey, we gotta stop making this a customer problem to solve, or a downstream tool problem. Like we can, we can solve this if we just put our people together in the same room and create an ontology, create a data model Yep. That we can all get on board with and, and stop creating those, those, uh, blockages in the actual systems architecture that, that we can solve.

I like, I like the test you just said, which is we have to stop making the, creating this that is a CPU a problem the customer must solve. Yeah. Let's, uh, let's us figure out how to make this go away so our customers aren't burning their Cycles.

We, we saw this also at, at Tru. It's A great way To think about it. Which I was, I was a co-founder of that before Splunk.

Splunk acquired True Star a couple years ago. Mm-Hmm. We were doing the feed burner for threat intelligence, which is, you know, every different threat Intelligence Had its own, Had its own sort of data model.

They have their own sort of risk scoring, you know, virus total scores on a scale of one to a hundred. CrowdStrike does it with high mediums and lows. Digital shout.

Everybody was doing it a different way and the customer couldn't use it. It was choking when it was getting into the sock because you couldn't automate around all these different data models. There's no, there's no equation with that crazy stuff.

Yeah. From Feed, burger to ai. Yeah.

We've covered, we've covered it all. And o tell And O before I got, I, I know you can't talk a lot about it 'cause it's depending, but we, we, we would be negligent if we didn't say congratulations on the, the Splunk acquisition by announcement. Yeah.

Yeah. I'm sure it's gotta open up all kinds of possibilities for you and your team there. Yeah.

I mean, we have that kind of scale. We're Super excited about it. Um, you know, our, our, our CEO, Gary Steele and CEO of, of Cisco, Chuck Robs just came out with a, with a blog post around it that, that I think hits all the high notes.

The reality is, you know, our customers for the most part are incredibly excited about this for this, for them, it makes sense. You know, when you look at the capabilities across security and observability and, and how we can accelerate our investments in things like artificial intelligence as part of Cisco faster than we, than we could have done as a standalone company. I think the future is super bright.

It fills a big need for Cisco as well. I mean, huge, hugely valuable to Cisco. I, you know what, talking as a security guy, I think it's a good thing for security too, because think so too.

They already had a decent security Yeah. Portfolio there and putting Splunk into it at a very that level. I'm, I'm excited to see what comes outta it.

It makes that connection between ops and security that we talked about. It does. Right.

Can you imagine that How how we can Traverse across all those different teams and capabilities and the enterprise. And look, let's be honest, the last couple years has, has taught CIOs, CTOs, CISO around the power of tool consolidation and Information sharing. Yeah.

And, and, but make sure you've got a couple of strategic partners in your, in your, in your suite of vendors that you work with. There should be a couple that can provide you end-to-end capability across more than just one point solution. And when you look at Splunk and Cisco combined, I think you see, uh, a really powerful strategic partner, even more so than we are today.

It's been A lot fun chatting with you. We hope we'll you to this s ECC suit, but that's a whole nother story. We haven't gotten that.

No. Anyway, hey Patrick, thanks for stopping by and talking to us. Thanks for having, I hope you enjoy the rest of your week here at AWS Reinvent.

We're gonna take a break here on our Techstrong TV coverage from Reinventing Las Vegas.