Detecting Security Flaws in Software – Frank Huerta, Curtail

This is Textron tv. Hey guys, thanks for the room. We're here with Frank wta, the C e O for Curtail, and we're talking about how to figure out whether or not there are vulnerabilities and flaws in your software the easy way.

'cause today we all seem to do it a little bit too much on the hard way, which is somebody else discovers it after something bad has already happened. Frank, welcome to the show. Thank you very much.

Walk us through, why is it that we are so challenged in figuring out where all these vulnerabilities and flaws are? I think we all know that any piece of software written by a human as has a problem. It's generally gonna have some mistake, but we seem to be enabled to kind of find this stuff until there's a crisis.

So is there a smarter way to do this or to think about this? Sure. Mike, I, I, I think there is, um, you know, one is the, the software we have, it's grown.

We have a lot of software out there. Um, it's complicated, but also we really want to get things out fast to users, right? And there has been this, uh, you know, belief that, you know, you can move fast and break things.

And so maybe if it's not, you know, fully baked, we can still throw it out there, right? But that we do get burned by that a lot, right? And that certainly has a lot of costs, um, has a lot of, you know, um, reputation issues and things like that.

So, um, you know, we do try to, to, you know, as software publishers, we do try to be very careful about, you know, what we do and, and, and use the, you know, the latest and the best types of testing, but we also want to be expedient about it, right? Um, and so we use commercially known tests. We use, you know, security testing, dass, dynamic application security testing, sass, we actually look at the code, you know, um, and then we also have homegrown tests.

We have our manual tests, but most of those are static in time, right? So they are looking for known things. So they're saying, Hey, I'm testing X to find Y, but if it's the test, if it's not, if, if, if X isn't in there to test, you're not gonna find y.

And so in inevitably, um, we miss things or we have regressions, we're constantly changing software and we change something and everything looks fine. And then, you know, we put it out into production and it breaks, right? Or, and usually that's found by customers.

And, um, you know, because customers do weird things with your software that you don't, obviously you, you don't anticipate a lot of times. And, um, that's really where we started, you know, believing we could make a difference. My history, I don't know, Mike has been in computer security.

I founded co-founded a company called Recourse Technologies, where we, um, we were actually the first company to commercialize honeypots, but we were focused on finding zero day, um, attacks, if you will, you know, novel attacks. Um, and you know, you, you understand the security problems and you say, geez, you know, we really want to, we don't want to have let these bad guys in, but the bad guys really are exploiting bad code in a lot of times. So if you could say, Hey, could we move back up earlier in time and help ourselves write better code so that these things couldn't get exploited earlier?

And could we find an effect zero day bugs, if you will, right? Those bugs that you can find beforehand. Most of the testing we have out there, as I said before, is looking for known things, right?

And even a lot of the AI based things that you're doing, right? Ai a AI is really still relying on known data sets, right? It's like the i it is like, hey, we have a big enough data set where we can approximate things that we've seen before.

But what if you haven't seen it before? What if it's been designed to specifically evade those things you've been seeing, you think you've seen before? So at curtail, we've developed a process to, um, a technology to actually find those, those flaws, or at least see the potential flaws that might happen from your customer, normal customer interactions.

And we do that by a fairly simple process where we actually compare your current version of software that is running, maybe, you know, maybe your production version and maybe say your next version coming up, uh, it's your release candidate and you can actually play your customer traffic at both of those. Obviously one is already running, and you can see the effects of how that customer traffic will, um, impact that new version, right? And you can see what changes have been introduced.

You can see if it will be running faster, you will can see or slower, and by how much. And we can do that on a request by request basis. We can see if APIs have changed.

We can see if security vulnerabilities have been introduced in certain cases. So, um, you know, um, CloudFlare had a, an issue several years back where they released a, they did an upgrade, released some code, and, um, it was releasing p i i for six months. Um, with our technology, we could have found that before they released right?

Customers eventually, actually a Google researcher eventually found it. Um, but these are the kinds of things that that, that we could actually help find and, and, and alleviate the pain before it becomes an issue. Um, and, uh, so it's really can, can bolster and supplement the current testing we have because testing isn't, like I said, a static thing.

It things change, right? So the test you used two or three releases ago, they may not be any good anymore, right? Um, and so how do you accommodate that?

And that's what we're trying to do at Curtails is, is bring in real life, uh, data, real life interaction with your software and see how it will, how it will change, How far left does that process get? Am I doing that say within the I d e of the developer or is it more embedded within my CI/CD platforms? Where does this all happen?

Yeah, this can take place at any point in your CI/CD pipeline. So it can be used as you said, shifting left. You could actually have your developers compare the new version of software they're coding as they're, you know, bringing in, you know, writing in new code and play it against a known, um, a comparison of the last software and see what's changed.

Uh, think of the, of the case of a refactoring, Hey, I'm, I'm trying to rewrite this fragile piece of code over there. I want, I, I want it to still function the same way I'm, I'm, uh, it does now, but um, but, but I need to rewrite it, right? And I can, you know, I can't break.

And so as you introduce new changes, you can see that the code is still doing the same thing as before. We can actually help you detect that. We look at the network responses from each of those and saying, Hey, if I make this change, is the code still doing the same thing as before?

We can also be used as a QA test, right? So we can say, Hey, you know, as I explained before, we've got a good release candidate. Let me see how this is actually gonna perform, how it's going to work, um, compared to our current production version that's out there.

Um, so really any of the CI/CD pipeline, and it can actually be used even after the fact. So you do your new release and then you run the, the old release, um, you know, the prior product release that was in production. You continue to run that in shadow, if you will, and look at the interactions of both of those and such that if you have a problem with that new release and you're gonna have to, um, roll back anyway, you can actually see where that problem occurred.

And then from a root cause perspective, we can actually tell you where the actual, um, what requests caused that problem. So when you're tracking it down, you can say, oh, this kind of request caused this issue, this change. We can go find that.

Mm-hmm. What do you think is the issue that we have with testing in general these days? It seems like we are building more code than ever.

We're shipping it out faster than ever, and then we're hoping to kind of maybe fix it in the upgrade cycle. As you mentioned earlier, it almost feels like we're experimenting on the end customer, and I think that their tolerance for that is probably dropping. So what is your sense of why are we a little bit, um, not so inclined to test as aggressively as maybe we should be?

Well, as, as I said earlier, there's, there's a lot of pressure to get releases out quickly, right? Um, there's demand and just, you know, I gotta move this software. Um, I've, you know, I, and so, so there is a lot of pressure being felt to get that software out there.

And then also, um, you know, it's the tests that you're running. So, you know, most people are well-intentioned. They want to have good quality software.

Um, but a lot of the tests are, um, are aren't comprehensive enough to cover the problems that might be out there. And, um, and, and, and so that's, that's part of the challenge, right? You can, you know, that's why we talk about these zero day bugs, right?

You can, you can have all of the, you know, the manual tests and the, you know, the commercial tests that you want, but if they aren't designed to find the bugs that are in there, they're not gonna find them. And so you need different approaches to see how those bugs might surface beforehand so you can fix them. There's a lot of focus these days on all things SecOps and some of what you do addresses that and other parts of what you do seem to be a little more on the quality assurance side.

Do you think that quality assurance and security testing is all gonna converge into one motion? 'cause ultimately it is all about the impact of the experience? Yeah, I mean, we're certainly, uh, true believers in SecOps.

And, and really, you know, at least the way we interpret that is that, you know, the developers, and we talked, you talked about shifting left a little earlier. The developers, um, are really not only concerned with their own development, but they're concerned with the quality and security of the software. And so if you can start doing those tests or, you know, trying to screen for some of those problems earlier on, you won't have to find out those problems later in production or try and do, you know, even if you're testing for them later, it's more expensive to test for these things later in the process, right?

So now your security team, you're at the end of release now your security team says, oh, I gotta, I gotta do all this testing for security problems, right? And so I gotta run all these, you know, DASS tests or whatever. Um, if you can do that earlier in the process, it's more efficient.

It saves time. You're gonna find those, you know, bugs potentially earlier. But, um, DevOps SecOps is also sort of a cultural change that needs to be adopted throughout the entire, you know, really engineering organization.

Um, they can't be siloed. You can't just say, well, you know, the developers, I'm gonna throw my code over the wall. QA will test it, they'll throw it over the wall and security guys will look at it, and then we'll see what happens if you really integrate everything that's, that's SecOps.

As you think through this whole process, um, what is the role of the testing teams that used to be there? Are they still part of this process or have we embedded this so far into the DevOps workflow that those folks aren't required because everything is being surfaced as we're writing the code? No, the, the testing teams are still going to be there.

The, the, the, the tests themselves don't go away. Um, it's just things may change a bit, right? Um, and it's not all gonna be on the developers.

There still has to be a team that, you know, you know, puts this stuff through and puts it into operations and things like that. So that's not gonna go away. It may be reimagined in some way, right?

I mean, you know, we've, I've read articles about all the AI stuff and AI can, you know, take away all the, you know, the, the testers that are out there, maybe some, but it, again, those people can be repurposed and it's not going to take away everything. Um, it's going to, they're, they're, you know, the way they interact with the software and test for it, test it now is, is still going to be there. There may be aids that come through with some of the, um, some of the AI based things, but in general, um, those, those jobs are still gonna, you know, be there.

There may have to, they may be repurposed in some way, but they're, those people are still gonna need, are, are gonna, the, the need for those people are still gonna be there. You mentioned AI and there are lots of new forms of it, generative ai, everybody's kind of talking about it right now. What is your sense of how smart can all this automated testing get?

What's the future look like? Well, I, I think, you know, we'd like to say, Hey, let's just push a button and let the robots handle it, right? And let's make it easy.

Um, and, and, and I think AI's got great promise. Um, it's been, you know, I be, I I was doing stuff with it in 30 years ago in, in an engineering job, right? And so, um, and, and, and the things we're doing tonight, these days at scale are really, are really impressive.

Um, but, you know, we also just can't blindly trust it, right? And so when I see things, and I, I, I, you know, I read an article I think on your site that said, you know, hey, that, you know, AI generated code can generate, can, can cause problems, right? There are bugs in that.

And, and, and so you always have to sort of ask yourself who's checking the, who's checking the AI itself? How are we checking it? Um, and, and so that's gonna be something that that's going to be, um, you know, something we need to be concerned about.

And, uh, I would like to think that curtail could help in that process because we have an objective way of testing, you know, Hey, here was one version, here's the next version that came out, what changed, what objectively changed about it? And, um, you know, if, if you are going to automate, you know, the AI certainly can run lots of tests, lots of known tests, right? But again, how is it going to find for those unknown things that it hasn't seen before?

Um, and so its role will be critical, but it may not be ex exclusive. To your point, we may need AI to check on the ai, right? Well, or, or we may need a system that, that, that is a different AI system to check on it, right?

Because if it's policing itself in some ways, you gotta watch out that it's not the fox guarding the hen henhouse. So ultimately, what's your best advice to organizations? What do you see folks who are doing testing, right?

Doing that you kind of wish other folks would, um, implement a similar set of best practices? Well, I think you, you just mentioned there at the end, implement those, those those best practices. Um, you know, put your pipeline in place.

Use the best sets of, of automated tests that are out there. Um, you know, uh, run the actual, um, uh, you know, security tests that are, that are out there. Have a, have a good platform and do those, you know, do your, you know, the, the, the, you know, the, the, you know, the, the basics.

Um, I would like to say you should add some of the things that we're doing as a, as a, to make your testing more comprehensive. Um, and also, um, you know, really try to instill that culture of DevOps and SecOps we're, we still see a lot of things still being siloed and really, if you to to, to get that working together. It, it can be a really, uh, you know, strong value proposition.

Um, where again, if you're able to shift left, get some of these things done earlier in the process, it really does make a, a tangible difference in cost savings, um, and, and product quality at the end. All right. Cool.

All right, folks. You heard it here. Good processes still make for good software and there is no substitute.

Hey Frank, thanks for being on the show. Thank you very much. All right.

And back to you guys in the studio.