Six Ways to Foster a Security Mindset in Engineering Teams | DevOps Onramp 2023

here everyone welcome to developing a security mindset some tips for software Engineers. I'm Jamiah Cyrus from New Relic. And before we get started.

Let me throw this up here quick Safe Harbor illegal Department makes a show this I'm not sharing anything that can be traded on but none of the information I provide about should give you any advice about things that you want to do with making trades about the company. So just throw it up there for legal purposes. And I'm gonna go ahead and Dive Right In, right?

So as software developers, we want to build amazing things. If you ask any developer what they rather spend their time doing 90% of the time we're gonna say that we want to be building something new or interesting and we do that a hundred percent of our time as we're working if we could but realistically problems arise and there's always things that we have to think about a software engineers. All right.

So our expectations are changing the landscape is rapidly evolving. We're expecting to ship products faster with fewer errors. We're dealing with more modern applications that are in some instances.

It's hard to even fully understand your entire stack or know where things are coming from because a software applications are so distributed at a distributed at this point, right the day of Monolithic applications are over some people say they might return but for right now the monolith is dead and we're just dealing with these very modern applications. We're expected to drive outcomes with the products that we're building. And then also we're dealing with more risk, right more bugs more complex systems more security risk just on a daily basis that we have to think about and so the idea of being able to build new things 100% of the time just isn't true for software Engineers, right solving problems fixing ER Us dealing with security threats are affect our effect in a developer's life, unfortunately, and normally we're acting pretty reactive to that.

Like something has to break. There's a security issue we go back and fix it. We're dealing with a lot of noise when we're on call.

Sometimes we don't have the right data to fully understand what's going on or even understand the environment right? We don't have enough context. And so that's just what the life of being an engineer will look like and You know, there's this this notion of developers being able to say hey, well, it worked fine and in Dev, it's Ops problems now, right but that world is changing as an engineer.

Now, we're building it and we're running it at the same time. So we have to really be thinking about multiple things outside of just writing code or just building the shiny new service that we're going to be shipping into production. Now, we really have to think about how it's operating.

Is it secure the type of performance that we're providing to our customers? And so that's where this talk is gonna come in and I know I said a lot of things but and let's talk we're gonna talk about how is the software developer? You can develop a security mindset, right?

So what is a security mindset and why is that so important? All right, so As an engineer having a security mindset. 99% of the time.

Hopefully it's gonna be safe. It's gonna be secured. It's gonna provide your users with a great experience and having the security mindset goes beyond just thinking about application vulnerabilities or that threat prevention and also goes into thinking about the health and the Integrity of your system.

As I said before if it's reliable understanding how your software is impacting your customers understanding the performance of it. Once it's gone out in the world and have an idea of how you gonna handle downtime. If something was to go down or if you were totally lose your application.

How Could You rebuild from there? So all of those things go into having the security mindset and making sure that you're putting out the best software possible. Alright, so I'm gonna dive directly into the tips.

And the first one I think is one of the most important ones so it even gets its own special second slide. It's gonna be shift your security process left, right? And so it's like well, what does that mean?

What is Shifting left? Right. So the idea of Shifting left is when you're starting to think about things earlier in your software development life cycle instead of waiting to things are actually deploying into production to start thinking about issues.

You're starting to think about The way that you're going to bill your software how you're gonna handle security how you're gonna observe its performance all in the design and plan phase throughout the entire development process of building that software and so shifting left looks something like this, right? So as you're actually building an application you're going through multiple phases from planning and designing it to where you're actually develop in building the tool you want to test it after you have all of your green tests. Then you're gonna go ahead and ship that into production and afterwards once it's in production, you're gonna Monitor and analyze it and you kind of repeat this process over and over right?

So in the traditional sense, we start to think about things in that testing through the deploy and release phase right? That's when we really start to think about the quality and the performance of the applications that we're shipping on into production. But if you were to look at this error curve, that's when you start to actually experience a spike and error because user you like actual usage is coming in.

And so there's always going to be things that you haven't thought about or anomaly. So they're gonna pop up that you just hadn't considered before and so the idea of Shifting left is instead of thinking about things like software security or observability or application monitoring when you're in that testing and deploy phase you want to pull that thought process much earlier into your software development lifecycle. You want to start thinking about that in your plan and your bill phase as well thinking about the instrumentation that you're gonna need the data that you may want to have access to if something was to go down and production.

You just start thinking about an implementing that strategy a bit early. So once you've tested and you deployed it out into the world, you have the right information you have a strategy in place that's gonna allow you to be more effective in resolving security threats or you know issues and downtime but also as you're shifting your your security left or Thing you're observability left that's gonna allow you to start to think about that process without also having to deal with the errors, right? So instead of you putting out fires and trying to figure out the data that you need you can build that strategy early in the process not have those fires that you need to put out and then be able to handle it a bit smoother.

Once you actually have things shipped out into the world and that's the idea of Shifting your software development or shifting your security or shifting your observability to the left. That's gonna be the first tip that I give you as a software engineer how you can develop a security mindset. It's to start thinking about things earlier in the process.

And that leads me directly into step two, which is to make the switch from devops to devsec devsecops. Right? And and I'm sure we're familiar with this process of the idea of devops right where there's a life cycle in the way that you actually build site software where you're planning you're building the coach.

You're gonna test it deploy it you operate it out in production and you monitor it and that's a life cycle that just repeats over and over. Right? So one of the best tips that I can give you is to also start to think about security and I would even say observability in that process.

So going just from devops to where I'm building and releasing so I'm building and I'm releasing and I'm thinking about the security of my software and my services through this entire process. I'm thinking about the observability of these Services through the entire process, right? So shifting left to think about it as early as possible automating as much of it as you can simulating threat.

An incident so you or your team can actually deal with a real incident in production better? Because you've had this process of actually testing and running game days and and seeing what would happen if a user put in the wrong input or trying to break your application on purpose, right? So that's gonna be going from devops into devsecops and then it's gonna take me directly into the third tip.

Which is thinking of the unhappy path? I was a front end engineer for a while. And when I first started I would always test things.

I would always test things like based off of the way. I knew that a user would be using my software and the first thing that happens when you actually see someone using it out in the world is that they're not using it the way that you built it. They're not using it the way that you had it in your head.

And so you as engineer, you have to think of these unhappy past you have to look for edge cases. You want to test them you want to plan for, you know use the errors you want to plan ways to prevent things from happening, right? So what if a user try to enter a my sequel query inside of a form field to see if they could access something from your database you have to plan for all these these situations and think of all these unhappy paths and building resiliency for that.

So be aware of using default passwords and configurations using default configurations on the operating system. Make sure that you tailor things and you limit access as much as possible try to break your code on Purp. This one testing it limit your permissions that you're giving so users only have access to exactly what they need.

And that's the folks that are building a software and the folks that are using out and in production. You just really want to think about just the right permission for anyone that's dealing with your software and services and that way you'll be able to stop things from happening. Once you've actually shipped your software out in the world.

So not only do you want to make sure that you're you're apps work in the way that you expect it for them to write. So I want to sign up for a product I go in and for my into my information I create a past code and everything works wonderfully, but also want to test what if someone puts JavaScript inside of this form fill what would happen right? And how do I plan for that?

Because that's what I used to do as an engineer just to see if I could and there's always gonna be people out there that will do the same thing just to see if they can all right. And so that's gonna take me over into tip number four. Which is conducting peer reviews of your code and your systems and that's going back to that planning for the unhappy path.

Sometimes you're so close to the application that you're developing that you're just gonna miss things naturally, right? So having a peer of yours or having a code review will allow you to step back a little bit have someone else look at code make sure that you reduce the the possibility of making any critical mistakes, make sure that you reduce the possibility of pushing bugs or security voter vulnerabilities into production. So I'd say this is a really big step before you send anything out into production.

You want to make sure that it's had a code review that you've tested it and that you're making sure that you get other eyes to look at what you're building before it goes out into the world. So a lot of people think that being an engineer is being a team of one, but I'm sure we all know that working collaboratively just makes you a better software engineer it makes for Software going out into the world and we could always use an extra set of eyes. So that's gonna be tip number four making sure that you conduct a peer review and you have someone else take a look at your code and the systems that you're building.

This is gonna take me into tip number five. All right, so tip number five is making sure that the libraries that you're using are regularly maintained and I would even add here is that when you're maintaining them and your updating your library is you want to make sure that you're dealing with any vulnerabilities that exist or you're you're choosing libraries and you're thinking about the vulnerabilities that they could be exposing your applications to right? So it's actually been stated that 80.

Well about 92% of all software in the world uses open source libraries, right? And so 80% of those libraries are never updated once we've at them to our code and the interesting thing is that 84% of all code bases contain at least one security vulnerability with about 50% of all of our code basis having a high risk security vulnerability. So we're choosing packages that are making our jobs easier.

They're allowing us to develop quickly giving us functionality. We don't have to think about building our Close but as we're doing that we're also exposing ourselves to these security risks by having them in our code base. And so we should be thinking about that when we're choosing the packages that we're going to be using in our applications.

We should be looking at the last time it's been updated. We should be going back as a software engineer and actually making sure that we're updating those libraries over time to reduce any risk or any possible exploits that we've exposed to applications to right and so for every piece of software that that we're sitting out into the world we want to make sure that we're trying to provide our customers with the best experience possible. Right?

So having a security mindset is understanding how your customers use it and making sure that you're providing them with a reliable experience. It's also having an understanding of the packages that we're using it and making sure that we're keeping everything up to date. So one of the things that I'd say is make sure that you're checking all of your libraries and your third party packages for unknown vulnerabilities making sure that you maintain them there's multiple ways that you could do it.

Here's a Shameless plug New Relic is free, right and in New Relic, we have a vulnerability management that you could use and we're gonna check your your services that are running in production and let you know if there's a vulnerability within your application you could also have that data directly in your IDE using coach dream, which is an open source third party plugin that's gonna allow you to see the performance of your applications and those security vulnerabilities directly in your IDE. So you can make decisions around how you want to update them. It's gonna take me into tip number six and you want to develop a process of redundancies.

You want to develop a system redundancy. What if you were to lose everything for some reason? You're a your server caught on fire, right or AWS just vanished and now we have no access to our code that's running in production somewhere.

It's it's not the most likely situation. But you do have to plan for what if I were to lose everything and no matter how hard and your applications or your services is there's always an unpredictable chance that that may happen. And so just having in an idea or some plan of system redundancy in place that's gonna help you to prevent as much downtime as possible that's gonna help you rebuild your systems and services.

So you don't have to start from scratch could save you countless hours of work or can save your company from totally burning down to the ground, right? So that's one tip that I say that it's just nice to have in there that if we were to lose everything if everything were to go down this is how quick we can get it back up and sometimes you may want to test for that as well. Right and that's gonna take me into my final tip for be having A security mindset as a software engineer.

It's gonna be tip number seven and that's keeping up to date with the best practices. Nothing that I said is finite. Nothing is static.

It's all dynamic as the way that we build software changes. These tips will change and we have to go back and reiterate a similar to the devops cycle. Right?

We're going back and we're checking everything over and over every few months every year. So you want to make sure that you're also doing that with yourself and you're in your skill set, right? So being aware of what the security best practices are understanding how rapidly technology advances and staying up to date on what's essential for building a Reliable Software also understanding what what you need to know as a professional for maintaining your security mindset.

So I would say making sure that you read industry Publications follow a few blogs a 10 software seminars and conferences. Just you can hear kind of like what's on The Cutting Edge and they're helpful. dev the open ssf project sneak has some really good resources that are free on their sites.

Just making sure that you invest time to keep growing your skill set understanding what's going on in the street and just knowing what's going on with regards to like latest practices in building software, but also making sure that it's reliable and safe for the users that are using it. All right, and so that's all the tips that I have for you today. I am Jemiah science the head of developer experience here at New Relic Miami native.

So go heat. We're in the playoffs right now. I'm a software developer that started out as a designer self taught so start teaching myself HTML CSS JavaScript eventually worked in PHP and see and all of the random things have been developing now for about 15 years.

I love building Community lifelong learner. I think any software engineer should be if you want to connect with me. Here's my LinkedIn and my Twitter.

Thank you all so much for tuning into this talk today. Hope it's helpful and developing yourself for your security mindset, and if you have any tips for me at love for you to reach out and just let me know. So, thank you all and other than that.

I hope you all have a great day.