Matt Tesauro – Black and Blue APIs: Attacker’s and Defender’s View of API Vulnerabilities

Hi, we're here to talk about black and blue apis and get the attackers and Defenders perspective on API vulnerabilities. Quick overview of what we're going to talk about today. I'm going to do a very quick intro just a little bit of background on me, honestly.

I'm gonna go into why attacking apis like why are these juicy Target for the nefarious people on the internet? And then the meat of the talk will be attacking apis where I'll walk through many different attacks that can happen for API specific to apis and tell you what they look like from both the attacker. and the defender perspective As well as a conclusion, we're all wrap up and give you some key takeaways.

So who am I I'm Matt tesaro. I'd like to call myself a reformed programmer and abstick engineer. I'm currently a distinguished engineer for No, Name security.

I've got 14 plus years in the OS Community. I've run several large projects for owasp. I've 22 plus years of using Linux and open source software.

So if you notice a little open source bent that's my background. I currently write in go. That's my current favorite language and I have a Edon in Tong sudo umigukawan, which is a second degree black belt.

That's actually me doing my double board break. For my second degree black belt that I was shocked. I made it through that.

So let's get started. So why attack apis? Well, there is a British mathematician who came up with the phrase.

Data is the new oil and I think we'll all agree data kind of is the new oil. I'm certainly more green than oil. But how does that how does that new oil get around apis are the pipelines for that new oil?

That's how data gets around these days. You do something on your phone that data generally comes from an API. You do something with the website.

A lot of the websites are backed by apis. You have home automation apis Cloud kubernetes apis. So kind of apis are everywhere.

And even if you do have a good solid your confidence in your absec program. There's still some gaps right some of the tooling that works. Well in a more normal and traditional whatever you want to call it.

Absec program will work for apis but there's a whole bunch of items. in that right side like rate limiting jwt's API gateways all these things are you not part of a typical appsec program unless you've embraced API security. So you may think you sort of have apps like nailed down, but without some special attention to API specific things.

You likely have gaps. And then of course, there's always what I love to call pro bono pen testing which is Hackers attack your website for fun and for free because it's on the internet which is unfortunately the way the world works these days. And then I'd like to Define three different sort of core categories or pillars whatever you want to call it of API security.

So there's three major areas that if you look at the tooling that they fall into so the first one I like to call Api security posture. This is to get an idea really a full inventory of all of your apis. You know, who should be calling it what data is sent and received ideally with some classification in terms of what sensitive or regulatory data versus not.

Where does it call originate is this an internal? Is this an external API should be going through an API Gateway? Just a very good map.

of your API landscape and then runtime security is watching or monitoring or keeping an eye on API traffic. Well initially to get an idea. What's normal and also ideally to find out what's abnormal do anomaly detection do response and alerting.

So that as your apps are running a production, you have a window and how they're running and understand any attacks that are happening. And then finally API security testing and this is a way to assess the security state of your apis. Now I like to make a distinguished or a distinction between dashed and fast.

So SAS to static analysis looking at the source code. Honestly fast tools don't care if your code is an API if it's something for an Apple watch. If it's a traditional web app, right?

It's asked understands the language that can evaluate. That source code. So I like to leave SAS off the table for here because it starts put it's not particular to apis.

And in fact, I like to focus on dash which is dynamic testing or testing a running API. And then once you run those Dash tests, you're going to have some results more than likely and you need to get those into issues trackers or whatever is the normal way to give bugs to the development teams. And then finally, I like to have a better and more security focused security control Focus definition of an API and you'll find some tools to a better or a worse job of this honestly having the granularity.

You need to make the proper kind of risk and security control decisions. So first one is a host name. That's obvious.

com second one is a path. So V1 slash cart slash add item and third one is the HTTP method. So post put it delete patch, right all those good HTTP methods, but you really need a combination of those three to make it good security decision.

com V2 users all presumably I'm getting a list of all the users. Well that get request. It's very different than the delete request that is otherwise the same.

So just by changing the method, I'm either listing all users or nuking all users right very different security. Text around those two calls, but only one thing changed. And then the second example is a post.

To V2 user admin in uat or a testing environment. com. And that's a post to production that's very interesting from a security perspective right in a very different thing and here we're only changing the host.

Otherwise, it's all the same. So let's get into the meat of the thing. Let's talk about attacking apis.

So we're going to talk about active attacks and I'm going to focus mainly on the active side of attacking. I'm going to leave the normal part of pen testing Recon and Discovery those steps off because we're just gonna assume that attackers have found and have some knowledge of your apis. And then what and I'm also going to use the oh wasp API top 10 to sort of categorize the well the top 10 risks or attacks or vulnerabilities that can happen with apis.

So that's going to be the structure that we use to walk through all these texts. So the first one broken object level authorization. This is a case where I'm using my API token to request somebody else's data fundamentally.

That's what ola is broken object level authorizations shortened usually to Bola. So in that in that an example of that would be me using Matt's token to request Sarah's data, right and I probably shouldn't have access to Sarah's data. So as an attacker, what do I do?

I look for IDs these could be numeric. They might be just strings. It might be uuids but I look for IDs and then play around with those.

Can I change those IDs and get somebody else's information just by modifying my request. I can create things as one user where I have the full information about their ID and what their contents is and try to request them as another user and see if that's allowed. Now as a Defender.

This can be pretty interesting because it really requires very deep inspection of the HTTP or the API traffic. Because that that inspection has to not only know that this is Matt's token, but Matt has substituted in Sarah's ID in his request and that's something that is it takes a very deep level of inspection something that wax generally don't have honestly the time to do because they are in line. So a lot of times you need to rely on ML or AI or whatever kind of magic buzzword you want to say, but you need a computer to do the thinking for you and to track those those different values.

You will notice generally a whole bunch of increased authorization errors, hopefully your monitoring for those but if you are monitoring for those, you'll see an increased level authorization errors as attackers are probing to see if these there are examples of Bola in your API. Broken user authentication. So there's two real cases here.

One is where you have a air quote login to apis, right? I provide a username and password. The API gives me back a token.

And then I take that token and make the rest of my requests the normal requests to the API. So it's a way to provide a token a bearer token where somebody used to talk to an API. The other case of this is where you have week.

Crypto or week coding around the tokens that are used for authentication for the API. So there's kind of two cases here that can happen. so for an attackers, obviously if there is a API I found that allows me to log in here quotes send a username and password and get a token.

I'm going to brute force that one or password spray attack that right and see if I can get anywhere likely I will because password reuse is very very common. Also if there's not a log and I can investigate the tokens to look for weaknesses. There's a ton of tools that exist for this like JWT underscore tool, right you can use to examine and find weaknesses.

These are things like not checking the signatures on a sign JWT Etc. So as a Defender while brute force and password spraying, those are super noisy attacks, you should very much see that if you're doing any kind of monitoring. For the other half of it the the poorly configured or created tokens.

This is where abstick training policy and review really comes into play and there's not much you can do. Well, you can do the testing will catch a lot of this honestly, but A lot of this is just developer education to avoid the problem in the first place and I really like the JWT best practice RFC. I'd highly recommend reading that.

Now one interesting side note about off broken user Authentication. is because we've gotten so many more improvements in browser security and ways to control logins and get a handle on logins MFA capture a whole nine. We have a whole bunch of tools.

In the tool belt in the sort of browser or mobile app world that don't exist for apis. right So what's happening now is attackers are stopping to attack normal air quote web logins, but they're actually attacking API logins instead and this became so significant. For GitHub that they actually removed the ability to log in air quotes to their API provide a username password and get a token.

They removed that completely from their API and the only way now to get a token is to log into the web app because every time they tighten their login, they saw a spike in attacks on their API. They're their web login. They saw a spike in attacks of their API.

So interesting interesting problem with logging into a to an API so excessive data exposure This is where developers use programming kind of helpers which take a data object usually from a database or some other data store convert those automatically into Json or XML and ship them out as the response to an API request. And a lot of times also hat will happen is devs make the assumption that only their clients are going to call their API. So hey, I can send them all the data.

They really only need the seven pieces of it. But I've got 10 in the database. I'll just send all 10.

They can ignore those other three. well as an attacker I don't need to ignore those other three. In fact those other three might be really interesting to me.

So you cannot assume that the client that calls your API is always going to be your client or even a friendly client, right? We're usually talking HTTP to talk to an API and you can do that in any number of ways doesn't have to be a mobile app. So if I can intercept that the traffic or just pretend to be a different actually use a different method to talk to the API different client that I can actually get and find that exposed data.

So a lot of times as an attacker, I want to look for Pages where there's likely more data store than what's displayed. And look at what those requests send when they when you make the API call. so for Defender unfortunately These are single requests.

That are legitimate. They just happen to return way more data than they should so they don't look weird or suspicious. So this is something where you really need to rely on source code tools like sast Define those two Json things.

um, it would depending on your language then maybe a different method call but to find those data conversions and make sure that you're not accidentally exposing data. Now one thing that will happen is if I as an attacker find it excessive data exposure. I'm likely to want to not only take it for one user but take it for all your users.

So in many instances after I find that initial instance, I'm going to go try to scrape all the user base or all the whatever to get as much data as I can and those Mass scrape data attacks. Can be seen by monitoring or runtime tooling. Lack of lack of resource and rate limiting.

So this is failure to restrict the amount of requests that an API gets so it can be overwhelmed. This might be just sending a whole bunch of network traffic to it the traditional kind of network boss. This may just be a bunch of legitimate requests that knock it over because there's only a certain capacity for your API.

Those could be not even malicious. It could just be one client kind of hogging all of your resources and keeping other clients from connecting. And then the other area of concern is any kind of calls that are particularly resource intensive.

And that may have inadequate protection because that to me is as an attacker is a recipe for a dose right any request where I can send a tiny amount of data and get a bunch back that asymmetry allows me to have a potential for a dose an at level dos. Now as a Defender, I want to look for calls. Like I just said that have small requests.

And give me large responses, right? That's a great way to get that kind of a Doss. And they're harder to rate limit because they are by designed asymmetric.

Now one of the things you can do as a Defender is look for strange requests because a lot of times to get around rate limiting controls. You'll play around with headers. You'll do some odd encoding you may drop like null terminators.

Into the request to try to trip up those rate limiting control devices, right or if you're monitoring sudden spikes and usage. Those all give you an idea that an attacker is having it go at your rate limiting and resource limiting. All right number five broken function level authorization.

These are bugs that allow one roll to request to make requests successfully to other roles restricted methods. So the easy example is I'm a normal user and I can make admin calls. That's a broken function level.

authorization issue So obviously it's an attacker. I want to look for apis that have multiple roles. I you know even try to see if there's any backplane or admin inadvertently or maybe on purpose got help you exposed to the internet, right?

And then as an attacker also, you can do interesting things like try undocumented methods, right if the documentation says this. API endpoint only accepts get well, why don't I send a post and a delete and see if those work maybe they just didn't document it but it actually the code will accept those. Why not give it a try.

You can also do things like create items as one roll and try to request them from a second roll and see if that's successful. It's another way to discover these issues. There was a Defender obviously, I want to sort of keep a special eye on apis that have multiple rules or privileges and you know, please please keep your admin and backplane completely separated from your end user or client facing apis.

There's no reason to mix those. and then If you have decent monitoring particularly with ML, you can find things like a same client that changes its role multiple times of our short period of time that's a very unusual usage pattern. Most people have a role and they make requests with that role.

They don't be an admin then be a normal user then be an editor all in a short period of time. So that's another hint that somebody is playing naughty with your API. number six Mass assignment This is where apis trust and don't filter client input.

And you have that with the combination of sort of the the opposite or the inverse of excessive data exposure you have code that automatically takes what is sent by request and saves it to a DB and doesn't make sure there aren't additional bonus data right in that request. So as an attacker, what I can do is make a legitimate request and add in extra data elements that then get saved on the back end system. right, so A lot of times I'm gonna look for requests that have more internal data.

Like if I'm looking for my profile, maybe my profile has my profile. Pic and you know my favorite color, but I probably have a role too, but they may not let me submit that well, what if I submitted anyway, right and if I'm lucky as an attacker unlucky for you as a Defender that works. And so what's interesting too is an attacker is I can put multiple guesses and generally these things silently fail.

And then if I can combine this with something like broken function level authorization, I can do really awful things like change a different users email address for password resets to when I control and suddenly I can become that user. And for Defenders, this is another place where you need deep inspection. So these Oddball requests will stand out because they'll be larger and they'll have data elements.

They don't normally have so it's pretty easy to find. I mean, you also may see a lot of failed or invalid requests while attackers are probing to try to find these vulnerabilities. You will see an increase in request size.

This can only be a small amount. So it kind of depends on how granular and How Deeply your inspecting your traffic, but it's another way you can notice these attacks happening. number seven so miss security misconfiguration, this is your Quite honestly standard or traditional misconfigured it resources in this case just for apis.

And this includes all of the the satellite of infrastructure that can be around apis like API gateways and firewalls and vpcs and API management councils and all these other things right that can exist in sort of the API landscape. And so for an attacker, I'm kind of doing the basics. We TLS information leaks verbose error messages.

Those are all kind of interesting ways to understand more about what is the back end of the infrastructure of a API as well as what technology they're running. And you can do things like purposefully generate errors by making purposely wrong requests. And if those were those are verbose you can get additional information also things like framework settings.

You can try to set things to debug Etc. And that's based on what you've learned. particularly in your Discovery portion of your say if you're doing a pen test.

so for Defenders I'll hopefully you're running the doing the bog standard running Network vulnerability scanning on a regular basis. A lot of those will find these luckily for you. These kind of probes like checking TLS tools.

That checks say TLS. Are very noisy and they make they send lots of probes. So you should find this if you're doing decent monitoring or runtime.

monitoring of your apis And then passive even just passive traffic monitoring can find things like header issues or Gateway Bypass or an external IP hitting an internal only API you think right? These are things that can be found by Defenders and let you know that there's a Mis configuration. You need to go address.

Eight is another good oldie but a goodie been around in the oh lost top 10 forever. It's injection. Right and this is the same kind of injection that can happen to air quote traditional apps just for apis and it's basically a failure to keep separate code or control characters.

If you want to call it that way from data and because of the your program gets confused. It ends up running the code or control characters of the attacker instead of treating it as data. So as an attacker, I want to inject or fuzz request data pick interesting things like tokens headers.

Things that feel like there may be query items to a database. And reconna Discovery will hopefully give you a pretty good idea of what infrastructure is there. So you can tailor right?

If there's no ldap, you don't need to send ldap injections strings at it because it doesn't matter. and then for Defenders This is input validation and output and coding right is the standard thing to do for this has been for years. This is where SAS can help you at the code level.

Um, but outside of the code level you'll see a lot of failed air or erroring requests if they're doing fuzzing that's very very noisy. Lots of requests that have Oddball characters in them. So this is actually fairly easy to spot from a monitoring or a runtime protection perspective.

number nine so you need to know what you have if you want to secure it and this one's all about knowing what you have knowing all the apis that are running in your infrastructure. So for attackers, there's really not much to do here. To be honest with you.

This really only comes into play because you're very successful in your pen test, right? They don't understand what assets they have. They're probably not allocating protection and controls correctly and you find those gaps as a pen tester and You're Off to the Races, right?

It's a good time. So really it's success in a pen test that tells you that they don't have proper assets management as opposed to a distinct vulnerability that you can Leverage. and then for Defenders This is just making sure your posture.

Is Rock Solid right, you know all your apis down to that three level host path method. I mentioned earlier you've classified all the data, you know, which apis are public, you know, which ones are internal, you know, which one should never see a public IP, you know, which ones should always. Wear question should always go through an API Gateway right if you have that.

You've got this one covered. So this is really heavy on the posture and a little bit on the runtime because you don't want posture to be static. You really want to run time to update.

Of runtime to update your posture as you go and as things change. Okay, number 10 the last one. It's efficient logging and monitoring.

Right. Do you know what's actually going on with your apis? And this is where logging monitoring observability whatever you want to call it, right this really important to up production running in particular of secure apis.

So as an attacker, this is also when you sort of have to infer you don't generally see the logs. So you just notice this by the fact that I can fuzz the thing forever. And there's no response.

I don't get blocked. I don't get banned. Nothing happens to me.

I don't get d off, right? So this is mostly an inference for attackers. for Defenders You basically don't know you're being attacked.

So I don't you have to notice the lack of knowing you're being attacked. But you can notice this by things like it's really hard to diagnose issues. And these could just be performance issues not even security issues, right?

There are suddenly really hard to track down because they're just isn't any logging or monitoring in place. Or you're kind of Flying Blind and you're always sort of relying on that hero to come in for my tea and do crazy magic and sort things out that's a sign that you probably don't have sufficient logging and monitoring in place and you really want to get to a steady state where you can just kind of watch the logs and understand. Oh look we've ticked up three percent.

Maybe something's going on as opposed to the API fell over. Oops. We need to do something.

And I have a couple pieces of bonus material for things that didn't fit nicely into the top 10. so One way to think about a particularly from a testing perspective. Although this also works for defense.

Is there's two different ways to attack apis from a very fundamental high level. There's what I like to call structural attacks and data attacks. And let's start with data data is the easy one when I'm submitting a request.

There's certain elements of data in there. I can play around and change those values and see if I get somewhere particularly things. The easy example is injection, right?

I put a single tick in there and I get SQL injection kind of a thing. Right, that's data attacks. Now, these are nice because QA or normal HTTP testing tools will make legitimate looking requests.

Easily for you, so you can get a lot of automation out of the box. For data attacks, but the flip side is structural attacks and you don't see these as much and they're certainly not supported by normal clients or QA or testing tools because those are designed to make legitimate requests and just play around with the data but structural tester just doing crazy stuff. With request to apis what productive when I found one time was injecting 32 Meg of space between two ah XML tags.

It's a valid request. It's crazy, but it's valid that took out an API. Right.

So the interesting thing about structural attacks is you almost have to graph them yourself if I'm repeating an XML element 12 times just to be crazy and see what it does. I have to do that by manually crafting that request. So a lot of structural attacks end up being manual.

They don't end up getting done and they can be very productive if you are doing pentesting. So unfortunately, we're getting close on time. And I did not think I could fit a whole graphql thing in here.

So I'm leaving you to Google that apologies, but I just didn't have the space. So let's go to our conclusions and see where we're at. So key takeaways for API testers.

You got to have some kind of knowledge or if you do have some kind of knowledge of how to test web apps. You can apply most of that actually to API testing. The difference isn't all that big.

So if you're starting from scratch go out and find the ill lost testing guide and get a handle on General. a web testing and that'll apply in many many cases to API testing this just not a UI around. You will have to have some special knowledge and tools then I've got more in that in the next slide.

and if you if well as a tester be aware that there are likely gaps in controls. In both apps like programs and in the Frameworks that lead to the security gaps for apis apis are fairly new certainly newer than web apps. And since these are a Brave New World and you're off in the Prairies you end up usually getting shot you're off in the wild.

That's where the wild beast come and get you. So at API testing is likely to be very productive. Oh, I went too fast.

I also created for both API testers and Defenders. I created this web page on the owasp website. It lists all of the tools.

I'm aware of for API testing and also categorize them as if it does posture runtime or monitoring or testing, excuse me, right and I'm happy to take submissions you're happy to this is on the Olaf website. Anybody can contribute. I love getting contributions.

You can either put them directly in there. Send them to me whatever you need to do. I'm just trying to get a nice long list.

So people that are evaluating API tools have something to work with And then for Defenders these next two slides enumerate all of the different oauth top 10. API vulnerabilities And where those tools play. So the the weak arm is obviously weak.

The normal arm is obviously strong. And if you go to the next slide, you'll see some ones with arrows up. That means they're really strong and some places where they don't really work.

Well. And the Really the concept of these two slides are the idea. I'm trying to get across is you need a combination of these three categories of tools to truly get coverage over all the range of things that can happen with apis.

There's several of them that do a really good job in the bulk of them, but they don't give it all to you and some of them like testing if you do it far enough left, you can avoid problems ever making it to production or if you're using runtime almost by the nature of the thing you're finding problems in production. Whoo, okay, and sorry about the fire hose. This was probably a complete fire hose for you.

I apologize. I had a lot to say and not a lot of time to say it, but I hope you enjoyed it. And thank you very much.

And I'm happy to answer questions now. We're in the future. My email address is Matt with three T's Matt tesaro met.

com. Thank you very much.