Prerequisites

• Your AWS Organization

• The SecurityAudit account

The Lesson

When I first mapped out the CloudSLAW gameplan this was going to be lab #4, but I realized it would work much better once we had our organization, SSO, and delegated administration set up. Normally GuardDuty is something I try to enable the first day of any new account, which shows you how important it is. The nice thing is that after this lab it will just be on automatically for all your accounts.

Putting on my historian hat again (which I’m allowed to do because I have a history degree — I don’t know why anyone trusts me on technology) I’d like to take you back to the bad old early days of cloud, when we lacked any sort of visibility (or IAM, for that matter). These were dark days, or they would have been had any attackers understood cloud, or anyone other than Netflix put something important up there. And this was before Stranger Things, so even that is debatable. CloudTrail was our first massive blast of sunlight, allowing us to know what was going on in our accounts — and I know it was a huge challenge for AWS to implement.

In 2017 Amazon added GuardDuty. This is basically a kind of Intrusion Detection System (IDS) for the cloud. IDS has been around for a while and used to refer only to network monitors that sniffed all network traffic at the point of your Internet connection, looking for any signs of attacks. We have different flavors of IDS now, and use the term “threat monitoring” more generally; GuardDuty fits neatly into that bucket. The TL;DR is that GuardDuty monitors a bunch of telemetry feeds in AWS, and uses different detection technologies to produce findings, which are signs of potential attacks. There are some cool aspects of how GuardDuty works, which most people don’t realize when they first start:

• GuardDuty monitors CloudTrail, VPC Flow Logs (we’ll get there), and DNS Logs (we’ll get there also) even if you don’t enable them. How? Because AWS is always producing those events — it’s just that when you turn on the service, AWS starts recording them (and charging you). With GuardDuty you don’t have to turn them on or pay the per-service fees, although you still pay GuardDuty fees.

• GuardDuty pricing is based on activity. That means there is little to no cost to turn it on in all accounts and regions, even if you don’t use them. You only pay for active accounts and regions.

• AWS uses a mix of threat intel feeds, heuristics, and their own detectors. You don’t get any insight into how they work.

• There are additional monitoring capabilities and detectors, which are enabled when you use certain features. For example you can selectively enable S3 or EKS Protection. These usually cost extra and use additional telemetry sources. As with the main log sources, you don’t need to explicitly activate the new telemetry source (like S3 Data Events, which we will cover later). GuardDuty will monitor the feed and you will pay the GuardDuty rates, but you won’t also pay for the Data Events.

• Findings can create real-time events, and stored them as files in S3 for analysis.

GuardDuty has advanced a ton even since I started CloudSLAW. It offers container detection, endpoint detection, and more.

But GuardDuty does have drawbacks, one of which we will see today:

• It’s a regional service. You can’t flip a switch and turn it on everyone like we’ve done with most of these labs. You have to explicitly activate each region.

• Not all findings are created equal for all environments. In a development environment you might get a bunch of alerts. Or you might get alerts you don’t care about. And it can miss things. As we progress with our usage and start (safely) generating, findings I’ll show you how to tune and manage these. Just don’t go turn it on for 2,765 corporate accounts, send all the alerts to your email, and then blame me. Please.

• GD ain’t free (that rhymes, cool!). For our purposes it will be inexpensive, but it will start creating charges in your accounts. Again, we are talking pennies to a couple dollars every month, and I’ll keep an eye on things to adjust later, but this is an important security capability we all need to know how to use.

Lesson Key Points

• GuardDuty is effectively an Intrusion Detection System for your AWS. It monitors a bunch of telemetry sources and uses Amazon-managed detectors to alert you to potential malicious activity. Examples include cryptomining and unusual IAM activity.

• GuardDuty uses a range of AWS telemetry sources, including ones you would otherwise need to turn on and pay for to review directly, including VPC Flow Logs. But when you use GuardDuty (if you don’t turn on the log feed) you only pay for GuardDuty, and for those feeds themselves. You don’t get a copy of the data, but AWS still watches it for malicious activity.

• GuardDuty charges based on activity volume. I get asked a lot whether orgs should turn it on in regions they don’t use: the answer is hell yes! Real-world major breaches would have been caught if GuardDuty was enabled in an unused region, and you pay basically nothing for it in inactive regions. In our case we have our unused regions locked out with an SCP so I’m not as worried about turning it on everywhere, but we will definitely use it in our two live regions.

The Lab

Today we will go into our management account and enable GuardDuty for us-east-1 (Virginia) and us-west-2 (Oregon). We will configure delegated administration again and delegate to our SecurityAudit account. We will also click a setting so that GD will be deployed, by default, in those two regions in all our accounts.

It’s a bit repetitive because we need to enable each region individually. To be completely honest, that’s one reason I had us lock out all the other regions.

When we are finished we will log into SecurityAudit and generate some sample findings.

As a reminder, after the 30-day free trial, you will see small charges in your account. This is to be expected, and I’ll watch mine closely to see if we ever need to adjust our plan.

Video Walkthrough

Step-by-Step

We have a bit of a different start today. First log into your Identity Center Sign In Portal and then copy the ID of your SecurityAudit account.

Then log into your CloudSLAW management account with AdministratorAccess:

Important note: we will repeat this process in 2 regions. Start in Oregon and go to GuardDuty (just search for it in the search bar). Then click Get Started:

Before we enable GuardDuty we want to delegate administration. This will save us some clicks. Remember, every time you save a click, a fairy gets its wings. Paste in the SecurityAudit account ID and then click Delegate:

Then Enable GuardDuty:

That’s it. Really. All the foundational checks are up and running. But now we need to set it up in Virginia. Go to the upper right corner and switch your region, then repeat these steps exactly!

At this point you should have it configured in Oregon and Virginia. Remember we have an SCP which locks out all of our other regions. In a major enterprise environment I would still enable it in those regions during my initial organizations setup, because strange things can happen, or someone mightt later activate a new region. It’s just best to have things in place. But for our purposes that’s overkill.

Okay? We good? Both regions done? Now let’s auto-enable it for all our other accounts. Technically this is only running in one account, in two regions. For this part we will log out of our management account and start working with our delegated admin. Log out of the CloudSLAW account in the upper right corner, then swap back to your sign-in portal tab and log into SecurityAudit using the SecurityFullAdmin role:

And go to GuardDuty > Accounts. The UX is a little weird here, so look for Auto Enable is off and click Edit:

Keep in mind that I’m glossing over a ton of GuardDuty features, including all the additional protection plans. They all cost more, and we aren’t doing anything with them yet. We will definitely learn about them all eventually.

You can see all of these on the auto-enable popup, but we will gloss over them. We want GD running everywhere, so click Enable for all accounts and then Save changes:

There’s a bug in the console at this point. Technically we just told AWS to turn on GuardDuty everywhere, but it only auto-enables for new accounts. It actually took me a day to realize the process didn’t work. Fortunately we can fix it with 3 clicks. Check the selector next to Accounts to pick all 5 of our accounts, then click Actions and Add member. There will be a confirmation popup which isn’t worth my time to screenshot, so just click Add member on that one again. Don’t worry if Total member accounts in the upper right doesn’t update — that isn’t real-time, even when you refresh the page. It’ll catch up eventually.

Got it? Good. If you ran things in the order I did, you just set up auto-enable in Virginia. Now swap to Oregon and repeat there.

You did just auto-enable twice, right? Like, you didn’t just do it once and think I wasn’t paying attention? Okay. With Virginia and Oregon both set, with GuardDuty running on all 5 accounts and auto-enabled for any new accounts, let’s take a peek at what it looks like.

Go to Settings > Generate sample findings:

One of the problems with a tool like GuardDuty that… isn’t really a problem, is that you won’t see anything unless you are under attack (or simulate one). This little button will create a bunch of dummy findings so you can see what they look like.

This finding would trigger if there was malware on a system and it was connecting to a known command and control server. How would AWS know this? AWS uses proprietary (meaning secret) threat intelligence, which includes subscriptions to commercial sources, to track the IP addresses of known C&C systems (not music factories). They correlate this with VPC Flow Logs (which tracks network connections) for your accounts. We’ll get to VPCs soon but those are the virtual networks where you run things. AWS also knows the port being used, and that this port is associated with that C&C activity.

This is exactly how an IDS works! Monitoring network traffic for connections to known hostile stuff. Now this example is very “traditional IDS” and there are a bunch of other findings here that use alternative techniques. Over time we’ll learn more about them, but I encourage you to poke around a bit, look at the info GuardDuty provides, and get a general sense of things until future labs where we have the time to go deeper.

Lab Key Points

• GuardDuty is a regional service. Even if you enable it for all accounts, you need to change that setting in every region. To enable it for all accounts, you need to change that setting in every region.

• Auto-enable for all accounts doesn’t work for your existing accounts, so make sure you add those manually. This is the first time I used this feature myself since I usually use Infrastructure as Code (CloudFormation/Terraform) instead.

• You will start getting charged in 30 days but it won’t cost much.

-Rich