OUs, SCPs, and a Root User Account Recovery

This lab focuses more on some AWS mechanics as we rework our SCPs and learn how to safely recover, then forget, a root user account

Special Announcement

I started CloudSLAW with two big goals in mind. First, to improve the security skills of anyone with an interest. you don’t need to be a full time security pro to care about security. But for those that do want to enter the field, I hope CloudSLAW will lower the barrier to entry.

That’s why I’m excited that my friend and collaborator Will Bengtson co-published a new book with the same goal. The Security Path is a collection of dozens of interviews with security professionals in all sorts of different roles; having them tell their stories of how they each started in their journey into the profession. It’s a cheat code for understanding where to start and how to direct your career.

Will and I have been teaching together at Black Hat for years and he provided a 40% off discount code for CloudSLAW subscribers. Just use the super secret code, cl0udslaw.

Prerequisites

The Lesson

To date most labs have illustrated a general security principle. Like how we talked about entities, identities, and personas, and showed what those meant by creating IAM users and roles. This week we focus on specific AWS mechanics you’ll use quite a bit in the future, similar to what you might see on Azure or GCP, but which don’t map to some 50-year-old core security concept.

Our objective today is to rename our SecurityAudit account to LogArchive, to better align with how most people name it. This is a great excuse to practice some techniques around SCPs, OUs, and using the root user account (one of the rare occasions).

We will do three things:

  • Move the SCP we use to protect the root account and keep it in the organization. We will take it off the root of our org and, instead, apply it to every Organizational Unit — with exceptions.

  • We will recover the root user account for the SecurityAudit account in a secure way, so we can change the account name and root user email. This is a three-step process:

    • Move the SecurityAudit account to the exceptions OU so we can use the root account. Otherwise the Service Control Policy would block all the API calls from root, and we couldn’t do anything.

    • “Recover” the root user account by resetting the password. Except, in our case, we will set it to a strong random password we won’t save anywhere, so we can’t ever use it again.

    • Make our changes, then log out and move the account back into the Security OU, where the SCP will resume blocking any activity by root.

All these steps are processes you will use in the future for different reasons, including:

  • As we build out we will have a bunch of different SCPs to set different security requirements on our OUs. For example our Workloads > Prod will be pretty locked down for our production accounts, while Sandbox accounts will be more loosey-goosey (apologies to my international readers for that Americanism).

  • We will regularly move accounts between OUs to change the security protections provided by SCPs. For example when an account needs to be promoted from Dev to Prod.

  • While we want to minimize using the root user account, sometimes we need to. Perhaps for incident response to kick out someone who compromised that admin IAM user which shouldn’t be there, or just to change billing settings. So this is a process you need to understand, along with how to do it as safely as possible.

  • I can’t tell you how many times I’ve walked in on a consulting engagement to perform an assessment, and found a root user account tied to the personal email of a developer who hasn’t worked there in 8 years. Or the email was compromised. Or they changed email providers and… you get the point. Stuff happens, and changing a root user email is another mechanic with security implications that you need to know.

This collection is easier to show than tell, so let’s get right into the lab.

Lesson Key Points:

  • SCPs allow you to define security at the OU or account level, or both. For now we will work at the OU level.

  • Moving an account to a different OU immediately changes which SCPs are enforced.

  • When we create accounts using AWS Organizations, we set the root email but no password is defined. To use that root user account, we perform account recovery. It is important to do this as safely as possible and to limit future exposure.

  • Sometimes we need the root user account, but we still really want to use it as little as possible.

  • We will sometimes need an OU without any restrictions on it to perform sensitive actions we normally block, as we will today.

The Lab

I already covered it up above. As a quick overview:

  • We will move our SCP off the root of the org (yeah, AWS uses the word ‘root’ too much) onto individual OUs.

  • We will move our SecurityAudit account into the Exceptions OU, which will not have the SCP protecting the root user on it.

  • We will perform a safe (as much as possible) root user account recovery, and briefly discuss one additional security control in case you are still worried.

  • We will change the name and email of the root user account.

  • We will move the root account out of Exceptions and back into Security, so it is blocked again.

  • We will create a new SecurityAudit account.

This sounds like a lot, but I timed it out when I recorded the video, and even making mistakes and explaining everything, it all took 12 minutes.

Although we use the SCP to lock down that root user account, it can still log into the console. The login process is not an API call we can block. While our SCP will block nearly everything, there are some things they cannot deny. From the AWS documentation:

  • Any action performed by the management account

  • Any action performed using permissions that are attached to a service-linked role

  • Register for the Enterprise support plan as the root user

  • Change the AWS support level as the root user

  • Provide trusted signer functionality for CloudFront private content

  • Configure reverse DNS for an Amazon Lightsail email server and Amazon EC2 instance as the root user

  • Tasks on some AWS-related services:

    • Alexa Top Sites

    • Alexa Web Information Service

    • Amazon Mechanical Turk

    • Amazon Product Marketing API

Video Walkthrough

Step-by-Step

First, go to your AWS IAM Identity Center Sign-In page and log in, then select your management account (mine is called CloudSLAW):

Stage 1: Set the SCPs

Then Organizations > Policies > Service control policies:

Now click on ProtectRootAndOrg. Don’t have that? Go back and do your labs, people!

As a quick reminder, this SCP prevents you from leaving the organization and blocks any action by the root user:

Now click Targets to see where that SCP is attached. Then click Attach so we can attach it to our other Organizational Units. Right now it’s on root protecting everything, but that prevents us from having our Exceptions OU allow root account actions, which is a big reason we created it.

On the next screen, attach the SCP to every OU except Exceptions! Let me repeat: do not attach it to Exceptions, but attach it to every other OU. Then Attach policy. What interesting is that if you watch the little spinner icon on the bottom of the screen, you can see it cycle as it issues an API call for each OU. That’s all the console does — make API calls for you.

When that’s done we finish by detaching the SCP from Root:

The first stage of the lab is complete. We moved the SCP onto all OUs except Exceptions. Then we removed it from root — otherwise it would still protect Exceptions. This gives us one OU where we can move accounts if we need to access root. If I was really good and really not lazy, I would have separated out the policy to Deny LeaveOrganization just in case someone tried that during the narrow window when the account is in Exceptions. And maybe we’ll do that later, but I am lazy — and also didn’t want to add any more steps to this lab.

Stage 2: Move SecurityAudit into the Exceptions OU

I put a bunch on one screenshot, but here are the steps:

  • Navigate to AWS accounts.

  • Expand the Security OU.

  • Check the box for SecurityAudit.

  • Copy the email address you used for SecurityAudit (either leave it in your clipboard, or better, paste it into a scrap text file)

  • Click Actions > Move.

Then pick Exceptions and Move AWS account:

If you did everything correctly it will look like this:

Now:

  • Click in the upper-right and Sign out.

  • Close the tab.

  • On your IAM Identity Center page, also click in the upper-right and Sign out.

  • If you need screenshots for that send me a letter in the US Mail. I’ll get back to ya.

Stage 3: Recover the Root User Account

Go to https://console.aws.amazon.com. Make sure Root user is selected, and enter the email address of your SecurityAudit account.

Now click Forgot password. After that screenshot I have a BUNCH of warnings and information about what to do next. Read all that before you go put 1234 as your new password. I don’t care if it works for your luggage.

On Recovering Root

This is an important and sensitive process. It’s critical you read and follow these steps:

  • When you click Forgot password, AWS will send a reset email to that email address. They will first hit you with the worst CAPTCHA in the history of humanity. I only get it right maybe 40% of the time.

  • That will send you the reset email with a link. Click it.

  • Use a password manager or a random text generator to create a long and strong password. Like, 20 characters or more.

    • DO NOT SAVE THIS PASSWORD, EVEN IN YOUR PASSWORD MANAGER.

    • Copy it to your clipboard, but don’t save it or write it down ANYWHERE. You want that thing to fade into the ether when you are done.

  • Set that password.

  • Log in.

  • KEEP THE PASSWORD IN YOUR CLIPBOARD MEMORY for a few more steps! Don’t copy anything else, until the step where we change the root account user email address.

  • When I tell you, forget that password ever existed

  • Send a T1000 back in time to kill that password’s mom. Don’t feel bad — the machine is the murderer, not you. She was probably a bad mom anyway — did you see that CAPTCHA?

WHY?!?

This is the process I use. I know other people who also set the MFA and then either forget that or don’t fully activate it. In my mind, unless your system is compromised at the time when you set that password, there should be no way for an attacker to get or recover it. But if even you paste it into a temporary text file, there is a chance of an autosaved version hiding somewhere.

Alternatively you can change the password to a random one generated by the AWS console when you are finished, but never look at or download it. That also works.

Stage 4: Change the Root User Account Email and Account Name

At this point you should be logged into the console as root. Go to the upper right corner and confirm you see a name which looks similar to this screenshot. If you see an @ sign, or your account ID, or anything else up there, you aren’t logged in as root. Then click Account.

On the Account screen, go to Account settings (right at the top) and then click Edit.

On the next screen click in the boxes to make the following changes:

  • First change the name of the account to LogArchive.

  • Then change the email so the “+” address part is “slawlogarchive”. In my case that means instead of [email protected] the email became [email protected].

In the first screenshot you can see what it looks like after I change the account name. In the second you’ll see you need to change the email and paste in your current password, which should still be on your clipboard. AWS will then email you a validation code at the new email address. You will need to copy that out and paste it in. This is the point where the root user account email we set will be overwritten in your clipboard and you will send that T-1000 back in time. If you prefer a T-100, that’s fine — I don’t judge, and I assume it’s a budget issue. Your loss.

Do as I say, not as I do

All done. Click in the upper-left and Sign out. 

WARNINGWARNINGWARNING!!!

If you haven’t figured it out yet, ANYONE can put in an email address and request a password reset. Worst yet, the AWS error message will tell them whether that email is real or not. They can’t reset the password unless they also have access to the email to click the reset link, so make darn sure that email is well protected.

We covered this way back in our first lab, and that’s why you want to use a random seed as part of the root email address for your management account. I also will add a random string to sensitive accounts, like these, which I create in organizations. For our lab purposes I’m skipping that but you absolutely should consider it in any enterprise environment.

Now they know.

Stage 5: Move LogArchive Back to the Security OU and Create a New SecurityAudit Account

At this point you are out of the root account and the password should have been forgotten and overwritten by your last copy/paste operation (unless you use a multi-item clipboard like I do, but really, that is low risk at this point). Now we want to get that SCP back on so even in case someone manages to recover that root account, they can’t do much.

Log back into your IAM Identity Center Portal.

Then go to Organizations > AWS accounts > Exceptions > check LogArchive > Actions > Move.

\

Then pick Security > Move AWS account.

Now LogArchive is in the Security OU, protected by the ProtectRootAndOrg SCP. Nifty.

To finish all this up we need to create a new SecurityAudit account. We will use a slightly different email, since you can’t reuse emails for AWS root user accounts. Look up and click Add an AWS account.

Call it SecurityAudit, and then you can use the same base email address as before but add a “1” to the end so it’s unique.

It takes 30 seconds or so to create, and you might need to refresh the page to see it. Notice that it isn’t in an OU, which means it isn’t protected by an SCP. Click on the new account, then Action > Move, and move it into the Security OU. Since we’ve done that twice today already, I won’t bother boring you with screenshots for every step.

Phew! That was a lot of steps and screenshots and jumping around, but it doesn’t really take that long.

-Rich

Join the conversation

or to participate.