Tuesday January 7th, 2020

AWS SSO CLI across AWS Organization accounts

Once having a large fleet of AWS accounts through AWS organization can be sometime hard to manage using AWS command line. Using SSO can make it easy to managed those fleet of account through AWS CLI which I will demo in this post.

In this tutorial I will be using three AWS accounts setup in my AWS Organization a main (kalilou), stage and prod account in conjunction with SSO users with some set of permissions. If you do not have those resources in place, here are the following steps to create them.

Create an AWS Organization resources
Create the different accounts inside the Organization
Create AWS SSO which upon creation will provide a SSO portal login URL
Create User(s) in the SSO console
In the SSO console, from AWS Accounts assign user and permission sets

AWS Organization

I have an AWS Organization which contains a master account (kalilou), prod and stage account and for more on AWS Organization, please visit https://docs.aws.amazon.com/singlesignon/latest/userguide/what-is.html

AWS SSO

Make sure you have setup and configure your AWS SSO and created user with permissions associated to them. I have created two users: kalilou and product owner and assigned them to different groups: Admin and ReadOnly

For more on AWS SSO, please visit https://docs.aws.amazon.com/singlesignon/latest/userguide/what-is.html

SSO AWS Account

SSO Users

SSO User Group

SSO login portal

AWS CLI v2

AWS now offers a version 2 of their CLI called AWS CLI v2 a next major version of AWS CLI which is in developer preview meaning it is only recommended for testing and evaluation but advised to used for production resources at this of point of time.

For any issue found, please report it at https://github.com/aws/aws-cli/issues?q=is%3Aopen+is%3Aissue+label%3Av2.

Please, revisit your AWS CLI setup and configuration when AWS officially releases it.

Installation and configuration

For more information on how to install AWS CLI v2, please visit https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html

This version is written in Python and can be installed through the installer which is pre-built for different operating system, no python required to be on the OS and the installer is self contained environment no need to worry about some dependencies (python folk world knows what I am talking about, for example if you have a python package installed system wide, the AWS CLI v2 installation will not affect it). Since this version is in developer preview, the installer will create an executable named aws2 living side by side with the AWS CLI v1 named aws. I will be using the pre-built installer for Mac.

$ curl "https://d1vvhvl2y92vvt.cloudfront.net/awscli-exe-macos.zip" -o "awscliv2.zip"
$ unzip awscliv2.zip
$ sudo ./aws/install

Now verify the installation by checking the current version and also your CLI v1 should be intact.

$ aws2 --version 
aws-cli/2.0.0dev3 Python/3.7.4 Darwin/18.7.0 botocore/2.0.0dev2

$ aws --version 
aws-cli/1.16.250 Python/3.7.6 Darwin/18.7.0 botocore/1.12.240

Now let's configure aws2 to make use of the SSO feature by running aws2 configure sso which is pretty much a guided prompt for assisting you on the setup you wanna achieve. When running the CLI, there will be steps where you will be prompted to choose which permission and account to use as well as the CLI will open your SSO login portal to confirm the login, please have a look at those images (IMAGE 1, 2 and 3) below.

$ aws2 configure sso 
SSO start URL [None]: https://REDACTED.awsapps.com/start
SSO Region [None]: eu-west-1
Attempting to automatically open the SSO authorization page in your default browser.
If the browser does not open or you wish to use a different device to authorize this request, open the following URL:

https://device.sso.eu-west-1.amazonaws.com/

Then enter the code:

# AT THIS STEP THE CLI WILL OPEN YOUR BROWSER POINTING TO THE SSO PORTAL URL (SEE IMAGE 1 BELOW) 
    
GFDV-DGGH
There are 3 AWS accounts available to you.

# AT THIS STEP THE CLI WILL LIST THE EXISTING ACCOUNT IN YOUR ORGANIZATION TO CHOOSE FROM (SEE IMAGE 2 BELOW)

Using the account ID prod-account-REDACTED

# AT THIS STEP THE CLI WILL LIST A NUMBER OF ROLE ASSIGNED TO YOUR USER FOR THE CHOSEN ACCOUNT ABOVE (SEE IMAGE 3)

There are 2 roles available to you.
Using the role name "AdministratorAccess"
CLI default client Region [None]: eu-west-1
CLI default output format [None]:

To use this profile, specify the profile name using --profile, as shown:

aws2 s3 ls --profile prod-account-sso

SSO login portal URL (IMAGE 1).

Choose which account (IMAGE 2).

Choose which Role to assume (IMAGE 3).

Now that we have the SSO setup and configured I will create 3 AWS profiles to make it easy to switch account. Create these in your _~/.aws/config _file.

[profile prod-sso]
sso_start_url = https://REDACTED.awsapps.com/start
sso_region = eu-west-1
sso_account_id = prod-account
sso_role_name = AdministratorAccess
region = eu-north-1
output = json


[profile sso-stage]
sso_start_url = https://REDACTED.awsapps.com/start
sso_region = eu-west-1
sso_account_id = stage-account
sso_role_name = AdministratorAccess
region = eu-north-1
output = json

[profile sso-master]
sso_start_url = https://REDACTED.awsapps.com/start
sso_region = eu-west-1
sso_account_id = master-account
sso_role_name = AdministratorAccess
region = eu-north-1

Now let test the setup and verify everything works as expected, using sso-prod to perform some API calls (list buckets)

$ aws2 s3 ls --profile sso-prod  # You can also export AWS_PROFILE=sso-prod and skip typing --profile 
...

Create buckets

# sso-test-revolight-1 bucket
$ aws2 s3api create-bucket --profile sso-prod --bucket sso-test-revolight-1 --region eu-west-1 --create-bucket-configuration LocationConstraint=eu-west-1
{
    "Location": "http://sso-test-revolight-1.s3.amazonaws.com/"
}

# sso-test-revolight-2 bucket
$ aws2 s3api create-bucket --profile sso-prod --bucket sso-test-revolight-2 --region eu-west-1 --create-bucket-configuration LocationConstraint=eu-west-1
{
    "Location": "http://sso-test-revolight-2.s3.amazonaws.com/"
}

# sso-test-revolight-3 bucket
$ aws2 s3api create-bucket --profile sso-prod --bucket sso-test-revolight-3 --region eu-west-1 --create-bucket-configuration LocationConstraint=eu-west-1
{
    "Location": "http://sso-test-revolight-3.s3.amazonaws.com/"
}

$ aws2 s3 ls --profile sso-prod
2020-01-07 14:40:10 sso-test-revolight-1
2020-01-07 14:40:37 sso-test-revolight-2
2020-01-07 14:41:25 sso-test-revolight-3

Now I can switch to other profiles to make some API calls

# sso-stage profile 
$ aws2 s3 ls --profile sso-stage 

# sso-master profile 
$ aws2 s3 ls --profile sso-master

Now let modify the sso-prod profile ssorolename to assume ViewOnlyAccess in _~/.aws/config _

[profile prod-sso]
sso_start_url = https://REDACTED.awsapps.com/start
sso_region = eu-west-1
sso_account_id = prod-account
sso_role_name = ViewOnlyAccess  # Changed role to ViewOnlyAccess
region = eu-north-1
output = json

We should be able to list all buckets

$ aws2 s3 ls --profile sso-prod 
2020-01-07 14:40:10 sso-test-revolight-1
2020-01-07 14:40:37 sso-test-revolight-2
2020-01-07 14:41:25 sso-test-revolight-3

With the ViewOnlyAccess Role, we should not able to create a bucket

$ aws2 s3api create-bucket --profile sso-prod --bucket sso-test-revolight-4 --region eu-west-1 --create-bucket-configuration LocationConstraint=eu-west-1

An error occurred (AccessDenied) when calling the CreateBucket operation: Access Denied

SSO login and logout

The SSO session will eventually expire and to refresh the session, run the login CLI command.

$ aws2 sso login

To logout, you can run the logout CLI command.

$ aws2 sso logout

Other interesting features

YAML output format

$ aws2 supports **_yaml_** format as output.

$ aws2  ec2 describe-subnets --profile sso-stage --output yaml

Subnets:
- AssignIpv6AddressOnCreation: false
    AvailabilityZone: eu-north-1c
    AvailabilityZoneId: eun1-az3
    AvailableIpAddressCount: ***
    ...............
- AssignIpv6AddressOnCreation: false
    AvailabilityZone: eu-north-1b
    AvailabilityZoneId: eun1-az2
    AvailableIpAddressCount: ***
    ................
............................

Auto completion

Auto completion for aws2 CLI is pretty handy when we don't always remember the options, commands or subcommands of the CLI. And the feature killer here is that resource can be autocompleted as well, which is an API call made in the background to get the list of resources, for example getting an IAM user.

$ complete -C aws2_completer aws2

I have 4 IAM users and with autocompletion I can get all those 4 users to select from.

Paging improvement

For instance when listing all s3 objects can result into some seconds delay since all pages are being aggregated before the result is sent to output.

This following CLI may involve some delays in seconds

$ aws2 s3api list-objects --bucket sso-test-revolight-1 --profile sso-prod --output yaml
Contents:
- ETag: '"26a498a85aa14f142c999675ec919085"'
    Key: Screenshot 2019-11-10 at 22.55.30.png
    LastModified: '2020-01-07T14:52:49+00:00'
    Size: 81822
    StorageClass: STANDARD
.......

But using yaml-stream output option instead of yaml will be faster since one page is retrieved at the time.

$ aws2 s3api list-objects --bucket sso-test-revolight-1 --profile sso-prod --output yaml-stream
- Contents:
    - ETag: '"26a498a85aa14f142c999675ec919085"'
    Key: Screenshot 2019-11-10 at 22.55.30.png
    LastModified: '2020-01-07T14:52:49+00:00'
    Size: 81822
    StorageClass: STANDARD
........

Interactive feature

In the console, basically for someone new to AWS, there is a get-started feature where through the console, the person is being assisted on how to start using the service and in aws2 a similar feature has been added.

# Interactively create new IAM user 
$ aws2 iam  create-user --cli-auto-prompt 

# Interactively create new dynamodb table 
$ aws2 dynamodb wizard new-table