AWS CLI

Tips and tricks to install and use the MacOS Terminal for Programmatic access to AWS.

Overview

This is a hands-on tutorial to get new users setup to effectively access and use the AWS cloud. Here you do some action and explanations and PROTIP advice is provided. PROTIPs included how to install and use AWS CLI automation, smart phone apps, and 3rd party tools used by the pros.

NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.

This is a part of my on-boarding and Amazon’s Getting Started tutorials. They cover getting API Keys and Credentials.

Instead of clicking and typing the manual AWS Management Console GUI, server administrators work with Programmatic access to AWS by crafting and running programs in the Command Line Interface (CLI).

CLI programs are used to invoke Cloud Formation or Terraform files which are the basis for creating resources in AWS.

This is so the build process can be debugged and changed slightly through the lifecycle from test to prod.

Instead of doing what other clouds do (some aws login command to prompt for a user name and password then storing credentials in a browser cookie), each aws command references a specifically-named file at $HOME/.aws/credentials created by command aws configure. The aws configure command creates that file after prompting for access key identifiers (AKIDs) to an AWS account. Press Enter to accept the value previously defined:

AWS CLI install

The following can be done from any folder within a MacOS Terminal.app:

Several ways are presented to install AWS CLI.

The simplest and most reliable for me is to use HomeBrew on Macs. If you have Homebrew installed:

brew info awscli

awscli: stable 2.2.39 (bottled), HEAD
Official Amazon AWS command-line interface
https://aws.amazon.com/cli/
/usr/local/Cellar/awscli/2.2.39 (12,035 files, 88.5MB)
  Poured from bottle on 2021-09-21 at 06:36:33
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/awscli.rb
License: Apache-2.0
==> Dependencies
Build: cmake ✘
Required: python@3.9 ✔, six ✔
==> Options
--HEAD
 Install HEAD version
==> Caveats
The "examples" directory has been installed to:
  /usr/local/share/awscli/examples
 
Bash completion has been installed to:
  /usr/local/etc/bash_completion.d
==> Analytics
install: 196,108 (30 days), 527,336 (90 days), 1,879,114 (365 days)
install-on-request: 194,585 (30 days), 523,050 (90 days), 1,859,544 (365 days)
build-error: 0 (30 days)

Notice that executables within folder path which does not require messing with PATH in ~/bash_profile:

/usr/local/Cellar/awscli/

Using Homebrew would allow you to upgrade version with a single command:
```
brew upgrade awscli
```
If awscli was not already installed:
```
brew install awscli
```
Surprisingly, the size decreased for a while over previous versions:
```
/opt/homebrew/Cellar/awscli/2.9.10: 12,995 files, 113.8MB
/usr/local/Cellar/awscli/2.2.39 (12,035 files, 88.5MB)
/usr/local/Cellar/awscli/2.2.21 (12,806 files, 100.3MB)
/usr/local/Cellar/awscli/2.2.14 (12,776 files, 101.8MB)
```
NOTE: Homebrew automatically installs the latest dependencies Ansible, ykman, etc.

Alternately, one can use pip install awscli –upgrade –user –ignore-installed six installed from https://pypi.org/project/awscli. But when I did, aws cannot be found.

Another alternative to install (on CentOS 7) is:
```
curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" \
   -o "awscli-bundle.zip"
unzip awscli-bundle.zip 
sudo ./awscli-bundle/install \
   -i /usr/local/aws -b /usr/local/bin/aws
```
Verify what version of awscli you have installed:
```
aws --version
```
Something went wrong if your response is:
```
-bash: aws: command not found
```
The expected sample response on an ARM M1 mac:
```
aws-cli/2.9.4 Python/3.11.0 Darwin/21.6.0 source/arm64 prompt/off
```
NOTE: “Darwin” is the name of the operating system internally within MacOS.

Previously:
```
aws-cli/2.2.22 Python/3.8.8 Darwin/18.7.0 exe/x86_64 prompt/off
aws-cli/2.2.21 Python/3.9.6 Darwin/18.7.0 source/x86_64 prompt/off
aws-cli/1.15.20 Python/3.6.5 Darwin/17.5.0 botocore/1.10.20
```
NOTE: Awscli now uses Python 3, not 2.7.

AWS Boto for Python

PROTIP: “AWS SDK for Python” enables your Python (.py) programs to invoke AWS CLI commands.

The Python package botocore on GitHub provides a low-level foundation for AWS CLI software.

Ansible internally uses Boto to connect to Amazon EC2 instances and hence you need Boto library in order to run Ansible on your laptop/desktop. TOOL: Use Ansible to copy files from local to remote host.
Make sure you’re not within Conda:
```
conda deactivate
```
If it’s already deactivated, you should not get any message.
To install Boto3:
```
pip install boto3 --upgrade --user --ignore-installed six
```
At the end of response:
```
Installing collected packages: botocore, boto3
Successfully installed boto3-1.18.47 botocore-1.21.47
```
Code for boto3 is obtained from https://github.com/boto/boto3. Read about it at https://aws.amazon.com/sdk-for-python.

NOTE: The package is installed into folder:
/usr/local/lib/python2.7/site-packages/boto3/*

It’s in /usr/local/anaconda3/lib/python3.7/site-packages (2.49.0)

The boto package is the hand-coded Python library that has been around since 2006. It is very popular and is fully supported currently by AWS. But because it is hand-coded and there are so many services available (with more appearing all the time) it is difficult to maintain.

boto3, generally available since 06/22/2015, is a new version of the boto library based on botocore. All of the low-level interfaces to AWS are driven from JSON service descriptions that are generated automatically from the canonical descriptions of the services. So, the interfaces are always correct and always up to date. There is a resource layer on top of the client-layer that provides a nicer, more Pythonic interface. The boto3 library is being actively developed by AWS and is the devs should use when starting new development.

BTW An example of a Lambda Python program to send email here.

Bash Shell completions

On Linux, to enable bash completion for aws commands:

echo "\n" >> ~/.bashrc
echo 'complete -C aws_completer aws' >> ~/.bashrc

Test out autocompletion by typing the first two characters and pressing Tab for a list of all aws cli commands that begin with those characters:

AWS Shell completion

PROTIP: For automatic complex autocompletion of AWS CLI commands, there is a 3rd-party utility that provides a shell GUI that suggest as you type:

Read about it at https://github.com/awslabs/aws-shell

To install the awesome AWS Shell:

pip install aws-shell

The package is installed in folders: /usr/local/bin/aws-shell

If you see these error messages:

ERROR: requests 2.22.0 has requirement urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1, but you'll have urllib3 1.26.6 which is incompatible.
ERROR: jupyter-console 6.0.0 has requirement prompt_toolkit<2.1.0,>=2.0.0, but you'll have prompt-toolkit 1.0.18 which is incompatible.
ERROR: ipython 7.6.1 has requirement prompt-toolkit<2.1.0,>=2.0.0, but you'll have prompt-toolkit 1.0.18 which is incompatible.

To enable AWS Shell:

aws-shell

First run, creating autocomplete index...
Creating doc index in the background. It will be a few minutes before all documentation is available.

You show now be in the sub-shell with prompt:

aws>

Exit aws-shell back to bash:
```
.exit
```
Alternately, .quit works too.

jp command

The jp command enables JSON to be manipulated within Bash scripts.

Install it on Macs, in any folder:

brew tap jmespath/jmespath
brew install jp

🍺  /usr/local/Cellar/jp/1.1.12: 3 files, 3MB

Verify it works by running a sample command:

For example, jp enables a simple syntax to extract text:
```
aws iam get-user | jq -r ".User.CreateDate[:4]"
```
The response should be: 2021 from the first 4 characters of CreateDate within User.
See other usage and examples at:
- https://github.com/jmespath/jp#usage
- https://shapeshed.com/jq-json/
- https://stedolan.github.io/jq/download/
jp is required by Aliases, below.

aws sub-commands

Further explained in this video:

AWS automatically reads and processes file “alias” in your local folder path ~/.aws/cli/.
Although https://github.com/awslabs/awscli-aliases has not been updated since 2016 (5 years ago), was tested with 1.11.24 of the AWS CLI and now obsolete, it has some convenient functions:
- aws whoami = sts get-caller-identity
- aws create-assume-role role
- aws running-instances
- aws ebs-volumes
- aws amazon-linux-amis
- aws list-sgs
- aws sg-rules
- aws tostring “string”
- aws tostring-with-jq “string”
- aws authorize-my-ip ip
- aws authorize-my-ip-by-name group_id
- aws get-group-id
- aws public-ports
- aws region
- aws find-access-key “access key id”
- aws docker-ecr-login
- aws myip
- aws allow-my-ip group, protocol, port
- aws allow-my-ip-all
- aws revoke-my-ip group, protocol, port
- aws revoke-my-ip-all my ip
If you forget to type a parameter for a subcommand that expects it, you’ll see:
```
Parameter validation failed:
Invalid length for parameter RoleName, value: 0, valid min length: 1
```
Create folder ~/.aws/cli/alias:

Load:

mkdir -p ~/.aws/cli
pushd ~/.aws/cli
curl -O https://raw.githubusercontent.com/awslabs/awscli-aliases/master/alias
popd

Try it: because alias defined aws whoami = sts get-caller-identity
```
aws whoami
```
should execute sts get-caller-identity.

Unlike AWS sub-commands, OS-level Keyboard shortcuts do not need “aws” in front of the subcommand.

TODO: pull in my custom keyboard.
```
mkdir -p ~/.aws/cli
pushd ~/.aws/cli
curl -O https://raw.githubusercontent.com/awslabs/awscli-aliases/master/alias
popd
```

aws configure

Amazon documentation says to run:

aws configure

That command prompts acceptance or override of default AWS ACCESS KEY ID, AWS SECRET ACCESS KEY, and region saved as a plain-text file at
```
~/.aws/credentials
```
Sample contents:
```
[default]
aws_access_key_id = ABCDEFGHIJKLMNOPQRST
aws_secret_access_key = 123456786iJsvzQbkIlDiFtBh6DrPzIw8r7hVb35
[py-ec2–1]
aws_access_key_id = ABCDEFGHIJKLMNOPQRST
aws_secret_access_key = 123456782Nwk156aPF0SxZ8KGY+RrhEbq3AIHUSS
```
BTW Progress toward AWS providing a more secure approach is at https://github.com/aws/aws-sdk/issues/41

Configure profiles

PROTIP: You’ll likely have a different AWS account for each enviornment (dev, qa, stage, prod), so specify a profile for each account.

NOTE: https://awsu.me/general/overview.html (Awsume) is a command-line utility for retrieving and exporting AWS credentials to your shell’s environment. With awsume, you can get credentials for any profile located in your config and credentials files (opens new window), including those that require MFA or an assume-role call.
Run the command to create files in folder ~/aws referenced by all other aws cli commands:
```
aws configure --profile dev
```
PROTIP: The example “root-admin-work” would be replaced with the user’s account name being created. Different accounts may be needed for different permissions in prod vs. dev use. Having separate access keys for different applications also generates distinct entries in AWS CloudTrail log files, which makes it easier to determine which application performed specific actions.

Without the profile specification, “aws configure” by itself defines default credentials.

The command prompts you for:
```
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-west-2
Default output format [None]: json
```
PROTIP: If you do not explicitly specify an endpoint, US West (Oregon) us-west-2 is the default Region.

The default output format is json.

PROTIP: The aws configure command creates key/value pairs “aws_access_key_id” and “aws_secret_access_key” in file credentials for use by all AWS SDKs. Key/value pairs “region” and “output” are saved in file config used by the CLI.

TODO: http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#cli-environment

NOTE: Set default region using PowerShell

The region in ~/.aws/config can be set also by:

aws configure set profile.prod.region us-west-1

Path ~/.aws/config is in variable $AWS_CONFIG_FILE

Path ~/.aws/credentials is in variable $AWS_SHARED_CREDENTIALS_FILE

aws configure set region \
   $(curl -s http://162.254.169.254/latest/dynamic/instance-identity/document \
   | jp -u 'region')

Roles for Tasks

TODO: Temporary security credentials Roles for Tasks are stored in the ~/.aws/config file:

[profile iam-role]
role_arn = arn:aws:iam::ACCOUNT_ID:role/IAM_ROLE
source_profile = iam-user
output = json
region = eu-west-1

Importantly, the default region is specified in ~/.aws/config.

PROTIP: The ~/.aws/config file also houses settings that speed up S3 sync.

[profile default]
...
s3 =
  max_concurrent_requests = 100
  max_queue_size = 10000
  use_accelerate_endpoint = true

aws-info.sh

In a Terminal, navigate to a folder in your PATH.

Download the aws-info.sh shell script:

curl -fsSL https://raw.githubusercontent.com/wilsonmar/DevSecOps/master/aws/aws-info.sh

Get the menu by running the script without any parameters:

chown +x aws-info.sh
./aws-info.sh

Notice:

-userinfo    to show User info
-netinfo     to show Network info
-svcinfo     to show Services info with cost history
 
-lambdainfo  to show Lambda info
-amiinfo     to show AMI info
-ec2info     to show EC2 info
 
-s3info      to show S3 info
-diskinfo    to show Disk info
-dbinfo      to show Database info
-certinfo    to show Certificates info
-loginfo     to show Logging info

Run to obtain all data:
```
./aws-info.sh -v -allinfo
```
User Info (IAM)

Identity and Access Management (IAM) roles for Amazon EC2:

List users:


aws iam list-users --query Users[*].UserName

List groups which the user belongs to :

aws iam list-groups-for-user --username ???

Create a new user named “MyUser”:

aws iam create-user --user-name MyUser

Sample response:

{
 "User": {
     "UserName": "MyUser",
     "Path": "/",
     "CreateDate": "2012-12-20T03:13:02.581Z",
     "UserId": "AKIAIOSFODNN7EXAMPLE",
     "Arn": "arn:aws:iam::123456789012:user/MyUser"
 }
}

Add the user to the group:

aws iam add-user-to-group --user-name MyUser --group-name MyIamGroup

To verify that the MyIamGroup group contains the MyUser, use the get-group command:

aws iam get-group --group-name MyIamGroup

The response:

 {
     "Group": {
         "GroupName": "MyIamGroup",
         "CreateDate": "2012-12-20T03:03:52Z",
         "GroupId": "AKIAI44QH8DHBEXAMPLE",
         "Arn": "arn:aws:iam::123456789012:group/MyIamGroup",
         "Path": "/"
     },
     "Users": [
         {
             "UserName": "MyUser",
             "Path": "/",
             "CreateDate": "2012-12-20T03:13:02Z",
             "UserId": "AKIAIOSFODNN7EXAMPLE",
             "Arn": "arn:aws:iam::123456789012:user/MyUser"
         }
     ],
     "IsTruncated": "false"
 }

DOCS

aws sts assume-role --role-arn arn:aws:iam::123456789012:role/role-name \
--role-session-name "RoleSession1" \
--profile IAM-user-name > assume-role-output.txt

https://aws.amazon.com/blogs/security/how-to-rotate-access-keys-for-iam-users/

aws iam list-access-keys
{
 "AccessKeyMetadata": [
     {
         "UserName": "Wilson_Mar",
         "AccessKeyId": "ABCDEFGHIJKLMNOPQRST",
         "Status": "Active",
         "CreateDate": "2020-06-12T04:04:22+00:00"
     }
 ]
}

AWS IAM commands use unique access key identifiers (AKIDs) to refer to individual access keys.

aws iam create-access-key --user-name Alice

Services list

For a list of Amazon services with command access:

aws commands help

PROTIP: Drag the left/right edge of the Terminal to widen the screen.

usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
 
  aws help
  aws <command> help
  aws <command> <subcommand> help
 
aws: error: argument command: Invalid choice, valid choices are:
 
accessanalyzer                           | acm
acm-pca                                  | alexaforbusiness
amp                                      | amplify
amplifybackend                           | apigateway
apigatewaymanagementapi                  | apigatewayv2
appconfig                                | appflow
appintegrations                          | application-autoscaling
application-insights                     | applicationcostprofiler
appmesh                                  | apprunner
appstream                                | appsync
athena                                   | auditmanager
autoscaling                              | autoscaling-plans
backup                                   | batch
braket                                   | budgets
ce                                       | chime
cloud9                                   | clouddirectory
cloudformation                           | cloudfront
cloudhsm                                 | cloudhsmv2
cloudsearch                              | cloudsearchdomain
cloudtrail                               | cloudwatch
codeartifact                             | codebuild
codecommit                               | codeguru-reviewer
codeguruprofiler                         | codepipeline
codestar                                 | codestar-connections
codestar-notifications                   | cognito-identity
cognito-idp                              | cognito-sync
comprehend                               | comprehendmedical
compute-optimizer                        | connect
connect-contact-lens                     | connectparticipant
cur                                      | customer-profiles
databrew                                 | dataexchange
datapipeline                             | datasync
dax                                      | detective
devicefarm                               | devops-guru
directconnect                            | discovery
dlm                                      | dms
docdb                                    | ds
dynamodb                                 | dynamodbstreams
ebs                                      | ec2
ec2-instance-connect                     | ecr
ecr-public                               | ecs
efs                                      | eks
elastic-inference                        | elasticache
elasticbeanstalk                         | elastictranscoder
elb                                      | elbv2
emr                                      | emr-containers
es                                       | events
finspace                                 | finspace-data
firehose                                 | fis
fms                                      | forecast
forecastquery                            | frauddetector
fsx                                      | gamelift
glacier                                  | globalaccelerator
glue                                     | greengrass
greengrassv2                             | groundstation
guardduty                                | health
healthlake                               | honeycode
iam                                      | identitystore
imagebuilder                             | importexport
inspector                                | iot
iot-data                                 | iot-jobs-data
iot1click-devices                        | iot1click-projects
iotanalytics                             | iotdeviceadvisor
iotevents                                | iotevents-data
iotfleethub                              | iotsecuretunneling
iotsitewise                              | iotthingsgraph
iotwireless                              | ivs
kafka                                    | kendra
kinesis                                  | kinesis-video-archived-media
kinesis-video-media                      | kinesis-video-signaling
kinesisanalytics                         | kinesisanalyticsv2
kinesisvideo                             | kms
lakeformation                            | lambda
lex-models                               | lex-runtime
lexv2-models                             | lexv2-runtime
license-manager                          | lightsail
location                                 | logs
lookoutequipment                         | lookoutmetrics
lookoutvision                            | machinelearning
macie                                    | macie2
managedblockchain                        | marketplace-catalog
marketplace-entitlement                  | marketplacecommerceanalytics
mediaconnect                             | mediaconvert
medialive                                | mediapackage
mediapackage-vod                         | mediastore
mediastore-data                          | mediatailor
meteringmarketplace                      | mgh
mgn                                      | migrationhub-config
mobile                                   | mq
mturk                                    | mwaa
neptune                                  | network-firewall
networkmanager                           | nimble
opsworks                                 | opsworkscm
organizations                            | outposts
personalize                              | personalize-events
personalize-runtime                      | pi
pinpoint                                 | pinpoint-email
pinpoint-sms-voice                       | polly
pricing                                  | proton
qldb                                     | qldb-session
quicksight                               | ram
rds                                      | rds-data
redshift                                 | redshift-data
rekognition                              | resource-groups
resourcegroupstaggingapi                 | robomaker
route53                                  | route53domains
route53resolver                          | s3control
s3outposts                               | sagemaker
sagemaker-a2i-runtime                    | sagemaker-edge
sagemaker-featurestore-runtime           | sagemaker-runtime
savingsplans                             | schemas
sdb                                      | secretsmanager
securityhub                              | serverlessrepo
service-quotas                           | servicecatalog
servicecatalog-appregistry               | servicediscovery
ses                                      | sesv2
shield                                   | signer
sms                                      | snowball
sns                                      | sqs
ssm                                      | ssm-contacts
ssm-incidents                            | sso
sso-admin                                | sso-oidc
stepfunctions                            | storagegateway
sts                                      | support
swf                                      | synthetics
textract                                 | timestream-query
timestream-write                         | transcribe
transfer                                 | translate
waf                                      | waf-regional
wafv2                                    | wellarchitected
workdocs                                 | worklink
workmail                                 | workmailmessageflow
workspaces                               | xray
s3api                                    | s3
ddb                                      | configure
deploy                                   | configservice
opsworks-cm                              | history
cli-dev                                  | help
whoami                                   | create-assume-role
running-instances                        | ebs-volumes
amazon-linux-amis                        | list-sgs
sg-rules                                 | tostring
tostring-with-jq                         | authorize-my-ip
get-group-id                             | authorize-my-ip-by-name
public-ports                             | region
find-access-key                          | docker-ecr-login
myip                                     | allow-my-ip
revoke-my-ip                             | allow-my-ip-all
revoke-my-ip-all

See http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-using.html

https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html
https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys
https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html

Grant temporary access keys - aws sts assume-role.

https://docs.aws.amazon.com/secretsmanager/latest/userguide/rotating-secrets-one-user-multiple-passwords.html

Additionally, add conditions to the policy that further restrict access, such as the source IP address range of clients. The example policy below grants the needed permissions (PutObject) on to a specific resource (an S3 bucket named “examplebucket”) while adding further conditions (the client must come from IP range 203.0.113.0/24):

{
 "Version": "2012-10-17",
 "Id": "S3PolicyRestrictPut",
 "Statement": [
         {
         "Sid": "IPAllow",
         "Effect": "Allow",
         "Principal": "*",
         "Action": "s3:PutObject",
         "Resource": "arn:aws:s3:::examplebucket/*",
         "Condition": {
             "IpAddress": {"aws:SourceIp": "203.0.113.0/24"}
         } 
     } 
 ]
}

bash-my-aws

https://bash-my-aws.org/ features Short, memorable commands:

buckets
instances
stacks grep postgres
keypairs

https://github.com/bash-my-aws/bash-my-aws/

Linux AMIs

Types of operating system AMI:

Amazon Linux 2014.09.2 (CentOS)
Red Hat Enterprise Linux 6.6 (RHEL)
SUSE Linux Enterprise Server 12
Ubuntu Server 14.04

for loop

for region in $( aws ec2 describe-regions --output text --query "Regions[].[RegionName]" | tr "\\n" " "  ); do echo "some $region" ; done

for region in $( aws ec2 describe-regions –output text –query “Regions[].[RegionName]”

tr “\n” “ “ ); do echo “some $region” ; done

Define groups to assign permissions

PROTIP: For a user to do something usually require several AWS resources. So several permissions need to be granted to a user. To simplify assignments, we define Groups of permissions which we then can assign to each user.

In other words, An IAM group is a management convenience to manage the same set of permissions for a set of IAM users.

The AWS CLI command to create a group named “MyIamGroup” is:

aws iam create-group --group-name MyIamGroup

A sample response:

{
    "Group": {
        "GroupName": "MyIamGroup",
        "CreateDate": "2012-12-20T03:03:52.834Z",
        "GroupId": "AKIAI44QH8DHBEXAMPLE",
        "Arn": "arn:aws:iam::123456789012:group/MyIamGroup",
        "Path": "/"
    }
}

Create a S3 security group:

aws ec2 create-security-group --group-name my-sg --description "My security group"

A sample response:

{
"GroupId": "sg-903004f8"
}

Click Manage Groups then Create New Group.

PROTIP: Groups are usually associated with a particular job: admin, sales, HR, front-end developer, back-end developer, etc.

A user can belong to multiple groups. More complex organizations manage differences in permissions for company, division, project, location, job level, etc. So 128 characters may not be enough if large words are used. Thus, abbreviate and use acronyms.

PROTIP: Put abbreviations and acronyms in a wiki publicly available to avoid duplicate usage.
“aws_iot_buttons” is the group name I use as an example.

PROTIP: Use dashes. Space characters are not allowed. On March 1, 2018 AWS removed the ability to use underscores in S3 bucket names.

The list shown are “AWS Managed”.

Click on Policy Type to select Job function.
PROTIP: Instead of scrolling down the massive list in Attache Policy (Alexa, Amazon, AWS, etc.), type in the Filter field the first few letters (such as “IoT”) and the list gets smaller. Notice the filter you type is applicable to not just characters beginning with what you typed, but also characters inside names as well.
Click to select.
Click “Create Group”.

Note different policies have different levels of access, with admin having more capabilities than “read only” ones.
Names shown on the screen is called a “Policy Summary”.
Click “JSON” to see the file that AWS reads to assign policies. Here you seen what Actions the policy allows.
Click “Access Advisor” to see users who have been assigned to use the policy.

https://docs.aws.amazon.com/iot/latest/developerguide/create-iot-policy.html

AWS Policy Generator

The AWS Policy Generator at https://awspolicygen.s3.amazonaws.com/policygen.html creates policy files that control access to Amazon Web Services (AWS) products and resources:

SQS Queue Policy
S3 Bucket Policy
VPC Endpoint Policy
IAM Policy
SNS Topic Policy

https://aws.amazon.com/developer/?developer-center-activities-cards.sort-by=item.additionalFields.startDateTime&developer-center-activities-cards.sort-order=asc

References

https://awscli.amazonaws.com/v2/documentation/api/latest/reference/index.html

https://github.com/aws/aws-cli/blob/develop/README.rst

https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html

https://forums.aws.amazon.com/forum.jspa?forumID=150

TODO: Put each AWS CLI command in a script at https://medium.com/circuitpeople/aws-cli-with-jq-and-bash-9d54e2eabaf1 by Lee Harding

https://github.com/jlevy/the-art-of-command-line#macos-only

https://gist.github.com/mikepfeiffer/

https://aws.amazon.com/powershell
AWS Powershell for Windows</a>

https://github.com/jlevy/the-art-of-command-line#macos-only

AWS Config

DOCS

An example of a service that requires AWS CLI to turn on and off is AWS Config.

The AWS Config Recorder, by default, detects and captures changes in resource configurations as configuration items.

Costs are around $4 a month, based on:

the number of configuration items and
the number of rule evaluations performed

Conformance packs

https://console.aws.amazon.com/config/

List the rules:

aws configservice describe-config-rules --output json \
| grep ConfigRuleName \
| cut -d":" -f2 \
| cut -d"," -f1

Sample responses:

"eks-cluster-oldest-supported-version-conformance-pack-qmmhw2vhu"
 "eks-cluster-supported-version-conformance-pack-qmmhw2vhu"
 "eks-endpoint-no-public-access-conformance-pack-qmmhw2vhu"
 "eks-secrets-encrypted-conformance-pack-qmmhw2vhu"

Delete?

aws configservice describe-config-rules | grep ConfigRuleName | gawk &#39;match($0, /:.+"(.+)"/, a) {print a[1]}&#39; | while read rule_name; 
do
 echo $rule_name; 
 aws configservice delete-config-rule --config-rule-name $rule_name
done

List the rules:

aws configservice describe-config-rules --output json \
| grep ConfigRuleName \
| cut -d":" -f2 \
| cut -d"," -f1 \
| xargs -L1 aws configservice delete-config-rule \
--config-rule-name

Turn off Recording for that region using the console
Delete the Rule by going to actions, delete rule
Use the AWS CLI and delete the default recording by

aws configservice delete-configuration-recorder –configuration-recorder-name default –region region-name

Delete the service linked role created for AWS Config
Refresh the Config home page to make it appear fresh.
If necessary delete the config bucket and its objects.

for AWS_REGION in $(aws ec2 describe-regions –output text
–query ‘Regions[].[RegionName]’) ; do echo “$AWS_REGION:”;
for snap in $(aws ec2 describe-snapshots –owner self –output text –region $AWS_REGION
–query ‘Snapshots[*].SnapshotId’); do aws ec2 describe-snapshot-attribute
–snapshot-id $snap –region $AWS_REGION –output text –attribute createVolumePermission
–query ‘[SnapshotId,CreateVolumePermissions[?Group == all]]’; done; echo; done

https://opensourceconnections.com/blog/2015/07/27/advanced-aws-cli-jmespath-query/ aws ec2 describe-images –owner amazon –query ‘Images[].[ImageId,Name]’ –output text | grep -m5 “ami-“

https://www.nops.io/unused-aws-ebs-volumes/

Video Tutorials

A good one on this topic is David Clinton’s “Using Docker on AWS with the Command Line” on Pluralsight, which goes into ECS, Fargate, and EKS using Kubernetes. The course is the basis for “Teach yourself Data Analytics in 30 days: Learn to use Python and Jupyter Notebooks by exploring fun, real-world data projects” at bootstrap-it.com/docker4aws. For your convenience, I’ve made code from David’s webpage available at https://github.com/wilsonmar/DevSecOps/tree/main/docker4aws

More recently (2/2019) is Manuj Aggarwal’s $84.99 https://www.udemy.com/course/aws-masterclass-aws-command-line-interface-and-devops/

https://www.kofrimpong.com/azure-cli-and-jmespath-query-part-2/

machines[?state=='running'].name

yields:

{
  "machines": [
    {"name": "a", "state": "running"},
    {"name": "b", "state": "stopped"},
    {"name": "b", "state": "running"}
  ]
}

More on Amazon

This is one of a series on Amazon:

More on DevOps

This is one of a series on DevOps:

More on Security

This is one of a series on Security in DevSecOps:

Wilson Mar

AWS CLI

AWS CLI install

AWS Boto for Python

Bash Shell completions

AWS Shell completion

jp command

aws sub-commands