Wilson Mar bio photo

Wilson Mar


Email me Calendar Skype call

LinkedIn Twitter Gitter Instagram Youtube

Github Stackoverflow Pinterest

Tips and tricks to install and use the MacOS Terminal for Programmatic access to AWS.

US (English)   Español (Spanish)   Français (French)   Deutsch (German)   Italiano   Português   Estonian   اَلْعَرَبِيَّةُ (Egypt Arabic)   中文 (简体) Chinese (Simplified)   日本語 Japanese   한국어 Korean


This is a hands-on tutorial to get new users setup to effectively access and use the AWS cloud. Here you do some action and explanations and PROTIP advice is provided. PROTIPs included how to install and use AWS CLI automation, smart phone apps, and 3rd party tools used by the pros.

NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.

This is a part of my on-boarding and Amazon’s Getting Started tutorials. They cover getting API Keys and Credentials.

Instead of clicking and typing the manual AWS Management Console GUI, server administrators work with Programmatic access to AWS by crafting and running programs in the Command Line Interface (CLI).

CLI programs are used to invoke Cloud Formation or Terraform files which are the basis for creating resources in AWS.

This is so the build process can be debugged and changed slightly through the lifecycle from test to prod.

Instead of doing what other clouds do (some aws login command to prompt for a user name and password then storing credentials in a browser cookie), each aws command references a specifically-named file at $HOME/.aws/credentials created by command aws configure. The aws configure command creates that file after prompting for access key identifiers (AKIDs) to an AWS account. Press Enter to accept the value previously defined:

AWS CLI install

The following can be done from any folder within a MacOS Terminal.app:

Several ways are presented to install AWS CLI.

  1. The simplest and most reliable for me is to use HomeBrew on Macs. If you have Homebrew installed:

    brew info awscli
    awscli: stable 2.2.39 (bottled), HEAD
    Official Amazon AWS command-line interface
    /usr/local/Cellar/awscli/2.2.39 (12,035 files, 88.5MB)
      Poured from bottle on 2021-09-21 at 06:36:33
    From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/awscli.rb
    License: Apache-2.0
    ==> Dependencies
    Build: cmake ✘
    Required: python@3.9 ✔, six ✔
    ==> Options
     Install HEAD version
    ==> Caveats
    The "examples" directory has been installed to:
    Bash completion has been installed to:
    ==> Analytics
    install: 196,108 (30 days), 527,336 (90 days), 1,879,114 (365 days)
    install-on-request: 194,585 (30 days), 523,050 (90 days), 1,859,544 (365 days)
    build-error: 0 (30 days)

    Notice that executables within folder path which does not require messing with PATH in ~/bash_profile:

  2. Using Homebrew would allow you to upgrade version with a single command:

    brew upgrade awscli

    If awscli was not already installed:

    brew install awscli

    Surprisingly, the size decreased for a while over previous versions:

    /opt/homebrew/Cellar/awscli/2.9.10: 12,995 files, 113.8MB
    /usr/local/Cellar/awscli/2.2.39 (12,035 files, 88.5MB)
    /usr/local/Cellar/awscli/2.2.21 (12,806 files, 100.3MB)
    /usr/local/Cellar/awscli/2.2.14 (12,776 files, 101.8MB)

    NOTE: Homebrew automatically installs the latest dependencies Ansible, ykman, etc.

    Alternately, one can use pip install awscli –upgrade –user –ignore-installed six installed from https://pypi.org/project/awscli. But when I did, aws cannot be found.

    Another alternative to install (on CentOS 7) is:

    curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" \
       -o "awscli-bundle.zip"
    unzip awscli-bundle.zip 
    sudo ./awscli-bundle/install \
       -i /usr/local/aws -b /usr/local/bin/aws
  3. Verify what version of awscli you have installed:

    aws --version

    Something went wrong if your response is:

    -bash: aws: command not found

    The expected sample response on an ARM M1 mac:

    aws-cli/2.9.4 Python/3.11.0 Darwin/21.6.0 source/arm64 prompt/off

    NOTE: “Darwin” is the name of the operating system internally within MacOS.


    aws-cli/2.2.22 Python/3.8.8 Darwin/18.7.0 exe/x86_64 prompt/off
    aws-cli/2.2.21 Python/3.9.6 Darwin/18.7.0 source/x86_64 prompt/off
    aws-cli/1.15.20 Python/3.6.5 Darwin/17.5.0 botocore/1.10.20

    NOTE: Awscli now uses Python 3, not 2.7.

    AWS Boto for Python

    PROTIP: “AWS SDK for Python” enables your Python (.py) programs to invoke AWS CLI commands.

    The Python package botocore on GitHub provides a low-level foundation for AWS CLI software.

    Ansible internally uses Boto to connect to Amazon EC2 instances and hence you need Boto library in order to run Ansible on your laptop/desktop. TOOL: Use Ansible to copy files from local to remote host.

  4. Make sure you’re not within Conda:

    conda deactivate

    If it’s already deactivated, you should not get any message.

  5. To install Boto3:

    pip install boto3 --upgrade --user --ignore-installed six

    At the end of response:

    Installing collected packages: botocore, boto3
    Successfully installed boto3-1.18.47 botocore-1.21.47

    Code for boto3 is obtained from https://github.com/boto/boto3. Read about it at https://aws.amazon.com/sdk-for-python.

    NOTE: The package is installed into folder:

    It’s in /usr/local/anaconda3/lib/python3.7/site-packages (2.49.0)

    The boto package is the hand-coded Python library that has been around since 2006. It is very popular and is fully supported currently by AWS. But because it is hand-coded and there are so many services available (with more appearing all the time) it is difficult to maintain.

    boto3, generally available since 06/22/2015, is a new version of the boto library based on botocore. All of the low-level interfaces to AWS are driven from JSON service descriptions that are generated automatically from the canonical descriptions of the services. So, the interfaces are always correct and always up to date. There is a resource layer on top of the client-layer that provides a nicer, more Pythonic interface. The boto3 library is being actively developed by AWS and is the devs should use when starting new development.

    BTW An example of a Lambda Python program to send email here.

    Bash Shell completions

  6. On Linux, to enable bash completion for aws commands:

    echo "\n" >> ~/.bashrc
    echo 'complete -C aws_completer aws' >> ~/.bashrc
  7. Test out autocompletion by typing the first two characters and pressing Tab for a list of all aws cli commands that begin with those characters:

    AWS Shell completion

    PROTIP: For automatic complex autocompletion of AWS CLI commands, there is a 3rd-party utility that provides a shell GUI that suggest as you type:

    Read about it at https://github.com/awslabs/aws-shell

  8. To install the awesome AWS Shell:

    pip install aws-shell

    The package is installed in folders: /usr/local/bin/aws-shell

    If you see these error messages:

    ERROR: requests 2.22.0 has requirement urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1, but you'll have urllib3 1.26.6 which is incompatible.
    ERROR: jupyter-console 6.0.0 has requirement prompt_toolkit<2.1.0,>=2.0.0, but you'll have prompt-toolkit 1.0.18 which is incompatible.
    ERROR: ipython 7.6.1 has requirement prompt-toolkit<2.1.0,>=2.0.0, but you'll have prompt-toolkit 1.0.18 which is incompatible.
  9. To enable AWS Shell:

    First run, creating autocomplete index...
    Creating doc index in the background. It will be a few minutes before all documentation is available.

    You show now be in the sub-shell with prompt:



  10. Exit aws-shell back to bash:


    Alternately, .quit works too.

    jp command

    The jp command enables JSON to be manipulated within Bash scripts.

  11. Install it on Macs, in any folder:

    brew tap jmespath/jmespath
    brew install jp
    🍺  /usr/local/Cellar/jp/1.1.12: 3 files, 3MB
  12. Verify it works by running a sample command:

    For example, jp enables a simple syntax to extract text:

    aws iam get-user | jq -r ".User.CreateDate[:4]"

    The response should be: 2021 from the first 4 characters of CreateDate within User.

  13. See other usage and examples at:

    jp is required by Aliases, below.

    aws sub-commands

    Further explained in this video:

    AWS automatically reads and processes file “alias” in your local folder path ~/.aws/cli/.

  14. Although https://github.com/awslabs/awscli-aliases has not been updated since 2016 (5 years ago), was tested with 1.11.24 of the AWS CLI and now obsolete, it has some convenient functions:

    • aws whoami = sts get-caller-identity
    • aws create-assume-role role
    • aws running-instances
    • aws ebs-volumes
    • aws amazon-linux-amis
    • aws list-sgs
    • aws sg-rules
    • aws tostring “string
    • aws tostring-with-jq “string
    • aws authorize-my-ip ip
    • aws authorize-my-ip-by-name group_id
    • aws get-group-id
    • aws public-ports
    • aws region
    • aws find-access-key “access key id
    • aws docker-ecr-login
    • aws myip
    • aws allow-my-ip group, protocol, port
    • aws allow-my-ip-all
    • aws revoke-my-ip group, protocol, port
    • aws revoke-my-ip-all my ip

    If you forget to type a parameter for a subcommand that expects it, you’ll see:

    Parameter validation failed:
    Invalid length for parameter RoleName, value: 0, valid min length: 1
  15. Create folder ~/.aws/cli/alias:

  16. Load:

    mkdir -p ~/.aws/cli
    pushd ~/.aws/cli
    curl -O https://raw.githubusercontent.com/awslabs/awscli-aliases/master/alias
  17. Try it: because alias defined aws whoami = sts get-caller-identity

    aws whoami

    should execute sts get-caller-identity.

    Unlike AWS sub-commands, OS-level Keyboard shortcuts do not need “aws” in front of the subcommand.

    TODO: pull in my custom keyboard.

    mkdir -p ~/.aws/cli
    pushd ~/.aws/cli
    curl -O https://raw.githubusercontent.com/awslabs/awscli-aliases/master/alias

aws configure

  1. Amazon documentation says to run:

    aws configure

    That command prompts acceptance or override of default AWS ACCESS KEY ID, AWS SECRET ACCESS KEY, and region saved as a plain-text file at 


    Sample contents:

    aws_access_key_id = ABCDEFGHIJKLMNOPQRST
    aws_secret_access_key = 123456786iJsvzQbkIlDiFtBh6DrPzIw8r7hVb35
    aws_access_key_id = ABCDEFGHIJKLMNOPQRST
    aws_secret_access_key = 123456782Nwk156aPF0SxZ8KGY+RrhEbq3AIHUSS

    BTW Progress toward AWS providing a more secure approach is at https://github.com/aws/aws-sdk/issues/41

    Configure profiles

    PROTIP: You’ll likely have a different AWS account for each enviornment (dev, qa, stage, prod), so specify a profile for each account.

    NOTE: https://awsu.me/general/overview.html (Awsume) is a command-line utility for retrieving and exporting AWS credentials to your shell’s environment. With awsume, you can get credentials for any profile located in your config and credentials files (opens new window), including those that require MFA or an assume-role call.

  2. Run the command to create files in folder ~/aws referenced by all other aws cli commands:

    aws configure --profile dev

    PROTIP: The example “root-admin-work” would be replaced with the user’s account name being created. Different accounts may be needed for different permissions in prod vs. dev use. Having separate access keys for different applications also generates distinct entries in AWS CloudTrail log files, which makes it easier to determine which application performed specific actions.

    Without the profile specification, “aws configure” by itself defines default credentials.

    The command prompts you for:

    AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
    Default region name [None]: us-west-2
    Default output format [None]: json

    PROTIP: If you do not explicitly specify an endpoint, US West (Oregon) us-west-2 is the default Region.

    The default output format is json.

    PROTIP: The aws configure command creates key/value pairs “aws_access_key_id” and “aws_secret_access_key” in file credentials for use by all AWS SDKs. Key/value pairs “region” and “output” are saved in file config used by the CLI.

    TODO: http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#cli-environment

    NOTE: Set default region using PowerShell

  3. The region in ~/.aws/config can be set also by:

    aws configure set profile.prod.region us-west-1

    Path ~/.aws/config is in variable $AWS_CONFIG_FILE

    Path ~/.aws/credentials is in variable $AWS_SHARED_CREDENTIALS_FILE

    aws configure set region \
       $(curl -s \
       | jp -u 'region')

    Roles for Tasks

    TODO: Temporary security credentials Roles for Tasks are stored in the ~/.aws/config file:

    [profile iam-role]
    role_arn = arn:aws:iam::ACCOUNT_ID:role/IAM_ROLE
    source_profile = iam-user
    output = json
    region = eu-west-1

    Importantly, the default region is specified in ~/.aws/config.

    PROTIP: The ~/.aws/config file also houses settings that speed up S3 sync.

    [profile default]
    s3 =
      max_concurrent_requests = 100
      max_queue_size = 10000
      use_accelerate_endpoint = true


  1. In a Terminal, navigate to a folder in your PATH.

  2. Download the aws-info.sh shell script:

    curl -fsSL https://raw.githubusercontent.com/wilsonmar/DevSecOps/master/aws/aws-info.sh
  3. Get the menu by running the script without any parameters:

    chown +x aws-info.sh


    -userinfo    to show User info
    -netinfo     to show Network info
    -svcinfo     to show Services info with cost history
    -lambdainfo  to show Lambda info
    -amiinfo     to show AMI info
    -ec2info     to show EC2 info
    -s3info      to show S3 info
    -diskinfo    to show Disk info
    -dbinfo      to show Database info
    -certinfo    to show Certificates info
    -loginfo     to show Logging info
  4. Run to obtain all data:

    ./aws-info.sh -v -allinfo

    User Info (IAM)

    Identity and Access Management (IAM) roles for Amazon EC2:

  5. List users:

    aws iam list-users --query Users[*].UserName
  6. List groups which the user belongs to :

    aws iam list-groups-for-user --username ???
  7. Create a new user named “MyUser”:

    aws iam create-user --user-name MyUser

    Sample response:

     "User": {
         "UserName": "MyUser",
         "Path": "/",
         "CreateDate": "2012-12-20T03:13:02.581Z",
         "UserId": "AKIAIOSFODNN7EXAMPLE",
         "Arn": "arn:aws:iam::123456789012:user/MyUser"
  8. Add the user to the group:

    aws iam add-user-to-group --user-name MyUser --group-name MyIamGroup
  9. To verify that the MyIamGroup group contains the MyUser, use the get-group command:

    aws iam get-group --group-name MyIamGroup

    The response:

         "Group": {
             "GroupName": "MyIamGroup",
             "CreateDate": "2012-12-20T03:03:52Z",
             "GroupId": "AKIAI44QH8DHBEXAMPLE",
             "Arn": "arn:aws:iam::123456789012:group/MyIamGroup",
             "Path": "/"
         "Users": [
                 "UserName": "MyUser",
                 "Path": "/",
                 "CreateDate": "2012-12-20T03:13:02Z",
                 "UserId": "AKIAIOSFODNN7EXAMPLE",
                 "Arn": "arn:aws:iam::123456789012:user/MyUser"
         "IsTruncated": "false"
  10. DOCS

    aws sts assume-role --role-arn arn:aws:iam::123456789012:role/role-name \
    --role-session-name "RoleSession1" \
    --profile IAM-user-name > assume-role-output.txt


    aws iam list-access-keys
     "AccessKeyMetadata": [
             "UserName": "Wilson_Mar",
             "AccessKeyId": "ABCDEFGHIJKLMNOPQRST",
             "Status": "Active",
             "CreateDate": "2020-06-12T04:04:22+00:00"

    AWS IAM commands use unique access key identifiers (AKIDs) to refer to individual access keys.

    aws iam create-access-key --user-name Alice

    Services list

  11. For a list of Amazon services with command access:

    aws commands help

    PROTIP: Drag the left/right edge of the Terminal to widen the screen.

    usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
    To see help text, you can run:
      aws help
      aws <command> help
      aws <command> <subcommand> help
    aws: error: argument command: Invalid choice, valid choices are:
    accessanalyzer                           | acm
    acm-pca                                  | alexaforbusiness
    amp                                      | amplify
    amplifybackend                           | apigateway
    apigatewaymanagementapi                  | apigatewayv2
    appconfig                                | appflow
    appintegrations                          | application-autoscaling
    application-insights                     | applicationcostprofiler
    appmesh                                  | apprunner
    appstream                                | appsync
    athena                                   | auditmanager
    autoscaling                              | autoscaling-plans
    backup                                   | batch
    braket                                   | budgets
    ce                                       | chime
    cloud9                                   | clouddirectory
    cloudformation                           | cloudfront
    cloudhsm                                 | cloudhsmv2
    cloudsearch                              | cloudsearchdomain
    cloudtrail                               | cloudwatch
    codeartifact                             | codebuild
    codecommit                               | codeguru-reviewer
    codeguruprofiler                         | codepipeline
    codestar                                 | codestar-connections
    codestar-notifications                   | cognito-identity
    cognito-idp                              | cognito-sync
    comprehend                               | comprehendmedical
    compute-optimizer                        | connect
    connect-contact-lens                     | connectparticipant
    cur                                      | customer-profiles
    databrew                                 | dataexchange
    datapipeline                             | datasync
    dax                                      | detective
    devicefarm                               | devops-guru
    directconnect                            | discovery
    dlm                                      | dms
    docdb                                    | ds
    dynamodb                                 | dynamodbstreams
    ebs                                      | ec2
    ec2-instance-connect                     | ecr
    ecr-public                               | ecs
    efs                                      | eks
    elastic-inference                        | elasticache
    elasticbeanstalk                         | elastictranscoder
    elb                                      | elbv2
    emr                                      | emr-containers
    es                                       | events
    finspace                                 | finspace-data
    firehose                                 | fis
    fms                                      | forecast
    forecastquery                            | frauddetector
    fsx                                      | gamelift
    glacier                                  | globalaccelerator
    glue                                     | greengrass
    greengrassv2                             | groundstation
    guardduty                                | health
    healthlake                               | honeycode
    iam                                      | identitystore
    imagebuilder                             | importexport
    inspector                                | iot
    iot-data                                 | iot-jobs-data
    iot1click-devices                        | iot1click-projects
    iotanalytics                             | iotdeviceadvisor
    iotevents                                | iotevents-data
    iotfleethub                              | iotsecuretunneling
    iotsitewise                              | iotthingsgraph
    iotwireless                              | ivs
    kafka                                    | kendra
    kinesis                                  | kinesis-video-archived-media
    kinesis-video-media                      | kinesis-video-signaling
    kinesisanalytics                         | kinesisanalyticsv2
    kinesisvideo                             | kms
    lakeformation                            | lambda
    lex-models                               | lex-runtime
    lexv2-models                             | lexv2-runtime
    license-manager                          | lightsail
    location                                 | logs
    lookoutequipment                         | lookoutmetrics
    lookoutvision                            | machinelearning
    macie                                    | macie2
    managedblockchain                        | marketplace-catalog
    marketplace-entitlement                  | marketplacecommerceanalytics
    mediaconnect                             | mediaconvert
    medialive                                | mediapackage
    mediapackage-vod                         | mediastore
    mediastore-data                          | mediatailor
    meteringmarketplace                      | mgh
    mgn                                      | migrationhub-config
    mobile                                   | mq
    mturk                                    | mwaa
    neptune                                  | network-firewall
    networkmanager                           | nimble
    opsworks                                 | opsworkscm
    organizations                            | outposts
    personalize                              | personalize-events
    personalize-runtime                      | pi
    pinpoint                                 | pinpoint-email
    pinpoint-sms-voice                       | polly
    pricing                                  | proton
    qldb                                     | qldb-session
    quicksight                               | ram
    rds                                      | rds-data
    redshift                                 | redshift-data
    rekognition                              | resource-groups
    resourcegroupstaggingapi                 | robomaker
    route53                                  | route53domains
    route53resolver                          | s3control
    s3outposts                               | sagemaker
    sagemaker-a2i-runtime                    | sagemaker-edge
    sagemaker-featurestore-runtime           | sagemaker-runtime
    savingsplans                             | schemas
    sdb                                      | secretsmanager
    securityhub                              | serverlessrepo
    service-quotas                           | servicecatalog
    servicecatalog-appregistry               | servicediscovery
    ses                                      | sesv2
    shield                                   | signer
    sms                                      | snowball
    sns                                      | sqs
    ssm                                      | ssm-contacts
    ssm-incidents                            | sso
    sso-admin                                | sso-oidc
    stepfunctions                            | storagegateway
    sts                                      | support
    swf                                      | synthetics
    textract                                 | timestream-query
    timestream-write                         | transcribe
    transfer                                 | translate
    waf                                      | waf-regional
    wafv2                                    | wellarchitected
    workdocs                                 | worklink
    workmail                                 | workmailmessageflow
    workspaces                               | xray
    s3api                                    | s3
    ddb                                      | configure
    deploy                                   | configservice
    opsworks-cm                              | history
    cli-dev                                  | help
    whoami                                   | create-assume-role
    running-instances                        | ebs-volumes
    amazon-linux-amis                        | list-sgs
    sg-rules                                 | tostring
    tostring-with-jq                         | authorize-my-ip
    get-group-id                             | authorize-my-ip-by-name
    public-ports                             | region
    find-access-key                          | docker-ecr-login
    myip                                     | allow-my-ip
    revoke-my-ip                             | allow-my-ip-all

    See http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-using.html

    • https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html
    • https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys
    • https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html

  12. Grant temporary access keys - aws sts assume-role.


    Additionally, add conditions to the policy that further restrict access, such as the source IP address range of clients. The example policy below grants the needed permissions (PutObject) on to a specific resource (an S3 bucket named “examplebucket”) while adding further conditions (the client must come from IP range

     "Version": "2012-10-17",
     "Id": "S3PolicyRestrictPut",
     "Statement": [
             "Sid": "IPAllow",
             "Effect": "Allow",
             "Principal": "*",
             "Action": "s3:PutObject",
             "Resource": "arn:aws:s3:::examplebucket/*",
             "Condition": {
                 "IpAddress": {"aws:SourceIp": ""}


    https://bash-my-aws.org/ features Short, memorable commands:

    • buckets
    • instances
    • stacks grep postgres
    • keypairs


    Linux AMIs

    Types of operating system AMI:

    • Amazon Linux 2014.09.2 (CentOS)
    • Red Hat Enterprise Linux 6.6 (RHEL)
    • SUSE Linux Enterprise Server 12
    • Ubuntu Server 14.04

    for loop

    for region in $( aws ec2 describe-regions --output text --query "Regions[].[RegionName]" | tr "\\n" " "  ); do echo "some $region" ; done
    for region in $( aws ec2 describe-regions –output text –query “Regions[].[RegionName]” tr “\n” “ “ ); do echo “some $region” ; done

Define groups to assign permissions

PROTIP: For a user to do something usually require several AWS resources. So several permissions need to be granted to a user. To simplify assignments, we define Groups of permissions which we then can assign to each user.

In other words, An IAM group is a management convenience to manage the same set of permissions for a set of IAM users.

The AWS CLI command to create a group named “MyIamGroup” is:

aws iam create-group --group-name MyIamGroup

A sample response:

    "Group": {
        "GroupName": "MyIamGroup",
        "CreateDate": "2012-12-20T03:03:52.834Z",
        "GroupId": "AKIAI44QH8DHBEXAMPLE",
        "Arn": "arn:aws:iam::123456789012:group/MyIamGroup",
        "Path": "/"
  1. Create a S3 security group:

    aws ec2 create-security-group --group-name my-sg --description "My security group"

    A sample response:

    "GroupId": "sg-903004f8"
  2. Click Manage Groups then Create New Group.

    PROTIP: Groups are usually associated with a particular job: admin, sales, HR, front-end developer, back-end developer, etc.

    A user can belong to multiple groups. More complex organizations manage differences in permissions for company, division, project, location, job level, etc. So 128 characters may not be enough if large words are used. Thus, abbreviate and use acronyms.

    PROTIP: Put abbreviations and acronyms in a wiki publicly available to avoid duplicate usage.

  3. “aws_iot_buttons” is the group name I use as an example.

PROTIP: Use dashes. Space characters are not allowed. On March 1, 2018 AWS removed the ability to use underscores in S3 bucket names.

The list shown are “AWS Managed”.

  1. Click on Policy Type to select Job function.

  2. PROTIP: Instead of scrolling down the massive list in Attache Policy (Alexa, Amazon, AWS, etc.), type in the Filter field the first few letters (such as “IoT”) and the list gets smaller. Notice the filter you type is applicable to not just characters beginning with what you typed, but also characters inside names as well.

  3. Click to select.
  4. Click “Create Group”.

    Note different policies have different levels of access, with admin having more capabilities than “read only” ones.

  5. Names shown on the screen is called a “Policy Summary”.
  6. Click “JSON” to see the file that AWS reads to assign policies. Here you seen what Actions the policy allows.

  7. Click “Access Advisor” to see users who have been assigned to use the policy.


AWS Policy Generator

The AWS Policy Generator at https://awspolicygen.s3.amazonaws.com/policygen.html creates policy files that control access to Amazon Web Services (AWS) products and resources:

  • SQS Queue Policy
  • S3 Bucket Policy
  • VPC Endpoint Policy
  • IAM Policy
  • SNS Topic Policy







TODO: Put each AWS CLI command in a script at https://medium.com/circuitpeople/aws-cli-with-jq-and-bash-9d54e2eabaf1 by Lee Harding



AWS Powershell for Windows</a>


AWS Config

An example of a service that requires AWS CLI to turn on and off is AWS Config.

The AWS Config Recorder, by default, detects and captures changes in resource configurations as configuration items.

Costs are around $4 a month, based on:

  • the number of configuration items and
  • the number of rule evaluations performed

Conformance packs


  1. List the rules:

    aws configservice describe-config-rules --output json \
    | grep ConfigRuleName \
    | cut -d":" -f2 \
    | cut -d"," -f1 

    Sample responses:

  2. Delete?

    aws configservice describe-config-rules | grep ConfigRuleName | gawk &#39;match($0, /:.+"(.+)"/, a) {print a[1]}&#39; | while read rule_name; 
     echo $rule_name; 
     aws configservice delete-config-rule --config-rule-name $rule_name
  3. List the rules:

    aws configservice describe-config-rules --output json \
    | grep ConfigRuleName \
    | cut -d":" -f2 \
    | cut -d"," -f1 \
    | xargs -L1 aws configservice delete-config-rule \
  4. Turn off Recording for that region using the console

  5. Delete the Rule by going to actions, delete rule

  6. Use the AWS CLI and delete the default recording by

aws configservice delete-configuration-recorder –configuration-recorder-name default –region region-name

  1. Delete the service linked role created for AWS Config

  2. Refresh the Config home page to make it appear fresh.

  3. If necessary delete the config bucket and its objects.

for AWS_REGION in $(aws ec2 describe-regions –output text
–query ‘Regions[].[RegionName]’) ; do echo “$AWS_REGION:”;
for snap in $(aws ec2 describe-snapshots –owner self –output text –region $AWS_REGION
–query ‘Snapshots[*].SnapshotId’); do aws ec2 describe-snapshot-attribute
–snapshot-id $snap –region $AWS_REGION –output text –attribute createVolumePermission
–query ‘[SnapshotId,CreateVolumePermissions[?Group == all]]’; done; echo; done

https://opensourceconnections.com/blog/2015/07/27/advanced-aws-cli-jmespath-query/ aws ec2 describe-images –owner amazon –query ‘Images[].[ImageId,Name]’ –output text | grep -m5 “ami-“


Video Tutorials

A good one on this topic is David Clinton’s “Using Docker on AWS with the Command Line” on Pluralsight, which goes into ECS, Fargate, and EKS using Kubernetes. The course is the basis for “Teach yourself Data Analytics in 30 days: Learn to use Python and Jupyter Notebooks by exploring fun, real-world data projects” at bootstrap-it.com/docker4aws. For your convenience, I’ve made code from David’s webpage available at https://github.com/wilsonmar/DevSecOps/tree/main/docker4aws

More recently (2/2019) is Manuj Aggarwal’s $84.99 https://www.udemy.com/course/aws-masterclass-aws-command-line-interface-and-devops/




  "machines": [
    {"name": "a", "state": "running"},
    {"name": "b", "state": "stopped"},
    {"name": "b", "state": "running"}

More on Amazon

This is one of a series on Amazon:

More on DevOps

This is one of a series on DevOps:

  1. DevOps_2.0
  2. ci-cd (Continuous Integration and Continuous Delivery)
  3. User Stories for DevOps
  4. Enterprise Software)

  5. Git and GitHub vs File Archival
  6. Git Commands and Statuses
  7. Git Commit, Tag, Push
  8. Git Utilities
  9. Data Security GitHub
  10. GitHub API
  11. TFS vs. GitHub

  12. Choices for DevOps Technologies
  13. Pulumi Infrastructure as Code (IaC)
  14. Java DevOps Workflow
  15. Okta for SSO & MFA

  16. AWS DevOps (CodeCommit, CodePipeline, CodeDeploy)
  17. AWS server deployment options
  18. AWS Load Balancers

  19. Cloud services comparisons (across vendors)
  20. Cloud regions (across vendors)
  21. AWS Virtual Private Cloud

  22. Azure Cloud Onramp (Subscriptions, Portal GUI, CLI)
  23. Azure Certifications
  24. Azure Cloud

  25. Azure Cloud Powershell
  26. Bash Windows using Microsoft’s WSL (Windows Subsystem for Linux)
  27. Azure KSQL (Kusto Query Language) for Azure Monitor, etc.

  28. Azure Networking
  29. Azure Storage
  30. Azure Compute
  31. Azure Monitoring

  32. Digital Ocean
  33. Cloud Foundry

  34. Packer automation to build Vagrant images
  35. Terraform multi-cloud provisioning automation
  36. Hashicorp Vault and Consul to generate and hold secrets

  37. Powershell Ecosystem
  38. Powershell on MacOS
  39. Powershell Desired System Configuration

  40. Jenkins Server Setup
  41. Jenkins Plug-ins
  42. Jenkins Freestyle jobs
  43. Jenkins2 Pipeline jobs using Groovy code in Jenkinsfile

  44. Docker (Glossary, Ecosystem, Certification)
  45. Make Makefile for Docker
  46. Docker Setup and run Bash shell script
  47. Bash coding
  48. Docker Setup
  49. Dockerize apps
  50. Docker Registry

  51. Maven on MacOSX

  52. Ansible
  53. Kubernetes Operators
  54. OPA (Open Policy Agent) in Rego language

  55. MySQL Setup

  56. Threat Modeling
  57. SonarQube & SonarSource static code scan

  58. API Management Microsoft
  59. API Management Amazon

  60. Scenarios for load
  61. Chaos Engineering

More on Security

This is one of a series on Security in DevSecOps:

  1. Azure Security-focus Cloud Onramp

  2. AWS Security (certification exam)
  3. AWS IAM (Identity and Access Management)


  5. SOC2
  6. FedRAMP
  7. CAIQ (Consensus Assessment Initiative Questionnaire) by cloud vendors

  8. Git Signing
  9. Hashicorp Vault
  10. OPA (Open Policy Agent)

  11. SonarQube
  12. WebGoat known insecure PHP app and vulnerability scanners
  13. Test for OWASP using ZAP on the Broken Web App

  14. Encrypt all the things

  15. Cyber Security
  16. Security certifications