Wilson Mar bio photo

Wilson Mar

Hello!

Email me Calendar Skype call

LinkedIn Twitter Gitter Instagram Youtube

Github Stackoverflow Pinterest

Tips and tricks to install and use the MacOS Terminal for Programmatic access to AWS.

US (English)   Español (Spanish)   Français (French)   Deutsch (German)   Italiano   Português   Cyrillic Russian   中文 (简体) Chinese (Simplified)   日本語 Japanese   한국어 Korean

Overview

This is a hands-on tutorial to get new users setup to effectively access and use the AWS cloud. Here you do some action and explanations and PROTIP advice is provided. PROTIPs included how to install and use AWS CLI automation, smart phone apps, and 3rd party tools used by the pros.

This is a part of my on-boarding and Amazon’s Getting Started tutorials. They cover getting API Keys and Credentials.

Instead of clicking and typing the manual AWS Management Console GUI, server administrators work with Programmatic access to AWS by crafting and running programs in the Command Line Interface (CLI).

CLI programs are used to invoke Cloud Formation or Terraform files which are the basis for creating resources in AWS.

This is so the build process can be debugged and changed slightly through the lifecycle from test to prod.

Instead of doing what other clouds do (some aws login command to prompt for a user name and password then storing credentials in a browser cookie), each aws command references a specifically-named file at $HOME/.aws/credentials created by command aws configure. The aws configure command creates that file after prompting for access key identifiers (AKIDs) to an AWS account. Press Enter to accept the value previously defined:

AWS CLI install

The following can be done from any folder within a MacOS Terminal.app:

Several ways are presented to install AWS CLI.

  1. The simplest and most reliable for me is to use HomeBrew on Macs. If you have Homebrew installed:

    brew info awscli
    
    awscli: stable 2.2.39 (bottled), HEAD
    Official Amazon AWS command-line interface
    https://aws.amazon.com/cli/
    /usr/local/Cellar/awscli/2.2.39 (12,035 files, 88.5MB)
      Poured from bottle on 2021-09-21 at 06:36:33
    From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/awscli.rb
    License: Apache-2.0
    ==> Dependencies
    Build: cmake ✘
    Required: python@3.9 ✔, six ✔
    ==> Options
    --HEAD
     Install HEAD version
    ==> Caveats
    The "examples" directory has been installed to:
      /usr/local/share/awscli/examples
     
    Bash completion has been installed to:
      /usr/local/etc/bash_completion.d
    ==> Analytics
    install: 196,108 (30 days), 527,336 (90 days), 1,879,114 (365 days)
    install-on-request: 194,585 (30 days), 523,050 (90 days), 1,859,544 (365 days)
    build-error: 0 (30 days)
    

    Notice that executables within folder path which does not require messing with PATH in ~/bash_profile:

    /usr/local/Cellar/awscli/
  2. Using Homebrew would allow you to upgrade version with a single command:

    brew upgrade awscli
    

    If awscli was not already installed:

    brew install awscli
    

    Surprisingly, the size has decreased over previous versions:

    /usr/local/Cellar/awscli/2.2.39 (12,035 files, 88.5MB)
    /usr/local/Cellar/awscli/2.2.21 (12,806 files, 100.3MB)
    /usr/local/Cellar/awscli/2.2.14 (12,776 files, 101.8MB)
    

    NOTE: Homebrew automatically installs the latest dependencies Ansible, ykman, etc.

    Alternately, one can use pip install awscli –upgrade –user –ignore-installed six installed from https://pypi.org/project/awscli. But when I did, aws cannot be found.

    Another alternative to install (on CentOS 7) is:

    curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" \
       -o "awscli-bundle.zip"
    unzip awscli-bundle.zip 
    sudo ./awscli-bundle/install \
       -i /usr/local/aws -b /usr/local/bin/aws
    
  3. Verify what version of awscli you have installed:

    aws --version
    

    Something went wrong if your response is:

    -bash: aws: command not found

    The expected sample response:

    aws-cli/2.2.22 Python/3.8.8 Darwin/18.7.0 exe/x86_64 prompt/off
    

    NOTE: “Darwin” is the name of the operating system internals within MacOS.

    Previously:

    aws-cli/2.2.21 Python/3.9.6 Darwin/18.7.0 source/x86_64 prompt/off
    aws-cli/1.15.20 Python/3.6.5 Darwin/17.5.0 botocore/1.10.20
    

    NOTE: Awscli now uses Python 3, not 2.7.

    AWS Boto for Python

    PROTIP: “AWS SDK for Python” enables your Python (.py) programs to invoke AWS CLI commands.

    The Python package botocore on GitHub provides a low-level foundation for AWS CLI software.

    Ansible internally uses Boto to connect to Amazon EC2 instances and hence you need Boto library in order to run Ansible on your laptop/desktop. TOOL: Use Ansible to copy files from local to remote host.

  4. Make sure you’re not within Conda:

    conda deactivate
    

    If it’s already deactivated, you should not get any message.

  5. To install Boto3:

    pip install boto3 --upgrade --user --ignore-installed six
    

    At the end of response:

    Installing collected packages: botocore, boto3
    Successfully installed boto3-1.18.47 botocore-1.21.47
    

    Code for boto3 is obtained from https://github.com/boto/boto3. Read about it at https://aws.amazon.com/sdk-for-python.

    NOTE: The package is installed into folder:
    /usr/local/lib/python2.7/site-packages/boto3/*

    It’s in /usr/local/anaconda3/lib/python3.7/site-packages (2.49.0)

    The boto package is the hand-coded Python library that has been around since 2006. It is very popular and is fully supported currently by AWS. But because it is hand-coded and there are so many services available (with more appearing all the time) it is difficult to maintain.

    boto3, generally available since 06/22/2015, is a new version of the boto library based on botocore. All of the low-level interfaces to AWS are driven from JSON service descriptions that are generated automatically from the canonical descriptions of the services. So, the interfaces are always correct and always up to date. There is a resource layer on top of the client-layer that provides a nicer, more Pythonic interface. The boto3 library is being actively developed by AWS and is the devs should use when starting new development.

    BTW An example of a Lambda Python program to send email here.

    Bash Shell completions

  6. On Linux, to enable bash completion for aws commands:

    echo "\n" >> ~/.bashrc
    echo 'complete -C aws_completer aws' >> ~/.bashrc
    
  7. Test out autocompletion by typing the first two characters and pressing Tab for a list of all aws cli commands that begin with those characters:

    AWS Shell completion

    PROTIP: For automatic complex autocompletion of AWS CLI commands, there is a 3rd-party utility that provides a shell GUI that suggest as you type:

    Read about it at https://github.com/awslabs/aws-shell

  8. To install the awesome AWS Shell:

    pip install aws-shell
    

    The package is installed in folders: /usr/local/bin/aws-shell

    If you see these error messages:

    ERROR: requests 2.22.0 has requirement urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1, but you'll have urllib3 1.26.6 which is incompatible.
    ERROR: jupyter-console 6.0.0 has requirement prompt_toolkit<2.1.0,>=2.0.0, but you'll have prompt-toolkit 1.0.18 which is incompatible.
    ERROR: ipython 7.6.1 has requirement prompt-toolkit<2.1.0,>=2.0.0, but you'll have prompt-toolkit 1.0.18 which is incompatible.
    
  9. To enable AWS Shell:

    aws-shell
    
    First run, creating autocomplete index...
    Creating doc index in the background. It will be a few minutes before all documentation is available.
    

    You show now be in the sub-shell with prompt:

    aws>

    aws-onboarding-aws-shell-config-207x58-5051.jpg

  10. Exit aws-shell back to bash:

    .exit
    

    Alternately, .quit works too.

    jp command

    The jp command enables JSON to be manipulated within Bash scripts.

  11. Install it on Macs, in any folder:

    brew tap jmespath/jmespath
    brew install jp
    
    🍺  /usr/local/Cellar/jp/1.1.12: 3 files, 3MB
  12. Verify it works by running a sample command:

    For example, jp enables a simple syntax to extract text:

    aws iam get-user | jq -r ".User.CreateDate[:4]"
    

    The response should be: 2021 from the first 4 characters of CreateDate within User.

  13. See other usage and examples at:

    jp is required by Aliases, below.

    aws sub-commands

    Further explained in this video:

    AWS automatically reads and processes file “alias” in your local folder path ~/.aws/cli/.

  14. Although https://github.com/awslabs/awscli-aliases has not been updated since 2016 (5 years ago), was tested with 1.11.24 of the AWS CLI and now obsolete, it has some convenient functions:

    • aws whoami = sts get-caller-identity
    • aws create-assume-role role
    • aws running-instances
    • aws ebs-volumes
    • aws amazon-linux-amis
    • aws list-sgs
    • aws sg-rules
    • aws tostring “string
    • aws tostring-with-jq “string
    • aws authorize-my-ip ip
    • aws authorize-my-ip-by-name group_id
    • aws get-group-id
    • aws public-ports
    • aws region
    • aws find-access-key “access key id
    • aws docker-ecr-login
    • aws myip
    • aws allow-my-ip group, protocol, port
    • aws allow-my-ip-all
    • aws revoke-my-ip group, protocol, port
    • aws revoke-my-ip-all my ip

    If you forget to type a parameter for a subcommand that expects it, you’ll see:

    Parameter validation failed:
    Invalid length for parameter RoleName, value: 0, valid min length: 1
    
  15. Create folder ~/.aws/cli/alias:

  16. Load:

    mkdir -p ~/.aws/cli
    pushd ~/.aws/cli
    curl -O https://raw.githubusercontent.com/awslabs/awscli-aliases/master/alias
    popd
    
  17. Try it: because alias defined aws whoami = sts get-caller-identity

    aws whoami

    should execute sts get-caller-identity.

    Unlike AWS sub-commands, OS-level Keyboard shortcuts do not need “aws” in front of the subcommand.

    TODO: pull in my custom keyboard.

    mkdir -p ~/.aws/cli
    pushd ~/.aws/cli
    curl -O https://raw.githubusercontent.com/awslabs/awscli-aliases/master/alias
    popd
    

aws configure

  1. Amazon documentation says to run:

    aws configure

    That command prompts acceptance or override of default AWS ACCESS KEY ID, AWS SECRET ACCESS KEY, and region saved as a plain-text file at 

    ~/.aws/credentials

    Sample contents:

    [default]
    aws_access_key_id = ABCDEFGHIJKLMNOPQRST
    aws_secret_access_key = 123456786iJsvzQbkIlDiFtBh6DrPzIw8r7hVb35
    [py-ec2–1]
    aws_access_key_id = ABCDEFGHIJKLMNOPQRST
    aws_secret_access_key = 123456782Nwk156aPF0SxZ8KGY+RrhEbq3AIHUSS
    

    BTW Progress toward AWS providing a more secure approach is at https://github.com/aws/aws-sdk/issues/41

    Configure profiles

    PROTIP: You’ll likely have a different AWS account for each enviornment (dev, qa, stage, prod), so specify a profile for each account.

    NOTE: https://awsu.me/general/overview.html (Awsume) is a command-line utility for retrieving and exporting AWS credentials to your shell’s environment. With awsume, you can get credentials for any profile located in your config and credentials files (opens new window), including those that require MFA or an assume-role call.

  2. Run the command to create files in folder ~/aws referenced by all other aws cli commands:

    aws configure --profile dev
    

    PROTIP: The example “root-admin-work” would be replaced with the user’s account name being created. Different accounts may be needed for different permissions in prod vs. dev use. Having separate access keys for different applications also generates distinct entries in AWS CloudTrail log files, which makes it easier to determine which application performed specific actions.

    Without the profile specification, “aws configure” by itself defines default credentials.

    The command prompts you for:

    AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
    AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
    Default region name [None]: us-west-2
    Default output format [None]: json
    

    PROTIP: If you do not explicitly specify an endpoint, US West (Oregon) us-west-2 is the default Region.

    The default output format is json.

    PROTIP: The aws configure command creates key/value pairs “aws_access_key_id” and “aws_secret_access_key” in file credentials for use by all AWS SDKs. Key/value pairs “region” and “output” are saved in file config used by the CLI.

    TODO: http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#cli-environment

    NOTE: Set default region using PowerShell

  3. The region in ~/.aws/config can be set also by:

    aws configure set profile.prod.region us-west-1
    

    Path ~/.aws/config is in variable $AWS_CONFIG_FILE

    Path ~/.aws/credentials is in variable $AWS_SHARED_CREDENTIALS_FILE

    aws configure set region \
       $(curl -s http://162.254.169.254/latest/dynamic/instance-identity/document \
       | jp -u 'region')
    

    Roles for Tasks

    TODO: Temporary security credentials Roles for Tasks are stored in the ~/.aws/config file:

    [profile iam-role]
    role_arn = arn:aws:iam::ACCOUNT_ID:role/IAM_ROLE
    source_profile = iam-user
    output = json
    region = eu-west-1
    

    Importantly, the default region is specified in ~/.aws/config.

    PROTIP: The ~/.aws/config file also houses settings that speed up S3 sync.

    [profile default]
    ...
    s3 =
      max_concurrent_requests = 100
      max_queue_size = 10000
      use_accelerate_endpoint = true
    

aws-info.sh

  1. In a Terminal, navigate to a folder in your PATH.

  2. Download the aws-info.sh shell script:

    curl -fsSL https://raw.githubusercontent.com/wilsonmar/DevSecOps/master/aws/aws-info.sh
    
  3. Get the menu by running the script without any parameters:

    chown +x aws-info.sh
    ./aws-info.sh
    

    Notice:

    -userinfo    to show User info
    -netinfo     to show Network info
    -svcinfo     to show Services info with cost history
     
    -lambdainfo  to show Lambda info
    -amiinfo     to show AMI info
    -ec2info     to show EC2 info
     
    -s3info      to show S3 info
    -diskinfo    to show Disk info
    -dbinfo      to show Database info
    -certinfo    to show Certificates info
    -loginfo     to show Logging info
    
  4. Run to obtain all data:

    ./aws-info.sh -v -allinfo
    

    User Info (IAM)

    Identity and Access Management (IAM) roles for Amazon EC2:

  5. List users:

    
    aws iam list-users --query Users[*].UserName
    
  6. List groups which the user belongs to :

    
    aws iam list-groups-for-user --username ???
    
  7. Create a new user named “MyUser”:

    aws iam create-user --user-name MyUser
    

</pre>

Sample response:

{
    "User": {
        "UserName": "MyUser",
        "Path": "/",
        "CreateDate": "2012-12-20T03:13:02.581Z",
        "UserId": "AKIAIOSFODNN7EXAMPLE",
        "Arn": "arn:aws:iam::123456789012:user/MyUser"
    }
}
  1. Add the user to the group:

    aws iam add-user-to-group --user-name MyUser --group-name MyIamGroup
  2. To verify that the MyIamGroup group contains the MyUser, use the get-group command:

    aws iam get-group --group-name MyIamGroup

    The response:

     {
         "Group": {
             "GroupName": "MyIamGroup",
             "CreateDate": "2012-12-20T03:03:52Z",
             "GroupId": "AKIAI44QH8DHBEXAMPLE",
             "Arn": "arn:aws:iam::123456789012:group/MyIamGroup",
             "Path": "/"
         },
         "Users": [
             {
                 "UserName": "MyUser",
                 "Path": "/",
                 "CreateDate": "2012-12-20T03:13:02Z",
                 "UserId": "AKIAIOSFODNN7EXAMPLE",
                 "Arn": "arn:aws:iam::123456789012:user/MyUser"
             }
         ],
         "IsTruncated": "false"
     }
  3. DOCS

    aws sts assume-role --role-arn arn:aws:iam::123456789012:role/role-name \
    --role-session-name "RoleSession1" \
    --profile IAM-user-name > assume-role-output.txt
    

    https://aws.amazon.com/blogs/security/how-to-rotate-access-keys-for-iam-users/

    aws iam list-access-keys
    {
     "AccessKeyMetadata": [
         {
             "UserName": "Wilson_Mar",
             "AccessKeyId": "ABCDEFGHIJKLMNOPQRST",
             "Status": "Active",
             "CreateDate": "2020-06-12T04:04:22+00:00"
         }
     ]
    }
    

    AWS IAM commands use unique access key identifiers (AKIDs) to refer to individual access keys.

    aws iam create-access-key --user-name Alice

    Services list

  4. For a list of Amazon services with command access:

    aws commands help
    

    PROTIP: Drag the left/right edge of the Terminal to widen the screen.

    usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
    To see help text, you can run:
     
      aws help
      aws <command> help
      aws <command> <subcommand> help
     
    aws: error: argument command: Invalid choice, valid choices are:
     
    accessanalyzer                           | acm
    acm-pca                                  | alexaforbusiness
    amp                                      | amplify
    amplifybackend                           | apigateway
    apigatewaymanagementapi                  | apigatewayv2
    appconfig                                | appflow
    appintegrations                          | application-autoscaling
    application-insights                     | applicationcostprofiler
    appmesh                                  | apprunner
    appstream                                | appsync
    athena                                   | auditmanager
    autoscaling                              | autoscaling-plans
    backup                                   | batch
    braket                                   | budgets
    ce                                       | chime
    cloud9                                   | clouddirectory
    cloudformation                           | cloudfront
    cloudhsm                                 | cloudhsmv2
    cloudsearch                              | cloudsearchdomain
    cloudtrail                               | cloudwatch
    codeartifact                             | codebuild
    codecommit                               | codeguru-reviewer
    codeguruprofiler                         | codepipeline
    codestar                                 | codestar-connections
    codestar-notifications                   | cognito-identity
    cognito-idp                              | cognito-sync
    comprehend                               | comprehendmedical
    compute-optimizer                        | connect
    connect-contact-lens                     | connectparticipant
    cur                                      | customer-profiles
    databrew                                 | dataexchange
    datapipeline                             | datasync
    dax                                      | detective
    devicefarm                               | devops-guru
    directconnect                            | discovery
    dlm                                      | dms
    docdb                                    | ds
    dynamodb                                 | dynamodbstreams
    ebs                                      | ec2
    ec2-instance-connect                     | ecr
    ecr-public                               | ecs
    efs                                      | eks
    elastic-inference                        | elasticache
    elasticbeanstalk                         | elastictranscoder
    elb                                      | elbv2
    emr                                      | emr-containers
    es                                       | events
    finspace                                 | finspace-data
    firehose                                 | fis
    fms                                      | forecast
    forecastquery                            | frauddetector
    fsx                                      | gamelift
    glacier                                  | globalaccelerator
    glue                                     | greengrass
    greengrassv2                             | groundstation
    guardduty                                | health
    healthlake                               | honeycode
    iam                                      | identitystore
    imagebuilder                             | importexport
    inspector                                | iot
    iot-data                                 | iot-jobs-data
    iot1click-devices                        | iot1click-projects
    iotanalytics                             | iotdeviceadvisor
    iotevents                                | iotevents-data
    iotfleethub                              | iotsecuretunneling
    iotsitewise                              | iotthingsgraph
    iotwireless                              | ivs
    kafka                                    | kendra
    kinesis                                  | kinesis-video-archived-media
    kinesis-video-media                      | kinesis-video-signaling
    kinesisanalytics                         | kinesisanalyticsv2
    kinesisvideo                             | kms
    lakeformation                            | lambda
    lex-models                               | lex-runtime
    lexv2-models                             | lexv2-runtime
    license-manager                          | lightsail
    location                                 | logs
    lookoutequipment                         | lookoutmetrics
    lookoutvision                            | machinelearning
    macie                                    | macie2
    managedblockchain                        | marketplace-catalog
    marketplace-entitlement                  | marketplacecommerceanalytics
    mediaconnect                             | mediaconvert
    medialive                                | mediapackage
    mediapackage-vod                         | mediastore
    mediastore-data                          | mediatailor
    meteringmarketplace                      | mgh
    mgn                                      | migrationhub-config
    mobile                                   | mq
    mturk                                    | mwaa
    neptune                                  | network-firewall
    networkmanager                           | nimble
    opsworks                                 | opsworkscm
    organizations                            | outposts
    personalize                              | personalize-events
    personalize-runtime                      | pi
    pinpoint                                 | pinpoint-email
    pinpoint-sms-voice                       | polly
    pricing                                  | proton
    qldb                                     | qldb-session
    quicksight                               | ram
    rds                                      | rds-data
    redshift                                 | redshift-data
    rekognition                              | resource-groups
    resourcegroupstaggingapi                 | robomaker
    route53                                  | route53domains
    route53resolver                          | s3control
    s3outposts                               | sagemaker
    sagemaker-a2i-runtime                    | sagemaker-edge
    sagemaker-featurestore-runtime           | sagemaker-runtime
    savingsplans                             | schemas
    sdb                                      | secretsmanager
    securityhub                              | serverlessrepo
    service-quotas                           | servicecatalog
    servicecatalog-appregistry               | servicediscovery
    ses                                      | sesv2
    shield                                   | signer
    sms                                      | snowball
    sns                                      | sqs
    ssm                                      | ssm-contacts
    ssm-incidents                            | sso
    sso-admin                                | sso-oidc
    stepfunctions                            | storagegateway
    sts                                      | support
    swf                                      | synthetics
    textract                                 | timestream-query
    timestream-write                         | transcribe
    transfer                                 | translate
    waf                                      | waf-regional
    wafv2                                    | wellarchitected
    workdocs                                 | worklink
    workmail                                 | workmailmessageflow
    workspaces                               | xray
    s3api                                    | s3
    ddb                                      | configure
    deploy                                   | configservice
    opsworks-cm                              | history
    cli-dev                                  | help
    whoami                                   | create-assume-role
    running-instances                        | ebs-volumes
    amazon-linux-amis                        | list-sgs
    sg-rules                                 | tostring
    tostring-with-jq                         | authorize-my-ip
    get-group-id                             | authorize-my-ip-by-name
    public-ports                             | region
    find-access-key                          | docker-ecr-login
    myip                                     | allow-my-ip
    revoke-my-ip                             | allow-my-ip-all
    revoke-my-ip-all
    

    See http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-using.html

    • https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html
    • https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys
    • https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html

  5. Grant temporary access keys - aws sts assume-role.

    https://docs.aws.amazon.com/secretsmanager/latest/userguide/rotating-secrets-one-user-multiple-passwords.html

    Additionally, add conditions to the policy that further restrict access, such as the source IP address range of clients. The example policy below grants the needed permissions (PutObject) on to a specific resource (an S3 bucket named “examplebucket”) while adding further conditions (the client must come from IP range 203.0.113.0/24):

    {
     "Version": "2012-10-17",
     "Id": "S3PolicyRestrictPut",
     "Statement": [
             {
             "Sid": "IPAllow",
             "Effect": "Allow",
             "Principal": "*",
             "Action": "s3:PutObject",
             "Resource": "arn:aws:s3:::examplebucket/*",
             "Condition": {
                 "IpAddress": {"aws:SourceIp": "203.0.113.0/24"}
             } 
         } 
     ]
    }
    

    bash-my-aws

    https://bash-my-aws.org/ features Short, memorable commands:

    • buckets
    • instances
    • stacks grep postgres
    • keypairs

    https://github.com/bash-my-aws/bash-my-aws/

    Linux AMIs

    Types of operating system AMI:

    • Amazon Linux 2014.09.2 (CentOS)
    • Red Hat Enterprise Linux 6.6 (RHEL)
    • SUSE Linux Enterprise Server 12
    • Ubuntu Server 14.04

Define groups to assign permissions

PROTIP: For a user to do something usually require several AWS resources. So several permissions need to be granted to a user. To simplify assignments, we define Groups of permissions which we then can assign to each user.

In other words, An IAM group is a management convenience to manage the same set of permissions for a set of IAM users.

The AWS CLI command to create a group named “MyIamGroup” is:

aws iam create-group --group-name MyIamGroup
   

A sample response:

{
    "Group": {
        "GroupName": "MyIamGroup",
        "CreateDate": "2012-12-20T03:03:52.834Z",
        "GroupId": "AKIAI44QH8DHBEXAMPLE",
        "Arn": "arn:aws:iam::123456789012:group/MyIamGroup",
        "Path": "/"
    }
}
   
  1. Create a S3 security group:

    aws ec2 create-security-group --group-name my-sg --description "My security group"
    

    A sample response:

    {
    "GroupId": "sg-903004f8"
    }
  2. Click Manage Groups then Create New Group.

    PROTIP: Groups are usually associated with a particular job: admin, sales, HR, front-end developer, back-end developer, etc.

    A user can belong to multiple groups. More complex organizations manage differences in permissions for company, division, project, location, job level, etc. So 128 characters may not be enough if large words are used. Thus, abbreviate and use acronyms.

    PROTIP: Put abbreviations and acronyms in a wiki publicly available to avoid duplicate usage.

  3. “aws_iot_buttons” is the group name I use as an example.

PROTIP: Use dashes. Space characters are not allowed. On March 1, 2018 AWS removed the ability to use underscores in S3 bucket names.

The list shown are “AWS Managed”.

  1. Click on Policy Type to select Job function.

  2. PROTIP: Instead of scrolling down the massive list in Attache Policy (Alexa, Amazon, AWS, etc.), type in the Filter field the first few letters (such as “IoT”) and the list gets smaller. Notice the filter you type is applicable to not just characters beginning with what you typed, but also characters inside names as well.

  3. Click to select.
  4. Click “Create Group”.

    Note different policies have different levels of access, with admin having more capabilities than “read only” ones.

  5. Names shown on the screen is called a “Policy Summary”.
  6. Click “JSON” to see the file that AWS reads to assign policies. Here you seen what Actions the policy allows.

  7. Click “Access Advisor” to see users who have been assigned to use the policy.

    https://docs.aws.amazon.com/iot/latest/developerguide/create-iot-policy.html

AWS Policy Generator

The AWS Policy Generator at https://awspolicygen.s3.amazonaws.com/policygen.html creates policy files that control access to Amazon Web Services (AWS) products and resources:

  • SQS Queue Policy
  • S3 Bucket Policy
  • VPC Endpoint Policy
  • IAM Policy
  • SNS Topic Policy

https://aws.amazon.com/developer/?developer-center-activities-cards.sort-by=item.additionalFields.startDateTime&developer-center-activities-cards.sort-order=asc

References

https://awscli.amazonaws.com/v2/documentation/api/latest/reference/index.html

https://github.com/aws/aws-cli/blob/develop/README.rst

https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html

https://forums.aws.amazon.com/forum.jspa?forumID=150

TODO: Put each AWS CLI command in a script at https://medium.com/circuitpeople/aws-cli-with-jq-and-bash-9d54e2eabaf1 by Lee Harding

https://github.com/jlevy/the-art-of-command-line#macos-only

https://gist.github.com/mikepfeiffer/

https://aws.amazon.com/powershell
AWS Powershell for Windows</a>

https://github.com/jlevy/the-art-of-command-line#macos-only

#


More on Amazon

This is one of a series on Amazon:

More on DevOps

This is one of a series on DevOps:

  1. DevOps_2.0
  2. ci-cd (Continuous Integration and Continuous Delivery)
  3. User Stories for DevOps
  4. Enterprise Software)

  5. Git and GitHub vs File Archival
  6. Git Commands and Statuses
  7. Git Commit, Tag, Push
  8. Git Utilities
  9. Data Security GitHub
  10. GitHub API
  11. TFS vs. GitHub

  12. Choices for DevOps Technologies
  13. Pulumi Infrastructure as Code (IaC)
  14. Java DevOps Workflow
  15. Okta for SSO & MFA

  16. AWS DevOps (CodeCommit, CodePipeline, CodeDeploy)
  17. AWS server deployment options
  18. AWS Load Balancers

  19. Cloud services comparisons (across vendors)
  20. Cloud regions (across vendors)
  21. AWS Virtual Private Cloud

  22. Azure Cloud Onramp (Subscriptions, Portal GUI, CLI)
  23. Azure Certifications
  24. Azure Cloud

  25. Azure Cloud Powershell
  26. Bash Windows using Microsoft’s WSL (Windows Subsystem for Linux)
  27. Azure KSQL (Kusto Query Language) for Azure Monitor, etc.

  28. Azure Networking
  29. Azure Storage
  30. Azure Compute
  31. Azure Monitoring

  32. Digital Ocean
  33. Cloud Foundry

  34. Packer automation to build Vagrant images
  35. Terraform multi-cloud provisioning automation
  36. Hashicorp Vault and Consul to generate and hold secrets

  37. Powershell Ecosystem
  38. Powershell on MacOS
  39. Powershell Desired System Configuration

  40. Jenkins Server Setup
  41. Jenkins Plug-ins
  42. Jenkins Freestyle jobs
  43. Jenkins2 Pipeline jobs using Groovy code in Jenkinsfile

  44. Docker (Glossary, Ecosystem, Certification)
  45. Make Makefile for Docker
  46. Docker Setup and run Bash shell script
  47. Bash coding
  48. Docker Setup
  49. Dockerize apps
  50. Docker Registry

  51. Maven on MacOSX

  52. Ansible

  53. MySQL Setup

  54. SonarQube & SonarSource static code scan

  55. API Management Microsoft
  56. API Management Amazon

  57. Scenarios for load
  58. Chaos Engineering

More on Security

This is one of a series on Security in DevSecOps:

  1. Git Signing
  2. Hashicorp Vault

  3. WebGoat known insecure PHP app and vulnerability scanners
  4. Test for OWASP using ZAP on the Broken Web App

  5. Encrypt all the things

  6. AWS Security (certification exam)
  7. AWS IAM (Identity and Access Management)

  8. Cyber Security
  9. Security certifications