Wilson Mar bio photo

Wilson Mar

Hello!

Email me Calendar Skype call

LinkedIn Twitter Gitter Instagram Youtube

Github Stackoverflow Pinterest

Tips and tricks to get account. Lock down root accounts. Install and use the AWS CLI, securely

US (English)   Español (Spanish)   Français (French)   Deutsch (German)   Italiano   Português   Cyrillic Russian   中文 (简体) Chinese (Simplified)   日本語 Japanese   한국어 Korean

Overview

This is a hands-on tutorial to get new users setup to effectively access and use the AWS cloud. Here you do some action and explanations and PROTIP advice is provided. PROTIPs included how to install and use AWS CLI automation, smart phone apps, and 3rd party tools used by the pros.

This is adapted from what is in Amazon’s Getting Started tutorials.

GUI, CLI, API

There are several ways to interact with AWS:


API Keys

API Keys are assigned to developers using the AWS CLI (Command Line Interface) for programmatic (by a program) rather than manual clicking and typing on a keyboard.

API keys make use of pairs of public (access) key and private (secret) key which stand in for real users typing in passwords.

SSH Keys

SSH keys are used only with AWS CodeCommit to access their repositories.

This document describes steps and scripts to store your AWS credentials securely (below), not in clear text as described by AWS.

Mobile apps for smart phones

  1. On your Android, Get the AWS Console installed:

    On Google Android mobile phones

  2. On your iOS, open the Store app and search to get AWS Console. It’s from “AMZN Mobile LLC” which creates all Amazon’s apps.

    PROTIP: These app got low review scores because the app only lets people read-only, but not change anything. And the 2FA is clunky.

  3. In the Store app, search for “Google Authenticator” and install it for multi-factor authentication to strength security of your Amazon cloud account.

    PROTIP: Many keep the Authenticator running on their smart phone.

TODO: To avoid embedding an access key with the app (even in encrypted storage), use Amazon Cognito to manage user identity by authenticating users using Login with Amazon, Facebook, Google, or any OpenID Connect (OIDC)–compatible identity provider.*

AWS accounts

In enterprises, identify the Administrator who dispenses user accounts.

If you’re the Global Administrator, see my htts://wilsonmar.github.io/aws-iam

The remainder of this is for users and super users.

Root account sign-up

The account which controls billing is called the root account, which as unlimited access to AWS resources and unlimited ability to rack up charges. By resources I mean: users, groups, roles, IAM Access Policies, API keys, etc. globally for all regions.

  1. Use an internet browser to get on the AWS marketing page at

    https://aws.amazon.com

  2. Get your credit card number ready.

    PROTIP: CAUTION: Once you give Amazon a credit card number, you cannot remove it. Amazon can continue to charge for it until the card expires in several years.

    PROTIP: You need a credit card to open an account. But to limit exposure, some people provide to AWS numbers from a pre-paid reloadable Visa gift (debit) card pre-paid online (which has an expiration date and some have a monthly service fee). The Drawpay card provides a 1% refund on purchases and a mobile app to view balances. Others provide fee-Free cash withdrawal at over 25,000 MoneyPass ATMs.

    *

  3. Click the yellow “Sign-Up” button if you don’t already have an account.

  4. PROTIP: If you are creating a production account for an organization, create an email address which you use only for managing AWS and not for regular email use and certainly not for doing shopping on Amazon.

    The account which controls billing is called the root account, which as unlimited access to AWS resources and unlimited ability to rack up charges. By resources I mean: users, groups, roles, IAM Access Policies, API keys, etc. globally for all regions.

    Secure that email address with multi-factor authentication with Google or whoever hosts your email server. Also have a way for one person (or maximum two) you trust to be able to access the account in case you are not able to.

  5. PROTIP: When providing answers to Security Challenge Questions, do not specify the real answer, which someone stole or figured out through social engineering. Instead, answer with nonsense

  6. Write that secret information down in 1Password or a paper in your fire-proof vault.

  7. Write down your Account Id number (12 digits).

  8. Supply a strong password.

    PROTIP: Use 1Password so that you can easily generate up to 64 character password, but remember only one password to access the 1Password database of secrets. 1Password encrypts its database so that you can make backups (to a USB drive or secure cloud). I favor 1Password because it provides a way to sync changes with your smartphone without going through the internet.

    Because you only have to remember one master password, you can are free to change various passwords as often as you want with no fear of forgetting them.

  9. An example of a value for “AWS account name” is “master-billing”.

  10. Click “Continue”.

    If you have 1Password installed, you would be prompted to create a new account.

  11. Provide phone number, address, and credit card.

    Students may want to create several accounts to take advantage of the free tier multiple times. However, uniquely different phone numbers, addresses, and credit cards are not needed for each identity.

  12. PROTIP: Where you keep information about your credit card, note the email address and account name using that credit card.

  13. Confirm the phone number by answering Amazon’s phone call.

  14. For now, click “Free” to select a plan. A comparison on plans is discussed below.

  15. Click “Free” to be prompted to sign-in with your new credentials.


When signing in under IAM, type your Account Id number rather than your root email address.

aws-singnin-333x362

To identify your Account ID:

  1. Click on your name on the upper black menu at the top of the page, then select “My Account”.

  2. Click your account name at the top black menu for this menu:

    aws-onboarding-myaccount-184x222-9824.jpg

  3. Copy the Account Id and paste it in the notes associated with where you saed your account email and password (within 1Password).

    PROTIP: This 12 digit number is given out for others to use to sign in using sub-accounts.

  4. Scroll down to click “Edit” next to “Alternate Contacts” and put the other person who knows how to get into the account in for the Billing.

  5. Scroll down to click Edit to the right of “Configure Security Challenge Questions”.
  6. Write down your security challenge questions and answers where you wrote your Account Id.

    PROTIP: Treat the answers as another set of passwords because others my discover the real answers via social engineering. Answer with some nonsense that has no basis in reality.

    AWS Services Management Console

  7. If you are at the AWS marketing page, click “My Account” for this menu:

    aws-onboarding-landing-250x252-18241

  8. Get the AWS Management Console:

    https://console.aws.amazon.com/console/home

  9. PROTIP: Bookmark this URL

    All Amazon services

  10. Click to view all Services at the upper-left black menu band.

  11. Scroll to the category “Security, Identify, and Compliance” list of ever-growing services:

    aws-iam-svcs-cat-207x318-16992

    • WAF (Web Application Firewall) provides application-level attacks such as SQL injection and cross-site scripting.
    • Shield protects against DDoS (Denial of Service) attacks

    • Click “Artifact” (at the bottom of the list) to read documents associated with security certifications.
    • Cognito provides an API to federate authentication with various social identity providers (Facebook, Twitter, etc.)
    • GuardDuty
    • Inspector
    • Amazon Macie
    • AWS Single Sign-On
    • Certificate Manager manages security certificates
    • Cloud HSM provides
    • Directory Service
    • Cloud Trail audits usage

    PROTIP: What’s not listed above is the AWS Best Practices which this tutorial addresses.

  12. Read the User Guide for each service at:

    https://aws.amazon.com/documentation

    Quick Access icons

    Save time by quickly get to the most frequently used services by having their icons at the top (black) menu bar.

  13. Click the push-pin icon.
  14. One by one, drag the icon on the list and drop it on the top black menu to the left of the orange push pin. If you don’t see the black menu, pause just under the browser URL for the browser to automatically scroll.

    PROTIP: The services most often used are IAM, VPC, EC2, S3

  15. If you have good memory of what icons mean, change the Settings to “Icons only”.

    aws-onboarding-icons-only-277x112-9365.jpg

    Compare Support Plans

  16. Click Amazon’s Support Plan page here.

    The Basic account does not enable you to communicate with Amazon people who can answer technical questions.

    The $29/month Developer Plan enables you to open an unlimited number of support cases only via email, with a 12-hour response time if “system impaired”. Otherwise, the SLA is 24 hours.

    The $100/month Business Plan enables you to have 24/7 chat, phone, as well as email access with AWS Support people on an unlimited number of support cases, with a 1-hour response time for “production down” issues, or 4-hour response for “production impaired” issues.

    Amazon’s Enterprise Plan for $15,000/month gets you 15 minute response on “business critical system down” issues. This plan also comes with an assigned TAM (Technical Account Manager).

    These dollar amounts are minimums, not fixed prices.

    https://aws.amazon.com/premiumsupport/programs/iem/ mentions “AWS Infrastructure Event Management (IEM) offers architecture and scaling guidance and operational support during the preparation and execution of planned events, such as shopping holidays, product launches, and migrations.”

  17. Scroll down to mouse over the “$29” on the Pricing line at the bottom of the table.

    aws-onboarding-pricing-179x101-7688

    PROTIP: Pricing for Developer support is the Greater of $29 or 3% of monthly AWS usage, so you will pay more than $29 if you spend more than $966.67.

  18. Scroll back up to click the “Pricing example” link on the right.
  19. Notice that if your spend is $2,000, Amazon bills you $60 for support, not $29.

    aws-onboarding-price-example-533x307-27004.jpg</a>

  20. Click the “Business” and “Enterprise” buttons in the pop-up to see sample volume pricing tiers.

    Cases in Support Center

  21. To view support cases filed and their status, see:

    https://console.aws.amazon.com/support/home

    Policies for this are:

    • AWSSupportAccess (Allows users to access the AWS Support Center)
    • SupportUser (This policy grants permissions to troubleshoot and resolve issues in an AWS account. This policy also enables the user to contact AWS support to create and manage cases)

  22. Scroll down to view videos on specific technical issues by Amazon people.

    On the lower-right corner, there are links to AWS Documentation, Getting Started Guides, Knowledge Center, Whitepapers, and AWS Forums.


Claim S3 Bucket names

The AWS Account Administrator has a fudiciary responsibility to secure Intellectual Property assets.

S3 Bucket names are universally unique among all AWS customers. So just as there are domain name squatters who register and sit on .com host names for sale at high prices to those who actually use the names, the administrator of root accounts for an organization should register your organization’s brand names before others get them first.

To create a bucket for each host name registered on GoDaddy, Google Domains, etc.

  1. Click S3 from among services.
  2. Click the blue “Create bucket” button.
  3. Type in the host name (such as “wilsonmar.com”) in the Bucket name field.
  4. Select your home Region.

    PROTIP: Claiming a Bucket name in one region locks it up for all Regions.

  5. Click “Next”.
  6. Click “Next”.
  7. Click “Next” to manage users.
  8. Click “Create Bucket”.

Root account lockdown

  1. On a browser in the AWS Management Console, select IAM (for Identity Access Management) for the list Security Status

    A new account will have this:

    aws-iam-status-334x256-24837

    To get back to this later, click “Dashboard” on the IAM menu on the left.

    The FAQ to this is at https://aws.amazon.com/iam/faqs

  2. Click on “Delete your root access key”.

  3. Check “Don’t show me this message again” and Continue to Security Credentials.

    Password

  4. PROTIP: Use 1Password to store your passwords so that you can use a “strong” password of so many characters that it will take hackers too much time to crack it. Because you only have to remember one master password, you can are free to change various passwords as often as you want with no fear of forgetting them.

    Apply an IAM password policy

  5. Click “Manage Password Policy” so AWS will ensure that “strong” passwords are used (and not easy to guess ones). AWS defaults are terrible:

aws-iam-weak-386x336-39852

Over time, as hackers have access to more powerful computers that can guess passwords quicker, larger passwords are necessary to make it more difficult to crack.

  1. PROTIP: The largest Minimum password length AWS allows is 128 characters. But 1Password can generate up to only 64 characters. Practically, 22 characters is a reasonable minimum. Require at least one number (digits) and one non-alphanumeric symbol character.

    aws-iam-1password-291x259-19343

  2. Scroll down to “Security Token Service Regions” and deactivate regions your organization will never use.

    PROTIP: The region is where most of your users are located. New services are usually restricted to one region, such as N. Virginia or N. California where AWS does development work.

    MFA (Multi-Factor Authentication)

    This has AWS text or call your smartphone (a virtual device) to make sure that it’s really you logging in.

  3. Click Activate MFA
  4. Click “A virtual MFA device”.
  5. Click Next Steps.

    Install MFA app

  6. On your iPhone or Android mobile app, open the Store app.
  7. Search for Google Authenticator app (if you don’t already have it installed).
  8. Click “Get” to install it.

  9. Click “Open”.

  10. In the the Google Authenticator app, click the “+” icon at the top of the screen.
  11. Click “Scan barcode”.
  12. Align the QR code (with the square of dots) within the green box.
  13. Wait for the Google Authenticator app to display two codes. Under the codes we want now begins with “root-account-mfa-device@” followed by the 12-digit Account Id.
  14. Type the first code for the account into the AWS Console website “Authentication code 1”.

    PROTIP: Do not type the space between numbers so that you enter only 6 digits.

  15. Press Tab and type the second code in “Authentication code 2”.

    PROTIP: A new code is created every minute.

  16. Scroll down to click “Activate virtual MFA” at the bottom of the screen.

    MFA in profile

    To specify use of MFA in an assumed role provider profile, see this example of credentials file:

     [profile prod-access]
     role_arn=arn:aws:iam::123456789012:role/ReinventProdAccess
     source_profile=development
     
     [profile prod-full-s3-access]
     role_arn=arn:aws:iam::123456789012:role/FullS3Access
     source_profile=development
     mfa_serial=arn:aws:iam::18490616333:mfa/james
    
  17. Test on Console: VIDEO:

    aws s3 ls --profile prod-full-s3-access

    The response is a prompt waiting for manual input:

    Enter MFA code: _

    ### Create Admin sub-account

  18. In the IAM page click “Create individual IAM users”. What it says is important:

    “Create IAM users and give them only the permissions they need. Do not use your AWS root account for day-to-day interaction with AWS, because the root account provides unrestricted access to your AWS resources.”

  19. Click “Manage users”.
  20. Click “Add User”.
  21. PROTIP: For the user name field, define a pattern of up to 64 characters with dashes (instead of spaces and underlines) to separate words.

    For the Administrator to do work (of assigning):

    root-admin-work

  22. Click “Programmatic access”.
  23. If you would like to use AWS Management Console access, leave the default for Autogenerated password because you’ll create a new password at next sign-in.
  24. Click “Next: Permissions”.

    We’ll add groups later, below.

  25. Click “Attach existing policies directly” because the Admin account it is limited.

  26. Rather than granting “AdministratorAccess” which gives all access, give policy to what :

    • SystemAdministrator
    • IAMFullAccess covers the others:

      • IAMReadOnlyAccess
      • IAMSelfManageServiceSpecificCriteria
      • IAMUserChangePassword
      • IAMUserSSHKeys
  27. Click “Next: Review”.
  28. Click “Create user”.

    Inform user of credentials

  29. To see what is sent if you click “Send email”, right-click on the link and “Copy Link”, then paste in a text editor to see:

    subject=Welcome to Amazon Web Services
    body=Hello,  You have been given access to the AWS Management Console for the Amazon Web Services account ID ending in 8630. You can get started by using the sign-in information provided below.%0A%0ASign-in URL: https://103265058630.signin.aws.amazon.com/console%0AUser name: root-admin-work   
    Your initial sign-in password will be provided separately from this email. When you sign in for the first time, you must change your password. 
    Sincerely, Your AWS Account Administrator
  30. PROTIP: Along with the Access Key Id and Secret access key, the default Region and format are also required to perform “aws configure”, so add that information in the email.

    PROTIP: Add what AWS Groups and associated Policies the user has been given.

    PROTIP: Also include in the email, for those who use AWS CLI, how to install it and 3rd-party tools.

    For those who use the AWS Console GUI, explain the mobile apps to install. Provide them the URL with the region included, such as:

    https://us-west-2.console.aws.amazon.com/lambda/home?region=us-west-2

    NOTE: Baking different zones into Console URLs makes for more direct connections and removes issues from using a single URL/DNS.

  31. Click “Download .csv” to download a “credentials.csv” file to your Downloads folder. It contains columns are a couple columns different than the “Add User” GUI:

    User name, Password, Access key ID, Secret access key, Console login link

    The “Console login link” is the “Sign-in URL” in the email.

    Apply an IAM password policy

  32. Click “Manage Password Policy” so AWS will ensure that “strong” passwords are used (and not easy to guess ones).

AWS defaults are terrible: aws-iam-weak-386x336-39852</a>

PROTIP: Over time, as hackers have access to more powerful computers that can guess passwords quicker, larger passwords are necessary to make them more difficult to crack.

  1. PROTIP: The largest Minimum password length AWS allows is 128 characters. 1Password can generate up to only 64 characters. Practically, 22 characters is a reasonable minimum. Require at least one number and one non-alphanumeric character.

    aws-iam-1password-291x259-19343

    PROTIP: Each site may have different rules about what special characters are allowed. So generate a smaller string, then manually add special characters. Copy the final string before pasting into the form.

  2. Click “Apply password policy”.

    Deactivate regions not used

    On the same “Account settings” page:

  3. Scroll down to “Security Token Service Regions” and deactivate regions your organization are not using.

    PROTIP: Select a Region where most of your target users are located. New services are usually restricted to one region, such as N. Virginia or N. California where AWS does development work.

    Admin Sign In

  4. Sign out and sign in again to the AWS Console using the newly created admin sub-account.

    Programmatic Access

    Instead of doing what other clouds do (an aws login command which prompt for a user name and password), aws commands reference a specifically-named file at $HOME/.aws/credentials created by command aws configure.

    The aws configure command creates that file after prompting for access key identifiers (AKIDs) to an AWS account. Press Enter to accept the value previously defined:

    • AWS Access Key ID [******L5ZQ]:
    • AWS Secret Access Key [******+1MD]:

Stored with credentials are also:

  • Default region name [us-east-1]:
  • Default output format [json]:

To create AKID credentials, AWS asks that account owners to manually use the IAM GUI to disable programmatic access to their root (email) account and protect it with MFA (Multi-factor Authentication)

The AWS Management Console provides a way for account owners (administrators) to manually create IAM user accounts for programmatic access.

For programmatic access to resources running inside AWS, the best practice is to use IAM roles which are not associated with a specific user or group. Any trusted entity can assume the role to perform a specific business task. A resource can be granted access without hardcoding an access key ID and secret access key into the configuration file. For example, you can grant an Amazon Elastic Compute Cloud (EC2) instance access to an Amazon Simple Storage Service (Amazon S3) bucket by attaching a role with a policy that defines this access to the EC2 instance. IAM dynamically manages the credentials for you with temporary credentials it rotates automatically.

Outside AWS (on a Terminal/Console on your laptop), a dedicated service account should be created for each use case with only the permissions needed to limit the “blast radius” if credentials are compromised. For example, if a monitoring tool and a release management tool both require access to your AWS environment, create two separate service accounts with two separate policies that define the minimum set of permissions for each tool.

CAUTION: The problem with IAM user account secrets is that they are long-running secrets stored in the credentials file in clear-text. Someone who clicks on a roque link on a phishing email would expose that file for theft. Many who lose control of their AWS credentials see bills from Amazon of thousands of dollars in unauthorized use (mining Bitcoins).

CloudAcademy.com and many enterprises create a centrally-administered https://aws.amazon.com/code/token-vending-machine-for-identity-registration-sample-java-web-application/ “Vending Machine” application to generate and dispense temporary IAM user accounts with access keys. Such credentials are valid for only 12 hours or less.

But that requires tedious repeated manual effort. Securing temporary accounts with MFA adds to that toil.

Automatic key rotation

https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html describes automatic rotatation of AKID credentials (with a quick MFA challenge answered on a mobile phone)

aws-AccessKeyAutoRotate-799x830

The auto-rotation of AWS IAM User Access Keys diagrammed above is from these guideline from Feb. 2019 uses MIT-licensed CloudFormation templates and Python scripts defined in https://github.com/aws-samples/aws-iam-access-key-auto-rotation and described step-by-step in this Word-format Document.

Setup S3 buckets in the US East (N. Virginia) Region (us-east-1). It runs every 90 days. At 100 days it disables and at 110 days it deletes the old Access Keys. It sets up a secret inside AWS Secrets Manager to store the new Access Keys, with a resource policy that permits only the AWS IAM User access to them.

Another automation sets up an Amazon DynamoDB table to house the email addresses of accounts rotated. These emails are used by a SNS Topic to send alerts when rotation occurs.

Alternately, you can refactor to send a Slack message instead of email (not shown in the diagram).

DOCS

aws sts assume-role --role-arn arn:aws:iam::123456789012:role/role-name \
   --role-session-name "RoleSession1" \
   --profile IAM-user-name > assume-role-output.txt

https://aws.amazon.com/blogs/security/how-to-rotate-access-keys-for-iam-users/

$ aws iam list-access-keys { “AccessKeyMetadata”: [ { “AccessKeyId”: “AKIAI2YGLLOSZDQ3L5Z1”, “Status”: “Active”, “CreateDate”: “2020-06-12T04:04:22+00:00” } ] }

AWS IAM commands use unique access key identifiers (AKIDs) to refer to individual access keys.

$ aws iam create-access-key –user-name Alice

Identity and Access Management (IAM) roles for Amazon EC2.

https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys

https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html

  1. Grant temporary access keys - aws sts assume-role.

https://docs.aws.amazon.com/secretsmanager/latest/userguide/rotating-secrets-one-user-multiple-passwords.html

Additionally, add conditions to the policy that further restrict access, such as the source IP address range of clients. The example policy below grants the needed permissions (PutObject) on to a specific resource (an S3 bucket named “examplebucket”) while adding further conditions (the client must come from IP range 203.0.113.0/24):

{
    "Version": "2012-10-17",
    "Id": "S3PolicyRestrictPut",
    "Statement": [
            {
            "Sid": "IPAllow",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::examplebucket/*",
            "Condition": {
                "IpAddress": {"aws:SourceIp": "203.0.113.0/24"}
            } 
        } 
    ]
}

IAM CLI

AWS Identity and Access Management (IAM) controls access to users, groups, roles, and policies.

  1. List users:

    
    aws iam list-users --query Users[*].UserName
    
  2. List groups which the user belongs to :

    aws iam list-groups-for-user --username ???

  3. Create a new user named “MyUser”:

    aws iam create-user --user-name MyUser
    

</pre>

The response is:

{
    "User": {
        "UserName": "MyUser",
        "Path": "/",
        "CreateDate": "2012-12-20T03:13:02.581Z",
        "UserId": "AKIAIOSFODNN7EXAMPLE",
        "Arn": "arn:aws:iam::123456789012:user/MyUser"
    }
}
  1. Add the user to the group:

    aws iam add-user-to-group --user-name MyUser --group-name MyIamGroup
  2. To verify that the MyIamGroup group contains the MyUser, use the get-group command:

    aws iam get-group --group-name MyIamGroup

    The response:

     {
         "Group": {
             "GroupName": "MyIamGroup",
             "CreateDate": "2012-12-20T03:03:52Z",
             "GroupId": "AKIAI44QH8DHBEXAMPLE",
             "Arn": "arn:aws:iam::123456789012:group/MyIamGroup",
             "Path": "/"
         },
         "Users": [
             {
                 "UserName": "MyUser",
                 "Path": "/",
                 "CreateDate": "2012-12-20T03:13:02Z",
                 "UserId": "AKIAIOSFODNN7EXAMPLE",
                 "Arn": "arn:aws:iam::123456789012:user/MyUser"
             }
         ],
         "IsTruncated": "false"
     }

Linux AMIs

Types of operating system AMI:

  • Amazon Linux 2014.09.2 (CentOS)
  • Red Hat Enterprise Linux 6.6 (RHEL)
  • SUSE Linux Enterprise Server 12
  • Ubuntu Server 14.04

Advanced User Data

https://gist.github.com/mikepfeiffer/

   
  • https://aws.amazon.com/powershell
    AWS Powershell for Windows</a>

    aws Get-AWSCredentials -ListProfiles

Diagrams

ProcessOn.com provides a free on-line tool to draw diagrams such as this

At architecture/icons Amazon provides a sample .PPTX (PowerPoint 2010+) file (AWS_Simple_Icons_PPT_v16.2.22.zip). Lines used to illustrate the hierarchy:

PROTIP: Use different colors for lines and text to reduce visual confusion.

You can also download a zip containing .png and .svg files of icons (AWS_Simple_Icons_EPS-SVG_v16.2.22.zip).


Hashicorp Terraform Enterprise

VIDEO: Hashicorp has a “Sentinal” product component which enforces various fine-grained rules (policy sets) to what can be done by each role. It also estimates monthly cost from cloud usage.

Rules in Hashicorp’s Foundational Policy library is at https://github.com/hashicorp/terraform-foundational-policies-library. Such “Policies as Code” are crafted based on Center for Internet Security (CIS) Benchmarks [pdf] (including Compute, Databases, Kubernetes, Storage, Networks) covering Azure and GCP as well as AWS.


Social

https://www.twitch.tv/aws/videos/all videos include:

[_] Create Forum Account

  1. PROTIP: To ensure anonymity interacting on public forums, the Administrator should create in a public email system (such as gmail.com, hotmail.com, etc.) an email address for use on forums. Don’t use a real name in the email address, but a positive adjective with a number to ensure it’s unique, such as “concerned123”.

    AWS says “Your email will be kept private” but I don’t trust that they can’t be hacked.

  2. Go to the AWS forums at URL:

    https://forums.aws.amazon.com/forum.jspa?forumID=150

  3. Register the new email address along with an AWS Nickname without a proper name, such as, again, “concerned123”.

  4. Use that email in StackOverflow.com and other public forums.

Tutorial Rock Stars and their presentations

Jeff Barr (@jeffbarr), AWS Chief Evangelist makes announcements of all new stuff at the company’s AWS Blog and #AWS Twitter hash-tag

Yan Kurniawan

J O’connner:

  • http://joconner.com/

Ryan Scott Brown @ryan_sb

  • https://serverlesscode.com/post/new-ssl-tls-cert-manager-acm/

Matt Wood, @mza, Product Strategy @ Amazon Web Services

AWS Certifications

Practice exams

CloudAcademy has a 90-minute practice exam. Each retake shows 65 questions from a large pool.

Because a minimum score of 35 percent is needed on each exam domain, Pearson Practice tests at OReilly has a “study mode” which allow you to provide answers after every question from across 4 complete exams. You can also filter by AWS Certified Cloud Practioner (CLF-C01) domains (number of questions in bank):

* 01 - The AWS Cloud Defined (26)
* 02 - Advantages of the AWS Cloud (10)
* 03 - Core AWS Services
* 04 - Cloud Architecture Design Principles
* 05 - The AWS Shared Responsibility Model
* 06 - Cloud Security and Compliance
* 07 - AWS Access Management Capabilities
* 08 - Resources for Security Support
* 09 - Methods of Deploying and Operating in AWS
* 10 - The AWS Global Infrastructure
* 11 - Resources for Technology Support
* 12 - Using the Free Tier to Build a Web Server
* 13 - AWS Pricing Models
* 14 - Account Structures for Billing and Pricing
* 15 - Resources for Billing Support    <br /><br />

For the AWS Certified SysOps Administrator Associate (SOA-C01) test

For the AWS Certified Solutions Architect Associate SAA-C02

For the AWS Certified Security – Specialty SCS-C01

For the AWS Certified Machine Learning-Specialty (ML-S)

For the WS Certified Big Data - Specialty (CCENT)

Other Practice Tests

AWS Training Resources

References

After signing up for https://www.aws.training, Authentication and Authorization with AWS Identity and Access Management 15 minutes

SWF (Simple Workflow Functions) sequences manual work.

AppStream streams desktop apps (like Citrix).

Elastic Transcoder of videos into various sizes and formats (ogg, mp4, etc.)

Orion Papers on Lucidchart

https://scriptcrunch.com/aws-certification-iam-essentials-cheat-sheet/

https://www.youtube.com/watch?v=e2A8K47Fj6s&index=4&list=PLZbbT5o_s2xoWPNdBbqi9eWnMJ5cDrr1M How to Configure the AWS CLI | Amazon Web Services | AWS</a> Nov 26, 2017 by deeplizard

https://docs.aws.amazon.com/cli/latest/index.html AWS CLI Command Reference

More on Amazon

This is one of a series on Amazon:

  1. To verify the identity being used in AWS CLI:

    aws sts get-caller-identity

    A sample response:

     "Account": "103265058630", 
     "UserId": "AIDAJHXCZNAH2MEXAMPLE",
     "Arn": "arn:aws:iam::103265058630:user/root-admin-work"
    

    Alternately, use an alias defined:

    aws whoami

    Define groups to assign permissions

    PROTIP: For a user to do something usually require several AWS resources. So several permissions need to be granted to a user. To simplify assignments, we define Groups of permissions which we then can assign to each user.

    In other words, An IAM group is a management convenience to manage the same set of permissions for a set of IAM users.

    The AWS CLI command to create a group named “MyIamGroup” is:

    aws iam create-group --group-name MyIamGroup
    

    A sample response:

    {
     "Group": {
         "GroupName": "MyIamGroup",
         "CreateDate": "2012-12-20T03:03:52.834Z",
         "GroupId": "AKIAI44QH8DHBEXAMPLE",
         "Arn": "arn:aws:iam::123456789012:group/MyIamGroup",
         "Path": "/"
     }
    }
    

    The AWS CLI command to create a S3 security group:

    aws ec2 create-security-group --group-name my-sg --description "My security group"
    

    A sample response:

    {
    "GroupId": "sg-903004f8"
    }
  2. Click Manage Groups then Create New Group.

    PROTIP: Groups are usually associated with a particular job: admin, sales, HR, front-end developer, back-end developer, etc.

    A user can belong to multiple groups. More complex organizations manage differences in permissions for company, division, project, location, job level, etc. So 128 characters may not be enough if large words are used. Thus, abbreviate and use acronyms.

    PROTIP: Put abbreviations and acronyms in a wiki publicly available to avoid duplicate usage.

  3. “aws_iot_buttons” is the group name I use as an example.

PROTIP: Use dashes. Space characters are not allowed. On March 1, 2018 AWS removed the ability to use underscores in S3 bucket names.

The list shown are “AWS Managed”.

  1. Click on Policy Type to select Job function.

  2. PROTIP: Instead of scrolling down the massive list in Attache Policy (Alexa, Amazon, AWS, etc.), type in the Filter field the first few letters (such as “IoT”) and the list gets smaller. Notice the filter you type is applicable to not just characters beginning with what you typed, but also characters inside names as well.

  3. Click to select.
  4. Click “Create Group”.

    Note different policies have different levels of access, with admin having more capabilities than “read only” ones.

  5. Names shown on the screen is called a “Policy Summary”.
  6. Click “JSON” to see the file that AWS reads to assign policies. Here you seen what Actions the policy allows.

  7. Click “Access Advisor” to see users who have been assigned to use the policy.

    https://docs.aws.amazon.com/iot/latest/developerguide/create-iot-policy.html


Manually Rotate Access keys

See that "AWS recommends that you rotate your access keys every 90 days"?
Some find it easier to remember by doing it on the first day of each month.
Why? There are thousands of big computers around the world literally staying up at night trying different combinations.
  1. PROTIP: Make an appointment on your Calendar with a recurring schedule.

    PROTIP: Rotation applies to access key of IAM child accounts, not the root account.

    You don’t want programmatic access to your root account, so you don’t need no stinkin’ keys.

  2. Click Delete to the key. Write down the date Created.

    Don’t create a new Access Key.

    Use groups to assign permissions

    PROTIP: For a user to do something usually require several AWS resources. So several permissions need to be granted to a user. To simplify assignments, we define Groups of permissions which we then can assign to each user.

    In other words, An IAM group is a management convenience to manage the same set of permissions for a set of IAM users.

  3. Click Manage Groups then Create New Group.

    PROTIP: Groups are usually associated with a particular job: admin, sales, HR, front-end developer, back-end developer, etc.

    A user can belong to multiple groups. More complex organizations manage differences in permissions for company, division, project, location, job level, etc. So 128 characters may not be enough if large words are used. Thus, abbreviate and use acronyms.

    PROTIP: Put abbreviations and acronyms in a wiki publicly available to avoid duplicate usage.

  4. “aws_iot_buttons” is the group name I use as an example.

PROTIP: Use underlines or dashes. Space characters are not allowed.

The list shown are “AWS Managed”.

  1. Click on Policy Type to select Job function.

  2. PROTIP: Instead of scrolling down the massive list in Attache Policy (Alexa, Amazon, AWS, etc.), type in the Filter field the first few letters (such as “IoT”) and the list gets smaller. Notice the filter you type is applicable to not just characters beginning with what you typed, but also characters inside names as well.

  3. Click to select.
  4. Click “Create Group”.

    Note different policies have different levels of access, with admin having more capabilities than “read only” ones.

  5. Names shown on the screen is called a “Policy Summary”.
  6. Click “JSON” to see the file that AWS reads to assign policies. Here you seen what Actions the policy allows.

  7. Click “Access Advisor” to see users who have been assigned to use the policy.

    https://docs.aws.amazon.com/iot/latest/developerguide/create-iot-policy.html

    Create IAM Users

  8. Click Users on the left menu.
  9. Click Add User.
  10. Specify User Name. For example: user1@myco.com

    PROTIP: Use underscores to separate words in IAM User Names rather than spaces.

  11. Check “Programmatic Access”.
  12. Uncheck “User must create a new password at next sign-in”.
  13. Click “Next: Permissions”.
  14. Click “Attach existing policies directly” for the first user.

    PROTIP: The policy attached depends on what the user will be allowed to do.

  15. Send to each user the AccountId, UserName using a different mode of communication than the password.
  16. User signs in using the credentials Account Id, the UserName, and password
  17. Click “Send email”

    PROTIP: Send credentials to your alternate email rather than to a cloud drive (Amazon, Google, Box, etc.); an email account that you setup with a fake birthdate and other personal information; one you never give out to anyone.


Roles for federated access

An analogy is a private ball where royal guests arrive wearing formal attire present an invitation card to enter. The fancy outfits with sashes and medals are kinda like group permissions that confer permissions to someone. The invitation card is kinda like IAM roles which are only for specific times.

The host of the party is kinda like AWS’s STS (Security Token Service) identify broker which grants access tokens to enable services to “assume” a role to perform on AWS services.

IAM roles are used by computer programs reaching through Enterprise identity federation into Microsoft Active Directory using SAML (Security Assertion Markup Language) or through Web identity federation into Google, Facebook, Amazon, etc.

IAM roles issue keys are valid for short durations, making them a more secure way to grant access.

An IAM user needs to be granted two distinct permissions to launch EC2 instances with roles:

  • Permission to launch EC2 instances.
  • Permission to associate an IAM role with EC2 instances.

STS returns:

  • A Security Token
  • An Access Key ID
  • A Secret Access Key

More security

  • egress rules on your Security Groups (after all there’s no reason ever that your database should be connecting to IP addresses in Russia),
  • vulnerability scanning,
  • Host-Based Intrusion Detection (HIDS) systems

AWS CLI Automation

The Command line interface (CLI) is used by programs rather than the manual AWS Console GUI.

In enterprises today, servers are built by scripts and configuration files generated from templates (usually multi-platform Hashicorp Terraform more than AWS CloudFormation templates).

This is so the build process can be debugged and changed slightly through the lifecycle from test to prod.

Instead of clicking and typing, server administrators work with template files in JSON format for Cloud Formation or Terraform to process.

Atlas generates JSON files based on information typed into their web Consoles.

AWS CLI install

Several ways are presented to install AWS CLI using Python. Homebrew is my favorite becuase you can upgrade easily:

  1. The simplest and most reliable for me is to use HomeBrew on Macs, from any folder:

    brew upgrade awscli

    If awscli was not already installed:

    brew install awscli

    🍺  /usr/local/Cellar/awscli/2.2.21: 12,806 files, 100.3MB
    Removing: /usr/local/Cellar/awscli/2.2.14... (12,776 files, 101.8MB)
    

    NOTE: awscli installs the latest dependencies Ansible, ykman, etc.

    Alternately, one can use pip install awscli –upgrade –user –ignore-installed six installed from https://pypi.org/project/awscli. But when I did, aws cannot be found.

    Another alternative to install (on CentOS 7) is:

    curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" \
       -o "awscli-bundle.zip"
    unzip awscli-bundle.zip 
    sudo ./awscli-bundle/install \
       -i /usr/local/aws -b /usr/local/bin/aws
    
  2. Verify what version of awscli you have installed:

    aws --version
    

    Something went wrong if your response is:

    -bash: aws: command not found

    The expected sample response (May 28, 2018):

    aws-cli/2.2.21 Python/3.9.6 Darwin/18.7.0 source/x86_64 prompt/off
    

    NOTE: Awscli now uses Python 3, not 2.7.

    Also previously:

    aws-cli/1.15.20 Python/3.6.5 Darwin/17.5.0 botocore/1.10.20
    

    AWS Boto for Python

    PROTIP: “AWS SDK for Python” enables your Python (.py) programs to invoke AWS CLI commands.

    The Python package botocore on GitHub provides a low-level foundation for AWS CLI software.

    Ansible internally uses Boto to connect to Amazon EC2 instances and hence you need Boto library in order to run Ansible on your laptop/desktop. TOOL: Use Ansible to copy files from local to remote host.

  3. Make sure you’re not within Conda:

    conda deactivate

    If it’s already deactivated, you should not get any message.

  4. To install Boto3:

    pip install –upgrade boto3 –user –ignore-installed six

    Code for boto3 is obtained from https://github.com/boto/boto3. Read about it at https://aws.amazon.com/sdk-for-python.

    NOTE: The package is installed into folder:
    /usr/local/lib/python2.7/site-packages/boto3/*

  5. Install Boto as well:

    pip install boto --user

    It’s in /usr/local/anaconda3/lib/python3.7/site-packages (2.49.0)

    The boto package is the hand-coded Python library that has been around since 2006. It is very popular and is fully supported currently by AWS. But because it is hand-coded and there are so many services available (with more appearing all the time) it is difficult to maintain.

    boto3, generally available since 06/22/2015, is a new version of the boto library based on botocore. All of the low-level interfaces to AWS are driven from JSON service descriptions that are generated automatically from the canonical descriptions of the services. So, the interfaces are always correct and always up to date. There is a resource layer on top of the client-layer that provides a nicer, more Pythonic interface.

The boto3 library is being actively developed by AWS and is the one I would recommend people use if they are starting new development.

### Bash Shell completions

  1. On Linux, to enable bash completion for aws commands:

    echo “\n” » ~/.bashrc echo ‘complete -C aws_completer aws’ » ~/.bashrc

  2. Test out autocompletion by typing the first two characters and pressing Tab for a list of all aws cli commands that begin with those characters:

    AWS Shell completion

    PROTIP: For automatic complex autocompletion of AWS CLI commands, there is a 3rd-party utility that provides a shell GUI that suggest as you type:

    Read about it at https://github.com/awslabs/aws-shell

  3. To install the awesome AWS Shell:

    pip install aws-shell

    The package is installed in folders: /usr/local/bin/aws-shell

    If you see these error messages:

    ERROR: requests 2.22.0 has requirement urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1, but you'll have urllib3 1.26.6 which is incompatible.
    ERROR: jupyter-console 6.0.0 has requirement prompt_toolkit<2.1.0,>=2.0.0, but you'll have prompt-toolkit 1.0.18 which is incompatible.
    ERROR: ipython 7.6.1 has requirement prompt-toolkit<2.1.0,>=2.0.0, but you'll have prompt-toolkit 1.0.18 which is incompatible.
    
  4. To enable AWS Shell:

    aws-shell

    First run, creating autocomplete index...
    Creating doc index in the background. It will be a few minutes before all documentation is available.
    

    You show now be in the sub-shell with prompt:

    aws>

    aws-onboarding-aws-shell-config-207x58-5051.jpg

  5. Exit aws-shell back to bash:

    .exit

    Alternately, .quit works too.

    jp command

    The jp command enables JSON to be manipulated within Bash scripts.

  6. Install it on Macs, in any folder:

    brew tap jmespath/jmespath
    brew install jp
    
    🍺  /usr/local/Cellar/jp/1.1.12: 3 files, 3MB
  7. Verify it works by running a sample command:

    For example, jp enables a simple syntax to extract the 1st value from bar within foo:

    echo '{"foo": {"bar": ["a", "b", "c"]}}' | jp foo.bar[1]

    The response should be: “b”

    WHOOPS: 2021/07/21 09:11:38 line.go:44: no valid y values given

  8. See other usage and examples at https://github.com/jmespath/jp#usage

    jp is required by Aliases, below.

    Aliases

    Create folder ~/.aws/cli/alias:

    mkdir -p ~/.aws/cli
    pushd ~/.aws/cli
    # From git clone https://github.com/awslabs/awscli-aliases --depth=1 alias
    curl -O https://raw.githubusercontent.com/awslabs/awscli-aliases/master/alias
    popd
    

    Further explained in video https://www.youtube.com/watch?v=Xc1dHtWa9-Q&t=26m35s

    Configure for Login

    Regardless of how you get the command:

    Configure profiles

    PROTIP: You’ll likely need to use several AWS accounts, so specify a profile for each account.

  9. Run the command to create files in folder ~/aws referenced by all other aws cli commands:

    aws configure –profile root-admin-work

    PROTIP: The example “root-admin-work” would be replaced with the user’s account name being created. Different accounts may be needed for different permissions in prod vs. dev use. Having separate access keys for different applications also generates distinct entries in AWS CloudTrail log files, which makes it easier to determine which application performed specific actions.

    Without the profile specification, “aws configure” by itself defines default credentials.

    The command prompts you for:

    AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
    AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
    Default region name [None]: us-west-2
    Default output format [None]: json
    

    PROTIP: If you do not explicitly specify an endpoint, US West (Oregon) us-west-2 is the default Region.

    The default output format is json.

    PROTIP: The aws configure command creates key/value pairs “aws_access_key_id” and “aws_secret_access_key” in file credentials for use by all AWS SDKs. Key/value pairs “region” and “output” are saved in file config used by the CLI.

    TODO: http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#cli-environment

  10. The region in ~/.aws/config can be set also by:

    aws configure set profile.prod.region us-west-1
    

    Path ~/.aws/config is in variable $AWS_CONFIG_FILE

    Path ~/.aws/credentials is in variable $AWS_SHARED_CREDENTIALS_FILE

    aws configure set region \
       $(curl -s http://162.254.169.254/latest/dynamic/instance-identity/document \
       | jp -u 'region')
    

    Roles for Tasks

    TODO: Temporary security credentials Roles for Tasks are stored in the ~/.aws/config file:

    [profile iam-role]
    role_arn = arn:aws:iam::ACCOUNT_ID:role/IAM_ROLE
    source_profile = iam-user
    output = json
    region = eu-west-1
    

    Importantly, the default region is specified in ~/.aws/config.

    PROTIP: The ~/.aws/config file also houses settings that speed up S3 sync.

    [profile default]
    ...
    s3 =
      max_concurrent_requests = 100
      max_queue_size = 10000
      use_accelerate_endpoint = true
    

    Services list

    Now that you have permissions after configuration:

    curl -fsSL https://raw.githubusercontent.com/wilsonmar/DevSecOps/master/aws/aws-info.sh

  11. For a list of Amazon services with command access:

    aws commands help

    PROTIP: Drag the left/right edge of the Terminal to widen the screen.

    usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
    To see help text, you can run:
     
      aws help
      aws <command> help
      aws <command> <subcommand> help
     
    aws: error: argument command: Invalid choice, valid choices are:
     
    accessanalyzer                           | acm
    acm-pca                                  | alexaforbusiness
    amp                                      | amplify
    amplifybackend                           | apigateway
    apigatewaymanagementapi                  | apigatewayv2
    appconfig                                | appflow
    appintegrations                          | application-autoscaling
    application-insights                     | applicationcostprofiler
    appmesh                                  | apprunner
    appstream                                | appsync
    athena                                   | auditmanager
    autoscaling                              | autoscaling-plans
    backup                                   | batch
    braket                                   | budgets
    ce                                       | chime
    cloud9                                   | clouddirectory
    cloudformation                           | cloudfront
    cloudhsm                                 | cloudhsmv2
    cloudsearch                              | cloudsearchdomain
    cloudtrail                               | cloudwatch
    codeartifact                             | codebuild
    codecommit                               | codeguru-reviewer
    codeguruprofiler                         | codepipeline
    codestar                                 | codestar-connections
    codestar-notifications                   | cognito-identity
    cognito-idp                              | cognito-sync
    comprehend                               | comprehendmedical
    compute-optimizer                        | connect
    connect-contact-lens                     | connectparticipant
    cur                                      | customer-profiles
    databrew                                 | dataexchange
    datapipeline                             | datasync
    dax                                      | detective
    devicefarm                               | devops-guru
    directconnect                            | discovery
    dlm                                      | dms
    docdb                                    | ds
    dynamodb                                 | dynamodbstreams
    ebs                                      | ec2
    ec2-instance-connect                     | ecr
    ecr-public                               | ecs
    efs                                      | eks
    elastic-inference                        | elasticache
    elasticbeanstalk                         | elastictranscoder
    elb                                      | elbv2
    emr                                      | emr-containers
    es                                       | events
    finspace                                 | finspace-data
    firehose                                 | fis
    fms                                      | forecast
    forecastquery                            | frauddetector
    fsx                                      | gamelift
    glacier                                  | globalaccelerator
    glue                                     | greengrass
    greengrassv2                             | groundstation
    guardduty                                | health
    healthlake                               | honeycode
    iam                                      | identitystore
    imagebuilder                             | importexport
    inspector                                | iot
    iot-data                                 | iot-jobs-data
    iot1click-devices                        | iot1click-projects
    iotanalytics                             | iotdeviceadvisor
    iotevents                                | iotevents-data
    iotfleethub                              | iotsecuretunneling
    iotsitewise                              | iotthingsgraph
    iotwireless                              | ivs
    kafka                                    | kendra
    kinesis                                  | kinesis-video-archived-media
    kinesis-video-media                      | kinesis-video-signaling
    kinesisanalytics                         | kinesisanalyticsv2
    kinesisvideo                             | kms
    lakeformation                            | lambda
    lex-models                               | lex-runtime
    lexv2-models                             | lexv2-runtime
    license-manager                          | lightsail
    location                                 | logs
    lookoutequipment                         | lookoutmetrics
    lookoutvision                            | machinelearning
    macie                                    | macie2
    managedblockchain                        | marketplace-catalog
    marketplace-entitlement                  | marketplacecommerceanalytics
    mediaconnect                             | mediaconvert
    medialive                                | mediapackage
    mediapackage-vod                         | mediastore
    mediastore-data                          | mediatailor
    meteringmarketplace                      | mgh
    mgn                                      | migrationhub-config
    mobile                                   | mq
    mturk                                    | mwaa
    neptune                                  | network-firewall
    networkmanager                           | nimble
    opsworks                                 | opsworkscm
    organizations                            | outposts
    personalize                              | personalize-events
    personalize-runtime                      | pi
    pinpoint                                 | pinpoint-email
    pinpoint-sms-voice                       | polly
    pricing                                  | proton
    qldb                                     | qldb-session
    quicksight                               | ram
    rds                                      | rds-data
    redshift                                 | redshift-data
    rekognition                              | resource-groups
    resourcegroupstaggingapi                 | robomaker
    route53                                  | route53domains
    route53resolver                          | s3control
    s3outposts                               | sagemaker
    sagemaker-a2i-runtime                    | sagemaker-edge
    sagemaker-featurestore-runtime           | sagemaker-runtime
    savingsplans                             | schemas
    sdb                                      | secretsmanager
    securityhub                              | serverlessrepo
    service-quotas                           | servicecatalog
    servicecatalog-appregistry               | servicediscovery
    ses                                      | sesv2
    shield                                   | signer
    sms                                      | snowball
    sns                                      | sqs
    ssm                                      | ssm-contacts
    ssm-incidents                            | sso
    sso-admin                                | sso-oidc
    stepfunctions                            | storagegateway
    sts                                      | support
    swf                                      | synthetics
    textract                                 | timestream-query
    timestream-write                         | transcribe
    transfer                                 | translate
    waf                                      | waf-regional
    wafv2                                    | wellarchitected
    workdocs                                 | worklink
    workmail                                 | workmailmessageflow
    workspaces                               | xray
    s3api                                    | s3
    ddb                                      | configure
    deploy                                   | configservice
    opsworks-cm                              | history
    cli-dev                                  | help
    whoami                                   | create-assume-role
    running-instances                        | ebs-volumes
    amazon-linux-amis                        | list-sgs
    sg-rules                                 | tostring
    tostring-with-jq                         | authorize-my-ip
    get-group-id                             | authorize-my-ip-by-name
    public-ports                             | region
    find-access-key                          | docker-ecr-login
    myip                                     | allow-my-ip
    revoke-my-ip                             | allow-my-ip-all
    revoke-my-ip-all
    

    See http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-using.html


Encrypt AWS Credentials

Use my shell script to log into AWS by decrypting credentials stored securely (instead of in plain text).

One reason to encrypt credentials is because it’s wise to have a backup copy of the secret file, in an encrypted format, somewhere else. This enables you to retrieve secrets in case you lose your laptop.

This article covers use of AWS (Amazon Web Services) on MacOS. In the future I’ll be updating this article to cover use of Windows and other secret-handling utilities (Microsoft Azure, Google Cloud Platform, Hashicorp Vault, Akeyless, etc.).

After obtaining an AWS Access Key ID, AWS Secret Access Key for your account (described above), use the credentials on your local machine (laptop), install the AWS CLI locally. Although there is a “awscli” Homebrew formula, but it has been deprecated. So follow this doc to manually install a pkg file for awscli2:

Installing, updating, and uninstalling the AWS CLI version 2 on macOS

AWS CLI versions 1 and 2 use the same aws command name.

If you have both versions installed, your computer uses the…docs.aws.amazon.com

The installer automatically creates a symlink in a folder in your PATH which links to the main program in the installation folder you chose:

ls -al $(which aws)

If you see a response such as this:

-rwxr-xr-x  1 wilsonmar  staff  830 Jul 21 09:07 /usr/local/anaconda3/bin/aws
  1. Verify install:

    aws --version

    A sample response (at time of writing):

    aws-cli/1.20.3 Python/3.7.3 Darwin/18.7.0 botocore/1.21.3

    QUESTION: Why does the pkg say “1.20.3”?

  2. Amazon documentation says to run:

    aws configure

    That command prompts acceptance or override of default AWS ACCESS KEY ID, AWS SECRET ACCESS KEY, and region saved as a plain-text file at 

    ~/.aws/credentials

    Sample contents:

    [default]
    aws_access_key_id = ABCDEFGHIJKLMNOPQRST
    aws_secret_access_key = 123456786iJsvzQbkIlDiFtBh6DrPzIw8r7hVb35
    [py-ec2–1]
    aws_access_key_id = ABCDEFGHIJKLMNOPQRST
    aws_secret_access_key = 123456782Nwk156aPF0SxZ8KGY+RrhEbq3AIHUSS
    

    BTW Progress toward AWS providing a more secure approach is at https://github.com/aws/aws-sdk/issues/41

    Meanwhile, to avoid having credentials in clear text, store them in encrypted form:

  3. Install GPG locally using my instructions at

    https://wilsonmar.github.io/git-signing

  4. Generate encrypted file “credentials.gpg” from file “credentials”. See:

    https://wilsonmar.github.io/git-signing/#bonus-encrypting-whole-files-using-gpg

  5. To be able to retrieve secrets in case you lose your laptop, for backup make a copy of the secret file in encrypted format, somewhere else.

  6. Make a backup of GPG keys somewhere else (in a key vault) so you can decrypt. One way is to store your private key in a Yubikey USB chip you plug into your laptop.

  7. Using the GPG private key, encrypt the aws/credentials file to a new credentials.gpg file also in the same ~/.aws folder.

  8. Delete the file at ~/.aws/credentials

  9. Download my shell script:

    curl "https://raw.githubusercontent.com/wilsonmar/DevSecOps/main/bash/awslogin.sh" -o "awslogin.sh"

    NOTE: It works similar to https://github.com/99designs/aws-vault, but with no external dependencies (other than GPG). However, aws-vault supports several vaulting backends.

  10. Run the script to login based on the encrypted credential.gpg file:

    source ~/awslogin.sh

    Alternately, run the script to use the “susan” profile defined:

    source ~/awslogin.sh -p susan

    The script unencrypts the gpg file, invokes aws login, then removes the unencrypted file.

    BONUS: To parse variables from within an AWS credentials file, consider: GitHub - whereisaaron/get-aws-profile-bash: Fetch AWS keys and secrets from ~/.aws/credentials…

    This is a pure bash script that can parse and extract AWS credentials (key id and secret) from a ~/.aws/credentials…github.com

    If you use it, remember to clear out variables after usage, so they don’t linger in memory.

References

TODO: Put each AWS CLI command in a script at https://medium.com/circuitpeople/aws-cli-with-jq-and-bash-9d54e2eabaf1 by Lee Harding

More on Amazon

This is one of a series on Amazon:

More on DevOps

This is one of a series on DevOps:

  1. DevOps_2.0
  2. ci-cd (Continuous Integration and Continuous Delivery)
  3. User Stories for DevOps
  4. Enterprise Software)

  5. Git and GitHub vs File Archival
  6. Git Commands and Statuses
  7. Git Commit, Tag, Push
  8. Git Utilities
  9. Data Security GitHub
  10. GitHub API
  11. TFS vs. GitHub

  12. Choices for DevOps Technologies
  13. Pulumi Infrastructure as Code (IaC)
  14. Java DevOps Workflow
  15. Okta for SSO & MFA

  16. AWS DevOps (CodeCommit, CodePipeline, CodeDeploy)
  17. AWS server deployment options
  18. AWS Load Balancers

  19. Cloud services comparisons (across vendors)
  20. Cloud regions (across vendors)
  21. AWS Virtual Private Cloud

  22. Azure Cloud Onramp (Subscriptions, Portal GUI, CLI)
  23. Azure Certifications
  24. Azure Cloud

  25. Azure Cloud Powershell
  26. Bash Windows using Microsoft’s WSL (Windows Subsystem for Linux)
  27. Azure KSQL (Kusto Query Language) for Azure Monitor, etc.

  28. Azure Networking
  29. Azure Storage
  30. Azure Compute
  31. Azure Monitoring

  32. Digital Ocean
  33. Cloud Foundry

  34. Packer automation to build Vagrant images
  35. Terraform multi-cloud provisioning automation
  36. Hashicorp Vault and Consul to generate and hold secrets

  37. Powershell Ecosystem
  38. Powershell on MacOS
  39. Powershell Desired System Configuration

  40. Jenkins Server Setup
  41. Jenkins Plug-ins
  42. Jenkins Freestyle jobs
  43. Jenkins2 Pipeline jobs using Groovy code in Jenkinsfile

  44. Docker (Glossary, Ecosystem, Certification)
  45. Make Makefile for Docker
  46. Docker Setup and run Bash shell script
  47. Bash coding
  48. Docker Setup
  49. Dockerize apps
  50. Docker Registry

  51. Maven on MacOSX

  52. Ansible

  53. MySQL Setup

  54. SonarQube & SonarSource static code scan

  55. API Management Microsoft
  56. API Management Amazon

  57. Scenarios for load
  58. Chaos Engineering

More on Security

This is one of a series on Security in DevSecOps:

  1. Git Signing
  2. Hashicorp Vault

  3. WebGoat known insecure PHP app and vulnerability scanners
  4. Test for OWASP using ZAP on the Broken Web App

  5. Encrypt all the things

  6. AWS Security (certification exam)
  7. AWS IAM (Identity and Access Management)

  8. Cyber Security
  9. Security certifications