Wilson Mar bio photo

Wilson Mar

Hello!

Email me Calendar Skype call

LinkedIn Twitter Gitter Instagram Youtube

Github Stackoverflow Pinterest

How to store and send files securely using AWS KMS (Key Management Service)

US (English)   Español (Spanish)   Français (French)   Deutsch (German)   Italiano   Português   Estonian   اَلْعَرَبِيَّةُ (Egypt Arabic)   中文 (简体) Chinese (Simplified)   日本語 Japanese   한국어 Korean

Overview

For storage “at rest” and while “in transit” over telecommunication lines, we encrypt clear (plain) text into unreadable (scrambled) cyphertext.

NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.

This tutorial aims to organize deep-dive insights and advice based on the combination of advice from several sources. Unlike others which first numb you with theory then have you mindlessly follow steps, I aim to provide commentary after each action.

Each cloud service (AWS with Azure with GCP, etc.) has its own mechanisms.

Personas and their tasks

Let’s pretend there are these users:

  • Mary, the Key Administrator
  • Alice, a valued user
  • Snape, a user who should no longer have access

AWS KMS

The AWS KMS (Key Management Service) manages CMKs (Customer Master Keys) for use with most other AWS services.

PROTIP: AWS is trying to replace the term Customer Master Key (CMK) with “KMS key”. Its concepts have not changed. To prevent breaking changes, KMS is keeping some variations of this term.

REMEMBER: KMS itself can only encrypt a maximum of 4 KB. So Data Keys are used to encrypt larger objects.

https://github.com/cipherstash/terraform-provider-kms/blob/main/docs/resources/kms_data_key_without_plaintext.md

Data Keys

To encrypt objects larger than 4KB,

REMEMBER: AWS KMS does not store Data Keys.

PROTIP: Use a separate Data key for each different dataset, so that if one key falls into the wrong hands, your whole system won’t be completely compromised. This is a “Zero Trust” approach.

Encryption can occur on the client or server, using several mechanisms:

REMEMBER:
SSE = Server-Side Encryption
CSE = Client-Side Encryption

  • … with S3 Managed Keys (SSE-S3)

  • … with KMS Managed Keys (SSE-KMS)
  • … with KMS Managed Keys (CSE-KMS)

  • … with Customer Provided keys (SSE-C)
  • … with Customer Provided Keys (CSE-C)

Hands-on

Instructions below are an enhanced version of thetext tutorial:

  1. Create a CMK Encryption Key using GUI AWS Management Console
  2. Create a CMK Encryption Key using GUI AWS CLI
  3. Create a CMK Encryption Key using Terraform
  4. Create a CMK Encryption Key using a Python program calling the KMS API

  5. Encrypt text using CLI

  6. Create an S3 bucket with CloudTrail logging functions
  7. Use an encryption key to encrypt data stored in a S3 bucket

  8. Monitor encryption key usage using CloudTrail
  9. Manage encryption keys for users and roles

Create a CMK (KMS Key) using GUI AWS Management Console

VIDEO

  1. Use an internet browser to get on the AWS Management Console, such as:

    https://us-east-2.console.aws.amazon.com/console/home?region=us-east-2#

  2. Select Key Management Service (KMS) from among AWS services:

    aws-kms-svc-802x308

  3. Upon entry, “Customer-managed keys” is auto-selected from the left menu:

    • AWS-managed keys
    • Customer-managed keys (symmetric or asymmetric)
    • Customer key stores

    About “AWS-managed keys”: AWS creates a Default master key that protects the data of each service (such as Cloud9) when no other key is defined.

  4. Click “Create Key” (in orange) for the “Configure keys” page.

  5. Select a region.

    Private CMK (Customer Master Keys) are created in KMS and remain there.

    REMEMBER: Internally, AWS KMS uses a HSM (Hardware Security Module) to store keys.

    Asymmetric encryption is not available in some regions (such as China).

    REMEMBER: A CMK (KMS Key) never leaves the HSM in the region where it was created.

    KMS keys were once specific to a region. But they recently became multi-region for client-side encryption in:

    • AWS Encryption SDK
    • AWS S3 Encryption Client, and
    • AWS DynamoDB Encryption Client.

  6. Click “Help me choose” for a lesson:

    REMEMBER: Symmetric keys are like a password, a single key is used to both encrypt and decrypt. It is fast and efficient. But they cannot be used to sign and verify.

    REMEMBER: Asymmetric keys are public/private key pairs. Key pairs generated using the RSA algorithm are used to encrypt/decrypt or sign/verify operations. Key pairs generated using using ECC (Elliptic curve) algorithms are used to only sign and verify.

  7. Click Symmetric.

  8. Click “Advanced options” to view “Key material origin”. Read the KMS docs at

    https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html

    Advanced Options: Key material origins: KMS, External, Customer key store (CloudHSM):

    • KMS are validated to FIPS 140-2 level 2
    • CloudHSM are validated to FIPS 140-2 level 3, keys and hardware exclusive to customer, either symmetric or asymmetric

    WARNING: Using AWS Cloud HSM cluster incurs an hourly fee. And AWS has no visibility or access to encryption keys in HSM.

  9. Click “Next” for the “Add labels” page.
  10. Type in an Alias and Description.

    PROTIP: Define aliases to differentiate keys within the account.

    PROTIP: Establish a convention for naming keys for all departments, projects, etc.

    Each key has an Alias and Key ID, which are GUIDs with dashes, and enabled.

  11. Add Tags?
  12. Click “Next” for the “Define key administrative permissions” page.

    Root and Administrator

  13. Select the Key Administrators already defined:

    To ensure that KMS root account has access, its Key Policy allows all actions to all resources:

    {
     "Sid": "Enable IAM User Permissions",
     "Effect": "Allow",
     "Principal": {"AWS": "arn:aws:iam:123456789123:root},
     "Action": "kms:*",
     "Resource": "*"
    }
    

    When using the AWS Management Console GUI, define the Key Administrator as Principals who administer the CMK, and can perform all but encryption functionality: Create, Describe, Enable, List, Put, Update, Revoke, Disable, Get, Delete, TagResource, UntagResource, ScheduleKeyDeletion, CancelKeyDeletion.

    PROTIP: Enable the Key Administrator to be the only one with the ability to Delete, to ensure against other accounts from making accidental or malicious deletions which make data unreadable. However, the Key Administrator should be easily reachable and quickly responsive to valid requests for deletion when needed.

  14. Leave default-checked “Allow key administrators to delete this key”.
  15. Click “Next” for the “Define key usage permissions” page.
  16. Select from “This Account” list your account.
  17. Click “Next” for the “Review and edit key polcy” page. A sample:

    {
       "Id": "key-consolepolicy-3",
       "Version": "2012-10-17",
       "Statement": [
          {
             "Sid": "Enable IAM User Permissions",
             "Effect": "Allow",
             "Principal": {
                "AWS": "arn:aws:iam::11111111:root"
             },
             "Action": "kms",
             "Resource": "*"
          },
    ...
    
  18. Click “Finish” to see the Alias name you created.

Create a CMK (KMS Data Key) using CLI

VIDEO

  1. To generate a CMK using the Advanced Encryption Standard:

    aws kms generate-data-key --key-id alias/demo1 --key-spec AES_256 \
    --region us-east-2 > keys.txt
    

    The command returns two versions of Data Keys in the file specified:

    • Plaintext
    • KeyId “arn:aws:kms:us-east-2:11111:key/24234-1fac-2222-3333-44444444”,
    • CiphertextBlob

    The above strings are in Base64 encoding.


Encrypt text using CLI and CMK

VIDEO

  1. Verify version installed:

    aws --version
  2. To encrypt a short sentence using the AWS CLI:

    aws kms encrypt --plaintext "My little secret" --key-id alias/DemoKey \
    --profile Alice
    

    PROTIP: KMS operations (commands) within AWS CLI are arranged by topic here:

  • update-primary-region

  • tag-resource, list-resource-tags, untag-resource

  • create-custom-key-store, connect-custom-key-store, describe-custom-key-stores, update-custom-key-store, disconnect-custom-key-store, delete-custom-key-store

  • get-key-policy, list-key-policies, put-key-policy
  • Grants: create-grant, list-grants, revoke-grant, list-retirable-grants, retire-grant
  • create-key, describe-key, list-keys, replicate-key, enable-key, disable-key, schedule-key-deletion
  • enable-key-rotation, get-key-rotation-status, disable-key-rotation

  • import-key-material, delete-imported-key-material
  • generate-data-key, generate-data-key-pair, generate-data-key-without-plaintext, generate-data-key-pair-without-plaintext

  • encrypt, decrypt, re-encrypt,
  • sign, verify
  • generate-random, GenerateDataKey, GenerateDataKeyWithoutPlaintext
  • get-public-key,
  • update-key-description
  • get-parameters-for-import

  • create-alias, list-aliases, update-alias, delete-alias
  • cancel-key-deletion


Create a CMK (KMS Key) using Terraform

Links to Terraform IaC YAML:

Resource Data sources
Define/import aws_kms_alias
Define/import aws_kms_ciphertext
Define/import aws_kms_key
aws_kms_external_key -
- aws_kms_public_key
- aws_kms_secret
- aws_kms_secrets
aws_kms_grant -
aws_kms_replica_key -
aws_kms_replica_external_key -

Envelop Encryption

References at CloudAcademy.com:

LAB: Encrypting S3 objects using SSE-KMS

When Customer keys are used, AWS KMS uses what is known as “envelope encryption”. An application’s cleartext data (of any size) is encrypted using two keys: the plaintext CMK and the Data Encryption Key (DEK) created from plaintext CMK (Customer-supplied Master Key) using the FIPS 140-2 validated cryptographic module.

Outside AWS, OpenSSL or AWS Encryption SDK is used to encrypt data with Data Keys.

Anyway, S3 uses the plaintext CMK to encrypt, then store each encrypted object with the encrypted CMK. The plaintext CMK is deleted from memory immediately after use.

When a user requests an encrypted object from S3, S3 makes a request to KMS with the encrypted CMK stored with the object. From that, KMS generates a plaintext DEK for return to S3 for use to decrypt.

Key policies

Access to each CMK is governed by key policies for that CMK. At least one Key Policy is required for all CMKs. Much like IAM policies, Key policies define (in JSON) who can use and access a key in KMS. A template of a Key Policy:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "statement identifier",
    "Effect": "effect",
    "Principal": "principal",
    "Action": "action",
    "Resource": "resource",
    "Condition": {"condition operator": {"condition context key": "context key value"} }    
  }]
}

An example of a Key Policy with IAM Policies:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": [
      "kms:Encrypt",
      "kms:Decrypt"
    ]
    "Resource": [
      "arn:aws:kms:us-east-1:123456789123:key/1234abcd-12ab-34cd-56ef-1234567890ab",
      "arn:aws:kms:eu-west-1:123456789123:key/0987dcba-09fe-87dc-65ba-ab0987654321"
    ]
  }
}

Grants

Grants allow delegation of access to another principal, such as a service integrated with KMS or another user.

Grants eliminates the possibility of anyone using the permission kms:PutKeyPolicy.

Grants are created using the AWS KMS APIs.

{
    "Sid": "Allow attachment of persistent resources",
    "Effect": "Allow",
    "Principal": [
      "AWS": [
         "arn:aws:iam::456789123345:user/BigCorp},
      ]
  ],
  "Action": [
     "kms:CreateGrant",
     "kms:ListGrants",
     "kms:RevokeGrant",
  ]
  "Resource": "*",
  "Condition": [
    "Bool": {
      "kms:GrantIsForAWSResource": true
    }
  ]
}

A GrantToken and GrantID are issued.

Using Key Policies with IAM:

Using Key Policies with Grants:

Logging in CloudTrail

AWS CloudTrail logs each API action within AWS, including actions using KMS. Audits of CloudTrail logs would reveal when KMS encryption keys are used, for what reason, and by whom.

AWS Tutorials about KMS


Generate secret keys using AWS KMS


Encrypt AWS Network in transit

GUI demo [3:05] AWS Networking Deep Dive: Virtual Private Cloud (VPC) 8 Aug 2019 by Ben Piper

Transport layer: Amazon S2N (Signal-to-Noise), AWS-managed VPN, AWS-client VPN, AWS VPN cloud hub, third-party VPN tunnel.

To create VPC : Customer Gateway : VPN Site-to-site IPSEC

  1. In AWS Console, select VPC in search bar or link.
  2. Customer Gateway menu, Create Customer Gateway
  3. Type Name, Select Routing, IP Address of firewall in front of network,

    You can leave blank Certificate ARN, Device.

  4. Virtual Private Gateways on left menu. Create VPC Gateway. Type name.

    “IPSec.1”

  5. Attach (explictly) in Actions drop-down.

  6. Site-to-Site VPN Connection in left menu to Create VPN Connection.
  7. Type Name tag, Virtual Private Gateway
  8. Add Another Rule. Type IP prefix (“192.68”)

    Customer Gateway and Tunnel Options can be left as is.

  9. Download configuration. In pop-up select Vendor (“Openwan”), Platform, Software.
  10. Upload configuration file to firewall.

  11. Route Tables in left menu to Edit Routes.
  12. Type Destination IP & Target of on-premise network. Add route.

Generate secret key using AWS KMS

AWS KMS uses the AWS Encryption SDK of cryptographic algorithms.

VIDEO: AWS re_Infoce 2019: Achieving Security Goals with AWS CloudHSM

CMK + Encryption algorithm yields the Plaintext key and Encrypted key.

Plaintext key + Data are fed into the Encryption algorithm yields Encrypted data.

Encrypted key + CMK fed into Decryption algorithm yields Plaintext key.

Videos

VIDEO: AWS re_Infoce 2019: How Encryption Works in AWS

VIDEO: AWS re_Infoce 2019

Build and Monitor Security into Your Golden AMI Pipeline

Introduction to AWS Services by the AWS Training Center Jun 9, 2019 [38:53] is highly rated introduction

More on Security

This is one of a series on Security in DevSecOps:

  1. SOC2
  2. CAIQ (Consensus Assessment Initiative Questionnaire) by cloud vendors

  3. Git Signing
  4. Hashicorp Vault

  5. WebGoat known insecure PHP app and vulnerability scanners
  6. Test for OWASP using ZAP on the Broken Web App

  7. Encrypt all the things

  8. AWS Security (certification exam)
  9. AWS IAM (Identity and Access Management)

  10. Cyber Security
  11. Security certifications