Wilson Mar bio photo

Wilson Mar

Hello!

Calendar YouTube Github

LinkedIn

Write dynamic Python, Java, .NET, NodeJs, or TypeScript code which are synth’d into Cloud Formation or Terraform static yaml

US (English)   Norsk (Norwegian)   Español (Spanish)   Français (French)   Deutsch (German)   Italiano   Português   Estonian   اَلْعَرَبِيَّةُ (Egypt Arabic)   Napali   中文 (简体) Chinese (Simplified)   日本語 Japanese   한국어 Korean

Overview

A procedural language to define infrastructure is useful to create abstractions to help manage complexity, such as a reusable infrastructure pattern composed of multiple resources and convenience methods.

Pulumi was created just for that. But rather than completely rewriting all Terraform HCL, CDKTF (CDK for Terraform) interoperates with existing Terraform providers and modules.

This article is a concise, yet deep technical description without generalizations that waste your time. NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.

Kumar’s Packt video course “Create Your First CDK Stack and Deploy It” on OReilly.com and Udemy.com uses node v12.12.0, Python 3.7.6, pip3 v19.3.1, VSCode, cdk v

  1. Create / navigate to a folder to hold github repos:
  2. Install the course author’s repo:

    git clone https://github.com/miztiik/my-first-cdk-project.git cd my-first-cdk-project

  3. If you’re on a Mac, edit .gitignore to add “.DS_folder”.
  4. Position to the branch:

    git checkout ec2_with_latest_ami_in_any_region

  5. Setup ~/.aws/credentials
  6. Install on your laptop:

    npm install -g aws-cdk  
    # If this is first time you are using cdk, run cdk bootstrap
    # cdk bootstrap
    pip3 install --upgrade aws-cdk.core
    cdk --version   # 2.43.1 (build c1ebb85)
    cdk init app --language python
    python3 -m venv .venv
    source .env/bin/activate
    # to install dependencies:
    pip3 install -r requirements.txt   
    cdk synth  # Synthesize the template
    cdk deploy
    

HashiCorp’s CDK for Terraform on AWS

The Cloud Development Kit for Terraform (CDKTF) generates JSON Terraform configuration from code in several programming languages: TypeScript (JavaScript), Python, Java, C#, and Go.

The HCL generated refer all Terraform providers, modules, and resources to define infrastructure.

  1. Learn hands-on how to install and use CDKTF

    Installing CDKTF requires Node (npm)

  2. Use NPM to install CDKTF in your glogal NPM folder:

    npm install --global cdktf-cli
    
    added 279 packages, and audited 331 packages in 17s
     
    26 packages are looking for funding
      run `npm fund` for details
     
    found 0 vulnerabilities
    npm notice 
    npm notice New minor version of npm available! 8.15.0 -> 8.17.0
    npm notice Changelog: https://github.com/npm/cli/releases/tag/v8.17.0
    npm notice Run npm install -g npm@8.17.0 to update!
    npm notice 
    
    Alternately:
      brew install cdktf
      ==> Installing dependencies for cdktf: terraform
      ==> Installing cdktf dependency: terraform
      Error: Cannot install terraform because conflicting formulae are installed.
        tfenv: because tfenv symlinks terraform binaries
       
      Please `brew unlink tfenv` before continuing.
       
      Unlinking removes a formula's symlinks from /opt/homebrew. You can
      link the formula again after the install finishes. You can --force this
      install, but the build may fail or cause obscure side effects in the
      resulting software.
      (
  3. Create a folder if this is new.
  4. Generate scaffold files:

    cdktf init
    
    Welcome to CDK for Terraform!
     
    By default, cdktf allows you to manage the state of your stacks using Terraform Cloud for free.
    cdktf will request an API token for app.terraform.io using your browser.
     
    If login is successful, cdktf will store the token in plain text in
    the following file for use by subsequent Terraform commands:
     /Users/wilsonmar/.terraform.d/credentials.tfrc.json
     
    Note: The local storage mode isn't recommended for storing the state of your stacks.
     
    ? Do you want to continue with Terraform Cloud remote state management? (Y/n) 
    
  5. Press control+C to cancel.

    Alternately,

    cdktf init -template=typescript --local
    
  6. For now, type “N”

    Alternately, if you answer “Yes” you’ll need to sign into an account on HashiCorp’s Terraform Cloud:

    https://app.terraform.io/session

  7. Answer the prompt with selecting “Typescript”

    ? What template do you want to use? typescript
     
    Initializing a project using the typescript template.
    ? Project Name test1
    ? Project Description A simple getting started project for cdktf.
    ? Do you want to start from a Terraform project? No
    ? Do you want to send crash reports to the CDKTF team? See https://www.terraform.io/cdktf/create-and-deploy/configuration-file#enable-crash-reporting-for-the-cli for more information No
     
    added 2 packages, and audited 54 packages in 1s
     
    5 packages are looking for funding
      run `npm fund` for details
     
    found 0 vulnerabilities
     
    added 298 packages, and audited 352 packages in 19s
     
    35 packages are looking for funding
      run `npm fund` for details
     
    found 0 vulnerabilities
    ========================================================================================================
     
      Your cdktf typescript project is ready!
     
      cat help                Print this message
     
      Compile:
     npm run get           Import/update Terraform providers and modules (you should check-in this directory)
     npm run compile       Compile typescript code to javascript (or "npm run watch")
     npm run watch         Watch for changes and compile typescript in the background
     npm run build         Compile typescript
     
      Synthesize:
     cdktf synth [stack]   Synthesize Terraform resources from stacks to cdktf.out/ (ready for 'terraform apply')
     
      Diff:
     cdktf diff [stack]    Perform a diff (terraform plan) for the given stack
     
      Deploy:
     cdktf deploy [stack]  Deploy the given stack
     
      Destroy:
     cdktf destroy [stack] Destroy the stack
     
      Test:
     npm run test        Runs unit tests (edit __tests__/main-test.ts to add your own tests)
     npm run test:watch  Watches the tests and reruns them on change
     
      Upgrades:
     npm run upgrade        Upgrade cdktf modules to latest version
     npm run upgrade:next   Upgrade cdktf modules to latest "@next" version (last commit)
     
     Use Providers:
     
      You can add prebuilt providers (if available) or locally generated ones using the add command:
       
      cdktf provider add "aws@~>3.0" null kreuzwerker/docker
     
      You can find all prebuilt providers on npm: https://www.npmjs.com/search?q=keywords:cdktf
      You can also install these providers directly through npm:
     
      npm install @cdktf/provider-aws
      npm install @cdktf/provider-google
      npm install @cdktf/provider-azurerm
      npm install @cdktf/provider-docker
      npm install @cdktf/provider-github
      npm install @cdktf/provider-null
     
      You can also build any module or provider locally. Learn more https://cdk.tf/modules-and-providers
     
    ========================================================================================================
    
  8. Tree

    ├── __tests__
    │  └── main-test.ts
    ├── cdktf.json
    ├── help
    ├── jest.config.js
    ├── main.ts
    ├── package-lock.json
    ├── package.json
    ├── setup.js
    └── tsconfig.json
    

https://cdk.tf/modules-and-providers

CDKTF reached its first GA release in August 2022.

  • https://www.terraform.io/cdktf

  • https://www.terraform.io/cdktf/concepts/cdktf-architecture

Use of TypeScript is recommended for those who plan to create and package custom constructs. TypeScript allows use of the CDKTF Constructs Package Generator to build and publish custom constructs in other languages.

MS-CRDS

CDK also provides A GitHub action to use with HashiCorp’s Terraform Cloud.

AWS

AWS defines their CDK (Cloud Development Kit) as:

“A multi-language software development framework for modeling cloud infrastructure as reusable components”

Programmatic coding enable reusableclass libraries (templates) which define entire sets of services using a single line:*

#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from '@aws-cdk/core';
import { TheBigFan Stack} from '../lib/the-big-fan-stack';
 
const app = new cdk.App();
new TheBigFanStack(app, 'TheBigFanStack');
   

The above code references “Constructs” which bundles infrastructure into reusable components deployed by a stack which “synth” out templates in CloudFormation or Terraform.

VIDEO: aws-cdk-flow-3118x880

aws-cdk-744x432

AWS and open-source contributors develop construct libraries for each of its many services (RDS, CloudFront, Kinesis, etc.). Reinvent 2019: How to make contributions ? https://www.youtube.com/watch?v=9As_ZIjUGmY

Template code contains:

  • if/else and for loops (control logic)
  • autocomplete
  • in-line documentation
  • compile-time warnings
  • modeling of cross-account and cross-region pipeline configurations

https://cdkworkshop.com provides instructions for a “Hello, world” Lambda function fronted by an API Gateway endpoint for users to call via an HTTP request.

https://aws.amazon.com/solutions/implementations/centralized-logging/

*

VIDEO: Apps withine a CDK Pipeline: aws-cdk-apps-1704x788

vs Proton

How is CDK different versus the AWS Proton app delivery service which monitors and update deployments?


Install AWS CDK CLI

  1. Have Homebrew installed for brew commands?

  2. The AWS CDK is not “cdk” in Homebrew:

    brew info aws-cdk

    aws-cdk: stable 1.111.0 (bottled)
    AWS Cloud Development Kit - framework for defining AWS infra as code
    https://github.com/aws/aws-cdk
    Not installed
    From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/aws-cdk.rb
    License: Apache-2.0
    ==> Dependencies
    Required: node ✘
    ==> Analytics
    install: 3,718 (30 days), 11,115 (90 days), 35,166 (365 days)
    install-on-request: 3,719 (30 days), 11,119 (90 days), 35,167 (365 days)
    build-error: 0 (30 days)
    
  3. No need to first install NodeJs

    brew install aws-cdk

    ==> Installing dependencies for aws-cdk: libuv, nghttp2 and node
    ==> Installing aws-cdk dependency: libuv
    ==> Pouring libuv--1.41.1.mojave.bottle.tar.gz
    🍺  /usr/local/Cellar/libuv/1.41.1: 49 files, 3.3MB
    ==> Installing aws-cdk dependency: nghttp2
    ==> Pouring nghttp2--1.44.0.mojave.bottle.tar.gz
    🍺  /usr/local/Cellar/nghttp2/1.44.0: 23 files, 2.6MB
    ==> Installing aws-cdk dependency: node
    ==> Pouring node--16.5.0.mojave.bottle.tar.gz
    🍺  /usr/local/Cellar/node/16.5.0: 2,467 files, 48.3MB
    ==> Installing aws-cdk
    ==> Pouring aws-cdk--1.114.0.mojave.bottle.tar.gz
    🍺  /usr/local/Cellar/aws-cdk/1.114.0: 8,201 files, 146.5MB
    
  4. How much disk space?

    df -h $(pwd)
  5. See if it works:

    cdk version
    (node:8758) ExperimentalWarning: The fs.promises API is experimental
    1.114.0 (build 7e41b6b)
    

    Alternately, get a menu:

    cdk

    (node:8732) ExperimentalWarning: The fs.promises API is experimental
    Usage: cdk -a <cdk-app> COMMAND
     
    Commands:
      cdk list [STACKS..]             Lists all stacks in the app      [aliases: ls]
      cdk synthesize [STACKS..]       Synthesizes and prints the CloudFormation
                                   template for this stack       [aliases: synth]
      cdk bootstrap [ENVIRONMENTS..]  Deploys the CDK toolkit stack into an AWS
                                   environment
      cdk deploy [STACKS..]           Deploys the stack(s) named STACKS into your
                                   AWS account
      cdk destroy [STACKS..]          Destroy the stack(s) named STACKS
      cdk diff [STACKS..]             Compares the specified stack with the deployed
                                   stack or a local template file, and returns
                                   with status 1 if any difference is found
      cdk metadata [STACK]            Returns all metadata associated with this
                                   stack
      cdk init [TEMPLATE]             Create a new, empty CDK project from a
                                   template.
      cdk context                     Manage cached context values
      cdk docs                        Opens the reference documentation in a browser
                                                                   [aliases: doc]
      cdk doctor                      Check your set-up for potential problems
     
    Options:
      -a, --app                 REQUIRED: command-line for executing your app or a
                             cloud assembly directory (e.g. "node bin/my-app.js")
                                                                         [string]
      -c, --context             Add contextual string parameter (KEY=VALUE)  [array]
      -p, --plugin              Name or path of a node package that extend the CDK
                             features. Can be specified multiple times    [array]
       --trace               Print trace for stack warnings             [boolean]
       --strict              Do not construct stacks with warnings      [boolean]
       --lookups             Perform context lookups (synthesis fails if this is
                             disabled and context lookups need to be performed)
                                                        [boolean] [default: true]
       --ignore-errors       Ignores synthesis errors, which will likely produce
                             an invalid output         [boolean] [default: false]
      -j, --json                Use JSON output instead of YAML when templates are
                             printed to STDOUT         [boolean] [default: false]
      -v, --verbose             Show debug logs (specify multiple times to increase
                             verbosity)                  [count] [default: false]
       --debug               Enable emission of additional debugging information,
                             such as creation stack traces of tokens
                                                       [boolean] [default: false]
       --profile             Use the indicated AWS profile as the default
                             environment                                 [string]
       --proxy               Use the indicated proxy. Will read from HTTPS_PROXY
                             environment variable if not specified       [string]
       --ca-bundle-path      Path to CA certificate to use when validating HTTPS
                             requests. Will read from AWS_CA_BUNDLE environment
                             variable if not specified                   [string]
      -i, --ec2creds            Force trying to fetch EC2 instance credentials.
                             Default: guess EC2 instance status         [boolean]
       --version-reporting   Include the "AWS::CDK::Metadata" resource in
                             synthesized templates (enabled by default) [boolean]
       --path-metadata       Include "aws:cdk:path" CloudFormation metadata for
                             each resource (enabled by default)
                                                        [boolean] [default: true]
       --asset-metadata      Include "aws:asset:*" CloudFormation metadata for
                             resources that uses assets (enabled by default)
                                                        [boolean] [default: true]
      -r, --role-arn            ARN of Role to use when invoking CloudFormation
                                                                         [string]
       --toolkit-stack-name  The name of the CDK toolkit stack           [string]
       --staging             Copy assets to the output directory (use
                             --no-staging to disable, needed for local debugging
                             the source files with SAM CLI)
                                                        [boolean] [default: true]
      -o, --output              Emits the synthesized cloud assembly into a
                             directory (default: cdk.out)                [string]
       --no-color            Removes colors and other style from console output
                                                       [boolean] [default: false]
       --fail                Fail with exit code 1 in case of diff
                                                       [boolean] [default: false]
       --version             Show version number                        [boolean]
      -h, --help                Show help                                  [boolean]
    
    

If your app has a single stack, there is no need to specify the stack name   If one of cdk.json or ~/.cdk.json exists, options specified there will be used as defaults. Settings in cdk.json take precedence. </pre>

### Node version alignment

  1. If you get an error such as this when you run:

    /usr/local/Cellar/aws-cdk/1.114.0/libexec/lib/node_modules/aws-cdk/node_modules/yargs-parser/build/index.cjs:1007
         throw Error(`yargs parser supports a minimum Node.js version of ${minNodeVersion}. Read our version support policy: https://github.com/yargs/yargs-parser#supported-nodejs-versions`);
               ^
    Error: yargs parser supports a minimum Node.js version of 10. Read our version support policy: https://github.com/yargs/yargs-parser#supported-nodejs-versions
    
  2. What version of NodeJs do you have installed?

    node -v

    If you have a version less that (10), such as “v9.11.1”, upgrade Node to the version specified (v10).

  3. To enable multiple versions of Node to be installed simultaneously, install NVM:

    brew install nvm
  4. Identify an LTS version to install:

    nvm ls
         v8.11.1
         v8.11.4
    ->      v9.11.1
          system
    default -> node (-> v9.11.1)
    node -> stable (-> v9.11.1) (default)
    stable -> 9.11 (-> v9.11.1) (default)
    iojs -> N/A (default)
    lts/* -> lts/dubnium (-> N/A)
    lts/argon -> v4.9.1 (-> N/A)
    lts/boron -> v6.17.1 (-> N/A)
    lts/carbon -> v8.16.0 (-> N/A)
    lts/dubnium -> v10.16.0 (-> N/A)
    
  5. Based on the above:

    nvm install 10.16.0
    Now using node v10.16.0 (npm v6.9.0)

https://wilsonmar.github.io/node-osx-install/

NPM Code

  1. In the https://docs.aws.amazon.com/cdk/latest/guide/cli.html install a fall-back global version:

    npm install -g aws-cdk # install latest version

    PROTIP: Be prepared to work with multiple versions of the AWS CDK installed. Install a matching version of the AWS CDK Toolkit in individual CDK projects (omitting -g from the npm install command).

  2. use npx cdk to invoke it; this will run the local version if one exists, falling back to a global version if not.

However, there is also:

Initialize language folder

cdk init --language python

cdk deploy

cdk synth stack1

CDKGoat

https://github.com/bridgecrewio/cdkgoat is a “Vulnerable by Design” AWS CDK repository. CdkGoat is a learning and training project that demonstrates how common configuration errors can find their way into production cloud environments.

It also shows how Bridgecrew can be used with the AWS CDK to provide CloudFormation template vulnerability scanning at build time, even though no CloudFormation templates exist in the source repository.

Rockstars

Elad Ben-Israel (@emeshbi)

Jason Fulghum (CDK team Sr Mgr Dev Tools)

Jun Fisk at CloudAcademy.com:

VIDEO: “Multi-Stack Deployments with CDK for Terraform” discusses https://github.com/hashicorp/cdktf-multistack-serverless-example used by the end to end example in TypeScript for a serverless web application hosted on AWS and deployed with the CDK for Terraform.

  • Frontend: React, Create React App, statically hosted via AWS S3 + CloudFront

  • Backend API: AWS Lambda + API Gateway + DynamoDB

Resources

  • https://www.youtube.com/watch?v=1ps0Wh19MHQ

  • https://www.youtube.com/watch?v=Q1FcifrDocE

  • https://www.youtube.com/watch?v=Cf3yJv3klsg

  • https://www.youtube.com/watch?v=ZWCvNFUN-sU

  • https://www.hashicorp.com/blog/cdk-for-terraform-enabling-python-and-typescript-support

  • AWS Cloud Development Kit - From Biginner to … by Valaxy Technologies on OReilly.com with code at https://github.com/