Write dynamic Python, Java, .NET, NodeJs, or TypeScript code which are synth’d into Cloud Formation or Terraform static yaml
Overview
A procedural language to define infrastructure is useful to create abstractions to help manage complexity, such as a reusable infrastructure pattern composed of multiple resources and convenience methods.
Pulumi was created just for that. But rather than completely rewriting all Terraform HCL, CDKTF (CDK for Terraform) interoperates with existing Terraform providers and modules.
This article is a concise, yet deep technical description without generalizations that waste your time. NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.
Kumar’s Packt video course “Create Your First CDK Stack and Deploy It” on OReilly.com and Udemy.com uses node v12.12.0, Python 3.7.6, pip3 v19.3.1, VSCode, cdk v
- Create / navigate to a folder to hold github repos:
-
Install the course author’s repo:
git clone https://github.com/miztiik/my-first-cdk-project.git cd my-first-cdk-project
- If you’re on a Mac, edit .gitignore to add “.DS_folder”.
-
Position to the branch:
git checkout ec2_with_latest_ami_in_any_region
- Setup ~/.aws/credentials
-
Install on your laptop:
npm install -g aws-cdk # If this is first time you are using cdk, run cdk bootstrap # cdk bootstrap pip3 install --upgrade aws-cdk.core cdk --version # 2.43.1 (build c1ebb85) cdk init app --language python python3 -m venv .venv source .env/bin/activate # to install dependencies: pip3 install -r requirements.txt cdk synth # Synthesize the template cdk deploy
HashiCorp’s CDK for Terraform on AWS
The Cloud Development Kit for Terraform (CDKTF) generates JSON Terraform configuration from code in several programming languages: TypeScript (JavaScript), Python, Java, C#, and Go.
The HCL generated refer all Terraform providers, modules, and resources to define infrastructure.
-
Learn hands-on how to install and use CDKTF
Installing CDKTF requires Node (npm)
-
Use NPM to install CDKTF in your glogal NPM folder:
npm install --global cdktf-cli
added 279 packages, and audited 331 packages in 17s 26 packages are looking for funding run `npm fund` for details found 0 vulnerabilities npm notice npm notice New minor version of npm available! 8.15.0 -> 8.17.0 npm notice Changelog: https://github.com/npm/cli/releases/tag/v8.17.0 npm notice Run npm install -g npm@8.17.0 to update! npm notice
Alternately:brew install cdktf
==> Installing dependencies for cdktf: terraform ==> Installing cdktf dependency: terraform Error: Cannot install terraform because conflicting formulae are installed. tfenv: because tfenv symlinks terraform binaries Please `brew unlink tfenv` before continuing. Unlinking removes a formula's symlinks from /opt/homebrew. You can link the formula again after the install finishes. You can --force this install, but the build may fail or cause obscure side effects in the resulting software. (
- Create a folder if this is new.
-
Generate scaffold files:
cdktf init
Welcome to CDK for Terraform! By default, cdktf allows you to manage the state of your stacks using Terraform Cloud for free. cdktf will request an API token for app.terraform.io using your browser. If login is successful, cdktf will store the token in plain text in the following file for use by subsequent Terraform commands: /Users/wilsonmar/.terraform.d/credentials.tfrc.json Note: The local storage mode isn't recommended for storing the state of your stacks. ? Do you want to continue with Terraform Cloud remote state management? (Y/n)
-
Press control+C to cancel.
Alternately,
cdktf init -template=typescript --local
-
For now, type “N”
Alternately, if you answer “Yes” you’ll need to sign into an account on HashiCorp’s Terraform Cloud:
https://app.terraform.io/session
-
Answer the prompt with selecting “Typescript”
? What template do you want to use? typescript Initializing a project using the typescript template. ? Project Name test1 ? Project Description A simple getting started project for cdktf. ? Do you want to start from a Terraform project? No ? Do you want to send crash reports to the CDKTF team? See https://www.terraform.io/cdktf/create-and-deploy/configuration-file#enable-crash-reporting-for-the-cli for more information No added 2 packages, and audited 54 packages in 1s 5 packages are looking for funding run `npm fund` for details found 0 vulnerabilities added 298 packages, and audited 352 packages in 19s 35 packages are looking for funding run `npm fund` for details found 0 vulnerabilities ======================================================================================================== Your cdktf typescript project is ready! cat help Print this message Compile: npm run get Import/update Terraform providers and modules (you should check-in this directory) npm run compile Compile typescript code to javascript (or "npm run watch") npm run watch Watch for changes and compile typescript in the background npm run build Compile typescript Synthesize: cdktf synth [stack] Synthesize Terraform resources from stacks to cdktf.out/ (ready for 'terraform apply') Diff: cdktf diff [stack] Perform a diff (terraform plan) for the given stack Deploy: cdktf deploy [stack] Deploy the given stack Destroy: cdktf destroy [stack] Destroy the stack Test: npm run test Runs unit tests (edit __tests__/main-test.ts to add your own tests) npm run test:watch Watches the tests and reruns them on change Upgrades: npm run upgrade Upgrade cdktf modules to latest version npm run upgrade:next Upgrade cdktf modules to latest "@next" version (last commit) Use Providers: You can add prebuilt providers (if available) or locally generated ones using the add command: cdktf provider add "aws@~>3.0" null kreuzwerker/docker You can find all prebuilt providers on npm: https://www.npmjs.com/search?q=keywords:cdktf You can also install these providers directly through npm: npm install @cdktf/provider-aws npm install @cdktf/provider-google npm install @cdktf/provider-azurerm npm install @cdktf/provider-docker npm install @cdktf/provider-github npm install @cdktf/provider-null You can also build any module or provider locally. Learn more https://cdk.tf/modules-and-providers ========================================================================================================
-
Tree
├── __tests__ │ └── main-test.ts ├── cdktf.json ├── help ├── jest.config.js ├── main.ts ├── package-lock.json ├── package.json ├── setup.js └── tsconfig.json
https://cdk.tf/modules-and-providers
CDKTF reached its first GA release in August 2022.
-
https://www.terraform.io/cdktf
-
https://www.terraform.io/cdktf/concepts/cdktf-architecture
Use of TypeScript is recommended for those who plan to create and package custom constructs. TypeScript allows use of the CDKTF Constructs Package Generator to build and publish custom constructs in other languages.
MS-CRDS
CDK also provides A GitHub action to use with HashiCorp’s Terraform Cloud.
AWS
- github.com/aws/aws-cdk is where the AWS CDK is open-sourced.
-
github.com/awsdocs/aws-doc-sdk-examples is referenced in AWS documentation, AWS SDK Developer Guides, etc.
- github.com/aws-samples/aws-cdk-examples
- Stack Overflow
- gitter.im/awslabs/aws-cdk
AWS defines their CDK (Cloud Development Kit) as:
“A multi-language software development framework for modeling cloud infrastructure as reusable components”
Programmatic coding enable reusableclass libraries (templates) which define entire sets of services using a single line:*
#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from '@aws-cdk/core';
import { TheBigFan Stack} from '../lib/the-big-fan-stack';
const app = new cdk.App();
new TheBigFanStack(app, 'TheBigFanStack');
The above code references “Constructs” which bundles infrastructure into reusable components deployed by a stack which “synth” out templates in CloudFormation or Terraform.
AWS and open-source contributors develop construct libraries for each of its many services (RDS, CloudFront, Kinesis, etc.). Reinvent 2019: How to make contributions ? https://www.youtube.com/watch?v=9As_ZIjUGmY
Template code contains:
- if/else and for loops (control logic)
- autocomplete
- in-line documentation
- compile-time warnings
- modeling of cross-account and cross-region pipeline configurations
https://cdkworkshop.com provides instructions for a “Hello, world” Lambda function fronted by an API Gateway endpoint for users to call via an HTTP request.
https://aws.amazon.com/solutions/implementations/centralized-logging/
VIDEO: Apps withine a CDK Pipeline:
vs Proton
How is CDK different versus the AWS Proton app delivery service which monitors and update deployments?
-
AWS re:Invent 2020: AWS Proton: Automating infrastructure provisioning & code deployments Feb 5, 2021 by Rafa Alvarez
-
Unboxing AWS Proton! Dec 1, 2020 by Containers from the Couch
Install AWS CDK CLI
-
Have Homebrew installed for brew commands?
-
The AWS CDK is not “cdk” in Homebrew:
brew info aws-cdk
aws-cdk: stable 1.111.0 (bottled) AWS Cloud Development Kit - framework for defining AWS infra as code https://github.com/aws/aws-cdk Not installed From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/aws-cdk.rb License: Apache-2.0 ==> Dependencies Required: node ✘ ==> Analytics install: 3,718 (30 days), 11,115 (90 days), 35,166 (365 days) install-on-request: 3,719 (30 days), 11,119 (90 days), 35,167 (365 days) build-error: 0 (30 days)
-
No need to first install NodeJs
brew install aws-cdk
==> Installing dependencies for aws-cdk: libuv, nghttp2 and node ==> Installing aws-cdk dependency: libuv ==> Pouring libuv--1.41.1.mojave.bottle.tar.gz 🍺 /usr/local/Cellar/libuv/1.41.1: 49 files, 3.3MB ==> Installing aws-cdk dependency: nghttp2 ==> Pouring nghttp2--1.44.0.mojave.bottle.tar.gz 🍺 /usr/local/Cellar/nghttp2/1.44.0: 23 files, 2.6MB ==> Installing aws-cdk dependency: node ==> Pouring node--16.5.0.mojave.bottle.tar.gz 🍺 /usr/local/Cellar/node/16.5.0: 2,467 files, 48.3MB ==> Installing aws-cdk ==> Pouring aws-cdk--1.114.0.mojave.bottle.tar.gz 🍺 /usr/local/Cellar/aws-cdk/1.114.0: 8,201 files, 146.5MB
-
How much disk space?
df -h $(pwd)
-
See if it works:
cdk version
(node:8758) ExperimentalWarning: The fs.promises API is experimental 1.114.0 (build 7e41b6b)
Alternately, get a menu:
cdk
(node:8732) ExperimentalWarning: The fs.promises API is experimental Usage: cdk -a <cdk-app> COMMAND Commands: cdk list [STACKS..] Lists all stacks in the app [aliases: ls] cdk synthesize [STACKS..] Synthesizes and prints the CloudFormation template for this stack [aliases: synth] cdk bootstrap [ENVIRONMENTS..] Deploys the CDK toolkit stack into an AWS environment cdk deploy [STACKS..] Deploys the stack(s) named STACKS into your AWS account cdk destroy [STACKS..] Destroy the stack(s) named STACKS cdk diff [STACKS..] Compares the specified stack with the deployed stack or a local template file, and returns with status 1 if any difference is found cdk metadata [STACK] Returns all metadata associated with this stack cdk init [TEMPLATE] Create a new, empty CDK project from a template. cdk context Manage cached context values cdk docs Opens the reference documentation in a browser [aliases: doc] cdk doctor Check your set-up for potential problems Options: -a, --app REQUIRED: command-line for executing your app or a cloud assembly directory (e.g. "node bin/my-app.js") [string] -c, --context Add contextual string parameter (KEY=VALUE) [array] -p, --plugin Name or path of a node package that extend the CDK features. Can be specified multiple times [array] --trace Print trace for stack warnings [boolean] --strict Do not construct stacks with warnings [boolean] --lookups Perform context lookups (synthesis fails if this is disabled and context lookups need to be performed) [boolean] [default: true] --ignore-errors Ignores synthesis errors, which will likely produce an invalid output [boolean] [default: false] -j, --json Use JSON output instead of YAML when templates are printed to STDOUT [boolean] [default: false] -v, --verbose Show debug logs (specify multiple times to increase verbosity) [count] [default: false] --debug Enable emission of additional debugging information, such as creation stack traces of tokens [boolean] [default: false] --profile Use the indicated AWS profile as the default environment [string] --proxy Use the indicated proxy. Will read from HTTPS_PROXY environment variable if not specified [string] --ca-bundle-path Path to CA certificate to use when validating HTTPS requests. Will read from AWS_CA_BUNDLE environment variable if not specified [string] -i, --ec2creds Force trying to fetch EC2 instance credentials. Default: guess EC2 instance status [boolean] --version-reporting Include the "AWS::CDK::Metadata" resource in synthesized templates (enabled by default) [boolean] --path-metadata Include "aws:cdk:path" CloudFormation metadata for each resource (enabled by default) [boolean] [default: true] --asset-metadata Include "aws:asset:*" CloudFormation metadata for resources that uses assets (enabled by default) [boolean] [default: true] -r, --role-arn ARN of Role to use when invoking CloudFormation [string] --toolkit-stack-name The name of the CDK toolkit stack [string] --staging Copy assets to the output directory (use --no-staging to disable, needed for local debugging the source files with SAM CLI) [boolean] [default: true] -o, --output Emits the synthesized cloud assembly into a directory (default: cdk.out) [string] --no-color Removes colors and other style from console output [boolean] [default: false] --fail Fail with exit code 1 in case of diff [boolean] [default: false] --version Show version number [boolean] -h, --help Show help [boolean]
If your app has a single stack, there is no need to specify the stack name If one of cdk.json or ~/.cdk.json exists, options specified there will be used as defaults. Settings in cdk.json take precedence. </pre>
### Node version alignment
-
If you get an error such as this when you run:
/usr/local/Cellar/aws-cdk/1.114.0/libexec/lib/node_modules/aws-cdk/node_modules/yargs-parser/build/index.cjs:1007 throw Error(`yargs parser supports a minimum Node.js version of ${minNodeVersion}. Read our version support policy: https://github.com/yargs/yargs-parser#supported-nodejs-versions`); ^ Error: yargs parser supports a minimum Node.js version of 10. Read our version support policy: https://github.com/yargs/yargs-parser#supported-nodejs-versions -
What version of NodeJs do you have installed?
node -v
If you have a version less that (10), such as “v9.11.1”, upgrade Node to the version specified (v10).
-
To enable multiple versions of Node to be installed simultaneously, install NVM:
brew install nvm
-
Identify an LTS version to install:
nvm ls
v8.11.1 v8.11.4 -> v9.11.1 system default -> node (-> v9.11.1) node -> stable (-> v9.11.1) (default) stable -> 9.11 (-> v9.11.1) (default) iojs -> N/A (default) lts/* -> lts/dubnium (-> N/A) lts/argon -> v4.9.1 (-> N/A) lts/boron -> v6.17.1 (-> N/A) lts/carbon -> v8.16.0 (-> N/A) lts/dubnium -> v10.16.0 (-> N/A) -
Based on the above:
nvm install 10.16.0
Now using node v10.16.0 (npm v6.9.0)
https://wilsonmar.github.io/node-osx-install/
NPM Code
-
In the https://docs.aws.amazon.com/cdk/latest/guide/cli.html install a fall-back global version:
npm install -g aws-cdk # install latest version
PROTIP: Be prepared to work with multiple versions of the AWS CDK installed. Install a matching version of the AWS CDK Toolkit in individual CDK projects (omitting -g from the npm install command).
-
use npx cdk to invoke it; this will run the local version if one exists, falling back to a global version if not.
However, there is also:
Initialize language folder
cdk init --language python
cdk deploy
cdk synth stack1
CDKGoat
https://github.com/bridgecrewio/cdkgoat is a “Vulnerable by Design” AWS CDK repository. CdkGoat is a learning and training project that demonstrates how common configuration errors can find their way into production cloud environments.
It also shows how Bridgecrew can be used with the AWS CDK to provide CloudFormation template vulnerability scanning at build time, even though no CloudFormation templates exist in the source repository.
Rockstars
Elad Ben-Israel (@emeshbi)
Jason Fulghum (CDK team Sr Mgr Dev Tools)
Jun Fisk at CloudAcademy.com:
- Working with the AWS CDK Toolkit
- Writing Your Own AWS CDK Constructs in TypeScript
- Evaluating CDK Constructs with Fine-Grained Assertions and Validation Tests
- AWS CDK Construct Testing Challenge
- AWS CDK Toolkit Challenge Using Python
VIDEO: “Multi-Stack Deployments with CDK for Terraform” discusses https://github.com/hashicorp/cdktf-multistack-serverless-example used by the end to end example in TypeScript for a serverless web application hosted on AWS and deployed with the CDK for Terraform.
-
Frontend: React, Create React App, statically hosted via AWS S3 + CloudFront
-
Backend API: AWS Lambda + API Gateway + DynamoDB
Resources
-
https://www.youtube.com/watch?v=1ps0Wh19MHQ
-
https://www.youtube.com/watch?v=Q1FcifrDocE
-
https://www.youtube.com/watch?v=Cf3yJv3klsg
-
https://www.youtube.com/watch?v=ZWCvNFUN-sU
-
https://www.hashicorp.com/blog/cdk-for-terraform-enabling-python-and-typescript-support
-
AWS Cloud Development Kit - From Biginner to … by Valaxy Technologies on OReilly.com with code at https://github.com/