Wilson Mar bio photo

Wilson Mar

Hello. Hire me!

Email me Calendar Skype call 310 320-7878

LinkedIn Twitter Gitter Google+ Youtube

Github Stackoverflow Pinterest

AWS CloudFormation + DSC for many clouds


Overview

This tutorial is a step-by-step hands-on introduction to use of Terraform to deploy a cluster of web servers and a load balancer on AWS and other providers (clouds).

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently.

Repeatable. Versioned. Documented. Automated. Testable. Shareable.

Competitors to Terraform

NOTE: Other IAC (Infrastructure as Code) tools include Chef, Puppet, Ansible, SaltStack, AWS CloudFormation.

https://www.terraform.io/intro/index.html

Competitors to Terraform

NOTE Other IAC tools include Chef, Puppet, Ansible, SaltStack, AWS CloudFormation.

  CloudFormation Terraform
Configuration format JSON HCL JSON
State management JSON HCL JSON
Providers support AWS only AWS, GCE, Azure (20+)
Execution control No Yes
Iterations No Yes
Manage already created resources No Yes (hard)
Failure handling Optional rollback Fix & retry
Open Source contributions? No Yes (GitHub issues)
Logical comparisons No Limited
Extensible Modules No Yes

Terraform also provides execution control, iterations, and (perhaps most of all) management of resources already created (desired state configuration) over several cloud providers (not just AWS).

Installation

Install on MacOS

In a Terminal window:

brew install -g terraform

The response at time of writing:

==> Downloading https://homebrew.bintray.com/bottles/terraform-0.10.6.sierra.bot
######################################################################## 100.0%
==> Pouring terraform-0.10.6.sierra.bottle.tar.gz
==> Caveats
zsh completions have been installed to:
  /usr/local/share/zsh/site-functions
==> Summary
🍺  /usr/local/Cellar/terraform/0.10.6: 7 files, 63.6MB
   

PROTIP: Terraform is written in the Go language, so there is no JVM to download as well. “Read all about it here”.

  1. Proceed to Get sample Terraform scripts.

Install on Windows

  1. In a Run command window as Administrator.
  2. Install Chocolatey cmd:
  3. Install Terraform using Chocolatey:

    choco install terraform -y

    The response at time of writing:

    Chocolatey v0.10.8
    Installing the following packages:
    terraform
    By installing you accept licenses for the packages.
    Progress: Downloading terraform 0.10.6... 100%
     
    terraform v0.10.6 [Approved]
    terraform package files install completed. Performing other installation steps.
    The package terraform wants to run 'chocolateyInstall.ps1'.
    Note: If you don't run this script, the installation will fail.
    Note: To confirm automatically next time, use '-y' or consider:
    choco feature enable -n allowGlobalConfirmation
    Do you want to run the script?([Y]es/[N]o/[P]rint): y
     
    Removing old terraform plugins
    Downloading terraform 64 bit
      from 'https://releases.hashicorp.com/terraform/0.10.6/terraform_0.10.6_windows_amd64.zip'
    Progress: 100% - Completed download of C:\Users\vagrant\AppData\Local\Temp\chocolatey\terraform\0.10.6\terraform_0.10.6_windows_amd64.zip (12.89 MB).
    Download of terraform_0.10.6_windows_amd64.zip (12.89 MB) completed.
    Hashes match.
    Extracting C:\Users\vagrant\AppData\Local\Temp\chocolatey\terraform\0.10.6\terraform_0.10.6_windows_amd64.zip to C:\ProgramData\chocolatey\lib\terraform\tools...
    C:\ProgramData\chocolatey\lib\terraform\tools
     ShimGen has successfully created a shim for terraform.exe
     The install of terraform was successful.
      Software installed to 'C:\ProgramData\chocolatey\lib\terraform\tools'
     
    Chocolatey installed 1/1 packages.
     See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
     

Sample Terraform scripts

Sample scripts have been prepared by several helpful people.

  • https://github.com/brikis98/infrastructure-as-code-talk/tree/master/terraform-configurations

### This tutorial

The sample scripts referenced by this tutorial contain moustache variable mark-up so that you can generate a set for your organization.

https://www.terraform.io/docs/providers/azurerm/r/virtual_machine_scale_set.html

  1. Install a Git client if you haven’t already.
  2. Create or navigate to a container folder where new repositories are added. For example:

    ~/gits/terraform/gruntwork-io

  3. Get a sample repo (word-wrapped command):

    git clone https://github.com/gruntwork-io/intro-to-terraform.git --depth=1 && cd intro-to-terraform

    At time of writing:

    Cloning into 'intro-to-terraform'...
    remote: Counting objects: 12, done.
    remote: Compressing objects: 100% (12/12), done.
    remote: Total 12 (delta 1), reused 9 (delta 0), pack-reused 0
    Unpacking objects: 100% (12/12), done.
    

    Plug-in Initialization

  4. Initialize Terraform plug-ins:

    terraform init

    Response:

    Initializing provider plugins...
           - Checking for available provider plugins on https://releases.hashicorp.com...
           - Downloading plugin for provider "aws" (0.1.4)...   
     
    The following providers do not have any version constraints in configuration,
    so the latest version was installed.
     
    To prevent automatic upgrades to new major versions that may contain breaking
    changes, it is recommended to add version = "..." constraints to the
    corresponding provider blocks in configuration, with the constraint strings
    suggested below.
     
           * provider.aws: version = "~> 0.1"
     
    Terraform has been successfully initialized!
     
    You may now begin working with Terraform. Try running "terraform plan" to see
    any changes that are required for your infrastructure. All Terraform commands
    should now work.
     
    If you ever set or change modules or backend configuration for Terraform,
    rerun this command to reinitialize your working directory. If you forget, other
    commands will detect it and remind you to do so if necessary.
    

    .tf files

  5. Navigate into one of the folders (in the case of the example repo):

    cd single-web-server

  6. Open folder using Atom or list files:

    ls -al

    Main.tf

    PROTIP: There should be only one main.tf per folder.

    NOTE: Terraform declarative files coded end with .tf file type.

    .tfstate files (containing JSON) are autogenerated to persist the state of runs. This can also be in a Hasicorp Consul server. Terraform creates a dependency graph (specfically, a Directed Acyclic Graph). This is so that nodes are built in the order they are needed.

    NOTE: State files can contain secrets.

    It also dentifies additions and deletions.

    Terraform apply creates a dev.state.lock.info file as a way to signal to other processes to stay away while changes to the environment are underway.

  7. Edit file main.tf. Widen the screen width to avoid wrapping.

    Both CloudFormation and Terraform work with JSON, but Terraform works with HCL (Hashicorp Configuratio Language) that is both human and machine friendly. https://github.com/hashicorp/hcl and described at https://www.terraform.io/docs/configuration/syntax.html

    NOTE: Terraform code is written in a declarative language called HCL (HashiCorp Configuration Language). It’s less verbose than JSON and more concise than YML.

    Unlike JSON and YML, HCL allows annotations as in bash scripts: Single line comments start with # (pound sign).
    Multi-line comments are wrapped between /* and */. Values can be interpolated usning syntax wrapped in ${}, in the format of ${type.name.attribute}. Literal $ are coded by doubling up $$.

    Back-slashes specify continuation.

    HCL does not have conditional if/else logic, which is why modules are necessary.

    Providers

    Terraform providers reference APIs. Examples are AWS, Google, Azure, Kubernetes, GitLab, DigitalOcean, Heroku, GitHub, OpenStack.

    https://github.com/terraform-providers

    https://github.com/hashicorp/terraform/tree/master/builtin/providers

    Metadata related to each provider are defined like this:

    provider "aws" {
      alias = "NorthEast"
      region = "us-east-1"
    }
    

    resource definitions specify the desired state of resources.

    Provisioners configurations are also plugins:

    Provisioner definitions define the properties of each resource, such as initialization commands. For example, this installs an nginx web server and displays a minimal HTML page:

    provisioner "remote-exec" {
      inline = [
     "sudo yum install nginx -y",
     "sudo service nginx start",
     "echo "<title>NGINX server</head><body style=\"background-color"></body></html>"
      ]
    }
    </pre>
    
    
    
  8. PROTIP: Make sure that the AWS region is what you want.

    https://www.terraform.io/docs/providers/aws/r/instance.html AWS provider

    NOTE: The contents of the repo were written based on Terraform 0.7.x.

  9. Variables file:

    vars.tf file contains specifcation of values to variables, such as the server port (8080).

    Defaults and lookup function

    PROTIP: Variables can be assigned multiple default values selected by a lookup function:

    # AWS_ACCESS_KEY_ID
    # AWS_SECRET_ACCESS_KEY
    # export AWS_DEFAULT_REGION=xx-yyyy-0
     
    variable "server_port" {
      description = "The port the server will use for HTTP requests"
      default = 8080
    }
    variable "amis" {
      type = "map”"
      default = {
     us-east-1 = "ami-1234"
     us-west-1 = "ami-5678"
      }
    }
    ami = ${lookup(var.amis, "us-east-1")}
    

    PROTIP: With AWS EC2, the us-east-1 region must be used as the basis for creating others.

    NOTE: Amazon has an approval process for making AMIs available on the public Amazon Marketplace.

    CIDR Subnet function

    variable network_info {
    default = “10.0.0.0/8” #type, default, description
    }
    cidr_block = ${cidrsubnet(var.network_info, 8, 1)} # returns 10.1.0.0/16
    cidr_block = ${cidrsubnet(var.network_info, 8, 2)} # returns 10.2.0.0/16
      
    variable network_info {
    default = “10.0.0.0/8” #type, default, description
    }
    cidr_block = ${cidrsubnet(var.network_info, 8, 1)} # returns 10.1.0.0/16
    cidr_block = ${cidrsubnet(var.network_info, 8, 2)} # returns 10.2.0.0/16
      

    Output

    outputs.tf file

    output "aws_elb_public_dns" {
      value = "${aws_elb.web.dns_name}"
    }
    output "public_ip" {
      value = "${aws_instance.example.public_ip}"
    }
    output "azure_rm_dns_cname" {
      value = "${azurerm_dns_cname_record.elb.id}"
    }
    
  10. PROTIP: If the AMI is no longer available, you will get an error message.

    NOTE: Components of Terrform are: Providers, Resources, Provisioners.

    AWS Configuration

export AWS_ACCESS_KEY_ID=(your access key id) export AWS_SECRET_ACCESS_KEY=(your secret access key)

Alternatively, specify a file named hoge containing credentials:

[hoge]
aws_access_key_id = blahblahbalh
aws_secret_access_key = FugaFuga
   

so you can pass profile name by –profile option:

terraforming s3 --profile hoge

### Terraform Plan

  1. Have Terrform evaluate based on vars in a different (parent) folder:

    terraform plan -var-file=’..\terraform.tfvars’ -var-file=’.\Development\development.tfvars’ -state=’.\Development\dev.state’

    The two dots specifies to look above the current folder.

    In this example terraform.tfvars file are credentials for both AWS EC2 and Azure ARM providers:

    bucket_name = "mycompany-sys1-v1"
    arm_subscription_id = "???"
    arm_principal = "???"
    arm_passsord = "???"
    tenant_id = "223d"
    aws_access_key = "insert access key here>"
    aws_secret_key = "insert secret key here"
    private_key_path = "C:\\MyKeys1.pem"
    

    The private_key_path should be a full path, containing \\ so that the single slash is not interpreted as a special character.

    bucket_name must be globally unique within all of the AWS provider customers.

    States & Environments

    PROTIP: Define a state file for each environment:

    • dev.state
    • uat.state
    • perftest.state
    • stage.state
    • prod.state

    A development.tfvars file:

    environment_tag = "dev"
    tenant_id = "223d"
    billing_code_tag = "DEV12345"
    dns_site_name = "dev-web"
    dns_zone_name = "mycorp.xyz"
    dns_resource_group = "DNS"
    instance_count = "2"
    subnet_count = "2"
    

    The production.tfvars file usually instead contain more instances and thus subnets that go through a load balancer for auto-scaling:

    environment_tag = "prod"
    tenant_id = "223d"
    billing_code_tag = "PROD12345"
    dns_site_name = "marketing"
    dns_zone_name = "mycorp.com"
    dns_resource_group = "DNS"
    instance_count = "6"
    subnet_count = "3"
    

    All these would use main_config.tf and variables.tf files that are commonly used for all environments:

    terraform apply -state=”.\develop\dev.state” -var=”environment_name=development”

    1. Generate model from logical definition (the Desired State).
    2. Load current model (preliminary source data).
    3. Refresh current state model by querying remote provider (final source state).
    4. Calculate difference from source state to target state (plan).
    5. Apply plan.

    A sample response:

    “&LT;computered>” means Terraform figures it out.

    Remote state

    NOTE

    terraform.tfstate can be stored in S3, Atlas, Consul, etcd, HTTP

    terraform remote pull obtains state.

0.

vars.tf file contains specifcation of values to variables, such as the server port (8080).

# AWS_ACCESS_KEY_ID
# AWS_SECRET_ACCESS_KEY

variable "server_port" {
  description = "The port the server will use for HTTP requests"
  default = 8080
}
    

outputs.tf file

output "public_ip" {
  value = "${aws_instance.example.public_ip}"
}
   
  1. PROTIP: Make sure that the AWS region is what you want.

    https://www.terraform.io/docs/providers/aws/r/instance.html AWS provider

    NOTE: The contents of the repo were written based on Terraform 0.7.x.

  2. PROTIP: If the AMI is no longer available, you will get an error message.

    ami = "ami-2d39803a"

    Terraform Plan

  3. Have Terrform evaluate based on vars in a different (parent) folder:

    terraform plan -var-file=’..\terraform.tfvars’

    A sample response:

    “&LT;computered>” means Terraform figures it out.

    • terraform.tfstate stored in S3, Atlas, Consul, etcd, HTTP

    Hashicorp Atlas is a licensed solution

    Version

  4. Obtain version installed:

    terraform version

    Alternately:

    terraform --version

    The response at time of writing:

    	Terraform v0.10.6
    

    QUESTION: Pace of change in Terraform?

    Release 0.6.8 (2.12.2015)

    TODO: Update terraform

    Command list

  5. For a list of commands:

    terraform

    The response at time of writing:

    Usage: terraform [--version] [--help] <command> [args]
     
    The available commands for execution are listed below.
    The most common, useful commands are shown first, followed by
    less common or more advanced commands. If you're just getting
    started with Terraform, stick with the common commands. For the
    other commands, please read the help and docs before usage.
     
    Common commands:
     apply              Builds or changes infrastructure
     console            Interactive console for Terraform interpolations
     destroy            Destroy Terraform-managed infrastructure
     env                Workspace management
     fmt                Rewrites config files to canonical format
     get                Download and install modules for the configuration
     graph              Create a visual graph of Terraform resources
     import             Import existing infrastructure into Terraform
     init               Initialize a Terraform working directory
     output             Read an output from a state file
     plan               Generate and show an execution plan
     providers          Prints a tree of the providers used in the configuration
     push               Upload this Terraform module to Atlas to run
     refresh            Update local state file against real resources
     show               Inspect Terraform state or plan
     taint              Manually mark a resource for recreation
     untaint            Manually unmark a resource as tainted
     validate           Validates the Terraform files
     version            Prints the Terraform version
     workspace          Workspace management
     
    All other commands:
     debug              Debug output management (experimental)
     force-unlock       Manually unlock the terraform state
     state              Advanced state management
    
  6. Define enviornment variables:

    TF_VAR_first_name=John terraform apply
    
  7. Define Terraform variable:

    terraform apply -var 'first_name=John' -var 'last_name=Bunyan'
    

    Values to Terraform variables define inputs such as run-time DNS/IP addresses into Terraform modules.

    NOTE: Built-in functions:

    https://terraform.io/docs/configuration/interpolation.html

    Apps to install

    NOTE: Software can be specified for installation using Packer’s local-exec provisioner which has Terraform on host machines executes commands. For example, on a Ubuntu machine:

    resource "null_resource" "local-software" {
      provisioner "local-exec" {
     command = <<EOH
    sudo apt-get update
    sudo apt-get install -y ansible
    EOH
      }
    }
    

    NOTE: apt-get is in-built within Ubuntu Linux distributions.

    PROTIP: Use this to bootstrap automation such as assigning permissions and running Ansible or PowerShell DSC, then use DSC scripts for more flexibility and easier debugging.

    Output variables

  8. Output Terraform variable:

    output "loadbalancer_dns_name" {
      value = "${aws_elb.loadbalancer.dns_name}"
    }
    

    Processing flags

    HCL can contain flags that affect processing. For example, within a resource specification, force_destroy = true forces the provider to delete the resource when done.

    Verify websites

  9. The website accessible?

  10. In the provider’s console (EC2), verify

    Destroy clean up

  11. Destroy instances so they don’t rack up charges unproductively:

    terraform destroy

    PROTIP: Amazon charges by the hour while others charge by the minute.

  12. Verify in the provider’s console.

Terraform Console

  1. On Windows, open the Terraform Console from a command Prompt rather than from within PowerShell.

    terraform console

    element(list(“one”,”two”,”three”),0,2)

    The response is (because counting begins from zero):

    three

    element(list(“one”,”two”,”three”),0,2)

    one, two

Automation

NOTE: Automating infrastructure deployment consists of:

  • Provisioning resources
  • Planning updates
  • Using source control
  • Reusing templates

PROTIP: Terrform files are “idempotent” (repeat runs don’t change anything if nothing is changed). Thus Terraform defines the “desired state configuration”.

Modules

https://github.com/terraform-community-modules

https://github.com/objectpartners/tf-modules

module "rancher" {
  source = "github.com/objectpartners/tf-modules//rancher/server-standalone-elb-db&ref=9b2e590"
}
   

The double slashes separate the repo from the subdirectory.

The ref is a commit ID

Plugins into Terraform

All Terraform providers are plugins - multi-process RPC (Remote Procedure Calls).

https://github.com/hashicorp/terraform/plugin

https://terraform.io/docs/plugins/index.html

Terraform expect plugins to follow a very specific naming convention of terraform-TYPE-NAME. For example, terraform-provider-aws, which tells Terraform that the plugin is a provider that can be referenced as “aws”.

PROTIP: Establish a standard for where plugins are located:

For *nix systems, ~/.terraformrc

For Windows, %APPDATA%/terraform.rc

https://www.terraform.io/docs/internals/internal-plugins.html

PROTIP: When writing your own terraform plugin, create a new Go project in GitHub, then locally use a directory structure:

$GOPATH/src/github.com/USERNAME/terraform-NAME

where USERNAME is your GitHub username and NAME is the name of the plugin you’re developing. This structure is what Go expects and simplifies things down the road.

Learning Resources

Video courses at Pluralsight.com:

YouTube videos:

Rock Stars

James Turnbull

Yevgeniy (Jim) Brikman (ybrikman.com), co-founder of DevOps as a Service Gruntwork.io :

Comprehensive Guide to Terraform includes:

  1. Why we use Terraform and not Chef, Puppet, Ansible, SaltStack, or CloudFormation

  2. How to manage Terraform state

  3. Terraform tips & tricks: loops, if-statements, and gotchas

dtan4

http://terraforming.dtan4.net/

https://github.com/dtan4/terraforming is a Ruby

AWS Cloud Formation

Puppet, Chef, Ansible, Salt AWS API libraries Boto, Fog

NOTE: Terraform remote configures remote state storage with Terraform.

References

https://github.com/brikis98/infrastructure-as-code-talk Infrastructure-as-code: running microservices on AWS with Docker, ECS, and Terraform