Wilson Mar bio photo

Wilson Mar

Hello. Hire me!

Email me Calendar Skype call 310 320-7878

LinkedIn Twitter Gitter Google+ Youtube

Github Stackoverflow Pinterest

I declare! Client-only immutable multi-cloud provisioning, with open-sourced Enterprise support


Overview

This tutorial is a step-by-step hands-on introduction to use of Terraform for creating a cluster of web servers through a load balancer on AWS, Azure, and Google Cloud.

Terraform, at terraform.io, is a tool for building, changing, and versioning infrastructure in the cloud.

Repeatable. Versioned. Documented. Automated. Testable. Shareable.

Competitors to Terraform

NOTE: Other IAC (Infrastructure as Code) tools include Chef, Puppet, Ansible, SaltStack, AWS CloudFormation.

terraform-comp-colored-650x261-36439Click to pop-up full screen image

“Immutable” means servers are treated like “cattle” (removed from the herd) and not as “pets” (kept alive as long as possible).

Feature CloudFormation Terraform
Source code closed-source open source
Configuration format JSON HCL JSON
State management JSON HCL JSON
Cloud Providers support AWS only AWS, GCE, Azure (20+)
Execution control No Yes
Iterations No Yes
Manage already created resources No Yes (hard)
Failure handling Optional rollback Fix & retry
Open Source contributions? No Yes (GitHub issues)
Logical comparisons No Limited
Extensible Modules No Yes

Terraform also provides execution control, iterations, and (perhaps most of all) management of resources already created (desired state configuration) over several cloud providers (not just AWS).

Installation

PROTIP: Terraform is written in the Go language, so there is no JVM to download as well.

Consider tfenv

If you plan on frequently switching among several versions installed of Terraform:

https://github.com/kamatama41/tfenv

Install on MacOS

  1. In a Terminal window:

    brew install -g terraform

    The response at time of writing:

    ==> Downloading https://homebrew.bintray.com/bottles/terraform-0.10.6.sierra.bot
    ######################################################################## 100.0%
    ==> Pouring terraform-0.10.6.sierra.bottle.tar.gz
    ==> Caveats
    zsh completions have been installed to:
      /usr/local/share/zsh/site-functions
    ==> Summary
    🍺  /usr/local/Cellar/terraform/0.10.6: 7 files, 63.6MB
    
  2. Proceed to Get sample Terraform scripts.

Install on Windows

  1. In a Run command window as Administrator.
  2. Install Chocolatey cmd:
  3. Install Terraform using Chocolatey:

    choco install terraform -y

    The response at time of writing:

    Chocolatey v0.10.8
    Installing the following packages:
    terraform
    By installing you accept licenses for the packages.
    Progress: Downloading terraform 0.10.6... 100%
     
    terraform v0.10.6 [Approved]
    terraform package files install completed. Performing other installation steps.
    The package terraform wants to run 'chocolateyInstall.ps1'.
    Note: If you don't run this script, the installation will fail.
    Note: To confirm automatically next time, use '-y' or consider:
    choco feature enable -n allowGlobalConfirmation
    Do you want to run the script?([Y]es/[N]o/[P]rint): y
     
    Removing old terraform plugins
    Downloading terraform 64 bit
      from 'https://releases.hashicorp.com/terraform/0.10.6/terraform_0.10.6_windows_amd64.zip'
    Progress: 100% - Completed download of C:\Users\vagrant\AppData\Local\Temp\chocolatey\terraform\0.10.6\terraform_0.10.6_windows_amd64.zip (12.89 MB).
    Download of terraform_0.10.6_windows_amd64.zip (12.89 MB) completed.
    Hashes match.
    Extracting C:\Users\vagrant\AppData\Local\Temp\chocolatey\terraform\0.10.6\terraform_0.10.6_windows_amd64.zip to C:\ProgramData\chocolatey\lib\terraform\tools...
    C:\ProgramData\chocolatey\lib\terraform\tools
     ShimGen has successfully created a shim for terraform.exe
     The install of terraform was successful.
      Software installed to 'C:\ProgramData\chocolatey\lib\terraform\tools'
     
    Chocolatey installed 1/1 packages.
     See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
     

Get Sample Terraform scripts

  1. Install a Git client if you haven’t already.
  2. Use an internew browser (Chrome) to my sample PowerShell DSC scripts at:

    https://github.com/wilsonmar/terraform-starter.git

    (I would be honored if you click Star)

  3. Create a GitHub account for yourself if you haven’t already.
  4. Click the Fork button to make it yours, since you may be making changes.
  5. Create or navigate to a container folder where new repositories are added. For example:

    ~/gits/wilsonmar/terraform

  6. Get my sample PowerShell scripts onto your laptop (substituting “wilsonmar” with your own account name):

    git clone https://github.com/wilsonmar/terraform-starter.git --depth=1 && cd terraform-starter

    The above is one line, but may be word-wrapped on your screen.

    The response at time of writing:

    Cloning into 'terraform-starter'...
    remote: Counting objects: 12, done.
    remote: Compressing objects: 100% (12/12), done.
    remote: Total 12 (delta 1), reused 9 (delta 0), pack-reused 0
    Unpacking objects: 100% (12/12), done.
    

    Other scripts

    These scripts is a combination of scripts prepared by several helpful people:

    https://github.com/gruntwork-io/intro-to-terraform.git

    TODO: The sample scripts referenced by this tutorial contain moustache variable mark-up so that you can generate a set for your organization.

    https://www.terraform.io/docs/providers/azurerm/r/virtual_machine_scale_set.html

    Environment folders Validate

    PROTIP: Separate Terraform configurations by a folder for each environment.

    • base
    • dev
    • loadtest (performance testing)
    • stage
    • uat (User Acceptance Testing)
    • prod

  7. Navigate into the base folder.

    PROTIP: Terraform commands act only on the current directory, and does not recurse into sub directories.

  8. View the development.tfvars file:

    environment_tag = "dev"
    tenant_id = "223d"
    billing_code_tag = "DEV12345"
    dns_site_name = "dev-web"
    dns_zone_name = "mycorp.xyz"
    dns_resource_group = "DNS"
    instance_count = "2"
    subnet_count = "2"
    

    The production.tfvars file usually instead contain more instances and thus subnets that go through a load balancer for auto-scaling:

    environment_tag = "prod"
    tenant_id = "223d"
    billing_code_tag = "PROD12345"
    dns_site_name = "marketing"
    dns_zone_name = "mycorp.com"
    dns_resource_group = "DNS"
    instance_count = "6"
    subnet_count = "3"
    

    All these would use main_config.tf and variables.tf files that are commonly used for all environments:

    Plug-in Initialization

    Providers are not included with the installer, so…

  9. Initialize Terraform plug-ins:

    terraform init

    Response:

    Initializing provider plugins...
           - Checking for available provider plugins on https://releases.hashicorp.com...
           - Downloading plugin for provider "aws" (0.1.4)...   
     
    The following providers do not have any version constraints in configuration,
    so the latest version was installed.
     
    To prevent automatic upgrades to new major versions that may contain breaking
    changes, it is recommended to add version = "..." constraints to the
    corresponding provider blocks in configuration, with the constraint strings
    suggested below.
     
           * provider.aws: version = "~> 0.1"
     
    Terraform has been successfully initialized!
     
    You may now begin working with Terraform. Try running "terraform plan" to see
    any changes that are required for your infrastructure. All Terraform commands
    should now work.
     
    If you ever set or change modules or backend configuration for Terraform,
    rerun this command to reinitialize your working directory. If you forget, other
    commands will detect it and remind you to do so if necessary.
    

    This creates a hidden .terraform\plugins" folder path containing a folder for your os - darwin_amd64` for MacOS.

    .tf files

  10. Navigate into one of the folders (in the case of the example repo):

    cd single-web-server

  11. Open folder using Atom or list files:

    ls -al

    https://www.terraform.io/docs/commands/validate.html

  12. Navigate into

  13. Validate the folder:

    terraform validate

    If no issues are identified, no message appears. (no news is good news)

    pre-commit hook to validate in your Git repository

    Main.tf

    PROTIP: There should be only one main.tf per folder.

    NOTE: Terraform files coded end with .tf file type.

  14. Edit file main.tf. Widen the screen width to avoid wrapping.

    Both CloudFormation and Terraform work with JSON, but Terraform works with HCL (Hashicorp Configuratio Language) that is both human and machine friendly. https://github.com/hashicorp/hcl and described at https://www.terraform.io/docs/configuration/syntax.html

    NOTE: Terraform code is written in a language called HCL (HashiCorp Configuration Language). It’s less verbose than JSON and more concise than YML.

    Unlike JSON and YML, HCL allows annotations as in bash scripts: Single line comments start with # (pound sign).
    Multi-line comments are wrapped between /* and */.

    Values can be interpolated usning syntax wrapped in ${}, called interpolation syntax, in the format of ${type.name.attribute}. Literal $ are coded by doubling up $$. For example, ${aws.instance.base.id} is interpolated to i-28978a2.

    Back-slashes specify continuation.

    More importantly, tf files are declarative, meaning that they define the desired end-state (outcomes). If 15 servers are declared, Terraform automatically adds or removes servers to end up with 15 servers rather than specifying procedures to invoke (such as add 5 servers). Terraform know how many servers it has setup already.

    Paid Pro and Premium licenses of Terraform add version control integration, MFA security, and other enterprise features.

    HCL does not have conditional if/else logic, which is why modules are necessary.

    Cloud Providers

    Terraform providers reference APIs. Examples are AWS, Google, Azure, Kubernetes, GitLab, DigitalOcean, Heroku, GitHub, OpenStack.

    https://github.com/terraform-providers

    https://github.com/hashicorp/terraform/tree/master/builtin/providers

  15. https://aws.amazon.com/

    Metadata related to each provider are defined like this:

    provider "aws" {
      alias = "NorthEast"
      region = "us-east-1"
      instance_type = "t1.micro"
    }
    

    resource definitions specify the desired state of resources.

    “t1.micro” qualifies for the Amazon free tier available to first-year subscribers.

    NOTE: Components of Terrform are: provider, resource, provision.

    Provisioners

    Provisioners configurations are also plugins:

    Provisioner definitions define the properties of each resource, such as initialization commands. For example, this installs an nginx web server and displays a minimal HTML page:

    provisioner "remote-exec" {
      inline = [
     "sudo yum install nginx -y",
     "sudo service nginx start",
     "echo "<title>NGINX server</head><body style=\"background-color"></body></html>"
      ]
    }
    </pre>
    
    
    
  16. PROTIP: Make sure that the AWS region is what you want.

    https://www.terraform.io/docs/providers/aws/r/instance.html AWS provider

    NOTE: The contents of the repo were written based on Terraform 0.7.x.

  17. Variables file:

    vars.tf file contains specifcation of values to variables, such as the server port (8080).

    Defaults and lookup function

    PROTIP: Variables can be assigned multiple default values selected by a lookup function:

    # AWS_ACCESS_KEY_ID
    # AWS_SECRET_ACCESS_KEY
    # export AWS_DEFAULT_REGION=xx-yyyy-0
     
    variable "server_port" {
      description = "The port the server will use for HTTP requests"
      default = 8080
    }
    variable "amis" {
      type = "map”"
      default = {
     us-east-1 = "ami-1234"
     us-west-1 = "ami-5678"
      }
    }
    ami = ${lookup(var.amis, "us-east-1")}
    

    PROTIP: With AWS EC2, the us-east-1 region must be used as the basis for creating others.

    NOTE: Amazon has an approval process for making AMIs available on the public Amazon Marketplace.

    CIDR Subnet function

    variable network_info {
    default = “10.0.0.0/8” #type, default, description
    }
    cidr_block = ${cidrsubnet(var.network_info, 8, 1)} # returns 10.1.0.0/16
    cidr_block = ${cidrsubnet(var.network_info, 8, 2)} # returns 10.2.0.0/16
      
    variable network_info {
    default = “10.0.0.0/8” #type, default, description
    }
    cidr_block = ${cidrsubnet(var.network_info, 8, 1)} # returns 10.1.0.0/16
    cidr_block = ${cidrsubnet(var.network_info, 8, 2)} # returns 10.2.0.0/16
      

    In this example terraform.tfvars file are credentials for both AWS EC2 and Azure ARM providers:

    bucket_name = "mycompany-sys1-v1"
    arm_subscription_id = "???"
    arm_principal = "???"
    arm_passsord = "???"
    tenant_id = "223d"
    aws_access_key = "insert access key here>"
    aws_secret_key = "insert secret key here"
    private_key_path = "C:\\MyKeys1.pem"
    

    The private_key_path should be a full path, containing \\ so that the single slash is not interpreted as a special character.

    bucket_name must be globally unique within all of the AWS provider customers.

    Terraforming AWS Configuration

    PROTIP: Install from https://github.com/dtan4/terraforming a Ruby script that enables a command such as:

    terraforming s3 --profile dev

    You can pass profile name by –profile option.

    Terraform Plan

  18. Have Terrform evaluate based on vars in a different (parent) folder:

    terraform plan -var-file=’..\terraform.tfvars’ -var-file=’.\Development\development.tfvars’ -state=’.\Development\dev.state’ -out base-date-+'%s'.plan

    The two dots in the command specifies to look above the current folder.

    The -out parameter outputs to a file name that begins with base- and ends with .plan. The date is like 147772345 which is the numer of seconds since 1/1/1970.

    A sample response:

    "<computered>" means Terraform figures it out.
    

    Output

    outputs.tf file

    output "aws_elb_public_dns" {
      value = "${aws_elb.web.dns_name}"
    }
    output "public_ip" {
      value = "${aws_instance.example.public_ip}"
    }
    output "azure_rm_dns_cname" {
      value = "${azurerm_dns_cname_record.elb.id}"
    }
    
  19. PROTIP: If the AMI is no longer available, you will get an error message.

    Terraform apply

    terraform apply -state=”.\develop\dev.state” -var=”environment_name=development”

    1. Generate model from logical definition (the Desired State).
    2. Load current model (preliminary source data).
    3. Refresh current state model by querying remote provider (final source state).
    4. Calculate difference from source state to target state (plan).
    5. Apply plan.

    Ignore state files

    Terraform apply generates .tfstate files (containing JSON) to persist the state of runs. It contains a maps resources IDs to their data.

  20. View the .gitignore file:

    terraform.tfstate*
    *.tfstate
    *.tfstate.backup
    .terraform/
    *.iml
    *.plan
    vpc
    

    CAUTION: State files can contain secrets.

    .terraform/ specifies that the folder is ignored when pushing to GitHub.

    Terraform apply creates a dev.state.lock.info file as a way to signal to other processes to stay away while changes to the environment are underway.

    tfstate.backup is created from the most recent previous execution before the current tfstate file contents.

    Remote state

    NOTE terraform.tfstate can be stored over the network in S3, etcd distributed key value store (used by Kubernetes), or a Hashicorp Atlas or Consul server.

    Hashicorp Atlas is a licensed solution.

    State can be obtained using command:

    terraform remote pull

  21. vars.tf file contains specifcation of values to variables, such as the server port (8080).

    # AWS_ACCESS_KEY_ID
    # AWS_SECRET_ACCESS_KEY
    
    

variable “server_port” { description = “The port the server will use for HTTP requests” default = 8080 } </pre>

outputs.tf file

output "public_ip" {
  value = "${aws_instance.example.public_ip}"
}
   
  1. PROTIP: Make sure that the AWS region is what you want.

    https://www.terraform.io/docs/providers/aws/r/instance.html AWS provider

    NOTE: The contents of the repo were written based on Terraform 0.7.x.

  2. PROTIP: If the AMI is no longer available, you will get an error message.

    ami = "ami-2d39803a"

    Terraform Plan

  3. Have Terrform evaluate based on vars in a different (parent) folder:

    terraform plan -var-file=’..\terraform.tfvars’

    In the sample response:

    Pluses and minuses flag additions and deletions. This is a key differentiator for Terraform as a “”

    “&LT;computered>” means Terraform figures it out.

    Terraform creates a dependency graph (specfically, a Directed Acyclic Graph). This is so that nodes are built in the order they are needed.

    Version

  4. Obtain version installed:

    terraform version

    Alternately:

    terraform --version

    WARNING: The response at time of writing, Terraform is not even “1.0” release, meaning it’s in beta maturity.:

    Terraform v0.10.6
    

    QUESTION: Pace of change in Terraform?

    Release 0.6.8 (2.12.2015)

    TODO: Update terraform

    Commands list & help

  5. For a list of commands:

    terraform

    The response at time of writing:

    Usage: terraform [--version] [--help] <command> [args]
     
    The available commands for execution are listed below.
    The most common, useful commands are shown first, followed by
    less common or more advanced commands. If you're just getting
    started with Terraform, stick with the common commands. For the
    other commands, please read the help and docs before usage.
     
    Common commands:
     apply              Builds or changes infrastructure
     console            Interactive console for Terraform interpolations
     destroy            Destroy Terraform-managed infrastructure
     env                Workspace management
     fmt                Rewrites config files to canonical format
     get                Download and install modules for the configuration
     graph              Create a visual graph of Terraform resources
     import             Import existing infrastructure into Terraform
     init               Initialize a Terraform working directory
     output             Read an output from a state file
     plan               Generate and show an execution plan
     providers          Prints a tree of the providers used in the configuration
     push               Upload this Terraform module to Atlas to run
     refresh            Update local state file against real resources
     show               Inspect Terraform state or plan
     taint              Manually mark a resource for recreation
     untaint            Manually unmark a resource as tainted
     validate           Validates the Terraform files
     version            Prints the Terraform version
     workspace          Workspace management
     
    All other commands:
     debug              Debug output management (experimental)
     force-unlock       Manually unlock the terraform state
     state              Advanced state management
    
  6. Help on a specific command:

    terraform plan --help

    Environment variables

  7. Define enviornment variables:

    TF_VAR_first_name=John terraform apply
    
  8. Define Terraform variable:

    terraform apply -var 'first_name=John' -var 'last_name=Bunyan'
    

    Values to Terraform variables define inputs such as run-time DNS/IP addresses into Terraform modules.

    NOTE: Built-in functions:

    https://terraform.io/docs/configuration/interpolation.html

    Apps to install

    NOTE: Software can be specified for installation using Packer’s local-exec provisioner which has Terraform on host machines executes commands. For example, on a Ubuntu machine:

    resource "null_resource" "local-software" {
      provisioner "local-exec" {
     command = <<EOH
    sudo apt-get update
    sudo apt-get install -y ansible
    EOH
      }
    }
    

    NOTE: apt-get is in-built within Ubuntu Linux distributions.

    PROTIP: Use this to bootstrap automation such as assigning permissions and running Ansible or PowerShell DSC, then use DSC scripts for more flexibility and easier debugging.

    Output variables

  9. Output Terraform variable:

    output "loadbalancer_dns_name" {
      value = "${aws_elb.loadbalancer.dns_name}"
    }
    

    Processing flags

    HCL can contain flags that affect processing. For example, within a resource specification, force_destroy = true forces the provider to delete the resource when done.

    Verify websites

  10. The website accessible?

  11. In the provider’s console (EC2), verify

    Destroy to clean up

  12. Destroy instances so they don’t rack up charges unproductively:

    terraform destroy

    PROTIP: Amazon charges by the hour while others charge by the minute.

  13. Verify in the provider’s console.

Terraform Console

  1. On Windows, open the Terraform Console from a command Prompt rather than from within PowerShell.

    terraform console

    element(list(“one”,”two”,”three”),0,2)

    The response is (because counting begins from zero):

    three

    element(list(“one”,”two”,”three”),0,2)

    one, two

Automation

NOTE: Automating infrastructure deployment consists of:

  • Provisioning resources
  • Planning updates
  • Using source control
  • Reusing templates

PROTIP: Terrform files are “idempotent” (repeat runs don’t change anything if nothing is changed). Thus Terraform defines the “desired state configuration”.

NOTE: Terraform remote configures remote state storage with Terraform.

Modules

https://github.com/terraform-community-modules

https://github.com/objectpartners/tf-modules

module "rancher" {
  source = "github.com/objectpartners/tf-modules//rancher/server-standalone-elb-db&ref=9b2e590"
}
   

The double slashes separate the repo from the subdirectory.

The ref is a commit ID

Bootstrap Terraform

https://forge.puppet.com/inkblot/terraform

https://supermarket.chef.io/cookbooks/terraform

https://github.com/migibert/terraform-role Ansible role to install Terraform on Linux machines

https://github.com/hashicorp/docker-hub-images/tree/master/terraform builds Docker containers for using the terraform command line program.

Plugins into Terraform

All Terraform providers are plugins - multi-process RPC (Remote Procedure Calls).

https://github.com/hashicorp/terraform/plugin

https://terraform.io/docs/plugins/index.html

Terraform expect plugins to follow a very specific naming convention of terraform-TYPE-NAME. For example, terraform-provider-aws, which tells Terraform that the plugin is a provider that can be referenced as “aws”.

PROTIP: Establish a standard for where plugins are located:

For *nix systems, ~/.terraformrc

For Windows, %APPDATA%/terraform.rc

https://www.terraform.io/docs/internals/internal-plugins.html

PROTIP: When writing your own terraform plugin, create a new Go project in GitHub, then locally use a directory structure:

$GOPATH/src/github.com/USERNAME/terraform-NAME

where USERNAME is your GitHub username and NAME is the name of the plugin you’re developing. This structure is what Go expects and simplifies things down the road.

TODO:

  • Grafana or Kibana monitoring
  • PagerDuty alerts
  • DataDog metrics

Learning Resources

Video courses at Pluralsight.com:

YouTube videos:

Rock Stars

James Turnbull

https://github.com/turnbullpress/tfb-code

https://github.com/jason-azze/tf-web-exercise

James Nugent

Engineer at Hashicorp

Yevgeniy (Jim) Brikman (ybrikman.com), co-founder of DevOps as a Service Gruntwork.io :

zero-downtime deployment, are hard to express in purely declarative terms.

Comprehensive Guide to Terraform includes:

  1. Why we use Terraform and not Chef, Puppet, Ansible, SaltStack, or CloudFormation

  2. How to manage Terraform state

  3. Terraform tips & tricks: loops, if-statements, and gotchas

dtan4

http://terraforming.dtan4.net/

https://github.com/dtan4/terraforming is a Ruby

AWS Cloud Formation

Puppet, Chef, Ansible, Salt AWS API libraries Boto, Fog

References

https://github.com/brikis98/infrastructure-as-code-talk Infrastructure-as-code: running microservices on AWS with Docker, ECS, and Terraform

https://www.youtube.com/channel/UCgWfCzNeAPmPq_1lRQ64JtQ/videos SignalWarrant’s videos on PowerShell by David Keith Hall includes:

More on DevOps

This is one of a series on DevOps:

  1. DevOps_2.0
  2. ci-cd (Continuous Integration and Continuous Delivery)
  3. User Stories for DevOps

  4. Git and GitHub vs File Archival
  5. Git Commands and Statuses
  6. Git Commit, Tag, Push
  7. Git Utilities
  8. Data Security GitHub
  9. GitHub API
  10. TFS vs. GitHub

  11. Choices for DevOps Technologies
  12. Java DevOps Workflow
  13. AWS DevOps (CodeCommit, CodePipeline, CodeDeploy)
  14. AWS server deployment options

  15. Digital Ocean
  16. Cloud regions
  17. AWS Virtual Private Cloud
  18. Azure Cloud Onramp
  19. Azure Cloud
  20. Azure Cloud Powershell

  21. Packer automation to build Vagrant images
  22. Terraform multi-cloud provisioning automation

  23. Powershell Ecosystem
  24. Powershell on MacOS
  25. Powershell Desired System Configuration

  26. Jenkins Server Setup
  27. Jenkins Plug-ins
  28. Jenkins Freestyle jobs
  29. Jenkins2 Pipeline jobs using Groovy code in Jenkinsfile

  30. Dockerize apps
  31. Docker Setup
  32. Docker Build

  33. Maven on MacOSX

  34. Ansible

  35. MySQL Setup

  36. SonarQube static code scan

  37. API Management Microsoft
  38. API Management Amazon

  39. Scenarios for load