Wilson Mar bio photo

Wilson Mar

Hello. Hire me!

Email me Calendar Skype call 310 320-7878

LinkedIn Twitter Gitter Google+ Youtube

Github Stackoverflow Pinterest

Your robot butler is simple, but not stupid


Overview

The object of this tutorial is to succintly present, with step-by-step instructions but without much marketing hype, how to make use of Ansible to install software on servers.

How Ansible works

Ansible’s Control Server (acs) communicates with servers to download and provision software locally in them.

The name “ansible” is popularized by the science-fiction book and movie Enders Game which uses an “ansible” to communicate, in real-time, with many ships at once, galaxies away.

One of the distinguishing technologies Ansible uses is the simplicity of it using the standard SSH (secure shell) protocol built into all Linux distributions. Windows Remote Management (WinRM) is used to connect with Windows (from Vista SP1 or Windows 2008 R1 and up).

So one does not need to beg for special ports to be opened through the enterprise firewall, which one needs to do with Chef and Puppet.

By default, JSON messages are communicated back to the Control Server’s API listening on standard port 80. Internally, Ansible uses the Django-REST framework, PyYAML.

But plug-ins can be installed so Ansible can communicate via ZeroMQ “fireball mode” or other means.

.yml files in Git vs. database

Instead of a database server, Ansible stores declarations in text files of yml (yamil) format that are both human and machine readable.

These Playbooks can be edited by any text editor.

Playbooks from others are available as roles in the Ansible Galaxy community website.

Being text files, most enterprises put Ansible configuration files in a Git repository (such as GitHub or Bitbucket) to maintain back versions for the team.

Modules do the work within the server are invoked by tasks specified in plays.

Modules can apply plays on several servers defined in an inventory file which can be dynamically generated from an enterprise CMDB (Configuration Management DataBase) cataloguing assets in AWS, Azure, GCP, or private cloud.

Additional modules can be defined, such as for building assets within AWS using CloudFormation.

Windows modules include win_feature (to installs and uninstall Windows Features) and win_regedit (Add, Edit, or Remove Registry Keys and Values). WinRM python module

To recap:


Let’s look at a playbook with full annotations:

Ansible works under the concept of “idempotance”, where repeated executions of the same script results in the same state at the end of each run. If something doesn’t exist, it is created. If something does exist already, it is left alone and another isn’t created.

Ansible reads declarations of desired state (what is wanted after processing) rather than imperative programming commands (to do this and that in a specified sequence).

This is like when you get in a taxi and you provide a destination address rather than directions to that location.

This makes definitions more reusable.

Windows

This yaml file launches the hello.ps1 PowerShell script:

- name: Run Powershell Scripts
  hosts: test
  tasks:
    - name: run a powershell script
      script: scripts/hello.ps1
      register: out
    - debug: var=out
   

The script:

https://github.com/dstamen/Ansible/tree/master/ansible_powershell

See http://davidstamen.com/ansible/using-ansible-to-run-powershell-scripts/

To execute the script, run:

ansible-playbook powershell.yml -i hosts

Videos:

Alternatives

This mentions the free Remote Execution Enabler for PowerShell tool from Solarwinds.

Ansible with Cloud Formation

Yan Kurniawan’s book provides Ansible playbook these procedural examples:

  • vpc_create.yml
  • sg_empty.yml to create empty security groups.
  • sg_modify.yml to modify security groups for each type of server
  • sg_delete.yml
  • ec2_vpc_web_create.yml to launch an instance in a particular subnet
  • ec2_vpc_db_create.yml without assigning a public IP address
  • nat_launch.yml to launch a “staging_nat” paravirtual t1.micro instance (with AMI name that includes “amzn-ami-vpc-nat”)
  • vpc_delete.yml
  • vpc_create_multi_az.yml
  • sg_jumpbox.yml
  • ec2_vpc_jumpbox.yml to launch jump box instance in public subnet A
  • ansible -i ec2.py tag_class_jumpbox -m ping
  • sg_openvpn.yml still requires manual retrieval of the AMI ID on https://openvpn.net/index.php/access-server/docs.html

The book provides an Ansible module in library/vpc_lookup

  • an update of https://github.com/edx/configuration/blob/master/playbooks/library/vpc_lookup (from John Jarvis) to lookup a VPC or subnets ID stored in local (safe) folder based on a particular filter specified in a script.

The suggested hashtag for the book is Tweet #ansible4aws.

PROTIP: Disable host key checking in ssh configuration so ssh will automatically add new host keys to the user known hosts files without asking (the default is “ask”).

  • Disable host key checking with StrictHostKeyChecking set to “no” in /etc/ssh/ssh_config file.

View sample configurations

  1. Use an internet browser to open galaxy.ansible.com/explore

  2. Search.

  3. Open a sample playbook.

    Playbooks are defined in .yml files, which begin with three dashes in the first line.

    Playbooks define plays. consisting of one or a set of tasks.

    tasks invoke modules.

    Tasks trigger handlers which are run on some condition, such as once at the end of plays.

    Spaces after dashes and colons are required.

An Ansible Config define Ansible control server configuration.

Notice the repos downloaded more than anyone is from @geerlingguy, Jeff Geerling (jeffgeerling.com) has been contributing to Ansible community since early 2013, and wrote ansiblefordevops.com.

Encrypted data within playbooks stored in GitHub can be unencrypted in memory using Ansible Vault.

Roles

Role files encapsulate instructions on how do a discrete unit of work, such as building a webserver. A role file contains for each type of resource tasks, variables, configuration templates, etc.

acme/
  webserver/
    README.md
    defaults/
    files/
    handlers/
    meta/main.yml
    tasks/
    templates/
    tests/
    vars/

The main.yml in meta defines dependencies:

---
galaxy_info:
  author: John Doe
  description: Quick and easy acme web installer.
  company: Acme
  license: MIT
  min_ansible_version: 1.9
  platforms:
  - name: EL
    versions:
    - all
  galaxy_tags:
    - acme
    - installer
    - web
dependencies:
  - { role: username.common, some_parameter: 3 }
  - { role: username.iptables, open_port: 80 }

On Linux, not Windows

Ansible Control Server core is written in Python 2.6+ (not 3.0). Thus, it can run natively on *NIX (Linux/Unix/Mac) - Windows not currently supported nor recommended.

However, run virtual instances on a Windows, Mac, or other native OS if you want to use them to run Ansible.

Setup Vagrant and Virtualbox

  1. Download and install:

    • A virtual image manager from VagrantUp.com (87.9 MB for vagrant_1.8.1.dmg).
    • A vm provider (hypervisor) to run virtual machines from Oracle’s VirtualBox

     

  2. Verify availability from a command-line Terminal:

    vagrant
    vboxmanager

  3. Create a folder (of any name) for Ansible configuration files. This is typically for a project. This can be in a git folder if you’d like version management.

    cd ~
    mkdir ansible
    cd ansible

    The ~ (tilde character above) refers to your home folder.

  4. Switch to an internet browser to open a repository of Vagrant server base images:

    http://vagrantcloud.com (which redirects to a site owned by hashicorp, who owns Vagrant, thus the advert for the Atlas licensed product)

    NOTE: Many enterprises instead use an internal repository.

  5. In the box under “Discover Vagrant Boxes”, search for ubuntu or CentOS, etc.

  6. Choose one and copy its text in blue, such as “nrel/CentOS-6.5-x86_64” from contributor nrel or “ubuntu/trusty64”.

  7. Close down any process making use of port 8080, as that’s Vagrant’s default port. (Jenkins also uses port 8080 by default)

  8. Initialize a Vagrantfile for use by Vagrant:

    vagrant init

    Sample response:

    A `Vagrantfile` has been placed in this directory. You are now
    ready to `vagrant up` your first virtual environment! Please read
    the comments in the Vagrantfile as well as documentation on
    `vagrantup.com` for more information on using Vagrant.
    
  9. If you have a file named Vagrantfile from another source, copy it into the folder to replace the file generated.

    Alternately, open a text editor to create a file name Vagrantfile in end up with this sample content to specific the acs (Ancible Control Server), web, and db servers:

   Vagrant.configure(2) do |config|

     config.vm.define "acs" do |acs|
       acs.vm.box = "nrel/CentOS-6.5-x86_64"
       acs.vm.hostname = "acs"
       acs.vm.network "private_network", ip: "192.168.33.10"
     end

     config.vm.define "web" do |web|
       web.vm.box="nrel/CentOS-6.5-x86_64"
       web.vm.hostname = "web"
       web.vm.network "private_network", ip: "192.168.33.20"
       web.vm.network "forwarded_port", guest: 80, host: 8080
     end

     config.vm.define "db" do |db|
       db.vm.box = "nrel/CentOS-6.5-x86_64"
       db.vm.hostname = "db"
       db.vm.network "private_network", ip: "192.168.33.30"
     end
   end
  

The (2) in Vagrant.configure(2) configures the configuration version.

Names between | (pipe) characters provide handles to identify each server.

Two spaces are used to indent.

Internal IP addresses (192.168.33.xxx) are used in this example.

Change 8080 to another port if it is already used by another process on your computer.

Vagrant up

  1. Navigate to a folder containing a Vagrantfile specification file.
  2. Bring up a machine based on the Vagrantfile in the folder:

    vagrant up

    This can take several minutes if this is the first time, since images for servers specified need to be downloaded.

  3. Switch to a Finder to see that a .vagrant (hidden) folder has been added. Under the machines folder is a folder for each type specified between pipe characters (acs, web, db, etc).

  4. Open another terminal shell to check what is running:

    vboxmanage list runningvms

    The response is are hashes:

   "ansible_acs_1463860205025_4852" {128ce450-8384-4adb-a4fd-7f4ac5c1f0b8}
   "ansible_web_1463862332570_44406" {dd044db3-ecf1-4b9b-9c42-96952172bd4d}
   "ansible_db_1463882256962_22323" {411c8704-f220-4188-8b94-d1bfb093e1b4}
   

Provision Ansible Control Server

  1. SSH into the acs server via vagrant:

    vagrant ssh acs

    This takes several seconds to connect.

    This adds the ey to the known_hosts file within the .ssh folder for future reference.

  2. When you’re done:

    exit

  3. Use a package manager to download bits. On a CentOS or RHEL server:

    sudo yum -y install ansible

    Alternately, on a Debian Ubuntu server:

    sudo apt-get -y install ansible

    Notice the log says Python is installed as well.

  4. Verify:

    ansible --version

    Provision web server

  5. SSH into the web server via vagrant:

    vagrant ssh web

  6. Use a package manager to download bits:

    sudo yum -y install epel-release

Install by Compiling Source Code

  1. Install the C compiler used with Python:

    sudo yum install gcc

    sudo yum install python-setuptools

    sudo easy_install pip

    sudo yum install python-devel

    sudo pip install ansible

Tasks

Ansible tasks are commands executed from command line terminals.

Tasks are shereable and repeatable.

Steps Modules do

  1. Gather facts on hosts into variables such as ansible_os_family.
  2. Fetch md5 checksum from remote to verify downloaded file
  3. Create and manage local users and groups
  4. Enable and disable OS features and preferences

  5. Fetch files from remote sites
  6. Install software (web server, app server, database, virus scanner, etc.)
  7. Update software security patches

  8. Copy app configurations
  9. Copy files into server
  10. Call databases to retrieve data

  11. Enable service to start on reboot
  12. Start web service
  13. Deploy load balancer configurations (put in or take out server on rotation)

Install sample environment

Install:

Ansible covers more functionality:

  • Provisioning - install software, patch security, copy files in, customize configurations, start web service.
  • Change management of configurations with configuration remediation.
  • Automation - make decisions. A single change can impact several machines.
  • Complex Orchestration of dependencies.

Ansible evaluates to mark changed states.

A function is “idempotent” if repeated applications has the same affect as a single application.

Inventory file

An example:

   [webservers]
   192.168.33.20
   192.168.33.30

   [db]
   db-a.example.com

   [lbservers]
   lbserver

   [monitoring]
   nagios
   

To get the status of servers under [webservers] in the inventory file above:

ansible webservers -m ping

In addition to this ad-hoc run, Ansible can be run based on the contents of Playbooks with a command such as:

ansible-playbook file.yml

Add -v for more detailed response.

Ansible Tower for more

Additionally, licensed product Ansible Tower runs playbooks for enterprises.

Next, let’s look at examples of some Ansible playbook files.

Modules in various languages

Unlike Puppet, Ansible does not require agent software to be installed and thus potentially leave residual bits on servers.

Modules are the “brains” of Ansible.

Various modules running on remote hosts provide the plumbing for other networking protocols, such as HTTP, runing on remote machines.

List of available modules, or locally:

ansible-doc -l

Press q to quit list, cursor up/down individual line, or space bar to page down.

Responses returned to the Ansible Control Server are in JSON messages.

Modules (hopefully written by following Module Development Guide) can be selected from various sources:

Ansible Module development can be in any dynamic language, not just Python on the server.

  • Simplejson library on *NIX.

Windows support

Ansible’s native Windows support uses Windows PowerShell remoting to manage Windows like Windows in the same Ansible agentless way that Ansible manages Linux like Linux.

  • Windows Remote PowerShell 2.0 enabled.

  • Windows modules

    • Push and execute any PowerShell scripts you write

Playbooks

Play behavior can be controlled several ways:

   with_items, 
   failed_when, 
   changed_when, 
   until, 
   ignore_errors
   

Register Output to Variable

To capture the result or output of a task so that follow-on tasks can act accordingly:

  tasks:
    - shell: /usr/bin/whoami
      register: username
    - debug: msg="Host=, User="
    - file: path=/home/myfile.txt
            owner=

Conditional Handlers

An example:

  tasks:
    - name: Deploy configuration file
      template: src=templates/httpd.j2 dest=/etc/httpd/conf/httpd.conf
      notify:
        - Restart Apache
  handlers:
    - name: Restart Apache
      service: name=httpd state=restarted

NOTE: .j2 files are processed by Jinja2, the template engine for Python, which replace variables with data values in static files.

To set a register to put result in a variable, then if the debug sees that a previous task failed, it would send a message.

  tasks:
    - command: ls /bad/path
      register: result
      ignore_errors: yes

    - debug: msg="Failure!"
      when: result|failed

NOTE: Handlers don’t run until all playbook tasks have executed.

  tasks:
    - copy: src=files/httpd.conf
            dest=/etc/httpd/conf/
      notify:
        - Apache Restart
  handlers:
    - name: Apache Restart
      service: name=httpd state=restarted

NOTE: A particular handler only executes once if needed.

NOTE: Handlers don’t run until all playbook tasks have executed.

Config. settings

The precedence Ansible looks for configuration variables. (stop searching once it finds one):

  1. $ANSIBLE_CONFIG environment variable
  2. ./ansible_cfg in current directory
  3. ~/ansible.cfg (home directory of currently logged in account)
  4. /etc/ansible/ansible.cfg global config. file

An example $ANSIBLE_CONFIG environment variable from the full list is:

   $ANSIBLE_FORKS=5
   

This sets the maximum number of parallel operations allowed on an Ansible server, determined through performance and capacity testing.

Include files

  tasks:
    - include: wordpress.yml
      vars:
        sitename: My Site
    - include_vars: variables.yml

Community

  • Twitter: @ansible by Red Hat, @robynbergeron
  • https://groups.google.com/forum/#!forum/ansible-announce
  • On a IRC client, select Destination: Freenode, and add channel #ansible.
  • AnsibleFest (SF July 28, 2016)
  • Ansible-Galaxy.com/explore/ is the community hub to find and share reusable Ansible content.
  1. Link to GitHub https://galaxy.ansible.com/accounts/github/login/
  2. Confirm email.

Roles

  • https://bitbucket.org/fquffio/ansible-elasticsearch/src
  • https://bitbucket.org/fquffio/ansible-kibana/src
  • https://bitbucket.org/fquffio/ansible-iptables

Rolling updates

Ansible achieves zero-downtime deployments with multi-tear rolling updates to each specific node in a cluster.

This specifies taking 5 machines at a time out of a cluster:

   - hosts: webservers
     serial: 5

   pre_tasks:

   - name: take out of load balancer pool
     local_action: command /usr/bin/take_out_of_pool 

  roles:
   - common
   - webserver
   - monitored

  post_tasks:
   - name: add back to load balancer pool
     local_action: command /usr/bin/add_back_to_pool 
   

Popularity

Ansible is open-sourced under the github.com/ansible organization on GitHub. The repo is among the top 10 Python projects.

The GitHub Octoverse report for 2013 featured Ansible as being #5 on the list of open source projects with the most contributors.

Its documentation is at ansible.com.

The @ansible Twitter account is titled “Red Hat Ansible” because it’s initiating author, Michael DeHaan from North Carolina (@laserllama), began writing Ansible in his spare time while working at RedHat. Quotes from his lightning talk at All Things Open Dec 3, 2014:

  • “Your IT infrastructure should be boring”
  • “How do we get sysadmins and developers together to cheat off each other, even at competing companies”
  • “Automation should not be your day job”
  • “Build early and often. Build a culture of testing.”
  • “Have Dev/QA/Stage environments that mirror production to see what can go wrong early.”

See the slides to his video “Python-Powered Radically Simple IT Automation” at PyCon 2014:

  • https://www.youtube.com/watch?time_continue=80&v=Qi0AhK7PMCI

From Ansible.com

Within Ansible’s YouTube channel

  • ansible.com/quick-start-video provide your email because it is a high-level, high-flautin’ marketing pitch which introduces Ansible Tower proprietary software.

Tim Gerla of Ansibleworks:

Videos

This tutorial presents related material in a different sequence for better understanding and updated.

Gwyn Price:

Others:

Misc

To enable Python to talk with Windows WinRM:

sudo pip install pywinrm

  1. Test whether a connection can be made:

    Test-WsMan 192.168.5.3

    https://github.com/PowerShell/PowerShell/issues/1883

https://github.com/PowerShell/PowerShell/blob/master/docs/KNOWNISSUES.md#remoting-support (WinRM does not run within MacOS 10) PowerShell https://quizlet.com/178078947/ansible-devops-automation-mamun-flash-cards/

https://github.com/PowerShell/psl-omi-provider