Your robot butler is simple, but not stupid
- How Ansible works
- Python Ansible Control Server on Linux, not Windows
- Install sample environment
- Windows support
- Config. settings
- Include files
- Daemon Sets
- Rolling updates
- Social Community
- From Ansible.com
- More on DevOps
The object of this tutorial is to succintly present, with step-by-step instructions but without much marketing hype, how to make use of Ansible to install software on servers.
Ansible was written in Python 2.6 and open-sourced under the ansible organization on GitHub. So to build Ansible from source (containing sub-modules):
git clone https://github.com/ansible/ansible.git --recursive cd ./ansible make rpm sudo rpm -Uvh ./rpm-build/ansible-*.noarch.rpm
The GitHub Octoverse report has dentified Ansible as being among the top 10 projects with most contributors and most discussed projects.
Its documentation is at ansible.com.
The @ansible Twitter account is titled “Red Hat Ansible” because it’s initiating author, Michael DeHaan from North Carolina (@laserllama), began writing Ansible in his spare time while working at RedHat. Quotes from his lightning talk at All Things Open Dec 3, 2014:
- “Your IT infrastructure should be boring”
- “How do we get sysadmins and developers together to cheat off each other, even at competing companies”
- “Automation should not be your day job”
- “Build early and often. Build a culture of testing.”
- “Have Dev/QA/Stage environments that mirror production to see what can go wrong early.”
See the slides to his video “Python-Powered Radically Simple IT Automation” at PyCon 2014:
How Ansible works
Ansible’s Control Server (acs) communicates with servers to download and provision software locally in them.
The name “ansible” is popularized by the science-fiction book and movie Enders Game which uses an “ansible” to communicate, in real-time, with many ships at once, galaxies away.
One of the distinguishing technologies Ansible uses the SSH (secure shell) protocol (OpenSSH program) in all Linux distributions. Windows Remote Management (WinRM) is used to connect with Windows (from Vista SP1 or Windows 2008 R1 and up).
With SSH on port 22, one does not need to beg for special ports to be opened through the enterprise firewall, which one needs to do with Chef and Puppet agents.
Use of SSH does require use of the ssh-keygen program to create private/public key pairs.
By default, JSON messages are communicated back to the Control Server’s API listening on standard port 80. Internally, Ansible uses the Django-REST framework, PyYAML.
But plug-ins can be installed so Ansible can communicate via ZeroMQ “fireball mode” or other means.
Ansible works with only text files, so it does not add a database nor the daemons to start or keep a database running.
New major releases of Ansible come out approximately every two months, with release cycles of about four months.
Python Ansible Control Server on Linux, not Windows
Ansible Control Server core is written in Python 2.6+. Thus, it can run natively on NIX (Linux/Unix/Mac), but Windows not currently supported nor recommended.
The “ansible_python_interpreter” variable in inventory points to the Python executable folder.
However, run virtual instances on a Windows, Mac, or other native OS if you want to use them to run Ansible.
Additionally, licensed product Ansible Tower runs playbooks for enterprises.
.yml files in Git vs. database
Instead of a database server, Ansible stores declarations in text files of yml (yamil) format that are both human and machine readable.
These Playbooks can be edited by any text editor.
Playbooks from others are available as roles in the Ansible Galaxy community website.
Being text files, most enterprises put Ansible configuration files in a Git repository (such as GitHub or Bitbucket) to maintain back versions for the team.
Modules do the work within the server are invoked by tasks specified in plays.
Modules can apply plays on several servers defined in an inventory file which can be dynamically generated from an enterprise CMDB (Configuration Management DataBase) cataloguing assets in AWS, Azure, GCP, or private cloud.
Additional modules can be defined, such as for building assets within AWS using CloudFormation.
Windows modules include win_feature (to installs and uninstall Windows Features) and win_regedit (Add, Edit, or Remove Registry Keys and Values). WinRM python module
Let’s look at a playbook with full annotations:
Ansible works under the concept of “idempotance”, where repeated executions of the same script results in the same state at the end of each run. If something doesn’t exist, it is created. If something does exist already, it is left alone and another isn’t created.
Ansible reads declarations of desired state (what is wanted after processing) rather than imperative programming commands (to do this and that in a specified sequence).
This is like when you get in a taxi and you provide a destination address rather than directions to that location.
This makes definitions more reusable.
This yaml file launches the hello.ps1 PowerShell script:
- name: Run Powershell Scripts hosts: test tasks: - name: run a powershell script script: scripts/hello.ps1 register: out - debug: var=out
To execute the script, run:
ansible-playbook powershell.yml -i hosts
Using Ansible for your Windows Configuration Management by David O’Brien (by @david_OBrien, david-obrien.net) at NICconf 24 Feb 2016.
Ansible with Cloud Formation
Yan Kurniawan’s book provides Ansible playbook these procedural examples:
- sg_empty.yml to create empty security groups.
- sg_modify.yml to modify security groups for each type of server
- ec2_vpc_web_create.yml to launch an instance in a particular subnet
- ec2_vpc_db_create.yml without assigning a public IP address
- nat_launch.yml to launch a “staging_nat” paravirtual t1.micro instance (with AMI name that includes “amzn-ami-vpc-nat”)
- ec2_vpc_jumpbox.yml to launch jump box instance in public subnet A
- ansible -i ec2.py tag_class_jumpbox -m ping
- sg_openvpn.yml still requires manual retrieval of the AMI ID on https://openvpn.net/index.php/access-server/docs.html
The book provides an Ansible module in library/vpc_lookup
- an update of https://github.com/edx/configuration/blob/master/playbooks/library/vpc_lookup (from John Jarvis) to lookup a VPC or subnets ID stored in local (safe) folder based on a particular filter specified in a script.
The suggested hashtag for the book is Tweet #ansible4aws.
PROTIP: Disable host key checking in ssh configuration so ssh will automatically add new host keys to the user known hosts files without asking (the default is “ask”).
- Disable host key checking with StrictHostKeyChecking set to “no” in /etc/ssh/ssh_config file.
View sample configurations
Use an internet browser to open galaxy.ansible.com/explore
Open a sample playbook.
Playbooks are defined in .yml files, which begin with three dashes in the first line.
Playbooks define plays. consisting of one or a set of tasks.
tasks invoke modules.
Tasks trigger handlers which are run on some condition, such as once at the end of plays.
Spaces after dashes and colons are required.
An Ansible Config define Ansible control server configuration.
Encrypted data within playbooks stored in GitHub can be unencrypted in memory using Ansible Vault.
acme/ webserver/ README.md defaults/ files/ handlers/ meta/main.yml tasks/ templates/ tests/ vars/
The main.yml in meta defines dependencies:
Setup Vagrant and Virtualbox
Download and install:
- A virtual image manager from VagrantUp.com (87.9 MB for vagrant_1.8.1.dmg).
- A vm provider (hypervisor) to run virtual machines from Oracle’s VirtualBox
Verify availability from a command-line Terminal:
Create a folder (of any name) for Ansible configuration files. This is typically for a project. This can be in a git folder if you’d like version management.
The ~ (tilde character above) refers to your home folder.
Switch to an internet browser to open a repository of Vagrant server base images:
http://vagrantcloud.com (which redirects to a site owned by hashicorp, who owns Vagrant, thus the advert for the Atlas licensed product)
NOTE: Many enterprises instead use an internal repository.
In the box under “Discover Vagrant Boxes”, search for ubuntu or CentOS, etc.
Choose one and copy its text in blue, such as “nrel/CentOS-6.5-x86_64” from contributor nrel or “ubuntu/trusty64”.
Close down any process making use of port 8080, as that’s Vagrant’s default port. (Jenkins also uses port 8080 by default)
Initialize a Vagrantfile for use by Vagrant:
A `Vagrantfile` has been placed in this directory. You are now ready to `vagrant up` your first virtual environment! Please read the comments in the Vagrantfile as well as documentation on `vagrantup.com` for more information on using Vagrant.
If you have a file named Vagrantfile from another source, copy it into the folder to replace the file generated.
Alternately, open a text editor to create a file name Vagrantfile in end up with this sample content to specific the acs (Ancible Control Server), web, and db servers:
The (2) in Vagrant.configure(2) configures the configuration version.
Names between | (pipe) characters provide handles to identify each server.
Two spaces are used to indent.
Internal IP addresses (192.168.33.xxx) are used in this example.
Change 8080 to another port if it is already used by another process on your computer.
- Navigate to a folder containing a Vagrantfile specification file.
Bring up a machine based on the Vagrantfile in the folder:
This can take several minutes if this is the first time, since images for servers specified need to be downloaded.
Switch to a Finder to see that a .vagrant (hidden) folder has been added. Under the machines folder is a folder for each type specified between pipe characters (acs, web, db, etc).
Open another terminal shell to check what is running:
vboxmanage list runningvms
The response is are hashes:
Provision Ansible Control Server
SSH into the acs server via vagrant:
vagrant ssh acs
This takes several seconds to connect.
This adds the ey to the known_hosts file within the .ssh folder for future reference.
When you’re done:
Use a package manager to download bits. On a CentOS or RHEL server:
sudo yum -y install ansible
Alternately, on a Debian Ubuntu server:
sudo apt-get -y install ansible
Notice the log says Python is installed as well.
Provision web server
SSH into the web server via vagrant:
vagrant ssh web
Use a package manager to download bits:
sudo yum -y install epel-release
Install by Compiling Source Code
Install the C compiler used with Python:
sudo yum install gcc
sudo yum install python-setuptools
sudo easy_install pip
sudo yum install python-devel
sudo pip install ansible
Ansible tasks are commands executed from command line terminals.
Tasks are shereable and repeatable.
Steps Modules do
- Gather facts on hosts into variables such as ansible_os_family.
- Fetch md5 checksum from remote to verify downloaded file
- Create and manage local users and groups
Enable and disable OS features and preferences
- Fetch files from remote sites
- Install software (web server, app server, database, virus scanner, etc.)
Update software security patches
- Copy app configurations
- Copy files into server
Call databases to retrieve data
- Enable service to start on reboot
- Start web service
- Deploy load balancer configurations (put in or take out server on rotation)
Install sample environment
Ansible covers more functionality:
- Provisioning - install software, patch security, copy files in, customize configurations, start web service.
- Change management of configurations with configuration remediation.
- Automation - make decisions. A single change can impact several machines.
- Complex Orchestration of dependencies.
Ansible evaluates to mark changed states.
A function is “idempotent” if repeated applications has the same affect as a single application.
The inventory of hosts can be defined within /etc/ansible/hosts containing .ini format Microsoft uses or yml format:
An example of ini format:
[webservers] 192.168.33.20 192.168.33.30 ansible_connection=ssh ansible_user=mpdehaan [db] db-a.example.com [lbservers] lbserver http_port=80 maxRequestsPerChild=808 [monitoring] nagios
Items in square brackets define group names.
To get the status of servers under [webservers] in the inventory file above:
ansible webservers -m ping
In addition to this ad-hoc run, Ansible can be run based on the contents of Playbooks with a command such as:
Add -v for more detailed response.
Such inventory files are typically kept in a Git repository.
Modules in various languages
Unlike Puppet, Ansible does not require agent software to be installed and thus potentially leave residual bits on servers.
Modules are the “brains” of Ansible.
Various modules running on remote hosts provide the plumbing for other networking protocols, such as HTTP, runing on remote machines.
List of available modules, or locally:
Press q to quit list, cursor up/down individual line, or space bar to page down.
Responses returned to the Ansible Control Server are in JSON messages.
Modules (hopefully written by following Module Development Guide) can be selected from various sources:
ansible-modules-core are writtin Python.
ansible-modules-extras developed by others have slightly lower use or priority.
Ansible Module development can be in any dynamic language, not just Python on the server.
- Simplejson library on *NIX.
Ansible’s native Windows support uses Windows PowerShell remoting to manage Windows like Windows in the same Ansible agentless way that Ansible manages Linux like Linux.
Windows Remote PowerShell 2.0 enabled.
- Push and execute any PowerShell scripts you write
Play behavior can be controlled several ways:
with_items, failed_when, changed_when, until, ignore_errors
Register Output to Variable
To capture the result or output of a task so that follow-on tasks can act accordingly:
NOTE: .j2 files are processed by Jinja2, the template engine for Python, which replace variables with data values in static files.
To set a register to put result in a variable, then if the debug sees that a previous task failed, it would send a message.
NOTE: Handlers don’t run until all playbook tasks have executed.
NOTE: A particular handler only executes once if needed.
NOTE: Handlers don’t run until all playbook tasks have executed.
The precedence Ansible looks for configuration variables. (stop searching once it finds one):
- $ANSIBLE_CONFIG environment variable
- ./ansible_cfg in current directory
- ~/ansible.cfg (home directory of currently logged in account)
- /etc/ansible/ansible.cfg global config. file
An example $ANSIBLE_CONFIG environment variable from the full list is:
This sets the maximum number of parallel operations allowed on an Ansible server, determined through performance and capacity testing.
Scalyr.com has a DaemonSet for Kubernetes monitoring.
Ansible achieves zero-downtime deployments with multi-tear rolling updates to each specific node in a cluster.
This specifies taking 5 machines at a time out of a cluster:
- hosts: webservers serial: 5 pre_tasks: - name: take out of load balancer pool local_action: command /usr/bin/take_out_of_pool roles: - common - webserver - monitored post_tasks: - name: add back to load balancer pool local_action: command /usr/bin/add_back_to_pool
- Twitter: @ansible by Red Hat, @robynbergeron
- On a IRC client, select Destination: Freenode, and add channel #ansible.
- AnsibleFest (SF July 28, 2016)
- Ansible-Galaxy.com/explore/ is the community hub to find and share reusable Ansible content.
- Link to GitHub https://galaxy.ansible.com/accounts/github/login/
- Confirm email.
Within Ansible’s YouTube channel
- ansible.com/quick-start-video provide your email because it is a high-level, high-flautin’ marketing pitch which introduces Ansible Tower proprietary software.
Tim Gerla of Ansibleworks:
- Continuous Deployment with Ansible USENIX 38:38 on 11 Jul 2013
This tutorial presents related material in a different sequence for better understanding and updated.
Hands-on Ansible Pluralsight 3h 53m video course 29 Dec 2015 by Aaron Paxson | @Neelixx | myteneo.net
Continuous Delivery Using Docker and Ansible Pluralsight video course 10 May 2016 by Justin Menga (@jmenga | pseudo.co.de) in 7h 13m creates Ansible files for a Python “TO DO” app using the Django REST framework. The files create Docker image.
Ansible and Docker by Patrick Galbraith from HP’s Advanced Technologies group.
Ansible Hands-On Training by Glen Jarvis
Getting Started With Jenkins edureka!
Ansible 101 - on a Cluster of Raspberry Pi 2s by Jeff Geerling
http://devopsguys.com in the UK
To enable Python to talk with Windows WinRM:
sudo pip install pywinrm
Test whether a connection can be made:
https://github.com/PowerShell/PowerShell/blob/master/docs/KNOWNISSUES.md#remoting-support (WinRM does not run within MacOS 10) PowerShell https://quizlet.com/178078947/ansible-devops-automation-mamun-flash-cards/
https://github.com/sthulb-attic/laptop-osx by https://twitter.com/sthulb an Amazon Solution Architect
https://github.com/monfresh/laptop Those are my personal playbooks and scripts to install a laptop from scratch including some dotfiles. Based on Fedora 27 Not idempotent
https://github.com/vaskas/laptop-ansible for Fedora 26
More on DevOps
This is one of a series on DevOps:
- ci-cd (Continuous Integration and Continuous Delivery)
- Git and GitHub vs File Archival
- Git Commands and Statuses
- Git Commit, Tag, Push
- Git Utilities
- Data Security GitHub
- GitHub API
- Choices for DevOps Technologies
- Java DevOps Workflow
- AWS DevOps (CodeCommit, CodePipeline, CodeDeploy)
- Cloud regions
- AWS Virtual Private Cloud
- Azure Cloud Onramp
- Azure Cloud
- Azure Cloud Powershell
- Digital Ocean
- Packer automation to build Vagrant images
Terraform multi-cloud provisioning automation
- Powershell Ecosystem
- Powershell on MacOS
- Jenkins Server Setup
- Jenkins Plug-ins
- Jenkins Freestyle jobs
- Dockerize apps
- Docker Setup
- API Management Microsoft
- Scenarios for load