Wilson Mar bio photo

Wilson Mar

Hello. Hire me!

Email me Calendar Skype call 310 320-7878

LinkedIn Twitter Gitter Google+ Youtube

Github Stackoverflow Pinterest

Get little bits working together


Overview

The object of this tutorial is to succintly present step-by-step instructions to contrast the setup of Docker on Mac OSX, CentOS Linux, and Windows.

I want you to feel confident that you’ve mastered this skill. That’s why this takes a hands-on approach where you type in commands and we explain the responses and possible troubleshooting. This is a “deep dive” because all details are presented.

Like a good music DJ, I’ve carefully arranged the presentation of concepts into a sequence for easy learning, so you don’t have to spend as much time as me making sense of the flood of material around this subject.

Sentences that begin with PROTIP are a high point of this website to point out wisdom and advice from experience. NOTE point out observations that many miss. Search for them if you only want “TL;DR” (Too Long Didn’t Read) highlights.

Stuck? Contact me and I or one of my friends will help you.

VMs vs. Docker

This is a more complex diagram than others so that interrelationships can be illustrated. TODO: Video of this.

docker flowchart v04-650x312-97kb
Click this for a pop-up containing a larger sized image.

Prior to Docker, a developer working on a Mac Pro can, in addition to native apps for Mac, also run Microsoft Visual Studio for Windows in a full install of Windows 10 by running them within a VMware Fusion hypervisor that manages Virtual Memory instances.

Each additional VM instance would take several more Gigabytes of disk space and memory because the operating system is duplicated inside each VM instance.

By contrast, Docker’s containers take a more lightweight approach. Docker sees its job as not to provide a complete machine but just to cleanly separate applications that need the same oerating system.

Docker for Mac

The Docker for Mac installer is downloaded from Docker’s website. It makes use of hypervisor software that doesn’t need Oracle’s Virtualbox previously needed.

Docker images are obtained from Docker Hub on-line or a more private and trusted image repository. Many images are visible searches within Docker Hub.

That same container image can run on another platform unchanged, typically a GNU/Linux kernel such as Debian running on a commodity server within AWS EC2 cloud. Docker is designed with cloud computing providers in mind.

The Docker Engine takes care of loading images and running them. It is installed from the package manager over the internet.

Docker is lighter weight because it provides the read-only portion of a Linux operating system to be shared among several images. Each image has its own bin and lib folders.

Docker is also lightweight and thus more scalable because each image intends to run and watch one single process. SSH ports are not opened into images.

A data volume outside the image is referenced to read private keys.

Additional data volumes, such as InfluxDB, are shared for logging with an additional image. It receives stats collected by the cAdvisor image (from Google) and structures data for display in a Grafana dashboard.

Images that go into container repositories are created by a build command in Docker for Mac and Windows. The build is based on source code for an app in GitHub having a Dockerfile and .dockerignore file.

Docker for Mac was released in 2016 as a native app that keeps itself updated, so no Homebrew for it as with previous versions. This new version makes use of 64-bit processor in a 2010 or newer Mac, which has Intel’s hardware support for virtualization. The operating system is MacOS Yosemite version 10.10 or newer, such as El Capitan 10.11 or Sierra 10.12.

Docker for Mac uses underlying software not in prior versions, so docker-machine commands that load and manage images depend on the installation of Virtualbox by the Docker Toolbox.

The new Docker for Mac makes obsolete the docker-machine commands, Docker Toolbox, and Virtualbox.

Recap

docker flowchart v04-650x312-97kb
Click this for a pop-up containing a larger sized image.

To recap, Docker containers are created using docker images, built by parsing a Dockerfile containing Docker commands.

“By using containers, resources can be isolated, services restricted, and processes provisioned to have a private view of the operating system with their own process ID space, file system structure, and network interfaces. Multiple containers can share the same kernel, but each container can be constrained to only use a defined amount of resources such as CPU, memory and I/O.” – Wikipedia

Each container runs as an isolated process on a shared kernel.

Competition

http://www.boycottdocker.org/ raises some technical concerns.

Alternatives to Docker:

CoreOS developed appc with a rkt (pronounced “rocket”) implementation in their Open Containers Initiative (OCI).

Canonical of Ubuntu has their LXD


Linux installer downloads

Docker was originally created for different flavors of Linux:

docker-filesystems-multilayer 650x534-211kb

Different operating systems use different file-system software for union-mount.

For example, Debian uses bootfs.

aufs (Advanced multi-layered Unification File System). Wikipedia notes it was rejected for merging into mainline Linux. Its code was criticized for being “dense, unreadable, and uncommented”. Instead, OverlayFS was merged in the Linux kernel.

btrfs (B-tree File System) is intended to address the lack of pooling, snapshots, checksums, and integral multi-device spanning in Linux file systems.

Install Docker on Mac OSX

Docker for Mac was added 2016 for installing Docker on Mac OSX. It uses the HyperKit VM (“xhyve”) to virtualize the Docker Engine environment. That technology requires OS X 10.10.3 Yosemite or newer.

Versions of Mac before Yosemite would need to use the deprecated Docker Toolbox.

NOTE: The version of Linux that comes with Mac isn’t completely compatible with Linux. So an extra layer is needed to emulate a Docker host. That’s Boot2Docker.

  1. If you’ve previously installed Boot2Docker or Docker for Mac, uninstall it by deleting it within your Applications folder.

  2. Get to downloads folder for Mac at:
    https://www.docker.com/products/docker#/mac

    Note “Docker Toolbox” is no longer used, which depended on Virtualbox. At time of writing I had Virtualbox version 5.1.2 installed, so can be left installed. Docs here says VirtualBox prior to version 4.3.30 must NOT be installed (it is incompatible with Docker for Mac).

    Alternately, install https://www.docker.com/products/docker-toolbox if you have an old edition of Mac running OS X 10.8 “Mountain Lion” or later.

  3. Click “Get Docker for Mac” to download to your Downloads folder.
  4. In Finder, navigate to you Download folder to double-click Docker.dmg

    PROTIP: Change the file name if you’re keeping back versions.

    Date of file Version Download Folder
    Oct 17, 2016 Docker.dmg 111 MB 208.1 MB
    July 19, 2016 Docker.dmg 114.0 MB 225.1 MB
    July 19, 2016 Docker.dmg 107.0 MB 225.1 MB
    June 17, 2016 1.12.0-rc2-beta16 113.5 MB 224.1 MB
  5. Double-click on Docker.dmg to open it.
  6. Drag and drop the whale into the Applications folder.
  7. Click Replace the previous version, if applicable.
  8. Click X to dismiss the pop-up.
  9. In the Applications folder, open the Docker app.
  10. Click Next.
  11. Click OK.
  12. Input Apple password.
  13. Click Got it!.

    TECHNICAL NOTE: The Docker command line stores its configuration files in a hidden directory .docker within your $HOME directory (cd ~).

  14. Click the whale icon at the top of your Mac for this menu:

    docker mac 20161110-270x248

  15. Click Preferences.
  16. Un-check “Automatically start Docker when you log in.”
    if you are not a frequent user.

  17. Skip to verify Docker install.

    Previously

    Obsolete instructions to install using Homebrew cask:

    Linux kernel-specific features for the Docker daemon.

    Alternately, Docker 0.8 or newer can be run on Macs thanks to a specially developed, lightweight VirtualBox VM.

    To start this, use the “quickstart terminal” which fires up Virtualbox.

    https://www.youtube.com/watch?v=v1BfbZu8EMw

Install Docker on Windows 10

The below enhances https://docs.docker.com/docker-for-windows/

CAUTION: A 64-bit machine is necessary.

  1. Enable Microsoft Hyper-V necessary to run Docker for Windows.

    CAUTION: This renders ususable Oracle Virtual Box VMs running Docker Toolbox.

  2. Upgrade Windows 10 to the “Anniversary Update” 1607 (KB3194798) released October 11, 2016 or more recent.

    See the video.

    In Settings > Updates & Security > Windows Update > Update & Security > Check for Updates

    Update.

  3. At https://docs.docker.com/docker-for-windows,
    Click “Get Docker for Windows (stable)” (not the more volatile Beta).

    This downloads InstallDocker.msi (105 MB), typically to your Downloads folder.

    NOTE: There is no Chocolatey module yet.

  4. Double-click InstallDocker.msi to run the installer.

  5. If you haven’t already downloaded the installer (InstallDocker.msi), you can get it here. It

  6. Follow the install wizard to accept the license, authorize the installer, and proceed with the install.

  7. Authorize Docker.app with your system password when prompted, since privileged access is needed to install networking components, links to the Docker apps, and manage the Hyper-V VMs.

  8. Click Finish on the setup complete dialog to launch Docker.

    The same modules are installed.

  9. Skip to verify Docker install.

Install in clouds

  1. Create ~/.aws/credentials file based on keys copied during user creation in AWS IAM, such as:

    [default]
    aws_access_key_id = AKID1234567890
    aws_secret_access_key = MY-SECRET-KEY
    

    This is so you don’t need to specify the keys in the command line:
    --amazonec2-access-key AKI******* --amazonec2-secret-key 8T93C*******

  2. In AWS, create an EC2 instance named “aws-sandbox”

    
    docker-machine create --driver amazonec2 --amazonec2-region us-west-1 aws-sandbox
     --volumes-from ?
    

    “amazonec2” is the driver name for Amazon EC2. Alternatives are
    “digitalocean” (with --digitalocean-access-token=) and
    “azure” (with --azure-subscription-id).

    --volumes-from is how data volumes are associated (for sharing).

    Default folders

    The default NGINX configuration root directory

    • /usr/share/nginx/html is the root directory
    • /etc/nginx contains configuration files

    To remap configuration files to a different location, specify that first:

    
    -v /var/nginx/conf:/etc/nginx:ro
    

Alpine Linux

Within Docker for Mac, the Docker engine runs in an Alpine Linux distribution on top of a Mac xhyve (pronounced “x-hive”) Virtual Machine.

The Alpine Linux distribution (distro) is so small that there is an edition of it for the Raspberry Pi.

Alpine comes with Docker for Mac, but for additional information, see:

https://alpinelinux.org

File Date Download Type Size MB
2016-09-28 alpine-virt-3.4.4-x86_64.iso Virtual 39 MB
2016-09-28 alpine-3.4.4-x86_64.iso Standard 85 MB
2016-09-28 alpine-extended-3.4.4-x86_64.iso Extended 311 MB

Install Docker in CentOS

There are two methods for installing Docker on CentOS 7:

  1. Installing Docker on an existing installation of the operating system.

  2. Spin up a server with Docker Machine which auto-installs Docker.

See https://docs.docker.com/machine/reference/ls/

The Docker installation package available in the official CentOS 7 repository may not be the latest version.

To get the latest and greatest version, install Docker from the official Docker repository.

  1. First, update the package database:

    sudo yum check-update

    The response:

    Failed to set locale, defaulting to C
    Loaded plug-ins: fastestmirror
    Loading mirror speeds from cached hostfile
            * base: mirror.n5tech.com
            * extras: mirror.pac-12.org
            * updates: mirror.math.princeton.edu
     
  2. Add the official Docker repository, download the latest version of Docker, and install it:

    curl -fsSL https://get.docker.com/ | sh

    The response:

           + sh -c 'sleep 3; yum -y -q install docker-engine'
    Failed to set locale, defaulting to C
    warning: /var/cache/yum/x86_64/7/docker-main-repo/packages/docker-engine-selinux-1.11.2-1.el7.centos.noarch.rpm: Header V4 RSA/SHA512 Signature, key ID 2c52609d: NOKEY
    Public key for docker-engine-selinux-1.11.2-1.el7.centos.noarch.rpm is not installed
    Importing GPG key 0x2C52609D:
     Userid     : "Docker Release Tool (releasedocker) <docker@docker.com>"
     Fingerprint: 5811 8e89 f3a9 1289 7c07 0adb f762 2157 2c52 609d
     From       : https://yum.dockerproject.org/gpg
    setsebool:  SELinux is disabled.
    libsemanage.semanage_exec_prog: Child process /sbin/setfiles did not exit cleanly.
    libsemanage.semanage_install_active: setfiles returned error code -1.
    libsemanage.semanage_exec_prog: Child process /sbin/setfiles did not exit cleanly.
    libsemanage.semanage_install_active: setfiles returned error code -1.
    /usr/sbin/semodule:  Failed!
     
    If you would like to use Docker as a non-root user, you should now consider
    adding your user to the "docker" group with something like:
     
      sudo usermod -aG docker your-user
     
    Remember that you will have to log out and back in for this to take effect!
    
  3. With installation complete, start the Docker daemon:

    sudo systemctl start docker

    No response is returned.

    NOTE: By default, docker commands require root privileges. So commands need to be prefixed with Linux sudo.

  4. Verify it’s running:

    sudo systemctl status docker -l

    The response:

    The output should be similar to the following, showing that the service is active and running:

    ● docker.service - Docker Application Container Engine
    Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
    Active: active (running) since Wed 2016-07-27 22:37:40 UTC; 1min 10s ago
      Docs: https://docs.docker.com
     Main PID: 3158 (docker)
    Memory: 16.3M
    CGroup: /system.slice/docker.service
            ├─3158 /usr/bin/docker daemon -H fd://
            └─3161 docker-containerd -l /var/run/docker/libcontainerd/docker-containerd.sock --runtime docker-runc --start-timeout 2m
     
    Jul 27 22:37:40 centos-512mb-sfo2-01 docker[3158]: time="2016-07-27T22:37:40.656780521Z" level=warning msg="Running modprobe bridge br_netfilter failed with message: modprobe: WARNING: Module br_netfilter not found.\ninsmod /lib/modules/3.10.0-327.22.2.el7.x86_64/kernel/net/llc/llc.ko \ninsmod /lib/modules/3.10.0-327.22.2.el7.x86_64/kernel/net/802/stp.ko \ninsmod /lib/modules/3.10.0-327.22.2.el7.x86_64/kernel/net/bridge/bridge.ko \n, error: exit status 1"
    Jul 27 22:37:40 centos-512mb-sfo2-01 docker[3158]: time="2016-07-27T22:37:40.673905283Z" level=info msg="Firewalld running: false"
    Jul 27 22:37:40 centos-512mb-sfo2-01 docker[3158]: time="2016-07-27T22:37:40.749552123Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
    Jul 27 22:37:40 centos-512mb-sfo2-01 docker[3158]: time="2016-07-27T22:37:40.812028054Z" level=warning msg="mountpoint for pids not found"
    Jul 27 22:37:40 centos-512mb-sfo2-01 docker[3158]: time="2016-07-27T22:37:40.812680271Z" level=info msg="Loading containers: start."
    Jul 27 22:37:40 centos-512mb-sfo2-01 docker[3158]: time="2016-07-27T22:37:40.812815226Z" level=info msg="Loading containers: done."
    Jul 27 22:37:40 centos-512mb-sfo2-01 docker[3158]: time="2016-07-27T22:37:40.812841211Z" level=info msg="Daemon has completed initialization"
    Jul 27 22:37:40 centos-512mb-sfo2-01 docker[3158]: time="2016-07-27T22:37:40.812872955Z" level=info msg="Docker daemon" commit=b9f10c9 graphdriver=devicemapper version=1.11.2
    Jul 27 22:37:40 centos-512mb-sfo2-01 docker[3158]: time="2016-07-27T22:37:40.850984219Z" level=info msg="API listen on /var/run/docker.sock"
    Jul 27 22:37:40 centos-512mb-sfo2-01 systemd[1]: Started Docker Application Container Engine.
    
  5. Configure to git it to start at every server reboot:

    sudo systemctl enable docker

    NOTE: This only needs to be done once per instance.

    The response:

    Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.

  6. Skip to verify Docker install.


After install

After installation, Docker commands are similar on all operating systems:

  1. Open a Terminal shell window.

  2. Get Docker version property:

    
    docker --version
    

    The response:

    Docker version 1.12.3, build 6b644ec
    

    Notice “experimental” has been removed.

  3. List all docker commands:

    docker

    The response:

    Usage: docker [OPTIONS] COMMAND [arg...]
        docker [ --help | -v | --version ]
     
    A self-sufficient runtime for containers.
     
    Options:
     
      --config=~/.docker              Location of client config files
      -D, --debug                     Enable debug mode
      -H, --host=[]                   Daemon socket(s) to connect to
      -h, --help                      Print usage
      -l, --log-level=info            Set the logging level
      --tls                           Use TLS; implied by --tlsverify
      --tlscacert=~/.docker/ca.pem    Trust certs signed only by this CA
      --tlscert=~/.docker/cert.pem    Path to TLS certificate file
      --tlskey=~/.docker/key.pem      Path to TLS key file
      --tlsverify                     Use TLS and verify the remote
      -v, --version                   Print version information and quit
     
    Commands:
     attach    Attach to a running container
     build     Build an image from a Dockerfile
     commit    Create a new image from a container's changes
     cp        Copy files/folders between a container and the local filesystem
     create    Create a new container
     deploy    Create and update a stack from a Distributed Application Bundle (DAB)
     diff      Inspect changes on a container's filesystem
     events    Get real time events from the server
     exec      Run a command in a running container
     export    Export a container's filesystem as a tar archive
     history   Show the history of an image
     images    List images
     import    Import the contents from a tarball to create a filesystem image
     info      Display system-wide information
     inspect   Return low-level information on a container, image or task
     kill      Kill one or more running container
     load      Load an image from a tar archive or STDIN
     login     Log in to a Docker registry.
     logout    Log out from a Docker registry.
     logs      Fetch the logs of a container
     network   Manage Docker networks
     node      Manage Docker Swarm nodes
     pause     Pause all processes within one or more containers
     plugin    Manage Docker plugins
     port      List port mappings or a specific mapping for the container
     ps        List containers
     pull      Pull an image or a repository from a registry
     push      Push an image or a repository to a registry
     rename    Rename a container
     restart   Restart a container
     rm        Remove one or more containers
     rmi       Remove one or more images
     run       Run a command in a new container
     save      Save one or more images to a tar archive (streamed to STDOUT by default)
     search    Search the Docker Hub for images
     service   Manage Docker services
     stack     Manage Docker stacks
     start     Start one or more stopped containers
     stats     Display a live stream of container(s) resource usage statistics
     stop      Stop one or more running containers
     swarm     Manage Docker Swarm
     tag       Tag an image into a repository
     top       Display the running processes of a container
     unpause   Unpause all processes within one or more containers
     update    Update configuration of one or more containers
     version   Show the Docker version information
     volume    Manage Docker volumes
     wait      Block until a container stops, then print its exit code
     
    Run 'docker COMMAND --help' for more information on a command.
    
  4. Obtain the version number using a sub-command:

    docker version

    Sample response on the Mac:

    Client:
     Version:      1.12.3
     API version:  1.24
     Go version:   go1.6.3
     Git commit:   6b644ec
     Built:        Wed Oct 26 23:26:11 2016
     OS/Arch:      darwin/amd64
     
    Server:
     Version:      1.12.3
     API version:  1.24
     Go version:   go1.6.3
     Git commit:   6b644ec
     Built:        Wed Oct 26 23:26:11 2016
     OS/Arch:      linux/amd64
    

    Sample response on Centos:

    Client:
     Version:      1.11.2
     API version:  1.23
     Go version:   go1.5.4
     Git commit:   b9f10c9
     Built:        Wed Jun  1 21:23:11 2016
     OS/Arch:      linux/amd64
     
    Server:
     Version:      1.11.2
     API version:  1.23
     Go version:   go1.5.4
     Git commit:   b9f10c9
     Built:        Wed Jun  1 21:23:11 2016
     OS/Arch:      linux/amd64
    

    Sample response on Windows:

    Client:
    Version:      1.12.0
    API version:  1.24
    Go version:   go1.6.3
    Git commit:   8eab29e
    Built:        Thu Jul 28 21:04:48 2016
    OS/Arch:      windows/amd64
    Experimental: true
     
    Server:
    Version:      1.12.0
    API version:  1.24
    Go version:   go1.6.3
    Git commit:   8eab29e
    Built:        Thu Jul 28 21:04:48 2016
    OS/Arch:      linux/amd64
    Experimental: true
    
  5. Obtain status using the info sub-command:

    docker info

    An example of command output for docker info on Windows:

    Containers: 0
     Running: 0
     Paused: 0
     Stopped: 0
    Images: 0
    Server Version: 1.12.3
    Storage Driver: aufs
     Root Dir: /var/lib/docker/aufs
     Backing Filesystem: extfs
     Dirs: 0
     Dirperm1 Supported: true
    Logging Driver: json-file
    Cgroup Driver: cgroupfs
    Plugins:
     Volume: local
     Network: host bridge overlay null
    Swarm: inactive
    Runtimes: runc
    Default Runtime: runc
    Security Options: seccomp
    Kernel Version: 4.4.27-moby
    Operating System: Alpine Linux v3.4
    OSType: linux
    Architecture: x86_64
    CPUs: 4
    Total Memory: 1.951 GiB
    Name: moby
    ID: 6SFI:4ZSJ:27OO:6L65:VZXB:MM6S:TIAT:MC7Z:CVNS:4ECH:N6AY:KTDJ
    Docker Root Dir: /var/lib/docker
    Debug Mode (client): false
    Debug Mode (server): true
     File Descriptors: 15
     Goroutines: 27
     System Time: 2016-11-10T20:54:34.119254981Z
     EventsListeners: 1
    No Proxy: *.local, 169.254/16
    Registry: https://index.docker.io/v1/
    WARNING: No kernel memory limit support
    Insecure Registries:
     127.0.0.0/8
    

Modules installed

Installing Docker gives you not just the Docker service (daemon) but also the docker command line utility, or the Docker client.

Regardless of the OS, the installation provides:

  1. See the list of supported drivers at
    https://docs.docker.com/machine/drivers

Extensions

Docker UCP (Universal Control Plane)

UCP enables you to control Docker environments through a web interface. This is helpful if you want to steer clear of the command line.

You can use Docker UCP to deploy to various cloud solutions, tie into your existing authentication infrastructure, and in turn control user access.

Dockur UCP also provides specialized monitoring.

https://docs.docker.com/ucp
has more information about Docker UCP.

Run Hello Container

PROTIP: Similar to Python, most people run commands within a Docker machine (which is the whole point of installing Docker).

This section describes how to run a Docker image, then remove it.

  1. See if it can run anything:

    docker run hello-world

    This uses the default “library” user, so the command is equivalent to:

    docker run library/hello-world
    

    If you get this:

    docker: Cannot connect to the Docker daemon. Is the docker daemon running on this host?.
    See 'docker run --help'.
    

    If the image specified is not found, Docker gets it for you from the default registry:

    Unable to find image 'hello-world:latest' locally
    latest: Pulling from library/hello-world
    c04b14da8d14: Pull complete 
    Digest: sha256:0256e8a36e2070f7bf2d0b0763dbabdd67798512411de4cdcf9431a1feb60fd9
    Status: Downloaded newer image for hello-world:latest
     
    Hello from Docker!
    This message shows that your installation appears to be working correctly.
     
    To generate this message, Docker took the following steps:
    \1. The Docker client contacted the Docker daemon.
    \2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    \3. The Docker daemon created a new container from that image which runs the
     executable that produces the output you are currently reading.
    \4. The Docker daemon streamed that output to the Docker client, which sent it
     to your terminal.
     
    To try something more ambitious, you can run an Ubuntu container with:
     $ docker run -it ubuntu bash
     
    Share images, automate workflows, and more with a free Docker Hub account:
     https://hub.docker.com
     
    For more examples and ideas, visit:
     https://docs.docker.com/engine/userguide/
    

    See https://docs.docker.com/docker-for-windows/

    It automatically ran:

    
    docker pull library/hello-world
    

    Alternately:

  2. This blog shows this command to run image named “mynginx1”:

    docker run –name mynginx1 -P -d nginx \ fcd1fb01b14557c7c9d991238f2558ae2704d129cf9fb97bb4fadf673a58580d

    “-P” (capital P) tells Docker to map the Ports exposed by the NGINX image – ports 80 and 443 – to ports on the Docker host randomly selected from the range between 49153 and 65535 each time the container is started or restarted. This is to avoid conflicts on standard ports 80 and 443 if we later create multiple NGINX containers on the same Docker host. The docker ps command under PORTS would show something like this:

    0.0.0.0:49166->443/tcp, 0.0.0.0:49167->80/tcp

    “-p” (lower case p) is used to set port mappings manually.

    “-d” specifies running in detached mode so the container continues to run until stopped, but does not respond to commands run on the command line.

  3. To interact with a detached container:

    TODO: ???

    NGINX example

    Alternately:

  4. To run an NGINX web server:

    docker run -d -p 8000:80 nginx

    The “-d” parameter …

    The “8000:80” means we’ll use localhost:8080.

  5. To see if that machine responds:

    curl $(docker-machine ip default):8000

    A common error message is:

    curl: (7) Failed to connect to 192.168.99.100 port 8000: Connection refused
    

    Ubuntu inside Mac

  6. To run the latest Ubuntu box inside your Mac:

    docker run -it –rm –publish 3000:3000 ubuntu bash

    “-it” means interactive (tty) terminal, specifying that the image should contain a shell when it runs so it can be terminated manually by Ctrl+C.

    ”–publish” forwards port 3000 on the host from port 3000 in the container.

    “bash” is the command issued in the container when it becomes active. Alternately, “ruby /app/hello_world.rb” would invoke a ruby program.

    Alternately, run version 14.04 of Ubuntu:

    docker run –net=host -ti ubuntu:14.04 bash

    After downloads, you should see a bash prompt such as:

    root@ee355a835ff8:/# 
    
  7. Get version information:

    cat /etc/lsb-release

    The response:

    DISTRIB_ID=Ubuntu
    DISTRIB_RELEASE=16.04
    DISTRIB_CODENAME=xenial
    DISTRIB_DESCRIPTION="Ubuntu 16.04.1 LTS"
    

    NOTE: You can’t run docker commands on this prompt because you’re inside.

  8. Press Control+C or type exit:

    exit

    List Docker Containers

  9. List Docker containers and their identifiers:

    docker ps -a

    PROTIP: In Linux the ps command is for processes. In a way, that’s what Docker containers are, a process.

    -a shows inactive as well as the default active listing. (Kinda counter-intuitive)

    Widen your screen to avoid wrapping:

    CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                      PORTS                                           NAMES
    2289fc019878        nginxdemos/hello    "nginx -g 'daemon off"   32 minutes ago      Up 32 minutes               0.0.0.0:32769->80/tcp, 0.0.0.0:32768->443/tcp   trusting_euler
    75ae035ab68b        hello-world         "/hello"                 39 minutes ago      Exited (0) 39 minutes ago                                                   serene_sammet
    
  10. PROTIP: Customize the layout by specifying a memory variable containing format template using tabs and line breaks.

    \nID\t\nIMAGE\t\nCOMMAND\t\nCREATED\t\nSTATUS\t\n
    

    docker ps -a –format $FORMAT

  11. To list Docker images downloaded:

    docker images

    REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
    nginxdemos/hello    latest              0ec165d1eb3a        12 hours ago        54.24 MB
    ubuntu              latest              f753707788c5        4 weeks ago         127.2 MB
    hello-world         latest              c54a2cc56cbb        4 months ago        1.848 kB
    
  12. List Docker machines:

    docker-machine ls

    Example response:

    NAME      ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER    ERRORS
    default   -        virtualbox   Running   tcp://192.168.99.100:2376           v1.12.3   
    

    See https://docs.docker.com/machine/get-started/

  13. Connect your shell to the new machine (per https://docs.docker.com/machine/reference/env/):

    eval “$(docker-machine env default)”

    No response is displayed becuase the “eval” command above runs the output of the command:

    
    docker-machine env default
    

    which is:

    export DOCKER_TLS_VERIFY="1"
    export DOCKER_HOST="tcp://192.168.99.100:2376"
    export DOCKER_CERT_PATH="/Users/mac/.docker/machine/machines/default"
    export DOCKER_MACHINE_NAME="default"
    # Run this command to configure your shell: 
    # eval $(docker-machine env)
    

    PROTIP: On a Mac, the docker-machine VM is called “default”, existing in directory
    /Users/<username>/.docker/machine/machines/default/

  14. See if the environment variables are set:

    env | grep DOCKER

  15. To unset commands:

    eval $(docker-machine env -u)

    which runs:

    unset DOCKER_TLS_VERIFY
    unset DOCKER_HOST
    unset DOCKER_CERT_PATH
    unset DOCKER_MACHINE_NAME
    # Run this command to configure your shell: 
    # eval $(docker-machine env -u)
    

    Remove images

  16. To remove an individual Docker image listed above (to free up disk space):

    docker rmi hello-world

    A common error response if you have not first

    Error response from daemon: conflict: unable to remove repository reference "hello-world" (must force) - container 75ae035ab68b is using its referenced image c54a2cc56cbb
    
  17. To stop all running docker containers:

    docker stop $(docker ps -a -q)

  18. To delete all containers in a single command (to save disk space):

    docker rm $(docker ps -a -q)

    No Error Report

    PROTIP: Whenever an attempt to provision a Dockerized host using Docker Machine fails, or Docker Machine crashes, some diagnostic information is sent automatically to a Docker account on Bugsnag.

  19. Disable this reporting by creating an empty file called no-error-report under your installations .docker/machine directory:

    touch ~/.docker/machine/no-error-report

    Start using old boot2docker

  20. Start again, you don’t need to specify “default”:

    docker-machine start default

  21. Connect:

    docker-machine ssh

    You should see a whale:

                         ##         .
                   ## ## ##        ==
                ## ## ## ## ##    ===
            /"""""""""""""""""\___/ ===
       ~~~ {~~ ~~~~ ~~~ ~~~~ ~~~ ~ /  ===- ~~~
            \______ o           __/
              \    \         __/
               \____\_______/
     _                 _   ____     _            _
    | |__   ___   ___ | |_|___ \ __| | ___   ___| | _____ _ __
    | '_ \ / _ \ / _ \| __| __) / _` |/ _ \ / __| |/ / _ \ '__|
    | |_) | (_) | (_) | |_ / __/ (_| | (_) | (__|   <  __/ |
    |_.__/ \___/ \___/ \__|_____\__,_|\___/ \___|_|\_\___|_|
    Boot2Docker version 1.12.3, build HEAD : 7fc7575 - Thu Oct 27 17:23:17 UTC 2016
    Docker version 1.12.3, build 6b644ec
    

    NOTE: Docker deprecated the Boot2Docker command line in favor of Docker Machine.

    https://docs.docker.com/machine/migrate-to-machine/

  22. Now do whatever you need to do here.

    Stop

  23. Hard stop the default machine:

    docker-machine stop default

    PROTIP: Those who use this a often create aliases to limit typing. For example, “dmon” for the above command.

  1. See the Officially supported repositories at:
    https://hub.docker.com/explore

    They include:

    • operating systems (ubuntu, centos)
    • database server base (redis, mongo, mysql, postgres, elasticsearch)
    • app server base (node, nginx, httpd)
    • blog app server base (wordpress, alpine)

  2. Search for Ubuntu images on the free and public repository at
    https://hub.docker.com

    docker search ubuntu

    The response (27 Jul 2016):

    NAME                              DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
    ubuntu                            Ubuntu is a Debian-based Linux operating s...   4343      [OK]       
    ubuntu-upstart                    Upstart is an event-based replacement for ...   65        [OK]       
    rastasheep/ubuntu-sshd            Dockerized SSH service, built on top of of...   30                   [OK]
    torusware/speedus-ubuntu          Always updated official Ubuntu docker imag...   26                   [OK]
    ubuntu-debootstrap                debootstrap --variant=minbase --components...   25        [OK]       
    nickistre/ubuntu-lamp             LAMP server on Ubuntu                           8                    [OK]
    nuagebec/ubuntu                   Simple always updated Ubuntu docker images...   6                    [OK]
    nickistre/ubuntu-lamp-wordpress   LAMP on Ubuntu with wp-cli installed            6                    [OK]
    nimmis/ubuntu                     This is a docker images different LTS vers...   5                    [OK]
    maxexcloo/ubuntu                  Docker base image built on Ubuntu with Sup...   2                    [OK]
    admiringworm/ubuntu               Base ubuntu images based on the official u...   1                    [OK]
    darksheer/ubuntu                  Base Ubuntu Image -- Updated hourly             1                    [OK]
    jordi/ubuntu                      Ubunt
    u Base Image                               1                    [OK]
    esycat/ubuntu                     Ubuntu LTS                                      0                    [OK]
    datenbetrieb/ubuntu               custom flavor of the official ubuntu base ...   0                    [OK]
    life360/ubuntu                    Ubuntu is a Debian-based Linux operating s...   0                    [OK]
    konstruktoid/ubuntu               Ubuntu base image                               0                    [OK]
    widerplan/ubuntu                  Our basic Ubuntu images.                        0                    [OK]
    croscon/ubuntu                    Crosconized Ubuntu                              0                    [OK]
    teamrock/ubuntu                   TeamRock's Ubuntu image configured with AW...   0                    [OK]
    ustclug/ubuntu                    ubuntu image for docker with USTC mirror        0                    [OK]
    smartentry/ubuntu                 ubuntu with smartentry                          0                    [OK]
    dorapro/ubuntu                    ubuntu image                                    0                    [OK]
    lynxtp/ubuntu                     https://github.com/lynxtp/docker-ubuntu         0                    [OK]
    webhippie/ubuntu                  Docker images for ubuntu                        0                    [OK]
    
    
    
  3. Widen the Terminal window so lines don’t wrap.

  4. Search for centos images on the free and public repository at
    https://hub.docker.com

    docker search centos

    The response (27 Jul 2016):

    NAME                          DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
    centos                        The official build of CentOS.                   2475      [OK]       
    jdeathe/centos-ssh            CentOS-6 6.8 x86_64 / CentOS-7 7.2.1511 x8...   26                   [OK]
    nimmis/java-centos            This is docker images of CentOS 7 with dif...   13                   [OK]
    million12/centos-supervisor   Base CentOS-7 with supervisord launcher, h...   12                   [OK]
    consol/centos-xfce-vnc        Centos container with "headless" VNC sessi...   10                   [OK]
    torusware/speedus-centos      Always updated official CentOS docker imag...   8                    [OK]
    nickistre/centos-lamp         LAMP on centos setup                            4                    [OK]
    centos/mariadb55-centos7                                                      3                    [OK]
    nathonfowlie/centos-jre       Latest CentOS image with the JRE pre-insta...   3                    [OK]
    consol/sakuli-centos-xfce     Sakuli end-2-end testing and monitoring co...   2                    [OK]
    timhughes/centos              Centos with systemd installed and running       1                    [OK]
    darksheer/centos              Base Centos Image -- Updated hourly             1                    [OK]
    blacklabelops/centos          CentOS Base Image! Built and Updates Daily!     1                    [OK]
    ericuni/centos                centos dev                                      0                    [OK]
    kz8s/centos                   Official CentOS plus epel-release               0                    [OK]
    grossws/centos                CentOS 6 and 7 base images with gosu and l...   0                    [OK]
    harisekhon/centos-scala       Scala + CentOS (OpenJDK tags 2.10-jre7 - 2...   0                    [OK]
    grayzone/centos               auto build for centos.                          0                    [OK]
    aguamala/centos               CentOS base image                               0                    [OK]
    dmglab/centos                 CentOS with some extras - This is for the ...   0                    [OK]
    jsmigel/centos-epel           Docker base image of CentOS w/ EPEL installed   0                    [OK]
    januswel/centos               yum update-ed CentOS image                      0                    [OK]
    ustclug/centos                 USTC centos                                    0                    [OK]
    smartentry/centos             CentOS with smartentry                          0                    [OK]
    repositoryjp/centos           Docker Image for CentOS.                        0                    [OK]
    
    
    
    
  5. Alternately, if you want security and have money for license, a more secure commercial (paid) repository is at
    https://www.docker.com/products/docker-trusted-registry

    A third alternative is
    https://docs.docker.com/registry

  6. Pull down an image from a private repository:

    docker pull my-reistry.net:5000/activemq

    Remove image

  7. To remove an image:

    docker rmi ubuntu:trusty

  8. Run the latest version image pulled for nginx:

    docker run -i -t nginx:latest /bin/bash


docker-machine install

Docker Machine provisions Docker on virtual machines that reside on local or on a cloud provider.

NOTE: Docker Machine makes it easy to provision and manage multiple Docker hosts remotely from your personal computer.

  1. Type the command by itself for a list of sub-commands:

    docker-machine

    The response shows the sub-commands and options available:

    Usage: docker-machine [OPTIONS] COMMAND [arg...]
     
    Create and manage machines running Docker.
     
    Version: 0.8.2, build e18a919
     
    Author:
      Docker Machine Contributors - <https://github.com/docker/machine>
     
    Options:
      --debug, -D           Enable debug mode
      --storage-path, -s "/Users/mac/.docker/machine" Configures storage path [$MACHINE_STORAGE_PATH]
      --tls-ca-cert           CA to verify remotes against [$MACHINE_TLS_CA_CERT]
      --tls-ca-key            Private key to generate certificates [$MACHINE_TLS_CA_KEY]
      --tls-client-cert           Client cert to use for TLS [$MACHINE_TLS_CLIENT_CERT]
      --tls-client-key          Private key used in client TLS auth [$MACHINE_TLS_CLIENT_KEY]
      --github-api-token          Token to use for requests to the Github API [$MACHINE_GITHUB_API_TOKEN]
      --native-ssh            Use the native (Go-based) SSH implementation. [$MACHINE_NATIVE_SSH]
      --bugsnag-api-token           BugSnag API token for crash reporting [$MACHINE_BUGSNAG_API_TOKEN]
      --help, -h            show help
      --version, -v           print the version
       
    Commands:
      active    Print which machine is active
      config    Print the connection config for machine
      create    Create a machine
      env     Display the commands to set up the environment for the Docker client
      inspect   Inspect information about a machine
      ip      Get the IP address of a machine
      kill      Kill a machine
      ls      List machines
      provision   Re-provision existing machines
      regenerate-certs  Regenerate TLS Certificates for a machine
      restart   Restart a machine
      rm      Remove a machine
      ssh     Log into or run a command on a machine with SSH.
      scp     Copy files between machines
      start     Start a machine
      status    Get the status of a machine
      stop      Stop a machine
      upgrade   Upgrade a machine to the latest version of Docker
      url     Get the URL of a machine
      version   Show the Docker Machine version or a machine docker version
      help      Shows a list of commands or help for one command
       
    Run 'docker-machine COMMAND --help' for more information on a command.
    
  2. Widen the Terminal window so lines don’t wrap.

  3. Create a Docker machine on the Digital Ocean cloud:

    docker-machine create -d digitialocean –digitalocean-access-token=secret

  4. List Docker machine instances:

    docker-machine ls </strong>

    The response is a heading above lines, if any:

    NAME   ACTIVE   DRIVER   STATE   URL   SWARM   DOCKER   ERRORS
    
  5. List Docker machine Internet Protocol addresses:

    docker-machine ip </strong>

    The response:

    192.168.99.100
    

    Create Docker machine

  6. Using your Mac’s Finder, look in your Applications folder for a Docker folder. In there is a Docker Quickstart Terminal.

    You can also invoke it by typing on the Mac’s Search box.

    It creates a window containing:

    Creating CA: /Users/mac/.docker/machine/certs/ca.pem
    Creating client certificate: /Users/mac/.docker/machine/certs/cert.pem
    Running pre-create checks...
    (default) Default Boot2Docker ISO is out-of-date, downloading the latest release...
    (default) Latest release for github.com/boot2docker/boot2docker is v1.12.3
    (default) Downloading /Users/mac/.docker/machine/cache/boot2docker.iso from https://github.com/boot2docker/boot2docker/releases/download/v1.12.3/boot2docker.iso...
    Creating machine...
    (default) Copying /Users/mac/.docker/machine/cache/boot2docker.iso to /Users/mac/.docker/machine/machines/default/boot2docker.iso...
    (default) Creating VirtualBox VM...
    (default) Creating SSH key...
    (default) Starting the VM...
    (default) Check network to re-create if needed...
    (default) Found a new host-only adapter: "vboxnet1"
    (default) Waiting for an IP...
     
                         ##         .
                   ## ## ##        ==
                ## ## ## ## ##    ===
            /"""""""""""""""""\___/ ===
       ~~~ {~~ ~~~~ ~~~ ~~~~ ~~~ ~ /  ===- ~~~
            \______ o           __/
              \    \         __/
               \____\_______/
     
    docker is configured to use the default machine with IP 192.168.99.100
    For help getting started, check out the docs at https://docs.docker.com
     
    bash: print: command not found
    
  7. Create a Docker machine named node1 in the local Virtualbox, in debug mode:

    docker-machine create -d virtualbox node1

    Alternately, if you are running docker-machine on windows, you should use Hyper-V :

    docker-machine create –driver hyperv vm

Docker Compose

See https://docs.docker.com/compose/install/.

  1. There’s a different version of each Docker module:

    
    docker-compose --version
    

    The responses:

    docker-compose version 1.8.1, build 878cff1
    
  2. Describe you stack in a docker-compose.yml. Example:

    web:
       build .
       command: python app.py
    ports:
     - "5000:5000"
    volumes:
     - .:/code
    links:
     - redis:redis
    redis:
      image : redis
    

    NOTE: Another example docker-compose.yml here

  3. Type the command by itself for a list of sub-commands:

    docker-compose

    The response:

    Define and run multi-container applications with Docker.
     
    Usage:
      docker-compose [-f <arg>...] [options] [COMMAND] [ARGS...]
      docker-compose -h|--help
     
    Options:
      -f, --file FILE             Specify an alternate compose file (default: docker-compose.yml)
      -p, --project-name NAME     Specify an alternate project name (default: directory name)
      --verbose                   Show more output
      -v, --version               Print version and exit
      -H, --host HOST             Daemon socket to connect to
     
      --tls                       Use TLS; implied by --tlsverify
      --tlscacert CA_PATH         Trust certs signed only by this CA
      --tlscert CLIENT_CERT_PATH  Path to TLS certificate file
      --tlskey TLS_KEY_PATH       Path to TLS key file
      --tlsverify                 Use TLS and verify the remote
      --skip-hostname-check       Don't check the daemon's hostname against the name specified
                               in the client certificate (for example if your docker host
                               is an IP address)
     
    Commands:
      build              Build or rebuild services
      bundle             Generate a Docker bundle from the Compose file
      config             Validate and view the compose file
      create             Create services
      down               Stop and remove containers, networks, images, and volumes
      events             Receive real time events from containers
      exec               Execute a command in a running container
      help               Get help on a command
      kill               Kill containers
      logs               View output from containers
      pause              Pause services
      port               Print the public port for a port binding
      ps                 List containers
      pull               Pulls service images
      push               Push service images
      restart            Restart services
      rm                 Remove stopped containers
      run                Run a one-off command
      scale              Set number of containers for a service
      start              Start services
      stop               Stop services
      unpause            Unpause services
      up                 Create and start containers
      version            Show the Docker-Compose version information
      
  4. Docker compose creates multiple containers with a single command:

    docker-compose up –x-smart-recreate

    The above command refers to Dockerfile and compose.yml files.

    Monitoring

    Monitor using cAdvisor collecting stats to write to InfluxDB, displayed by Grafana, described here

    Alternatives include Kubernetes by Google,

    Mesos Centos Atomic Consul, Terraform Serf Cloudify Helios

Docker Swarm

Docker Swarm creates and manages clustered (pool of) Docker servers. It scales containers by dispersing containers across multiple hosts.

docker run swarm create

Installing Docker Swarm launches a container that is used as the Swarm Manager master to communicate to all the nodes in a Swarm cluster.

See https://docs.docker.com/swarm/

docker run -d swarm join –addr=node_ip:2375 token://cluster_id

docker run -d -p swarm_port:2375 swarm manage token://cluster_id

See http://autopilotpattern.io/

Additional notes on security

Docker does have different security requirements which will be a hindrance. Security can be integrated but it does require knowledge of the Linux container environment.

https://blog.docker.com/2013/08/containers-docker-how-secure-are-they/

Resources

Articles

http://jdlm.info/articles/2016/03/06/lessons-building-node-app-docker.html March 6, 2016 by Dr John Lees-Miller (@jdleesmiller)

http://jdlm.info/ds-docker-demo

https://github.com/jdleesmiller/ds-docker-demo

  • Docker is not designef for SSH to “get inside” a running image. You don’t need SSH to restart the process, tweak configurations, debug with gdb, strace, etc.

Pluralsight video courses

Pluralsight’s Container Management using Docker skill path of video tutorials by Nigel Poulton (@NigelPoulton)

  1. Docker and Containers: The Big Picture Dec 22, 2015 / 1h 47m

  2. Docker Deep Dive Jan 28, 2015 / 5h 38m 2s

  3. Docker Swarm: Native Docker Clustering Mar 30, 2016 / 2h 22m 28s

  4. Integrating Docker with DevOps Automated Workflows Sept 21, 2015 / 1h 1m 50s

Dan Wahlin

YouTub videos

  1. Continuous Delivery with Jenkins Workflow and Docker Explained by Cloudbees at DevOpsTV channel

  2. Continuous Integration Using Docker by DevOpsTV

  3. Moving from Jenkins to Docker for build pipelines from Matt Bostock

  4. Continuous Delivery Pipeline with Docker and Jenkins Javaforum Göteborg

  5. Introduction to Docker by Twitter University 47:14

  6. Jenkins 2 and Beyond DevOpsTV

  7. Jenkins 2.0. What? When? What is in it for me? on Praqma Channel

  8. how-to-provision-and-manage-remote-docker-hosts-with-docker-machine-on-centos-7

  9. Docker First Impressions on Ubuntu

Notes

Docker was initially developed to run under GNU/Linux. So not FreeBSD.

Virtualization

Later Macs have Intel’s hardware support for MMU (memory management unit) EPT (Extended Page Tables) virtualization.

Docker for Mac makes use of a Hyperkit hypervisor which runs Docker images such as NGINX.

More on DevOps

This is one of a series on DevOps:

  1. DevOps_2.0
  2. User Stories for DevOps

  3. Choices for DevOps Technologies
  4. Java DevOps Workflow
  5. AWS DevOps (CodeCommit, CodePipeline, CodeDeploy)
  6. AWS server deployment options

  7. Digital Ocean
  8. Cloud regions
  9. AWS Virtual Private Cloud
  10. Azure Cloud Powershell

  11. Git and GitHub vs File Archival
  12. Git Commands and Statuses
  13. Data Security GitHub
  14. Git Commit, Tag, Push
  15. Git Utilities
  16. GitHub API

  17. TFS vs. GitHub

  18. Jenkins Server Setup
  19. Jenkins Plug-ins
  20. Jenkins Freestyle jobs
  21. Jenkins2 Pipeline jobs using Groovy code in Jenkinsfile

  22. Dockerize apps
  23. Docker Setup
  24. Docker Build

  25. Maven on MacOSX

  26. Powershell Ecosystem
  27. Powershell on MacOS
  28. Powershell Desired System Configuration

  29. Ansible

  30. MySQL Setup

  31. SonarQube static code scan

  32. API Management Microsoft
  33. API Management Amazon

  34. Scenarios for load