Wilson Mar bio photo

Wilson Mar

Hello. Hire me!

Email me Calendar Skype call 310 320-7878

LinkedIn Twitter Gitter Google+ Instagram Youtube

Github Stackoverflow Pinterest

One integrated pipepline, for free

Overview

This tutorial is a hands-on technically deep but succinct tutorial about how to use GitLab’s cloud-based CI-CD service to build programming code into packages, test it, deploy it on servers, and monitor apps as they run.

Use GitLab’s CI/CD capability instead of standing up another package such as Jenkins because it is integrated with GitLab’s Git repository and web portal.

In 2017 Forrester rated GitLab v9.3 the top CI vendor:
gitlab-ci-forrester-445x435.gif

And that’s before AutoDevOps was announced with v11.

Forrester cites “GitLab has a broad market reach, with over 80,000 active instances using the open source GitLab Community edition and over 500 enterprises paying for GitLab CI/CD.”

Among limitations Forrester quotes “Despite best-in-class extensibility, reference customers cited limitations in notifications and alerts, calling out difficulty integrating with Slack, among other platforms.”

Marketing landing pages:

  • https://about.gitlab.com/auto-devops/
  • https://about.gitlab.com/features/gitlab-ci-cd/

They say “you can use it for free on GitLab.com”. CI/CD capabilitis are included with both the open source GitLab Community Edition (CE) and the proprietary (licensed for money) GitLab Enterprise Edition. However, free accounts have a limit of 2,000 CI pipeline minutes per group per month.

AutoDevOps / Complete DevOps

The brand name was introduced with GitHub 11 to highlight all its capabilities.

Servers

To avoid issues, GitLab runs CI/CD work on different machines than ones holding GitLab projects (repositories). you should too.

In GitLab Land, a “runner” is equivalent to a “node” worker in Jenkins. Runners run on the Digital Ocean cloud managed by GitLab. GitLab currently recommends standing up a server with at least 2 CPU cores and 4GB of RAM.

Source to build gitlab-runner is at https://gitlab.com/gitlab-org/gitlab-runner

GitLab Account

PROTIP: CI/CD is enabled on individual projects by an admin of that instance..

  1. Login to https://gitlab.com
  2. The landing page should be a list of your projects (repositories), as if you clicked Project, then Your projects.

    Fork a repo

  3. If you don’t already have one, fork one, such as this associated with the Demo: CI/CD with GitLab Mar 14, 2017 by Joshua Lambert, Product Manager at GitLab:

    https://???

  4. Click to select the project you want to enable for CI-CD.
  5. Scroll down to Settings in the left menu and select CI / CD.

    The screen provides a short description of ther different types of GitLab Runners:

     SharedSpecificGroup
    Use case:Quick startCOE & Mix of servicesIsolated team
    Privacy level:PublicKinda PrivatePrivate
    Scheduling:fair usageFIFOFIFO
    • PROTIP: If you want a quick start, use
      Shared Runners by clicking the green “Enable shared Runners”.

      Shared Runners are available to any project, like a cafeteria.

      For convenience, this is the default for new Runners created.

      Shared Runners select jobs to run from a process queue using a “fair usage” algorithm. It selects where the lowest number of jobs are running.

      A Shared Runner can be made into a Specific Runner (but not the reverse way).

    • PROTIP: If you have complex set of different services running on different Runners, use
      Specific Runners which have been set to “Restrict projects for this Runner”.

      From GitLab CI, go to Admin on the topbar which should take you to the Manage Projects page. Click the name of your project, then choose ‘Runners’ from the right hand menu. On that page should be a list of instructions, with the third instruction detailing the alternative token. For example:

      http://ci.gitlab.home/projects/6/runners
    • PROTIP: If you are in an isolated team who don’t want to share, use
      Group Runners to execute code all projects belonging to a designated group name.

      Group Runners can be managed using the Runners API (below).

      See http://ci.gitlab.home/admin/runners

Shared Runner token

Tokens for shared Runners are like “9538b0ab”.

Tag “gce” stands for “Google Compute Engine”. In 2018 GitLab migrated from Azure to Google Cloud after Microsoft bought GitHub.

PROTIP: Alternately, automate this step by making a Registration API call.

Specific Runner token

To uniquely identify Runners on the public internet, Tokens (of random characters) are generated by the GitLab server so Runners can authenticate themselves.

  1. Highlight the token displayed and copy to your Clipboard:

    Use the following registration token during setup: 
    1HEHB_jN-wttCDBQnK2n

    Tags

    The type of jobs that a Runner is equipped to run, such as “Rails”, “Node”, etc. are specified by tags.

    PROTIP: Tags should be defined when a Docker image is defined with components (such as Node, etc.)

gitlab-runner

In order for client machines to communicate CI / CD commands with GitLab, install the gitlab-runner client executable.

The GitLab Runner client is written in the Go language. So it can run on Linux, macOS, Windows, FreeBSD, Docker – any platform for which can build Go binaries.

Auto DevOps automatically build, test, and deploy apps based on a predefined Continuous Integration and Delivery configuration.

  1. Click button “Reset runners registration token”.

    NOTE: The Runner tokens in the list are the first 8 characters of that runner’s full token.

    PROTIP: If you are concerned about compromise, put an entry on your calendar to reset periodically. TODO: Script to automatically reset and update scripts using the token.

See https://docs.gitlab.com/ce/ci/runners/

and https://wiki.samba.org/index.php/CI_using_Your_own_gitlab_runner

Runner Installation

https://docs.gitlab.com/runner/install/

Until this issue to create a brew package is available to manage versions, follow https://docs.gitlab.com/runner/install/osx.html

  1. Use my bash script to install gitlab-runner on your Mac at https://github.com/wilsonmar/DevSecOps/blob/master/gitlab/gitlab-runner-mac.sh

    Highlight and copy this and paste it in your Terminal:

    sh -c “$(curl -fsSL https://raw.githubusercontent.com/wilsonmar/DevSecOps/master/gitlab/gitlab-runner-mac.sh)”

    There are comments in that script.

    PROTIP: The folder /usr/local/bin/ is also where git modules are installed.

  2. Verify version installed:

    gitlab-runner --version
    Version:      11.2.0
    Git revision: 35e8515d
    Git branch:   11-2-stable
    GO version:   go1.8.7
    Built:        2018-08-22T15:58:04+00:00
    OS/Arch:      darwin/amd64
    
  3. Run command on its own for sub-commands:

    gitlab-runner
    NAME:
    gitlab-runner - a GitLab Runner
     
    USAGE:
    gitlab-runner [global options] command [command options] [arguments...]
     
    VERSION:
    11.2.0 (35e8515d)
     
    AUTHOR:
    GitLab Inc. <support@gitlab.com>
     
    COMMANDS:
      exec                  execute a build locally
      list                  List all configured runners
      run                   run multi runner service
      register              register a new runner
      install               install service
      uninstall             uninstall service
      start                 start service
      stop                  stop service
      restart               restart service
      status                get status of a service
      run-single            start single runner
      unregister            unregister specific runner
      verify                verify all registered runners
      artifacts-downloader  download and extract build artifacts (internal)
      artifacts-uploader    create and upload build artifacts (internal)
      cache-archiver        create and upload cache artifacts (internal)
      cache-extractor       download and extract cache artifacts (internal)
      help, h               Shows a list of commands or help for one command
     
    GLOBAL OPTIONS:
    --debug                      debug mode [$DEBUG]
    --log-level value, -l value  Log level (options: debug, info, warn, error, fatal, panic)
    --cpuprofile value           write cpu profile to file [$CPU_PROFILE]
    --help, -h                   show help
    --version, -v                print the version
    
  4. To verify its’s location:

    gitlab-runner list
    
    Listing configured runners ConfigFile=/Users/wilsonmar/.gitlab-runner/config.toml
    
  5. cd to ~/.gitlab-runner folder created by the installer to view config.toml. It contains:

    concurrent = 1
    check_interval = 0
    

    TODO: Describe config.toml

.gitlab-ci.yml

  1. Place a file named .gitlab-ci.yml in the root of any project. This is what the gitlab-runner looks for and runs every time a commit is pushed to the GitLab server.

    QUESTION: Is there a hook file?

    Another example is about CI for publishing NPM modules: https://www.exclamationlabs.com/blog/continuous-deployment-to-npm-using-gitlab-ci/

    A colection of such files are at: https://gitlab.com/gitlab-org/gitlab-ci-yml

  2. PROTIP: Programs thar read Yml files can be finicky. So run a linter by pasting into the online form at:

    http://www.yamllint.com/

    PROTIP: Better yet, specify a hook to run the linter automatically with every commit.

    The .gitlab-ci.yml file contains a series of tasks.

    Caches

    All the files and modifications put in or do inside the .gitlab-ci.yml are reverted each time a commit is pushed to the server. This can be avoided by specifying caches.

    Stages in pipeline

  3. Stages defined in the file are defined to specify the order to be attempted:

    stages:
             - build
             - test
             - staging
             - openMr
             - prod
    

    PROTIP: Each column in the “pipeline” displayed on GitLab’s dashboard is named for each stage. So keep stage names short. Thus, many abbreviate “production” with “prod”.

    Errors in any stage would stop progress to follow-on stages.

    NOTE: “openMr” is a custom stage. Some use a stage named “review” instead of “test”, etc.

  4. Actions are defined for each stage. For example:

    Build My App:
      stage: build
      tags:
    - node
      before_script:
    - yarn config set cache-folder .yarn
    - yarn install
      script:
     - npm run build
    

    Notice the sample action name “Build My App” can contain spaces, and is the action executed for the stage “build” defined in the list of stages above it.

    In this example, the yarn utility is being used instead of npm. And it’s creating a cache folder that will contain all the yarn configurations that will not be recreated when each project runs (each time it’s pushed).

  5. These and addtional steps are defined in the sample at

    https://gist.github.com/superjose/709989dd58aa90bfeda75767668482b2

    as described in blog https://hackernoon.com/configuring-gitlab-ci-yml-150a98e9765d

    Another example is https://linuxhint.com/gitlab_runner_gitlab_ci/ which provides code.

    Deploying to AWS?

    See instructions from @autonix at https://stackoverflow.com/a/38672045/1057052


Get Token

  1. Grab the shared-Runner token on the admin/runners page

    sudo gitlab-runner register
    Running in system-mode.                            
     
    Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
    

    http://ci.gitlab.home/

  2. Supply the URL, as there is no “press return for default”:

    https://gitlab.com/
    Please enter the gitlab-ci token for this runner:
    
  3. Enter a description for the Runner, you can change this later in GitLab’s UI:

    Please enter the gitlab-ci description for this runner

    [hostame] my-runner
  4. Enter the tags associated with the Runner, you can change this later in GitLab’s UI:

    Please enter the gitlab-ci tags for this runner (comma separated):

  5. Enter the Runner executor:

    Please enter the executor: ssh, docker+machine, docker-ssh+machine, kubernetes, docker, parallels, virtualbox, docker-ssh, shell:

Docker executor

docker
  1. If you chose Docker as your executor, you’ll be asked for the default image to be used for projects that do not define one in .gitlab-ci.yml:

    Please enter the Docker image (eg. ruby:2.1):
    alpine:latest
  2. Be sure Docker.app is installed on your mac

Variables

To hold passwords, secret keys, define variables which the runner applies to environments.

?? They can be protected by only exposing them to protected branches or tags.

Pipeline triggers

To force a specific branch or tag to get rebuilt with an API call, define triggers

to impersonate their associated user including their access to projects and their project permissions.

Docker Container Registry

Installing the Docker image enables auto-scaling of additional instances to automatically meet demand. See https://docs.gitlab.com/runner/executors/docker_machine.html

https://docs.gitlab.com/runner/install/registry_and_cache_servers.html

Login to a dedicated machine where the container registry proxy will be running Make sure that Docker Engine is installed on that machine

Optionally create a new container registry:

docker run -d -p 6000:5000 \
    -e REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io \
    --restart always \
    --name registry registry:2
   

Configure Runners

  1. Configure Runner to “Run untagged jobs option”.

    Click Save changes for the changes to take effect

  2. Set the maximum job timeout

  3. Verify all registered Runners:

    gitlab-runner verify

Configure Jobs

GitLab looks for a .gitlab-ci.yml file to specify what jobs do. Its format is described at:

https://docs.gitlab.com/ee/ci/yaml/

A tutorial is at:

https://docs.gitlab.com/ee/ci/quick_start/

We don’t use a shared Runner because then you have to enable each project explicitly for the Runner to run its jobs.

Executor

To trigger your build script, the default is to use shell.

Your Runner should show in the Manager Runners page on the GitLab CI Admin Dashboard (topbar > Admin > Runners). Depending on your Runner type, it may be green, blue, or red.

git submodule update –init ls -la

Monitoring

As of Omnibus GitLab 9.0, Prometheus and its related exporters are enabled by default, to enable easy and in depth monitoring of GitLab. Approximately 200MB of memory will be consumed by these processes, with default settings.

If you would like to disable Prometheus and it’s exporters or read more information about it, check the Prometheus documentation. GitLab Ru

Do it

When a developer finishes a set of changes, they can open a merge request, which is equivalent to a “pull request” in GitHub. This is so other contributors to the repo can make comments.

A pipeline is a collection of jobs split in different stages. Every commit pushed to GitLab generates a pipeline attached to that commit. If multiple commits are pushed together the pipeline will be created for the last one only.

API

Based on https://gitlab.com/help/api/runners.md which at time of this writing is v4 with endpoint “https://gitlab.example.com/api/v4/runners”

NameCall(s)With a...
Register a new RunnerPOST /runners
Verify authentication for a registered RunnerPOST /runners/verify
List all runnersGET /runners/all
GET /runners/all?scope=online
List owned runnersGET /runners
GET /runners?scope=active
scope
Get runner's detailsGET /runners/:id
Update runner's detailsPUT /runners/:id
Remove a runnerDELETE /runners/:id
List runner's jobsGET /runners/:id/jobs
List project's runnersGET /projects/:id/runners
Enable a runner in projectPOST /projects/:id/runners
Disable a runner from projectDELETE /projects/:id/runners/:runner_id
Delete a registered RunnerDELETE /runners token

References

This was created based partly on documentation at:

  • https://docs.gitlab.com/ee/ci/

  • https://about.gitlab.com/2018/01/22/a-beginners-guide-to-continuous-integration/

  • https://codereviewvideos.com/course/your-own-private-github/video/gitlab-ci-runner-tutorial

  • https://neoteric.eu/automate-your-work-with-gitlab-ci-cd-tool

Udemy video: GitLab/GitLab CI Course For Developers and DevOps 7/2017 [2 hrs] by Peng Xiao

More on DevSecOps

This is one of a series on DevSecOps:

  1. DevOps_2.0
  2. ci-cd (Continuous Integration and Continuous Delivery)
  3. User Stories for DevOps

  4. Git and GitHub vs File Archival
  5. Git Commands and Statuses
  6. Git Commit, Tag, Push
  7. Git Utilities
  8. Data Security GitHub
  9. GitHub API
  10. TFS vs. GitHub

  11. Choices for DevOps Technologies
  12. Java DevOps Workflow
  13. AWS DevOps (CodeCommit, CodePipeline, CodeDeploy)
  14. AWS server deployment options

  15. Cloud regions
  16. AWS Virtual Private Cloud
  17. Azure Cloud Onramp
  18. Azure Cloud
  19. Azure Cloud Powershell
  20. Bash Windows using Microsoft’s WSL (Windows Subystem for Linux)

  21. Digital Ocean
  22. Cloud Foundry

  23. Packer automation to build Vagrant images
  24. Terraform multi-cloud provisioning automation

  25. Powershell Ecosystem
  26. Powershell on MacOS
  27. Powershell Desired System Configuration

  28. Jenkins Server Setup
  29. Jenkins Plug-ins
  30. Jenkins Freestyle jobs
  31. Jenkins2 Pipeline jobs using Groovy code in Jenkinsfile

  32. Dockerize apps
  33. Docker Setup
  34. Docker Build

  35. Maven on MacOSX

  36. Ansible

  37. MySQL Setup

  38. SonarQube static code scan

  39. API Management Microsoft
  40. API Management Amazon

  41. Scenarios for load