GitLab CI (and CD and CD too if you dare)

One integrated pipepline, for free

Overview

This tutorial is a hands-on technically deep but succinct tutorial about how to use GitLab’s cloud-based CI-CD service to build programming code into packages, test it, deploy it on servers, and monitor apps as they run.

Use GitLab’s CI/CD capability instead of standing up another package such as Jenkins because it is integrated with GitLab’s Git repository and web portal.

In 2017 Forrester rated GitLab v9.3 the top CI vendor:

And that’s before AutoDevOps was announced with v11.

Forrester cites “GitLab has a broad market reach, with over 80,000 active instances using the open source GitLab Community edition and over 500 enterprises paying for GitLab CI/CD.”

Among limitations Forrester quotes “Despite best-in-class extensibility, reference customers cited limitations in notifications and alerts, calling out difficulty integrating with Slack, among other platforms.”

Marketing landing pages:

https://about.gitlab.com/auto-devops/
https://about.gitlab.com/features/gitlab-ci-cd/

They say “you can use it for free on GitLab.com”. CI/CD capabilitis are included with both the open source GitLab Community Edition (CE) and the proprietary (licensed for money) GitLab Enterprise Edition. However, free accounts have a limit of 2,000 CI pipeline minutes per group per month.

AutoDevOps / Complete DevOps

The brand name was introduced with GitHub 11 to highlight all its capabilities.

GitLab’s Pricing page details the features offered for each level of pricing.
https://gitlab.com/help/topics/autodevops/index.md
https://www.youtube.com/watch?v=0Tc0YYBxqi4
Idea to Production with GitLab 10.2
https://www.youtube.com/watch?v=YHznYB275Mg

Servers

To avoid issues, GitLab runs CI/CD work on different machines than ones holding GitLab projects (repositories). you should too.

In GitLab Land, a “runner” is equivalent to a “node” worker in Jenkins. Runners run on the Digital Ocean cloud managed by GitLab. GitLab currently recommends standing up a server with at least 2 CPU cores and 4GB of RAM.

Source to build gitlab-runner is at https://gitlab.com/gitlab-org/gitlab-runner

GitLab Account

PROTIP: CI/CD is enabled on individual projects by an admin of that instance..

Login to https://gitlab.com
The landing page should be a list of your projects (repositories), as if you clicked Project, then Your projects.

Fork a repo
If you don’t already have one, fork one, such as this associated with the Demo: CI/CD with GitLab Mar 14, 2017 by Joshua Lambert, Product Manager at GitLab:

https://???
Click to select the project you want to enable for CI-CD.
Scroll down to Settings in the left menu and select CI / CD.

The screen provides a short description of ther different types of GitLab Runners:

Shared Specific Group

Use case: Quick start COE & Mix of services Isolated team

Privacy level: Public Kinda Private Private

Scheduling: fair usage FIFO FIFO
- PROTIP: If you want a quick start, use
  Shared Runners by clicking the green “Enable shared Runners”.
  
  Shared Runners are available to any project, like a cafeteria.
  
  For convenience, this is the default for new Runners created.
  
  Shared Runners select jobs to run from a process queue using a “fair usage” algorithm. It selects where the lowest number of jobs are running.
  
  A Shared Runner can be made into a Specific Runner (but not the reverse way).
- PROTIP: If you have complex set of different services running on different Runners, use
  Specific Runners which have been set to “Restrict projects for this Runner”.
  
  From GitLab CI, go to Admin on the topbar which should take you to the Manage Projects page. Click the name of your project, then choose ‘Runners’ from the right hand menu. On that page should be a list of instructions, with the third instruction detailing the alternative token. For example:
```
http://ci.gitlab.home/projects/6/runners
```
- PROTIP: If you are in an isolated team who don’t want to share, use
  Group Runners to execute code all projects belonging to a designated group name.
  
  Group Runners can be managed using the Runners API (below).
  
  See http://ci.gitlab.home/admin/runners

	Shared	Specific	Group
Use case:	Quick start	COE & Mix of services	Isolated team
Privacy level:	Public	Kinda Private	Private
Scheduling:	fair usage	FIFO	FIFO

Shared Runner token

Tokens for shared Runners are like “9538b0ab”.

Tag “gce” stands for “Google Compute Engine”. In 2018 GitLab migrated from Azure to Google Cloud after Microsoft bought GitHub.

PROTIP: Alternately, automate this step by making a Registration API call.

Specific Runner token

To uniquely identify Runners on the public internet, Tokens (of random characters) are generated by the GitLab server so Runners can authenticate themselves.

Highlight the token displayed and copy to your Clipboard:
```
Use the following registration token during setup: 
1HEHB_jN-wttCDBQnK2n
```
Tags

The type of jobs that a Runner is equipped to run, such as “Rails”, “Node”, etc. are specified by tags.

PROTIP: Tags should be defined when a Docker image is defined with components (such as Node, etc.)

gitlab-runner

In order for client machines to communicate CI / CD commands with GitLab, install the gitlab-runner client executable.

The GitLab Runner client is written in the Go language. So it can run on Linux, macOS, Windows, FreeBSD, Docker – any platform for which can build Go binaries.

Auto DevOps automatically build, test, and deploy apps based on a predefined Continuous Integration and Delivery configuration.

Click button “Reset runners registration token”.

NOTE: The Runner tokens in the list are the first 8 characters of that runner’s full token.

PROTIP: If you are concerned about compromise, put an entry on your calendar to reset periodically. TODO: Script to automatically reset and update scripts using the token.

See https://docs.gitlab.com/ce/ci/runners/

and https://wiki.samba.org/index.php/CI_using_Your_own_gitlab_runner

Runner Installation

https://docs.gitlab.com/runner/install/

Until this issue to create a brew package is available to manage versions, follow https://docs.gitlab.com/runner/install/osx.html

Use my bash script to install gitlab-runner on your Mac at https://github.com/wilsonmar/DevSecOps/blob/master/gitlab/gitlab-runner-mac.sh

Highlight and copy this and paste it in your Terminal:

sh -c “$(curl -fsSL https://raw.githubusercontent.com/wilsonmar/DevSecOps/master/gitlab/gitlab-runner-mac.sh)”

There are comments in that script.

PROTIP: The folder /usr/local/bin/ is also where git modules are installed.

Verify version installed:

gitlab-runner --version

Version:      11.2.0
Git revision: 35e8515d
Git branch:   11-2-stable
GO version:   go1.8.7
Built:        2018-08-22T15:58:04+00:00
OS/Arch:      darwin/amd64

Run command on its own for sub-commands:

gitlab-runner

NAME:
gitlab-runner - a GitLab Runner
 
USAGE:
gitlab-runner [global options] command [command options] [arguments...]
 
VERSION:
11.2.0 (35e8515d)
 
AUTHOR:
GitLab Inc. <support@gitlab.com>
 
COMMANDS:
  exec                  execute a build locally
  list                  List all configured runners
  run                   run multi runner service
  register              register a new runner
  install               install service
  uninstall             uninstall service
  start                 start service
  stop                  stop service
  restart               restart service
  status                get status of a service
  run-single            start single runner
  unregister            unregister specific runner
  verify                verify all registered runners
  artifacts-downloader  download and extract build artifacts (internal)
  artifacts-uploader    create and upload build artifacts (internal)
  cache-archiver        create and upload cache artifacts (internal)
  cache-extractor       download and extract cache artifacts (internal)
  help, h               Shows a list of commands or help for one command
 
GLOBAL OPTIONS:
--debug                      debug mode [$DEBUG]
--log-level value, -l value  Log level (options: debug, info, warn, error, fatal, panic)
--cpuprofile value           write cpu profile to file [$CPU_PROFILE]
--help, -h                   show help
--version, -v                print the version

To verify its’s location:

gitlab-runner list

Listing configured runners ConfigFile=/Users/wilsonmar/.gitlab-runner/config.toml

cd to ~/.gitlab-runner folder created by the installer to view config.toml. It contains:
```
concurrent = 1
check_interval = 0
```
TODO: Describe config.toml

.gitlab-ci.yml

Place a file named .gitlab-ci.yml in the root of any project. This is what the gitlab-runner looks for and runs every time a commit is pushed to the GitLab server.

QUESTION: Is there a hook file?

Another example is about CI for publishing NPM modules: https://www.exclamationlabs.com/blog/continuous-deployment-to-npm-using-gitlab-ci/

A colection of such files are at: https://gitlab.com/gitlab-org/gitlab-ci-yml
PROTIP: Programs thar read Yml files can be finicky. So run a linter by pasting into the online form at:

http://www.yamllint.com/

PROTIP: Better yet, specify a hook to run the linter automatically with every commit.

The .gitlab-ci.yml file contains a series of tasks.

Caches

All the files and modifications put in or do inside the .gitlab-ci.yml are reverted each time a commit is pushed to the server. This can be avoided by specifying caches.

Stages in pipeline
Stages defined in the file are defined to specify the order to be attempted:
```
stages:
         - build
         - test
         - staging
         - openMr
         - prod
```
PROTIP: Each column in the “pipeline” displayed on GitLab’s dashboard is named for each stage. So keep stage names short. Thus, many abbreviate “production” with “prod”.

Errors in any stage would stop progress to follow-on stages.

NOTE: “openMr” is a custom stage. Some use a stage named “review” instead of “test”, etc.
Actions are defined for each stage. For example:
```
Build My App:
  stage: build
  tags:
- node
  before_script:
- yarn config set cache-folder .yarn
- yarn install
  script:
 - npm run build
```
Notice the sample action name “Build My App” can contain spaces, and is the action executed for the stage “build” defined in the list of stages above it.

In this example, the yarn utility is being used instead of npm. And it’s creating a cache folder that will contain all the yarn configurations that will not be recreated when each project runs (each time it’s pushed).
These and addtional steps are defined in the sample at

https://gist.github.com/superjose/709989dd58aa90bfeda75767668482b2

as described in blog https://hackernoon.com/configuring-gitlab-ci-yml-150a98e9765d

Another example is https://linuxhint.com/gitlab_runner_gitlab_ci/ which provides code.

Deploying to AWS?

See instructions from @autonix at https://stackoverflow.com/a/38672045/1057052

Get Token

Grab the shared-Runner token on the admin/runners page

sudo gitlab-runner register

Running in system-mode.                            
 
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):

http://ci.gitlab.home/

Supply the URL, as there is no “press return for default”:

https://gitlab.com/

Please enter the gitlab-ci token for this runner:

Enter a description for the Runner, you can change this later in GitLab’s UI:

Please enter the gitlab-ci description for this runner
```
[hostame] my-runner
```
Enter the tags associated with the Runner, you can change this later in GitLab’s UI:

Please enter the gitlab-ci tags for this runner (comma separated):

Enter the Runner executor:

Please enter the executor: ssh, docker+machine, docker-ssh+machine, kubernetes, docker, parallels, virtualbox, docker-ssh, shell:

Docker executor

docker

If you chose Docker as your executor, you’ll be asked for the default image to be used for projects that do not define one in .gitlab-ci.yml:
```
Please enter the Docker image (eg. ruby:2.1):
```
```
alpine:latest
```
Be sure Docker.app is installed on your mac

Variables

To hold passwords, secret keys, define variables which the runner applies to environments.

?? They can be protected by only exposing them to protected branches or tags.

Pipeline triggers

To force a specific branch or tag to get rebuilt with an API call, define triggers

to impersonate their associated user including their access to projects and their project permissions.

Docker Container Registry

Installing the Docker image enables auto-scaling of additional instances to automatically meet demand. See https://docs.gitlab.com/runner/executors/docker_machine.html

https://docs.gitlab.com/runner/install/registry_and_cache_servers.html

Login to a dedicated machine where the container registry proxy will be running Make sure that Docker Engine is installed on that machine

Optionally create a new container registry:

docker run -d -p 6000:5000 \
    -e REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io \
    --restart always \
    --name registry registry:2

Configure Runners

Configure Runner to “Run untagged jobs option”.

Click Save changes for the changes to take effect
Set the maximum job timeout
Verify all registered Runners:
```
gitlab-runner verify
```

Configure Jobs

GitLab looks for a .gitlab-ci.yml file to specify what jobs do. Its format is described at:

https://docs.gitlab.com/ee/ci/yaml/

A tutorial is at:

https://docs.gitlab.com/ee/ci/quick_start/

We don’t use a shared Runner because then you have to enable each project explicitly for the Runner to run its jobs.

Executor

To trigger your build script, the default is to use shell.

Your Runner should show in the Manager Runners page on the GitLab CI Admin Dashboard (topbar > Admin > Runners). Depending on your Runner type, it may be green, blue, or red.

git submodule update –init ls -la

Monitoring

As of Omnibus GitLab 9.0, Prometheus and its related exporters are enabled by default, to enable easy and in depth monitoring of GitLab. Approximately 200MB of memory will be consumed by these processes, with default settings.

If you would like to disable Prometheus and it’s exporters or read more information about it, check the Prometheus documentation. GitLab Ru

Do it

When a developer finishes a set of changes, they can open a merge request, which is equivalent to a “pull request” in GitHub. This is so other contributors to the repo can make comments.

A pipeline is a collection of jobs split in different stages. Every commit pushed to GitLab generates a pipeline attached to that commit. If multiple commits are pushed together the pipeline will be created for the last one only.

API

Based on https://gitlab.com/help/api/runners.md which at time of this writing is v4 with endpoint “https://gitlab.example.com/api/v4/runners”

Name	Call(s)	With a...
Register a new Runner	POST /runners
Verify authentication for a registered Runner	POST /runners/verify
List all runners	GET /runners/all GET /runners/all?scope=online
List owned runners	GET /runners GET /runners?scope=active	scope
Get runner's details	GET /runners/:id
Update runner's details	PUT /runners/:id
Remove a runner	DELETE /runners/:id
List runner's jobs	GET /runners/:id/jobs
List project's runners	GET /projects/:id/runners
Enable a runner in project	POST /projects/:id/runners
Disable a runner from project	DELETE /projects/:id/runners/:runner_id
Delete a registered Runner	DELETE /runners	token

References

This was created based partly on documentation at:

https://docs.gitlab.com/ee/ci/
https://about.gitlab.com/2018/01/22/a-beginners-guide-to-continuous-integration/
https://codereviewvideos.com/course/your-own-private-github/video/gitlab-ci-runner-tutorial
https://neoteric.eu/automate-your-work-with-gitlab-ci-cd-tool

Udemy video: GitLab/GitLab CI Course For Developers and DevOps 7/2017 [2 hrs] by Peng Xiao

More on DevSecOps

This is one of a series on DevSecOps:

Wilson Mar