Run pipelines from within GitHub, for free (instead of Jenkins, CircleCI, etc.)
Overview
This tutorial is a step-by-step hands-on deep yet succinct introduction to using GitHub’s Actions to build at low cost, quickly.
NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.
GitHub Actions enables software development teams to configure Infrastructure as Code (IaC) for Continuous Integration for NodeJs and a wide range of programming languages.
When developers can merge and deploy code many times in a single day, they can achieve Agile DevOps.
Documentation
GitHub Actions Documentation is at https://help.github.com/en/actions
https://help.github.com/en/categories/automating-your-workflow-with-github-actions
https://help.github.com/en/actions/configuring-and-managing-workflows
https://help.github.com/en/actions/language-and-framework-guides
https://help.github.com/en/actions/migrating-to-github-actions
Actions in Jobs triggering Workflows
Click image to pop-up full-size display.
DEFINITION: In GitHub Actions, a workflow is a configurable automated process made up of one or more jobs.
DEFINITION: Workflows are run (invoked) by Runners within a GitHub hosted environment or a self-hosted environment.
Actions are individual steps within a job.
Run locally
Actually, you can run GitHub Actions locally on your laptop using github.com/nektos/act.
https://github.com/cplee/github-actions-demo
Hello World
-
Create a new Git repo.
-
Create a .github folder within your repository.
This follows the same convention as .circleci.
-
Create a workflows folder within your repository.
Each workflow is defined by a yaml-formatted file.
PROTIP: To start, rather than creating your own a yaml-formatted file to define each Workflow configuration.
An example (using NodeJs) from https://github.com/cplee/github-actions-demo/blob/master/.github/workflows/main.yml
name: CI on: push jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - uses: actions/setup-node@v1 - run: npm install - run: npm test
-
To view the status of workflows, press the Actions tab at the top menu.
PROTIP: To get to the top of the screen to see GitHub’s Tabs, on macOS, press command + up_arrow.
-
Click “Set up a workflow yourself” or select a template containing pre-populated yml files from various people.
PROTIP: You can create and share templates for use by others in your own organization. See https://help.github.com/en/actions/hosting-your-own-runners
-
PROTIP: Protect the master branch so it can’t be inadvertently deleted or broken.
-
PROTIP: Setup required reviews so that any pull requests are double checked by teammates.
Community
https://github.community/t5/GitHub-Actions/bd-p/actions
Sample NPM workflow
-
Let’s look at a yaml workflow file used by GitHub Actions.
A workflow is a unit of automation from start to finish, including the definition of what triggers the automation, what environment or other aspects should be taken account during the automation, and what should happen as a result of the trigger.
-
See https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions
on: scheduled actions
on:
specifications inside that file define a scheduled time when the workflow is triggered.Alternately, workflows can be triggered by events in or outside GitHub, such as a git push or a scheduled time.
The default trigger is to run on every push to every branch:
on: [push]
This example is triggered upon a push to either the master branch or a release branch:
on: push: branches: - master - release/*
In this example, the workflow is triggered to run the master branch anytime there’s a push or pull request.
on: push: branches: [ master ] pull_request: branches: [ master ]
PROTIP: To set a workflow (using crontab specifications) to run at 2:00 AM UTC every day, 1=Monday to 5=Friday:
on: schedule: - cron: "0 2 * * 1-5"
- Minute 0 to 59, or * (no specific value)
- Hour 0 to 23, or * for any value. All times UTC.
- Day of the month 1 to 31, or * (no specific value)
- Mont 1 to 12, or * (no specific value)
- Day of the wee 0 to 7 (0 and 7 both represent Sunday), or * (no specific value)
jobs: block
Workflows are made of jobs, and the template workflow defines a single job with the identifier build.
jobs: build: name: 'Build' runs-on: ubuntu-latest
Several
jobs:
blocks define different sections of a Workflow.runs-on: job host environment
Every job needs a specific host machine specified by the runs-on: field. This template workflow specifies using the latest version of Ubuntu, a Linux-based operating system.
- ubuntu-latest, ubuntu-18.04, or ubuntu-16.04
- windows-latest or windows-2019
- macos-latest or macos-10.15
The above specify the Runner within a GitHub hosted environment or a self-hosted environment.
Ubuntu contains Docker.
Alternately,
runs-on: $
refers to the “os” alternatives in the strategy section.job strategy: matrix
A Job Matrix is designed to build and test code with different environments and configurations.
strategy: matrix: node-version: [10.x, 12.x] os: [ubuntu-latest, windows-latest, macOS-latest]
PROTIP: The code above defines variable
$
which resolves to “10.x”, or “12.x” when referenced in the set of steps below, which are repeated automatically for each node-version specified.CAUTION: Reference the list of releases for the language you’re using, such as this one for NodeJs.
You can also vary the host operating system environment:
strategy: matrix: node-version: [10, 12, 14] os: [ubuntu-latest, windows-latest, macOS-latest]
The above would generate 3 x 3 = 9 job runs.
PROTIP: Different jobs in the matrix are run simultaneously.
Cost of GitHub Actions jobs
GitHub charges on a “pay as you go” basis two ways: by the minute used by each job and what operating system:
There are limits on the number of concurrent jobs: Enterprise licensees have a limit of 180 jobs, of which 50 are macOS jobs, but only 5 macOS jobs for others. Even free accounts get up to 20 concurrent jobs. 40 for those who pay $4 a month. Each team gets 60 jobs at a time.
PROTIP: A job matrix can generate a maximum of 256 jobs per workflow run. This limit also applies to self-hosted runners.
Steps
Each job is made up of one or more steps. In the sample template:
steps: - uses: actions/checkout@v2 - name: Use Node.js $ uses: actions/setup-node@v1 with: node-version: $ - run: npm ci - run: npm run build --if-present - run: npm test
-
(a dash) precedes each action.Echo
Issue a message by running an echo command:
jobs: build: steps: name: Run one-liner run: echo Hello, world!
Notice no quote characters.
- uses: step in Actions coding
- uses: actions/checkout@v2
actions
defines an action from GitHub’s public Marketplace of Actions.checkout@v2
retrieves the latest (such as v2.1.0) in https://github.com/actions/checkout/releases. The action’s home page is at https://github.com/marketplace/actions/checkoutPROTIP: Monitor when versions are updated. When an upgrade is available, search through GitHub repos to see which ones should be upgraded.
- name: step in Actions coding
Because the Node.js version needs to be specified several times:
- run: step in Actions coding
- run: npm ci - run: npm run build --if-present - run: npm test
npm ci
was introduced in NodeJs 5.1 (2018) in place of “npm install” (or yarn) for faster downloading and installation of package dependencies (based on specifications in the package.json file) into the node_modules folder.BTW the new GitHub Package Registry only supports npm as a client for JavaScript packages (at least for now).
npm run build
runs the build field defined in the scripts field within package.json.BTW npm build no longer exists as of 2019.
--if-present
is an optional flag to avoid exiting with a non-zero exit code when the script is undefined.npm test
executes all tests defined.PROTIP: Consider separate test jobs to separate build from test details.
build and publish
PROTIP: Include where you’re publishing if you’re publishing to the gpr (Google Package Registry) as well as NPM.
jobs: build: ... publish-npm: ... publish-gpr:
Slack notification
-
Post to a Slack channel when a new issue is added on GitHub:
name: Slack Issue on: issues: types: [opened] job: post_slack_message: runs-on: ubuntu-latest steps: - uses: rtCamp/action-slack-notify@2.0.0 - env: SLACK_WEBHOOK: $ SLACK_USERNAME: memyselfandi SLACK_CHANNEL: gh-issues
Clear-text of secrets are input in the Security tab.
env: ci: true
env: ci: true
Sample repo for GitHub’s Tutorial
A sample repo was provided in VIDEO: Continuous integration with GitHub Actions [1:55:24] at GitHub Satellite 2020 on 7 May 2020
- Create and use multiple, customized workflows
- Implement a unit testing framework using GitHub Actions
- Use multiple jobs in a workflow and pass artifacts between jobs
- Configure a repository to work in conjunction with GitHub Actions workflows and your team’s workflow.
curl https://api.github.com/octocat
-
Go to and fork
https://github.com/githubsatelliteworkshops/ci-with-actions
BLAH: The pdf in the link satellite-2020-workshops-ci-with-actions.pdf does not have links enabled.
- @pprmk, Sr. Implementation Engineer
- @dechyper, Solutions Architect
- @iamhughes, Sr. DevOps Engineer
-
Throughout the course, return to the list of course agenda at:
- Click “Start free course”. You may be asked to login GitHub.
- [20:01] Choose either “Public” or “Private”, then “Begin GitHub Actions: Continuous Integration”.
- [20:35] Wait for message “you can start your first step”. Scroll down to notice the other courses.
-
Among the 16 steps:
-
Use a templated workflow Create a pull request with a templated workflow
-
Run a templated workflow Wait for GitHub to run the templated workflow and report back the results
-
Add your first test Add your first test script for CI to pick up
-
Read an Actions log Tell the bot which test is failing so we can fix it
-
Fix the test Edit the file that’s causing the test to fail
-
Share the workflow with the team Merge the pull request containing your first workflow so the entire team can use it
-
Create a custom GitHub Actions workflow Edit the existing workflow with new build targets
-
Target a Windows environment Edit your workflow file to build for Windows environments
-
Use multiple jobs Edit your workflow file to separate build and test jobs
-
Run multiple jobs Wait for the result of multiple jobs in your workflow
-
Upload a job’s build artifacts Use the upload action in your workflow file to save a job’s build artifacts
-
Download a job’s build artifacts Use the download action in your workflow file to access a prior job’s build artifacts
-
Share the improved CI workflow with the team Merge the pull request with the improved workflow
-
Automate the review process Add a new workflow file to automate the team’s review process
-
Use an action to automate pull request reviews Use the community action in your new workflow
-
Create an approval job in your new workflow In your new workflow file, create a new job that’ll use the community action
-
Automate approvals Use the community action to automate part of the review approval process
-
Use branch protections Complete the automated review process by protecting the master branch
-
-
[29:34] Click “Start: Use a templated workflow” for the Issue#1 page on your own repo such as this (but with your name instead):
https://github.com/wilsonmar/github-actions-for-ci/issues/1
-
[30:58] Click Actions tab, click “Set up this workflow” or navigate within the repo’s .github/workflows folder to edit file nodejs.yml (the Actions file).
- Copy “Paste “CI for Node” into your invisible Clipboard.
- [31:29] Click “Start commit” to a new branch.
- [32:02] Commit new file.
- [32:05] Double-click to select all of the suggested name to Paste “CI for Node” insted. Click “Create pull request”.
- [32:22] “Review required” and “Merging is blocked” apprears until …
- [32:44] Click on “Details” or Actions tab to see jobs running. Click on a build.
-
[33:11] Click “Pull Requests” tab to return to “CI for Node”.
Vocabulary is defined by the bot.
- [34:08] Add your first test: Click “Pull requests” tab. Click “Add Jest tests”. Click “Merge pull request”.
-
[34:53] Click “Delete branch”.
Read Actions Log
- [35:06] Click on “next step” (created by the bot).
- [35:42] Navigate to the log output: Click on “Actions” tab.
- [36:22] Click the latest “CI for Node” run (at the top. Click a build. Identify a name of a failing test with red “x”. Expand it by clicking it.
- [36:48] Identify the name “Initialize with two players” and copy it.
-
[37:23] To “Pull requests”. In the Comment paste the name of the failing test. Click “Comment”.
Fix the test
- [37:50] Click “Commit suggestion” of “Update src/game.js”.
- [38:10] Click “Commit Changes”.
-
[38:47] Refresh screen until bot makes “Changes approved”.
Share
-
[39:34] Click “Merge pull request”. “Confirm Merge”. “Delete branch”.
-
[56:11] https://github.com/wilsonmar/github-actions-for-ci/issues/1
Step 7: (Work Session 2) Create a custom GitHub Actions workflow
- [1:06:43] Click “Resume”.
Create Badge
Within Actions tab:
References
https://www.youtube.com/watch?v=T6sW1Dk9B4E What every GitHub user should know about VS Code - GitHub Satellite 2020 24:08
Continuous integration with GitHub Actions presented at GitHub Satellite 2020 7 May, 2020
[2] Advanced GitHub Actions: workflows for production grade CI/CD - GitHub Universe 2019
GitHub Actions Now Supports CI/CD | Getting Started Hong Ly
GitHub Actions (CI/CD Flow) Coding Tech
5 Ways to DevOps-ify your App - Github Actions Tutorial Fireship
Introduction to GitHub Actions [38:46] by BlackMarbleLtd CEO @RichardFennell references https://github.com/rfennell/ActionPlayground/blob/master/src/helloworld.ts (Typescript)
Introducing GitHub Package Registry GitHub
GitHub Actions: Open Source Workflow Automation by Bas Peters DATA MINER
Github Actions | Open CICD Platform by Github by Tech Primers
https://www.youtube.com/watch?v=F3wZTDmHCFA GitHub Actions: How to Set Up a Simple Workflow CodingWithChandler
https://www.youtube.com/watch?v=2Ym94MfScZ4 GitHub Actions CI/CD Workflow for a Laravel Application - Part 1: Introduction Oh See Media
https://coletiv.com/blog/how-to-setup-continuous-integration-and-deployment-workflows-for-reactjs-using-github-actions/
VIDEO: Unlocking the Cloud Operating Model with GitHub Actions by Steve Winton, Senior Partner Engineer, GitHub
https://github.com/actionsdesk
https://dev.to/github/export-github-issues-commit-history-and-more-github-artifact-exporter-2ok6 Export GitHub Issues, Commit History and More | GitHub Artifact Exporter by Davide ‘CoderDave’ Benvegnù
-
VIDEO “GitHub Actions Tutorial - Basic Concepts and CI/CD Pipeline with Docker” by TechWorld with Nana
-
VIDEO: Visual Studio Toolbox at Microsoft:
-
VIDEO: Automatic Deployment With Github Actions Traversy Media
https://sanderknape.com/2021/01/go-crazy-github-actions/
More
This is one of a series on Git and GitHub:
- Why Git? (file-based backups vs Git clone)
- Git basics (script)
- Git whoops (correct mistakes)
- Git command shortcuts
- Git interactive merge (imerge)
- Git patch
- Git utilities
- Git hooks
- GitHub data security
- GitHub actions for automation JavaScript
- GitHub REST API
- GitHub GraphQL API
- GitHub PowerShell API Programming
- GitHub GraphQL PowerShell Module