GitHub Actions

Run pipelines from within GitHub, for free (instead of Jenkins, CircleCI, etc.)

This article describes a production-worthy baseline professional developers and DevSecOps platform engineers can collaborate on refining over time.

https://github.com/bomonike/gha-baseline

At the bottom of this article is my list of video classes, YouTube videos, blogs, and vendor documentation about learning this topic from scratch.

So this aims to be hands-on and deep, yet succinct.

Here we start with our Baseline code.

NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.

Baseline Production example

1. Create a new Git repo (with a README.md).

From-scratch Tutorials

This section summarizes their content.

GitHub added an “Actions” tab to repos (in 2019) to perform Continuous Integration (CI) like Jenkins.

GitHub Actions enables software development teams to configure Infrastructure as Code (IaC) for Continuous Integration for NodeJs and a wide range of programming languages.

When developers can merge and deploy code many times in a single day, they can achieve Agile DevSecOps.

Actions in Jobs triggering Workflows

The “Actions” tab within a repository display Workflows stored within the repo’s .github folder. Notice the leading dot to specify a hidden folder.

Click image to pop-up full-size display.

Within the .github folder is a workflows folder whichcontain declarative yml files. Each “workflow” is a separate yaml file, each an automated process that contain one or more logically related jobs.

Each jobs contains one or more steps – tasks executed through a GitHub Actions YAML config file, such as building source code, run tests, or deploy the code that has been built to some remote server.

Build and run tests jobs can be in the same workflow, with the deployment job into a different workflow.

PROTIP: Within a Workflow file named (for example) “build_and_test.yml”, specify a corresponding name such as:

name: Build and Test

A runner is the remote computer that GitHub Actions uses to execute the jobs. Runners can be local, in AWS. Runners are specified by runs-on: lines such as:

runs-on: ubuntu-latest

In addition to Ubuntu, GitHub provides Microsoft Windows, and macOS runners.

A job is trigged for execution by a GitHub Action when some event occurs. Jobs can be scheduled too. Events are specified by the on: section.

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

DEFINITION: Actions are individual steps within a job – commands that can be reused in your config file. You can write your custom actions or use existing ones.

Each step has a hyphen and name: and uses:. For an example running Python:

    steps:
    - name: Checkout code
      uses: actions/checkout@v2
    - name: Set up Python Environment
      uses: actions/setup-python@v2
      with:
        python-version: '3.x'
    - name: Install Dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt
    - name: Run Tests
      run: |
        python manage.py test

The Scripts folder contain programmatic sh (Bash shell) files which carry out actions.

Run locally

You can run GitHub Actions locally on your laptop using github.com/nektos/act.

https://github.com/cplee/github-actions-demo

Baseline Production example

1. At github.com, navigate to the repo you want to add GitHub Actions:

2. Create new file .github/workflows folder path from the root of your repo.

   This follows the same convention as .circleci.

   Each workflow is defined by a yaml-formatted file.

3. Create a workflow yml file named main.yml

   PROTIP: To start, rather than creating your own a yaml-formatted file to define each Workflow configuration. An example (using NodeJs) from https://github.com/cplee/github-actions-demo/blob/master/.github/workflows/main.yml

4. PROTIP: Create in your internet browser a bookmark so you can return to this quickly.

5. Edit the main.yml workflow file:

   name: 'baseline-workflow'
   # **What it does**: Scan Terraform code. Save results on S3 buckets based on credentials from HashiCorp Vault.
   # **Why we have it**: So secrets are not static in GitHub Actions GUI, needing to be repeated in each Action.
   # **Who does it impact**: Docs content.
   on: [push]
   jobs:
     test-job:
   runs-on: ubuntu-latest
   steps:
   - name: 🚀 Conditions at start
     run: echo "Stats at start of job ..."
   - name: 🫶 Get code
     uses: actions/checkout@v2
   - uses: actions/setup-node@v1
   - run: 🎉 npm install
   - run: npm test
   - name: 🫶 Conditions at end
     run: echo "Stats at end of job ..."
     deploy:
   needs: test-job
     ...
   

</pre>

### Job name & environment

PROTIP: The name value should match the name of the yml file. 
Encase the name value in single quotes if there is a space or other special character.

See https://docs.github.com/en/actions/using-jobs/using-environments-for-jobs

Notice indents are two spaces by default.

PROTIP: Add "-job" at the end of job names 

### on: triggers

<tt>on: push</tt> defines one of the <a target="_blank" href="https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows">>events that trigger</a> a workflow to start:

* watch (repo starred)
* fork (repo forked)
* issues (opened or deleted)
* issue_commment
* create (branch or tag)
* pull_request (opened or closed)

* push (of a commit)
* workflow_dispatch
<br /><br />

### Runners Pricing

REMEMBER: Each job has its own runner (virtual machine isolated from other jobs)

<tt>runs-on:</tt> defines the <a target="_blank" href="https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners">runner</a> within a GitHub hosted environment. Instead of <tt>ubuntu-latest</tt> a version specification can be specified. Alternately, <a href="#MatrixVariations">several versions</a>.

<tt>with:</tt> configures the runner.

CAUTION: See <a target="_blank" href="https://docs.github.com/en/billing/managing-billing-for-github-actions/about-billing-for-github-actions">cost implications</a> depending on the platform, number CPU cores, etc.

GiHub pre-installs Clang, Bash, Python, Node, etc. for use on each runner.

### Steps with emojis

PROTIP: Use emoji's to visually differentiate step names.
❤️ Initial greeting<br />
👀 Verify Terraform<br />
🫶 Goodbye<br />

* https://emojipedia.org/

### Sample code

https://github.com/SamGallagher95/best-terraform-cd-article/tree/main/terraform

<a name="Marketplace"></a>

### Actions Marketplace

CAUTION: GitHub currently does not dynamically scan 3rd-party actions for malicious activity.

Among 3rd-party Actions in GitHub's public Marketplace, <a target="_blank" href="https://github.com/marketplace?category=&query=sort%3Apopularity-desc&type=actions&verification=">sorted by number of stars</a>:

* https://github.com/marketplace/actions/super-linter (from GitHub)
* https://github.com/marketplace/actions/trufflehog-oss to scan for leaked secrets
* https://github.com/marketplace/actions/configure-aws-credentials-action-for-github-actions
* https://github.com/marketplace/actions/checkout to a specific version of your GitHub repo
</br /><br />

"Verified creator" only means that GitHub has been able to contact the creator.

<tt>needs: test</tt> enforces a dependency to finish successfully.

### Environment secrets

To create buckets in S3 or other AWS services, sepecify:

<pre>env:
   GITHUB_TOKEN: $
   AWS_ACCESS_KEY_ID: $
</pre>

1. Save the changes with a comment. Click the green “Start commit”.

2. To view the status of workflows, press the Actions tab at the top menu.

   PROTIP: To get to the top of the screen to see GitHub’s Tabs, on macOS, press command + up_arrow.

   

3. Wait past “Queued” to click the run at the top of the list.

4. Click a job box with green check icon to see step info.

   Set up job and Complete job (“Cleaning up orphan processes”) are added by GitHub.

   PAT

5. In GitHub Settings > Developer Settings > Define a PAT (Personal Access Token) for expiration in 30 days.

   PROTIP: For note, add a time stamp such as “expires 23-12-31”.

   Select scopes repo and workflow

6. Click “Set up a workflow yourself” or select a template containing pre-populated yml files from various people.

   PROTIP: You can create and share templates for use by others in your own organization. See https://help.github.com/en/actions/hosting-your-own-runners

7. PROTIP: Protect the master branch so it can’t be inadvertently deleted or broken.

8. PROTIP: Setup required reviews so that any pull requests are double checked by teammates.

   Strategy Matrix of variations

   The “ubuntu-vers” job in the code here run each possible combination of variables, one for each combination of the version and os.

   jobs:
     ubuntu-vers:
   strategy:
     matrix:
       version: [10, 12, 14]
       os: [ubuntu-latest]
   

   See https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs

Community

https://github.community/t5/GitHub-Actions/bd-p/actions

Sample NPM workflow

1. Let’s look at a yaml workflow file used by GitHub Actions.

   https://github.com/wilsonmar/ci-with-actions/blob/master/github-actions-for-ci/.github/workflows/nodejs.yml

   A workflow is a unit of automation from start to finish, including the definition of what triggers the automation, what environment or other aspects should be taken account during the automation, and what should happen as a result of the trigger.

2. See https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions

   on: scheduled actions

   on: specifications inside that file define a scheduled time when the workflow is triggered.

   Alternately, workflows can be triggered by events in or outside GitHub, such as a git push or a scheduled time.

   The default trigger is to run on every push to every branch:

   on: [push]
   

   This example is triggered upon a push to either the master branch or a release branch:

   on:
     push:
    branches:
    - master
    - release/*
   

   In this example, the workflow is triggered to run the master branch anytime there’s a push or pull request.

   on:
     push:
    branches: [ master ]
     pull_request:
    branches: [ master ]
   

   PROTIP: To set a workflow (using crontab specifications) to run at 2:00 AM UTC every day, 1=Monday to 5=Friday:

   on:
     schedule:
            - cron: "0 2 * * 1-5"
   

   1. Minute 0 to 59, or * (no specific value)
   2. Hour 0 to 23, or * for any value. All times UTC.
   3. Day of the month 1 to 31, or * (no specific value)
   4. Mont 1 to 12, or * (no specific value)
   5. Day of the wee 0 to 7 (0 and 7 both represent Sunday), or * (no specific value)

   jobs: block

   Workflows are made of jobs, and the template workflow defines a single job with the identifier build.

   jobs:
     build:
    name: 'Build'
    runs-on: ubuntu-latest
    

   Several jobs: blocks define different sections of a Workflow.

   runs-on: job host environment

   Every job needs a specific host machine specified by the runs-on: field. This template workflow specifies using the latest version of Ubuntu, a Linux-based operating system.

   Others:

   • ubuntu-latest, ubuntu-18.04, or ubuntu-16.04
   • windows-latest or windows-2019
   • macos-latest or macos-10.15
     
     

   The above specify the Runner within a GitHub hosted environment or a self-hosted environment.

   Ubuntu contains Docker.

   Alternately,

   runs-on: $ refers to the “os” alternatives in the strategy section.

   job strategy: matrix

   A Job Matrix is designed to build and test code with different environments and configurations.

      strategy:
      matrix:
        node-version: [10.x, 12.x]
        os: [ubuntu-latest, windows-latest, macOS-latest]
   

   PROTIP: The code above defines variable $ which resolves to “10.x”, or “12.x” when referenced in the set of steps below, which are repeated automatically for each node-version specified.

   CAUTION: Reference the list of releases for the language you’re using, such as this one for NodeJs.

   You can also vary the host operating system environment:

      strategy:
      matrix:
        node-version: [10, 12, 14]
        os: [ubuntu-latest, windows-latest, macOS-latest]
   

   The above would generate 3 x 3 = 9 job runs.

   PROTIP: Different jobs in the matrix are run simultaneously.

   Cost of GitHub Actions jobs

   GitHub charges on a “pay as you go” basis two ways: by the minute used by each job and what operating system:

   

   There are limits on the number of concurrent jobs: Enterprise licensees have a limit of 180 jobs, of which 50 are macOS jobs, but only 5 macOS jobs for others. Even free accounts get up to 20 concurrent jobs. 40 for those who pay $4 a month. Each team gets 60 jobs at a time.

   PROTIP: A job matrix can generate a maximum of 256 jobs per workflow run. This limit also applies to self-hosted runners.

   Steps

   Each job is made up of one or more steps. In the sample template:

       steps:
    - uses: actions/checkout@v2
    - name: Use Node.js $
      uses: actions/setup-node@v1
      with:
        node-version: $
    - run: npm ci
    - run: npm run build --if-present
    - run: npm test
    

   - (a dash) precedes each action.

   Echo

   Issue a message by running an echo command:

   jobs:
   build:
      steps:
        name: Run one-liner
        run: echo Hello, world!
   

   Notice no quote characters.

   - uses: step in Actions coding

   - uses: actions/checkout@v2

   actions defines an action from GitHub’s public Marketplace of Actions.

   checkout@v2 retrieves the latest (such as v2.1.0) in https://github.com/actions/checkout/releases. The action’s home page is at https://github.com/marketplace/actions/checkout

   PROTIP: Monitor when versions are updated. When an upgrade is available, search through GitHub repos to see which ones should be upgraded.

   - name: step in Actions coding

   Because the Node.js version needs to be specified several times:

   - run: step in Actions coding

       - run: npm ci
    - run: npm run build --if-present
    - run: npm test
   

   npm ci was introduced in NodeJs 5.1 (2018) in place of “npm install” (or yarn) for faster downloading and installation of package dependencies (based on specifications in the package.json file) into the node_modules folder.

   BTW the new GitHub Package Registry only supports npm as a client for JavaScript packages (at least for now).

   npm run build runs the build field defined in the scripts field within package.json.

   BTW npm build no longer exists as of 2019.

   --if-present is an optional flag to avoid exiting with a non-zero exit code when the script is undefined.

   npm test executes all tests defined.

   PROTIP: Consider separate test jobs to separate build from test details.

   build and publish

   PROTIP: Include where you’re publishing if you’re publishing to the gpr (Google Package Registry) as well as NPM.

     jobs:
   build:
      ...
   publish-npm:
      ...
   publish-gpr:
   

   Slack notification

3. Post to a Slack channel when a new issue is added on GitHub:

     name: Slack Issue
    on:
      issues:
         types: [opened]
     job:
     post_slack_message:
       runs-on: ubuntu-latest
       steps:
         - uses: rtCamp/action-slack-notify@2.0.0
         - env:
             SLACK_WEBHOOK: $
             SLACK_USERNAME: memyselfandi
             SLACK_CHANNEL: gh-issues
   

   Clear-text of secrets are input in the Security tab.

   env: ci: true

      env:
   ci: true
   

Sample repo for GitHub’s Tutorial

A sample repo was provided in VIDEO: Continuous integration with GitHub Actions [1:55:24] at GitHub Satellite 2020 on 7 May 2020

• Create and use multiple, customized workflows
• Implement a unit testing framework using GitHub Actions
• Use multiple jobs in a workflow and pass artifacts between jobs
• Configure a repository to work in conjunction with GitHub Actions workflows and your team’s workflow.
  
  

curl https://api.github.com/octocat

1. Go to and fork

   https://github.com/githubsatelliteworkshops/ci-with-actions

   BLAH: The pdf in the link satellite-2020-workshops-ci-with-actions.pdf does not have links enabled.

   • @pprmk, Sr. Implementation Engineer
   • @dechyper, Solutions Architect
   • @iamhughes, Sr. DevOps Engineer
     
     

2. Throughout the course, return to the list of course agenda at:

   https://git.io/Jewra which goes to
   https://lab.github.com/githubtraining/github-actions:-continuous-integration

3. Click “Start free course”. You may be asked to login GitHub.
4. [20:01] Choose either “Public” or “Private”, then “Begin GitHub Actions: Continuous Integration”.
5. [20:35] Wait for message “you can start your first step”. Scroll down to notice the other courses.

6. Among the 16 steps:

   1. Use a templated workflow Create a pull request with a templated workflow

   2. Run a templated workflow Wait for GitHub to run the templated workflow and report back the results

   3. Add your first test Add your first test script for CI to pick up

   4. Read an Actions log Tell the bot which test is failing so we can fix it

   5. Fix the test Edit the file that’s causing the test to fail

   6. Share the workflow with the team Merge the pull request containing your first workflow so the entire team can use it

   7. Create a custom GitHub Actions workflow Edit the existing workflow with new build targets

   8. Target a Windows environment Edit your workflow file to build for Windows environments

   9. Use multiple jobs Edit your workflow file to separate build and test jobs

   10. Run multiple jobs Wait for the result of multiple jobs in your workflow

   11. Upload a job’s build artifacts Use the upload action in your workflow file to save a job’s build artifacts

   12. Download a job’s build artifacts Use the download action in your workflow file to access a prior job’s build artifacts

   13. Share the improved CI workflow with the team Merge the pull request with the improved workflow

   14. Automate the review process Add a new workflow file to automate the team’s review process

   15. Use an action to automate pull request reviews Use the community action in your new workflow

   16. Create an approval job in your new workflow In your new workflow file, create a new job that’ll use the community action

   17. Automate approvals Use the community action to automate part of the review approval process

   18. Use branch protections Complete the automated review process by protecting the master branch

7. [29:34] Click “Start: Use a templated workflow” for the Issue#1 page on your own repo such as this (but with your name instead):

   https://github.com/wilsonmar/github-actions-for-ci/issues/1

8. [30:58] Click Actions tab, click “Set up this workflow” or navigate within the repo’s .github/workflows folder to edit file nodejs.yml (the Actions file).

9. Copy “Paste “CI for Node” into your invisible Clipboard.
10. [31:29] Click “Start commit” to a new branch.
11. [32:02] Commit new file.
12. [32:05] Double-click to select all of the suggested name to Paste “CI for Node” insted. Click “Create pull request”.
13. [32:22] “Review required” and “Merging is blocked” apprears until …
14. [32:44] Click on “Details” or Actions tab to see jobs running. Click on a build.

15. [33:11] Click “Pull Requests” tab to return to “CI for Node”.

    Vocabulary is defined by the bot.

16. [34:08] Add your first test: Click “Pull requests” tab. Click “Add Jest tests”. Click “Merge pull request”.

17. [34:53] Click “Delete branch”.

    Read Actions Log

18. [35:06] Click on “next step” (created by the bot).
19. [35:42] Navigate to the log output: Click on “Actions” tab.
20. [36:22] Click the latest “CI for Node” run (at the top. Click a build. Identify a name of a failing test with red “x”. Expand it by clicking it.
21. [36:48] Identify the name “Initialize with two players” and copy it.

22. [37:23] To “Pull requests”. In the Comment paste the name of the failing test. Click “Comment”.

    Fix the test

23. [37:50] Click “Commit suggestion” of “Update src/game.js”.
24. [38:10] Click “Commit Changes”.

25. [38:47] Refresh screen until bot makes “Changes approved”.

    Share

26. [39:34] Click “Merge pull request”. “Confirm Merge”. “Delete branch”.

27. [56:11] https://github.com/wilsonmar/github-actions-for-ci/issues/1

    Step 7: (Work Session 2) Create a custom GitHub Actions workflow

28. [1:06:43] Click “Resume”.

Create Badge

Within Actions tab:

AWS in GitHub Actions

First, preconfigure the IAM IdP in your AWS account (see Assuming a Role for details).

Configure your AWS credentials and region environment variables for use in GitHub Actions, add action https://github.com/aws-actions/configure-aws-credentials

    - name: Configure AWS Credentials
      uses: aws-actions/configure-aws-credentials@v2
      with:
        role-to-assume: arn:aws:iam::123456789100:role/my-github-actions-role
        aws-region: us-east-2

the action implements the AWS SDK credential resolution chain and exports environment variables for other Actions to use.

v2 of the action uses the Node 16 runtime by default.

This causes the action to perform an AssumeRoleWithWebIdentity call and return temporary security credentials for use by other actions.

https://www.freecodecamp.org/news/how-to-setup-a-ci-cd-pipeline-with-github-actions-and-aws/

Environment variable exports are detected by both the AWS SDKs and the AWS CLI for AWS API calls.

https://github.com/aws-actions

Alternately, https://www.freecodecamp.org/news/how-to-setup-a-ci-cd-pipeline-with-github-actions-and-aws/ Use the AWS Elastic Beanstalk compute service pulled from AWS S3 buckets uploaded from GitHub.

1. Setup an AWS Account
2. Get into Elastic Beanstalk environment https://us-west-2.console.aws.amazon.com/elasticbeanstalk/home?region=us-west-2#/welcome
3. “Create Application” (formerly “Create a New Environment”).
4. Application name: PROTIP: Type your name so it’s unique.
5. Application tags
6. Platform: Choose Python if you like.
7. Platform branch
8. Platform version
9. Application code: select “Sample application” or “Upload your code”.
10. Click “Create application”.
11. Grab the application name and the environment name at the upper-left: Wilson230321-env
    http://wilson230321-env.eba-iqusqyih.us-west-2.elasticbeanstalk.com/

What’s Next?

• AWS Elastic Beanstalk overview - What is AWS Elastic Beanstalk?
• AWS Elastic Beanstalk concepts
• Deploying a Django Application to AWS Elastic Beanstalk
• Deploy a Flask Application to AWS Elastic Beanstalk
• Using the Elastic Beanstalk Python platform (Customizing and Configuring a Python Container)
• Working with Logs

The new Elastic Beanstalk environment management console described at:
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/environments-console.html such as:
https://us-west-2.console.aws.amazon.com/elasticbeanstalk/home?region=us-west-2#/environments

Video classes

https://www.udemy.com/course/github-actions-the-complete-guide/ 10.5 hour $16.99 GitHub Actions - The Complete Guide Nov 2022 referencing https://github.com/academind/github-actions-course-resources by Maximilian Schwarzmuller

YouTube videos

https://www.youtube.com/watch?v=T6sW1Dk9B4E What every GitHub user should know about VS Code - GitHub Satellite 2020 24:08

Continuous integration with GitHub Actions presented at GitHub Satellite 2020 7 May, 2020

[2] Advanced GitHub Actions: workflows for production grade CI/CD - GitHub Universe 2019

GitHub Actions Now Supports CI/CD | Getting Started Hong Ly

GitHub Actions (CI/CD Flow) Coding Tech

5 Ways to DevOps-ify your App - Github Actions Tutorial Fireship

Introduction to GitHub Actions [38:46] by BlackMarbleLtd CEO @RichardFennell references https://github.com/rfennell/ActionPlayground/blob/master/src/helloworld.ts (Typescript)

Introducing GitHub Package Registry GitHub

GitHub Actions: Open Source Workflow Automation by Bas Peters DATA MINER

Github Actions | Open CICD Platform by Github by Tech Primers

https://www.youtube.com/watch?v=F3wZTDmHCFA GitHub Actions: How to Set Up a Simple Workflow CodingWithChandler

https://www.youtube.com/watch?v=2Ym94MfScZ4 GitHub Actions CI/CD Workflow for a Laravel Application - Part 1: Introduction Oh See Media

VIDEO: Unlocking the Cloud Operating Model with GitHub Actions by Steve Winton, Senior Partner Engineer, GitHub

https://github.com/actionsdesk

https://dev.to/github/export-github-issues-commit-history-and-more-github-artifact-exporter-2ok6 Export GitHub Issues, Commit History and More | GitHub Artifact Exporter by Davide ‘CoderDave’ Benvegnù

• VIDEO “GitHub Actions Tutorial - Basic Concepts and CI/CD Pipeline with Docker” by TechWorld with Nana

• VIDEO: Visual Studio Toolbox at Microsoft:

• VIDEO: Automatic Deployment With Github Actions Traversy Media

• https://sanderknape.com/2021/01/go-crazy-github-actions/

• Github Actions and GitOps in One Hour Video Course by Alfredo Deza and Noah Gift

Blogs

https://coletiv.com/blog/how-to-setup-continuous-integration-and-deployment-workflows-for-reactjs-using-github-actions/

Documentation

GitHub Actions Documentation is at https://help.github.com/en/actions

Setup Continuous Integrations

https://help.github.com/en/categories/automating-your-workflow-with-github-actions

https://help.github.com/en/actions/configuring-and-managing-workflows

https://help.github.com/en/actions/language-and-framework-guides

https://help.github.com/en/actions/migrating-to-github-actions

More

This is one of a series on Git and GitHub:

1. Git and GitHub videos

2. Why Git? (file-based backups vs Git clone)

3. Git Markdown text

4. Git basics (script)
5. Git whoops (correct mistakes)

6. Git messages (in commits)

7. Git command shortcuts

8. Git custom commands

9. Git-client based workflows

10. Git HEAD (Commitish references)

11. Git interactive merge (imerge)
12. Git patch

13. Git rebase

14. Git utilities

15. Git-signing

16. Git hooks
17. GitHub data security

18. TFS vs GitHub

19. GitHub actions for automation JavaScript
20. GitHub REST API
21. GitHub GraphQL API
22. GitHub PowerShell API Programming
23. GitHub GraphQL PowerShell Module