GitHub Hooks

Let’s all hold hands and sing songs

Overview

What Hooks can do
Service hooks
- Git Notifier
Receivers first
- Begin from Jenkins
Set up CRSF on Jenkins
Get GitHub secret
Set up webhook on GitHub
- Test if it works
- Pull Request
SSH Agent Forwarding
References
More about Git & GitHub

This article shows you how to install and configure GitHub hooks into Jenkins version 2 for Continuous Delivery (CD) as well as Continuouse Integration (CI).

I want you to feel confident that you’ve mastered this skill. That’s why this takes a hands-on approach where you type in commands and we explain the responses and possible troubleshooting. This is a “deep dive” because all details are presented.

Like a good music DJ, I’ve carefully arranged the presentation of concepts into a sequence for easy learning, so you don’t have to spend as much time as me making sense of the flood of material around this subject.

Sentences that begin with PROTIP are a high point of this website to point out wisdom and advice from experience. NOTE point out observations that many miss. Search for them if you only want “TL;DR” (Too Long Didn’t Read) highlights.

Stuck? Contact me and I or one of my friends will help you.

What Hooks can do

GitHub’s Repository WebHooks API is described at
https://developer.github.com/v3/repos/hooks

GitHub has an email hook that sends out an email for each change committed.

This repo provides code to email detailed diffs of each commit. It addresses security configurations.

I would like to see each commit appear on my Gmail calendar to help me see commits in context of my other appointments.

https://gist.github.com/ngs/3550670
https://zapier.com/zapbook/github/google-calendar/
https://zapier.com/zapbook/cronofy-calendar-api/github/

PythonAnywhere.com

The documentation is at https://developer.github.com/webhooks/

Service hooks

Services hooks are pre-built within GitHub to take action when events occur on GitHub.

On the GitHub project sending the hook.
Click Settings on the top bar.
Click Webhooks & Services (previously “Service Hooks”).
Click Browse the directory.
Search for emails.

Git Notifier

http://www.icir.org/robin/git-notifier/

Alternately:

https://www.git-notifier.com
Activate via the Google account.
Specify the Account/Repo and the email to be notified.

Emails are sent free. But phone message cost one Euro each.

Receivers first

We would like Jenkins to attempt a new build when a change is committed in GitHub.com or pushed to it from Git.

The alternative to this is polling on a scheduled interval, which can be a little bit inefficient if nothing was changed. However, a regular schedule is useful when people work strict hours.

Begin from Jenkins

In Manage Jenkins | Manage Plugins, Available tab, find “GitHub plugin” at
https://wiki.jenkins-ci.org/display/JENKINS/Github+Plugin
Return to the Jenkins Dashboard.
Create a new item or click an existing build job.
Select Configure from the menu.
In v2, under Build Triggers, Check “GitHub project”.

In v1, scroll to “Source Code Management”. Check “Git”.
Specify the Project URL.
Check “Build when a change is pushed to GitHub”.

Set up CRSF on Jenkins

Cross Site Request Forgery (CRSF) exploits Cross-origin_resource_sharing, so all PUTs are excluded, except when an exception is added in Jenkins.
Type in the Filter field CORS support for Jenkins until the plugin appears on the Manage Jenkins, Plugin Manager, Available page.
If it appears, check the box under “Installed” .

If it doesn’t appear, click on the Installed tab to see if it’s there already.
If you click on the link, the name is “Cors Filter Plugin” on
https://wiki.jenkins-ci.org/display/JENKINS/CSRF+Protection
“Prevent Cross Site Request Forgery exploits” needs to be unchecked disabled in order for webhooks to work.
In the authentication section, near the bottom,
check the box “Prevent Cross Site Request Forgery exploits”.
Click “Install without restart” of the Jenkins server.

The response is temporarily until “Success” appears.
```
Checking internet connectivity
Checking update center connectivity
Success
```
Click “Restart Jenkins when no jobs are running”.

Get GitHub secret

The below describes how Jenkins ensures security by making use of a “deploy key” where a public-private SSH key pair is generated for each GitHub SCM repository. Each key pair is attached to a repository instead of to a personal user account. The public key is stored on the Jenkins server. The correpsonding public key is stored in the repository along with the code. Jenkins matches the public and public keys before granting access to each single GitHub repository.

https://developer.github.com/guides/managing-deploy-keys
discusses the advantage of “Deploy keys” versus other approaches for authentication and authorization.

SSH into a command-line in your Jenkins server</a>, generate a keypair:

ssh-keygen -C “jenkins123@xyz.com”

The default is RSA.

PROTIP: Some enterprises have a central list of servers with unique “service account” names associated with a server (hardware) instead of email addresses for individual people.

NOTE: Organizations that don’t believe in SSH have an alternative to Jenkins provide a UI to generate keys.

The response:
```
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
```
“id_rsa” is the default name of the private key file.

“id_rsa.pub” contains the public key.
Press Enter to accept the default prefix.

The response:
```
Enter passphrase (empty for no passphrase): 
```
Press Enter twice to not specify a passphrase.
```
Enter same passphrase again: 
```

Bring the public key to GitHub:

cd ~/.ssh
cat id_rsa.pub

You should see a file beginning with

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOgkh/0a1it6moT/ueEN3c/5CsX6x619icK9wMRpIWNMyZQGaSnU8a1xl6ZWuwIRM1LzjcS/7JeI1sYiW4K0MhYXYtR7b693L8jLppSKL8/+zWQ+kiPDpGLV7hOU1wINrVDsjWvRSIi+ihb7wy6atQRTtAq/0mStc6sBLNYJoVMzaOw+a6cob+IJQ0VNH4wVdImnCQU8T13afFLNwEvXMHdjXYHnmt/V+nQV7omb6oCX/xnqdDiI0LEQLZu+EI5j2ELdeBF9p9Nk4ko0JJknDrR0OCOAoZJVebQDG+ZsHwha9Xw1dDTVJJkmNGvp3pi0BegSPk4sAbZg2zIVHUpqtJ jenkins123@xyz.com

Highlight the private key file and press Command/Control+C to copy the contents to your laptop’s Clipboard.
Switch to GitHub.com.
In the top right corner of any GitHub page, click your profile photo.
On your profile page, click the Repositories tab.
Click the name of your repository.
Click Settings in your repository’s right sidebar.
Click Deploy Keys In the sidebar.
Click Add deploy key.
Paste your public key in and click Submit. ???
Files in the .ssh folder should be 600:
```
chmod 600 ~/.ssh
```
Enable verbose logging (-vT) to see what permissions it complains about:

ssh -vT git@github.com

The response can be:
```
Permission denied (publickey).
```
The desired response is:
Append the public key to the bottom of the authorized_keys file:

cat id_rsa.pub » ~/.ssh/authorized_keys

NOTE: On some systems it’s authorized_keys2.

PROTIP: Use the verbose option on the ssh command to say which key files it’s actually trying.

Set up webhook on GitHub

Sign into the repository. You won’t see the “Settings” tab unless you have permissions.
Click on the Settings tab.
Click on “Webhooks & services” from the left menu.
Click on “Add webhook” on the upper right.

Webhooks allow external services to be notified when certain events happen within your repository. When the specified events happen, we’ll send a POST request to each of the URLs you provide.

https://developer.github.com/webhooks/creating
explains each field:
In Payload URL goes a URL such as “http://localhost:4567/payload” or

http://138.68.1.138:8080/github-webhook/

CAUTION: Use http://
For Content type, “application/json” sends a HTTP POST. Older form data is sent with “application/x-www-form-urlencoded”.
For secret, leave blank ???
Select “Just the push event” for “Which events would you like to trigger this webhook?” or too much will be sent.

The technical name of all events are described at
https://developer.github.com/webhooks/#events
Click Add Webhook.

GitHub sends a Ping
https://developer.github.com/webhooks/#ping-event.

Test if it works

There are several ways to check:
- changes on GitHub.com by those with permissions
- changes pushed to GitHub from a Git client
- changes in a Pull Request accepted.
Do each of the above when the Jenkins server is down, then start it up again to see if queuing worked.

Pull Request
On GitHub, at your repo, fork the repo to your account.
On your local machine, create a folder and clone from your account’s copy of the repo.
Checkout the current commit to a new branch.
Make a change in the README.md file.
Git add and commit.
Git Push it back to GitHub.
Accept the PR. Note the time, and if the Jenkins server is on another time zone (UTC), translate the time on the Jenkins server.
On the Jenkins jobs Dashboard, select the project you updated to see a new job invoked.
See the new job from “anonymous user”?

SSH Agent Forwarding

For those who don’t want to have a private key file, even in encrypted form, stored anywhere away from a trusted machine, there is the “same agent” protocol. An “ssh-agent” program runs in the background and keeps private key loaded into memory. Servers call the ssh-agent as if they were already running on the server. This is like asking a friend to enter their password so that you can use their computer.

The passphrase need to be entered only once, when the ssh-agent is invoked.

SSH agent forwarding can be used to make deploying to a server simple. It allows you to use your local SSH keys instead of leaving keys (without passphrases!) sitting on your server.

If you’ve already set up an SSH key to interact with GitHub, you’re probably familiar with ssh-agent. It’s a program that

At the risk of However, there is different vulnerability: agent hijacking over the wire.

In short, this allows a chain of ssh connections to forward key challenges back to the original agent, obviating the need for passwords or private keys on any intermediate machines.

http://www.unixwiz.net/techtips/ssh-agent-forwarding.html#fwd by Steve Friedl

https://developer.github.com/guides/using-ssh-agent-forwarding/

References

https://help.github.com/articles/about-webhooks/
http://lxanders.github.io/posts/jenkins-with-github-integration/#prepare-github
https://thepracticalsysadmin.com/setting-up-a-github-webhook-in-jenkins/
http://fourkitchens.com/blog/article/trigger-jenkins-builds-pushing-github
http://blog.shippable.com/configure-web-hooks-to-trigger-continuous-integration

TODO: Quote URL which says “Payload sizes need to be monitored because GitHub caps them at 5 MB each.”

The objective of Jenkins2 is to install with a recommended set of plugins (a more “curated” experience than v1) that cover 80% of use cases out of the box.

More about Git & GitHub

This is one of a series on Git and GitHub:

Wilson Mar