One git commit and you’re hooked
Overview
Here is a tutorial on how to make Git on a local machine automatically run a “hook” script in response to git commands.
Dimensions to Hooks
Hooks scripts run on two locations:
- client-side hooks (on your laptop) and
- server-side hooks (on GitHub, GitLab, BitBucket, etc.).
Git and GitHub have been written to look for a hook program before and after each of these commands:
- git clone
- git checkout
- git commit
- git rebase
- git patch (applypatch-msg)
- git push
- git merge
- git am
- git gc –auto
Just so we can use nerdy language, hooks before each event begin with “pre-“ and
hooks after each even begin with “post-“. This illustration (by Sarah Goff-Dupont of Atlassian) identifies common uses across four situations:
Ensure adherance to standards
PROTIP: Some organizations impose a set of hook files on repositories in order to impose some standards, such as:
-
Reject passwords found in files. Doing it on the client is better because once some text in a repository, it is necessary to git rebase, which is rather messy.
-
Reject very short commit messages, such as 10 characters or less.
- Move media files (.mp4, .mp3, .jpg, .png) from repositories that are only “supposed” to contain text to a repository hold media files that humans can’t read. The job of git-lfs (Large File System) is to move and replace binary files with a (texual) link to binary repositories.
- etc.
Hooks On the client
The trouble with client hooks is that it’s rather intrusive to install them on each developer’s laptop.
The free Community Edition (CE) of IntelliJ IDEA from Jetbrains provides check-boxes to activate built-in Git hook functionality:
PROTIP: We don’t see how to get to Version Control in IntelliJ on videos because it’s often accessed via a hotkey - Apple command 9 on macs and Alt+9 on Windows.
See this video about VCS improvements in IntelliJ 2016.3.
Let’s use a sample project that already has .idea folder containing files that define configurations for IntelliJ.
IntelliJ has a project wizard to start from scratch on various languages.
Default Git Hook files
PROTIP: Every Git repository is created with a hooks folder containing sample hook files that are named so they don’t execute.
-
Navigate into any Git repository you want to automate.
cd .git/hooks
Git and GitHub looks into this specific folder name for scripts to run when specific events occur.
PROTIP: Git and GitHub have been written such that it recognizes specific file names for each internal event.
PROTIP: Each Git repository is created with a set of sample automation files in the folder. But the file names end with “.sample” so that they won’t run.
Listed alphabetically:
applypatch-msg.sample commit-msg.sample post-update.sample pre-applypatch.sample pre-commit.sample pre-push.sample pre-rebase.sample pre-receive.sample prepare-commit-msg.sample update.sample
There are others that Git automatically runs.
pre-checkout pre-applypatch post-applypatch post-merge post-receive pre-auto-gc post-rewrite runs after a commit is modified by a git commit --amend or git rebase.
-
PROTIP: Click each link above for the lastest version online at:
https://github.com/git/git/tree/master/templates
CAUTION: Nobody uses the default files as they are by just removing the “.sample” part of the file name because they probably don’t do what you want.
-
Delete the sample files in your repo because it’s best to get the latest version from GitHub.
No file extension
Due to their Linux origins, Git and GitHub were written to recognize command files that have no file extension, since in Linux the type of command is defined in the first line within each file (with a “SheBang magic number”) rather than in the file name’s extension as in Windows.
Multiple languages
Git will run any script that can run on the command line (and properly installed):
- Bash shell is the default
- JavaScript (mocha)
- Ruby
- Python
- Groovy
- PowerShell (see below)
All commands in a particular hook file must be of the same language.
Bash hook script
During installation of git-lfs (GitHub’s Large File System), commands like these are placed in hook files:
#!/bin/sh command -v git-lfs >/dev/null 2>&1 || { echo >&2 "\nThis repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting .git/hooks/post-commit.\n"; exit 2; } git lfs post-commit "$@"
The above commands first checks if git-lfs is installed, then performs the git lfs command, with “$@” forwarding the attributes passed into the hook file.
The hook files used by git-lfs are:
- pre-push
- post-checkout
- post-commit
- post-merge
Python hook script
NOTE: The minimal Python hook script (commit-msg):
#!/usr/bin/env python # post-checkout hook import argparse def parse_args(): pass def main(args=None): pass if __name__ == "__main__": args = parse_args() main(args)
TODO: Print out values of arguments supplied when calling the hook.
PowerShell script
NOTE: To have Git run a PowerShell script, a shell script invokes an additional PowerShell (.ps1) file added in the hooks folder:
#!/bin/sh # pre-commit (no file extension) c:/Windows/System32/WindowsPowerShell/v1.0/powershell.exe \ -ExecutionPolicy RemoteSigned -Command .\.git\hooks\pre-commit.ps1
Demo simple hooks
I have prepared a repository to demo use of local Git hooks to run.
The repo contains a “hooks” folder containing script files.
Files in the folder are copied into the .git/hooks
folder where Git looks.
-
In an internet browser, go to this repository (which we will later call the “upstream”):
-
Sign in using your account. The rest of this document assumes “hotwilson” is your account name. So replace “hotwilson” with your own account whereever you see it.
-
Fork the repo under your own account.
- Open a Terminal window.
-
Make and cd into a folder where you will the repository will be downloaded.
Again, in this example, you would change “hotwilson” to your own account name.
cd ~ && mkdir gits && cd gits && mkdir hotwilson && cd hotwilson
-
Confirm you are indeed where you want to create a new repository:
pwd
-
Clone to download:
git clone https://github.com/hotwilson/git-utilities.git cd git-utilities
-
Optionally, to enable updates from the original repo:
git remote add upstream https://github.com/wilsonmar/git-utilities.git git remote -v
Global templates
NOTE Git looks for hooks in a global template folder applicable to all repositories when either a GIT_TEMPLATE_DIR variable is set in .bash_profile or .gitconfig contains a setting defined by command:
git config --global init.templatedir /path/to/your/templates/
Alternatively, symbolic links or symlinks can be used to link custom hooks to the ones in the .git/hooks folder.
-
Copy these lines and paste in Terminal (at the root folder) to copy demo hook files from the hooks folder into the .gits/hooks folder:
cp hooks/* .git/hooks chmod +x .git/hooks/*
PROTIP: In Mac and Linux machines, scripts must be authorized to be run using the chmod command.
PROTIP: It is too difficult and dangerous to edit and add/commit files inside the .git folder, which Git itself uses to track changes. So copying files in is the better approach than error messages such as:
fatal: This operation must be run in a work tree
-
Copy these lines and paste in the Terminal to make a change, then add and commit it for the Git Hook to take action:
echo "random text" >>README.md git add . git commit -m"try git commit hooks"
Prettier node module code
To ensure consistency in formating within code files (without spending time caholing people), automatically format the code using prettier.io (github.com/prettier/prettier).
Here’s why automatica reformatting by Prettier is awesome:
- No effort spent fixing formatting
-
Code copied in (from another project or from StackOverflow) automatically adjusts
- No looking up rules in a style guide
-
When the style guide changes (or when it is first introduced) Prettier can automatically apply it across the whole code base
- No time wasted (ill will created) discussing style in pull requests
- Differences in coding style by different people won’t appear as different
- White space can be removed automatically to safe space, and re-formatted for human reading
Below are steps in the shell script that installs prettier for use by Git.
-
Create or navigate to a folder containing JavaScript NodeJs code (file index.js). That folder should have been initialized as a Git repository (has a .git folder at root). In my example, its:
~/gits/wilsonmar/node-sample1
-
Install yarn or npm. Upgrade if already installed:
brew upgrade yarn
-
Install using yarn/npm,
brew upgrade yarn yarn add --dev prettier pretty-quick
npm install prettier --global
The above adds file yarn.lock and package.json, which contains a list of dependencies expanded into folder node_modules.
https://npm.im/pretty-quick (https://github.com/azz/pretty-quick)
-
Edit file _____ to edit setting to control specific file formats by adding the format between square brackets: javascript, TypeScript, HTML, CSS, JSON, etc. Specify “true” or false for automatic “editor.formatOnSave”:
{ "folders": [], "settings": {}, "[javascript]": { "editor.formatOnSave": true } }
-
Invoke from command line :
yarn pretty-quick --global
Alternately, update your current code:
prettier "*/.ts" --write
Example response (with either “Everything is awesome” or changefix message):
yarn run v1.21.1 $ /Users/wilson_mar/gits/wilsonmar/node-sample1/node_modules/.bin/pretty-quick 🔍 Finding changed files since git revision null. 🎯 Found 3 changed files. ✍️ Fixing up README.md. ✍️ Fixing up index.js. ✅ Everything is awesome! ✨ Done in 0.31s.
https://www.npmjs.com/package/pretty-quick
-
Install the Prettier in your code editor. In VS Code, open the Command Palette (under the View submenu, or using Cmd+Shift+P on Mac and Ctrl+Shift+P on Windows) to type “Extensions: Install Extensions”. In place of “Search Extensions in Markeplace”, type “Prettier” (by Esben Petersen), then click “Install”. Click “Install” in the detail screen.
Search for “Prettier”, click “Install”, and then “Reload” once the installation is complete.
Alternately, via the command line:
code --list-extensions code --install-extension esbenp.prettier-vscode
To upgrade, install with –force option.
This is like SublimeLinter
-
Invoke pretty-quick manually in your code editor opened to a file. In VSCode, Command Palette, type “Format Document” and select it so Prettier will tidy up your code.
-
For automatic invocation on commit via Git hooks, add a dependency to husky:
yarn add --dev husky
-
Edit package.json to add the hook to husky:
{ "devDependencies": { "husky": "^3.1.0", "prettier": "^1.19.1", "pretty-quick": "^2.0.1" } "husky": { "hooks": { "pre-commit": "pretty-quick --staged" } } }
The utility automatically adds 21 hook files (with no file extension) to .sample files within folder .git/hooks:
- applypatch-msg for applypatch-msg.sample
- commit-msg for commit-msg.sample
- fsmonitor-watchman.sample
- post-applypatch
- post-checkout
- post-commit
- post-merge
- post-receive
- post-rewrite
- post-update for post-update.sample
- pre-applypatch for pre-applypatch.sample
- pre-auto-gc
- pre-commit for pre-commit.sample
- pre-merge-commit
- pre-push for pre-push.sample
- pre-rebase for pre-rebase.sample
- pre-receive for pre-receive.sample
- prepare-commit-msg for prepare-commit-msg.sample
- push-to-checkout
- sendemail-validate
- update for update.sample
These are described at https://github.com/typicode/husky#readme
Variables in hooks
Each of the demo hook files can contain variables. For example:
#!/bin/sh COMMIT_MSG_FILE=$1 # ".git/COMMIT_EDITMSG" echo "prepare-commit-msg: $(cat $COMMIT_MSG_FILE)" exit 0
CheatSheet Matrix
Click to expand this cheatsheet (from Daniel Convissor) to see the 15 hooks:
The table lists for each hook what command triggers it, when it runs, what parameters are passed into it, and what happens when the hook exits abnormally (with a 1).
-
You may delete “.sample” files there.
-
Make a git commit:
git commit -m"test commit with husky prettier"
Skip hook invocation
One can suppress hooks from firing with the
--no-verify
command attribute:git commit --no-verify
-
Create aliases to make git commands with flags defined in
-
Use git diff utility
Notice the changes Prettier made to JavaScript coding:
- Use “ (double quotes) instead of ‘ (single quotes)
- Add an extra space before the function body (after the parameter list)
- Add a semicolon at the end of the return statement
JavaScript Style Guides
As a starting point for your own organization:
More about Prettier:
- https://www.codereadability.com/automated-code-formatting-with-prettier/
Flake8 Linter for Python
flake8 is a python wrapper that glues together plugins to check the style and quality of Python code. It runs in GitLab.
-
To Install, run (within virtualenv):
pip install flake8 pip install flake-bugbear # to add additional rules. flake8 --version pip install pre-commit # to $PROJ/.git/hooks/pre-commit pre-commit install pip install black # to reformat code # bandit # to check for python code vulnerabilities # sign-commit # to add signature verification
This is explained by Kenneth H. East.
-
Add a .flake8 configuration file at the root of the project repo.
-
Variables are used to select and ignore specific warnings or errors:
flake8 --select W292,E712 --ignore E501,F401,F841,W293,E303,W291,E261,E262,E265,E231,E251,E302 \ path/to/code/
Ones that should be fixed:
- W292 no newline at end of file - this can cause the line to not be read.
- E712 comparison to False should be ‘if cond is False:’ or ‘if not cond:’
The whole team should decide whether to enforce:
- W191 indentation contains tabs
- E305 expected 2 blank lines after class or function definition, found 1
- E302 expected 2 blank lines, found 1
Stylistic choices than can be ignored, but maybe not:
- E501 line too long
- F401 module imported but unused
- F841 local variable ___ is assigned to but never used
- W293 blank line contains whitespace
- E303 too many blank lines
- W291 trailing whitespace
- E261 at least two spaces before inline comment
- E262 inline comment should start with ‘# ‘
- E265 block comment should start with ‘# ‘
- E231 missing whitespace after ‘,’
- E251 unexpected spaces around keyword / parameter equals
Flake8 calls pep8, PyFlakes, pycodestyle, Ned Batchelder’s McCabe, and third-party plugins.
PROTIP: Flake8 does not issue warnings on lines that contain a # noqa in-line comment at the end.
git commit hooks
First of all, there is already some editing to git commit. Git does not allow empty commit messages. If a commit message does not include a message, your favorite editor is opened for you to enter one. If you still haven’t typed anything in your editor, Git aborts the commit.
The flow of processing on the client is illustrated by this (from Johan Abildskov & Jan Krag of Praqma)
(click for full screen image)
Upon git commit
, client hooks are executed in this order:
-
The pre-commit hook runs after a git commit is executed, but before the commit message editor is displayed. Since an exit from this with anything other than zero aborts the commit, it is the place to check assets involved in the commit itself (rather than the commit message), to run linters and unit tests on the local laptop. This program has access to the commit date/time, author name and email.
NOTE: pre-commit hooks do not support hooks with side effects (such as modifying files and adding them to the index with git add).
A non-zero exit aborts the commit, which can happen if a file being committed contains “#DONTPUSH” (or other keyword recognized by the hook program) developers put in their code as a reminder to hold off pushing to the team repository.
The following pre-commit code keeps debugging code (such as “console.log” debugging statements) from reaching the shared code base:
FILES_PATTERN='\.(js|coffee)(\..+)?$' FORBIDDEN='console.log' git diff --cached --name-only | \ grep -E $FILES_PATTERN | \ GREP_COLOR='4;5;37;41' xargs grep --color --with-filename -n $FORBIDDEN && echo 'COMMIT REJECTED Found "$FORBIDDEN" references. Please remove them before commiting' && exit 1
FILES_PATTERN above contains a Regular Expression.
-
The prepare-commit-msg hook is invoked after receiving a git commit, just prior to firing up the commit message editor. This hook can edit the commit message in a way that cannot be suppressed. For example, ensuring a capital letter.
QUESTION: This has access to the commit SHA-1 (when operating on an existing commit)?
-
The commit-msg hook adjusts the commit message after it has been edited in order to ensure conformity to a standard or to reject based on any criteria. It can abort the commit if it exits with a non-zero value.
-
The post-commit hook is called after the actual commit is made so that it cannot disrupt the commit. It is mainly used for sending notifications (emails, SMS, Slack, Twitter, etc.). This NOTE describes deploying to a Local Web Server with a Post-Commit Hook.
git push hooks
The order of execution is:
-
pre-receive run just before pushed files are updated. So it can abort the receive process by exiting with a non-zero status. Thus, it is used to enforce commit policies and reject the entire commit if it is deemed unsatisfactory. (code coverage)
-
update filters each commit ref made to the remote repository independently. It can be used to reject or accept each ref being pushed.
-
post-receive is triggered after an update has been done on the remote repository. So it cannot abort the update process. But it can trigger notifications on a successful remote repository update. (send email, run load tests, so that a log of notifications is stored on a remote server.
Sample JavaScript Mocha linter
From https://scotch.io/tutorials/using-git-hooks-in-your-development-workflow#client-side-hooks
#!/bin/bash # Exits with non zero status if tests fail or linting errors exist num_of_failures=`mocha -R json | grep failures -m 1 | awk '{print $2}' | sed 's/[,]/''/'` errors=`jscs -r inline ./test/test.js` num_of_linting_errors=`jscs -r junit ./test/test.js | grep failures -m 1 | awk '{print $4}' | sed 's/failures=/''/' | sed s/">"/''/ | sed s/\"/''/ | sed s/\"/''/` if [ $num_of_failures != '0' ]; then echo "$num_of_failures tests have failed. You cannot commit until all tests pass. Commit exiting with a non-zero status." exit 1 fi if [ $num_of_linting_errors != '0' ]; then echo "Linting errors present. $errors" exit 1 fi
Useful Hook examples
-
A list of websites offering Git hooks is at Matthew Hudson’s githooks.com, along with projects others have done with hooks. The canonical documentation is at https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks
-
https://longair.net/blog/2011/04/09/missing-git-hooks-documentation/
-
https://github.com/brigade/overcommit#built-in-hooks
-
https://git-scm.com/book/en/v2/Customizing-Git-An-Example-Git-Enforced-Policy
-
get push heroku master
-
Setting up the Heroku git-push workflow on your Kubernetes cluster in 60 seconds with Gitkube Jun 26, 2018 [50:46] by Tanmai Gopal shows how you simply git push to update into a Kubernetes cluster. See https://github.com/hasura/gitkube described by https://gitkube.sh/ (@gitkube)
pre-rebase hooks
-
Use a text editor to create in the hooks folder a “pre-rebase” containing this text described in YouTube video Life’s better with Git hooks - Grumpy Gits [17:58] Nov 16, 2015 by Deepank Vora.
#!/bin/sh echo "Custom rebasing message." exit 1
(Unlike Windows, Linux defines file handling in the first line inside the file).
PROTIP: Non-zero finish aborts the commit.
-
Enable the new script:
chmod +x pre-rebase
This would avoid messages such as:
-bash: ./pre-rebase: No such file or directory
-
Run the new script to make sure it works:
./pre-rebase
-
Open another Terminal shell window to the working directory of the repo to run the
git rebase master
command.You should see the custom message.
QUESTION: If instead you see:
Current branch master is up to date.
View pre-commit.sample
-
Use a text editor Set to always open .sample files with a text editor.
#!/bin/sh
at the top of the file means that it’s a Bash shell script.(Author Junio C Hamano is the primary developer of Git, now works at Google.)
applypatch-msg
GitLab
You can connect to the GitLab API using your API key to perform actions inside GitLab when performing, for example, a commit. This you can do from the client side, but of course, you can also use the server-side hooks, such as post-receive to run programs and scripts after they have reached the remote.
GitLab has a GraphQL API (in alpha as of this writing) among its many APIs
closes or fixes
-
The following script is saved as
.git/hooks/commit-msg
(with no file extension):#!/bin/sh PRIVATE_TOKEN="API_KEY" GITLAB_URL="https://your_gitlab_domain.com" URL=`git config --get remote.origin.url` PROJECT=`basename ${URL} .git | cut -d':' -f2` for issue_id in `grep -o -e "\(closes\|fixes\) #[0-9]\+" $1 | cut -d'#' -f2`; do curl -X PUT -d "closed=1" \ ${GITLAB_URL}api/v3/projects/${PROJECT}/issues/${issue_ id}/?private_token=${PRIVATE_TOKEN} done
-
Edit the program with your gitlab URL.
If this script sees a commit message text containing “closes #” or “fixes #”, the hook closes the issue identified by the number after the # symbol.
-
Make the file executable to enable this functionality.
chmod +x *
Resources
The cananoical resources on Git Hooks:
More Resources
Atlassian’s tutorial on Git Hooks - Local hooks
023 Introduction to Git Hooks by Dan Gitschooldude
gated-commit with git pre-commit [12:41] by fullstack
Andrew Burgess for Tuts [7:28] June 28, 2012 describes run of mocha test processed by a Bash script.
contains contributor code in the git team’s repo
Git Hooks Practical uses on Windows
Git - How to validate commit messages? 27 May 2019 by Marcell Lipp
https://confluence.atlassian.com/bitbucketserverkb/how-to-scan-for-and-remove-passwords-or-secrets-in-bitbucket-server-repositories-973473524.html
More on DevOps
This is one of a series on DevOps:
- DevOps_2.0
- ci-cd (Continuous Integration and Continuous Delivery)
- User Stories for DevOps
- Git and GitHub vs File Archival
- Git Commands and Statuses
- Git Commit, Tag, Push
- Git Utilities
- Data Security GitHub
- GitHub API
- Choices for DevOps Technologies
- Pulumi Infrastructure as Code (IaC)
- Java DevOps Workflow
- AWS DevOps (CodeCommit, CodePipeline, CodeDeploy)
- AWS server deployment options
- Cloud services comparisons (across vendors)
- Cloud regions (across vendors)
- Azure Cloud Onramp (Subscriptions, Portal GUI, CLI)
- Azure Certifications
- Azure Cloud Powershell
- Bash Windows using Microsoft’s WSL (Windows Subsystem for Linux)
- Azure Networking
- Azure Storage
- Azure Compute
- Digital Ocean
- Packer automation to build Vagrant images
- Terraform multi-cloud provisioning automation
-
Hashicorp Vault and Consul to generate and hold secrets
- Powershell Ecosystem
- Powershell on MacOS
- Jenkins Server Setup
- Jenkins Plug-ins
- Jenkins Freestyle jobs
- Docker (Glossary, Ecosystem, Certification)
- Make Makefile for Docker
- Docker Setup and run Bash shell script
- Bash coding
- Docker Setup
- Dockerize apps
- Ansible
- Kubernetes Operators
- Threat Modeling
- API Management Microsoft
- Scenarios for load
- Chaos Engineering