Wilson Mar bio photo

Wilson Mar

Hello. Hire me!

Email me Calendar Skype call 310 320-7878

LinkedIn Twitter Gitter Google+ Instagram Youtube

Github Stackoverflow Pinterest

This is the way to merge. It needs to be setup, but you’ll love it.


Overview

This article presents a step-by-step tutorial so you can confidently use an alternative to the standard git merge and git rebase that comes with Git.

The annoyance

Although superior to others, is rather “cumbersome and scary”.

  • Merging is all-or-nothing process.

  • You can’t save your progress.

  • You can’t switch to another branch temporarily.

  • There is no way to test a partly-done merge.

  • There is no way to save a partly-done merge.

  • If you make a mistake, you can’t go back.

  • If you cannot resolve the whole conflict, there is nothing to do but to abort and start over.

Alternatives

PROTIP: If you would like to compare differences in files manually, use a comparison utility. A free one for Windows is:

http://meldmerge.org

Interactive merge

The interactive approach to merging two branches together (safely) is incrementally in steps that allows for manual fixing. Some call this “rebase with history” because the technique creates new commits based on previous commits like rebase, but retains the previous commit history (which rebase currently does not do).

The helper module that does this runs using Python (either version 2 or 3).

It was mentioned by GitHub Data Scientist Patrick McKenna in a YouTube video “Greatest Hits of the Git Maintainers Room - Git Merge 2017” at his talk during the GitMerge May 2017 conference.

It was actually created in May 2013 by Michael Haggerty (mhagger@alum.mit.edu), a GitHub Core committer and “theoretical physicist turned software developer”.

Imerge was described in this video from the GitMerge 2013 conference which discusses the approach in a May 2013 blog post.

This article combines all the above into a step-by-step tutorial so “newbies” can easily benefit from this game-changing technology.

Installation

  1. Install Python.
  2. Install Ruby (for its Make utility).
  3. Install Git.

    On Windows, install Chocolatey.org and in a Run command window, choco install msysgit.

  4. If you are running Windows, be in Git Bash to run.

    Install test modules

  5. Create and cd to a folder where you will clone a repository containing scripts:

    cd ~
    mkdir git-imerge-test
    cd git-imerge-test
    
  6. Clone a repo containing scripts that creates test repos containing sample conflicts:

    git clone 
    https://github.com/wilsonmar/git-utilities
    cd git-utilities
    
  7. Create a folder where you want the test repo created.
  8. Copy the git-imerge script to that new folder.

    • git-imerge-test-create.sh

  9. Give run permissions to the scripts:

    On Mac and Linux:

    chmod 555 git-imerge-test-create.sh
    

    The above only needs to be done once.

    View git-imerge

  10. Clone the git-imerge module onto your computer:

    git clone https://github.com/mhagger/git-imerge
    cd git-imerge
    

    The “active ingredient” is the git-imerge file. The file has no file extension because it’s a Git custom command.

  11. View the git-imerge file

    cat git-imerge | more
    

    Notice from the top line that it’s run by a Python interpreter.

    The part that starts with r""" is what is displayed by the git imerge -h command.

  12. Press q to escape the display or press Spacebar key for next page.

    Mac Homebrew install

  13. If you have a Mac, view git-imerge specifications for Homebrew:

    https://github.com/Homebrew/homebrew-core/blob/master/Formula/git-imerge.rb

    PROTIP: The advantage of using Homebrew instead of manually installing is that it handles upgrades automatically. So before you upgrade MacOS again, check to see if there is an entry for that new version, then do a brew upgrade. The other advantage is that executing brew uninstall git-imerge is easier than hunting down files to delete.

    The Ruby-language code has a work-around for the make installation path.

    It installs to the /bin folder.

    If you are OK with the above, install git-imerge:

    brew install git-imerge
    

    The response I got:

    ==> Downloading https://homebrew.bintray.com/bottles/git-imerge-1.0.0.sierra.bot
    ######################################################################## 100.0%
    ==> Pouring git-imerge-1.0.0.sierra.bottle.tar.gz
    ==> Using the sandbox
    ==> Caveats
    Bash completion has been installed to:
      /usr/local/etc/bash_completion.d
    ==> Summary
    🍺  /usr/local/Cellar/git-imerge/1.0.0: 6 files, 168.1KB
    

    Notice the brew formula takes care of installing the bash_completion where other daemons are also installed (/usr/local/etc/) and automatically invoked without you needing to manually add it in your bash_profile.

  14. Skip to verify.

    Option B: Windows and Linux install

    Sorry, instructions for Mac and Linux will be coming.

    Manual install on Mac

  15. Use Make to run Makefile to install Bash completions:

    On a Mac:

    make
    

    This copies file <strong>git-imerge.bashcomplete</strong> to $(DESTDIR)/etc/bash_completion.d/git-imerge

    PRPTIP: In Make files, tabs should be used instead of spaces to indent items.

    See: https://github.com/bobthecow/git-flow-completion/wiki/Install-Bash-git-completion

  16. Make Bash completion occur at Terminal start-up:

    On a Mac:

    vim ~/.bash_profile
    

    Add near other Python settings:

    if [ -f $(brew --prefix)/etc/bash_completion ]; then
    . $(brew --prefix)/etc/bash_completion
    fi
    

    CAUTION: The above needs to be confirmed.

  17. Re-start the Terminal to take the changes:

    source ~/.bash_profile
    

    Get menu to verify

    Regardless of how it was installed:

  18. Verify by obtaining the list of commands:

    If you want just a reminder of the keywords:

    git imerge
    

    The response should be this (error message):

    usage: git-imerge [-h]
                   {start,merge,rebase,drop,revert,continue,finish,diagram,list,init,record,autofill,simplify,remove,reparent}
                   ...
    git-imerge: error: too few arguments
    

    (ignore the “too few arguments”)

    Alternately, for a whole “man page” as well:

    git imerge -h
    

    The response adds to the above:

    Git incremental merge
     
    Perform the merge between two branches incrementally.  If conflicts
    are encountered, figure out exactly which pairs of commits conflict,
    and present the user with one pairwise conflict at a time for
    resolution.
     
    Multiple incremental merges can be in progress at the same time.  Each
    incremental merge has a name, and its progress is recorded in the Git
    repository as references under 'refs/imerge/NAME'.
     
    An incremental merge can be interrupted and resumed arbitrarily, or
    even pushed to a server to allow somebody else to work on it.
     
    Instructions:
     
    To start an incremental merge or rebase, use one of the following
    commands:
     
     git-imerge merge BRANCH
         Analogous to "git merge BRANCH"
     
     git-imerge rebase BRANCH
         Analogous to "git rebase BRANCH"
     
     git-imerge drop [commit | commit1..commit2]
         Drop the specified commit(s) from the current branch
     
     git-imerge revert [commit | commit1..commit2]
         Revert the specified commits by adding new commits that
         reverse their effects
     
     git-imerge start --name=NAME --goal=GOAL BRANCH
         Start a general imerge
     
    Then the tool will present conflicts to you one at a time, similar to
    "git rebase --incremental".  Resolve each conflict, and then
     
     git add FILE...
     git-imerge continue
     
    You can view your progress at any time with
     
     git-imerge diagram
     
    When you have resolved all of the conflicts, simplify and record the
    result by typing
     
     git-imerge finish
     
    To get more help about any git-imerge subcommand, type
     
     git-imerge SUBCOMMAND --help
     
    positional arguments:
      {start,merge,rebase,drop,revert,continue,finish,diagram,list,init,record,autofill,simplify,remove,reparent}
                         sub-command
     start               start a new incremental merge (equivalent to "init"
                         followed by "continue")
     merge               start a simple merge via incremental merge
     rebase              start a simple rebase via incremental merge
     drop                drop one or more commits via incremental merge
     revert              revert one or more commits via incremental merge
     continue            record the merge at branch imerge/NAME and start the
                         next step of the merge (equivalent to "record"
                         followed by "autofill" and then sets up the working
                         copy with the next conflict that has to be resolved
                         manually)
     finish              simplify then remove a completed incremental merge
                         (equivalent to "simplify" followed by "remove")
     diagram             display a diagram of the current state of a merge
     list                list the names of incremental merges that are
                         currently in progress. The active merge is shown with
                         an asterisk next to it.
     init                initialize a new incremental merge
     record              record the merge at branch imerge/NAME
     autofill            autofill non-conflicting merges
     simplify            simplify a completed incremental merge by discarding
                         unneeded intermediate merges and cleaning up the
                         ancestry of the commits that are retained
     remove              irrevocably remove an incremental merge
     reparent            change the parents of the HEAD commit
     
    optional arguments:
      -h, --help            show this help message and exit
    

Run scripts

Create test repo

  1. Run the script which sets up data containing conflicts and perform a merge using git-imerge:

    ./git-imerge-test-create.sh
    

    (You may want to later adapt this to create your own data and commands.)

    How the test repo is created

    To avoid problems, the script aims to be “idempotent” in that each time it’s run, the same result is produced. To achieve this, the script creates a repo. In subsequent runs the repo (.git folder) is deleted before starting over.

    Each branch contains a single file named somefile.md.

    The script makes a first commit with a blank file so branches can be created.

    In the repo, a branch named feature1 is created so that commits can be added to it in parallel with master.

    A “for” loop in the script alternates between the two branches to add a line at the bottom of the file, then makes another commit. Here are the message text of commits:

    To make conflicting lines, each line that will conflict contains its own branch name (master or feature1). A cat of the file within the master branch contains:

    A1
    B2 master
    C3
    D4
    E5
    F6 
    G7 master
    H8
    I9 master
    10
    11
    

    A cat of the file within the feature1 branch contains:

    A1
    B2 feature1
    C3
    D4
    E5
    F6 
    G7 feature1
    H8
    I9 feature1
    10
    11
    

    After construction, the branch list shows the last commits for each branch. For example:

      feature1 55d4211 I
           * master   80fa56b 9
    

    The asterisk (*) indicates that the currently checked-out branch is “master”.

    After printing out the above, the script pauses with this message:

    Press enter to continue

    Analyze native merge

    Let’s pause here to review the repo created.

  2. Do a diff to see:

    git diff master feature1
    

    The default display has a column in front of each line of content (with commit ids that will be different):

    diff --git a/somefile.md b/somefile.md
    index 8cb48cc..db324d6 100644
    --- a/somefile.md
    +++ b/somefile.md
    @@ -1,9 +1,9 @@
     A1
    -B2 master
    +B2 feature1
    

    The “—” at the top indicates where a - (minus sign) marks lines from file “a”.

    The “+++” at the top indicates where a + (plus sign) marks lines from file “b”.

    Different colors may also appear depending on your setup.

    The difference between an imerge versus a native git merge is that imerge presents only a single instance of such markers, whereas a native git merge put such markers in several places.

  3. As an aside, let’s do a native git merge as the basis for comparison.

    On a Mac, press control+C to exit the running script.

    Be at the master source branch and merge in the “up-start” branch into it:

    git checkout master
    git merge feature1
    

    The response we expect is:

    CONFLICT (content): Merge conflict in somefile.md
    Automatic merge failed; fix conflicts and then commit the result.
    
  4. View modifications about conflicts Git made to the file:

    cat somefile.md
    

    You should now see:

    A1
    <<<<<<< HEAD
    B2 feature1
    =======
    B2 master
    >>>>>>> master
    C3
    D4
    E5
    F6
    <<<<<<< HEAD
    G7 feature1
    H8
    I9 feature1
    =======
    G7 master
    H8
    I9 master
    >>>>>>> master
    

    The format of above markers for a two-way merge is:

    git config --global merge.conflictstyle merge
    
  5. Try the three-way merge formatting by typing this:

    git config --global merge.conflictstyle diff3
    

    The first line below the <<<<<< HEAD is the master (HEAD) branch.

    The bottom marker is the branch containing conflicting text right above it.

    The line above the “=======” is the original branch before change.

    Hello, master change.”, and the “b1” branch has “Hello, branch b1 change.”. This three-way diff can be very helpful in determining what really changed.

  6. Re-run the script above.

  7. Edit and save the file.

    However, if you want to be adament and overwrite what’s in master with the entirety of them up-start’s file from the feature1 branch:

    git checkout --theirs somefile.md
    

    Alternately, if you want to just keep whatever was in the original source (master) branch:

    git checkout --ours somefile.md
    
  8. Add and commit the change:

    git add somefile.md && git commit -m"resolved"
    

    NOTE: All files need to be added again, not just the ones in conflict.

    Resources

    The steps described above combines the best of advice from others about traditional git merge:

    • http://genomewiki.ucsc.edu/index.php/Resolving_merge_conflicts_in_Git

    Interactive merge

    Anyway, back to interactive merge:

    Essentially, we want to end up with this in a Git Network Diagram: *

    o - 0 - 1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 - 10 - 11 - I11'  ← master branch
      \                                               /
       A -- B -- C --- D --- E --- F --- G --- H --- I       ← branch feature1
    

    However, instead of merging commit I with 11, we want to do an interactive merge by merging incrementally as illustrated by this diagram:

    o - 0 - 1  - 2  - 3  - 4  - 5  - 6  - 7  - 8  - 9  - 10  - 11    ← master
     |   |    |    |    |    |    |    |    |    |     |     |
     A -   --   --   --   --   -- A6 -   -- A8 - A9 - A10 - A11
     |   |    |    |    |    |    |    |    |
     B -   --   --   --   --   -- B6 - B7 - B8   X
     |   |    |    |    |    |    |
     C -   --   --   --   --   -- C6   X
     |   |    |    |    |    |    |
     D -   --   --   --   --   -- D6
     |   |    |    |    |    |    |
     E - E1 - E2 - E3 - E4 - E5 - E6
     |   |
     F - F1   X
     |   |
     G - G1
     |   |
     H - H1
     |   |
     I - I1
     
     ↑
      branch feature1
    

    Commit A is merged with 1, etc.

    “X” in the diagram below marks where is conflict is designed to occur.

    To see this in action:

  9. Run the script again:

    ./git-imerge-test-create.sh
    
  10. But this time, press Enter for the script to begin merge much like with standard git merge by checking out the destination branch:

    git checkout master
    
  11. Tell git imerge what branch you want to merge into:

    git imerge start --name=NAME --goal=full feature1
    

    NAME stands for the commit message of your final commit when merging is finished.

    QUESTION: How is this different than --first-parent feature1? When do we use that?

    Intermediate state handling

    Internally, the tool uses git bisect to find pairwise merges that conflict.

    When it hits a conflict, it asks for help.

  12. When imerge processing stops due to a conflict, notice you are at branch “imerge/NAME” automatically created to hold results:

    git branch -avv
    

    The response (where each run will have different commit IDs):

      feature1    2fe920d I
           * imerge/NAME b9a54e5 imerge 'NAME': automatic merge 2-1
      master      7b85c5f 9
    

    Below are internals information you may not care about: During an incremental merge, intermediate results are stored directly in your repository as special references:

    refs/imerge/NAME/state - A blob containing a little bit of metadata.

    refs/imerge/NAME/manual/M-N - Manual merge including all of the changes through commits M on master and N on branch.

    refs/imerge/NAME/auto/M-N - Automatic merge including all of the changes through commits M on master and N on branch.

    refs/heads/imerge/NAME - Temporary branch used when resolving merge conflicts.

    refs/heads/NAME - Default reference name for storing final results.

    I mention all this because this error occurs if you try to checkout a different branch after

    somefile.md: needs merge
    error: you need to resolve your current index first
    

    Cycle of fixes

    Resolve conflicts in the sample the usual way:

  13. In larger files in real life, you may need to use a diff utility to identify differences. I will be move material from my class here.

  14. Resolve deliberate conflicts in the example by using a text editor on somefile.md :

    A1
    <<<<<<< HEAD
    B2 feature1
    =======
    B2 master
    >>>>>>> 1905858068225d58f2f36fdce243bc2a663ced36
    

    The ======= separates content were inserted by Git to divide what is conflicting between the two branches.

    That and the beginning and ending markers also inserted by Git need to be removed before saving the file.

    At the bottom is the SHA1 commit ID.

  15. Remove lines added to end up with:

    A1
    B2 feature1
    ...
    

    NOTE: We keep the “feature1” version because that’s the change we typically want to make.

  16. Save the file.

  17. Add and commit the change:

    git add somefile.md && git commit -m"Fix B2"
    
  18. Resume:

    git imerge continue
    

    If you forgot to do a commit, you’ll see this message:

    [file]: needs merge
    
  19. Repeat the cycle of fixes above until you see:

    Merge is complete!
    
    
    BLAH QUESTION for Michael: Why does "B2 master" appear again?
    And seeminly out of order?
    
    
    A1
    <<<<<<< HEAD
    B2 master
    C3
    =======
    B2 feature1
    >>>>>>> 94e7ec48c4cd902c0512ae69ae44d8d9ffa057a2
    
    And so on:
    A1
    B2 feature1
    C3
    D4
    E5
    F6
    <<<<<<< HEAD
    G7 master
    =======
    G7 feature1
    >>>>>>> c4d21840f2bcd6cd51060b101dac9289b782fbde
    
    QUESTION: Why again?
    A1
    B2 feature1
    C3
    D4
    E5
    F6
    <<<<<<< HEAD
    G7 master
    H8
    =======
    G7 feature1
    >>>>>>> 303e8288568ecc67b7e8bc248359c32122a46808
    

The last one:

A1
B2 feature1
C3
D4
E5
F6
G7 feature1
H8
<<<<<<< HEAD
I9 master
=======
I9 feature1
>>>>>>> 195454a77bb6805954f70de5bb76879fe026210b
  

### Diagram

  1. Obtain a diagram to visualize:

    git imerge diagram
    

    An example of the output:

    **********
    *??|?????|
    *--+-----+
    *??|#?????
    *??|??????
    *??|??????
    *--+??????
     
    Key:
      |,-,+ = rectangles forming current merge frontier
             * = merge done manually
      . = merge done automatically
      # = conflict that is currently blocking progress
      @ = merge was blocked but has been resolved
      ? = no merge recorded
    

    Abort

  2. The same command is used to abandon the merge process for both the atraditional git merge and git-imerge:

    git merge --abort
    

    The difference is that the traditional git merge takes an “all or nothing” approach, and you wold have to start over.

    However, with git-imerge, unlike regular git merge, aborting does not abandon all previous changes because git-imerge has recorded each of the intermediate merges in additional separate branches.

    Therefore…

    Abort remove

  3. If you are using incremental merge and need to completely abort, remove the temporary branches git-imerge created:

    git imerge remove
    
  4. In either the traditional git merge and git imerge, get back to the branch you were in before starting merge. For example:

    git checkout master
    

    Interactive Merge Final Merge

  5. If you are using imerge, you can simplify commits for the permanent record, to omit the intermediate results (like a rebase):

    git imerge finish --goal=merge
    

    Verify

    By default, the steps above creates a new branch NAME that points at the result, and checks out that branch.

  6. See the branches:

    git branch -avv
    

    A sample response:

    
      
  7. See the final commit created by git-imerge:

    git log --decorate --graph --all
    

    The most recent log message for me:

           *   commit 02631c734be37a281a0676aaa851d58a11b904af (NAME)
    |\  Merge: 72fc5b9 8e2b346
    | | Author: Wilson Mar <wilsonmar@gmail.com>
    | | Date:   Thu Jun 1 18:33:44 2017 -0400
    | | 
    | |     Merge feature1 into master (using imerge)
    

Other merging solutions

Phil Price of Microsoft wrote Typescript “codelens” to provide a better display of merge conflicts in VS Code

Inspired by https://atom.io/packages/merge-conflicts for Atom

Resources

  • https://sethrobertson.github.io/GitFixUm/fixup.html

More

This is one of a series on Git, GitHub, and GitLab:

  1. Git and GitHub videos

  2. Why Git? (file-based backups vs Git clone)
  3. Git Markdown text
  4. Git basics (script)
  5. Git command shortcuts

  6. Git-client based workflows
  7. Git whoops (correct mistakes)
  8. Git rebase
  9. Git interactive merge (imerge)
  10. Git HEAD (Commitish references)
  11. Git commits with a Tag and Signature

  12. Git custom commands
  13. Git utilities
  14. Git hooks

  15. GitHub data security

  16. TFS vs GitHub
  17. GitHub REST API
  18. GitHub GraphQL API
  19. GitHub PowerShell API Programming
  20. GitHub GraphQL PowerShell Module