AdamFranco.com » source-control

Git Tip: Grouping feature-branch commits when merging.

Adam — Sun, 12 Dec 2010 18:32:02 +0000

Let’s say you are working on a large feature or update that requires a bunch of commits to complete. You finish up with your work and are then ready to merge it onto your master branch.

For example, here is the history of my drupal repository after some work updating the cas module to the latest version (and to support the new version of phpCAS):

As you can see, I have a number of commits, followed by a merge in with the new module code, followed by some more commits.

Now, if I merge my feature branch (master-cas3-simple) into the master via

git merge  master-cas3-simple

then the history will look like this:

While the history is all there, it isn’t obvious that all of the commits beyond “Convert MS Word quote…” are a single unit of work. They all kind of blend together because git performed a “fast-forward” commit. Usually fast-forward commits are helpful since they keep the history from being cluttered with hundreds of unnecessary merge commits, but in this case we are loosing the context of these commits being a unit of work.

To preserve the grouping of these commits together I can instead force the merge operation to create a merge commit (and even append a message) by using the --no-ff option to git merge:

git merge --no-ff -m "Upgraded CAS support to to cas-6.x-3.x-dev and phpCAS 1.2.0 RC2.5" master-cas3-simple

This results in the history below:

As you can see, merging with the --no-ff option creates a merge commit which very obviously delineates work on this feature. If we decided that we wanted to roll back this feature it would be much easier to sort out where the starting point before the feature was.

Thanks to Vincent Driessen for turning me onto the utility of the the --no-ff merge option via his post “A successful Git branching model“.

Mirroring a Subversion repository on Github

Adam — Sun, 05 Dec 2010 05:30:12 +0000

For the past few months I have been doing a lot of work on the phpCAS library, mostly to improve the community trunk of phpCAS so that I wouldn’t have to maintain our own custom fork with support for the CAS attribute format we use at Middlebury College. The phpCAS project lead, Joachim Fritschi, has been great to work with and I’ve had a blast helping out with the project.

The tooling has involved a few challenges however, since Jasig (the organization that hosts the CAS and phpCAS projects) uses Subversion for its source-code repositories and we use Git for all of our projects. Now, I could just suck it up and use Subversion when doing phpCAS development, but there are a few reasons I don’t:

We make use of Git submodules to include phpCAS along with the source-code of our applications, necessitating the use of a public Git repository that includes phpCAS.
The git-svn tools allow me to use git on my end to work with a Subversion repository, which is great because…
I find that Git’s fast history browsing and searching make troubleshooting and bug fixing much easier than any other tools I’ve used.

For the past two years I have been using git-svn to work with the phpCAS repository and every so often pushing changes up to a public Git repository on GitHub. Our applications reference this repository as a submodule when they need to make use of phpCAS. Now that I’ve been doing more work on phpCAS (and am more interested in keeping our applications using up-to-date versions), I’ve decided to automate the process of mirroring the Subversion repository on GitHub. Read on for details of how I’ve set this up and the scripts for keeping the mirror in sync.

On my development server I have a git repository I’ve cloned from the Jasig Subversion repository via:

git svn clone --stdlayout https://source.jasig.org/cas-clients/phpcas/

I use this repository for my phpCAS development and am continually using git svn rebase and git svn dcommit to update my branches from Subversion and commit changes back to the Subversion repository.

My goal was to fetch from Subversion and push all of the branches and tags from the Subversion repository to GitHub while ignoring any private branches or un-dcommited changes I might have kicking about my development repository.

Step 1: Add the GitHub repository as a remote

git remote add github git@github.com:adamfranco/phpcas.git

Step 2: Fetch the latest changes from the svn repository
To do this, I just needed to run git svn fetch to import commits from the Subversion repository into my Git repository.

Step 3: Make Git tags for Subversion tag-branches
I may be doing something wrong, but it seems that Subversion tags come through git svn as git branches rather than as git “tag” objects. Basically they are a branch with a single commit that just adds the tag message, but no content change. Using git show I found I could grab the parent id, message, and other metadata from the “tag-branch”, then feed that into git tag to create actual tag objects in the git repository.

Step 4: Push subversion branches and newly created tags to GitHub
When called with no parameters git push will push all branches that have matching names in both the source and the destination repository. This wasn’t going to work for me since I want to only automatically push the branch-state that exists in subversion (not any un-dcommitted changes in my Git repository) and want to create mirrors of any new branches that appear in the Subversion repository. To accomplish this I needed to specify every branch individually. I determined the list of branches via:

git branch -r | grep -v '/' | grep -v trunk

then looped through them and appended them to the hard-coded mapping between the svn “trunk” and the GitHub “master”:

$cmd = 'git push --tags github refs/remotes/trunk:refs/heads/master ';
foreach ($svnBranches as $branch) {
	$cmd .= 'refs/remotes/'.$branch.':refs/heads/'.$branch.' ';
}

All together: update_github_phpcas
Below is a script which performs the tasks above. I’ve added it to my crontab so that it runs every half-hour and keeps my GitHub repository in-sync with the Jasig Subversion repository.



#!/usr/local/bin/php


/**

* Script to mirror a Subversion repository on GitHub or another public Git repository.

*

* Author: Adam Franco (afranco@middlebury.edu)

* Date: 2010-12-04

* License: GNU General Public License (GPL) version 2 or later.

*/ 
chdir('/home/afranco/private_html/phpcas/');
// Fetch from svn.

`git svn fetch`;
// Lookup all of the svn branches

$svnBranches = explode("\n", trim(`git branch -r | grep -v '/' | grep -v trunk`));

$svnBranches = array_map('trim', $svnBranches);
// Add all of our branches to the list of those to push

$cmd = 'git push --tags github refs/remotes/trunk:refs/heads/master ';

foreach ($svnBranches as $branch) {

$cmd .= 'refs/remotes/'.$branch.':refs/heads/'.$branch.' ';

}
// Ensure that Git tags are created for every SVN tag branch.

$svnBranches = explode("\n", trim(`git branch -r | grep 'tags/'`));

$svnBranches = array_map('trim', $svnBranches);

foreach ($svnBranches as $svnTag) {

$ref = "refs/remotes/$svnTag";

$parent = shell_exec("git show --format=\"format:%P\" $ref");
// If there are no tags on the parent of the tag branch, add one.

if (!strlen(trim(`git tag --contains $parent`))) {

$message = shell_exec("git show --format=\"format:%s%ntagged by %aN on %aD\" $ref");

$date = shell_exec("git show --format=\"format:%ai\" $ref");

$tagName = str_replace('tags/', '', $svnTag);
$tagCmd = 'GIT_COMMITTER_DATE="'.$date.'" git tag -a -m "'.$message.'" '.$tagName.' '.$parent;

#        print $tagCmd ."\n";

#        print "Creating tag $tagName\n";

`$tagCmd`;

}

}
#print $cmd;

#print "\n";
$output = `$cmd 2>&1`;
if (trim($output) != 'Everything up-to-date')

print $output."\n";

I think that this script should work with very few changes for mirroring any Subversion repository as a Git repository.

Git Tip of the Day: Stage Hunks

Adam — Tue, 13 Jan 2009 16:19:43 +0000

One of the great things about the Git version-control system is the ability to incrementally commit your changes on a private branch to keep a step-by-step record of your thought and writing process on a fix or a feature, and then merge the completed work onto your main [or public] branch after your feature or fix is all done and tested. By keeping an incremental log of your changes — rather than just committing one giant set of code with changes to 30 files — it becomes much easier to know why a certain line was changed in the future when bugs are discovered with it.

One thing that often happens to me though, is that I work for about a half hour to an hour trying to get a new piece of code working and in the process make several sets of changes to one file that are only loosely related.

Let’s say that I am fixing a bug in my ‘MediaLibrary’ class and while doing so notice some some spelling mistakes in some comments that I fix. Now my one file has two changes my bug fix, and the spelling fix. Rather than committing both changes together with one comment describing both changes, I can highlight one of the changes in git-gui and select the “Stage Hunk for Commit” option.

With that one hunk staged I can now commit with a message applicable to that change. Other changes can then be staged and committed with their own messages resulting in a very understandable history of changes.

“Stage Hunk for Commit” can also be used to commit important changes while not including debugging lines inserted in your code.