Git Superprojects & Mercurial Forests

git-logologo-droplets-50I know most of the world is still trying to grok distributed SCM while it is something that I have simply come to expect. I probably should write up something about Git and Mercurial to help bridge the gap between my mindset and where most people probably (currently) are, as this writing will show my complete acceptance of the distributed model. But that article will have to wait as I am actually trying to grok how to use these tools in more advanced ways.

So today, I've decided I need to string together separate, generally unrelated projects as dependencies of a project I'm working on. As an example of what I'm talking about, I have a project that depends on the Sparkle Update Framework for Mac OS X and also Lua. It would be nice to coordinate everything together. There are two pieces to this. One part is getting the build system to understand how to build everything including the dependencies and how to create the proper final .app bundle. The other part is getting the SCM system to connect all the components even though they are separate projects.

So first a word on the build system. I am using Xcode. I have separate independent Xcode projects for Lua, Sparkle, and my project. In my project, I use the cross-project dependency feature (or whatever you call it) that allows my Xcode project to include and reference other Xcode projects. I have decided that I will keep all 3 projects in separate directories, but all as peers at the same directory level. I let Xcode use relative paths to reference the other projects.

This pretty much works fine except for one really annoying aspect which is a bug. I have 'Copy Files Build Phases' setup in my project's settings to copy the frameworks and executables from the dependent Xcode projects (Lua and Sparkle) into my application's bundle. Xcode will let me drag-and-drop the Products from the dependent Xcode projects into the Copy Files Build Phases. But when I actually build, Xcode fails to copy these files because it is being stupid and looking for the files in a location relative to my project instead of the location relative to the dependent project. I filed a bug almost 2 years ago about this (22 months ago), and it hasn't been touched. The bug id is 4487731. This one really irks me.

So the other aspect is the SCM and what this article is really about. So I've been using both Git and Mercurial, trying to figure out which one I like more. Both have their strengths and weaknesses and I still haven't committed to one. So the next challenge is how to do this. 

In Git 1.5.3, a new feature called superprojects or submodules was introduced. This appears to be the way to do this in Git. There's a pretty good write up on how to use the feature here (Chapter 8 Submodules):

http://www.kernel.org/pub/software/scm/git/docs/user-manual.html#submodules

Unfortunately, it seems rather complicated to use, or maybe just more steps than I care to remember. I am also uneasy about how submodule add expects you to reference a different location with warnings about not using local URLs if you plan to publish. I'm rather confused by this. First, for now, I am predominantly local. However, if I want to change this later, I'm not sure what kind of trouble I will get into. Also, I had already laid out the dependency directories below the super directory. This is not allowed. It must be referenced from somewhere else to clone into the super directory. The command fails when I try any way.

Meanwhile, Mercurial doesn't seem to officially support this feature, but there is an extension called Forest that supports this. Unfortunately, this extension doesn't seem to be included with the OS X installer and I had to go hunting for it. Also, the documentation doesn't really say if there are problems/issues one should be aware of (reading into why it is not part of the official core).

http://www.selenic.com/mercurial/wiki/index.cgi/ForestExtension

After playing with both of these for a little bit, I found Git a little easier to setup and use compared to Mercurial in this feature. However, neither has really sunk in yet.


Update: (August 28, 2008)

I have posted a concrete example of how to setup and use a Git SuperProject and a Mercurial Forest. See the two articles entitled: 

Git Superproject and Submodules: A Real World Example (LuaDoc with CMake)

and

Mercurial Forests: A Real World Example (LuaDoc with CMake)


Update: (September 4, 2009)

I have posted a new followup example to Mercurial Forests using the new (experimental) Mercurial Subrepos feature.

Copyright © PlayControl Software, LLC / Eric Wing