Subrepositories (submodules) as a dependency tracking solution - yes or no?

Question

What do you think about SCM-based (subrepositories in mercurial, submodules in git) dependencies management?

Is it definitely a good way of managing dependencies? Or definitely bad way?

Should I prefer some build system (ant/phing/rake) over subrepositories?

Some particular example for you to have a context:

I have several web applications that share across reusable components, say jquery plugins.

Answer 1

If you have the option, then go with a dedicated tool instead of using the SCM for this. I've written about this on SO, but the argument boils down to coupling, in particular overly tight coupling. Flexible software development is all about avoiding tight coupling unless it's really needed and both Mercurial subrepositories and Git submodules introduce a very tight coupling.

In addition, subrepos/submodules introduce extra complexity in the daily workflow. If people are already familiar with the SCM, then this might be acceptable. But I've seen many organizations where they tries to introduce Mercurial and subrepos at the same time — I consider that a mistake since it just makes everything seem extra complicated.

The Mercurial wiki has a set of recommendations for how subrepos should be used. These include the use of a thin shell repository. This avoids coupling the components together. You can work in either subrepo as needed and push/pull to update them. Once in a while someone (maybe a build engineer, maybe a continuous integration server) will test new configurations of the components. If they pass tests, then a new commit is made in the shell repository so that people have a new base to use for their work.

Answer 2

When we switched from SVN to Kiln (Mercurial) a year ago we re-implemented our subrepositories also. I can honestly say that I wish I hadn't. Subrepositories quickly become a hassle to deal with keeping them up to date when the dependencies change. This is especially true when dealing with internal "third part" libraries that are under active development.

We are currently gradually converting over our dependencies to be NuGet packages built and hosted by our TeamCity server.

Before I could recommend subrepositories as a preferred solution choice, I would need to see a good argument in their favor.

Answer 3

I use Mercurial and use subrepositories for all my dependencies (GLEW, GLM, Boost, Assimp, Bullet) but am not convinced it's always a good thing. Sometimes I think it makes more sense to just link against a prebuilt library.

For me, it's nice to be able to build and step through all the external code I'm using for debugging's sake, plus I can make my own branches to make custom changes and still pull updates for my subrepositories.

I keep all my subrepositories in a Dependencies directory just below the project root, although Mercurial's docs recommend a master repository with your own project and any dependencies as subrepositories.

Answer 4

git submodules are great for this. It allows you to keep all of your code in a bundle, upgrade with simplicity, do it on a per-project basis, not worry about the others, and basically keep everything sane.

A couple of caveats:

1. git pull && git submodule init && git submodule update --- these need to be part of your STANDARD workflow.

2. Submodules are hard if people are changing them resulting in conflicts. Not too hard, but all you get is two SHA1's and you have to pick the best one, or manually merge them and upgrade the base repo.

3. Github is a great place to put all your git stuff. That's just a side note, but I use a mix of public repos for open source submodules and private repose for the main project.

Hope this helps!