Is switching from a CVCS to a DVCS a considerable bandwidth save?

Question

So, I'm in a big geographically-distributed company, we use perforce and I'm starting to make the case for a DVCS with a whitepaper I was requested to write.

I was thinking that one of the arguments I could use is the bandwidth saved using a DVCS, and technically if there were binaries to download (there are, but they're not sync'ed every day) and enough developers downloading them one could see a substantial difference. Unfortunately that's an edge case and its all that I can think of (other than individual developers being able to work faster when diffing, committing locally, and all those other DVCS perks).

So, would there be an actual noticeable bandwidth save?, in what other cases?. Free cookie to anyone with actual empirical evidence (confirming or refuting).

Edit: It is just one of the arguments I'm trying to support and it's not at all the main one, I got at least 10 others... this is one of the "side-benefits" but I want to find out how real it is. Plus, I want to win over the IT guys =)

Answer 1

This is an interesting question. Before the analysis, I'll say that bandwidth savings shouldn't be the reason you switch to a DVCS. However, a related point is the load on the server and that will naturally go down with a DVCS: people are simply not using the server as often. When they use it, they use it for "simple" things like push/pull. Heavy-duty operations such as annotate and grep are done on the client.

We can investigate the bandwidth usage by looking at an example where a DVCS works particularly poorly: an example with an image file. The setup is like this:

• You add a 10 MB image to your project. Since it's a compressed file format you cannot compress it further and the initial version takes up 10 MB in the repository.

• You make 9 changes to the image. Delta compression is often useless for compressed formats so each new revision takes up an additional 10 MB in the repository.

• The repository size is now 100 MB.

We can now compare the bandwidth used in with a centralized and a decentralized system. I'm a Mercurial developer so I'll use Mercurial as an example of a DVCS (but Git works the same). I'll use Subversion as the centralized version control system (CVCS):

• Pushing new revision to server: hg commit; hg push vs svn commit.

  The bandwidth is the same since you need to send a 10 MB delta to the server in both cases.

• Pulling new revision from server: hg pull --update vs svn update.

  A CVCS will let you download the 10 MB you need whereas a DVCS will ask you to download the intermediate revisions you're missing. So the bandwidth requirement depends on the update frequency:

  • If you do close collaboration and thus update often, then you end up with the same bandwidth requirement.

  • If you update less often, a DVCS will use more bandwidth.

• Fresh checkout: hg clone vs svn checkout.t

  Similar to the case above, but with very infrequent updates. So a DVCS will download more data than a centralized system.

• Updating to old version: hg update vs svn update

  A DVCS will require no bandwidth here, but a centralized tool will download 10 MB. Depending on your workflow, you might do this quite a lot when searching for bugs and so you can have significant savings here.

I think the requirements can be summarized as: you pay a larger up-front cost with a DVCS since you download everything. When that cost is paid, updates to old revisions are free. Updates to new revisions cost about the same as with a centralized tool, assuming you update frequently in both cases. Sending commits to the server is equal in the two systems.

This example shows that there is an overhead associated with a DVCS. However, in practice the overhead is manageable. For source code, the delta compression kicks in and does wonders to keep the size of your repository down.

An example from Mercurial:

• Our biggest source file (mercurial/commands.py) is a 200 KB Python file. Since it's plain text, the initial version can be compressed to about 50 KB inside the repository.

• We changed the file about 2200 times over the next five years. Delta compression means that each change takes up about 630 bytes in the repository.

• The total size in the repository is 1.4 MB.

For for that file (probably our most used file) it makes a lot of sense to just download all 2200 revisions up-front — it's just another 1.4 MB to download! So the overhead is extremely low for text files and this is basically why DVCS can even be considered in the first place.

I've also looked at OpenOffice. The working copy for a checkout of tip is 2.0 GB. They have 276,000 changesets in their repository and the entire history takes up 2.3 GB. That is a 15% overhead thanks to excellent delta compression. The overhead for an initial checkout will be larger since a CVCS could compress the 2.0 GB down to maybe 500 MB. But they save bandwidth every time someone has to checkout an old revision to fix a bug — with a DVCS you already have the data right where you need it.

Finally, let me mention that Mercurial has a largefiles extension for handling files that take up too much space in the repository. It works by externalizing them so that they're only downloaded when needed. This effectively turns Mercurial into a CVCS with regard to those files.

Answer 2

I'm in the process of writing my own white paper about my experience with converting from SVN to Git as I've mentioned in the comments.

Here's how I won over our IT guys. I don't have the exact numbers off hand, but the VM containing the SVN repo for our project was near 12-14gb of usage. When it came to an actual checkout, it was near 4.06gb coming down. When I experienced this on my first day, I made it my #1 priority to move away from SVN as quickly as possible.

The first population of our repo resulted in a size of about 1.3gb - once I tweaked the .gitignore file, we dropped the repo size even more. I was bent on making the repo as lean as possible, so with some analysis, we ended up at 346mb.

I received a new workstation yesterday, and much like my first day, I had to clone the repository. Downloading the repo was extremely fast with compression on, and took about 9 minutes.

I hesitated in initially posting this as a response, as your mileage may vary significantly - my predecessors may have setup SVN poorly for all that I know. I think that you could (and should) setup a test environment and try the conversion to see what kind of metrics you'll realize and publish those into your paper.

On a more day to day basis, I have a ton of hooks setup to automate pulling between a few different servers, but definitely not enough remote activity to monitor bandwidth usage on a day to day basis. It is very hard to say without more detail whether or not you will realize a benefit there.