Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

So, I'm in a big geographically-distributed company, we use perforce and I'm starting to make the case for a DVCS with a whitepaper I was requested to write.

I was thinking that one of the arguments I could use is the bandwidth saved using a DVCS, and technically if there were binaries to download (there are, but they're not sync'ed every day) and enough developers downloading them one could see a substantial difference. Unfortunately that's an edge case and its all that I can think of (other than individual developers being able to work faster when diffing, committing locally, and all those other DVCS perks).

So, would there be an actual noticeable bandwidth save?, in what other cases?. Free cookie to anyone with actual empirical evidence (confirming or refuting).

Edit: It is just one of the arguments I'm trying to support and it's not at all the main one, I got at least 10 others... this is one of the "side-benefits" but I want to find out how real it is. Plus, I want to win over the IT guys =)

share|improve this question
4  
In our SVN -> git conversion, we went from a 4.06gb codebase to 346mb. Imagine the bandwidth savings every time the codebase was checked out/cloned :) –  Nic Mar 11 '12 at 5:34
    
Wow. actual facts. Good job melee, I would post that as a 'yes answer' ! Plus I'd upvote it! –  Michael Durrant Mar 11 '12 at 5:37
2  
Personally I wouldn't focus on it as it isn't the main reason / biggest benefit. I would focus on the manage your local 'commits' while off-line feature. I'd also show gitx (Mac) and gitg (Linux) as great git gui tools that will help win some. One final point - Make sure to explain that branching is different from plain old CVS's as that is one item I have really seen trip up developers from say svn. Branching is done for slightly different reasons in git vs. svn Usually a lot less too which should be communicated. –  Michael Durrant Mar 11 '12 at 5:40
2  
Consider doing the repository conversion as a test and then do some experiments and compare the results. –  user1249 Mar 11 '12 at 7:57

2 Answers 2

up vote 11 down vote accepted

This is an interesting question. Before the analysis, I'll say that bandwidth savings shouldn't be the reason you switch to a DVCS. However, a related point is the load on the server and that will naturally go down with a DVCS: people are simply not using the server as often. When they use it, they use it for "simple" things like push/pull. Heavy-duty operations such as annotate and grep are done on the client.

We can investigate the bandwidth usage by looking at an example where a DVCS works particularly poorly: an example with an image file. The setup is like this:

  • You add a 10 MB image to your project. Since it's a compressed file format you cannot compress it further and the initial version takes up 10 MB in the repository.

  • You make 9 changes to the image. Delta compression is often useless for compressed formats so each new revision takes up an additional 10 MB in the repository.

  • The repository size is now 100 MB.

We can now compare the bandwidth used in with a centralized and a decentralized system. I'm a Mercurial developer so I'll use Mercurial as an example of a DVCS (but Git works the same). I'll use Subversion as the centralized version control system (CVCS):

  • Pushing new revision to server: hg commit; hg push vs svn commit.

    The bandwidth is the same since you need to send a 10 MB delta to the server in both cases.

  • Pulling new revision from server: hg pull --update vs svn update.

    A CVCS will let you download the 10 MB you need whereas a DVCS will ask you to download the intermediate revisions you're missing. So the bandwidth requirement depends on the update frequency:

    • If you do close collaboration and thus update often, then you end up with the same bandwidth requirement.

    • If you update less often, a DVCS will use more bandwidth.

  • Fresh checkout: hg clone vs svn checkout.t

    Similar to the case above, but with very infrequent updates. So a DVCS will download more data than a centralized system.

  • Updating to old version: hg update vs svn update

    A DVCS will require no bandwidth here, but a centralized tool will download 10 MB. Depending on your workflow, you might do this quite a lot when searching for bugs and so you can have significant savings here.

I think the requirements can be summarized as: you pay a larger up-front cost with a DVCS since you download everything. When that cost is paid, updates to old revisions are free. Updates to new revisions cost about the same as with a centralized tool, assuming you update frequently in both cases. Sending commits to the server is equal in the two systems.

This example shows that there is an overhead associated with a DVCS. However, in practice the overhead is manageable. For source code, the delta compression kicks in and does wonders to keep the size of your repository down.

An example from Mercurial:

  • Our biggest source file (mercurial/commands.py) is a 200 KB Python file. Since it's plain text, the initial version can be compressed to about 50 KB inside the repository.

  • We changed the file about 2200 times over the next five years. Delta compression means that each change takes up about 630 bytes in the repository.

  • The total size in the repository is 1.4 MB.

For for that file (probably our most used file) it makes a lot of sense to just download all 2200 revisions up-front — it's just another 1.4 MB to download! So the overhead is extremely low for text files and this is basically why DVCS can even be considered in the first place.

I've also looked at OpenOffice. The working copy for a checkout of tip is 2.0 GB. They have 276,000 changesets in their repository and the entire history takes up 2.3 GB. That is a 15% overhead thanks to excellent delta compression. The overhead for an initial checkout will be larger since a CVCS could compress the 2.0 GB down to maybe 500 MB. But they save bandwidth every time someone has to checkout an old revision to fix a bug — with a DVCS you already have the data right where you need it.

Finally, let me mention that Mercurial has a largefiles extension for handling files that take up too much space in the repository. It works by externalizing them so that they're only downloaded when needed. This effectively turns Mercurial into a CVCS with regard to those files.

share|improve this answer
    
When you tell about pushing, you forgot (?) about D in DVCS - (pull from me|push to neighbor) may happen more than 1 times, while in SVN I svn co only once. But for large files in repo I'll use LargeFiles as A MUST –  Lazy Badger Mar 11 '12 at 14:40
    
@LazyBadger: I think at least 90% of the pushes go to the server and I was trying to describe the bandwidth requirements of the server. The largefiles extension is of course an interesting work-around: it gives you CVCS-like bandwidth requirements, but with a local cache to avoid downloading a file more than once. –  Martin Geisler Mar 11 '12 at 17:23
    
You can also clone from a particular revision and not the entire history, and the largefiles extension allows you to have binary files in a network share. But you made an interesting case. –  dukeofgaming Mar 11 '12 at 17:25
    
@dukeofgaming: yeah, you're right that there is a lot of corner cases. I was trying to describe a simple but realistic use case. –  Martin Geisler Mar 11 '12 at 17:29
    
I think percentage is heavy workflow-dependent and in case of "managed anarchy" traffic will|can increase dramatically (in reasonable big teams) with "any-2-any" connections –  Lazy Badger Mar 11 '12 at 17:37

I'm in the process of writing my own white paper about my experience with converting from SVN to Git as I've mentioned in the comments.

Here's how I won over our IT guys. I don't have the exact numbers off hand, but the VM containing the SVN repo for our project was near 12-14gb of usage. When it came to an actual checkout, it was near 4.06gb coming down. When I experienced this on my first day, I made it my #1 priority to move away from SVN as quickly as possible.

The first population of our repo resulted in a size of about 1.3gb - once I tweaked the .gitignore file, we dropped the repo size even more. I was bent on making the repo as lean as possible, so with some analysis, we ended up at 346mb.

I received a new workstation yesterday, and much like my first day, I had to clone the repository. Downloading the repo was extremely fast with compression on, and took about 9 minutes.

I hesitated in initially posting this as a response, as your mileage may vary significantly - my predecessors may have setup SVN poorly for all that I know. I think that you could (and should) setup a test environment and try the conversion to see what kind of metrics you'll realize and publish those into your paper.

On a more day to day basis, I have a ton of hooks setup to automate pulling between a few different servers, but definitely not enough remote activity to monitor bandwidth usage on a day to day basis. It is very hard to say without more detail whether or not you will realize a benefit there.

share|improve this answer
    
The 346 MB figure, is the size of the checked out files, or the whole repository (in .git)? –  user1249 Mar 11 '12 at 7:56
2  
so was the key for the decrease in size the fact that you ignored files (which you could do in svn as well I guess) or git's compression? –  stijn Mar 11 '12 at 8:53
    
The size of the .git folder is/was ~346mb. I'm not sure what the entire directory tree is now, but my baseline is the size of our gitolite server repo versus the size of the svn repo. –  Nic Mar 11 '12 at 8:53
    
@stijn I'd say a combination of both. The straight-over conversion yielded quite a drastic increase, and tweaking it further reduced the size even more. In looking for evidence to support my experience, I found this article: codeforest.net/git-vs-svn which states that the Mozilla project found similar gains when switching away from SVN, although no direct link/evidence from Mozilla. –  Nic Mar 11 '12 at 9:00
    
Ah, so the ".gitignore" was when creating the git repository from a checked out repository. Don't you need the history?? –  user1249 Mar 11 '12 at 10:40

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.