Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Builds are slow and take time. We can get MSBuild to parallelize them, but only on a single machine, not across a cluster. Why hasn't anyone come up with clustered build solutions in the .Net space? I know such solutions exist in the C++ space (e.g., IncrediBuild, Electric Cloud, etc.) but we .Net developers are forced to build huge solutions on individual machines. Anyone know if there's some unsolvable technical difficulty in this?

Just curious.

share|improve this question

migrated from stackoverflow.com Apr 22 '11 at 19:32

This question came from our site for professional and enthusiast programmers.

    
I don't know what exactly you define as "slow" and "huge". But I'm pretty certain it's peanuts compared to what the larger, very template-heavy C++ projects take. You don't need to wait for thousands of template instanciations, the associated compiletime computations, countless optimization passes, native code generation, etc. that make C++ compile times so infamous. –  delnan Apr 22 '11 at 19:29
    
I've seen Visual Studio Solutions that take +60 minutes to compile; on a modern machine. –  Nate Apr 22 '11 at 19:32
    
Code-weaving usually slows down compile time quite significantly. PostSharp, CodeContracts... –  GregC Apr 22 '11 at 19:42
2  
@Nate: C# code? A C# project that I worked on (~600k lines spread across around 150 projects and 20-30 solutions) built from scratch in about 4 minutes. In my experience, it wouldn't be worthwhile having an IncrediBuild-like system for C#/.NET –  Dean Harding Apr 23 '11 at 8:07
1  
@Dmitri: Well, I've also worked on C++ projects that took over 4 hours to build, so yeah, I'm OK with 4 minute build times :-) That's a rebuild-from-scratch as well, just building a single project or two typically takes seconds... –  Dean Harding Apr 23 '11 at 8:30

2 Answers 2

up vote 3 down vote accepted

" Why hasn't anyone come up with clustered build solutions in the .Net space?"

If compilation were to be clustered:

  1. The compiler will have to first break the compilation into different threads which can be done on indepenent machines.
  2. Send the code to different machines for compliation.
  3. Wait till all machines have completed their task
  4. Get the results from all machines in the cluster together.
  5. Put together the results and link them up.

steps 1 and 5 are dependant on the nature of the project : Too many interdepencies and the number of distribuable tasks is drastically reduced. Step 2 and 4 depend on the status of your network. The time taken to send and recive the object files adds to the compile time and beats the benefit of clustering. Depending on the project you probably may end up with the same time for compiling on a single machine as you would for a cluster.

share|improve this answer
    
If you're saying that it's not worth it, I totally disagree :) Regarding your points, 1. is already being done locally so could be done remotely (e.g., with MSBuild), 2. is really a non-issue, I use DropBox and it does peer-to-peer on my LAN so, basically, it's instant, 3. is why we are doing it in the first place, 4. is same a 2. i.e. a non-issue, and 5. is the only really 'tough' thing. –  Dmitri Nesteruk Apr 23 '11 at 7:39
    
I am saying that it wont be worth it except for very large projects. Compilation is a CPU intesive task. It would have been a geat idea in the days of yore but not today. What appears to us as "instant" may not really be for the computer. What I understand is that network transfer is much slower than single CPu compilation. –  DPD Apr 23 '11 at 10:25

Pretty much any CI system does that. If you're talking specifically about on the dev machine: Automated Build Studio is one such example.

That said, most of the products I've worked on/with never require a full rebuild on the dev machine as the code is usually loosely coupled enough and the devs rarely need to make changes to that many different assemblies between checkins.

share|improve this answer
    
If it's loosely coupled, you're still limited by the # of cores on your machine, not by the number of cores on all your machines. –  Dmitri Nesteruk Apr 23 '11 at 7:40
    
@Dmitri Nesteruk: Yes, but if it's loosely coupled, an incremental build won't perform a whole rebuild, hence you only have to build a couple files/projects instead of the whole application - thus, distributed compilation isn't really necessary. –  Steve Evers Apr 23 '11 at 17:34

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.