Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Does this depend heavily on the type of project (e.g. scientific software vs banking system vs game)? If so, in which areas is the difference the highest and lowest? In your experience, is it safe to say that as a rule, the time it takes to run unit tests is at least one order of magnitude larger than that of building the software in the first place?

share|improve this question
    
It is highly dependant on many factors: money, project type, time, requirements etc. Some projects don't use unit tests at all. The question is very broad. –  oleksii Jul 23 '11 at 18:10
    
Is this a factor for adopting unit tests? –  JeffO Jul 23 '11 at 19:54

3 Answers 3

Not on the type of project, but rather the issues related to the project.

If you're developing a piece of software which will be used to control a nuclear plant, chances are you will have to do extensive testing, code review, etc. For such projects where huge amounts of money and people lives are concerned, it is not unusual to spend days on a single line of code, deciding if it must be written, and if yes, how, adding tests, etc. This also means that you will have lots of testing code, which will be slow to run compared to the compile time of a rather small codebase.

If on the other hand you're writing a piece of code you'll use for your personal website which is visited at best by a few members of your family or a few friends, chances are you will spend all your time developing new features, with no tests at all, so the time you spend running unit tests will be always zero.


This being said, I don't understand why do you compare the time spent running the tests with the time spent compiling the codebase. Those are two very different processes which has nothing in common. Say you want to test a simple Hello World app which must wait for ten seconds, then display "Hello World". Probably the tests will run approximately twenty seconds, maybe much more, while the compile time will be a few milliseconds.

share|improve this answer

In your experience, is it safe to say that as a rule, the time it takes to run unit tests is at least one order of magnitude larger than that of building the software in the first place?

No. It depends on too many things, like how many tests there are, how much setup they require, what language is used (C++ is notoriously slow to compile, but fast to run), how efficient the compiler is, etc.

Besides, what meaningful conclusion could be drawn from this ratio?

share|improve this answer

We are making a game engine, and our unit tests take about the same time to run as it takes the engine to compile. But:

  • some of our tests are supposed to take some time, e.g. timer tests
  • some of our tests show visualisations (well, you gotta test the graphics) and are not automated (cannot be, because we cannot check if it looks right automatically) and need the programmer to evaluate the test, which takes time
  • also, we have ~20 tests, and their linking time is quite high (around 1-2 seconds each, so you spend half a minute just linking your test) which is not really part of the engine build

Most of this does not apply to most applications, I guess. So yeah, I think the ratio you mentioned depends on the type of application (not exclusively, but it does depend).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.