Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

This question already has an answer here:

Why is it often said that the test cases need to be made before we start coding?
What are its pros and what the cons if we don't listen to this advice?

Moreover, does that advice refer to black box testing or White Box testing or both?

share|improve this question

marked as duplicate by JeffO, MichaelT, gnat, Martijn Pieters, Kilian Foth Mar 18 '13 at 9:53

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

    
Do (all/many) test cases need to be made before we start coding?. I know that spec shoud be done before coding. With tdd you usally do not write all testcases upfront before start coding but one at a time. –  k3b Jan 12 '13 at 7:55

5 Answers 5

up vote 27 down vote accepted

Writing tests before implementations is one of the core ideas behind Test Driven Development (TDD). The procedure is:

  • write a failing test
  • change the code to make it pass, without breaking any other test
  • refactor the code, verify that all tests still pass

This procedure is the same whether you implement a feature or fix a bug; it is often referred to as "Red-Green-Refactor" (red = test fails, green = test passes). The advantages of this method are numerous:

  • the test clearly defines what constitutes "done"
  • the test documents how you intend the code to be used
  • if you do it right, you build a complete test suite with 100% coverage as a by-product of your development process, which means you always have reliable regression tests at hand no matter what you do
  • coding to meet a test case (and nothing else) helps keep you focused on one task and prevents feature creep
  • the red-green-refactor cycle makes for a nice, clean and tracable commit history and fits a feature branch repository approach well (each feature branch starts with a commit that adds a failing test, then one or more commits until "green" is reached, and then one or more refactoring commits).
  • you produce working versions at regular intervals - ideally, each red-green-refactor cycle ends with a potentially shippable product, but at the very least, it should build without failures and pass all the tests.

Now for the downsides:

  • Not all code lends itself to automated testing. This can be because it's a legacy codebase written without automated tests in mind; it can be because the problem domain is such that certain things cannot be mocked; or maybe you rely on external data sources that are not under your control and would be too complicated to mock; etc.
  • Sometimes, correctness, maintainability and stability are not high enough on the priority list. This sounds bad, but it doesn't have to be: if you're writing a one-use script to batch convert a bunch of documents, writing tests at all is silly - you just run it on a few test documents, eyeball the result, and that's that. Automated testing wouldn't add any value here.
  • Writing the test before writing the code implies that you already know exactly what you want done. Quite often, though, you don't, and a significant amount of explorative programming is required to get a feeling for the particular problem domain; and more often than not, the results of such explorative coding are then bashed into shape to become the real product. Automated tests are just as valuable for such products, but it doesn't make much sense adding them before you know what you're going to make. In this case, it's better to take the prototype at the end of the explorative phase, add tests to cover everything you have so far, and switch to red-green-refactor from there.
  • Test-before-implementation isn't everyone's cup of tea; writing the implementation first feels more natural, and some people have an easier time getting into a flow that way
share|improve this answer
    
Good answer. I've noticed that when I have a good idea of what I want to build TDD comes easier than when I really have not clue where to start. –  ipaul Mar 18 '13 at 2:40

Why is it often said that the test cases need to be made before we start coding?

It's a basic principle of test-driven development, but not a general "best practice". Not everyone wants or needs to do TDD, even though its proponents often claim that everyone would benefit from it.

What are its pros and what the cons if we don't listen to this advice?

The advantage is that it forces you to think about your code from a perspective of testing before you write it, which has two main advantages: you're less likely to forget about special cases and boundary conditions, and your code has to be testable and thus modular. Another important benefit: at any time you know that your code is working as you thought it should.

The disadvantage is that it may encourage thinking about low-level details and overlook important design issues (but the same can happen if you just start coding). Also, the most easily testable design is not necessarily the best (but probably much better than one grown out of ad hoc coding).

Of course there's the question whether TDD saves more time than writing all those unit tests costs. Its proponents certainly claim so.

Moreover, does that advice refer to black box testing or White Box testing or both?

Usually, TDD deals exclusively with unit tests, which are white box tests. Integration tests have to be done additionally.

share|improve this answer
    
helpful of course. –  TheIndependentAquarius Jan 11 '13 at 10:11
6  
Another disadvantage is that, while proponents of TDD often tout it as a "design methodology", it has serious shortcomings as a design tool (e.g. the assumption that it's reasonable to expect a solution to grow organically out of a bunch of tests). Google the infamous Sudoku TDD affair! –  Andres F. Jan 11 '13 at 15:08
2  
@Andres F.: that's what I meant with "the most easily testable design is not necessarily the best". It practically forces you towards modularity and low coupling, which is certainly something you want in a good design - but not everything. On the other hand, the only way in which I see TDD leading to bad design is if you focus on it to the exclusion of everything else. –  Michael Borgwardt Jan 11 '13 at 15:20
2  
"it forces you to think about your code from a perspective of testing": yes, which means "as a user of the code." You can write your tests as though you had easy-to-use code and then write it that way. This includes things like "but I don't want to do a bunch of test setup", which will push you to fewer dependencies. –  Nathan Long Jan 11 '13 at 16:59
2  
TDD is not exclusively done with unit tests. ATDD is done with acceptance tests. Different level of granularity. All test driven means is that you are first writing a test (a contract) and then writing code to satisfy the test (meet the contract). –  dietbuddha Jan 12 '13 at 16:38

It is said in the context of practising TDD/BDD, which are design methodologies, not test methodologies.

With TDD/BDD, the idea is to drive out the API usage before writing the API, evolving it with tests.

Seen in this light, the point is to write a test for a non-existing API call or a test for a non-existing API behaviour and then implementing it.

As for blackbox/whitebox testing - again, this isn't about testing - it is about design.

share|improve this answer
    
what is TDD BDD? –  TheIndependentAquarius Jan 11 '13 at 9:54
    
TDD - Test Driven Design. BDD - Behaviour Driven Design. –  Oded Jan 11 '13 at 9:57

When I am writing test first, it isn't much a matter of verifying correct functioning of my code, not really a matter of black box or white box or whatever related to quality assurance, and here is why...

Over the years I practiced and polished a skill to convince management that I need good testers 1, 2, 3 and this suffices to ensure that my code will work as intended without me having to think much about it.

Main reason why I prefer test first is it helps me understand how to design my code so that it's convenient to use. I am not good at imagining in my head how my module will be used by other modules so I can do it overly complicated and that will bite me later and possibly even require redesign.

This is a problem no matter whether I am coding a module for myself or for someone else, it is equally painful using inconvenient interface myself as well as having to explain its usage to other programmers.

http://i.stack.imgur.com/JwlL8.jpg

"Test-first" approach saves me from that pain; it doesn't require imagination to immediately see how my module will be used if designed this or that way and makes it easy to choose the more convenient one.

When coding module for someone else, it also saves me from the need to painfully explain how to use it.

Look, code in this test shows how you should use that module and if you run it, you'll also see what results you should expect. Using any other way than shown in this test would be a mistake. Now go away and let me do something interesting.

share|improve this answer

Why is it often said that the test cases need to be made before we start coding?

There are a couple reasons why this is said.

  • Some people approach tests (blackbox) as contracts. This forces the person writing the test and the person implementing the code to have a clear set of criteria for success. This means you write code to meet the contract and that's how you know when the functionality required is met.
  • TDD (Test Driven Development) is a practice where you drive all development through tests. Classically this is done with unit tests, but more recently with tests at whatever level are appropriate to your product and process.

What are its pros and what the cons if we don't listen to this advice?

Pros

  • Drives out ambiguity earlier in the process because you must understand what the success criteria is before writing the test.
  • Some proponents suggest there are design benefits specifically to unit TDD because it enforces loose coupling and more modular design.
  • Confidence that your code is working (at least for the portions that are tested).

Cons

  • Understand how to write maintainable tests.
  • Understand how to write good tests.
  • Understand when to make the trade off between a very expensive good test and a low cost less good test.
  • More code to maintain

Moreover, does that advice refer to black box testing or White Box testing or both?

It depends. Most unit tests tend to be more blackbox in that they will examine input and output of the method or function. However, there are times when mocks or stubs are needed making it more of a whitebox test as it now requires information about the implementation. It may be a moot point for the purposes of TDD because the developer is typically the one writing the tests.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.