Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

For a while I have been trying to learn to write unit tests for my code.

Initially I started out doing true TDD, where I wouldn't write any code until I'd written a failing test first.

However, I recently had a thorny problem to solve which involved a lot of code. After spending a good couple of weeks writing tests and then code, I came to the unfortunate conclusion that my entire approach was not going to work, and I would have to throw out two weeks work and start again.

This is a bad enough decision to come to when you've just written the code, but when you've also written several hundred unit tests it becomes even more emotionally difficult to just throw it all away.

I can't help thinking that I've wasted 3 or 4 days of effort writing those tests when I could have just put the code together for proof of concept and then written the tests afterwards once I was happy with my approach.

How do people who practice TDD properly handle such situations? Is there a case for bending the rules in some cases or do you always slavishly write the tests first, even when that code may turn out to be useless?

share|improve this question
6  
Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away. - Antoine de Saint-Exupery –  mouviciel Feb 9 '12 at 9:46
11  
How is it possible that all your tests are wrong? Please explain how a change in implementation invalidates every single test you wrote. –  S.Lott Feb 9 '12 at 10:46
5  
@S.Lott: The tests were not wrong, they were just no longer relevant. Say you are solving part of a problem using prime numbers, so you write a class to generate prime numbers and write tests for that class to make sure it's working. Now you find another totally different solution to your problem that doesn't involve primes in any way. That class and it's tests are now redundant. This was my situation only with 10's of classes not just one. –  GazTheDestroyer Feb 9 '12 at 11:03
5  
@GazTheDestroyer its seems to me that distinguishing between the test code and the functional code is a mistake - its all part of the same development process. It is fair to note that TDD has an overhead that is usually recovered further down the developmet process and that it seems that that overhead has gained you nothing in this case. But equally how much did the tests inform your understanding of the failings of the architecture? Its also important to note that you are allowed (nay, encouraged) to prune your tests over time... although this is probably a bit extreme (-: –  Murph Feb 9 '12 at 11:54
9  
I'm going to be semantically pedantic and agree with @S.Lott here; what you did is not refactoring if it results in throwing away many classes and the tests for them. That's re-architecting. Refactoring, especially in the TDD sense, means that the tests were green, you changed some internal code, re-ran the tests, and they stayed green. –  Eric King Feb 9 '12 at 15:15
show 10 more comments

11 Answers

I feel there are two issues here. The first is that you didn't realize in advance that your original design may not be the best approach. Had you known this in advance, you may have chosen to develop a quick throw-away prototype or two, to explore the possible design options and to assess which is the most promising way to follow. In prototyping, you need not write production quality code and need not unit test every nook and cranny (or at all), as your sole focus is on learning, not on polishing the code.

Now, realizing that you need prototyping and experiments rather than starting the development of production code right away, is not always easy, and not even always possible. Armed with the knowledge just gained, you may be able to recognize the need for prototyping next time. Or may not. But at least you know now that this option is to be considered. And this in itself is important knowledge.

The other issue is IMHO with your perception. We all make mistakes, and it is so easy to see in retrospect what we should have done differently. This is just the way we learn. Write down your investment into unit tests as the price of learning that prototyping may be important, and get over it. Just strive not to make the same mistake twice :-)

share|improve this answer
2  
I did know that it was going to be a difficult problem to solve and that my code would be somewhat exploratory, but I was fired up with enthusiasm from my recent TDD successes, so I carried on writing tests as I had done since that's what all the TDD literature stresses so much. So yes, now I know that the rules can be broken (which is what my question was all about really) I will probably chalk this up to experience. –  GazTheDestroyer Feb 9 '12 at 10:50
2  
"I carried on writing tests as I had done since that's what all the TDD literature stresses so much". You probably ought to update the question with the source of your idea that all tests must be written before any code. –  S.Lott Feb 9 '12 at 14:37
1  
I have no such idea and I'm not sure how you got that from the comment. –  GazTheDestroyer Feb 9 '12 at 15:20
1  
I was going to write an answer, but upvoted yours instead. Yes, a million times yes: if you don't know what your architecture looks like yet, write a throwaway prototype first, and don't bother with writing unit tests during prototyping. –  Robert Harvey Feb 9 '12 at 15:25
1  
@WarrenP, surely there are people who think TDD is the One True Way (anything can be turned into a religion if you try hard enough ;-). I prefer to be pragmatic though. For me TDD is one tool in my toolbox, and I use it only when it helps, rather than hinders, solving problems. –  Péter Török Feb 9 '12 at 18:04
show 3 more comments

The point of TDD is that it forces you to write small increments of code in small functions, precisely to avoid this problem. If you have spent weeks writing code on one domain, and every single utility method you wrote becomes useless when you rethink the architecture, then your methods are almost certainly way too large in the first place. (Yes, I'm aware this isn't exactly comforting rght now...)

share|improve this answer
3  
My methods weren't large at all, they simply became irrelevant given the new architecture which bore no resemblance to the old architecture. Partly because the new architecture was far simpler. –  GazTheDestroyer Feb 9 '12 at 9:39
    
Alright, if there truly is nothing reusable you can only cut your losses and move on. But the promise of TDD is that it makes you reach the same goals faster, even though you write test code in addition to application code. If that is true, and I firmly think it is, then at least you reached the point where you realized how to do the architecture in "a couple weeks" rather than twice that time. –  Kilian Foth Feb 9 '12 at 11:02
1  
@Kilian, re "the promise of TDD is that it makes you reach the same goals faster" - what goals are you referring to here? It is pretty obvious that writing unit tests along with the production code itself makes you slower initially, compared to just churning out code. I would say TDD is only going to pay back in the long term, due to improved quality and reduced maintenance costs. –  Péter Török Feb 9 '12 at 17:08
    
@PéterTörök - There are people who insist TDD never has any cost because it pays for itself by the time you have written the code. That's certainly not the case for me but Killian appears to believe it for himself. –  psr Feb 10 '12 at 1:01
    
Well... if you don't believe that, in fact if you don't believe that TDD has a substantial payoff rather than a cost, then there isn't any point in doing it at all, is there? Not just in the very specific situation that Gaz described, but at all. I'm afraid I've now driven this thread completely off-topic :( –  Kilian Foth Feb 10 '12 at 8:55
show 1 more comment

Brooks said "plan to throw one away; you will, anyhow". It seems to me that you are doing just that. That said, you should write your unit tests to test unit of code and not large swath of code. Those are more functional tests and therefore should it over any internal implementation.

For example, if I want to write a PDE (partial differential equations) solver, I would write a few tests trying to solve things that I can solve mathematically. Those are my first "unit" tests -- read: functional tests run as part of a xUnit framework. Those will not change depending on what algorithm I use to solve the PDE. All I care about is the result. The second unit tests will focus on the functions used to code the algorithm and thus would be algorithm specific -- say Runge-Kutta. If I were to find out that Runge-Kutta was not suitable, then I would still have those top level tests (including the ones that showed that Runge-Kutta was not suitable). Thus the second iteration would still have many of the same tests as the first.

Your problem maybe on of design and not necessarily of code. But without more details, it is difficult to say.

share|improve this answer
    
It's only peripheral, but what's PDE? –  Michael Kjörling Feb 9 '12 at 12:40
1  
@MichaelKjörling I guess it's Partial Differential Equation –  foraidt Feb 9 '12 at 13:09
1  
Didn't Brooks retract that statement in his 2nd edition? –  Simon Feb 9 '12 at 16:51
    
How do you mean you'll still have the tests that show Runge-Kutta was not suitable? What do those tests look like? Do you mean that you saved the Runge-Kutta algorithm you wrote, before discovering it wasn't suitable, and running the end-to-end tests with RK in the mix would fail? –  moteutsch Jan 21 at 22:52
add comment

You should keep in mind that TDD is an iterative process. Write a small test (in most cases a few lines should be sufficient) and run it. The test should fail, now directly work on your main source and try to implement the tested functionality so that the test passes. Now start over again.

You should not try to write all the tests in one go, because, as you have noticed, this is not going to work out. This reduces the risk of wasting your time writing tests that aren't going to be used.

share|improve this answer
1  
I don't think I can have explained myself very well. I do write tests iteratively. That's how I ended up with several hundred tests for code that suddenly became redundant. –  GazTheDestroyer Feb 9 '12 at 10:01
1  
As above - I think it should be thought of as "tests and code" rather than "tests for code" –  Murph Feb 9 '12 at 11:56
1  
+1: "You should not try to write all the tests in one go," –  S.Lott Feb 9 '12 at 13:29
add comment

I think you said it yourself: you were not sure about your approach before you started writing all your unit tests.

The thing I learned comparing the real-life TDD projects I worked with (not that many in fact, only 3 covering 2 years of work) with what I had learned theoretically, is that Automated Testing != Unit Testing (without of course being mutually exclusive).

In other words, the T in TDD doesn't have to have a U with it... It is automated, but is less a unit test (as in testing classes and methods) than an automated functional test: it's at the same level of functional granularity as the architecture you are presently working on. You start high-level, with few tests and only the functional big picture, and only eventually you end up with thousands of UTs, and all your classes well-defined in a beautiful architecture...

Unit tests give you a great deal of help when you work in team, to avoid code changes creating endless cycles of bugs. But I never wrote anything so precise when starting to work on a project, before having at least a global working POC for each user story.

Maybe it's just my personal way of doing this. I don't have the sufficient experience to decide from scratch what patterns or structure my project will have, so indeed I won't waste my time writing 100s of UTs from the beginning...

More generally, the idea of breaking everything and throwing it all will always be there. As "continuous" as we can try to be with our tools and methods, sometimes the only way there is to fight entropy is to start over. But the goal is that when that happens, the automated and unit testing you implemented will have made your project already less costly than if there were not there - and it will, if you find the equilibrium.

share|improve this answer
3  
well said - it's TDD, not UTDD –  Steven A. Lowe Feb 9 '12 at 15:58
    
Excellent answer. In my experience of TDD it is important that the written tests focus on the functional behaviours of the software and away from unit testing. It is more difficult to think about the behaviours you need from a class, but it does lead to clean interfaces and potentially simplifies the resulting implementation (you don't add functionality that you don't actually need). –  JohnTESlade Feb 15 '12 at 13:30
add comment
How do people who practice TDD properly handle such situations?
  1. by considering when to prototype vs when to code
  2. by realizing that unit testing is not the same as TDD
  3. by writings TDD tests to verify a feature/story, not a functional unit

The conflation of unit testing with test-driven development is the source of much anguish and woe. So let's review it once more:

  • unit testing is concerned with verifying each individual module and function in the implementation; in UT you'll see an emphasis on things like code coverage metrics and tests that execute very quickly
  • test-driven development is concerned with verifying each feature/story in the requirements; in TDD you'll see an emphasis on things like writing the test first, ensuring that the code written does not exceed the intended scope, and refactoring for quality

In summary: unit testing has an implementation focus, TDD has a requirements focus. They are not the same thing.

share|improve this answer
    
"TDD has a requirements focus" I totally disagree with that. The tests your write in TDD are unit tests. They do verify each function/method. TDD does have an emphasis on code coverage and does care about tests that execute quickly (and they'd better do, since you run the tests every 30 seconds or so). Maybe you were thinking ATDD or BDD ? –  guillaume31 Feb 9 '12 at 17:35
1  
@ian31: perfect example of UT and TDD conflation. Must disagree, and refer you to some source material en.wikipedia.org/wiki/Test-driven_development - the purpose of the tests is to define code requirements. BDD is great. Never heard of ATDD, but at a glance it does look like how I apply TDD scaling. –  Steven A. Lowe Feb 9 '12 at 20:51
    
You can perfectly well use TDD to design technical code that is not directly related to a requirement or user story. You'll find countless examples of that on the web, in books, conferences, including from the people who initiated TDD and popularized it. TDD is a discipline, a technique for writing code, it won't stop being TDD depending on the context in which you use it. –  guillaume31 Feb 10 '12 at 10:52
    
Also, from the Wikipedia article you mentioned : "Advanced practices of test-driven development can lead to ATDD where the criteria specified by the customer are automated into acceptance tests, which then drive the traditional unit test-driven development (UTDD) process. [...] With ATDD, the development team now has a specific target to satisfy, the acceptance tests, which keeps them continuously focused on what the customer really wants from that user story." Which seems to imply ATDD is primarily focused on requirements, not TDD (or UTDD as they put it). –  guillaume31 Feb 10 '12 at 10:56
    
@ian31: The OP's question about 'throwing out several hundred unit tests' indicated a confusion of scale. You can use TDD to build a shed if you like. :D –  Steven A. Lowe Feb 10 '12 at 17:42
add comment

Test-driven development is meant to drive your development. The tests you write help you assert the correctness of the code you're currently writing and increase the development speed from the first line onward.

You seem to believe the tests are a burden and only meant for incremental development later on. This line of thinking is not in line with TDD.

Maybe you can compare it with static typing: although one can write code using no static type information, adding static type to code helps in asserting certain properties of the code, freeing the mind and allowing focus on important structure instead, thus increasing the velocity and efficacy.

share|improve this answer
add comment

I'm not sure about the reason why your approach turned out flawed after 3 days. Depending on your uncertainties in your architecture, you could consider changing your testing strategy:

  • If you are uncertain about performance, You might want to start with a few integration tests that assert performance?

  • When API complexity is what you are investigating, write some real bare, small unit tests to figure out what would be the best way to go about it. Don't bother implementing anything, just make your classes return hard coded values or make them throw NotImplementedExceptions.

share|improve this answer
add comment

The problem with doing a major refactoring is that you can and will sometimes follow a path that leads you to realize that you've bitten off more than you can chew. Giant refactorings are a mistake. If the system design is flawed in the first place, then refactoring may only take you so far before you need to make a hard decision. Either leave the system as it is and work around it, or plan to redesign and make some major changes.

There is however another way. The real benefit of refactoring code is to make things simpler, easier to read, and even easier to maintain. Where you approach a problem that you have uncertainty about, you spike a change, go so far to see where it might lead in order to learn more about the problem, then throw away the spike, and apply a new refactoring based on what the spike taught you. The thing is, you can really only improve your code with certainty if the steps are small and your refactoring efforts don't overrun your ability to write your tests first. The temptation is to write a test, then code, then code some more because a solution may seem obvious, but soon you realise that your change will change many more tests, so you need to be careful to only change one thing at a time.

The answer therefore is to never make your refactoring a major one. Baby steps. Start by extracting methods, then look to removing duplication. Then move to extracting classes. Each in tiny steps one minor change at a time. IF you're extracting code, write a test first. If you are removing code, then remove it and run your tests, and decide if any of the broken tests will be needed any more. One tiny baby step at a time. It seems like it will take longer, but will actually shorten your refactoring time considerably.

The reality is however, that every spike is seemingly a potential waste of effort. Code changes sometimes go nowhere, and you find yourself restoring your code from your vcs. This is just a reality of what we do from day to day. Every spike that fails is not wasted however, if it teaches you something. Every refactoring effort that fails will teach you that you are either trying to do too much too quickly, or that your approach may be wrong. That too is not a waste of time if you learn something from it. The more you do this stuff, the more you learn and the more efficient you will become at it. My advice is to just wear it for now, learn to do more by doing less, and accept that this is just the way things probably need to be until you get better at identifying how far to take a spike before it leads you nowhere.

share|improve this answer
add comment

For me unit tests are also as an occasion to put the interface under "real" use (well, as real as unit tests go!).

If I am forced to set up a test I have to exercise my design. This helps to keep things sane (if something is so complex that writing a test for it is a burden, what will it be like to use it?).

This doesn't avoid changes in the design, rather it exposes the need for them. Yes, a complete rewrite is a pain. To (try to) avoid it I usually set up (one or more) prototype, possibly in Python (with the final development in c++).

Granted, you don't always have the time for all these goodies. Those are precisely the cases when you will need a LARGER amount of time to achieve your goals...and/or to keep everything under control.

share|improve this answer
add comment

Welcome to creative developers circus.


Instead of respecting all 'legal/reasonable' way to code at the beginning,
try intuition, above all if it is important and new for you and if no sample around looks like you want:

- Write with your instinct, from things you already know, not with your mental and imagination.
- And stop.
- Take a magnifying glass and inspect all words you write : You write "text" because "text" is near to String, but "verb", "adjective" or something more accurate is needed, read again and adjust method with new sense
... or, you wrote a piece of code thinking about future? remove it
- Correct, do other task (sport, culture or other things outside business), come back and read again.
- All fit well, pass to UML
- Correct, do other task, come back and read again.
- All fit well, pass to TDD
- Now all is correct, good
- Try benchmark to point out things to be optimized, do it.

What's appear :
- you wrote a code respecting all rules
- you get an experience, a new way for work,
- something change in your mind, you'll never afraid by new configuration.

And now, if you see an UML looking like the above, you will be able to say
"Boss, I begin by TDD for this...."
it is another new thing ?
"Boss, I would try something before decide the way I'll code.."

Best regards from PARIS
Claude

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.