Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

In my current project (a game, in C++), I decided that I would use Test Driven Development 100% during development.

In terms of code quality, this has been great. My code has never been so well designed or so bug-free. I don't cringe when viewing code I wrote a year ago at the start of the project, and I have gained a much better sense for how to structure things, not only to be more easily testable, but to be simpler to implement and use.

However... it has been a year since I started the project. Granted, I can only work on it in my spare time, but TDD is still slowing me down considerably compared to what I'm used to. I read that the slower development speed gets better over time, and I definitely do think up tests a lot more easily than I used to, but I've been at it for a year now and I'm still working at a snail's pace.

Each time I think about the next step that needs work, I have to stop every time and think about how I would write a test for it, to allow me to write the actual code. I'll sometimes get stuck for hours, knowing exactly what code I want to write, but not knowing how to break it down finely enough to fully cover it with tests. Other times, I'll quickly think up a dozen tests, and spend an hour writing tests to cover a tiny piece of real code that would have otherwise taken a few minutes to write.

Or, after finishing the 50th test to cover a particular entity in the game and all aspects of it's creation and usage, I look at my to-do list and see the next entity to be coded, and cringe in horror at the thought of writing another 50 similar tests to get it implemented.

It's gotten to the point that, looking over the progress of the last year, I'm considering abandoning TDD for the sake of "getting the damn project finished". However, giving up the code quality that came with it is not something I'm looking forward to. I'm afraid that if I stop writing tests, then I'll slip out of the habit of making the code so modular and testable.

Am I perhaps doing something wrong to still be so slow at this? Are there alternatives that speed up productivity without completely losing the benefits? TAD? Less test coverage? How do other people survive TDD without killing all productivity and motivation?

share|improve this question

migrated from stackoverflow.com Jun 23 '11 at 23:28

This question came from our site for professional and enthusiast programmers.

    
@Nairou: You could always try "getting the project finished"! Make a branch now. Just write the code in there. But limit what you do, either by time or number of game entities and see if you went faster. You can then ignore that branch, go back to trunk and TDD from there and see what the difference is. –  quamrana Jun 22 '11 at 11:00
4  
To me, writing tests too early is like optimizing too early. You might be working hard on testing code you will remove in the future anyway. –  LennyProgrammers Jun 28 '11 at 13:49
24  
TDD is like sex. If you don't like it, you're doing it wrong. –  Falcon Jun 28 '11 at 15:28
1  
Back when I was learning, we had a trick for when we had to provide design documents. We wrote the code first, then wrote the documents to describe the code. Maybe you need to learn a moderate amount of that pragmatism for your TDD. If you already have a plan in mind, maybe it's better to get most of that into code before writing the tests. Whatever idealism suggests, sometimes it's better to do what you're already ready to do, rather than distract yourself with something else then come back when you're not fresh any more. –  Steve314 Jul 8 '11 at 0:22
2  
I'm going to go against the popular opinion and say that TDD may not always be the right choice if you're making games. Since someone at gamedev.stackexchange already answered this question spectacularly, I'll just link this here. –  l46kok Oct 28 '13 at 6:39

10 Answers 10

TDD is still slowing me down considerably

That's actually false.

Without TDD, you spend a few weeks writing code which mostly works and spend the next year "testing" and fixing many (but not all) of the bugs.

With TDD, you spend a year writing code which actually works. Then you do final integration testing for a few weeks.

The elapsed time is probably going to be the same. The TDD software will be substantially better quality.

share|improve this answer
5  
So why do I need TDD? "The elapsed time is the same" –  Peter Long Jun 22 '11 at 2:09
18  
@Peter Long: Code quality. The year "testing" is how we wind up with crap software that mostly works. –  S.Lott Jun 22 '11 at 2:10
1  
@Peter, you've got to be kidding. The quality of the TDD solution will be far superior. –  Mark Thomas Jun 22 '11 at 2:12
5  
Why do I need TDD? Kent Beck lists peace of mind as a big one, and it's very compelling to me. I am living in constant fear of breaking stuff when I work on code with no unit tests. –  Mathias Jun 22 '11 at 5:20
5  
@Peter Long: "The elapsed time is the same"... and at any point during that time you can deliver working code. –  Frank Shearar Jun 24 '11 at 16:34

Let me begin by thanking you to share your experience and voicing out your concerns... which I have to say are not uncommon.

  • Time/Productivity: Writing tests is slower than not writing tests. If you scope it to that, I'd agree. However if you run a parallel effort where you apply a non-TDD approach, chances are that the time you spend break-detect-debug-and-fix existing code will put you in the net negative. For me, TDD is the fastest I can go without compromising on my code-confidence. If you find things in your method, that are not adding value, eliminate them.
  • Number of tests: If you code up N things, you need to test N things. to paraphrase one of Kent Beck's lines "Test only if you would want it to work."
  • Getting stuck for hours: I do too (sometimes and not > 20 mins before I stop the line).. It's just your code telling you that the design needs some work. A test is just another client for your SUT class. If a test is finding it difficult to use your type, chances are so will your production clients.
  • Similar tests tedium : This needs some more context for me to write up a counterargument. That said, Stop and think about the similarity. Can you data-drive those tests somehow? Is it possible to write tests against a base-type? Then you just need to run the same set of tests against each derivation. Listen to your tests. Be the right kind of lazy and see if you can figure out a way to avoid tedium.
  • Stopping to think about what you need to do next (the test/spec) isn't a bad thing. On the contrary, it's recommended so that you build "the right thing". Usually if I can't think of how to test it, I usually can't think of the implementation either. It's a good idea to blank out implementation ideas till you get there.. maybe a simpler solution is overshadowed by a YAGNI-ish pre-emptive design.

And that brings me to the final query : How do I get better ? My (or An) answer is Read, Reflect and Practice.

e.g. Of late, I keep tabs on

  • whether my rhythm reflects RG[Ref]RG[Ref]RG[Ref] or is it RRRRGRRef.
  • % time spent in the Red / Compile Error state.
  • Am I stuck in a Red/Broken builds state?
share|improve this answer
1  
I'm very curious about your comment on data-driving the tests. Do you just mean a single set of tests that process external data (like from a file) rather than retesting similar code? In my game I have multiple entities, and each one is largely different, but there are some common things that get done with them (serializing them over the network, making sure they don't get sent to non-existing players, etc.) So far I haven't found a way to consolidate this, and so have sets of tests for each one that are almost identical, different only in what entity they use and what data it contains. –  Nairou Jun 22 '11 at 12:25
    
@Nairoi - Not sure what test runner you are using. I just learned a name for what I wanted to convey. Abstract fixture pattern[goo.gl/dWp3k]. This still requires you to write as many Fixtures as there are concrete SUT types. If you want to be even more concise, look at your runner's docs. e.g. NUnit supports Parameterized and Generic test fixtures (now that I searched for it) goo.gl/c1eEQ Seems like the very thing you need. –  Gishu Jun 22 '11 at 13:39
    
Interesting, I've never heard of abstract fixtures. I currently use UnitTest++ which has fixtures, but not abstract ones. It's fixtures are very literal, just a way of consolidating test code that you would otherwise repeat in each test, for a given group of tests. –  Nairou Jun 23 '11 at 2:20
    
Great answer. Can you fix the link to goo.gl/dWp3k –  asgeo1 Jun 24 '11 at 1:07
    
@asgeo - can't edit that comment.. the link has picked up a trailing bracket.This should work - goo.gl/dWp3k –  Gishu Jun 24 '11 at 2:34

I agree with the other answers, but I also want to add a very important point: Refactoring costs!!

With well written unit tests, you can safely do re-writes of your code. First, well written unit tests provide excellent documentation of the intent of your code. Second, any unfortunate side effects of your refactoring will be caught in the existing test suite. Thus, you have guaranteed that the assumptions of your old code are true for your new code as well.

share|improve this answer

Or, after finishing the 50th test to cover a particular entity in the game and all aspects of it's creation and usage, I look at my to-do list and see the next entity to be coded, and cringe in horror at the thought of writing another 50 similar tests to get it implemented.

This makes me wonder how much you're following the "Refactor" step of TDD.

When all your tests are passing, this is time for you to refactor the code and remove duplication. While people usually remember this, sometimes they forget that this is also time to refactor their tests as well, to remove duplication and simplify things.

If you have two entities that merge into one to enable code re-use, consider merging their tests as well. You really only need to test incremental differences in your code. If you're not regularly performing maintenance on your tests, they can quickly become unwieldy.

A couple of philosophical points about TDD that might be helpful:

  • When you can't figure out how to write a test, despite extensive experience writing tests, then that's definitely a code smell. Your code is somehow lacking in modularity, which makes writing small, simple tests difficult.
  • Spiking out a bit of code is perfectly acceptable when using TDD. Write the code you want to, to get an idea of what it looks like, then delete the code and start with tests.
  • I see practicing extremely strict TDD as a form of exercise. When you first start out, definitely write a test first every single time, and write the simplest code to make the test pass before moving on. However, this is not necessary once you become more comfortable with the practice. I don't have a unit test for every single possible code path I write, but through experience I'm able to choose what needs to be tested with a unit test, and what can be covered by functional or integration testing instead. If you've been practicing TDD in a strict fashion for a year, I would imagine you're close to this point as well.

EDIT: On the topic of philosophy of unit testing, I think this might be interesting for you to read: The Way of Testivus

And a more practical, if not necessarily very helpful, point:

  • You mention C++ as your development language. I've practiced TDD extensively in Java, using excellent libraries like JUnit and Mockito. However, I've found TDD in C++ to be very frustrating due to the lack of libraries (in particular, mocking frameworks) available. While this point doesn't help you in your current situation much, I hope you take it into consideration before ditching TDD altogether.
share|improve this answer
3  
Refactoring tests is dangerous. Nobody seems to talk about this, but it is. I certainly don't have unit tests to test my unit tests. When you refactor to reduce duplication, you typically increase complexity (because your code becomes more general). That means there's more likely to be a bug in your tests. –  Scott Whitlock Jun 24 '11 at 1:17
1  
I disagree that refactoring tests is dangerous. You are only refactoring when everything is passing, so if you do a refactoring and everything is still green, then you're just fine. If you're thinking you need to write tests for your tests, I feel like that's an indicator that you need to write simpler tests. –  jaustin Jun 24 '11 at 1:56
1  
C++ is hard to unit test (the language doesn't easily do things that make mocks easy). I've noticed that functions which are "functions" (only operate on arguments, results come out as return values/params) are much easier to test than "procedures" (return void, no args). I've found that it can actually be easier to unit test well-crafted modular C code than C++ code. You don't have to write in C, but you can follow the modular C example. It sounds completely insane, but I've put unit tests on "bad C" where everything was global and it was super easy- all the state is always available!. –  anon Jun 24 '11 at 2:58
1  
I think this is very true. I do a lot of RedGreenRedGreenRedGreen (or, more often, RedRedRedGreenGreenGreen), but I rarely refactor. My tests certainly have never been refactored, as I always felt it would waste even more time not coding. But I can see how it might be the cause of the issues I'm now facing. Time for me to seriously look into doing some refactoring and consolidation. –  Nairou Jun 25 '11 at 16:25
1  
Google C++ mocking framework (integrated with google C++ test fw) - very, very powerfull mocking library - flexible, feature rich - quite comparable with any other mocking framework out there. –  ratkok Jun 25 '11 at 18:25

You don't need 100% percent test coverage. Be pragmatic.

share|improve this answer
    
If you don't have 100% test coverage, you don't have 100% confidence. –  Christopher Mahan Jun 24 '11 at 23:32
42  
You don't have 100% confidence even with 100% of test coverage. That's testing 101. Tests cannot demonstrate that the code is defect-free; on the contrary, they can only demonstrate that it does contain defects. –  CesarGon Jun 25 '11 at 0:40
6  
For what it's worth, one of the most passionate TDD advocates, Bob Martin, does not recommend 100% coverage - blog.objectmentor.com/articles/2009/01/31/…. In the manufacturing industry (granted, different from software in many respects), no one goes for 100% confidence because they can spend a fraction of the effort to be 99% confident. –  Chance Jul 4 '11 at 20:26

How do other people survive TDD without killing all productivity and motivation?

This is completely different from my experiences. You are either amazingly intelligent and write code without bugs, (eg. off by one errors) or you don't realize your code has bugs which prevent your program working, and so aren't actually finished.

TDD is about having the humility to know that you (and me!) make mistakes.

For me the time writing unittests is more than saved in reduced debugging time for projects that are done using TDD from the beginning.

If you don't make mistakes then maybe TDD isn't as important for you as it is for me!

share|improve this answer
    
Well, you have bugs in your TDD code as well ;) –  Coder Jun 28 '11 at 13:53
    
That true! but they do tend to be a different type of bug, if TDD is done properly. I guess saying code has to be 100% bug free to be finished isn't right. Although if one defines a bug as a deviation from the unit test defined behavior, then I guess it is bug free :) –  Tom Jul 4 '11 at 20:02

Yes, writing tests and code might take longer than just writing code - but writing code and associated unit tests (using TDD) is much more predictable than writing code and then debugging it.

Debugging is almost eliminated when using TDD - which makes all development process much more predictable and in the end - arguably faster.

Constant refactoring - it is impossible to do any serious refactoring without comprehensive unit test suite. The most efficient way to build that unit testing based safety net is during TDD. Well refactored code significantly improves overall productivity of the designer/team who maintains the code.

share|improve this answer

Very interesting question.

What's important to note, is that C++ is not very easily testable, and gaming, in general, is also a very bad candidate for TDD. You can't test if OpenGL/DirectX draws triangle red with driver X, and yellow with driver Y easily. If the bump map normal vector is not flipped after shader transforms. Nor you can test clipping issues over driver versions with different precisions, and so on. Undefined drawing behavior because of incorrect calls can also be tested only with accurate code review and SDK at hand. Sound is also a bad candidate. Multithreading, which again is quite important for games, is pretty much useless to unit-test. So it's tough.

Basically gaming is a lot of GUI, sound and threads. GUI, even with standard components you can send WM_ to, is tough to unit-test.

So what you can test, is model loading classes, texture loading classes, matrix libraries and some-such, which is not a lot of code, and, quite often, not very reusable, if it's only your first project. Also, they are packed into proprietary formats, so it's not quite likely that 3rd party input can differ much, unless you release modding tools etc.

Then again, I'm not TDD guru or evangelist, so take all that with a grain of salt.

I would probably write some tests for main core components (for example matrix library, image library). Add a bunch of abort() on unexpected inputs in every function. And most importantly, concentrate on resistant/resilient code that doesn't break easily.

Concerning off by one errors, clever use of C++, RAII and a good design goes a long way to prevent them.

Basically you have a lot to do just to cover the basics if you want to release the game. I'm not sure if TDD will help.

share|improve this answer
3  
+1 I really like the concept of TDD and use it wherever I can, but you raise a very important point that TDD advocates are curiously silent on. There are lots of types of programming as you have pointed out for which writing meaningful unit tests is extremely hard if not impossible. Use TDD where it makes sense, but some types of code are better developed and tested in other ways. –  Mark Heath Jul 8 '11 at 9:58
    
@Mark: yup, no-one seems to care about integration tests nowadays, thinking that becuase they've got an automated test suite, everything wil magically work when put together and tried out with real data. –  gbjbaanb Jul 8 '11 at 10:24

I've only a few remarks:

  1. It seems you're trying to test everything. You probably shouldn't, just the high-risk and edge cases of a particular piece of code / method. I'm pretty sure the 80 / 20 rule applies here: You spend 80% writing tests for the last 20% of your code or cases that aren't covered.

  2. Prioritize. Get into agile software development, and make a list of what you really really really need to do to release in one month time. Then release, just like that. This'll make you think about the priority of features. Yeah, it'd be cool if your character could do a backflip, but does it have business value?

TDD is good, but only if you don't aim for 100% test coverage, and it's not if it keeps you from producing actual business value (i.e. features, things that add something to your game).

share|improve this answer

Consider narrowing the scope of your game and get it where someone can play it or you release it. Maintain your testing standards without having to wait too long to release your game might be a middle-ground to keep you motivated. The feedback from your users may provide benefits in the long-term and your testing allows you to feel comfortable with the additions and changes.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.