Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

When doing TDD and writing a unit test, how does one resist the urge to "cheat" when writing the first iteration of "implementation" code that you're testing?

For example:
Let's I need to calculate the Factorial of a number. I start with a unit test (using MSTest) something like:

[TestClass]
public class CalculateFactorialTests
{
    [TestMethod]
    public void CalculateFactorial_5_input_returns_120()
    {
        // Arrange
        var myMath = new MyMath();
        // Act
        long output = myMath.CalculateFactorial(5);
        // Assert
        Assert.AreEqual(120, output);
    }
}

I run this code, and it fails since the CalculateFactorial method doesn't even exist. So, I now write the first iteration of the code to implement the method under test, writing the minimum code required to pass the test.

The thing is, I'm continually tempted to write the following:

public class MyMath
{
    public long CalculateFactorial(long input)
    {
        return 120;
    }
}

This is, technically, correct in that it really is the minimum code required to make that specific test pass (go green), although it's clearly a "cheat" since it really doesn't even attempt to perform the function of calculating a factorial. Of course, now the refactoring part becomes an exercise in "writing the correct functionality" rather than a true refactoring of the implementation. Obviously, adding additional tests with different parameters will fail and force a refactoring, but you have to start with that one test.

So, my question is, how do you get that balance between "writing the minimum code to pass the test" whilst still keeping it functional and in the spirit of what you're actually trying to achieve?

share|improve this question
2  
It's a human thing: you have to resist the urge to cheat. There is nothing more to it. You could add more tests and write more test code than code to test, but if you don't have that luxury than you'll need to just resist. There are MANY places in coding where we have to resist the urge to hack or cheat, because we know that, while it might work today, it won't work later. –  Yar Nov 25 '10 at 16:17
3  
Surely, in TDD, doing it the other way around is cheating - i.e. return 120 is the proper way. I find it quite hard to make myself do that, and not race ahead and start writing the factorial calculation. –  Paul Butcher Nov 25 '10 at 16:32
2  
I would consider this a cheat, only because it may pass the test but it does not add any true functionality or get you closer to a final solution to the problem at hand. –  GrumpyMonkey Nov 25 '10 at 19:06
    
If it turns out that client code code only ever passes in a 5, returning 120 is not just a non-cheat, but is actually a legitimate solution. –  Kramii Nov 26 '10 at 17:41
    
I agree with @PaulButcher - in fact, a lot of unit testing examples in texts and articles would take this approach. –  HorusKol Aug 15 '11 at 1:31

11 Answers 11

up vote 26 down vote accepted

It's perfectly legit. Red, Green, Refactor.

The first test passes.

Add the second test, with a new input.

Now quickly get to green, you could add an if-else, which works fine. It passes, but you are not done yet.

The third part of Red, Green, Refactor is the most important. Refactor to remove duplication. You WILL have duplication in your code now. Two statements returning integers. And the only way to remove that duplication is to code the function correctly.

EDIT: I'm not saying don't write it correctly the first time. I'm just saying it's not cheating if you don't.

share|improve this answer
6  
This just raises the question, why not just write the function correctly in the first place? –  Robert Harvey Nov 25 '10 at 16:37
4  
@Robert, factorial numbers are trivially simple. The real advantage of TDD is when you write non-trivial libraries, and writing the test first forces you to design the API before the implementation, which - in my experience - leads to better code. –  user1249 Nov 25 '10 at 16:40
1  
@Robert, it is you who are concerned about solving the problem instead of passing the test. I am telling you that for non-trivial problems it simply works better to postpone the hard design until you have tests in place. –  user1249 Nov 25 '10 at 16:53
1  
@Thorbjørn Ravn Andersen, no, I'm not saying you can only have one return. There are valid reasons for multiple (ie, guard statements). The issue is, both return statements were "equal". They did the same 'thing'. They just happened to have different values. TDD isn't about rigidity, and adhering to a specific size of test/code ratio. It's about creating a comfort level within your code base. If you can write a failing test, then a function that will work for future tests of that function, great. Do it, then write your edge case tests ensuring your function still works. –  CaffGeek Nov 25 '10 at 17:00
1  
@Thorbjørn: With respect to factorials being simple, are you saying that the OP is oversimplifying with respect to his example? –  Robert Harvey Nov 25 '10 at 17:46

Clearly an understanding of the ultimate goal, and the achievement of an algorithm that meets that goal, is required.

TDD is not a magic bullet for design; you still have to know how to solve problems using code, and you still have to know how to do that at a level higher than a few lines of code to make a test pass.

I like the idea of TDD because it encourages good design; it makes you think about how you can write your code so that it is testable, and generally that philosophy will push the code towards a better design overall. But you still have to know how to architect a solution.

I do not favor reductionist TDD philosophies that claim you can grow an application by simply writing the smallest amount of code to pass a test. Without thinking about architecture, this won't work, and your example proves that.

Uncle Bob Martin says this:

If you're not doing Test Driven Development, it's very difficult to call yourself a professional. Jim Coplin called me on the carpet for this one. He didn't like that I said that. In fact, his position right now is that Test Driven Development is destroying architectures because people are writing tests to the abandon of any other kind of thought and tearing their architectures apart in the mad rush to get tests to pass and he's got an interesting point, that's an interesting way to abuse the ritual and lose the intent behind the discipline.

if you're not thinking through the architecture, if what you're doing instead is ignoring architecture and throwing tests together and getting them to pass, you're destroying the thing that will allow the building to stay up because it's the concentration on the structure of the system and solid design decisions that help the system maintain its structural integrity.

You cannot simply just throw a whole bunch of tests together and make them pass for decade after decade after decade and assume that you're system is going to survive. We don't want to evolve ourselves into hell. So a good test driven developer is always conscious of making architectural decisions, always thinking of the big picture.

share|improve this answer
    
Not really an answer to the question, but 1+ –  rmx Nov 25 '10 at 16:29
    
@rmx: Um, the question is: How do you get that balance between "writing the minimum code to pass the test" whilst still keeping it functional and in the spirit of what you're actually trying to achieve? Are we reading the same question? –  Robert Harvey Nov 25 '10 at 16:33
    
The ideal solution is an algorithm and has nothing to do with architecture. Doing TDD won't make you invent algorithms. At some point you need to make steps in terms of an algorithm/solution. –  Joppe Nov 25 '10 at 23:21
    
I agree with @rmx. This doesn't really answer my specific question, per se, but it does give rise to food for thought as to how TDD in general fits into the big picture of the overall software development process. So, for that reason, +1. –  CraigTP Nov 26 '10 at 9:11
    
I think you could substitute "algorithms" - and other terms - for "architecture" and the argument still holds; it's all about being unable to see the wood for the trees. Unless you are going to write a separate test for every single integer input, TDD will not be able to distinguish between a proper factorial implementation and some perverse hard-coding which works for all tested cases but not others. The problem with TDD is the ease with which "all tests pass" and "the code is good" are conflated. At some point a heavy measure of common sense needs to be applied. –  Julia Hayward Jan 16 at 9:51

A very good question...and I have to disagree with almost everyone except @Robert.

Writing

return 120;

for a factorial function to make one test pass is a waste of time. It's not "cheating", nor is it following red-green-refactor literally. It is wrong.

Here's why:

  • Calculate Factorial is the feature, not "return a constant". "return 120" is not a calculation.
  • the 'refactor' arguments are misguided; if you have two test cases for 5 and 6, this code is still wrong, because you are not calculating a factorial at all:

    if (input == 5) { return 120; } //input=5 case
    else { return 720; }   //input=6 case
    
  • if we follow the 'refactor' argument literally, then when we have 5 test cases we would invoke YAGNI and implement the function using a lookup table:

    if (factorialDictionary.Contains(input)) {
        return factorialDictionary[input]; 
    }
    throw new Exception("Input failure");
    

None of these are actually calculating anything, you are. And that's not the task!

share|improve this answer
    
I think the oposing argument was based on the BDD principle (which you seem to have missed) that unit tests encode requirements. Therefore if some code passes the tests, it meets requirements and is valid. –  rmx Nov 26 '10 at 16:48
1  
@rmx: nope, didn't miss it; "refactor to remove duplication" can be satisfied with a lookup table. BTW the principle that unit tests encode requirements is not specific to BDD, it is a general principle of Agile/XP. If the requirement was "Answer the question 'what is the factorial of 5'" then 'return 120;' would be legit ;-) –  Steven A. Lowe Nov 26 '10 at 16:54
    
@Steven A.Lowe, you refactor to remove code smells. Constants are code smells, hence, you would remove the constants, and you wouldn't replace them with a lookup table, because that lookup table is just another constant –  CaffGeek Nov 26 '10 at 17:12
1  
@Chad all of which is unnecessary work - just write the function the first time ;-) –  Steven A. Lowe Nov 26 '10 at 17:14
    
@Steven A.Lowe, by that logic, why write any tests?! "Just write the application the first time!" The point of TDD, is small, safe, incremental changes. –  CaffGeek Nov 26 '10 at 17:19

When you've written only one unit test, the one-line implementation (return 120;) is legitimate. Writing a loop calculating the value of 120 - that would be cheating!

Such simple initial tests are a good way to catch edge cases and prevent one-off errors. Five actually isn't the input value I'd start with.

A rule of thumb that could be useful here is: zero, one, many, lots. Zero and one are important edge cases for the factorial. They can be implemented with one-liners. The "many" test case (e.g. 5!) would then force you to write a loop. The "lots" (1000!?) test case could force you to implement an alternative algorithm to handle very large numbers.

Edit: fixed some typos

share|improve this answer

If you are able to "cheat" in such a way, it suggests that your unit tests are flawed.

Rather than testing the factorial method with a single value, test it was a range of values. Data-driven testing can help here.

View your unit tests as a manifestation of the requirements - they must collectively define the behaviour of the method which they test. (This is known as behaviour driven development - its the future ;-))

So ask yourself - if someone were to change the implementation to something incorrect, would your tests still pass or would they say "hang on a minute!"?

Bearing that in mind, if your only test was the one in your question, then technically, the corresponding implementation is correct. The problem is then viewed as poorly-defined requirements.

share|improve this answer
    
As nanda pointed out, you can always add an endless series of case statements to a switch, and you can't write a test for every possible input and output for the OP's example. –  Robert Harvey Nov 25 '10 at 16:03
    
You could technically test values from Int64.MinValue to Int64.MaxValue. It would take a long time to run but it would explicitly define the requirement with no room for error. With the current technology, this is unfeasable (I suspect that it might become more common in the future) and I agree, you could cheat but I think the OPs question was not a practical one (no-one would actually cheat in such a way in practice), but a theoretical one. –  rmx Nov 25 '10 at 16:09
    
@rmx: If you could do that, the tests would be the algorithm, and you would no longer need to write the algorithm. –  Robert Harvey Nov 25 '10 at 16:16
    
It's true. My university thesis actually involves the automatic generation of the implementation using the unit-tests as a guide with a genetic algorithm as an aid to TDD - and it is only possible with solid tests. The difference is that binding your requirements to your code typically are far more difficult to read and grasp than a single method that embodies the unit-tests. Then comes the question: if your implementation is a manifestation of your unit tests, and your unit-tests are a manifestation of your requirements, why not just skip testing altogether? I dont have an answer. –  rmx Nov 25 '10 at 16:24
    
Also, are'nt we, as humans, just as likely to make a mistake in unit-tests as we are in the implementation code? So why unit test at all? –  rmx Nov 25 '10 at 16:25

As long as you only have a single test, then the minimal code needed to pass the test is truly return 120;, and you can easily keep it for that as long as you don't have any more tests.

This allows you to postpone further design until you actually write the tests that exercise OTHER return values of this method.

Please remember that the test is the runnable version of your specification, and if all that specification says is that f(6)=120 then that fits the bill perfectly.

share|improve this answer
    
Seriously? By this logic, you will have to rewrite the code every time someone comes up with a new input. –  Robert Harvey Nov 25 '10 at 16:04
6  
@Robert, at SOME point adding a new case will not result in the simplest possible code anymore, at which point you write a new implementation. As you have the tests in place already, you know exactly when your new implementation does the same as the old one. –  user1249 Nov 25 '10 at 16:09
1  
@Thorbjørn Ravn Andersen, exactly, the most important part of Red-Green-Refactor, is the refactoring. –  CaffGeek Nov 25 '10 at 16:17
    
+1: This is the general idea from my knowledge as well, but something needs to be said about fulfilling the implied contract (ie. the method name factorial). If you only ever spec (ie test) f(6) = 120 then you only ever need to 'return 120'. Once you start adding tests to ensure that f(x) == x*x-1...*x-x-1 : upperBound >= x >= 0 then you will arrive at a function that satisfies the factorial equation. –  Steve Evers Nov 25 '10 at 16:21
1  
@SnOrfus, the place for "implied contracts" to be is in the test cases. If you contract is for factorials, you TEST if known factorials are and if known non-factorials aren't. Plenty of them. Doesn't take long to convert the list of the ten first factorials to a for-loop testing every number up to the tenth factorial. –  user1249 Nov 25 '10 at 16:31

Writing "cheat" tests is OK, for sufficiently small values of "OK". But recall - unit testing is only complete when all tests pass and no new tests can be written that will fail. If you really want to have a CalculateFactorial method that contains a bunch of if statements (or even better, a big switch/case statement :-) you can do that, and since you're dealing with a fixed-precision number the code required to implement this is finite (although probably rather large and ugly, and perhaps limited by compiler or system limitations on the maximum size of a procedure's code). At this point if you really insist that all development has to be driven by a unit test you can write a test that requires the code to compute the result in an amount of time shorter than that which can be accomplished by following all the branches of the if statement.

Basically, TDD can help you write code which implements requirements correctly, but it can't force you to write good code. That's up to you.

Share and enjoy.

share|improve this answer
    
+1 for "unit testing is only complete when all tests pass and no new tests can be written that will fail" Many people are saying its legit to return the constant, but don't follow with "for the short term", or "if the overall requirements only need those specific cases" –  Thymine May 10 '12 at 18:45

Just write more tests. Eventually, it would be shorter to write

public long CalculateFactorial(long input)
{
    return input <= 1 ? 1 : CalculateFactorial(input-1)*input;
}

than

public long CalculateFactorial(long input)
{
    switch (input) {
       case 0: return 1;
       case 1: return 1;
       case 2: return 2;
       case 3: return 6;
       case 4: return 24;
       case 5: return 120;
    }
}

:-)

share|improve this answer
1  
Wh not just write the algorithm correctly in the first place? –  Robert Harvey Nov 25 '10 at 16:38
1  
@Robert, it is the correct algorithm for calculating factorial of a number from 0 to 5. Besides, what does "correctly" mean? This is a very simple example, but when it goes more complex, there become many gradations of what "correct" means. Is a program that requires root access "correct" enough? Is using XML "correct", instead of using CSV? You can't answer this. Any algorithm is correct as long as it satisfies some business requirements, which are formulated as tests in TDD. –  Pavel Shved Nov 25 '10 at 16:57
1  
It should be noted that since the output type is long, there is only a small number of input values (20 or so) that the function can possibly handle correctly, therefore a large switch statement is not necessarily the worst implementation - if speed is more important than code size, the switch statement might be the way to go, depending on your priorities. –  user281377 Nov 25 '10 at 17:06

I do 100% agree with Robert Harveys suggestion here, it's not just about making tests pass, you need to keep the overall goal in mind too.

As a solution to your painpoint of "it's only verified to work with a given set of inputs" I'd propose using data driven tests, such as xunit theory. The power behind this concept is that it allows you to easily create Specifications of inputs to outputs.

For Factorials, a test would look like this:

    [Theory]
    [InlineData(0, 1)]
    [InlineData( 1, 1 )]
    [InlineData( 2, 2 )]
    [InlineData( 3, 6 )]
    [InlineData( 4, 24 )]
    public void Test_Factorial(int input, int expected)
    {
        int result = Factorial( input );
        Assert.Equal( result, expected);
    }

You could even implement a test-data provide (that returns IEnumerable<Tuple<xxx>>) and encode a mathematic invariant, such as repeatedly dividing by n will yield n-1).

I find this tp be a very powerful way of testing.

share|improve this answer

If you still able to cheat then the tests are not enough. Write more tests! For your example, I will try to add tests with input 1, -1, -1000, 0, 10, 200.

Nevertheless, if you're really commit to cheat you can write an endless if-then. In this case, nothing could help except code review. You would be soon caught on acceptance test (written by other person!)

The problem with unit tests is sometimes programmers look at them as unnecessary work. The correct way to see them is as tool for you to make the result of your work correct. So if you create an if-then, you know unconsciously that there are other cases to consider. This means you have to write another tests. And so on and so on until you realize the cheating is not working and it is better to just code the correct way. If you still feel that you are not finish, you are not finish.

share|improve this answer
1  
So it sounds like you are saying that merely writing just enough code for the test to pass (as TDD advocates) is not sufficient. You also have to keep sound software design principles in mind. I agree with you BTW. –  Robert Harvey Nov 25 '10 at 16:07

I would suggest that your choice of test is not the best test.

I would start off with:

factorial(1) as the first test,

factorial(0) as the second

factorial(-ve) as the third

and then continue on with non-trivial cases

and finish up with an overflow case.

share|improve this answer
    
What is -ve?? –  Robert Harvey Nov 27 '10 at 2:29
    
a negative value. –  Chris Cudmore Nov 28 '10 at 21:25

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.