Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

In an answer to another question, I suggested creating a randomized value for input to a specific method. In my experience this has been useful for making tests more readable and it lets you skip the "trivial phase" where you hardcode specific results in a method.

Here's the unit test code pasted from the other answer, for quick reference:

[Test]
public void FullHouseReturnsTrue()
{
    var roll = AnyFullHouse();

    Assert.That(sut.IsFullHouse(roll));
}
[Test]
public void StraightReturnsFalse()
{
    var roll = AnyStraight();

    Assert.That(sut.IsFullHouse(roll), Is.False);
}

A couple of comments in response suggested that this strategy would not work well, because the helper method would need to be tested also. In my experience, I've never had a need to test methods like this, since creating the corresponding production code also tests my test code.

Do AnyStraight() and AnyFullHouse() need to have their own unit tests? If so, how do you solve the chicken-and-egg problem that presents?

EDIT

Would I still want to create dedicated unit tests for the AnyFullHouse algorithm if I inlined the method?

[Test]
public void FullHouseReturnsTrue()
{
    var roll = listOfHardCodedFullHouseRolls[_random.Next(0, listOfHardCodedFullHouseRolls.Length)];

    Assert.That(sut.IsFullHouse(roll));
}
share|improve this question
1  
How would you effectively unit test the result of something with a non-deterministic outcome? (For what it's worth, I share your skepticism on the notion that unit test helper methods need to be tested) –  Erik Dietrich Feb 7 '13 at 6:40
1  
@ErikDietrich You would end up with something like Haskell's quick check. It is a little more involved than regular unit tests. You have to define invariants that are tested against random input. The library makes the random seed available so you can get repeatable results when necessary. –  stonemetal Feb 8 '13 at 19:15

2 Answers 2

Why randomize the values? Having helper fields of AnyStraight or AnyFullHouse with hardcoded values is just as readable, still provides the reduction in magic values, and reduces the chance of error in your randomization (as well as guaranteeing that your tests are repeatable).

In my experience, as soon as your tests are not repeatable, people make excuses as to why they fail. Actual bugs are ignored and your tests are quickly ignored for not catching the bugs. Don't start down that slope.

share|improve this answer
    
I believe tallseth's motivation for randomizing the values is to test multiple valid and invalid combinations. After a while you have tested [1,1,2,2,2], [1,2,2,2,1], [1,1,3,3,3], [1,1,4,4,4], ..., [5,5,5,6,6]. –  Kristof Claes Feb 7 '13 at 9:06
    
@Kristof Claes: yes, and Telastyn's criticism is fully valid: what do you do if suddenly one test fails, and when you run it again, it's successful? –  Michael Borgwardt Feb 7 '13 at 9:23
1  
@MichaelBorgwardt : Maybe you can pass the roll-value to the Assert? Like Assert.IsTrue(sut.IsFullHouse(roll), roll.ToString()); with NUnit. This way you know what values caused the test to fail. –  Kristof Claes Feb 7 '13 at 10:25
    
Certainly. One might also implement AnyFullhouse by having 3 hardcoded FullHouse values, and picking one at random. The worst case scenario is some random value hits a test case you haven't thought of, and the test failure is a valid one. The nunit RepeatAttribute has always allowed me to find such failures very quickly. –  tallseth Feb 8 '13 at 14:25
    
+1 for "people make excuses as to why they fail" though. I haven't experienced that, but I also didn't think about this problem on the team-psychology dimension. –  tallseth Feb 8 '13 at 14:27

Test suites like PHPUnit and JUnit, etc, are developed using the same TDD principles (and are usually tested against themselves).

Of course, this means that there is a minimal bootstrap suite required, but by taking an axiomatic approach means you can easily build up from simple assertions like "true === true" and "0 !== false".

In your case, you simply need to isolate the helpers and test them - it's not really a chicken and egg, more of an inception: you need to go deeper.

Alternatively, you could assume that the helpers are correct - but then that limits your assumed confidence in your tests, and therefore limits the value of them.

share|improve this answer
    
But it isn't a test framework that is being created here. The biggest reason to make AnyFullHouse a method is so I can give it an explanatory name. if it was inlined, would people react the same way? –  tallseth Feb 8 '13 at 14:30
    
Well, it kind of is a framework - you are putting in helper methods and extending the framework to your needs. If you don't confirm those helper methods independently, then you can't be confident in the tests that use them. –  HorusKol Feb 11 '13 at 0:17

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.