Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I am test-driving a method that is to generate a collection of data objects. I want to verify that the properties of the objects are being set correctly. Some of the properties will be set to the same thing; others will be set to a value which is dependent on their position in the collection. The natural way to do this seems to be with a loop. However, Roy Osherove strongly recommends against using logic in unit tests (Art of Unit Testing, 178). He says:

A test that contains logic is usually testing more than one thing at a time, whicfh isn't recommended, because the test is less readable and more fragile. But test logic also adds complexity that may contain a hidden bug.

Tests should, as a general rule, be a series of method calls with no control flows, not even try-catch, and with assert calls.

However, I can't see anything wrong with my design (how else do you generate a list of data objects, some of whose values are depended on where in the sequence they are?—can't exactly generate and test them separately). Is there something non-test-friendly with my design? Or am being too rigidly devoted to Osherove's teaching? Or is there some secret unit test magic I don't know about that circumvents this problem? (I'm writing in C#/VS2010/NUnit, but looking for language-agnostic answers if possible.)

share|improve this question
3  
I recommend not looping. If your test is that the third thing has its Bar set to Frob, then write a test to specifically check that the third thing's Bar is Frob. That's one test by itself, go straight to it, no loop. If your test is that you get a collection of 5 things, that's also one test. That's not to say you never have a loop (explicit or otherwise), it's just that you don't often need to. Also, treat Osherove's book as more guidelines than actual rules. –  Anthony Pegram May 7 '13 at 17:06
    
@AnthonyPegram Sets are unordered - Frob may sometimes be 3rd, may sometimes be 2nd. You can't rely on it, making a loop (or language feature like Python's in) necessary, if the test is "Frob was successfully added to an existing collection". –  Izkata May 7 '13 at 21:40
    
@Izbata, his question specifically mentions that ordering is important. His words: "others will be set to a value which is dependent on their position in the collection." There are plenty of collection types in C# (the language he references) that are insertion ordered. For that matter, you can also rely upon order with lists in Python, a language you mention. –  Anthony Pegram May 7 '13 at 22:56
    
Also, let's say you're testing a Reset method on a collection. You need to loop through the collection and check each item. Depending on the size of the collection, not testing it in a loop is ridiculous. Or let's say I'm testing something that is supposed to increment each item in a collection. You could set all items to the same value, call your increment, then check. That test sucks. You should set several of them to different values, call increment, and check that all the different values incremented correctly. Checking just one random item in the collection is leaving a lot to chance. –  iheanyi 2 days ago

2 Answers 2

up vote 7 down vote accepted

TL;DR:

  • Write the test
  • If the test does too much, the code may do too much too.
  • It may not be a unit test (but not a bad test).

The first thing for testing is about dogma being unhelpful. I enjoy reading The Way of Testivus which points out some issues with dogma in a lighthearted way.

Write the test that needs to be written.

If the the test needs to be written some way, write it that way. Attempting to force the test into some idealized test layout or not have it at all is not a good thing. Having a test today that tests it is better than having a "perfect" test some later day.

I will also point to the bit on the ugly test:

When the code is ugly, the tests may be ugly.

You don’t like to write ugly tests, but ugly code needs testing the most.

Don’t let ugly code stop you from writing tests, but let ugly code stop you from writing more of it.

These can be considered truisms to those who have been following for a long time... and they just become ingrained in the way of thinking and writing tests. For people that haven't been and are trying to get to that point, reminders can be helpful (I even find re-reading them helps me avoid getting locked into some dogma).


Do consider that when writing an ugly test, if the code it may be an indication that the code is trying to do too much too. If the code that you are testing is too complex to be properly exercised by writing a simple test, you might want to consider breaking the code into smaller parts that can be tested with the simpler tests. One should not write a unit test that does everything (it might not be a unit test then). Just as 'god objects' are bad, 'god unit tests' are bad too and should be indications to go back and look at the code again.

You should be able to exercise all the code with reasonable coverage through such simple tests. Tests that do more end to end testing that deal with larger questions ("I have this object, marshalled into xml, sent to the web service, through the rules, back out, and unmarshalled") is an excellent test - but it certainly isn't a unit test (and falls into the integration testing realm - even if it has mocked services that it calls and custom in memory databases to do the testing). It may still use the XUnit framework for testing, but the testing framework doesn't make it a unit test.

share|improve this answer

Don't try to test too many things at once. Each of the properties of each data object in the collection is too much for one test. Instead, I recommend:

  1. If the collection is fixed-length, write a unit test to validate length. If it is variable length, write several tests for lengths that will characterize its behavior (e.g. 0, 1, 3, 10). Either way, do not validate properties in these tests.
  2. Write a unit test to validate each of the properties. If the collection is fixed-length and short, just assert against one property of each of the elements for each test. If it is fixed-length but long, choose a representative but small sample of the elements to assert against one property each. If it is variable-length, generate a relatively short but representative collection (i.e. maybe three elements) and assert against one property of each.

Doing it this way makes the tests small enough that leaving out loops doesn't seem painful. C#/Unit example, given method under test ICollection<Foo> generateFoos(uint numberOfFoos):

[Test]
void generate_zero_foos_returns_empty_list() { ... }
void generate_one_foo_returns_list_of_one_foo() { ... }
void generate_three_foos_returns_list_of_three_foos() { ... }
void generated_foos_have_sequential_ID()
{
    var foos = generateFoos(3).GetEnumerable();
    foos.MoveNext();
    Assert.AreEqual("ID1", foos.Current.id);
    foos.MoveNext();
    Assert.AreEqual("ID2", foos.Current.id);
    foos.MoveNext();
    Assert.AreEqual("ID3", foos.Current.id);
}
void generated_foos_have_bar()
{
    var foos = generateFoos(3).GetEnumerable();
    foos.MoveNext();
    Assert.AreEqual("baz", foos.Current.bar);
    foos.MoveNext();
    Assert.AreEqual("baz", foos.Current.bar);
    foos.MoveNext();
    Assert.AreEqual("baz", foos.Current.bar);
}

If you are used to the "flat unit test" paradigm (no nested structures/logic), these tests seem pretty clean. Thus logic is avoided in the tests by identifying the original problem as trying to test too many properties at once, rather than lacking loops.

share|improve this answer
1  
Osherove would have your head on a platter for having 3 asserts. ;) The first one to fail means you never validate the rest. Note also that you didn't really avoid the loop. You just explicitly expanded it out into its executed form. Not a hard criticism, but just a suggestion to get some more practice isolating your test cases to the minimum amount possible, to give yourself more specific feedback when something fails, while continuing to validate other cases that could conceivably still pass (or fail, with their own specific feedbacks). –  Anthony Pegram May 8 '13 at 2:24
2  
@AnthonyPegram I know about the one-assert-per-test paradigm. I prefer the "test one thing" mantra (as advocated by Bob Martin, against one-assert-per-test, in Clean Code). Side note: unit testing frameworks that have "expect" versus "assert" are nice (Google Tests). As for the rest, why don't you split your suggestions into a full answer, with examples? I think I could benefit. –  Kazark May 8 '13 at 13:20

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.