Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

In the comment to this great post Roy Osherove pointed to the OAPT project that is designed to run each assert in a single test.

This is written on projects home page:

Proper unit tests should fail for exactly one reason, that’s why you should be using one assert per unit test.

And also Roy wrote in comments:

My guideline is usually that you test one logical CONCEPT per test. you can have multiple asserts on the same object. they will usually be the same concept being tested.

I think that there are some cases when multiple assertions are needed (e.g. Guard Assertion), but in general I try to avoid this. What is your opinion? Please provide a real word examples when multiple asserts are really needed.

share|improve this question
1  
How do you do mocking without having multiple assertions? Each expectation on the mock is an assertion in itself, including any order of calls you impose. –  Christopher Creutzig Mar 15 '13 at 19:05
3  
I've seen the one-assert-per-method philosophy abused in the past. An old co-worker used a funky inheritance mechanism to make this possible. It led to a lot of sub-classes (one per branch) and lots of tests that did the same set-up/tear-down process only to check the different results. It was slow, hard to read and a severe maintenance problem. I never convinced him to switch back to a more classic approach. Gerard Meszaros book talks about this topic in detail. –  Travis Parks Mar 15 '13 at 19:39
    
I think as a general rule of thumb you should try to minimize the number of asserts per test. However, as long as the test sufficiently narrows the problem to a specific place in the code, then it's a useful test. –  ConditionRacer Mar 29 '13 at 16:12
1  
I've seen cases where multiple asserts were used instead of RowTest (MbUnit) / TestCase (NUnit) to test a variety of edge-case behaviors. Use the proper tools for the job! (Unfortunately, MSTest doesn't seem to have a row-test capability yet.) –  GalacticCowboy Mar 29 '13 at 17:35
    
@GalacticCowboy You can get similar functionality of RowTest and TestCase using test data sources. I'm using a simple CSV file with great success. –  julealgon Mar 12 at 15:57

10 Answers 10

up vote 67 down vote accepted

Great question.

I don't think it's necessarily a bad thing, but I do think we should strive towards only having single asserts in our tests. This means you write a lot more tests and our tests would end up testing only one thing at a time.

Having said that, I would say maybe half of my tests actually only have 1 assert. I think it only becomes a code (test?) smell when you have about 5 or more asserts in your test.

How do you solve multiple asserts?

share|improve this answer
3  
Like this answer - i.e. its OK, but its not, in the general instance, good (-: –  Murph Sep 28 '10 at 11:45
    
I like this one too! –  Restuta Sep 28 '10 at 21:15
    
I do it a bit. For example, if I'm testing comparability, and that ItemA > ItemB I'll also assert that ItemB < ItemA in the same test. –  CaffGeek Oct 26 '12 at 13:53
1  
Ehh? Why would you do that? the method execution is exactly the same? –  jgauffin Oct 30 '12 at 15:08
26  
A single assert per unit test is a great way to test the reader's ability to scroll up and down. –  Tom Oct 30 '12 at 15:45

I don't even agree with the "fail for one reason only" in general. What is more important is that tests are short and reads clearly imo.

This is not always achievable though and when a test is complicated a (long) descriptive name and testing fewer things makes more sense.

share|improve this answer

I have never thought that more than one assert was a bad thing.

I do it all the time:

public void ToPredicateTest()
{
    ResultField rf = new ResultField(ResultFieldType.Measurement, "name", 100);
    Predicate<ResultField> p = (new ConditionBuilder()).LessThanConst(400)
                                                       .Or()
                                                       .OpenParenthesis()
                                                       .GreaterThanConst(500)
                                                       .And()
                                                       .LessThanConst(1000)
                                                       .And().Not()
                                                       .EqualsConst(666)
                                                       .CloseParenthesis()
                                                       .ToPredicate();
    Assert.IsTrue(p(ResultField.FillResult(rf, 399)));
    Assert.IsTrue(p(ResultField.FillResult(rf, 567)));
    Assert.IsFalse(p(ResultField.FillResult(rf, 400)));
    Assert.IsFalse(p(ResultField.FillResult(rf, 666)));
    Assert.IsFalse(p(ResultField.FillResult(rf, 1001)));

    Predicate<ResultField> p2 = (new ConditionBuilder()).EqualsConst(true).ToPredicate();

    Assert.IsTrue(p2(new ResultField(ResultFieldType.Confirmation, "Is True", true)));
    Assert.IsFalse(p2(new ResultField(ResultFieldType.Confirmation, "Is False", false)));
}

Here I use multiple asserts to make sure complex conditions can be turned into the expected predicate.

I am only testing one unit (the ToPredicate method), but I am covering everything I can think of in the test.

share|improve this answer
14  
Multiple asserts are bad because of error detection. If you have failed your first Assert.IsTrue, other asserts will not be executed, and you won't get any information from them. On other hand if you had 5 tests instead of 1 with 5 asserts you could get something usefull –  Sly Sep 28 '10 at 10:52
3  
Do you consider it still bad if all the asserts test the same kind of functionality? Like above, the example tests the conditionals and if any of this fails, you should fix it. Does it matter to you that you may miss the last 2 asserts if an earlier one fails? –  cringe Sep 28 '10 at 10:55
40  
I fix my problems one at a time. So the fact that the test could fail more than once doesn't bother me. If I split them up I would have the same errors come up, but all at once. I find it easier to fix things a step at a time. I admit, that in this instance, the last two assert could probably be refactored into their own test. –  Matt Ellen Sep 28 '10 at 11:00
13  
Your case is very representative, that is why NUnit has additional attribute TestCase - nunit.org/?p=testCase&r=2.5 –  Restuta Sep 28 '10 at 11:12
3  
google C++ testing framework has ASSERT() and EXPECT(). The ASSERT() stops on failure while EXPECT() continue. That's very handy when you want to validate more than one thing in the test. –  ratkok Jun 25 '11 at 4:28

If you have multiple asserts in a single test function, I expect them to be directly relevant to the test you are conducting. For example,

   @Test
   test_Is_Date_segments_correct {

   // It is okay if you have multiple asserts checking dd, mm, yyyy, hh, mm, ss, etc. 
   // But you would not have any assert statement checking if it is string or number, that is a different test and may be with multiple or single assert statement.
   }

Having a lot of tests (even when you feel that it is probably an overkill) is not a bad thing. You can argue that having the vital and most essential tests is more important. SO, when you are asserting, make sure your assert statements are correctly placed rather than worrying about multiple asserts too much. If you need more than one, use more than one.

share|improve this answer

The goal of the unit test is to give you as much information as possible about what is failing but also to help accurately pinpoint the most fundamental problems first. When you know logically that one assertion will fail given that another assertion fails or in other words there is a dependency relationship between the test then it makes sense to roll these as multiple asserts within a single test. This has the benefit of not littering the test results with obvious failures which could have been eliminated if we bailed out on the first assertion within a single test. In the case where this relationship does not exist the preference would naturally be then to separate these assertions into individual tests because otherwise finding these failures would require multiple iterations of test runs to work out all of the issues.

If you then also design the units/classes in such a way that overly complex tests would need to be written it makes less burden during testing and probably promotes a better design.

share|improve this answer

Tests should fail for one reason only, but that doesn't always mean that there should be only one Assert statement. IMHO it is more important to hold to the "Arrange, Act, Assert" pattern.

The key is that you have only one action, and then you inspect the results of that action using asserts. But it is "Arrange, Act, Assert, End of test". If you are tempted to continue testing by performing another action and more asserts afterwards, make that a separate test instead.

I am happy to see multiple assert statements that form parts of testing the same action. e.g.

[Test]
public void ValueIsInRange()
{
  int value = GetValueToTest();

  Assert.That(value, Is.GreaterThan(10), "value is too small");
  Assert.That(value, Is.LessThan(100), "value is too large");
} 

or

[Test]
public void ListContainsOneValue()
{
  var list = GetListOf(1);

  Assert.That(list, Is.Not.Null, "List is null");
  Assert.That(list.Count, Is.EqualTo(1), "Should have one item in list");
  Assert.That(list[0], Is.Not.Null, "Item is null");
} 

You could combine these into one assert, but that's a different thing from insisting that you should or must. There is no improvement from combining them.

e.g. The first one could be

Assert.IsTrue((10 < value) && (value < 100), "Value out of range"); 

But this is not better - the error message out of it is less specific, and it has no other advantages. I'm sure you can think of other examples where combining two or three (or more) asserts into one big boolean condition makes it harder to read, harder to alter and harder to work out why it failed. Why do this just for the sake of a rule?

NB: The code that I am writing here is C# with NUnit, but the principles will hold with other languages and frameworks. The syntax may be very similar too.

share|improve this answer
1  
I feel like this should be 1 assertion using Assert.IsTrue(10 < value < 100); –  Danny G Oct 29 '12 at 17:49
    
Danny - I have amended the answer to talk about this. –  Anthony Oct 30 '12 at 13:46

Another reason why I think, that multiple asserts in one method is not a bad thing is described in following code:

class Service {
    Result process();
}

class Result {
    Inner inner;
}

class Inner {
    int number;
}

In my test I simply want to test that service.process() returns the correct number in Inner class instances.

Instead of testing...

@Test
public void test() {
    Result res = service.process();
    if ( res != null && res.getInner() != null ) Assert.assertEquals( ..., res.getInner() );
}

I'm doing

@Test
public void test() {
    Result res = service.process();
    Assert.notNull(res);
    Assert.notNull(res.getInner());
    Assert.assertEquals( ..., res.getInner() );
}
share|improve this answer
1  
And that's a good thing, you shouldn't have any conditional logic in your tests. It makes test more complex and less readable. And I guess Roy outlined it right in his blog post that multiple asserts on one object is okay most of the time. So those that you have are just guard asserts and it's okay to have them. –  Restuta Oct 30 '12 at 20:44

If your test fails, you won't know whether the following assertions will break, too. Often, that means you'll be missing valuable information to figure out the source of the problem. My solution is to use one assert but with several values:

String actual = "val1="+val1+"\nval2="+val2;
assertEquals(
    "val1=5\n" +
    "val2=hello"
    , actual
);

That allows me to see all failed assertions at once. I use several lines because most IDEs will display string differences in a compare dialog side-by-side.

share|improve this answer

When I'm using unit testing to validate high-level behavior, I absolutely put multiple assertions into a single test. Here's a test I'm actually using for some emergency notification code. The code that runs before the test puts the system into a state where if the main processor gets run, an alarm gets sent.


    @Test
    public void testAlarmSent() {
        assertAllUnitsAvailable();
        assertNewAlarmMessages(0);

        pulseMainProcessor();

        assertAllUnitsAlerting();
        assertAllNotificationsSent();
        assertAllNotificationsUnclosed();
        assertNewAlarmMessages(1);
    }

It represents the conditions that need to exist at every step in the process in order for me to be confident that the code is behaving the way I expect. If a single assertion fails, I do not care that the remaining ones won't even get run; because the state of the system is no longer valid, those subsequent assertions wouldn't tell me anything valuable.* If assertAllUnitsAlerting() failed, then I wouldn't know what to make of assertAllNotificationSent()'s success OR failure until I determined what was causing the prior error and corrected it.

(* -- Okay, they might conceivably be useful in debugging the problem. But the most important information, that the test failed, has already been received.)

share|improve this answer
    
When you do that you you should better using testing frameworks with dependant tests, it is better (e.g. testng supports this feature) –  Kemoda Oct 26 '12 at 13:44
1  
I write tests like this too so that you can be confident of what the code is doing and state changes, I don't think this is a unit test, but an integration test. –  mdma Sep 11 '13 at 9:30

Having multiple assertions in the same test is only a problem when the test fails. Then you might have to debug the test or analyse the exception to find out which assertion it is that fails. With one assertion in each test it's usually easier to pinpoint what's wrong.

I can't think of a scenario where multiple assertions are really needed, as you can always rewrite them as multiple conditions in the same assertion. It may however be preferrable if you for example have several steps to verify the intermediate data between steps rather than risking that the later steps crash because of bad input.

share|improve this answer
1  
If you are combining multiple conditions into a single assert, then on failure all you know is that one failed. With multiple asserts you know specifically about some of them (the ones up to and including the failure). Consider checking a returned array contains a single value: check it is not null, then it has exactly one element and then then value of that element. (Depending on the platform) just checking the value immediately could give a null dereference (less helpful than the null assertion failing) and doesn't check the array length. –  Richard Sep 28 '10 at 15:41
    
@Richard: Getting a result and then extracting something from that result would be a process in several steps, so I covered that in the second paragraph in the answer. –  Guffa Sep 28 '10 at 16:42
2  
Rule of thumb: if you have multiple assertions in a test, each one should have a different message. Then you don't have this problem. –  Anthony Oct 26 '12 at 13:17

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.