Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I was following a book Art of Unit Testing, This book says that You must not have any logic in Your asserts as this reduces readability of test, or I may be testing too many things at a time. Suppose there is a method with signature as below:

public int DoSomething(int variable1, out string variable2, out float variable3)
{
...
}

How should I test this method, as I have to assert on return value, variable2, variable3 ???


Sample Test method that tests all of these three concerns:

    public void DosSomethingTest(){
        //add code for arrange
        int expectedvalue=12;
        string expectedVariable1="I am expected";
        string expectedVariable3=15.9;
        string actualVariable1, actualVariable3;
        var sut=new Foo();
        int actualValue=sut.DoSomething(15, out actualVariable1, out actualVariable3);

        //this is where I am testing 3 concerns in one test. Is there a better way? 
        if(actualValue==expectedvalue){
            if(expectedVariable1==actualVariable1){
                if(expectedVariable3!=actualVariable3){
                    Assert.Fail("Unexpected variable2 returned");//pseudo code
                }
            }
            else{
                Assert.Fail("Unexpected variable1 returned");//pseudo code
            }
        }
        else{
            Assert.Fail("Unexpected Value returned");//pseudo code
        }
    }

Approach 2


Seperate Test methods that tests concerns separately:

    public void DosSomething_Passing15_Returns12Test(){
        //add code for arrange
        int expectedvalue=12;
        string expectedVariable1="I am expected";
        string expectedVariable3=15.9;
        string actualVariable1, actualVariable3;
        var sut=new Foo();
        int actualValue=sut.DoSomething(15, out actualVariable1, out actualVariable3);

        Assert.IsTrue(actualValue, expectedvalue);
    }

    public void DosSomething_Passing15_ReturnsExpectedVariable1Test(){
        //add code for arrange
        int expectedvalue=12;
        string expectedVariable1="I am expected";
        string expectedVariable3=15.9;
        string actualVariable1, actualVariable3;
        var sut=new Foo();
        int actualValue=sut.DoSomething(15, out actualVariable1, out actualVariable3);

        Assert.IsTrue(actualVariable1, expectedVariable1);
    }

    public void DosSomething_Passing15_ReturnsExpectedVariable2Test(){
        //add code for arrange
        int expectedvalue=12;
        string expectedVariable1="I am expected";
        string expectedVariable3=15.9;
        string actualVariable1, actualVariable3;
        var sut=new Foo();
        int actualValue=sut.DoSomething(15, out actualVariable1, out actualVariable3);

        Assert.IsTrue(actualVariable3, expectedVariable3);
    }

Isn't approach 2 adhering to "Tests should not test more than one concern"?

Conclusion


If redesigning of SUT is possible:

  1. There should ideally be no out parameter instead an aggregate class should be evaluated in Asserts.
  2. If there are unrelated out parameters then SUT method should be refactored into multiple methods, then those new methods should have their own Test methods and they will then test only one concern.

If redesigning of SUT is not possible then as per the answer there will be three Asserts

share|improve this question
1  
Please don't put EDIT monikers in your questions. Every post on Stack Exchange already has a detailed edit history that anyone can review. The edit history for your question is here. –  Robert Harvey Apr 2 at 15:59

5 Answers 5

up vote 16 down vote accepted

If someone tells you there is something wrong with:

int actualValue=sut.DoSomething(15, out actualVariable1, out actualVariable2);

AssertEquals("actualValue", 1, actualValue);
AssertEquals("actualVariable1", 23, actualVariable1);
AssertEquals("actualVariable2", 14, actualVariable2);

then they are very likely being overly pedantic, applying a rule blindly without understanding anything of context. I mean, you probably generally should consider simplifying a function with multiple out arguments.

But if you have them, that's how you test them.

share|improve this answer
2  
+1 for for showing that I don't need any if/else branching. If first assert fails then it automatically jumps to next. –  shankbond Mar 30 at 11:57
    
as for as "pedantic" is concerned, I have already faced so many problems, written brittle test cases, and all that could potentially go wrong with a test project. At the end I have even abandoned using the Unit Test project. The context here was that Unit Tests should be more readable and if they are readable and simple, it is likely that those unit tests will be used for a longer time. –  shankbond Mar 30 at 12:21
    
@shankbond: that unit test is as simple as it can be without changing signatures. If you want it to be even simpler, change the API of your sut and remove the out parameters. –  Doc Brown Mar 30 at 13:08
2  
@shankbond the question is how your three values correlate... why does your method produce all three of them? because they are part of some higher-level entity? Then capturing them in an aggregate class seems a good idea. If not, why are they computed in the same method? Split the method up into three methods, each of which returns one of the values. –  Sven Mar 31 at 8:20
1  
@salsolatragus how would you handle intDivide(int numerator, int denominator, out int quotient, out int remainder)? Wouldn't packaging up the quotient and remainder into a class be a bit excessive? And if you want both, calling two functions (when you've already got the result from a single operation) becomes unnecessarily expensive. –  MichaelT Mar 31 at 16:12

You call this function a number of times with chosen values for variable1. For each input value, you check that each of the output variables and return value match the expected result. You write those as single asserts on each expected value, so that a failure pinpoints the fault. Each input value and set of output values constitutes a single test.

The values you choose should span the range of permitted values, and should particularly explore any known values that represent edge conditions for the algorithm the function uses. Some people call this "grey box" testing (not quite black).

You also test (or attempt to test) the function for aberrant behaviour (perhaps variable1 must be positive, so you check that it fails correctly for negative values.).

A data driven approach works well, where all the relevant values are pre-computed and stored in a database or other structure. The actual unit test should contain little or no computation.

share|improve this answer

The idea behind "a method should test one thing" is so you can immediately see what is causing the problem when the test fails.

If you have an assert on three different things then you will have to see what one is going wrong exactly.

Another benefit of splitting them up is so you have an overview over what works and what doesn't when an assert fails. If you put all three in the same method and the first assert fails, you won't know if the latter two work. This will increase the difficulty for solving the problem: you won't know when you accidentally break another output parameter.

At this point you have to make a decision for yourself: do you consider these three output variables closely related or are they worth splitting up?

If they are closely related you can keep them together but if they aren't then I would advise you to put them in separate methods. You should extract their common logic to another method so you don't have to write the setup twice.

share|improve this answer
    
If all of the values are closely related and I aggregate all of the return parameters into a new class, I still in the end will have to verify whether all the properties of this new aggregate class matches, and will again have three asserts. –  shankbond Apr 1 at 3:39
    
What's the reasoning behind that? This adds an extra layer of complexity because you have to lookup what values are being compared exactly. –  Jeroen Vannevel Apr 1 at 3:41
    
Then How do You suggest I should be asserting on this new aggregate class? Can You please provide an example –  shankbond Apr 1 at 3:46
    
I don't see the necessity for such an aggregate class in the first place. Why do you want one? –  Jeroen Vannevel Apr 1 at 4:12
    
I guess I was a bit confused and mixed two things from Your answer, testing approach, and the design advice for aggregating out parameters. :) –  shankbond Apr 1 at 15:42

Words Of Wisdom for those brave souls who are embracing unit testing

After struggling with this question, and googling endless days, I finally came to different conclusion:

A good Unit Test should test only one concern, but it can test multiple aspects.

In my question, I am asking about multiple aspects, so it is Ok to have three asserts and may/ may not be neccessary to refactor the SUT(System under Test) method as per the given answers:

In cases where there are different concerns which essentially means:

Different inputs and their corresponding outputs

as shown below:

//Works Only in case of Nunit.

 [TestCase(15, "I am expected", 15.9, 12)]
 [TestCase(17, "I am expected1", 100, 12)]
 [TestCase(18, "I am expected2", 112, 12)]
 public void DoSomething_ReturnsExpectedTest(int inputValue, string expectedVariable1, int expectedVariable3, int expectedValue){
    //add code for arrange
    string actualVariable1, actualVariable3;
    var sut=new Foo();
    int actualValue=sut.DoSomething(15, out actualVariable1, out actualVariable3);

    Assert.IsTrue(actualValue, expectedvalue);
    Assert.IsTrue(actualVariable1, expectedVariable1);
    Assert.IsTrue(actualVariable3, expectedVariable3);    }
share|improve this answer

An assert is an encapsulation of the test logic:

assert foo(x) is y

can be written as

if foo(x) is not y then throw AssertionError

Using assert is encouraged since it's shorter, more readable, and unit testing frameworks may recognize it and treat it specially.

Now, you want to have those benefits for assertions that encapsulate a different logic - one that also tests out parameters. For this - you should write your own assertion helper!

The logic will be inside the assertion helper and the assertion statement itself will remain logicless. True - it's syntax will be a bit complex, as you have to specify the method to be tested, the input parameters, the expected return value and the expected out parameter values. Still, the fact that you have out parameters means you are using a language that is not Java so you should be able to achieve a nice syntax, that can be readable with the right line-breaking and indentation conventions.

Maybe your testing framework even have such helper!

UPDATE

You are concerned about testing multiple things with the assertion. OOP can help you here - first create an object that encapsulates the specific call to the method, and then use methods of that object to assert the different parts of the result of that call:

ComplexAssert.AssertWithOutParameters(sut.DoSomething,15)
    .AssertReturnValue(expectedValue,"Unexpected Value returned")
    .AssertOutParameter(1,expectedVariable1,"Unexpected variable1 returned")
    .AssertOutParameter(2,expectedVariable3,"Unexpected variable2 returned");

Now everything is neat and readable - the tested method is only called once and still each assertions gets it's own method call.

share|improve this answer
    
Thanks for the reply, actually I am looking for the approach, Is it even legal(correct) to have one test that tests so many concerns? Also, I have updated my question with slight elaboration –  shankbond Mar 30 at 10:48
    
See my update to the answer –  Idan Arye Mar 30 at 11:14

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.