Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

From Martin Fowler's Mocks Aren't Stubs he states that Stubs provide canned answers to calls made during the test while Mocks objects pre-programmed with expectations which form a specification of the calls they are expected to receive.
In addition several sources state that Stubs shouldn't fail tests, but Mock could.

So what about this:

$validator= $this->getMock('ValidatorService');

        // Set up the expectation for the isValid() method
        // to be called only once and with the string 'textToBeValidated'
        // as its parameter and returns true or false.
        $validator->expects($this->once())
                 ->method('isValid')
                 ->with($this->equalTo('textToBeValidated'))
                 ->will($this->returnValue(true));

This came up in a code review in a project I'm working on and we called it a Stock (St ub - M ock). Why you ask?!

  1. It set an expectation for isValid() to be called only once and with certain input like a Mock.
  2. It returns a canned answer like a Stub.

I get a feeling that this is wrong, but what if we used a pure Stub only:

 $validator->expects($this->any())  //<-- can't fail the test now
                     ->method('isValid')
                     ->will($this->returnValue(true));   //<-- canned answer
  • I can't verify that the method isValid was called only once with the right parameters.
  • If the test passed I'm not sure that it passed for the right reason. Maybe isValid() was called twice and that caused it to pass.

So let's use a pure Mock instead:

$validator->expects($this->once())
                     ->method('isValid')
                     ->with($this->equalTo('textToBeValidated'))
  • I can't verify that my function works correctly, because I only verified that call to isValid() was successful. Maybe there is a bug in the rest of my code using the ValidatorService.

I know some people will talk about state vs behavior verification, but the way I see it the only way to be sure that the test passes for the correct reason is to to use a Stock (i.e. verify the behavior and final state).

Of course the other option is to write 2 test for every test criteria. One using a Mock and an identical test using a Stub. However, that will result in a TestOverflow (a huge number of nearly identical tests).

What is the correct way to have a concrete test of a function using a dependency?

share|improve this question
1  
@RobertHarvey well yea it's working. However, just coz it's working doesn't mean it's a good practice or maintainable. –  Songo Mar 3 at 17:18
1  
@RobertHarvey I have never seen a fake object used that way in a unit testing or TDD book (we r new to TDD btw). It seems like we are mixing the usage of Stubs with Mocks when the intent of each of them is different, however, I can't see how I would have 100% confidence in my test otherwise. –  Songo Mar 3 at 17:18
2  
@RobertHarvey I see what you mean. We aren't blindly trying to follow a dogma for unit testing. We are still trying to find the correct way to proceed. That's why we do code review sessions in the first place :) We have very limited access to unit testing or TDD expertise on site so most of us are trying to contribute by self study or open discussions during code review sessions. Nobody holds the "I read it then it's correct" attitude it's more like "I haven't seen it before, what's ur resource?" –  Songo Mar 3 at 17:31
1  
can you give an example why it is important that validate was called once? I can only imagine tests where stubs are sufficient. Example: having an orderclass that uses an addressvalidation service and the order should only be possible if the billing address is valid. so i have one test where validate returns true and one where validate returns false –  k3b Mar 3 at 17:51
1  
Most frameworks, nowadays, have a Verify method that can be called after the method under test. So you stub to get your canned response, regardless of the input, and then verify that the right input was given. This confirms more closely to Arrange-Act-Assert AND removes this confusion between stubbing and mocking. –  pdr Mar 3 at 18:30

3 Answers 3

Mocks will return stub answers if that is required by the contract of the class/interface they're mocking. That's just necessary. Your object isn't anything special, it's just a mock.

Nevertheless, prefer stubs to mocks when you can get away with it. Mocks generally lead to less flexible tests compared to tests without mocks. For example, do you have to validate that isValid was called exactly once? Can't you instead have a test case that stubs isValid to return true, another that stubs isValid to return false, and simply observe the results of the two tests? (The answer may be 'no' if you have to make sure the correct text is passed. It's just something to think about.)

share|improve this answer
1  
well that's exactly what I meant. If I stub only how could I be sure that there isn't another bug that causes it to pass? I need to verify the end result and the way I got there too. –  Songo Mar 3 at 17:22
2  
You want to test your real isValid function separately. You want it to be idempotent so that it doesn't matter how often it is invoked. And then when you test whatever you're testing in your question, you stub it out. Or you could fake it out: make a fake ValidatorService where isValid is implemented as $x === "foo" and write your tests under that premise. That allows you to test that the correct data reaches isValid. –  Sebastian Redl Mar 3 at 17:27
    
@SebastianRedl I think what Songo meant by "...another bug that causes it to pass" that there might be a bug in the function that uses isValid() not in isValid() itself. –  John Raya Mar 3 at 18:01
2  
If there is a bug in the function, but you can't detect that by external behaviour, then there isn't a bug in the function. At most there is a potential performance or style improvement - testing for those is like trying to test for typos in comments. –  soru Mar 3 at 21:20
    
@SebastianRedl The isValid method implies to me that it is some kind of business logic and like all business logic should be covered by its own test. Perhaps you can include your comment as part of your answer? –  maple_shaft Mar 11 at 11:32

Well, what you are describing as "Stock" is actually the definition of a Mock, at least in the book xUnit Patterns, which describes all the main types of test doubles.

The logic for the types of test doubles is that each inherits from the previous.

Consider a Dummy as the baseline, a NullObject implementation for any given interface. The next type is the Stub, which returns some values. Then there is the Spy, which returns values just as a Stub, but also records the details of its interactions. Then comes the Mock, which by inheriting from the Spy, can decide whether to fail a test if the interaction is not valid for the given expectations.

By this logic, a Mock is a Spy, which is a Stub, which is a Dummy, which is a Test Double (the completely abstract concept). There is also the concept of fakes, but they follow a different hierarchy.

You can see the details on the definition of each type here.

share|improve this answer
up vote 2 down vote accepted

OK I did some research and finally found what I was looking for in the post Mocks for Commands, Stubs for Queries by Mark Seemann.

To make a long story short Stocks are bad. They are a code smell or better a unit testing smell indeed.

Having experienced the pain of over specifying interactions between classes in my code I find "J. B. Rainsberger" post to reflect this ugly truth:

When one uses a mock in place of a stub, one tends to over-specify the collaboration between a Client and its Suppliers, and that leads to brittle tests. These tests tend to make hyperactive assertions, meaning assertions that fail even when the underlying production code works as expected. Such hyperactive assertions play the role of the boy who cried “wolf!”, distracting us with needless concern. Even when these superfluous mocks don’t over-specify a collaboration, they often duplicate specifications in other tests.

So next time you find yourself writing "expects" and "returns" in the same fake object think again.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.