What are the key factors in choosing a Mocking Framework?

Question

I'm looking to get started with objects in my unit tests. It seems there are tons of good mocking frameworks out there.

1. Do the different frameworks have different target audiences?
2. What factors should I consider when choosing which framework is right for my situation?

Answer 1

Do the different frameworks have different target audiences?

Yes. Some frameworks like Microsoft Moles, TypeMock Isolator, and JustMock, allow you to be able to mock just about anything. These mocking tools are generally better for developers wanting to use them on existing legacy code since it may not be possible to refactor such into a design that is more testable.*

Traditionally, testable designs mean that the codebase needs to make liberal use of interfaces, abstract classes, virtual methods, unsealed classes, etc. Therefore, traditional mocking frameworks like Moq and RhinoMocks work well with code developed using Test Driven Development, Dependency Injection, and other such concepts. By the way, I would highly recommend using Dependency Injection as you gain much more than just testable code, but more maintainable code as well.

What factors should I consider when choosing which framework is right for my situation?

• Development Activity. Tools like Moq, and RhinoMocks are very active and popular and thus are up to date.
• Open Source Vs. Commercial. Consider the various pros and cons typical for this comparison. Cost, Support, etc...
• Maturity. How new is the tool. Is it in beta (like Microsoft Moles) or has it had several stable releases? For example, I like Moles for legacy code, but there are several bugs that need to addressed in it and there is going to be along wait before they get addressed (next release Nov. 2011).
• Documentation. There are several books and blogs that cover unit testing, mocking, auto-mocking, etc. Additionally how good is the tool's own documentation?
• Syntax. Each tool has it's own way of saying the same thing. See which one is a better fit for you.
• Speed. Tools that use CLR profiling (TypeMock, Moles, JustMock), can be much slower than traditional ones (Moq, RhinoMocks). This speed penalty may be an issue as you amass many unit tests. The rule of thumb is if a test takes longer than 1/10 a second it is too slow.
• Community Support. Are other developers writing other tools that extend (or work in compliment) to the mocking tool? There is a Moq.Contrib project that adds an Auto-mocking ability to Moq (which helps speed up test writing time). Better yet, there is AutoFixture, AutoFixture.AutoMoq, AutoFixture.AutoRhinoMocks, which also allows for Auto-mocking, plus anonymous variable creation.

^{*See Working Effectively With Legacy Code, for ways on how to slowly refactor code without tests into code that can be used with traditional testing (and mocking) tools.}

Answer 2

The Moq tutorial has a section on background, philosophy, and controversy right at the beginning which discusses this in relation to a few specific tools: TypeMock Isolator, RhinoMocks, and Moq. It's written to explain Moq, so is naturally a bit skewed, but I found it to be quite helpful for me when trying to understand some of the differences in mocking frameworks.

I found the responses to this SO thread on C# Mocking Frameworks also useful. Most just refer to one Mocking Framework that the user really finds useful, but there is a response from HaraldV a ways down that discusses proxy-based mocks and profiler-based mocks.

I was also able to find a comparison chart online. Note that it's from 2009, so I'm not sure it is up-to-date; there is at least one comment stating that the info on TypeMock and callbacks is outdated, but the chart might be good for raising issues to consider even if you will need to do legwork to see what the current state is: RhinoMocks, Moq, NMock, and TypeMock comparison chart

There is a project on Google Code with test cases in multiple mocking frameworks for easy code comparison: mocking-frameworks-compare

Answer 3

1. Ease of use. Some of the frameworks have more advanced idioms of usage. For example, MOQ allows the use of lambdas to encode expectations. Some older libraries do not support this.
2. Speed. Each unit test should be fast so that your entire library doesn't take hours to run. Some mocking frameworks have statically generated mocks, which is fast. Other frameworks dynamically generate code at runtime, which is slower.
3. Support. You want a framework that is actively supported with fixes and updated to support new versions of .NET, as they are released.
4. Power. Most of the mocking frameworks I've researched are roughly the same in terms of power. There is one notable exception. Microsoft Moles allows for mocking of "non-virtual/static methods in sealed types." This is something that no other mocking framework supports, to my knowledge.

On my team, we chose Microsoft Moles. It wins significantly on #2, #3, and #4, though it is less idiomatic than most of the alternatives and is at the low end on #1.