Standards of unit testing output

Question

Simple question, what's more common / standard / won't make future developers look at me cross-eyed for unit testing:

Displaying just the errors, or reporting every passed test?

Answer 1

I would say that showing every test and its pass/fail/skipped status is the best, because it gives positive feedback that each test was successfully run or purposefully skipped.

However, if you are building your own tools, I recommend giving the user an option to choose what they want to see.

Answer 2

If you're using a unit testing framework:

• Use defaults. They're usually good, succinct and convey exactly the information you need. Look at nose in Python (find command line examples!) or the Test tools in Visual Studio 2010. In my experience, they're quite good.
• Less is more. Don't clutter results with lots of long-winded error messages or needless warnings. Again, see default results for tests in the previous frameworks to get a good idea of what looks good.

If you're building a unit testing framework:

• Above all make things intuitive and easy to understand. You can get pretty far just presenting pass/fail/skipped results for each test can be extremely helpful. Again, less is more: gussied-up presentations usually won't help things out too much. Even simple colour text-coding can help immensely, as can basic print-outs to a command window.

Answer 3

Depends what you want to see. It's always nice to have at least the option of seeing all the results (as in, a complete list of all tests that have been run, and their status), and if you run the tests as a developer, you are probably also interested in error details for the failed ones. You do want a way to boil it all down to one single exit status though: either everything passes (which means global pass) or one or more tests do not pass (which means global failure). This is especially important if you want to hook your test suite into a Continuous Integration framework or similar automated checking mechanism.

So, long story short: If you can, make it configurable. Most off-the-shelf solutions have this option, and if you're rolling your own, it's not hard to do either.