Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Simple question, what's more common / standard / won't make future developers look at me cross-eyed for unit testing:

Displaying just the errors, or reporting every passed test?

share|improve this question
2  
Most unit test frameworks will display the results of each test, whether it passes, failed, was inconclusive or was not run. –  Oded Aug 14 '12 at 15:44
    
@Oded Just to help me wrap my head around this, could you describe a unit test which was inconclusive? –  Philip Aug 14 '12 at 18:41
    
See here: nunit.org/index.php?p=utilityAsserts&r=2.5.10 –  Oded Aug 14 '12 at 18:42
1  
@Philip: "not implemented yet" is probably the most common cause for inconclusive unit test results - either the feature hasn't been implemented yet, or the unit test is just an auto-generated stub. –  tdammers Aug 14 '12 at 19:06
    
Also, if the setup for your test fails (and not the test itself), then that could warrant calling the result inconclusive instead of failed. –  MetaFight Jul 12 '13 at 7:46

4 Answers 4

up vote 3 down vote accepted

I would say that showing every test and its pass/fail/skipped status is the best, because it gives positive feedback that each test was successfully run or purposefully skipped.

However, if you are building your own tools, I recommend giving the user an option to choose what they want to see.

share|improve this answer

Depends what you want to see. It's always nice to have at least the option of seeing all the results (as in, a complete list of all tests that have been run, and their status), and if you run the tests as a developer, you are probably also interested in error details for the failed ones. You do want a way to boil it all down to one single exit status though: either everything passes (which means global pass) or one or more tests do not pass (which means global failure). This is especially important if you want to hook your test suite into a Continuous Integration framework or similar automated checking mechanism.

So, long story short: If you can, make it configurable. Most off-the-shelf solutions have this option, and if you're rolling your own, it's not hard to do either.

share|improve this answer

If you're using a unit testing framework:

  • Use defaults. They're usually good, succinct and convey exactly the information you need. Look at nose in Python (find command line examples!) or the Test tools in Visual Studio 2010. In my experience, they're quite good.
  • Less is more. Don't clutter results with lots of long-winded error messages or needless warnings. Again, see default results for tests in the previous frameworks to get a good idea of what looks good.

If you're building a unit testing framework:

  • Above all make things intuitive and easy to understand. You can get pretty far just presenting pass/fail/skipped results for each test can be extremely helpful. Again, less is more: gussied-up presentations usually won't help things out too much. Even simple colour text-coding can help immensely, as can basic print-outs to a command window.
share|improve this answer

I definitely vote for showing any test result, both passed and failed, maybe summarizing the failed tests at first.

IMHO when unit testing it's fundamental to get all details from test run.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.