Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

we're working on error handling in an application. We try to have fairly good automated test coverage. One big problem though is that we don't really know of a way to test some of our error handling.

For instance, we need to test that whenever there is an uncaught exception, a message is sent to our server with exception information. The big problem with this is that we strive to never have an uncaught exception(and instead have descriptive error messages).

So, how do we test something what we never want to actually happen?

share|improve this question
add comment

4 Answers

Simulate the failure / exceptional situation.

Your mock objects are crash test dummies and you already know what can possibly go wrong, it shouldn't be hard to emulate it (you know because you've already prepared for it, if you don't, well, you'll soon find out ;). And... it's always amazingly fun to physically unplug your database server while your processes are running. Doesn't really scale as a testing method, but it's awesome even if you do it just once.

Consider this trivial / overly simplistic example:

  • Class A does something, and throws an OMGSomethingBroke exception if something goes awry.
  • Class B calls Class A, and in case something breaks, catches the exception and logs it (or whatever).

What you need to do is create a mock of Class A, let's call it Mock A that does nothing more than throw the OMGSomethingBroke exception. Mock A and Class A should implement the same interface, and that's about it. You've emulated the failure, and you can now move on to testing Class B. Have it call Mock A instead of Class A, and test if it handles the failure gracefully.

Testing can be exceptionally simple if your design is solid (yes, both puns intended).


Just found a very interesting Udacity course, Software Testing (CS258) - How to Make Software Fail:

When writing software, destruction can be just as valuable as creation. Learn how to catch bugs and break software as you discover different testing methods that will help you build better software.

The course has a wider scope than your question, obviously, but the core message in its summary applies: Breaking stuff is fun! (Well, and a good way to test software).

share|improve this answer
1  
Yep, mocks are the way to go! Error testing is the #1 way I use mocks: "What happens to our Twitter-consuming service if Twitter goes down?", etc. –  Ethel Evans Oct 2 '12 at 17:30
add comment

This is one of the many reasons why modularization is a good idea.

Your test is supposed to test the behaviour of one bit of code, and one only. In this case, it's the code that sends email when a different bit of code generates an error. So what you do is hook up a mock version of the other bit of code that does create an error, to your actual, production-grade error handler, and then verify that it does send email. (You can also replace the email sending code with a mock version, too, so it doesn't actually send the mail but just reports that it would have done so - makes the test much easier and faster.)

That way, everyone is happy: Your error handler is tested, your production code never causes errors (that you know of), and you still know hat would happen if it did. If your code base doesn't support such flexible hook-ups, well this is one major reason why it should.

share|improve this answer
    
Well, technically, we use a separate product for the message sending which is tested, so that part is not a problem. It's more making sure that in the uncaught exception case that this bit of code gets called. I still do not quite understand how we'd cause an uncaught exception where there isn't suppose to be one. –  Earlz Oct 2 '12 at 15:00
    
Check out Mock Objects and Dependency Injection. If you program your business logic against interfaces that can be implemented both by production classes and by test classes, you can control the input and output to the code under test as closely as you want. In this case the input to the error handler is the fact that an exception is thrown, so you would inject a class that deliberately raises one - it can be as simple as dividing by zero (in a non-obvious way, so that the compiler won't forbid it). –  Kilian Foth Oct 2 '12 at 15:08
    
@Earlz Example: Class A does something, and throws an exception if something goes wrong, Class B calls Class A, catches the exception, logs it and sends the message. Your mock of Class A, Mock A will just throw the exception, and you'll be testing whether the exception was thrown and if it was of the correct type. Mock A and Class A should implement the same interface, and if there's a future change your test should fail. It's a very simple test really. BTW Class B in this example does a bit too much, should be broken down into distinct parts as well. –  Yannis Rizos Oct 2 '12 at 15:09
add comment

So, how do we test something what we never want to actually happen?

Sometimes, you can't, sometimes you don't want to. Boeing doesn't intentionally send a prototype jet into a nose dive 100 feet above the ground to test various "can't happen" type events. A nuclear power plant isn't going to intentionally drive the power plant past the point of no return to see if the tertiary backups work.

This one seems testable. Throw an exception you know isn't caught. Except via a catch-all (e.g., catch (...)), you almost certainly are not catching an exception of type BogusExceptionForTestPurposesOnly. So throw it in some test code. Just make sure this test code never makes it's way into the deliverable.

share|improve this answer
    
Boeing doesn't intentionally send a prototype jet into a nose dive 100 feet above the ground to test various "can't happen" type events Well, perhaps if they did, they'd avoid failures like this one: huffingtonpost.co.uk/2012/08/15/… –  Yannis Rizos Oct 2 '12 at 18:09
    
@YannisRizos: That is not a good example. The X-51's problem isn't that it was built wrong. The problem is that it is a vehicle that apparently can't be built right. Another problem is that there is no such thing as a Mach 5 full scale wind tunnel. The only way to test whether a vehicle intended to fly at Mach 5 can do so is to try to do so. –  David Hammen Oct 2 '12 at 21:56
add comment

Add a special "break the rules" request to debug builds and invoke it. For example add a "uncaught throw" request and make it throw a unique condition you think should be uncaught.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.