“Anything can go wrong will go wrong” - Murphy's Law. so is it necessary to test all the conditional, exception cases?

Question

If "Anything can go wrong will go wrong" is true, we need to test all the conditionals and exceptional cases in my code.

But sometimes it's hard to find all of them since many of them are corner cases, which are hard to test. How can I go about finding corner cases better or avoiding the issue?

Answer 1

Yes! Every block of code should be tested. But how do you test it? By Unit Testing in first place,

well, to let you code to be testable, it is another theory, it is a kind of programming technique. You can't let your code become smell.

1. Test your code part by part by function.
2. Your function must be specific, one purpose, simple, not complex.
   e.g. don't make Global variable in your function, otherwise (not testable)
3. Your function must be understandable.
4. Your function is loosely coupling, less dependency, so that you can test it easily. etc.etc....

Once your code style is improved, your test cases , corner cases would be easily to be tested.

Here is a good article of how to make your code become less smell. Hope it helps.

Answer 2

Consider code coverage analysis. In my experience it was pretty efficient way to discover un-tested use cases.

When tests already have high code coverage, using this approach to discover holes may be fun, too - because remaining cases turn out rather non-obvious (or, even more interesting, might correspond to bugs in specification).

• Just don't assume that coverage analysis will guarantee to cover all the relevant cases. Eg it won't protect you from missing feature or concurrency issues.

Thing to keep in mind is that it may be tricky to drill into every possible case. Imagine an application designed to output "Hello World" if and only if it's Thursday and it rains - to test it, you would have to wait until Thursday when it rains? or mock the necessary conditions. That may be too much effort to justify the risks you'd avoid by having it covered in tests.

Another thing to consider is how much time testing takes. If it takes a year to fully test your application then you better drop that idea (unless you're on mission critical product)

Answer 3

I find this very true. I sometimes tend to be lazy and dont test certain lines of code, but almost never fails to come back and hit me.

If you are writing OO code. Here are the inputs based on my learning.

• Use Occam's Razor. That is find the simplest way to implement it.
• Divide anything that can be divided into multiple classes or mixins until you see that each class or mixin is doing exactly one thing. That is respect SRP.
• For every mixin/class write atleast one test case initially covering every function.
• If you are using a few third party library see if there are chances that those functions can return unexpected results from the documentation.
• Now look at each if/for/while/case and similar constructs to see if that is covered in your test case. If not write more test cases.
• Check if any line can fail due to stuffs like NullPointerException or object not defined type of errors. Typically lines after some sort of object creation or stuff that depend on using library to set your object.
• Use Black box testing, where you test every function weather its available public or not. Most testing frameworks believe that testing public interfaces will do(white box testing) and some languages wont give you access to private interfaces. In these situations try to test the private functions through public functions.
• Use some kind of automated tools that can test for code coverage.

Even after this there will be some scenario's that one cannot easily test, especially the environment related stuff. You can either get the code peer reviewed by an experienced programmer or run your test cases in all possible environments.

One can never be fully confident that every case is covered. If you have spent a lot of time with the code while its developed you will gain confidence on it. Once you have enough confidence, release it bravely. Things will come back, since you would have read your code a 100 times, you can fix stuff really quickly and will be able to imagine other places this will happen. For every bug that comes back, cover it with test cases.

Answer 4

If everyone knew how to account for everything, there would basically be one version of software, no maintenance would be needed, testers wouldn't be required, etc. You can't follow a set of guidelines that will give you a magic black-and-white answer because the list of items to test is based on a multi-variable component (code).

The best thing you can do is learn. You'll make mistakes and forget to do things; make a conscious effort not to make the same mistake twice. If you can do this, you're probably better than the average software engineer.

Answer 5

Maybe

You cannot account for all exceptional cases unless you have a ton of time and money. The more interconnects in a software system the more complex it is and the more it take to cover even the standard test cases. We aren't even necessarily talking about code you wrote. Remember anytime your software isn't operating purely out of memory (that doesn't grow) you will have many more exception cases. Accessing network, disk, display, any other devices can all cause an exception. Growing memory can cause an exception. In some cases even detecting that the software was improperly stopped may be an exception case. The question is do you really need to cover every exception case.

Yes

If your working on software where lives hang in the balance, literally. Then yeah, you probably do need to at least consider every exception cases. Surgery software, Space Shuttle software, weapon system software, etc. In each of these cases there is often explicit analysis of what the all the exceptions are and if they need to be covered. These kinds of projects often end up spending alot more time and money to cover these cases. That is the cost of covering "all" exception cases.

No

Are you working on a social networking software. A Facebook app, an iPhone app, some motorcycle enthusiast site, etc. This kind of software doesn't usually warrant the kind of consideration I talked about above. Usually there is a cost to getting to the market late, and covering even some of the exceptional cases may be prohibitively expensive. This can often be a decision that will have business implications, especially in a startup. Do you cover all those cases and drive your company out of business, or lose a large portion of revenue?

It Depends

So the answer is "it depends". It depends on the needs of the company, the cost of those exceptions, the cost of the extra time and effort required to guard against those exceptions, and the time involved.

Ideally

Ideally you strike a balance. Using practices like automated tests to get at least as much coverage on your code as possible without too much cost in time or money. You can use practices like unit testing, functional testing, fuzz testing, etc; so that the software isn't too buggy, that it's easy to maintain and extend, and so you can look at it and take some pride in what you wrote.

Answer 6

The most important thing is that things will happen that you have not anticipated, and hence not coded for.

It is very important that your program handles this correctly. You must choose if "something unexpected happened" is bad enough that the program should shut down gracefully immediately for manual inspection or what else might be the correct procedure.

I have personally found that the "fail fast" approach is the best way to get good program behaviour after a few iterations.