Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

If "Anything can go wrong will go wrong" is true, we need to test all the conditionals and exceptional cases in my code.

But sometimes it's hard to find all of them since many of them are corner cases, which are hard to test. How can I go about finding corner cases better or avoiding the issue?

share|improve this question
    
This is a really broad question. Can you specify as to what type of code/cases you're worried about? For example, something like "validate input" would be useful if your method uses user input, but if it doesn't, then "validate input" might not be a good suggestion. –  BlackJack Sep 3 '11 at 6:09
5  
If it's really true, everything that can go wrong about your tests will go wrong as well ;) –  delnan Sep 3 '11 at 9:20

6 Answers 6

up vote 2 down vote accepted

Yes! Every block of code should be tested. But how do you test it? By Unit Testing in first place,

well, to let you code to be testable, it is another theory, it is a kind of programming technique. You can't let your code become smell.

  1. Test your code part by part by function.
  2. Your function must be specific, one purpose, simple, not complex.
    e.g. don't make Global variable in your function, otherwise (not testable)
  3. Your function must be understandable.
  4. Your function is loosely coupling, less dependency, so that you can test it easily. etc.etc....

Once your code style is improved, your test cases , corner cases would be easily to be tested.

Here is a good article of how to make your code become less smell. Hope it helps.

share|improve this answer
    
Code smell, what a good word! –  panda Sep 3 '11 at 8:43
1  
-1: Testing certain exceptional cases maybe prohibitively expensive. What may make sense to test if you working on Space Shuttle or surgery software may not make sense if your working on a Facebook app. There is alot more to determining what should and should not be tested depending on the context the software will be used in. –  dietbuddha Sep 3 '11 at 16:54
    
@dietbuddha: Those exceptional cases are almost always detected by a library class. They are easily tested by mocking the library class. –  kevin cline Sep 4 '11 at 19:02
1  
@kevin cline: Context also matters when you are assuming the libs you use will respond correctly to exceptional cases. You wouldn't want the x-ray machine to stay on because the lib causes your program to die before it gets to the shutoff code. –  dietbuddha Sep 5 '11 at 0:02
    
If loss of life or property is a possibility I would not only test the library code, I would inspect it. –  kevin cline Sep 5 '11 at 19:16

Consider code coverage analysis. In my experience it was pretty efficient way to discover un-tested use cases.

When tests already have high code coverage, using this approach to discover holes may be fun, too - because remaining cases turn out rather non-obvious (or, even more interesting, might correspond to bugs in specification).

  • Just don't assume that coverage analysis will guarantee to cover all the relevant cases. Eg it won't protect you from missing feature or concurrency issues.

Thing to keep in mind is that it may be tricky to drill into every possible case. Imagine an application designed to output "Hello World" if and only if it's Thursday and it rains - to test it, you would have to wait until Thursday when it rains? or mock the necessary conditions. That may be too much effort to justify the risks you'd avoid by having it covered in tests.

Another thing to consider is how much time testing takes. If it takes a year to fully test your application then you better drop that idea (unless you're on mission critical product)

share|improve this answer
3  
Code coverage is a start, but sometimes misleading, since even 100% coverage does not guarantee that you have tested every relevant case. For example, if I forget to add some necessary error handling to a function in my code, I can get 100% code coverage by unit tests using the the input data I was thinking of, but the code is buggy either since I overlooked a different case of input data. –  Doc Brown Sep 3 '11 at 9:21
    
@DocBrown good catch I'll update my answer –  gnat Sep 3 '11 at 9:22

I find this very true. I sometimes tend to be lazy and dont test certain lines of code, but almost never fails to come back and hit me.

If you are writing OO code. Here are the inputs based on my learning.

  • Use Occam's Razor. That is find the simplest way to implement it.
  • Divide anything that can be divided into multiple classes or mixins until you see that each class or mixin is doing exactly one thing. That is respect SRP.
  • For every mixin/class write atleast one test case initially covering every function.
  • If you are using a few third party library see if there are chances that those functions can return unexpected results from the documentation.
  • Now look at each if/for/while/case and similar constructs to see if that is covered in your test case. If not write more test cases.
  • Check if any line can fail due to stuffs like NullPointerException or object not defined type of errors. Typically lines after some sort of object creation or stuff that depend on using library to set your object.
  • Use Black box testing, where you test every function weather its available public or not. Most testing frameworks believe that testing public interfaces will do(white box testing) and some languages wont give you access to private interfaces. In these situations try to test the private functions through public functions.
  • Use some kind of automated tools that can test for code coverage.

Even after this there will be some scenario's that one cannot easily test, especially the environment related stuff. You can either get the code peer reviewed by an experienced programmer or run your test cases in all possible environments.

One can never be fully confident that every case is covered. If you have spent a lot of time with the code while its developed you will gain confidence on it. Once you have enough confidence, release it bravely. Things will come back, since you would have read your code a 100 times, you can fix stuff really quickly and will be able to imagine other places this will happen. For every bug that comes back, cover it with test cases.

share|improve this answer
    
I think you're reversed between black box testing (not peeking inside) and white box testing (peek inside) –  Lie Ryan Sep 3 '11 at 12:55

If everyone knew how to account for everything, there would basically be one version of software, no maintenance would be needed, testers wouldn't be required, etc. You can't follow a set of guidelines that will give you a magic black-and-white answer because the list of items to test is based on a multi-variable component (code).

The best thing you can do is learn. You'll make mistakes and forget to do things; make a conscious effort not to make the same mistake twice. If you can do this, you're probably better than the average software engineer.

share|improve this answer

Maybe

You cannot account for all exceptional cases unless you have a ton of time and money. The more interconnects in a software system the more complex it is and the more it take to cover even the standard test cases. We aren't even necessarily talking about code you wrote. Remember anytime your software isn't operating purely out of memory (that doesn't grow) you will have many more exception cases. Accessing network, disk, display, any other devices can all cause an exception. Growing memory can cause an exception. In some cases even detecting that the software was improperly stopped may be an exception case. The question is do you really need to cover every exception case.

Yes

If your working on software where lives hang in the balance, literally. Then yeah, you probably do need to at least consider every exception cases. Surgery software, Space Shuttle software, weapon system software, etc. In each of these cases there is often explicit analysis of what the all the exceptions are and if they need to be covered. These kinds of projects often end up spending alot more time and money to cover these cases. That is the cost of covering "all" exception cases.

No

Are you working on a social networking software. A Facebook app, an iPhone app, some motorcycle enthusiast site, etc. This kind of software doesn't usually warrant the kind of consideration I talked about above. Usually there is a cost to getting to the market late, and covering even some of the exceptional cases may be prohibitively expensive. This can often be a decision that will have business implications, especially in a startup. Do you cover all those cases and drive your company out of business, or lose a large portion of revenue?

It Depends

So the answer is "it depends". It depends on the needs of the company, the cost of those exceptions, the cost of the extra time and effort required to guard against those exceptions, and the time involved.

Ideally

Ideally you strike a balance. Using practices like automated tests to get at least as much coverage on your code as possible without too much cost in time or money. You can use practices like unit testing, functional testing, fuzz testing, etc; so that the software isn't too buggy, that it's easy to maintain and extend, and so you can look at it and take some pride in what you wrote.

share|improve this answer

The most important thing is that things will happen that you have not anticipated, and hence not coded for.

It is very important that your program handles this correctly. You must choose if "something unexpected happened" is bad enough that the program should shut down gracefully immediately for manual inspection or what else might be the correct procedure.

I have personally found that the "fail fast" approach is the best way to get good program behaviour after a few iterations.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.