Should a test plan account for configuration values?

Question

Should a test plan for internal software account for the error states of all possible configurations?

Ie: I have an app.config with some file paths. Should the test plan have steps for testing what happens when those values are incorrect?

Answer 1

Two schools of thought on this:

1. If it's likely to cause an issue, then yes it should.

2. On the other hand, since it's an internal app, you may not want to spend too much effort dotting every I and crossing every T.

The last one hinges on the scope of the software. If it's just a simple utility that makes something a bit easier to do, then it may be fine without very robust error handling. If it will get a good amount of use, especially by many different people, then it should be relatively robust as the users will likely come to depend on it.

Answer 2

Consider creating a tool that generates a random, but valid, configuration, to be used as part of your test scripts.

Answer 3

In an ideal world, yes.

Even if it an internal app, you can't control its use by users and the more it is used, the more it will probably be used in ways you hadn't thought of.

However, we don't always live in an ideal world, so justifying the "budget" may be tricky - "it works, move on".

At the very least, I would advise implementing some form of logging so when (if?) problems do arise, you have a head start on recreating them. As it is an internal app, getting hold of the log files shouldn't be a problem.

Answer 4

Yes.

(Unless you want the internal users calling you every time there is an error in the install)