Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am reading "The Pragmatic Programmer" . The book has the following paragraph

Dave once worked on an international telex switch. Quite understandably, the client demanded an exhaustive test specification and required that the software pass all tests on each delivery. To ensure that the tests accurately reflected the specification, the team generated them programmatically from the document itself. When the client amended their specification, the test suite changed automatically. Once the team convinced the client that the procedure was sound, generating acceptance tests typically took only a few seconds.

I am curious to know how he created tests from specification . What format was the specification in ? How to write a program that will automatically read the specification and write test cases. Have I misunderstood the above paragraph ?

share|improve this question
Although I don't know whether they used FitNesse or something else, it is worth having a look at. – Péter Török May 18 '11 at 9:29
up vote 9 down vote accepted

What format was the specification in ?

Not informal English, you can be sure.

There are a lot of more formal specification languages.

Start here:

Z is popular for this kind of thing.

To an extent, one can use UML for this. Although it's rarely rich enough to lead to unit tests. And if the UML picture is rich enough to lead to unit tests, it can also lead directly to working code.

However, I've been very successful using spreadsheets. Yes. Plain old spreadsheets.

Most folks who use ordinary business software can prepare a spreadsheet of examples.

How to write a program that will automatically read the specification and write test cases.

  1. Parse the formal specification language.

  2. Emit unit tests in your favorite programming language based on the specification.

This isn't as easy as it seems because it still requires considerable manual effort to work out detailed examples of the edge and corner cases.

Total automation from specification to test cases with no manual intervention is only possible when the changes are relatively minor or the test data is trivial to generate.

Phone switches are (just) state machines, so it's not difficult to work out example cases.

For accounting software, however, (or actuarial or financial modeling or econometrics, stuff I've been doing recently) it's a bit dicey to find examples "automatically". It usually requires a spreadsheet showing the edge case and how it should be handled.

share|improve this answer

Given the state of natural language processing, this can only have worked if the document was already in a nearly-machine-readable state. It may have listed the expected output signals for every desired combination of inputs (like the specification for an electric circuit) or given a similar exhaustive description in repetitive language. (Perhaps it was even automatically generated itself!) In general, you cannot transform a spec into tests or code automatically. That's why programming is still necessary, after all.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.