Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

What document types could be used when describing software testing automation (on any phase, automation planning, test design, implementation, reporting)? Is there any standard types of documents? If someone could provide any examples and templates for different types of documents? What is the difference between manual testing documentation and automated?

share|improve this question

migrated from stackoverflow.com Oct 21 '11 at 20:13

This question came from our site for professional and enthusiast programmers.

4 Answers 4

Here is what i would propose:

1. Test strategy document:
This outlines the over all testing objectives, what testing goals exists and how is the over all testing will be performed linking all levels from unit test, component test, system test and integration test. This is not a standard or something - but it can be something of this sort.

2. Test suit:
This is the collection of test cases and conditions on when and how each cases needs to be performed. Each set of inputs, procedures and expected output behavior against each elements. There are times when more than just success and failures are noted so that further analysis is done on this.

3. Test environment/setup and procedures
If you are automating the testing process fully or partly, it is worthwhile to document how exactly the (various elements of) testing is going to get executed. It should be debated and validated if the testing performed here is correct method or not. The developer and QA associated should know how to operate the set of tools and what procedures to follow.

4. Traceability Matrix:
This is a well defined matrix which identifies which set of test cases are relevant to promise each functionality point is assured to be steady. Read more here or this wiki link. Whenever a new bug is discovered or a new feature is requested, traceability matrix should be updated to capture these changes.

5. Test results
Whether generated automated or performed manually, the results (detailed and summerised) should be captured in a test execution sheet. Most important thing to note down is that a. original observation (such as logs, actual output of the application) should be capture as relevant so that conclusions can be validate. b. the document needs to capture the build against which these test were carried out; different build may not produce the same behavior against the same test.

Procedure and formats can be developed as needed. Most important thing, from my personal experience is that instead of making water tight compliance to some format allow people do document this like a running diary making only few things mandatory and let people pour in more information freely. Testing is never static (at least for any reasonably complex project) so over time all these template must evolve continuously - quite often every next step could be a major departure from the last one. If either the templates are stale, or people do not follow that because it is too rigid, eventually much of the relevant knowledge through testing procedure won't be reflected right.

share|improve this answer

The answer greatly depends on what the software does, how it is designed, and even the industry the software will be used in. Some industries have very extensive regulatory requirements for test documentation & validation, for example FDA CFR 21 Part 11in the Pharma and Biotech industries.

Also, consider that even if your organization would religiously follow all the relevant IEEE standards for testing documentation, such as:

  • IEEE 829: software test documentation
  • IEEE 1008: unit testing
  • IEEE 1012: software verification and validation
  • IEEE 1028: software inspections
  • IEEE 1044: classification of software anomalies

there is a plethora of directly and indirectly related documents and standards you should also define and use, like:

  • IEEE 830: system requirements specifications
  • IEEE 730: software quality assurance plans
  • IEEE 1061: software quality metrics and methodology
  • IEEE 12207: software life cycle processes and life cycle data

If all of those IE3 standards look insurmountable - especially for smaller organizations and teams - have a look at the Software Engineering Body of Knowledge which is less "scientific" and more concise.

Finally, to some extent, using ALM tools that keep electronic records of tests in stead of formal documents is a perfectly valid alternative as long as you have procedures that define a standardized use.

share|improve this answer

In my project, the automation tests are essentially documented using the same means as manual tests. In addition to all the other information, we also point to the test case in code.

We maintain an automation suite which the whole team knows how to execute. With both these pieces of information, any member on the team is able to identify the test case as well as run it.

Also, like our production code, we also version and review the automated test cases. These serve as living documentation for the code representation of the test.

So in a way, we are reusing our documentation ways for manual tests and production code for automated tests.

share|improve this answer

Did you look at IEEE 829-1998, Standard for Software Test Documentation? I know that in my company the test documents in this standard are used to describe the entire testing strategy. For example, the Master Test Plan will actually include information on testing automation in the approach section.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.