What document types could be used when describing software testing automation (on any phase, automation planning, test design, implementation, reporting)? Is there any standard types of documents? If someone could provide any examples and templates for different types of documents? What is the difference between manual testing documentation and automated?
migrated from stackoverflow.com Oct 21 '11 at 20:13
Here is what i would propose:
1. Test strategy document:
2. Test suit:
3. Test environment/setup and procedures
4. Traceability Matrix:
5. Test results
Procedure and formats can be developed as needed. Most important thing, from my personal experience is that instead of making water tight compliance to some format allow people do document this like a running diary making only few things mandatory and let people pour in more information freely. Testing is never static (at least for any reasonably complex project) so over time all these template must evolve continuously - quite often every next step could be a major departure from the last one. If either the templates are stale, or people do not follow that because it is too rigid, eventually much of the relevant knowledge through testing procedure won't be reflected right.
The answer greatly depends on what the software does, how it is designed, and even the industry the software will be used in. Some industries have very extensive regulatory requirements for test documentation & validation, for example FDA CFR 21 Part 11in the Pharma and Biotech industries.
Also, consider that even if your organization would religiously follow all the relevant IEEE standards for testing documentation, such as:
there is a plethora of directly and indirectly related documents and standards you should also define and use, like:
If all of those IE3 standards look insurmountable - especially for smaller organizations and teams - have a look at the Software Engineering Body of Knowledge which is less "scientific" and more concise.
Finally, to some extent, using ALM tools that keep electronic records of tests in stead of formal documents is a perfectly valid alternative as long as you have procedures that define a standardized use.
In my project, the automation tests are essentially documented using the same means as manual tests. In addition to all the other information, we also point to the test case in code.
We maintain an automation suite which the whole team knows how to execute. With both these pieces of information, any member on the team is able to identify the test case as well as run it.
Also, like our production code, we also version and review the automated test cases. These serve as living documentation for the code representation of the test.
So in a way, we are reusing our documentation ways for manual tests and production code for automated tests.
Did you look at IEEE 829-1998, Standard for Software Test Documentation? I know that in my company the test documents in this standard are used to describe the entire testing strategy. For example, the Master Test Plan will actually include information on testing automation in the approach section.