Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I am having a design problem in test automation:-

Requirements - Need to test different servers (using unix console and not GUI) through automation framework. Tests which I'm going to run - Unit, System, Integration

Question: While designing a test case, I am thinking that a Test Case should be a part of a test suite (test suite is a class), just as we have in Python's pyunit framework. But, should we keep test cases as functions for a scalable automation framework or should be keep test cases as separate classes(each having their own setup, run and teardown methods) ? From automation perspective, Is the idea of having a test case as a class more scalable, maintainable or as a function?

share|improve this question
add comment

3 Answers

In the past, a test case usually referred to a function / method. However, in our days, a Test Case usually refers to a "Test", meaning a class containing a set of methods which are strongly related to each-other from a testing point of view.

Methods in these Test Case classes are called Test Methods.

Considering this, the inclusion would be. Test Method (testing a specific single thing - state or behavior) is part of a Test Case (which refers probably to a functionality or submodule) which in turn is part of a Test Suite (which refers to a bunch of test cases and can be organized as you wish).

For more on these structures and test organization in general I recommend you xUnit Test Patterns: Refactoring Test Code by Gerard Meszaros

share|improve this answer
add comment

The following is my opinion:

Pros of writing tests as functions:

  • If you need any pre-requisites for that test case, just call another function which provides the pre-requisites. Do the same thing for teardown steps
  • Looks simple for a new person in the team. Easy to undertstand what is happening by looking into tests as functions

Cons of writing tests as functions:

  • Not maintainable - Because if there are huge number of tests where same kind of pre-requisites are required, the test case author has to maintain calling each pre-requisite function in the test case. Same for each teardown inside the test case

  • If there are so many calls to such a pre-requisite function inside many test cases, and if anything changes in the product functionality etc, you have to manually make efforts in many places again.

Pros of writing test cases as classes:

  • Setup, run and teardown are clearly defined. the test pre-requisites are easily understood
  • If there is Test 1 which is does something and the result of Test 1 is used as a setup pre-requisite in Test 2 and 3, its easy to just inherit from Test 1, and call its setup, run a teardown methods first, and then, continue your tests. This helps make the tests independent of each other. Here, you dont need to make efforts to maintain the actual calling of your code. It will be done implicitly because of inheritance.
  • Sometimes, if the setup method of Test 1 and run method of Test 2 might become the pre-requisites of another Test 3. In that case, just inherit from both of Test 1 and Test 2 classes and in the Test 3's setup method, call the setup of Test 1 and run of Test 2. Again you dont need to need to maintain the calling of the actual code, because you are calling the setup and run methods, which are tried and tested from the framework perspective.

Cons of writing test case as classes:

  • When the number of tests increase, you cant look into a particular test and say what it does, because it may have inherited so much levels that you cant back track. But, there is a solution around it - Write doc strings in each setup, run, teardown method of each test case. And, write a custom wrapper to generate doc strings for each test case. While/After inheriting, you should provide an option to add/Remove the docstring of a particular function (setup, run, teardown) to the inherited function. This way, you can just run that wrapper and get information about a test case from its doc-strings
share|improve this answer
add comment

I prefer Testcase as a class because

  • In Untitests you usally have several tests with a similar context that would logicaly grouped together by a class.
  • In Integration and System-Tests you usally have a kind of global variables that are updated/verified in the test-steps. If these global variables are not members of a class you are not able to run several tests concurrently (for performance or stresstest) because the tests would influence each other.

Actually i dont see any benefit of a non-class-testmethod.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.