Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

We use a detailed step-by-step user-interface regression test for our commercial web application. It has a "backbone" test for the most used / most important parts of the system, with optional tests for specific areas of functionality. Using this plan has definitely helped us ensure high quality software.

But, having very specific tests can be counter-productive. The tester concentrates on following the test and will completely miss usability issues, or not notice fairly obvious problems such as the bottom part of a page that is missing.

By contrast, some of the best UI testing happens when building a demo of a new feature. I often do my own best testing by pretending to demonstrate the system to an imaginary prospect. Yet when I tell the testers, "Just demonstrate the system to yourself" they don't cover nearly as much functionality as they do with a detailed point-by-point test.

I'm repeatedly asked to provide more and more detail in the test plan so that a new untrained tester can test with it without asking any questions. Yet details seem to be counter-productive. How much detail do you put in a regression test to make it effective? How do you write tests that make the tester focus more on the system than on checking off items on the test?

share|improve this question
    
I have a feeling that you know the answer better than anyone else –  superM Aug 31 '12 at 13:33
    
What answer is that? Test it myself? I'm posting here in hopes of finding a different answer! :-) –  GlenPeterson Aug 31 '12 at 15:41
1  
First, its not an answer but a comment. Second, no one here can know your team better than you do. So, since you tried many things and were looking for even more, I assumed you'll be able to know what's best for your team!!! –  superM Aug 31 '12 at 19:15
    
Should I have posted this on the User Interface forum instead? –  GlenPeterson Sep 13 '12 at 14:16

2 Answers 2

Technically speaking, a regression test should not be looking for usability issues. It may catch deviations from the UI design as part of the test cases but usability issues are best determined by UI design specialists. UI regression tests are usually performed by QA (testers), as you know.

If the product is too big for a detailed UI design, the UI designer produce should produce guidelines that the development team should follow and the QA should check for. For example, "Each page should contain a header with A, B and C" or "When asking for an address, it should have these fields laid out in this way".

It may be better to supply the UI verification steps in a different way. For example, instead of giving steps, include screen shots of that the app should look like (which should catch the bottom portion of a page missing).

You may also want to consider an automated UI testing tool like Selenium. This can automate much of the repetitive testing aspect you are capturing in the steps and free you and your testers up to look for harder to reproduce or more critical issues.

share|improve this answer
    
Ooh - Selenium looks pretty cool! To look for usability issues, do you do Usability Testing, Beta Testing, or what? Sometimes we change something and it makes a distant part of the system awkward/misleading/un-intuitive. –  GlenPeterson Aug 31 '12 at 15:37
    
In my organization, we have a specialist UI design team that development teams can use if needed. The UI design team, product managers and architects iterate until we have a good UI design, which is then implemented. If there are questions or contradictions, the product managers and architects can choose to involve the UI design team again to fix them. This may or may not work in your case. –  akton Aug 31 '12 at 15:56

You cannot command someone to be innovative. If you have only testers in your team without developing own ideas for good tests, I don't think a better test plan will solve that problem for you. Either they need more experience and training, or you need different people in your team.

Said that, I think you should have both:

  1. a very detailed step-by-step check list of things where you want testers to have an eye on

  2. some general questions / guidelines, for example, about testing usability or about things not explicitly mentioned in your plan, or a demand for creating missing points in the checklist by themselves.

Make clear that you expect your team not only to focus on point 1, but also on point 2, but don't expect too much because of what I wrote at the beginning.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.