Selenium and non technical team members

Question

I work on a team developing a web site with a large number of pages. The QA guy on the team currently runs all of the tests manually. I am thinking of having him create a bunch of Selenium scripts. I don't think he can automate all of the tests, but he could automate many of them.

Because he doesn't have a programming background, how hard will it be for him to create and manage a batch of Selenium scripts? What issues will he need help with?

Answer 1

I think Selenium as a stand-alone "record and replay" tool is ideal for non-technical users. The idiom of recording my path and then replaying it is intuitive, and the interface, while obviously geared toward non-newbs, isn't too intimidating.

Selenium 2 / Selenium RC is obviously a higher order of complexity, and I don't think you can ask a non-programmer to interface with that. (I've been frustrated with the lack of full interface for non-Java languages, actually. I know the current release is transitional, but I really could use a full PHP solution right now, not in some mythical future release. But that's not what you're asking.)

I think if your tester guy doesn't already know about Selenium, he'll kiss you square on the mouth for telling him about it.

Answer 2

Selenium-IDE is designed for the purpose of being used by non-programmers. But my experience is that the scripts that are produced that way (using generated XPath) are very brittle.

So create in them should be no problem at all. Managing will be not-so-bad if your application is fairly stable. Or nearly impossible if your application is in development and/or changing frequently.

Selenium-RC is purely a developer tool.

Answer 3

I strongly recommend you ask this question on the Software Quality Assurance & Testing beta site, or search for similar questions that have already been asked there. You will find a much greater depth of experience with using Selenium for record-and-replay QA.

Eric's answer is the experience of most QA Engineers who used record-and-playback tools: The resulting tests were brittle and difficult to maintain. A developer can use Page Objects with Selenium RC to make a far more maintainable UI test suite, and is a better option when it is available.

This isn't to say that Selenium IDE won't be useful for your tester; however, the usefulness (and the potential to hurt testing efforts rather than help) strongly depends on how fixed the UI is. You will get your best results with record-and-playback if the UI can be frozen early in the development cycle. You may need to teach the tester how to get new XPaths when the old XPaths are broken, and how to tell if the test is failing due to a UI change (because the XPath is no longer valid) or because of a functionality bug.

I do think using Selenium IDE is better than no automation at all for any project that has some generally-stable UI. Running hundreds of manual tests on stable UI every time that there is a minor change is a real time-sink. Just set expectations accordingly and realize that changing the UI will have a significantly bigger impact on the QA schedule once you start using record-and-replay automation for that UI. Your tester may want to pick-and-choose the areas s/he automates wisely.

Answer 4

I've had no problems training non-programming folk to write Selenium tests with relative XPaths in Java/JUnit. The programmer sets it up and writes a basic test template, the tester develops more tests - by recording and then tweaking them. If tester needs anything specific, he asks programmer to write it. Tester needs to have a really basic understanding of HTML and XPath, and a bit of help to pick up Selenium API (as well as Selenium plugin for Firefox and Firebug).

The funny thing is, at first I was considering something fancy like Cucumber or something similar, but this Selenium/JUnit combination proved good enough and simple enough. Curiously, testers without previous Java experience had no serious problems writing JUnit tests. The only important thing is that experienced programmer sets up the test, so that the tester basically just needs to produce new tests and call Selenium methods.