Tag Info

Hot answers tagged

17

A few of my thoughts: Be honest that writing automated tests will take more time. If you're doing unit level TDD (which I would recommend as a starting point if you're going to invest in automated testing), you can expect about 30% extra time needed to code a feature. The key here is explaining that this extra 30% (which is probably higher than 30% in ...


15

When my team implemented automated UI testing a lot of great things happened. First, the QA team became much more efficient at testing the application as well as more proficient with the application. The lead QA said that he was able to bring new QA members up to speed quickly by introducing them to the test suites for the UI. Second, the quality of QA ...


10

If the tests aren't 100% automated, they lose a tremendous amount of value. They won't be run reliably, and are a lot more error prone because of the human interaction. If you can't get it 100% automated, remove that test completely, even if it means not testing that piece of functionality for now. As an aside, have you asked for help on stackoverflow? ...


10

and we don't really want testers writing too much code We took the opposite approach. We wanted the testers writing code. Here's the workflow we started to adopt. It's not easy to do this because management doesn't absolutely depend on automated testing of the front-end. They're willing to settle for "close-enough". User stories. Operational ...


9

Acceptance tests act at a very different level than, say, unit tests. A unit test is very precise: it deals with one method, sometimes a part of a method. This makes it a perfect choice for regression testing. You make a change. A test fails while it passed during the previous commit. Great, you can easily pinpoint the source of the regression both in time ...


8

I think Selenium as a stand-alone "record and replay" tool is ideal for non-technical users. The idiom of recording my path and then replaying it is intuitive, and the interface, while obviously geared toward non-newbs, isn't too intimidating. Selenium 2 / Selenium RC is obviously a higher order of complexity, and I don't think you can ask a non-programmer ...


7

One thing of definite value is that automate tests can be run continuously; such as every hour on a rebuild or similar. Doing this forces any bugs or regressions out into the open rapidly within hours or days of a programmer working on the offending code, this makes context switching much easier. The second benefit to continuous testing is that it forces ...


7

Automated UI tests are the real integration tests. They test the entire system in the way it's actually used when it's live. That makes them the most meaningful tests. However, they also tend to be the most brittle, and the slowest to execute. Keep an eye on the cost/benefit ratio (with brittleness being a part of the cost) and don't hestitate to have some ...


6

Test Expense Once an automatic test is written, it can be run by a computer at the cost of a few joules. The equivalent manual test requires a person on payroll working down a list of instructions. Test Reliability The computer can be trusted to faithfully execute the same test procedure, every time. The human is apt to make mistakes and get lazy. The ...


6

The purpose of Selenium is to create UI-driven integration tests. Integration tests verify that all components of your system work correctly when deployed together. Integration tests are not a sufficient test strategy and complement other test strategies having a different focus, for example unit-testing and acceptance testing. UI-driven tests are ...


6

As far as I can tell, your example closely matches the strong coupling, as it is defined in Wikipedia: Strong coupling occurs when a dependent class contains a pointer directly to a concrete class which provides the required behavior. The dependency cannot be substituted, or its "signature" changed, without requiring a change to the dependent class. ...


5

Run Selenium from a separate programming language's testing framework. Guaranteeing the evironment is OK without manually doing any work is really hard to do if you're using the selenium recorder and playing back in their standard test language. You'll need to export the code from the testrunner (or write it yourself) using a language like Java or C# where ...


5

Unit testing is just one of the multiple types of testing. Basically, you'll find three types of tests: Unit tests. Each of those tests ensure that a tiny piece of application is working as expected. For example, a unit test will ensure that if, on an e-commerce website, I create an instance of a Product class with a price inferior to zero, an exception ...


4

So far the main benefit we can see is for regression testing, especially across multiple client deployments of our platform. Automating your regression testing is a good thing. This frees up your testers to do more interesting work - be this adding more automated tests, stress testing your application, or any number of other things. Also, by making it ...


4

The most important thing when creating tests like this once you get past a trivial number is the idea of symmetric change - a small change in the code should result in a small change in the test suite. For example, let's say you collect someone's name with two text boxes in 100 tests. If you write those tests niavely (or maybe are using record-playback) ...


4

Mostly (depending on your test coverage) bug-free code, and I would say that one of the biggest arguments is when you say to your manager that you can write a test for a discovered bug, ensuring you will always know in the future if that bug comes back :) My opinion is that unit / integration tests are most important, while if you apply some UI pattern like ...


4

Selenium-IDE is designed for the purpose of being used by non-programmers. But my experience is that the scripts that are produced that way (using generated XPath) are very brittle. So create in them should be no problem at all. Managing will be not-so-bad if your application is fairly stable. Or nearly impossible if your application is in development ...


4

Page Object elements should be private. The issue, especially with respect to the Page Object pattern, is that public objects expose their methods, not the page object's methods. If the page object's method would just be clickPrintButton(), there's not much difference, but imagine instead a printOrder() method that needs to click the button and wait for a ...


4

UI testing is hard. For the most part, it doesn't lend itself well to unit testing. Tools like Selenium are designed to fill this gap. For the most part, they do this by automating the user interface, simulating the operations that a user might perform via mouse clicks and typing. But these tests tend to be brittle, and they are generally not ...


4

Generic functions should not go into classes. Python is not Java, it does not require pure functions to live inside classes. Only use classes if you want to store state of some kind. Put the functions into a standalone utility module, and import that into the module that contains your test classes.


3

I strongly recommend you ask this question on the Software Quality Assurance & Testing beta site, or search for similar questions that have already been asked there. You will find a much greater depth of experience with using Selenium for record-and-replay QA. Eric's answer is the experience of most QA Engineers who used record-and-playback tools: The ...


3

Not all of the testers are programmer savvy. Teach them. Or, if some are bright enough and others not, separate them out into automated testing and support (mostly replicating problems that others have found). This might sound like an onerous task, but it's really not that hard. They don't need to know about hard-core programming to write tests, simply ...


3

The number of files comprising your tests will be mostly irrelevant - The time spent running the tests themselves will be much greater than the time spent loading and compiling the files. Instead try to keep your test code manageable - group related tests together, and try to keep the files withion the very rough ballpark of 50-500 lines.


3

Test code is code that needs to be read, executed and maintained, just like production code, so it should be subjected to the same standards as production code. Just use inheritance, interfaces, whatnot, where it makes sense and where it helps you build a maintainable selenium script. P.S.: the PageObject is a commonly used pattern for Selenium, where you ...


3

There is no "should". Testing is a tool. It's like asking "Should an artist always use oil-based paint". The answer is (of course!) "it depends". The goal of tests are to a) test the product, and b) be well engineered. You can write well-engineered tests in many styles. Your team should do what is most compatible with their skills.


3

At some point in time, someone has to make a decision how the UI has to look like. If this is done iteratively step-by-step in a dicussion between the customer and your team, fine - that is a feasible, pragmatic way. Whenever the decision is made for a particular part of your UI, you can start creating the tests. I end up creating the Automated test code ...


2

You won't be testing if Google Graphs work, because that's not your job. What you should test, is if you're sending correct API calls to the external service and if given a well formated reply your application process it as expected.


2

You're right that regression is a huge one. Also - if your tests are written modularly, you can get more bang for the buck by mixing and matching test sets we've reused automated test scripts for data load so that we don't have to kludge a database to do large size testing performance test multi thread tests on web systems - swapping between browsers ...


2

We've looked at Selenium and Telerik and have settled on the latter as the tool of choice due to its much more flexible recorder I'm not sure how much you've looked into it. There certainly are other options as well. Have you looked into Watir, WatiN, Sikuli to name a few? and we don't really want testers writing too much code. I feel bad for ...


2

I think you answered your own question in your question when you asserted "If the tests rely on the framework to define the expected results and then verify them against the output on the webpage, then if the logic in the framework is incorrect, then the defined expected results will also be incorrect and there is a risk that the tests will pass even though ...



Only top voted, non community-wiki answers of a minimum length are eligible