Hot answers tagged acceptance-testing
25
One important factor which makes unit tests extremely useful is fast feedback.
Consider what happens when you have your app fully covered with integration/System/functional tests (which is already an ideal situation, far from reality in most development shops). These are often run by a dedicated testing team.
You commit a change to the SCM repo,
sometime ...
21
Your team needs to do the data conversion for them. You really should have done it for them in the first place.
I've been involved in a number of expensive platform migrations and the vendor always, always has their own data conversion team who are responsible for understanding the legacy system, writing all the migration scripts, doing all the tests, and ...
13
Maybe I'm too old-fashioned, but even the most modern development or proccess techniques cannot substitute another set of eyes, fresh eyes, before releasing a product to your client.
Even if your product is simply an API for another developer, you can use QA to think as the API user, providing test/use scenarios that you or your client did not think in ...
10
I worked in the car navigation field over a decade ago.
Step A)
Use a reference package and select a large sample set, run A/B tests. Not looking for exactness, looking for outliers - The reference set showed Reroute 1234 as 10.34km, and we calculated 123.5km.
Step B)
- Refine our software and the reference software
- Add more samples and reduce the ...
10
End-to-end tests are also necessary. How else can you know you hooked up all the units together correctly? On very simple code, it is possible to test all the paths through the code with only end-to-end tests, but as you get more layers, it becomes prohibitively more costly to do so.
For example, say you have three layers, each with five possible paths. ...
9
The test plan should NOT be written by developers. Part of what the test plan is to do is to check to see if the developer correctly interpreted the requirement. A developer cannot effectively write a test plan on the code he is going to write. Test plans should be written by the people who are going to be doing the QA or by the business analysts. If ...
9
Is it true to say that the product manager and the QA should be responsible for taking the user-story and breaking it down to acceptance tests?
Mostly. They may not actually write the actual acceptance test. They may approve something you wrote. But they approve the acceptance tests. Yes.
the acceptance tests need to be formal, so they can be ...
8
Acceptance Test-Driven Planning is an extension of XP Planning. It involves the following steps:
Getting our story straight
Show and tell
Retrospective
(optional) Technical Retrospective
Iteration Planning Workshop
(optional) Big, up-front thinking
Cutting the iteration.
This planning process is supposed to take up about 10% of the time, the development ...
7
We use Robot Framework extensively and Fitnesse a little.
Fitnesse I think has an edge if you like the wiki interface that it exposes to people writing the tests - it's particularly good if your data consists of tables of primitives with expected inputs and outputs. Once you get beyond the basics though it gets clunkier and clunkier.
Robot Framework is ...
7
100% coverage is not the same as 100% tested.
I'd see a QA person in a ATDD project as someone that would help write the tests and perform the other types of testing that still exists. I.e. UI Testing, destruction testing and load/stress tests.
But I've never worked out an ATDD project.
7
It personally sounds as if your Acceptance tests have encompassed properties of Integration tests and that you are trying to "kill two birds with one stone" as the saying goes.
In the traditional Waterfall model a single Acceptance test should determine if a single requirement has been met. If developing based on a strict SRS document, you may find that ...
7
Let's suppose that it works for email, but not name.
Now your acceptance test will fail half the time. Just re-running the test will potentially change the result. That'll make it harder to notice errors and track down the problem with it occurs. It'll be way easier if you just write two tests, one for each login method.
7
When there is a dispute between the users and the program about how the program should work. The solution isn't to fix the users. That goes double when you're presenting data to domain experts.
In this case if the devs and the users have different ideas about what data should be presented, then correct response is to make the data presented match the user's ...
6
QA's job is to break the application, the devs job is to not break it. Therefore they write their tests from a different perspective. For instance the devs write tests to see if the expected behavior happens, QA writes tests to see what happens when the users does something the developer would never consider the user would do. Further, developers often ...
5
Typically, an acceptance test is written against a requirement. Regardless of how you capture requirements, you should have a set of things that your system must be able to do, expressed as a list in an Software Requirements Specification (SRS) or as a collection of user stories, among other options. Each requirement should have an acceptance test associated ...
5
All types of testing are very important, and ensure different aspects of the system are in spec. So to work backwards, "If I had to choose one type of testing..." I wouldn't. Unit testing provides me different feedback than integration testing or interactive testing by a person.
Here's the type/benefit of testing we do:
Unit testing--ensures that units ...
5
I don't think there should be too much (read: hardly any) randomness in regression tests. If your buzz test found an error that happens when "bvcyjkfa76!" is entered into "number of hats", make exactly this string (or "banana" if you like it better and it triggers the same error) a test case, but don't let your unit test throw a random string at the program.
...
5
We're one of Google's competitors. Our answer? Basically two.
First, we do calculate the complete address-to-address solution. Yup, that's a big matrix. Even worse, we do so for all times of day, all days of the week. There's sufficient similarity in the input domain to cache intermediate results, which makes the problem tractable. Still, try to get a bulk ...
4
In my acceptance suites I have stayed away from using technology specific controls i.e for web applications don't use css dont use html elements if you need to fill in a form do the specifics in the steps to setup the SUT not the actual acceptance tests
I use cucumber for my acceptance and have the following
Given A xxx
And I am on the xxx page
And a ...
4
At a previous employer the role of QA was to not test the product but to guarantee the developers essentially did what they said they were going to do with regard to the previously defined acceptance tests which were defined by QA.
The product owner on the other hand had absolutely nothing to do with the testing. Dealing with testing at any level IMHO is ...
4
Run Selenium from a separate programming language's testing framework.
Guaranteeing the evironment is OK without manually doing any work is really hard to do if you're using the selenium recorder and playing back in their standard test language. You'll need to export the code from the testrunner (or write it yourself) using a language like Java or C# where ...
4
Behavior driven development is about more than just automated acceptance tests.
For more information on that see the wikipedia page on BDD or check the excellent list resources Liz Keogh has compiled on BDD
Now on topic:
You could start automating the acceptance tests using BDD tools for .Net. StoryQ and SpecFlow are examples of BDD tools that support ...
4
The broader scope of software testing involves analyzing all of your products, supported platforms, and then identifying what needs to be tested to provide a particular degree of assurance about the results.
The most important aspect is to document what you decide and why you decided that. You'll want to provide that documentation to your clients so they ...
3
I suggest a big - panic smothering email, hit everyone associated not just their management. Keep it short and to the point.
2 points:
1) We can address your concerns at a meeting/phone call (propose a time)
2) We have complete confidence in the system as it is without the hassle and expense of additional changes
It sounds like you have a list of their ...
3
They are the ones paying the bills so in the end you have to give them what they are asking for even though it wouldn't be the best solution and a step backwards.
You have to consider however that perhaps the people who used to use the mainframe have a point. My wife used to work for a bank where she used some mainframe system to enter various financial ...
3
BDD scenarios generally have three purposes:
Having conversations about scenarios lets everyone discuss the behavior of the system and explore any misunderstandings or gaps in knowledge. This is a great thing to do anyway, since you'll gain a better understanding of how the system works and uncover assumptions about how it ought to work compared to how it ...
3
them that the new system doesn't provide data that the old system did
(when it really does).
You should take this VERY Seriously..
Then:
1) Assure management you are working with the Legacy guys to get all of there concerns resolved.
2)Be sure you fully understand what they are saying is missing and why it is needed. Work with the legacy guys to ...
3
I've used FitNesse and Selenium (essentially FitNesse driving Selenium).
When working with Selenium, I think Selenium test scripts should definitely be written by developers, with testers defining the acceptance criteria in a different tool (such as FitNesse). I've seen the mess that can be made out of selenium-only test scripts, which define long processes ...
3
These are different tools with different goals:
Unit testing is a design tool (TDD), and an almost prerequisite for refactoring.
Integration tests are great to visualize the progress of the project, and also great to avoid regression bugs.
The point to remember is that you can't really design with integration tests, and you won't find regressions with ...
3
It's not like google writes separate code for every pair of addresses in the world. With the exception of heuristics that kick in at a larger scale, the algorithm for a 3-leg journey is exactly the same as for a 3000-leg. You thoroughly test the shorter paths and use induction to show the testing applies to longer paths as well.
You pick a healthy ...
Only top voted, non community-wiki answers of a minimum length are eligible
