Tag Info

Hot answers tagged

28

One important factor which makes unit tests extremely useful is fast feedback. Consider what happens when you have your app fully covered with integration/System/functional tests (which is already an ideal situation, far from reality in most development shops). These are often run by a dedicated testing team. You commit a change to the SCM repo, sometime ...


21

Your team needs to do the data conversion for them. You really should have done it for them in the first place. I've been involved in a number of expensive platform migrations and the vendor always, always has their own data conversion team who are responsible for understanding the legacy system, writing all the migration scripts, doing all the tests, and ...


14

End-to-end tests are also necessary. How else can you know you hooked up all the units together correctly? On very simple code, it is possible to test all the paths through the code with only end-to-end tests, but as you get more layers, it becomes prohibitively more costly to do so. For example, say you have three layers, each with five possible paths. ...


13

Maybe I'm too old-fashioned, but even the most modern development or proccess techniques cannot substitute another set of eyes, fresh eyes, before releasing a product to your client. Even if your product is simply an API for another developer, you can use QA to think as the API user, providing test/use scenarios that you or your client did not think in ...


11

we don't want to modify code that could be shared, because that will cause a big regression test impact Above sounds about right to me. The more important is the code, the more it is shared, the higher are quality requirements, the more quality assurance should be involved when it changes. Since your system is implemented in Java, you can see an ...


10

I worked in the car navigation field over a decade ago. Step A) Use a reference package and select a large sample set, run A/B tests. Not looking for exactness, looking for outliers - The reference set showed Reroute 1234 as 10.34km, and we calculated 123.5km. Step B) - Refine our software and the reference software - Add more samples and reduce the ...


10

Yes, end-to-end tests (or integration tests) make a lot of sense, but so do unit tests, too. Ideally, you have both of them, since both typically catch different kind of bugs. Thus, having end-to-end tests should never be an excuse for not having unit tests.


9

The test plan should NOT be written by developers. Part of what the test plan is to do is to check to see if the developer correctly interpreted the requirement. A developer cannot effectively write a test plan on the code he is going to write. Test plans should be written by the people who are going to be doing the QA or by the business analysts. If ...


9

Is it true to say that the product manager and the QA should be responsible for taking the user-story and breaking it down to acceptance tests? Mostly. They may not actually write the actual acceptance test. They may approve something you wrote. But they approve the acceptance tests. Yes. the acceptance tests need to be formal, so they can be ...


9

I don't think there exist any metrics to calculate "cost for regression tests / LOC of reused code built". And I don't think anyone had ever invested so much time and money to build the same "big" system twice, one version with a lot of resuable components, and one without, to make any serious research on that. But I have seen problems caused by reuse like ...


9

If you are developing an online system for your company, that would imply that somebody besides you must have an idea of how it supposed to work at least at the UI level. So that would be the level at which your work can be reviewed by others within the company. Unfortunately, that means that there is nobody to review your code. One possibility is that you ...


9

What you are mentioning is a classic problem that many companies have introducing agile development. Unfortunately, there's no easy solution. For the specific problem of acceptance, the "standard" workaround is having someone in the team acting as a proxy for the product owner. This is not great but it can work as long as this person takes care to demo ...


8

Acceptance Test-Driven Planning is an extension of XP Planning. It involves the following steps: Getting our story straight Show and tell Retrospective (optional) Technical Retrospective Iteration Planning Workshop (optional) Big, up-front thinking Cutting the iteration. This planning process is supposed to take up about 10% of the time, the development ...


7

The QA team should write and execute the test plan. Ideally the test plan should be written in parallel with the functional specification - it's amazing how thinking about how to test functionality concentrates the mind and improves the specification.


7

100% coverage is not the same as 100% tested. I'd see a QA person in a ATDD project as someone that would help write the tests and perform the other types of testing that still exists. I.e. UI Testing, destruction testing and load/stress tests. But I've never worked out an ATDD project.


7

We use Robot Framework extensively and Fitnesse a little. Fitnesse I think has an edge if you like the wiki interface that it exposes to people writing the tests - it's particularly good if your data consists of tables of primitives with expected inputs and outputs. Once you get beyond the basics though it gets clunkier and clunkier. Robot Framework is ...


7

All types of testing are very important, and ensure different aspects of the system are in spec. So to work backwards, "If I had to choose one type of testing..." I wouldn't. Unit testing provides me different feedback than integration testing or interactive testing by a person. Here's the type/benefit of testing we do: Unit testing--ensures that units ...


7

It personally sounds as if your Acceptance tests have encompassed properties of Integration tests and that you are trying to "kill two birds with one stone" as the saying goes. In the traditional Waterfall model a single Acceptance test should determine if a single requirement has been met. If developing based on a strict SRS document, you may find that ...


7

Let's suppose that it works for email, but not name. Now your acceptance test will fail half the time. Just re-running the test will potentially change the result. That'll make it harder to notice errors and track down the problem with it occurs. It'll be way easier if you just write two tests, one for each login method.


7

When there is a dispute between the users and the program about how the program should work. The solution isn't to fix the users. That goes double when you're presenting data to domain experts. In this case if the devs and the users have different ideas about what data should be presented, then correct response is to make the data presented match the user's ...


7

In general no. For toy programs (say solutions to Project Euler problems), sure. This is before getting into a religious discussion of what are unit tests (if it uses a file system object, is it a unit test? Let the inquisition begin!) For N-Tier applications, where N > 1, integration tests are needed, and even they may not actually be suitable for ...


6

QA's job is to break the application, the devs job is to not break it. Therefore they write their tests from a different perspective. For instance the devs write tests to see if the expected behavior happens, QA writes tests to see what happens when the users does something the developer would never consider the user would do. Further, developers often ...


6

...shouldn't it be the role of TDD to emerge the design and therefore these components? No. This is a common misconception about TDD. The purpose of TDD is not to "grow a design." The purpose of TDD is to insure that a program stays "well-designed." TDD will force you to create an API that's testable, and specific functional requirements will ...


6

Acceptance tests access the application through a special purpose API. You presented this use case: Given Michael has just been created in the application, his status should be left to non-activated. The API implied from this use case is something like: CreateUser(String name); enum UserStatus {non-activated}; UserStatus GetUserStatus(String name); ...


5

Typically, an acceptance test is written against a requirement. Regardless of how you capture requirements, you should have a set of things that your system must be able to do, expressed as a list in an Software Requirements Specification (SRS) or as a collection of user stories, among other options. Each requirement should have an acceptance test associated ...


5

In my acceptance suites I have stayed away from using technology specific controls i.e for web applications don't use css dont use html elements if you need to fill in a form do the specifics in the steps to setup the SUT not the actual acceptance tests I use cucumber for my acceptance and have the following Given A xxx And I am on the xxx page And a ...


5

I don't think there should be too much (read: hardly any) randomness in regression tests. If your buzz test found an error that happens when "bvcyjkfa76!" is entered into "number of hats", make exactly this string (or "banana" if you like it better and it triggers the same error) a test case, but don't let your unit test throw a random string at the program. ...


5

Run Selenium from a separate programming language's testing framework. Guaranteeing the evironment is OK without manually doing any work is really hard to do if you're using the selenium recorder and playing back in their standard test language. You'll need to export the code from the testrunner (or write it yourself) using a language like Java or C# where ...


5

We're one of Google's competitors. Our answer? Basically two. First, we do calculate the complete address-to-address solution. Yup, that's a big matrix. Even worse, we do so for all times of day, all days of the week. There's sufficient similarity in the input domain to cache intermediate results, which makes the problem tractable. Still, try to get a bulk ...


5

In practice, I tend to leave the design a bit rough when first starting out with an acceptance test, just introducing simplistic versions of concepts to make progress. The trick is recognise when you have enough information about the implicit design to refactor. Actually, I think you have a bigger problem with your first test. You're starting with the ...



Only top voted, non community-wiki answers of a minimum length are eligible