If a new feature is implemented and there is a bug, should QA reject that feature or accept it and file a new bug?

Question

At work, we use a bug tracker called Pivotal Tracker (www.pivotaltracker.com) that allows engineers to file features and bugs. If a feature or bug fix is delivered, it is QA's job to accept or reject it accordingly.

If an engineer implements a new feature that has a bug in it, should QA accept that feature and file the bug separately, or should QA reject that feature and report the bug in a comment?

Potential advantage of rejecting the feature: Better chances of engineers noticing and fixing the bug before releasing the code

Potential advantage of filing a separate bug: It allows engineers to better assess how many bugs there have been (if that is really useful)

Answer 1

There is not a hard-and-fast answer; it is up to your organisation.

One thing to consider is that if QA find many bugs in the new feature, it would be better to track each of them separately, so I would tend towards filing individual bug reports.

If the bug is so major that there is no way the feature could ever be used as intended (e.g. the menu item to launch the feature is missing) it may make more sense to reject the feature outright - it hasn't even reached the sanity-test stage.

Don't forget if you go for a bug-report, that QA needs a meaningful version number to file the bug against - it is neither a bug in Version 1.0 nor a bug in Version 2.0 - it has to be against a particular interim build/load, or similar.

Obviously, QA should also have the power to reject the entire build from being delivered to customers - simply measuring that "there is only one bug in that new feature" isn't sufficient.

Answer 2

A small handbook as answer:

1.Allow feature, don't file bug

Don't do this! Even if the feature is allowed in production with bugs, the issues should be recorded (it might be never fixed but this is another story).

2.Allow feature, file bug

Use this when the issue is minor or has reasonable workaround and will not have a great impact. The bug is reported and will, hopefully, be fixed in the future. Example of bugs here: window moves 1 px to the right after 10 minimizes, comments are ignored in output file, minor UI bugs.

3.Don't allow feature, file bug

Don't allow the feature to get outside the dev environment when the bug that was found is critical/impacts a lot of people (as someone mentioned before, a menu item is missing and the new feature cannot be used, or data loss). However, opening a bug for such an issue could be tricky because the bug is usually visible for all people in the organization but makes sense just for a small group. It might create noise.

4. Don't allow feature, don't file bug

The same bug types as in 3 apply here. The only difference is that no bug is created but rather the dev group is informed about it. Eventually it might be logged somewhere else (on a white board)

Answer 3

A third option is to reject the new feature and file a bug. If you have a separate QA department that is the gatekeeper for quality they shouldn't ever accept broken code. IMO, "reject" should always be the first choice. Whether you subsequently file a bug or not is up to you.

This might be unpopular with development -- and even product managers) -- but that's a Good Thing. If something is painful, do it often. This will cause people to find a solution to the problem. That is, if every time they submit buggy code it comes back with a "reject", they'll get tired of that and work harder to find the defects in their own code.

Answer 4

I'm not familiar with Pivotal Tracker, so I'm not sure exactly what it can do.

Where I work we use FogBugz for bug tracking, and in a case like this we would mark the feature "Waiting for fixes", and open up separate cases for each of the bugs found. This solves both of your problems - it allows each bug to be tracked individually (and they really should be, in a complex feature), and it also lets you clearly know if the feature is ready to ship or not.

Answer 5

It depends on bug severity.

As a rule, a feature should be accepted if it passes acceptance testing. If that bug was found there, feature is normally rejected.

Regardless of whether that feature was rejected or not, bug should be filed separately. Engineers will definitely notice a failed feature, but if bug is mentioned only in comments, it will be harder to track this bug.