Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Does it make sense to give signoff authority to testers? Should a test team

  1. Just test features, issues, etc, and simply report on a pass/fail basis, leaving it up to others to act on those results, or
  2. Have authority to hold up releases themselves based on those results?

In other words, should testers be required to actually sign off on releases? The testing team I'm working with feels that they do, and we're having an issue with this because of "testing scope creep" -- the refusal to approve releases is sometimes based on issues explicitly not addressed by the release in question.

share|improve this question
2  
Please rephrase your question so that it is not a poll. What is the problem you are trying to solve? –  M. Dudley Jul 1 '13 at 14:58
5  
"should" largely depends on the organizational structure of the company. If they get measured on bugs found in production, being able to hold a buggy release is an essential tool. –  MichaelT Jul 1 '13 at 15:10
    
Excellent point, @MichaelT . In my organization, testing is a role rather than a job description, and evaluation is more ad-hoc and not at all quantitative. Successful deployments would feed into positive reviews, but bugs in production would not be specific negatives, any more than for anyone else on the team. –  Ernest Friedman-Hill Jul 1 '13 at 15:14
5  
If you have problem that testers refuse to approve releases based on issues not planned to be addressed, than you have communication problem (they don't know the issues are not planned to be addressed) or people problem (they are making themselves important; whether to release is ultimately business decision). –  Jan Hudec Jul 1 '13 at 15:23

5 Answers 5

up vote 26 down vote accepted

Most places I have worked the QA people do have some sort of sign-off step, but do not have final authority on if the release proceeds or not. Their sign-off represents that they completed the testing expected by the release plan, not that the release is flawless.

Ultimately QA != the business and the business needs to decide if they are OK with deploying the code in the current state or if the benefit outweighs the downside or whatever. This is often done by clients or stakeholders immediately prior to deploy and is often called User Acceptance.

If your QA is also your User Acceptance group then there is the potential that they do have the authority to define your release candidate as unacceptable, but if you are getting this over issues that are out of scope to the bugfix/iteration/sprint/change request/whatever you bucket your time in, then the Project Manager or the business line stakeholders needs to have a come to Jesus meeting with the QA team.

It is fine to report on preexisting defects or unintended outcomes of new requirements, but if it is out of scope and non-disastrous it is generally not acceptable to label it as a blocking issue. It goes in the backlog for the product owner to prioritize like everything else.

share|improve this answer
    
Who decides if you will invite the customer to perform an acceptance test on build 1234 or that it is not good enough for acceptance testing? –  Bart van Ingen Schenau Jul 1 '13 at 15:23
3  
@BartvanIngenSchenau: The product owner/project manager has to have last word on these. Because even the acceptance tests often will bulge if need be (do you want feature X now even though we can't get Y to work with it, or do you want to wait 2 more months until we fix it?) and the product owner is negotiating that. –  Jan Hudec Jul 1 '13 at 15:27
    
in addition to what Jan said, in many methodologies there would be schedule or cadence here. some folks deploy every candidate that survives the initial smoke test to UAT, some auto deploy anything checked into the candidate branch, some everything that is checked into main. ideally you have been showing the stakeholders progress as you go somehow so there shouldn't be much of a surprise at the end. in some of these cases you end up showing the stakeholders what the QA people have been dragging you through the coals on and they just say "who cares" and it is over. –  Bill Jul 1 '13 at 15:43

Someobody needs that authority. Whether it is a tester, the team of testers, the leader of the team of testers, or the leader of the development organization is somewhat irrelevant. Or perhaps more accurately, it depends on the organization.

Ultimately, the choice to release software is a business function. The business has to decide whether the quality is appropriate. Arguably, the director of quality assurance should make that decision, or feed that decision to the appropriate business unit. That all depends on the size of the company, the relative importance of quality, etc.

All that being said, the information used to make the decision starts with the tester. Whether they have the power to stop a release or not, they should feel the responsibility to inform the decision makers when they see something that they think should cause a delay in the release.

share|improve this answer

Giving sign-off authority (i.e. a veto right) for releases to testers makes as much sense as giving that right to developers: none at all.

Testers and developers are primarily technical people, so they are likely to make their decisions mostly technical grounds. But, the concerns that need to be weighed when making a release are both technical and business concerns. Obviously, the customer won't be happy if you deliver a bug-ridden product, but the customer will be equally unhappy if you keep postponing a release because there are still open issues on the product.

Someone needs to find the right balance between a good product and keeping to the schedule that was promised to the customer. To do that, you should not be involved in the project in a purely technical role, but rather in a more business/management oriented role like project manager or product owner and take your input from the testers and the developers.

share|improve this answer
1  
I voted this down because I fundamentally disagree with several points and assumptions that you are making. I disagree that QA shouldn't have authority to veto a release because many QA groups also operate in a User Acceptance role as well. Furthermore, I disagree with the assumption that testers are technical people. Not always the case, not every group that releases software can afford a full QA team, so that role can fall on business analysts that may not be technical at all. –  maple_shaft Jul 1 '13 at 15:37
1  
in addition to maple_shaft's point I often see the final call on this left to whoever is in the customer role unless there is something terrible identified. it is ultimately their deliverable and they are most likely to have the right point of view on risk assuming your provide them with accurate information. –  Bill Jul 1 '13 at 15:51

The decision to 'release' or 'not to release' is at the end of the day a business decision, where a rigorous risk/reward analysis needs to be performed.

It is insane for any organization to ask the test team to take on this responsibility or for the test team to agree to this responsibility.

The role of the test team is to provide an analysis of the quality of the software, its readiness to be released, and any risks identified as an input to the business decision to release or not to release.

As others have noted, _somebody_ (and I believe it is an individual) does need the authority to make the 'release' or 'not to release' decision. That same person can have delegated that decision under specific conditions (i.e. no P1 or P2 Bugs)

share|improve this answer

I've worked with the same situation of testers over-reaching and inventing ever more creative ways of breaking a system which, when risk assessed, are incredibly unlikely to ever happen in production.

While I understand and commend the test team for not wanting to send out imperfect release, it does require strong product ownership to define what is an "acceptable risk".

In my experience, the test team should be given a veto on releasing software but this veto should be overridable by the product owner but only after discussion with the lead testers.

Software will never be perfect, if you're suffering from test creep then you'll never get anything released until there's a major production issue (which won't be tested correctly) and rushed out.

share|improve this answer
1  
That is a real problem but I am not sure if that is necessarily the OP's problem though. My interpretation is that somehow the testers are interpreting new requirements that weren't originally defined. –  maple_shaft Jul 1 '13 at 15:42
2  
my experience with testing teams has lead me to fall on the other side of this. To me QA should have no expectation of being able to block a deploy without making their case to the rest of the team or getting the owner to override the team. –  Bill Jul 1 '13 at 15:47
1  
True - I probably wasn't explicit enough, the same issues occur when testers raise defects, and I quote, "in the spirt of the story" which leads to the same issues - nothing ever getting released. –  Michael Jul 1 '13 at 15:47
    
In my case, it's more @maple_shaft 's interpretation; not so much being devious in finding ways to break the software, as reporting failure to handle explicitly unsupported cases. –  Ernest Friedman-Hill Jul 1 '13 at 15:55
1  
@ErnestFriedman-Hill It sounds like you are describing Negative Requirements and these are what is missing from your functional requirement documents. A Negative Requirement is an explicit statement about what a system will NOT do, and can be as acceptable as regular requirements. If these are declared then their test cases against Negative Requirements are not valid. –  maple_shaft Jul 1 '13 at 16:59

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.