Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I'm looking for assistance in figuring out the best approach to managing User Acceptance Testing (UAT) and how users raise defects/bugs - I've been in the situation before of having to deal with a free for all in that every small error is raised as a bug which takes valuable time from actual bug fix whilst I sift through lots of invalid defects/bugs.

Don't get me wrong, i'm not trying to short shift users by saying "not a defect" but trying to make the process smoother so that things get fixed.

share|improve this question
add comment

4 Answers

up vote 3 down vote accepted

First of all, if you aren't yet using an issue tracker tool, get one and make it accessible to users. This way they can enter their bugs into it, rather than pestering you directly via emails or phone calls. You should of course train them to use the tool correctly - this may be a significant initial investment, but it will quickly pay off.

Let users freely report anything they consider a bug, but have them assign severity and/or priority to each bug. This allows you to focus on the most important / urgent ones, but still keep track of all. All decent bug trackers have built in support for these properties, and allow you to query / filter / order bugs according to them. In case the severity or priority of a bug is incorrect, you can discuss this with the user and modify it. If some users often report issues which aren't real bugs, or aren't reproduceable, you (or your management) may need to discuss this with them, and/or - again - educate them on how to document items to make them usable in the development process.

share|improve this answer
3  
To a user, everything is high severity, high priority. It's in their best interests to indicate this. The prioritization should be done by a project or product manager, a Product Owner, a quality lead, or someone else internal to the developing organization. This way, you can avoid the noise of users submitting too many high severity or high priority defects. Plus, developers can filter and only view those that have been accepted by those in a project leadership position. –  Thomas Owens Dec 5 '11 at 12:52
1  
+1 for the issue tracker. I however, do not agree with the suggestion 'have them assign severity and/or priority'. That will stir another hornet's nest. Moreover, such a ranking, by anyone, can mask out real and potential bugs and unnecessarily highlight others. –  Kris Dec 5 '11 at 12:52
    
@ThomasOwens, I think most users can make a difference between a typo or misaligned button on the GUI, and a crashing application. So in my experience, it is usually OK to let them do an initial classification of their reported bugs. Of course, as I noted, it can be overridden if needed. Indeed, in bigger projects you may need a separate person / board to do that. –  Péter Török Dec 5 '11 at 12:56
2  
I would still prefer the user to identify the type of defect in terms of a functional component (such as "UI", "saving/loading", "configuration", and so on), but I wouldn't trust them to provide valid prioritization for the developer to use to work on features. That's not eliminating any noise on the developer's end, but it is making the noise through a single, unified channel (better than nothing, I suppose). I don't want a developer taking a user's classification and spending time working on something that's not really of concern to the product. I only want them working on approved changes. –  Thomas Owens Dec 5 '11 at 13:01
1  
@ThomasOwens, I agree that ideally there should be a product owner or something similar to prioritize bugs and assign them to sprints, among others. This was not clear from the above, sorry. –  Péter Török Dec 5 '11 at 13:10
show 1 more comment

A change control board (CCB) would be helpful to shield the developers from the user requests. A CCB is a group (although it could be an individual) that meets on a regular basis to review the latest submitted defects, determine their impact, prioritize them, and in some cases, assign them to a member of the development team.

The CCB would review all of the defects and feature requests submitted since the last meeting. For each one, they would determine if it's a valid request. If it's not, it would get closed with a reason why. If it is valid, it would be assigned a criticality and perhaps a milestone for incorporation or deadline date. The development lead at the CCB might even assign the task to a specific developer, depending on the process you are using.

Developers only need to care about the defects and feature requests approved by the CCB, and not all defects submitted by the user. The CCB would also be responsible for transforming user requests into the expected input to the development team such as a formal bug report, a prioritizied user story, or an updated requirements specification.

Depending on your situation, it might not be possible to provide your users/customers with access to your issue tracking tool. In such an instance, you would want another standardized method for everyone to submit reports with all of the information the CCB would need to turn it into a bug report or feature request in the issue tracking tool of your choice. Quality assurance would most likely still be submitting right into the issue tracking tool that you are using, but they would be reviewed, prioritized, and assigned by the CCB before a developer starts working on it.

share|improve this answer
add comment

Proper Issue tracking is a must. I am assuming that you have efficient version control with appropriate branching as necessary.

However, i think more robust release deployment is what make most difference to reliability directly perceived by the end customer. Often, when few things are fixed, but at the same time more bugs gets added along with new features. The result is the customer has always something buggy. So best is to create well established fall-back-points and robust release methodology to ensure that there is always a very reliable way to deliver the software. See this for more.

Another thing (though not very common) but should be to use Test management tools. I am in search of such web-based systems but looks like best bet is Testopia

share|improve this answer
add comment

I think the way to answer this question, is to take a step back, and define what is your development process. If you are using Scrum, or some other agile development methodology which has a defined release schedule, and a process for determining what goes in each release, you can push off dealing with the bugs until the next release planning session. (Of course, any "show stopper" bugs have to be fixed immediately.) The release planning session should be about prioritizing the bugs, together with any new feature requests, and deciding what gets in, what gets left out. If you are doing fairly quick release cycles (like every 1-2 weeks), users will get the confidence that the important stuff is getting fixed quickly.

What Peter said above, about having an Issue Tracking system, which allows users to capture the bugs, and set priorities, is critical. However, that is only the first step, since the bug fix won't get into a release unless the "team" agrees on it.

This is actually a great question, with no simple answer. Whatever solution you use needs to fit your organization.

You might find this link interesting reading, as a starting point for more investigation on what methodology would work best for you.

Coping with Bugs on an Agile/Scrum Project

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.