Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Is it a bad sign if users submit bug reports for things that are by design?

Does it typically mean that the application is confusing or unclear, or should I just chalk it up to a one-off user mistake unless specifically stated?

(I don't actually have any such reports. This is a purely hypothetical question about whether or not the existence of by-design "bugs" is a bad thing.)

share|improve this question
18  
Perhaps some of the programers at Lotus Notes would care to comment, since there standard reply is "its not a bug its a feature you don't understand". –  James Anderson Sep 1 '11 at 6:29
5  
It suggests to me that the user requirements as given to the developers do not match what the users think they asked for. –  temptar Sep 1 '11 at 9:42
7  
@temptar "It's what I asked for, but it's not what I wanted." –  StuperUser Sep 1 '11 at 10:37
4  
I was assigned a "bug" work item recently for behavior that was exactly, specifically, what was explicitly spelled out in the specification (and which had been discussed during the specification phase). While the behavior may very well have been unexpected from a user's point of view (a particular setting, which was paid attention to elsewhere, was being ignored), if the spec literally all but says "let's ignore that for now to save time", then to me it's not a bug, but rather a change request. Certainly, a good case could be made for wanting that changed, but don't call it a bug. –  Michael Kjörling Sep 1 '11 at 12:43
6  
@Dunk: I've implemented a system that assumed 24 hours in every day, because we failed to convince the customer of the existance of Daylight Savings Time. They just wouldn't believe that there are days with 25 hours. Is that a bug in the program? –  MSalters Sep 1 '11 at 13:17

10 Answers 10

up vote 54 down vote accepted

Is it a bad sign? I think it's a warning that's worth looking into, but I also think it's bound to happen.

When people submit any kind of feedback to me, I try to filter it into three buckets:

  • Bugs
  • Feature Requests
  • Mis-communication

Bugs

Bugs are when something obviously doesn't work the way you would expect, nor the way the user would expect. Like, it asked me for my name, I entered "Scott", hit enter, and it said, "Hi Joe!"

Feature Requests

This is like "I know we never talked about this, but can the program infer from my mouse gestures that I'm left-handed and move the OK button to the left side of the screen?" This is when the current behaviour matches both your and the user's expectations, but they want to change the expectation.

Mis-communication

This is when you would expect one outcome from a scenario, but the user expects a different outcome. Sometimes this becomes a feature request, if they just haven't communicated their expectations, but they thought they did. Sometimes this becomes a bug if your expectation is proven to be wrong.

However, many times you have knowledge that the user doesn't have. What if they said, "On this screen, I can add a record for myself twice with the same first and last name! That's obviously a bug!" Your response might be, "There are lots of people in the world with the same first and last name, so we don't require that combination to be unique. We have a cleanup task that runs at night and emails a Possible Duplicates Report to customer service when it thinks it detects a duplicate with a similar name and address, and asks them to check it manually."

So you should read every bug report, but most complex systems are going to have bug reports that are really just feature requests, or possibly a mis-communication of the requirements. Not understanding the underlying complexity of the real world is probably the biggest source of these issues.

share|improve this answer
3  
Mis-communication also includes mis-remembering (or plain forgetting) the expectations which were communicated and agreed. –  Peter Taylor Sep 1 '11 at 9:02
    
+1 Paging Major Major Major Major... –  Mark Bannister Sep 1 '11 at 9:13
1  
Great distinctions, but I still think that the application should attempt to explain the mis-communication if it can. With your multiple people with the same name the app might say, "We already have 3 John Smiths are you sure it isn't one of these" with a 'No I'm sure this is a new John Smith' option. –  Alex Andronov Sep 1 '11 at 14:10
    
@Alex Andronov - a miscommunication doesn't mean there isn't an action to take: either change the documentation, tooltips, etc., updating the training material, or possibly just a brief explanation and moving on. –  Scott Whitlock Sep 1 '11 at 15:11

This wasn't touched on in previous answers thus far so I will add that it could also be a sign of an ignorant user base. I don't use the word "ignorant" in a derogatory or condescending way, merely as a way to express that they are without proper knowledge or education in their domain or of the complexities of the software itself.

Most users are not aware of just how complicated software has to be to meet certain requirements. Something to the effect of 80% of effort goes into only 20% of functionality (fringe and exception cases).

They sometimes don't understand why the software inherently needs to behave in a certain way, many times to prevent a multitude of defects or corruption of data, etc...

This isn't worrying as this becomes better with clear and concise documentation and communication.

share|improve this answer
2  
+1 but please look up the difference between complex and complicated... Software doesn't need to be complicated at all, but it is often times very complex. –  Marjan Venema Sep 1 '11 at 6:24
    
@Marjan, It was a poor choice of words, thank you –  maple_shaft Sep 1 '11 at 10:50
    
This is very true. The most common case I run across is users not realizing the difference between many-to-one vs. many-to-many relationships. A common request I get is, "can you make the report show the order number in the header?" and I have to explain that while in about 95% of the cases there is only one order number on the stuff they work on, there are many cases where the query might include data across multiple orders. Then I have to ask, do you want me to list all the order numbers in the header separated by commas or do you want the order number on each detail line in the report? –  Scott Whitlock May 21 '13 at 11:53
    
@ScottWhitlock Yes it is the same thing. A good developer or business analyst then shouldn't just do what the customer asks but try to uncover the core problem that they are experiencing for why they made such a request in the first place. Once you identify their problem you can identify a proper solution and write appropriate requirements or user stories. –  maple_shaft May 21 '13 at 16:47

If you have a user that is an expert in their field, you could have a major problem. Imagine an accountant that finds your software, by design, isn't following general accounting procedures or an engineer who discovers you have an incorrect formula. This shouldn't be too hard to research to see if they are correct. Redesign and fix it if needed-quick.

One opinion on a particular UI feature or a field they think should have a currency symbol or some other formating, should be considered, at least until you get more feedback. Researching this could be a little more difficult.

share|improve this answer
1  
There's no single answer, then, I suppose. It should be assessed on a case-by-case basis. I figured as much. –  Maxpm Sep 1 '11 at 2:18

By-design bugs in my experience means that your use cases don't fit your real users. Try reading about using real users for your use cases ( just giving them names and a thumbnail description does wonders for the quality of use cases.)

When prominent OS vendors say "this behaviour is by design" they generally had ease of implementation or commercial advantage in mind. If that's not you, try to find out your users real skill-set and relationship to your software. Do they use it all day ? For fun ? Once a week because it replaced the TPS forms ?

share|improve this answer

The UI should follow The Principle of Least Astonishment - repeated bug reports on a feature that works as designed are an indication that this principle has not been adhered to correctly.

share|improve this answer
3  
Or, that there is a divided user group. For example, let's say your webapp mimmicks a Desktop. Do you expect that closing a window closes the programm? Yes for Windows users, No for Mac-users. –  keppla Sep 1 '11 at 7:33
    
@Keppla - you make a good point. You cannot please everyone, so in the case of "bugs" like the ones mentioned here some investigation is needed to make sure you won't be getting more bug reports after the "fix" than before. –  Joris Timmermans Sep 1 '11 at 7:54
1  
maybe, but it's a lot more powerful to cite the data of "hundreds of people have filed bugs on this!" than it is to have a few rabid users telling you that you have violated their personal interpretation of the Magical Principle of Least Astonishment and they are astonished. I'm a bit weary of the latter, since one man's "astonishment" is another man's "straightforward". –  Jeff Atwood Sep 1 '11 at 9:51
    
@Jeff Atwood - as the saying goes "one swallow does not make a summer", "one bug report does not imply a design error". A single bug report on something that is a feature is no cause for a redesign, and even several reports on a common feature do not necessarily mean that the majority of your users would not be happier without a "fix". Especially if your original release is already popular and people are used to using it "that way". –  Joris Timmermans Sep 1 '11 at 13:28

There are two kinds of bugs: the functionality doesn't match the programmer's intentions, or the functionality doesn't match the requirements. Programmers have a tendency to focus on the former to the detriment of the latter. To put it simply, if your users are reporting a lot of "by-design bugs," your requirements aren't what you think they are.

share|improve this answer

Not necessarily. It could be that the reported bug is in a use of the software which is just outside of its originally defined scope. Consider software which was designed to do A, B, and C (for a trivial example, draw lines, triangles, and rectangles). If D is a logical next step (e.g. pentagons), the user may assume it should do that as well, and that not doing so is a bug. But if this is outside of the original scope, it's not a bug. It could instead be an oversight (bug) in design, or a gray area in the specifications, or a different set of assumptions made by the developer and the user.

(Edit - added my comment to the answer per @Marjan Venema's suggestion.

share|improve this answer
    
I don't really see that being marked as a bug report. More like a suggestion or feature request. (Although, with some bug-tracking systems, it might count as one anyway.) –  Maxpm Sep 1 '11 at 2:23
1  
I agree, most of the time, but it can mean an oversight (bug) in design, or a gray area in the specifications, or a different set of assumptions made by the developer and the user. –  James McLeod Sep 1 '11 at 2:30
1  
James, if you add that comment to your answer, the whole answer becomes better. –  Marjan Venema Sep 1 '11 at 6:25

I believe its a bad sign mainly due to the fact that the design is not generic and users requirement is not fully understood/analyzed.

I see two catogories of design, - Design by accident. - Design by intention.

Design by accident leads to such issues frequently and cannot sustain.

share|improve this answer

I'd like to add to maple_shaft's answer about users' "ignorance". You also have to keep in mind that 90% of the users will only care about their own experience and way to use the system. They don't care about other users, why should they? It's our job as designers/developers to take in input from all the different kinds of users and make something that fits everyone as good as possible. Most of the times you can't make a solution optimal for everyone.

But of course you need to read and evaluate feedback from your users! They are, after all, those who use your creation!

share|improve this answer

The users submitting the bug requests were generally not consulted on the design so it is not surprising that they see things as bugs that were deliberate design decisions. To a user anything that doesn't work as expected is a bug.

I have found the problem is often a sign that the BA or PM only got requirements from managers not actual users. What managers expect of the system is often vastly different from what the people entering the data actually need. I was taught to collect data directly from the actual users, but most BAs I have run into lately only talk to managers (and generally relatively senior managers at that).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.