Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

The company where I work is aiming to have a 10% maximum error margin.They expect analysts to not miss the estimate by more or less than 10%.

I don't know what to think about it, since I got nothing to compare it to.

What could be a good baseline to measure if we are estimating too wrong, for more or less? How much (in %) do you think is okay to to miss?

share|improve this question
3  
I'd want the error margin to be specified by the team putting together the estimate and submitted with the estimate. By and large, your first shot at an estimate, with just a brief product description and no solid requirements will have a high margin for error. As the project gets more and more defined, new estimates may be made with lesser error margins. –  Jesse C. Slicer May 18 '12 at 20:15
2  
Are you saying that if you come in UNDER budget by too much, someone's going to get into trouble? –  pdr May 18 '12 at 20:22
    
@pdr They didn't say anything about the consequences. –  tofs May 18 '12 at 20:28
    
Should be on se.pm? Just askin'. –  Peter K. May 18 '12 at 21:19
1  
I would suggest looking at the "Software Estimation" book by Steve McConnell. It has a tidbit in there that an accuracy of +/- 10% is possible - but only possible on well-controlled projects. This references a book by Jones in 1998. –  MichaelT May 18 '12 at 22:32

6 Answers 6

up vote 16 down vote accepted

Unless you are estimating something very similar to that which you and your co-workers have done before, +/-10% is ridiculously optimistic. Your management either doesn't have a lot of experience with software, or they're not aware of Large Limits to Software Estimation. That paper has some accompanying supporting material, and a lot of punditry can be found.

Let's examine a far simpler system than a typical software project: Rubik's Cube. You can solve any position in 20 moves, max. But since you're estimating, you can only look at a given cube for a few minutes before giving the solution. Can you give a good estimate? No, sometimes estimating a process takes longer than doing the process.

Another simple system: Pinocchio. A wooden automaton, his nose-piece grows when he utters a lie. What happens when Pinocchio is at rest, and then says "My nose is growing"? Some systems aren't amenable to prediction, they're undecidable.

These two problems are built-in to most software systems. Because of that, you'll never get estimates close to +/-10%.

My advice is to give a heavily padded estimate, work like a slave to get the project done as fast as you can, and then look busy until you're within 10% under or over. At that point, announce a spectacular success.

share|improve this answer
    
My advice is to give a heavily padded estimate, work like a slave to get the project done as fast as you can, and then look busy until you're within 10% under or over. At that point, announce a spectacular success. Along with your Dilbert like avatar got it right for me, thanks. –  tofs May 18 '12 at 20:33
6  
@tofs - Asking for estimates that accurate (unless you very nearly just do the exact same thing repeatedly) should be a warning flag to you. Their expectations are unrealistically optimistic, and you will be the one not meeting them. Better to tell them so now than after they spent a lot of money based on overconfidence in the estimates. Also, it looks less like making excuses if you tell them about this before hand. –  psr May 18 '12 at 20:52
    
@psr I guess I'll have to break their bubble, problem is, it comes from high above. If I can't, I will have to resort to heavily padded estimates unfortunately. –  tofs May 18 '12 at 21:09
2  
The Rubix Cube analogy only works if you posit that halfway through solving the cube, upper management decides that solid colors on all faces is too Web 1.0 and they want NoSql Ajax stripes instead. And then the users tell you that they can't use it unless it has a seventh side, with a picture of a cat on it... –  Tacroy May 18 '12 at 22:08

Back when I used to lead software teams, roughly around the Cretaceous/Tertiary boundary, we actually came close to achieving +/- 10% on estimation. It was about +/- 15% if my very dodgy memory serves. But this was because we were estimating for things we'd already done: relatively simple real-time communications firmware which took bytes from A and moved them to B using a language that we were familiar with, in a real-time environment designed by us, talking to hardware designed in-house by guys a few offices away. Lots of slight variants on the above theme, for literally years.

Expecting to achieve this kind of error rate in normal software project estimation is, frankly, ludicrous. When you see it apparently achieved, it's because either people over-estimated and padded (doing extra stuff and pet projects to use up the budget), or under-estimated and worked like dogs at evening and weekends to make up time.

share|improve this answer

There are lots of variables:

  1. How long is the project?

  2. How will the project be managed? Waterfall? Agile? SCRUM?

I would assume from the question Waterfall. If that is the case you are pretty much expected to fail by some percentage given the request for a margin.

If the answer is some Agile methodology, especially something like SCRUM. It doesn't really matter what the margin percentage is. A 50% margin of error on a 2 week Sprint is 1 week, on a 1 week Sprint it is 2.5 days, both extremely worst case scenarios. This is because the delivery date is re-estimated every Sprint, and it will get to be more and more accurate over time, instead of less and less.

With Waterfall 50% margin of error isn't un-heard of either, but over a 12 month project that is 6 months, and it doesn't really get discovered / admited it is that bad until 10 months into the 12.

share|improve this answer

It actually depends a lot on the size and complexity of the project.

If your project estimate is say 1 week - 10% is reasonable. It means +/- 1/2 day.
If it is 1 month 10% is shaky - from my experience it is impossible to forecast what you will discover in 1 month.

More than one month - all bets are off :).

These are per developer - so for a 4 developer team 1week == 1 month.

For a 4 developer team, mostly its good to come up with a list of features that can be done in 1 month, and be within 10% for those feature. Then, re-estimate for the next month.

I have made some naive assumptions here

  1. All developers can do work in parallel.
  2. Haven't considered tester - they should have some time to test
  3. All developers are equal - Frontend, Backend, Designers etc...

You have to factor those in.. but this is the general idea.

share|improve this answer

A good baseline would be one based upon real data you have collected.

The first step to doing this is recording all estimates. The second step is recording the actual results. Be honest, don't be tempted to 'self-adjust' the actual. Gather enough of this information and you have some data to statistical base how good your estimates are. Note, this can/will vary wildly based upon who did the estimation and who the work. Only by doing this can you be reasonably expected to give a 'margin of error' that is anything other pure rubbish.

It doesn't stop there either; analyzing what causes estimates to be off can help improve the accuracy of your future estimates. Note: They still remain estimates, and as such, are only estimates.

Estimation also doesn't finish after the first estimate; this is something that can be adjusted as the project progresses as more knowledge is gained, this reducing the possible margin of error as you go along. The more open you are with communication, the earlier surprises are discussed - leading to people been less surprised and allow more time to make adjustments in either the project or customers expectations.


Second, perhaps a better way of handling error margin is 'confidence internals' rather than merely % margins of error. You than based you estimated delivery date on a confidence interval, rather than a singular date.

PERT is one example of a framework to handle estimation based upon statistical reasoning. For example:

"Based upon what I know now, we have a 90% confidence level we can deliver in 8 months. 95% confidence in 10 months, 99% confidence in 2 years, etc"

Note here: The more confident they want you, the larger the estimate will be. Depending on upon the complexity (aka, how accurate you could possibly be), it could be a small difference between 80% and 90%, or it could be huge!


Lastly - Good luck ;) If you ever solve 'maximum error margin' in software estimation whereby you can specify something like 'all our estimates will be +/- 10%', be sure to commission a box-office movie for the rest of us in the software industry. I'm thinking something like a cross between Office Space and The Matrix :D

share|improve this answer

I'd be very hesitant about any sort of "target error margin" because it really depends on the project. If you are trying to estimate how long it is going to take to install, configure, and customize a new CRM system where no one is quite sure what sorts of customizations are going to be necessary and what sort of business process changes will be needed and the company has no history of similar large projects, your error margin ought to be pretty large (i.e. you might guess that it will take 18 months +/- 50% and quote 9-27 months). As the project goes forward, the specifications get clearer, decisions get made about business processes, your developers get more comfortable, etc. those error margins should get tighter. If you are trying to estimate how long it is going to build the 101st basic ecommerce site when you've got a good history from the first 100, you would expect to be able to give a much more accurate estimate. Most projects, though, are going to fall somewhere in the middle.

Now, if you are quoting a single number rather than a range, the answer is probably to start quoting the range so that the person doing the estimating can specify how accurate they expect their estimate to be.

share|improve this answer
    
While Bruce Ediger made a good point about how an analyst can approach the problem. I think you said something I can use when arguing with my boss. –  tofs May 18 '12 at 20:36

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.