How to learn to make better estimates?

Question

I suck at estimates. When someone asks me how long something will take, I don't even dare to make a guess since I will be completely off the mark. Usually I'm way too optimistic, and should probably multiply my guess with some large X factor...

How can I learn to make better estimates? It's not taught at my uni, and even though we have deadlines for all laborations I never think about how long something will actually take. Which I should. For everyone's sake (especially mine).

Answer 1

I'm still not great at it, but I have found that tracking how long you estimate for tasks and how long you actually take can be a big help. That way you can get a solid idea of how far off you usually are. Issue management software with time tracking (Jira in my case) or a spread sheet can be a big help with this.

I think more than anything it's an experience thing.

Answer 2

You can learn by doing them collectively.

I use Planning Poker. It a consensus-based technique for estimating.

Your estimation must be tracked and compared to what you have effectively done. You will get the Velocity.

Each time you estimate something, multiplicate by your recent velocity to get accurate estimation.

Answer 3

2 link by the blog of Joel Splosky:

• http://www.joelonsoftware.com/articles/fog0000000245.html (older)
• http://www.joelonsoftware.com/items/2007/10/26.html

Answer 4

Murphy's Law of Time Management: To figure out how long something will take, figure out how long it should take and double it.

Then, move up to the next higher unit of time. Thus, we allocate two weeks for a one-day project.

Answer 5

I find it easiest to do the process of division to the minimal tasks as mentioned above work out each one and then double that estimate. Then I add them together and add fifty percent. That gives me an approximate project time in ideal conditions. If the work is practically going to be happening in parallel with others it's going to need longer. If you are going to have to wait for other people, expect them to take twice as long as you think it will. Waiting for content or feedback or other information often takes far longer than seems possible.

Where I work we work out a best case/expected case/worst case estimate for each step of the process, which is useful as a guide and also for evaluating how your estimates have worked out.

The technique is not ever so important except that you need to be able to combat the programmer's temptation to underestimate tasks, but what is important is being conservative about when you can deliver something. If it takes you seven weeks to build something and you promised eight weeks, you can come in a little early and look good for it or do some extra testing and be assured of reliability. If you promised six weeks you can look bad even if it's absolutely not your fault. If in doubt, guess conservatively.

Answer 6

Whenever you encounter an estimation problem, try to split them in to smaller pieces. Then see if you have already done stuff similar to the pieces. If you have, you should already have a fair idea of how long each piece takes. If you don't, you should start to actively keep track of time taken for various kinds of tasks. This will help you in future estimations.

Total time needed will be more than the sum of the individual pieces, since you need some time for integration and testing.

If you haven't done something similar, you can probably rely on other peoples experience and get an estimate from them. Don't take this at face value though. Nothing teaches you like experience.

Its kind of like shooting a target. Previous shots at estimation should tell you how off the mark you are, so that you can correct it.

Answer 7

First and most important, you have to define a process and stick by it. Include revising the plan at the end of each phase of the process. You can also revise the process, but in an orderly way.

Second, do some kind of design. Design is the first step to planning, you don't build a house without drawings.

Third, track time (effort). You should at least differentiate:

• Analysis
• Design
• Code
• Unit testing (include fixing defects)
• Integration testing (include fixing defects)

• Acceptance testing, with the user (include fixing defects)

  It would be great if you measured the defects fixing effort for each testing type, but it adds complexity, so you can do it later on.

Fourth, identify key base items for estimating. For example:

• Number of processes to be automated (Analysis)
• Number of domain model entities (Design)
• Number of forms and reports (Code)

Fifth, correlate base items and effort. For example:

• Analysis effort = X man-hours / process to be automated
• Design effort = Y man-hours / domain model entity
• Code effort = Z man-hours / form (or report); number of forms = A * domain model entities
• Unit testing effort = M% * Code effort
• Integration testing effort = N% * Code effort
• Acceptance testing effort = P% * Code effort

Sixth, keep track of performance and deviation from estimates for each project. So you can fine tune your correlation factors.

Seventh, repeat and improve. You will gain a lot of insight just at the end of the first project, by the third you will feel at ease planning and estimating.

Have a look at http://en.wikipedia.org/wiki/Personal_Software_Process, it really works.

Answer 8

You could try to build a track record of what was the estimate and what was the actual for various tasks to build up enough of a record to then know what multiplier to have for specific things that get repeated in your list. Granted this is a trial and error exercise but it has seemed to work fine for me. There is also something to be said for many trials before the pattern emerges probably. This is likely similar to a lot of other answers that would say that one could boil down to, "Just do it!" as that is really how most of us developed the skill. Is it a major pain to see how wrong one can be when making estimates? Yes, but if the estimates get better then everyone can be happy eventually.

Answer 9

If you can decompose the project into smaller tasks and do estimates for those you will be more accurate over-all. Any task bigger than a couple days should be broken down further. If you can't break it down further than you probably have a requirements gap. If you have to do a back-of-the-napkin estimate for a one-line requirement well...nothing can really help you much. Sadly a lot of shops work this way much of the time.

Answer 10

Learn your own bias. If your last estimate has been too low by factor two, next time, double your initial estimate. (Of course, Hofstadter's law makes it hard to do that right...)

It's also always a good idea to remember how much work was needed after the initial release of the previous work, and add that to the next estimate. E.g. your last task took 2 months to complete, but after going live, support, hotfixes and additional changes have cost you another month, so, next time estimate 3 months for a similar task.

Answer 11

For openers, read "Software Engineering Economics", by Barry Boehm, and "Controlling Software Projects", by Tom DeMarco. After you have read and digested those two, read "Software Cost Estimation With COCOMO 2", by Barry Boehm.

For what I have to say next, it will help you a LOT to have taken a probability and statistics class, even a basic cookbook one.

No estimate is perfect. There is some probability of coming in early, and some probability of coming in late. Boehm's original detailed COCOMO model gave predictions that turned out to be within 30% of the actual result, better than 60% of the time. That was a lot better than what was common when he wrote and published the book.

When you take your best guess (and that's all an estimate is), you are including those probabilities. If you pull the estimate in, you are increasing the probability that you will come in late. If you increase the estimate, you are increasing the probability that you will come in early or finish on time. How much you pull it in or let it out controls how the probability changes, and must necessarily depend on the penalties for being early or late. (Insert horror stories here - and there've been a LOT of them over the years!)

DeMarco addresses this to some extent. He also points out that there is an "impossibility region": some schedules are just too tight to be made, no matter what kind of heroics are attempted.

Answer 12

Rather than write a book on it, I'll just offer a little advice on how to use the "break down" method of estimation:

• Break your assignment down into smaller component tasks. Estimate each task as best as possible.

• Add a task for planning and design (which includes what you're doing now.) Estimate it.

• If you don't already have one, add a task for "bringing the tasks together." This task may not seem useful at first. However, when you use this "break down" method of estimation, there are always time consuming things to do that "fall between tasks" and that "pull the tasks together." This one can be tricky to estimate. Try your best.

• Add a task for testing and documentation. Your assignment may not require a lot of testing and documentation, but you should at least spend a little time thinking about it.

• Add up the task estimates to get an overall estimate.

• Go ahead and multiply that total estimate by two††. This will give you padding time to:

  1. Finish things that you overlooked in your original task list
  2. Finish things that you couldn't have known about until getting under way
  3. Incorporate feedback from other people, and make changes
  4. Get interrupted by other things going on around you, like meetings
  5. Finish ahead of estimate more often than behind it

And last, but not least, don't be afraid to sketch out estimates for yourself that are probably totally wrong. Sometimes just sketching everything out, no matter how potentially wildly inaccurate, can help you start on the path to getting a better sense for what's involved.

††As you get more and more experience, this "fudge factor" can be tuned to suit your personal style, and your work environment.

Answer 13

Software Estimation by Steve McConnell (MS Press) is a good read.

The main thing with software estimate is summarized by the following

Without historical information, your estimates are useless.

This is one reason I think why iterative projects have much more success that large phased waterfall projects. They aren't trying to build out a plan for a year at a time with little information other than some black box voodoo of what they think it should be. Every iteration, they are reestimating/replanning and have the last several iterations to base their estimates on.

A few other points to keep in mind:

1. It will only get slower. Applying the 80/20 rule means that the harder work will come later unless your project management is very disciplined.
2. Estimation != Planning. Estimation is the process of figuring out the effort required to get something done. Planning is the process of fitting it into a schedule.
3. 60% efficiency is about all you can hope for. 70% is utopia. If you're estimating in days, build this in. If you're estimating in hours, don't forget to apply it later.
4. Remember the long tail. Estimates are a rough guess of how long it "probably" will take adjusted for some level of risk and unknowns. The long tail comes into play because the actual amount of work required will never be less than 0. OTOH, the maximum amount of time it will take is only limited by how long you're willing to spend on it before giving up. As a former boss of mine said "all estimates are +/- x% and it's never minus".

Answer 14

The formula that works when working for myself :

1. do a break down of todo's to a 1 - 4 hour granularity. I find that i'm usually accurate with these

2. the 'unknowns factor': Multiply by a factor of 2 raised to number of unknowns. I.e. if you are to develop a couchdb applicaiton, but do now know anything about javascript and http .. add 2 ^ 2 as mult factor.

3. context-switch-factor : multiple by 1.5 if you will work in perfect environment( at home in study corner etc ) , or 2.5 if you will work in imprefect environment ( office / crowded place etc )

I find this to be within +/- 20% of actual time taken !!