Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

This question already has an answer here:

If given too little time, quality suffers. If given too much time, developers will find a way to use all alloted time, no matter what. What would be a good way with dealing with estimates? Are they a necessary evil?

share|improve this question

marked as duplicate by gnat, Joris Timmermans, Jalayn, GlenH7, Yusubov May 16 '13 at 12:43

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

19  
In the words of the late great Douglas Adams "I love deadlines. I like the whooshing sound they make as they fly by." –  JohnFx Dec 1 '10 at 15:26
6  
I think your assumption that "developers will find a way to use all alloted time, no matter what" quite misguided. I've finished a lot of projects early and don't randomly use up alloted time. Perhaps rewording your question could eliminate your undocumented assumption. –  Walter Dec 1 '10 at 15:29
1  
@Walter - not so fast. This phenomenon is well established and is not specific to developers. For example, how often in college did you have your term paper written and ready to turn in a week before the due date? –  JohnFx Dec 1 '10 at 15:34
1  
@Walter - man, I must be working in the wrong place. That's how it is here at least, maybe we are really dysfunctional. –  noone Dec 1 '10 at 15:48
2  
@PeterAllenWebb - It doesn't make them lazy if they find a way to fill up the available time. There are always ways to improve on a software project (more testing, tweaking the UI, adding better error handling, general refactoring). I'm not suggesting they are wasting the extra time, just that they will fill the time available delivering the best product available given the time constraints. Also, my comment said specifically that this is true of everyone, not just developers. –  JohnFx Dec 1 '10 at 16:05

10 Answers 10

Of course they're necessary but they're not intrinsically evil, they're just done badly.

The basics of a good estimate:

1) Developers, ideally the ones who will do the work, have to be involved in generating them.

2) An estimate is a range (usually best case, worst case, most likely case), not a single value. You may use the range to approximate a single value ((best case + 4* most likely case + 2 * worst case) / 7 being a fair formula for doing so) but never forget that that's all it is - an approximation.

3) In my experience the most common reason for bad estimates is wishful thinking, either on the part of the developer (who is optimistic it will all go fine) or the manager (who is desparate for it all to go quickly). This needs to be fought - any estimate based on everything going well is pretty much a cert to be massively under. Most likely case should probably assume that a certain amount of stuff isn't going to go the way you want it to.

4) In my experience the second most common reason for bad estimates is not thinking through everything that has to be done. It's not just the coding, it's the unit testing, the admin, the user guide, the back end maintenance screens relating to the functionality. Break down the task into individual components each taking between quarter of a day and five days - that sort of size can be estimated with a reasonable level of accuracy.

5) Measure the actual time things took and review afterwards and use that to improve estimates. Refer to actuals for similar previous tasks as a bench mark and to justify why things are higher than might be expected.

6) Estimate both at the micro level (each individual task) and the macro level (gut feel on how long the whole thing should take). Usually the macro level estimate will be lower - this is an indication that you might want to review (but not necessarily change) the micro level estimates. The sweet spot in my experience is when they either agree or, more likely, the macro level estimate is still lower but no one is willing to change any of the individual task estimates. At that point you use the individual task estimates.

7) Don't confuse the estimate and the commercial aspects. If you estimate 100 days and the usual charge out rate is $1,000 per day that should obviously be $100,000. If they say they can't charge that and win the business that's fine but (unless the scope changes) the estimate doesn't change, the charge out rate does.

8) Developers should be challenged on their estimates (as in asked to explain why something takes that long) but not pressured to lower them. Certainly not just because the numbers aren't what someone wants in any case - if they can't justify them then they usually need to be pushed for a better breakdown and potentially a second set of eyes but in a non-confrontational manner - remind the PM (or if you are the PM remember) that they're going to look bad if the project isn't delivered on time too so they should be keen that the estimates are solid.

9) Contingency should be added both shit happens contingency (usually 10 - 20%) and to cover specific risks. Contingency should be planned in but is "owned" by the project manager - that is if a developer needs to go into it he has to speak to the PM and say why he's over run his estimate. This usually helps stop the just filling time and keeps contingency for genuine stuff. If anyone objects to your shit happens contingency budget then explain it's just the reality of IT projects - stuff happens you can't predict. If they still don't like it then remember to hide it inside the estimates next time.

10) (Edit) Don't confuse effort and elapsed time. Effort is the total time that will need to be spent doing something, elapsed is the difference between when it starts and when it ends. They're normally different - admin, support, mentoring and life mean that the chances are that you won't get 5 days effort completed in a working week. Take 4 as a rough guide but adjust according to the company and the other commitments of the team members.

11) (Edit - from Martin Wickman) Other work you've done or are planning can be a valuable source of information when estimating. In the comments below Martin refers to Relative Estimating which involves comparing like tasks with each other to ensure that the estimates assigned seem in proportion given the work involved. It's surprising how often you'll happily assign 1 week to one task and 1 week to another but when you look at the two things one is clearly more complex than the other - that's a good indication that one (or both) figures are wrong. Taking this further you get into evidence based scheduling which uses completed tasks and projects as a key input into estimates for future work - at a very basic level "how long did it take last time we did something like this?"

Steve McConnell wrote about all of this in Software Estimation. It's a bit dry but it's very thorough on the subject.

share|improve this answer
2  
One I'd like to add to your list, that Mr. McConnell really harps on in his book: an estimate is not a commitment is not a plan to meet a target. Always clarify what you're being asked for. People will often ask for an estimate when what they really want is a commitment or a plan to meet a target. –  nlawalker Dec 1 '10 at 20:05
    
+1 Personally I find point 7) is usually the definer, particularly in small businesses. Downward pressure from sales/marketing people corrupting time estimates to meet client cost or timeframe demands. –  Orbling Dec 1 '10 at 20:41
    
+1 Hell of an answer. What has personally been a game-changer for me, is my company now gives me one day every sprint (two weeks) to come up with estimates. When you spend 10% of your time going through code, you have a very clear idea of what needs to be changed before you dig in. This is time that is usually spent anyway (but on the wrong side of the estimate-giving). –  Mike M. Dec 2 '10 at 14:41
    
@Jon @noone. While Jons answer is classic and exhaustive, I think it is missing (what I consider) the most powerful way to estimate: relative estimation. This means you don't estimate in time, but compare task sizes against each other, and uses completed tasks as a way to know how much work you can handle each iteration. –  Martin Wickman Jan 26 '11 at 18:35
    
@Martin - it's a good point. I'll make some tweaks to it to incorporate this. –  Jon Hopkins Jan 27 '11 at 10:46

The phenomenon you are referring to is known as Parkinson's Law, which states "Work expands to fill the available time for its completion."

That is, people have a tendency to pace themselves to finish things just in time for the deadline. I don't think it is a result of laziness, but more a consequence that there is always more to do than time to do it, so you prioritize things according to when you have to deliver them.

The best solution to the deadline dilemma, in my experience, is an iterative process that breaks the project into small frequent deliverables. This keeps the sense of urgency up and also helps to keep the estimates realistic as it is much easier to estimate a small task than a larger one. Be careful though to give the developers a lot of input and control over the deadlines. A manager who dictates a million tiny deadlines = a micromanager.

In any case, deadlines should be set by real business constraints. Creating deadlines based on estimates of how long it will take to do the work is somewhat akin to using a ruler to measure itself.

share|improve this answer
    
Oh boy I so have a micromanager here... Thanks, John! –  noone Dec 1 '10 at 15:38
    
+1 For staged deadlines, this is general theory for deadlines, software or not. People work to the deadline as a rule. Or in a lot of cases, some way after it... –  Orbling Dec 1 '10 at 20:42
    
+1 For iterative and frequent releases. This (and relative estimation) is the key to good estimation. –  Martin Wickman Jan 26 '11 at 18:22

Estimates are not a necessary evil. If they're evil, you should question if they're necessary.

One of significant recent trends in software engineering is the proliferation of Lean software development philosophies, particularly Kanban. And - don't shoot the messenger -- the proponents of Lean and Kanban put it bluntly: "Estimation is waste." What do you do with waste? "Eliminate it."

(Henrik Kniberg, the famous author of Scrum and XP from the Trenches, has gotten into Kanban. His latest book, Lean from the Trenches is coming out soon. In the book, he contends that story-point, relative size estimation is a waste of time.)

What they do instead of estimation is a huge, separate topic, but the gist of it is: they embrace variability, optimize the flow of valuable features to the users, measure the lead time and optimize the whole system for lead time or throughput -- instead of controlling the time it takes to execute individual tasks.

If you visualize the journey of a user requirement from the point when it's identified to when it's delivered to the user, you can see how little of it actually involves typing code. Even in the development phase, lots of time can be lost while in-progress work items are:

  • in blocked state (how effective is your escalation process?)
  • waiting for non-instantly available resources
  • getting thorough other bottlenecks
  • reworked

A less-than-superefficient software development team can easily waste (as far as the customer is concerned, that's all waste) several days on any of the above causes, which brings into question whether a precise estimate that it would take 5 days instead of 2 to code some feature adds any value. Remember the customer doesn't care how many of 45 days from the feature request to delivery was spent on coding or something else and in what order.

To use an analogy, an Amazing Race contestant adds value by getting to the finish first. Estimating how long it would take him to eat 1 kilogram of local food is not a value-adding activity.

I don't know whether you can implement any of this thinking at your workplace, I contend this is a useful perspective.

share|improve this answer
    
Days? I've known teams to get blocked for months, particularly on the old 'waiting for resources' issue. Clients can be a primary bottleneck. –  Orbling Dec 1 '10 at 20:46
    
@Orbling: I used "days" instead of "months" mostly to soften the shock. ("Estimation is waste" is a shocking statement to many.) There are so many managers and developers fixated on finessing estimates in the 4-to-12-hour range, completely oblivious to the fact that their lead times are measured in months (at least). –  azheglov Dec 1 '10 at 20:55
    
I find that any estimate with accuracy below weeks is not suitable for anything other than elementary projects that are particularly well understood. Obviously this low accuracy magnitude has to scale with project size. If someone says "can you build me Facebook", then the minimum accuracy should probably be years. –  Orbling Dec 1 '10 at 21:04

Delevopers should be the ones producing estimates for their own work, not you. (Or, rephrased, you should only be estimating your own work.)

Time and experience will allow you to estimate more accurately, especially if you track your own estimates.

The real problem with programming is that for a particular language, on a particular project, you might be able to give highly accurate estimates to routine feature requests. But a lot of the time you have to give an estimate on something that you haven't done before (and possibly in a language you don't know, and/or with a framework you don't know). In which case your estimate is, in some form, a guess. Try guess anyway, keep track of your estimates, and reflect.

share|improve this answer
    
Thanks Frank. Do you see benefits on using methods such as function points? –  noone Dec 1 '10 at 15:15
    
I think it's more important to have a pretty good idea of the requirements you're implementing, and not get too hung up on estimates. They are just estimates. It's only when you actually implement something that you get to truly understand how complex the thing you're making actually is. –  Frank Shearar Dec 1 '10 at 15:27
    
+1 For the exceptionally important point, that for completely unploughed furrows, development mirrors research rather than construction and time estimates are then a guess. –  Orbling Dec 1 '10 at 20:43

The simple answer: use relative estimates and ship frequently. This is basically the approach used in the Extreme Programming method. The basic steps:

  1. The team creates a list of every useful step forward, generally written from the user's perspective. (E.g., "Customer adds item to cart".)
  2. The product manager sorts the list by priority, inserting release markers. (E.g., "first internal alpha", "first public beta".)
  3. The developers pick out the smallest non-trivial item. They call that 1 point.
  4. The developers estimate everything else in terms of that. Anything too large (e.g., 8 points) gets broken down into smaller pieces.
  5. The developers start in building things one feature at a time, making a working system (that is, something you could release if somebody put a gun to your head) at least once a week.
  6. Each week, the team counts up the points they completed. Next week, they try to take on about the same amount. If they complete all that early, they can opt to do more.
  7. If the product manager wants an estimate for some particular release, they can look at the weekly velocity, count up the points in the release, and do the math.

This may sound too simple, but it has worked very well for me, and plenty of other teams. In particular, because the product manager has full control over the schedule through arranging the upcoming work, schedule pressure is no longer a problem for developers. Business people get to argue with one another about priorities and schedules as much as they want, and I get to keep coding.

share|improve this answer
    
I should add that I think one a team gets good at this small-unit style of working, they can drop estimates altogether and just use a Kanban approach unless businesspeople have a particular need for numerical estimates or the ability to compare different paths. –  William Pietri Dec 2 '10 at 0:50
    
+100 Relative estimation is awesome, and imo the only truthful way to estimate. Why? Because you don't need any padding or "multiply with pi" to get it to work. –  Martin Wickman Jan 26 '11 at 18:26
  1. Break down the task into subtask that takes maximum time t to deliver. (For exampe t=1day)
  2. Do historical analysis to get a picture on how good your team estimated the time and try to improve.
share|improve this answer

Developers or tech leads estimate the time for a project, depending on the features to be added. If this time is much more then you will have to discuss with them for reducing the feature/functionality. The estimates will also change depending on the developer to developer. A fresher can not complete the job in same time as an experience developer can complete. So such things should be taken into consideration. And developers do take all the time probably because there requirements are kept changing.

share|improve this answer
    
+1 For an important point managers fail to take in to account often. Estimates must be for the people who will do the project. A lot of estimates are made by senior experienced developers, and then the work is given to juniors who take four times as long. In coding, experience is nine-tenths of skill. –  Orbling Dec 1 '10 at 20:48

I'm surprised there's no mention of agile paradigm. We've used agile on projects to break up the work into small manageable chunks. Each chunk is somewhat easy to estimate and usually shouldnt be more than a week's worth of work. Daily scrum's allow mgmt and other devs to know when something is starting to exceed the timeframe. Analyze your project daily so you always know where you are in the overall "race" to finish the total project. If something hold's you back, management doesn't find out days before the total deadline, you see it coming.

share|improve this answer

A completed project is not a set of code. A completed project is a set of code at a specific point in time. If your customers/users don't care about when they get the code, then it's probably not very important.

share|improve this answer

You really need to read and understand JP Lewis' "Large Limits to Software Estimation"

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.28.4205

http://scribblethink.org/Work/kcsest.pdf

http://scribblethink.org/Work/Softestim/softestim.html

Rather like the fact that you can't in general read a program's code and decide if it halts or not, in general, you can't read requirements and decide how long it will take to implement them. I don't believe this is arguable.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.