In IT we are often asked to estimate the expected time/schedule or cost of software development. Sadly, the desire of stakeholders to have “predictable” schedules or costs results in significant dysfunction within a software development team. When a software team is forced by their stakeholders to commit to a schedule/cost they must then ensure that the schedule/cost doesn’t slip. For example, to protect themselves from increased time and cost due to scope creep, software development teams will make it difficult for stakeholders to change their requirements during Construction and even go so far as to drop promised scope late in a project. The desire of stakeholders to reduce their financial risk often results in behaviors by the software development team that ensure that stakeholders don’t get what they actually want. Naturally IT gets blamed for this.
We need to do better. In this blog we summarize the things that we know to be true about software development estimation. In no particular order, they are:
- Estimates are guesses. Look up the word in the dictionary – An estimate is a rough approximation or calculation, in other words a guess. Unfortunately, too many people think that estimates are promises, or worse yet guarantees. In our opinion “guesstimate” is a far more appropriate word that “estimate”.
- Scope on IT projects is a moving target. Our stakeholders struggle to tell us what they want and even when they do they change their minds anyway. Any guesstimate based on varying scope must also vary in kind.
- Guesstimates are probability distributions. Although your stakeholders may ask for a fixed amount guesstimate, for example “this will cost $1 million”, the reality is that there’s a chance the cost will be less than $1 million and a very good chance that it will be more. There is ample evidence that the initial estimate for a software development project should be given in a range of -25% to +75%, so your million dollar project should be quoted as a range of $750,000 and $1,750,000. This is shown in the diagram above by the green distribution curve. In many organizations this can be politically difficult to do, and strangely enough in many cases stakeholders prefer to be lied to (it’s going to be $1,000,000) rather than be told the truth.
- Guesstimates must reflect the quality of the inputs. A guesstimate needs to reflect the quality of the information going into it – if your scope is fuzzy your guesstimate based on that scope needs to be equally fuzzy. Sometimes stakeholders want a guesstimate with a tight range, perhaps +/- 10% (the red curve in the diagram), early in a project. To provide a tight range such as this you need to have a very good understanding of the requirements and the design. Early in the software development process this can only be done through more detailed modeling, an expensive and risky proposition which often proves to be a wasted effort because the requirements will evolve over time anyway.
- Guesstimates anchor perception. The primary danger of providing guesstimates to people is that they believe them. Tell someone that it’s going to be $1,000,000 and they fixate on that cost even while they are changing their minds. Tell them that it’s going to be between $750,000 and $1,750,000 and most people will fixate on the cost of $750,000. Some people will focus on the average cost of $1,250,000 even though the median was $1,000,000 (guesstimates are in effect Weibull probability distributions).
- It’s easier to guesstimate small things. ‘Nuff said.
- It’s easier to guesstimate work you’re just about to do instead of work in the distant future. It is much easier to identify the details of work to be done right now, and thus turn a large piece of work into a collection of smaller pieces that are easier to guesstimate. In part this is because you have a much better understanding of the current situation you are working in and in part because you are more focused on the here and now.
- The people doing the work will likely give a better guesstimate. They are more motivated to get the guesstimate right, particularly when they must commit to it, and have a much better idea of their abilities. Granted, someone may need to coach people through the guesstimation effort. In Disciplined Agile this is a responsibility of the team lead.
- Someone who has done the work before will give a better guesstimate than someone who hasn’t. Experience counts.
- Guesstimates reflect the situation that you face. Which organizational situation do you think will result in a short schedule and lower cost: Five people co-located in a single room or the same five people working from different locations in difficult time zones? Or how about a team working under regulatory constraints versus the same team without those constraints? Context counts.
- Multiple guesstimates are better than a single guesstimate. Getting guesstimates from several people provides insights from several points of view, hopefully prompting an intelligent conversation that enables you to develop a guesstimate with better confidence. Similarly, the same person producing guesstimates for the same piece of work using different guesstimation strategies will also provide a range of answers that you can combine.
- Guesstimates should be updated over time. As your understanding of what stakeholders want improves, and your understanding of how well your team works together, you should update your guesstimates. As your understanding of the fundamental inputs into your estimate improves you are able to produce a better estimate, thus enabling the stakeholders of that guesstimate to make better decisions.
- It costs money to produce a guesstimate. The precision of an estimate is driven by the detail and stability of the information going into it. Want a tighter range on your estimate? Then you’re going to have to have a better handle on the requirements, design, and capabilities of the team doing the work. This greater precision requires greater cost. The fundamental question posed by the #NoEstimates community is effectively “Is the value of improved decision making capability from having the guesstimate greater than the cost of creating the guesstimate?” The implication is that you must ensure the cost is much less than the benefit, hence their focus on finding ways to streamline and even eliminate the guesstimation effort.
- Guesstimation is far more art than science. See point #1 about estimates being guesses. The best guesstimates are done by the people doing the work, just before they need to do the work, for small pieces of work.
- Formal software guesstimation schemes are little more than a scientific façade. Function point counting, feature point counting, and COCOMO II are all examples of formal strategies. They boil down to generating numeric guesses from your detailed requirements and design, plugging these guesses into an algorithm which then produces a guesstimate. These are all expensive strategies (they require detailed requirements and design work to be performed) that prove to be risky (because they often force you into a waterfall approach) in practice. Yes, they do in fact work to some extent, but in practice there are much less expensive and less risky strategies to choose from. People like these type of guesstimation strategies because they provide a false sense of security due to their complexity and cost.
- Past history isn’t as valuable as people hope. Some formal guesstimation strategies are based on past history, but this proves to be a false foundation from which to build upon for several reasons. First, people have different levels of capability which change over time as they learn. Capers Jones has shown that developers have productivity ranges of 1 to 25, the implication being that if you don’t know exactly who is on a team and how well they work together your corporate history will be questionable. Second, technologies evolve quickly so past history from working with older versions of technologies or completely different technologies becomes questionable at best. Third, people and teams change (hopefully for the better) over time, implying that an input into your guesstimate is fuzzy at best. Fourth, because every team is unique and faces a unique situation basing estimates on past history from other teams in different situations proves questionable.
- Beware professional guesstimators. They tend to break many of the rules we’ve described above.
To summarize, when you are required to provide estimates for your software development efforts that you should take a pragmatic, light-weight approach to doing so. This blog posting has provided many practical insights that should help guide your decisions. These insights and many more, are built right into the Disciplined Agile framework.