In the beginning... there is the estimate

From Chapter 2: The Mythical Man-Month

Excerpt
...our techniques of estimating are poorly developed. (page 14)

Adapted from "Creation" by Phillip Medhurst

In NASA, software managers obtain funding for software projects by writing proposals. The proposals are high-level descriptions of the task that include both cost and a schedule estimates that will govern the job.¹ These estimates are typically insufficient and will need augmentation. Enter the mythical man-month.

Underestimation is not a new problem. In chapter 8, 'Calling the Shot,' Books suggests a remedy. He outlines an engineering approach to estimation using a parametric cost model. A lot of progress has been made since then. We now have sophisticated analytic, cost-estimation tools like COCOMO and SEER-SEM . Both are commonly used to gauge proposed NASA and DoD budgets. Both use parametric models based on the software size, modifying parameters and consults a substantial dataset derived from 50 years of carefully scrubbed project data. For example COCOMO allows the user to dial in parameters for the estimated lines of source code (often called SLOCs), required reliability, anticipated complexity, performance constraints, and team experience. You feed in the parameter values and the tool spits out a cost.

I've used COCOMO and SEER-SEM -- at least I've worked with an expert who knew how to properly set the parameters. The results were credible; they matched my thumb-jump estimates. And, since the models used historic project data, there was a plausible basis-of-estimate.

Sadly, the use of COCOMO was mere makework. In practice, the submitted estimates were based on institutional conventions that relied on in-house cost models. In my view, these in-house models codify the underestimation that characterized earlier projects.

Why would an organization adopt an approach that leads to chronic under estimation? There are more reasons than you can shake a stick at, but they are all the rational outcome of the cultural, financial and engineering forces native to a large government-funded bureaucracy. Here's an illustration of how those forces might work:

• A proposal manager will strive to contain cost. Government cost guidelines or cost caps must be met or the proposal is a gonner.
• The proposed system will be assumed to have the same requirements as the last system. This will be perceived as cheaper and less risky.
• The separate engineering disciplines will compete for a funding wedge in a zero-sum game. A favorite will emerge that adds science value.
• A high degree of software inheritance will be assumed. Since nothing new is needed, the estimate becomes the cost of the last project.
• Since the inherited software already exists, the proposed software costs will be expected to be cheaper than the last project cost.
• The estimation process will repeat in a cost-cutting spiral until the engineering teams agree to a budget that is uncomfortable. The staff must be fed.
• Cost models will be used, but only to demonstrate that that a reduced cost is credible. (Skillful use of the parameters, particularly the reuse parameter is valuable).
• Experience shows that approved government contracts come with a wink and a nod. The government sponsor is more likely to add budget late in the project when program success is on the line.

If the proposal wins, the events leading to the appearance of the mythical man-month might go like this:

• The project manager keeps reserves as long as possible in the likely event of last minute cost surprises.
• The project customers make changes that drive requirement changes. Some change are important; some are frivolous. All must be addressed.
• Salary and overhead costs will have increased since proposal submission. The budget remains fixed, so the work must be done by a smaller workforce.
• Government cuts lead to project cuts. Then each discipline will fight push the cuts elsewhere in the project. The management must account for the hard fact that without hardware there is no system. The choices are limited. Software and operations budgets are the best candidates.
• The software for the new project will not be the same as the last project. The platforms will be different. The new requirements will impact the design. Additional resources are needed.
• The schedule remains fixed to meet the launch date. The engineering teams do their best to make up the differences with long hours.
• As the crisis become apparent, reserves become available.
• Management invokes the obvious remedy: add staff

I've heard different costing experts say that proposed estimates are typically off the actual costs by 50% or more.² Is it any wonder? In my view, good cost estimation is a black art--the problem has too many unconstrained degrees of freedom. The bigger the project, the greater the cultural and financial forces, the more degrees of freedom, the blacker the art.

I have little faith that better tools will lead to better estimates. The tools merely serve the institution. However, available budgets would be allocated more effectively if project management operated as if the mythical man-month was not a myth.

Easier said than done.

---

¹In some sense cost and schedule are two sides of the same coin. In practice they are not. Each estimate is constructed against different constraints.
²The actual hours worked are not reported and the actual man-months are not known.