A textbook case: next chapter

Update: A textbook case

In a previous posting I made the claim that the problems with the rollout of Healthcare.gov were a text book example of problems Brooks describes in the Mythical Man-Month. If misery loves company, and if you are a NASA software developer, the continuing reports about Healthcare.gov should provide reassuring proof you are not alone.

The Washington Post reported today that, last spring, the administration was warned by 'independent' consultants that the website deployment might be delayed. A number of risks were listed, but the following was especially telling:

...the policy and requirements of a program are best defined at the outset, leaving sufficient time for testing and revision. By contrast...the federal marketplace’s design was marked by “evolving requirements” that shifted throughout the design phase, leaving scant time to test the system before its launch.
(from Private consultants warned of risks before HealthCare.gov’s Oct. 1 launch, November 19, 2013 Edition of the Washington Post)

For many of us in the software development business, the report of requirement creep will have a creepily familiar ring. Requirement creep is inevitable. It's always been with us; it always will be. (A topic for later posting.) Here is particularly trenchant example of requirement creep and the Humpty-Dumpty Effect¹ that occurred four decades ago during development of the Shuttle.

Even though NASA engineers estimated the size of the flight software to be smaller than that on Apollo, the ubiquitous functions of the Shuttle computers meant that no one group of engineers and no one company could do the software on its own. This increased the size of the task because of the communication necessary between the working groups. It also increased the complexity of a spacecraft already made complex by flight requirements and redundancy. Besides these realities, no one could foresee the final form that the software for this pioneering vehicle would take, even after years of development work had elapsed, since there continued to be both minor and major changes. NASA and its contractors made over 2,000 requirements changes between 1975 and the first flight in 1981. As a result, about $200 million was spent on software, as opposed to an initial estimate of $20 million. ²

The Post article hints at another area that mirrors the NASA development culture. According to the article³ "programs of this scale are ideally pursued in a more orderly process..." That phrase was lifted by the Post from a slide deck that was provided by the House Energy and Commerce Committee. The consultants deserve kudos for showing a particularly deft touch at reporting what appears to be a poorly implemented development process.⁴

Soft pedaling concerns has become a way of life for reviewers--especially reviewers whose livelihood depends are the next contract. This sort of conflict of interest is commonplace because there is small community of experts with the requisite skills to offer a meaningful opinion.

For those in positions of high authority, this is an old problem. After all, can you believe someone who has a vested interest in telling you what you want to hear? Machiavelli weighed in on this point. He had the following advise for Lorenzo de’Medici.

It is an infallible rule that a prince who is no wise himself cannot be well advised...The counselors will all think of their own interests, and he will be unable to either to correct or to understand them. And it cannot be otherwise, for men will always be false to you unless they are compelled by necessity to be true. Therefore it must be concluded that wise counsels, from whoever they come, must necessarily be due to the prudence of the prince...⁵ [my emphasis]

In other words, Machiavelli suggests that for effective governance, the management must be wise about the subject at hand and receive true counsel from her advisors. Ironically, successful management of a government technology project depends heavily on the one's ability to craft the acceptable message. So much the better if the message is true.

^1. See The Humpty-Dumpty Effect for a discussion what happens when work is divided between engineering teams.
^2. Tomayko, J., Computers in Spaceflight: The NASA Experience, Chapter Four-Computers in the Space Shuttle Avionics System-Developing software for the space shuttle. NASA contractor report, 1988. Page 114.
^3. ibid
^4. Given the reported lack of coordination between development groups, this is anything but surprising.
^5. Michiavelli, N., The Prince. Translated by Ricci, L. Revised by Vincent, E.R.P., Oxford University Press, World Classics. Reprinted 1968. p.108.