Decomposing Software

More on the Humpty-Dumpty Effect

From Chapter 2:

Excerpt
Men and months are interchangeable commodities only when a task can be partitioned among many workers with no communication among them (page 16)

From "On the Criteria To Be Used in Decomposing Systems into Modules"¹

Excerpt
...it should be possible to study the system one module at a time. The whole system can therefore be better designed because it is better understood.

Resting place for collateral damage from decomposition

Brooks' Law describes how the number of intercommunication paths grow quadratically as staff is added. Each communication path adds overhead and eats away at productive programming time. Adding staff will slow progress. A schedule problem cannot be solved by simply increasing the number of man months.

Note that these intercommunication paths imply that the system has been decomposed into understandable design chucks that can be developed independently.

By and large, decomposition is considered necessary and inherently good. Decomposition is a fundamental systems-engineering method.¹ And, since Parnus², software engineers have obligingly decomposed code into functional parts and in pursuit of "modularity and information hiding." If there's any controversy about decomposition, it concerns type: functional vs object oriented, hierarchical vs flat, efficient-and-fast vs elegant-and-slow.

Perhaps a little controversy is warranted. It was during the Constellation program I noticed the secondary effects from decomposition. These secondary effects may have far greater consequences than 'just' overrunning budget and schedule.

Consider this sketch of what happens when NASA builds software system for a mission.

The work begins by splitting the system into flight and ground systems. The flight system might be decomposed into the following subsystems: Command and Data Handling, Attitude Control, Propulsion, Thermal, Power, Structural and Avionics. The ground system might be decomposed into the following subsystems: planning, commanding, space communication, telemetry, data management and science processing. Each function in each subsystem is processing data about the state of the system. In many, many cases the same underlying state is sensed, evaluated and represented by the different subsystems. For example a voltmeter on the power bus will report changes in voltage, but the same changes will also be sensed (sometimes indirectly) by each device that uses power.

Furthermore, when the system is decomposed into subsystems, the physical relationships (i.e. the connections inherent in nature) are lost. For example, when Propulsion turns on a catbed heater; Power will register a voltage drop on the bus and Thermal will sense temperature change in of nearby structures. Recreating the relationships between states is necessary for effective fault management and autonomous flight. The current generation of spacecraft capture only the most rudimentary physical relationships. If conventional decomposition approaches were used, developing a system that replicated nature's links would be prohibitively expensive and risky.

A similar loss of the physical relationships will occur in the data model when the end-to-end software in flight/ground system is partitioned. Each subsystem team defines the domain specific representations of the information it needs. As the data is passed around it will undergo many transformations. In the transformational process, temporal and semantic information of the actual system state is lost and the information must be cobbled together from the pieces.

Why does this matter? Here's a few of the things I've seen:

• Isolation of state information makes it impossible for anyone to know how all the parts will interact.
• Unintended relationships are spread around the system like seeds at a watermellon stand. These relationships are very difficult to track down.
• Change must be approached with great caution because code breaks in surprising places.
• Testing becomes arbitrarily narrow, obvious and monstrously expensive. Thorough testing is simply infeasible.
• Maintenance costs grow like kudzu and overwhelms any available funds that might be used for new approaches.

But, here's the real gotya: The effect decomposition has on the institution.

Just like the design is partitioned, the development work is split up and the funding divvied among organizations. These funds are the life blood of each organization and managers will fight like the dickens to keep a task alive--even if the technology they own is moribund. What began as an engineering decision soon becomes become a fixture in the political landscape that, once established, must be protected. And since the political divisions mirror the system decomposition, a new mission system will be architected along existing organizational lines. i.e. the organization becomes the system architecture--an ossified one at that.

Can you imagine what would happen if a methodology or technology came along that would perturb the subsystem boundaries? Not pretty. As the architecture becomes annealed to the organization, an organization becomes resistant to technology that threatens a charter. Likewise criticisms or advances in technology that come from outside the organization must be defended against since the claim of unique expertise is required for survival. (Otherwise, why not go elsewhere?)

Add it up. Once THE decomposition is place, the software becomes brittle and expensive to change just as the organization's very survival starts depending on the status quo. The result is a fusion of system and institution that is highly resistant to change and nearly impermeable by new ideas. When creative ideas and developments appear they tend to languish and die on the shelf--a damning claim for an organization like NASA that's chartered to be a world leader in technology.

I would be the last to advocate that we stop the decomposing our systems. However, if there's a scintilla of truth to this posting, decomposition inhibits change. Reconciling the need to decompose with the need to change may present the next generation of software engineers with one of their greatest challenges.

¹ Parnas, D., "On the Criteria To Be Used in Decomposing Systems into Modules." Communications of the ACM. 1972

²NASA Systems Engineering Handbook