IEEE Software Essays on Continuous Design

There are a lot of good ideas about software design, and they work great on a fixed, known target. In industry, that rarely happens. Instead, requirements are fuzzy, the desired outcome keeps shifting, and design information is fragmented across teammates. I refer to these challenges as Continuous Design because teams don’t design once, they design and re-design continuously.

I write the Pragmatic Designer column for IEEE Software. I’ve been using that to lay the foundations for my ideas on Continuous Design. The text below weaves a narrative across my essays, telling a story about how teams design, how that design evolves, and how teams keep the design on track.

Intellectual control

Teams have confidence in their code because their tests pass (statistical control), and because they designed the code to make sense (Intellectual Control). Today, we lean too heavily on testing to give us confidence and we lose intellectual control. Most developers haven’t noticed because Testing Numbs Us to Our Loss of Intellectual Control.

Teams do what they think is the modern, right thing, but The Rituals of Iterations and Tests don’t yield low tech debt or intellectual control. To get that, they must deliberately choose to keep the design conceptually simple and visible in the code. Once lost, intellectual control is expensive to recover.

Extended and distributed cognition

To create the conditions for intellectual control, teams must foster extended cognition (where the code becomes an extension of their thoughts) and distributed cognition (where the design of the system is shared across the team).

Extended cognition requires the code to be in a healthy state, because Healthy Code Reveals the Problem and Solution. A reader of healthy code can see what the team knows about the problem domain and what design they have chosen to solve the problem.

When you create the conditions for intellectual control, that allows you to Scale Your Team Horizontally. Each developer you add contributes proportionally to their ability, not based on their project tenure.

Ur-technical debt

Designing continuously means grappling with technical debt buildup, the kind that Ward Cunningham originally talked about. I call it Ur-Technical Debt (“ur” means “original”) to distinguish it from the modern, overly broad interpretation.

To understand ur-technical debt, we need to recognize that code has two jobs. Its first job is as a machine: it runs and does valuable work. Everyone recognizes this job. Code’s second job is to express the thoughts of developers. People typically underestimate the importance of this job. Ur-tech debt happens when we evolve the code as a machine but neglect to similarly update it to express our evolved thoughts.

Ur-tech debt is possible because code can be an acceptable machine (that is, compute the right answer) but be a lousy carrier of our thoughts. Code can express our thoughts and Healthy Code Reveals the Problem and Solution, but not all code is healthy, and not all teams are trying to express their thoughts in code.

Ur-technical is a natural result of building iteratively. Iteration only works alongside refactoring – not the superficial cleanup kind of refactoring, but the continuous reworking of the code so that it expresses our thoughts and it “looks like we knew what we were doing all along”, as Ward Cunningham said.

When code carries our thoughts, it enables extended cognition, where the code operates as long-term memory for developers. When you do this right, you think faster, can work on far larger problems, and the Code Is Your Partner in Thought.

Many of these ideas were already clear by the 1990s, as was software architecture. You might wonder, Why Is It Getting Harder to Apply Software Architecture? The reason is that iterative processes combined with factory metaphors (like reducing cycle times and work in progress) encourage developers to focus on the flow of features from request to delivery. That is in contrast to thinking of the system’s overall design, and its health. It’s possible to do both, but only for skilled, experienced developers.

Teams can minimize their tech debt by looking at their development process as an algorithm run by the developers, so you should Garbage Collect Your Technical Debt. Creating tech debt is inevitable. Teams do two things to reduce it: Avoid creating it and clean it up after it exists.

Lots of teams iterate and refactor (that is, they follow The Rituals of Iterations and Tests), but not all of them keep their code in a good state. To keep the code healthy over the long term, teams need Design Focused Iteration, where they iterate on the design, making it better suited to the problem at hand. In contrast, many teams instead follow Code Focused Iteration, where they iterate only on the code. To be agile, it’s critical to recognize these Two Kinds of Iteration.

Future essays will talk about:

• Peter Naur’s ideas on Theory Building. How the focus on incremental features makes it hard to do theory building.
• Distributed Cognition and how successful teams ensure that everyone shares a theory of the system and can evolve it.