IEEE Software - The Pragmatic Designer: Software Architecture Is a Set of Abstractions
This column was published in IEEE Software, The Pragmatic Designer column, July-August 2023, Vol 40, number 4.
ABSTRACT: Software architecture is a set of abstractions that helps you reason about the software you plan to build, or have already built. Our field has had small abstractions for a long time now, but it has taken decades to accumulate larger abstractions, including quality attributes, information hiding, components and connectors, multiple views, and architectural styles. When we design systems, we weave these abstractions together, preserving a chain of intentionality, so that the systems we design do what we want. Twenty years ago, in this magazine, Martin Fowler published the influential essay “Who Needs an Architect?” It’s time for developers to take another look at software architecture and see it as a set of abstractions that helps them reason about software.
IEEE Software - The Pragmatic Designer: Fix Technical Debt with Virtuous Cycles
This column was published in IEEE Software, The Pragmatic Designer column, March-April 2023, Vol 40, number 2.
ABSTRACT: In recent years, teams have found it easy to quickly deliver a working system, but increasingly hard to deliver new features because of tech debt. Tech debt arises from what teams do – and more importantly, what they don’t do – each day.
Teams can use virtuous cycles to produce great code and keep improving it. Virtuous cycles can be found at the scale of methods, modules, and systems. With minimal effort, developers can slow the buildup of tech debt by shifting their perspective on development activities from a checklist to virtuous cycles.
IEEE Software - The Pragmatic Designer: Two Kinds of Iteration
This column was published in IEEE Software, The Pragmatic Designer column, January-February 2022, Vol 39, number 1.
ABSTRACT: There are two kinds of iteration, but they are commonly conflated. The first is the evolution of a design and its implementation to become more suitable over time: design-focused iteration (DFI). The second is the evolution of an artifact (like code) to become more suitable over time: code-focused iteration (CFI).
CFI improves only the code and ignores the refinement relationship between design and code. In contrast, a goal of DFI is nurturing and improving the refinement relationship so that the design becomes more stable and valuable over time.
Refactoring happens in both kinds of iteration. In CFI, the refactoring is shallow and textual. In DFI, the refactoring is conceptual and yields what Domain Driven Design calls “deep models” and “supple designs”.
IEEE Software - The Pragmatic Designer: Garbage Collect Your Technical Debt
This column was published in IEEE Software, The Pragmatic Designer column, September-October 2021, Vol 38, number 5.
ABSTRACT: The iterative process that a team follows is a bit like a garbage collection algorithm, and we can compare software development processes like we can any algorithm. A process can help developers do two things: clean up tech debt after it exists, or avoid creating it. When an iterative process does neither, tech debt buildup will lead to bankruptcy, so it is only suitable for projects with a short lifespan. A process that does both has the best chance at minimizing tech debt over a long lifespan. In particular, focusing on the system’s design will keep tech debt low.
IEEE Software - The Pragmatic Designer: Why Is It Getting Harder to Apply Software Architecture?
This column was published in IEEE Software, The Pragmatic Designer column, July-August 2021, Vol 38, number 4.
ABSTRACT: By the late 1990s, we understood the basic abstractions of software architecture, and it was possible to choose an architecture that suited the problem at hand. Over two decades later, it’s difficult to apply these ideas to our projects, but why? Because we have changed our philosophy to iterative development using factory metaphors, and have reinforced that change with tooling. Developers now focus primarily on incremental changes, and secondarily on the overall design or architecture. These circumstances are inhospitable to the holistic system reasoning that architecture requires, and can explain why so many teams today regard technical debt as their biggest challenge.
Software Architecture 3 Ways
Abstract: Architecture is a specific abstraction, but it is hard to describe it to others. Most of our explanations refer back to shared experiences – but early career developers don’t yet have those experiences. In this talk, I will describe software architecture three ways, none of which refer to shared experiences.
- By showing a system that was redesigned several times with the same requirements.
- By charting the critical insights that added up to architecture, starting in the 1970s.
- By walking through the design of an example system and knocking down some misunderstandings, such as “the code is the truth” and “there’s one right way to build it”.
If you are new to architecture, I hope you will understand architecture better. If you already know architecture, I hope you will have new ways of explaining your passion to others.
This invited talk was delivered at the 2021 Codecamp Conference - The One With Architecture, 24 February 2021.
Contract-based design
This invited talk was delivered at the University of York in February 2021. Here’s a link to the YouTube playlist of all the lectures.
Contracts: an overview
Contracts will make you a stronger software developer. They change your thinking, actions, and values. As you internalize the contract metaphor, you’ll look at software differently.
You will apply logic to source code, not just animate code in your head. You will write contracts in your code, so both you and your team benefits. You will reward clean code and designs, not short-term hacking.
Contractual thinking applies to any kind of programming language, So you can use it with C, Java, Python, Haskell, or anything else.
It’s a transformation that takes time and effort, but the investment is worth it.
IEEE Software - The Pragmatic Designer: The Rituals of Iterations and Tests
This column was published in IEEE Software, The Pragmatic Designer column, November-December 2020, Vol 37, number 6.
ABSTRACT: Teams are right to use iterative processes and tests, but these rituals alone do not ensure a healthy project. Even with iterations and tests, teams face risks from two slippery slopes. The first is the buildup of ur-technical debt caused by a failure to refactor deeply enough, so although they intend to follow an iterative process, teams may be following a sedimentary process. The second is an over-reliance on tests to keep control of a project. By the nature of a slippery slope, a project may be in greater danger than the team perceives, so a project might already be too expensive to rescue, making it a technical zombie. To avoid these slippery slopes, teams must go beyond the mechanical rituals and invent a theory that explains the problem and solution, then ensure the code continues to act as an effective external representation for their theory.
IEEE Software - The Pragmatic Designer: Code is Your Partner in Thought
This column was published in IEEE Software, The Pragmatic Designer column, September-October 2020, Vol 37, number 5.
ABSTRACT: To understand technical debt, it’s necessary to see a program as having characteristics of both a machine and of pure thought. Technical debt arises when a program works fine as a machine but expresses my thoughts poorly. Debt in a program mentally impairs the programmers working on it. I can remove that impairment by refactoring or rewriting my program so that it again expresses my thoughts. Source code carries our thoughts with a directness and immediacy that is not possible with steel or concrete. It enables a cognitive coupling between programmers and their code, a kind of extended cognition, offloading mental burdens into the code and allowing us to produce far more complicated programs.
IEEE Software - The Pragmatic Designer: Ur-Technical Debt
This column was published in IEEE Software, The Pragmatic Designer column, July-August 2020, Vol 37, number 4.
ABSTRACT: The term technical debt was coined by Ward Cunningham in 1992. In recent years, people have broadened the definition to include ideas not present in the original formulation, including lack of skill, expedient hacking, and obliviousness to software architecture. By lumping these together, it’s harder to choose the right repair. This article proposes that we use the term ur-technical debt to refer to the original idea, which was: When I build systems iteratively, my understanding of the problem and solution grow gradually, and inevitably my current thoughts do not match code I wrote earlier, so I must expend effort to fix that code, much like paying interest on a debt.
O'Reilly Software Architecture NYC 2020: Intellectual Control
In the early days of software engineering, Edsger Dijkstra warned us not to let the size and complexity of our programs cause us to lose “intellectual control” due to the limited nature of our minds. Dijkstra never defined precisely what intellectual control was. Our software today is staggeringly larger than the programs of the 1960’s, so does that mean we have it under our intellectual control or did we find ways to make progress without Dijkstra’s high standards?
If you found this interesting, you’ll want to try these next:
- Intellectual Control
- Testing Numbs Us to Our Loss of Intellectual Control
- Code is King; Let’s Think in Code
- Continuous Design of IT Systems
This talk was a keynote at the O’Reilly Software Architecture conference NYC, 26 February 2020.
Here are the slides:
IEEE Software - The Pragmatic Designer: Testing Numbs Us to Our Loss of Intellectual Control
This column was published in IEEE Software, The Pragmatic Designer column, May-June 2020, Vol 37, number 3.
ABSTRACT: Software teams need a healthy balance of both intellectual control, which comes from reasoning, and statistical control, which comes from testing. Complexity is the enemy of reasoning; efforts to maintain intellectual control tend to push complexity down. In my experience, many teams let their intellectual control atrophy, then compensate with more testing. This approach works for a while, but without intellectual control to keep complexity down, progress becomes slower and more difficult. Once lost, intellectual control is expensive to recover, so the teams find themselves in a local maximum they cannot escape.
GSAS 2019: Code is King; Let's Think in Code
The goal of expressing design intent in code has been around since at least the 1970’s. What’s changed is that now code is king: Developers stare at code all day long, our processes track features, and our success metric is deployed code. Other, formerly first-class activities – like modeling, design docs, and software architecture – now happen largely to support code.
As a result, coding is a different cognitive task. Instead of code being the final expression of developers’ thoughts, it’s now a partner in their thinking. In a real sense, the thoughts they think are made possible only because they are staring at code.
This talk builds a cognitive model of software development in three parts.
- Theory building (from Peter Naur)
- Extended & distributed cognition
- Situational awareness
The implication is that we must go beyond the old advice to “express design intent in code” and actually change our development practices. To succeed like other types of engineering, we must support theory building and expression in teams of software developers.
Invited talk at the GSAS 2019 conference Barcelona, Spain.
IEEE Software - The Pragmatic Designer: Better Code Reviews with Design by Contract
This column was published in IEEE Software, The Pragmatic Designer column, Nov-Dec 2019, Vol 36, number 6.
ABSTRACT: Design by contract is a technique that improves the quality of your team’s code. It yields code with both a logical and a procedural nature, where the contracts state declaratively what will happen, and the implementations procedurally cause the desired effect. The team can reason either logically, by using the contracts, or procedurally, by following the code line by line, but the former allows them to reason about far larger programs. It also creates conditions for deliberate practice, so developers using design by contract will grow their design skills faster.
IEEE Software - The Pragmatic Designer: Healthy Code Reveals the Problem and Solution
This column was published in IEEE Software, The Pragmatic Designer column, Sep-Oct 2019, Vol 36, number 5.
ABSTRACT: Source code reveals abstractions from two places: the problem and the solution. It’s easier to design and evolve a system when you understand each of them separately before you combine them in code. With skill, it’s possible to separate those concerns in the code. Declarative understanding of the abstractions is the most useful and easy to convey. However, current software development processes rarely guide developers to do this.
IEEE Software - The Pragmatic Designer: Scale Your Team Horizontally
This column was published in IEEE Software, The Pragmatic Designer column, July-August 2019, Vol 36, number 4.
ABSTRACT: We’d like to add an engineer to our project and have our team get that much more power. The primary factor that allows developers to contribute is the state of the code. To keep the code clean, we must shoot for two goals to minimize a project’s ur-technical debt. First, enable developers to contribute according to their ability, not according to their tenure. Second, keep the design small enough to fit in everyone’s heads. This doesn’t contradict Brooks’ Law, which applies specifically to late projects.
SATURN 2019: Continuous Design of IT Systems
I’ve been applying a mixture of agile, Domain Driven Design, functional programming, and architecture ideas to Information Technology (IT) systems. Most architectural case studies report on one round of design but my projects were being designed and redesigned continuously. This talk covers some insights I’ve had and some ways of working that may help you on your projects.
-
Agile and architecture differ in their use of control systems (open loop, closed loop) to guide projects. We can mix them but only when the team agrees to the mixture, eg agilists don’t shout YAGNI and architects don’t stop the presses. We should use open loop control for some things and closed loop for others.
-
The team won’t have a single architect but neither will it have silos of expertise. Instead, the team creates and evolves a shared understanding of the architecture that is communicated verbally and represented in the codebase (including types, framework components, comments, and READMEs).
-
The team maintains intellectual control of the system. They keep the state space small and sufficiently well-understood to reason about it and derive conclusions. The design is continuously changing so they evolve the code to match that understanding and actively communicate relevant changes to the rest of the team.
-
Humans are not ideal computing machines and work better with some external representations (ie source code) than others. To facilitate human reasoning, teams should mercilessly refactor their code so that it matches their internal theories of the requirements and the design. Without aggressive refactoring, complexity quickly accumulates, preventing axiomatic reasoning and intellectual control. Theory building is refactoring.
-
We should expect the code tied to the domain to converge as the team masters the requirements. The architectural infrastructure, however, may never converge. As the team’s theories become clearer, the source code (in the small) will take on a functional, definitional flavor regardless of programming language.
Invited talk at the SATURN 2019 conference.
SATURN 2019 Workshop - Survivor: Architecture Island
This year’s workshop at SATURN asked its participants to narrow the long list of architecture ideas, much like the TV show Survivor. Our hope was to arrive at a short list we could use to teach newcomers.
What we discovered was that there were core ideas but they varied by the makeup of the team (experienced, distributed, etc.) and by the kind of project (safety-critical, IT, etc.). This was interesting and raises some good questions to dig into.
What are the big ideas in software architecture?
Position paper for the SATURN 2019 Workshop “Architecture Island”.
In this workshop, we’ll be discussing which ideas about software architecture we’d keep and which ones we’d be forced to abandon as we confront the dangers on Architecture Island (rather like the TV show Survivor). It seems like a good idea to survey some of the most popular books on software architecture to create an inventory of what ideas we are starting with.
I intended to survey about a dozen books but it took more time than I’d expected so I (so far) have only done two plus my Intro to Software Architecture lecture from 2012. For this workshop, any list is better than no list, but I’d find it interesting to see which ideas span all or most of the books on software architecture.
I picked up Documenting Software Architectures (2nd edition) first as I remembered it as being much like a catalog itself. I skimmed the book and jotted down ideas or topics. There’s a fair amount of bias in this activity: in my haste I may not have noticed an idea, a different reader might notice different ideas, and of course names differ across books and I was trying to align them while jotting down notes. Then I did the same for Software Architecture in Practice (3rd edition) and my intro lecture. I merged those lists into a table, doing some more subjective merging and omitting of topics. If by chance any of the workshop participants notice errors or omissions (I’m looking at you Len Bass) then please let me know and I’ll fix up the table.
Notation: A question mark in the table indicates that I don’t think the topic was covered but I wouldn’t be surprised if I’d overlooked it. The asterisk in the table is meant to be a bullet that nests under the line above, but my efforts to make it render that way have not yet worked.
I included that intro lecture not because it’s in the same league as the two books but because as of 2012 it was my idea of what I’d keep on Architecture Island and I was curious what I’d left out.
| Topic | DSA 2nd Edition | SAP 3rd Edition | Intro to SA |
|---|---|---|---|
| Views (module, C&C / Runtime, Allocation) | Yes | Yes | Yes |
| * Master model unifies views | No | No | Yes |
| Stakeholders | Yes | Yes | No |
| Quality attributes (vs functionality) | Yes | Yes | Yes |
| * Architecture inhibits or enables | Yes | Yes | Yes |
| * Tied to failure risks | ? | ? | Yes |
| Requirements: QA vs feature | Yes | Yes | Yes |
| * QA requirements drive architecture | Yes | Yes | Yes |
| * QA scenarios | No | Yes | Yes |
| * Argument: How this architecture meets requirements | Yes | No | No |
| * Architecturally significant requirements (ASRs) | ? | Yes | Yes |
| Architecture is a subset of design | Yes | ? | Yes |
| Contexts: Technical, lifecycle, business, professional | No | Yes | No |
| Architectural styles / patterns | Yes | Yes | Yes |
| * Module style: decomposition, uses, generalization, layers, aspects, data model | Yes | No | No |
| * Runtime style: pipe and filter, client-server, peer-to-peer, SOA, pub-sub, shared-data, multi-tier | Yes | No | No |
| * Allocation: deployment, install, work assignment | Yes | No | No |
| Tactics | No | Yes | No |
| * Attribute driven design | No | Yes | No |
| Architecture diagrams (vs cartoons) | Yes | ? | No |
| Notations and ADLs: element catalog | Yes | No | No |
| Product line architectures | Yes | Yes | No |
| Architectural decisions | Yes | ? | No |
| * Rational architecture decisions | No | No | Yes |
| Interfaces: provided and required | Yes | No | No |
| Design reviews | Yes | Yes | No |
| Align architecture and business goals | No | Yes | No |
| Tradeoffs (and ATAM) | No | Yes | Yes |
| Separate: Architecture, architects, architecting | No | Yes | Yes |
| Constraints / guide rails | Yes | Yes | Yes |
| Conceptual model of architecture | Yes | Yes | Yes |
| Architecturally evident coding style | No | No | Yes |
| Architectural hoisting | No | No | No |
IEEE Software - The Pragmatic Designer: Principle of Least Expressiveness
This column was published in IEEE Software, The Pragmatic Designer column, May-June 2019, Vol 36 number 3.
I’m always delighted to discover a connection between two ideas that I’m already fond of on their own, so I’d like to share a connection I found recently. The first idea is writing code that expresses my thinking about the problem domain, and the second is the principle of least expressiveness (PLE). The connection is that I can use the PLE to reveal my thinking about the problem domain, and because all ambiguity stops at the code, the act of programming using the PLE can help me simplify and debug the flawed ideas I have in my head.
The PLE [3] is as follows:
When programming a component, the right computation model for the component is the least expressive model that results in a natural program.
The least expressive model means that if you can express your idea with a constant, use that, and similarly for lookup tables, state machines, and so on. You should only use a Turing-complete language when you cannot use something simpler—with the caveat not to contort the code.
IEEE Software - The Pragmatic Designer: Ignore, Refactor, or Rewrite
This column was published in IEEE Software, The Pragmatic Designer column, 21 February 2019, Vol 36 number 2.
Imagine that you have some code written, but it has problems. The problems are small enough that you could imagine rewriting the code completely, and you must choose what do. You could do nothing (ignore it), make incremental changes (refactor it), or write new code from scratch (rewrite it). How do you choose? What factors do you consider?
There’s already a lot of guidance. In fact, the very existence of refactoring on the list of choices is special because the idea of refactoring code wasn’t well formed until the 1990s. When you refactor code, you make changes that improve its structure but do not change its visible behavior, and our tools are increasingly good at supporting refactoring, helping us make sweeping changes safely.
Most of the guidance applies to smaller chunks of code and decisions implemented in hours or days, not weeks or months. I’ve long wished that there were a great book on architecture-scale refactoring with distilled wisdom and case studies of successes and failures. This article only touches on that but it covers some topics that augment the good advice you will find in Martin Fowler’s Refactoring book [1] and Michael Feathers’ Working Effectively with Legacy Code [2]. For a good case study of architecture evolution and decision making, I suggest Tony Tsakiris’s article [3] in which he describes how Ford looked at its vehicle control interfaces across many car brands and how it chose to embrace, tolerate, or deprecate each of them.
IEEE Software - The Pragmatic Designer: Intellectual Control
This column was published in IEEE Software, The Pragmatic Designer column, 14 January 2019, Vol 36 number 1. It reached the home page of Hacker News but got few comments.
In the early days of software engineering, Edsger Dijkstra warned us not to let the size and complexity of our programs cause us to lose “intellectual control” due to the limited nature of our minds. To my knowledge, he never defined precisely what intellectual control was. Our software today is staggeringly larger than the programs of the 1960s, so does that mean we have it under our intellectual control, or did we find ways to make progress without Dijkstra’s high standards?
I see signs that we have some software that is under intellectual control and other software that is not. In this column, I’m going to discuss how we can recognize these two categories, what happens when engineers on a project have different attitudes about intellectual control, some advice on when we probably should insist on it, and some ideas about how we achieve it.
YOW! 2018: Code Is Your Partner in Thought
We used to build software like bridges. There was time for careful collection of requirements and the analysis of design options. But we don’t do that anymore, in part because the time we spent did not ensure we hit our targets or avoided risks. Today, most companies use some form of continuous design where the software changes in small steps, often weekly or even daily, which makes those companies responsive to environmental changes. The problem is that in just a few years the software becomes over-complicated and usually needs to be rewritten, even when the team refactors. The longer you let the team code, the worse the code gets. That’s a shame because other kinds of engineering don’t have this problem. Car engines, for example, improve every year. I’ll primarily discuss some ideas about why things work out this way and also some promising ideas for how we can practice continuous design longer, if not forever.
Invited talk at the YOW! 2018 conference series in Sydney, Brisbane, and Melbourne.
ILTAM 2018: Code Is Your Partner in Thought
We used to build software like bridges. There was time for careful collection of requirements and the analysis of design options. But we don’t do that anymore, in part because the time we spent did not ensure we hit our targets or avoided risks. Today, most companies use some form of continuous design where the software changes in small steps, often weekly or even daily, which makes those companies responsive to environmental changes. The problem is that in just a few years the software becomes over-complicated and usually needs to be rewritten, even when the team refactors. The longer you let the team code, the worse the code gets. That’s a shame because other kinds of engineering don’t have this problem. Car engines, for example, improve every year.
Keynote address at the 2018 ILTAM Israeli Conference on Software Architecture.
SATURN 2018: How are we growing great software designers?
This is my position paper for the SATURN 2018 workshop on Growing Great Software Designers. You may also be interested in the workshop’s summary outcomes.
How are we growing software designers?
George Fairbanks
6 May 2018
SATURN Workshop: Growing great software developers
Using broad strokes, we can paint a picture of how we grow developers today and compare that with how we did it in the past, then make some guesses about how we could do it in the future. We can start our picture by listing the activities that we use to grow. By looking at how frequently the activites have been used and who is driving them, I can see some trends.
SATURN 2018: Refactoring to Functional Architecture Patterns
Last year at SATURN I spoke about ideas from the functional programming (FP) community that are relevant to software architecture. This talk is an experience report based on applying those ideas and represents one way to marry functional programming with traditional OO design advice.
We followed a pattern we call the Rich Service layer. It is implemented as pure functions plus a modified version of what Fowler calls an Anemic Domain Model. Historically, the Transaction Script pattern has been contrasted with the Domain Model pattern, where objects have methods expressing the business logic.
Our domain model expresses strictly defined types with as little optional data as possible, not just jumbled data bags. Our domain types are as strictly defined as possible. We check integrity at the system boundary. Compared to object-oriented programming, we don’t push much behavior onto the types, which are immutable. We grew an (internal) Domain Specific Language (DSL) organically so the code reads naturally. But the DSL mainly expresses predicates on state rather than mutations. There is little business logic, which is instead in functions in the rich service layer.
We Are Developers 2017: Model-Minded Development (part 2)
Abstract:
We want to write good code but we don’t know exactly how to do that. This talk suggests one way, which is to focus on expressing your theories (i.e, models) in the code. That has three parts: a model of the domain, a model of the design, and an argument about why this design has the desired effect on the domain.
While this talk can stand alone, its best to see part 1 first.
Delivered at We Are Developers 2017 in Vienna.
SATURN 2017: Functional Programming Invades Architecture
Abstract:
A few decades ago when the first architecture books were being written, today’s large-scale web systems were an oddity, but now they are mainstream. Our architecture pattern languages have changed as have our development, deployment, and operating procedures. During this transition, one source of ideas has been the functional programming (FP) community.
FP itself mostly happens within a module, and many software architects treat it as an implementation choice, unrelated to architecture. But some ideas from the FP community are helpful in architectural design: statelessness, immutability, and pure functions. They underlie DevOps practices and are at the core of many distributed systems patterns in our architectures.
Although functional programming ideas have been around for a long time, conditions are ripe for applying them now. This talk surveys how modern web systems are built with FP patterns, shows how they work, and considers why they are increasingly popular. We will dig into one client-side example and demonstrate how FP ideas can simplify tasks.
Delivered at SATURN 2017
SATURN 2017: Functional Programming In Software Architecture
This is my position paper for the SATURN 2017 workshop on Architecture and the Functional Frontier. You may also be interested in my presentation on functional ideas in software architectures and the follow-up case study presentation.
ABSTRACT: Functional programming (FP) has been around for a while but has largely been treated as a concern inside a module, a Programming in the Small concern [1], rather than an architectural concern. The collection of ideas within FP are also found in other places and many originate in mathematics. So it is not surprising that those ideas at times were used in software architectures. It is interesting to see how many ideas are now found in both architecture and FP, often due to the challenges of reasoning about and building distributed systems.
GOTO Berlin 2016: Model-Minded Development
Abstract:
Senior software developers can walk up to a whiteboard and give an impromptu talk on how their software works, explaining both the details and the broad strokes. Most importantly, they connect the specifics of this system to general architecture patterns, domain models, design patterns, development patterns, or programming styles. How do we become “senior software developers”, other than waiting until we are older? The answer is models.
This talk covers Model-Minded Development. It briefly shows how we underestimate the cognitive task of programming and how we consequently use models to amplify our own cognitive abilities and to coordinate our ideas with others. Then we’ll dig into source code and compare equivalent programs that do better and worse jobs of expressing models in the code.
Delivered at GOTO Berlin 2016
GOTO London 2016: Model-Minded Development
Abstract:
Senior software developers can walk up to a whiteboard and give an impromptu talk on how their software works, explaining both the details and the broad strokes. Most importantly, they connect the specifics of this system to general architecture patterns, domain models, design patterns, development patterns, or programming styles. How do we become “senior software developers”, other than waiting until we are older? The answer is models.
This talk covers Model-Minded Development. It briefly shows how we underestimate the cognitive task of programming and how we consequently use models to amplify our own cognitive abilities and to coordinate our ideas with others. Then we’ll dig into source code and compare equivalent programs that do better and worse jobs of expressing models in the code.
Note: I had flown overnight and was jetlagged when delivering this talk. The content is OK but you can tell I’m nursing a cold and had 4 cups of coffee. The GOTO Berlin talk is the same content with better delivery.
Delivered at GOTO London 2016
SATURN 2016: Centralized vs. Decentralized Approaches to SOA: Hamilton vs. Jefferson
Talk from SATURN 2016 with Michael Keeling.
Modern service-oriented architecture (SOA) systems force teams to reconcile a multitude of organizational and technology decisions. With each decision, the team reaffirms allegiance to its chosen message passing, platform governance, and quality assurance philosophy. Which side are you on: Centralized SOA or Decentralized SOA?
During this session, we will explore essential topics in modern SOA including governance, message passing strategies, orchestration, “smart” and “dumb” pipes, quality assurance strategies, deployment, and many other topics. For each topic, we’ll cover the most important information you need to know and debate the pros and cons of a centralized and decentralized approach.
And because it’s Michael and George hosting this session, we can’t just leave it at that. In the spirit of the American Federalists (strong central government) vs. Antifederalists (decentralized government) of the 1790s, George, playing the role of a modern Jefferson, will advocate for decentralized SOA while Michael, as a modern Hamilton, will attempt to convince you, the audience, that centralized SOA is the best path forward. This is a session you will not want to miss!
SATURN 2016: Model-Minded Development
This is a shorter (30 minute) and less theoretical version of the WICSA 2016 keynote talk from earlier this year. I’d recommend watching this first until you decide you are curious for more.
- Domain-Driven Design (DDD) says that we should be mindful of domain models and embed them in our code.
- Design Patterns say that we should know a catalog of patterns so we can solve recurring problems that arise in object-oriented (OO) code.
- Software Architecture says that unless we are mindful of large-scale patterns and models, then our systems will not achieve the qualities we seek.
- Test-Driven Design (TDD) says that we should structure our code so that it can be more easily tested.
- Programming styles (functional, OO, procedural, etc.) say what the core abstractions of our programs should be.
The common denominator here is that software developers are expected to keep in mind many abstract yet complex models that constrain the code they write. In some ways, these constraints are a burden, and in other ways, they are the light that illuminates a path forward.
I will discuss an idea called Model-Minded Development that generalizes across DDD, design patterns, architecture, TDD, and coding styles. The defining characteristic of senior software developers is their facility with Model-Minded Development, and it enables them to operate at an advanced level.
An IT Application Developer's Perspective on Containers
Position paper for: SATURN 2016 Workshop on Containers George Fairbanks 17 April 2016
Introduction
Since I’ve been an IT application developer for most of my professional career (about 20 years) and have sat on the sidelines admiring and studying the work of the “systems developers”, my position paper describes a personal account of a few trends I’ve witnessed regarding how IT applications are built and how containers have affected the trends.
The first trend involves IT and “systems”. In the early days of application development, computing resources were scarce so all problems were systems problems where developers had to carefully choose algorithms to deliver anything successfully. Over time, resources became more abundant and tech like relational databases made routine IT application development easy. In recent years, distributed computing has become more prevalent and forced IT developers to again pay attention to “systems” concerns.
Another trend involves the hardware. In earlier days, developers chose the hardware that made their software easier to write and easier to achieve performance goals, but in modern times developers must figure out how to build software that runs on provided, homogenous data center hardware.
The final trend involves how containers have evolved. Early containers, often called “application engines” and eventually standardized via Servlets, J2EE, and .Net, focused on hosting the application logic only. As virtual machines became viable, an entire machine from the hardware API was hosted. Today’s containers host the operating system or its processes. This modern version of containers seems to be the best fit, the “mama bear”, for IT application developers and it has seen wide adoption and enthusiasm.
An IT Application Developer's Perspective on Containers
Position paper for the SATURN 2016 Workshop on Containers.
Introduction
Since I’ve been an IT application developer for most of my professional career (about 20 years) and have sat on the sidelines admiring and studying the work of the “systems developers”, my position paper describes a personal account of a few trends I’ve witnessed regarding how IT applications are built and how containers have affected the trends.
The first trend involves IT and “systems”. In the early days of application development, computing resources were scarce so all problems were systems problems where developers had to carefully choose algorithms to deliver anything successfully. Over time, resources became more abundant and tech like relational databases made routine IT application development easy. In recent years, distributed computing has become more prevalent and forced IT developers to again pay attention to “systems” concerns.
Another trend involves the hardware. In earlier days, developers chose the hardware that made their software easier to write and easier to achieve performance goals, but in modern times developers must figure out how to build software that runs on provided, homogenous data center hardware.
The final trend involves how containers have evolved. Early containers, often called “application engines” and eventually standardized via Servlets, J2EE, and .Net, focused on hosting the application logic only. As virtual machines became viable, an entire machine from the hardware API was hosted. Today’s containers host the operating system or its processes. This modern version of containers seems to be the best fit, the “mama bear”, for IT application developers and it has seen wide adoption and enthusiasm.
IT systems and “Systems systems”
Let’s admit that it sounds silly to say “systems systems”. But we have a subdiscipline in computer science that is called “systems” ( not to be confused with the overall engineering subdiscipline called “systems engineering”) and as far as I know there’s no easy term for the software that the CS systems subdiscipline builds. By “systems systems” I mean things like operating systems, device drivers, distributed systems, mail servers, and databases. Developers of such “systems systems” traditionally spend a lot of their time working on algorithmic problems and distributed systems problems. Systems developers tend to build long-lived software that evolves but is still recognizable decades later, often still using the same standard messages or protocols (eg TCP/IP, SMTP, SQL).
So-called “IT systems” (where the acronym IT unpacks to Information Technology, not much of a clarification in itself) aren’t generally treated by academics as a subdiscipline of computer science or engineering, despite being the most common (probably overwhelmingly) kind of system built in industry. IT applications facilitate running a commercial business and the users are (generally) not engineers or computer scientists. Developers of IT systems traditionally spend little of their time on algorithms but a lot on messy, frequently changing business requirements. They also do so under short deadlines. IT developers built huge numbers of short-lived systems and a few long-lived systems whose maintainers wished they had been retired long ago.
As a result, IT developers have created strategies for shooting at their moving target whose incoherent requirements they cannot anticipate nor refactor into consistency. In particular, relational databases have been a critical part of the strategy and I sometimes refer to them as the Jedi lightsaber of IT development. (NB I find that most developers don’t see them that way and instead use them out of inertia – either their project was already using one or it’s simply the presumptive architecture). Confronted with the need for multiple clients to access shared data and shifting, unpredictable requirements, relational databases provide a nearly perfect solution: (1) safe concurrent access to shared data with little to no burden on developers, and (2) an ability to store the data today in a normalized format and expect acceptable performance when accessing that data tomorrow, or even after evolving the schema.
In contrast, Systems developers often have the luxury of a relatively slow moving target (eg a mail server), standard formats/protocols, and flexible requirements allowing developers to homogenize or smooth over arbitrary differences. But their quality attributes are harder, often requiring vastly better reliability, efficiency, or scale than IT systems routinely attempt.
I’m sure that this broad-stroke depiction of IT and “systems” systems is inaccurate in its details. However, it should serve well enough to juxtapose two types of application development so I’ll now turn to how IT systems have evolved over time.
IT systems and “systems systems”: at first similar, then distinct, now becoming similar again
Here is a cartoon, oversimplified history of IT application development. In the early days, the scarcity of computing resources meant that every IT system developer was forced to confront “systems” challenges, choose algorithms carefully, and generally little few off-the-shelf tech to make their jobs easy. Early IT systems avoided being distributed systems as much as possible, with the earliest mainframe IT applications being basically one-tier applications. As personal computers became dominant in the late 1980’s, client-server computing became the norm for IT applications and developers had to confront distributed computing problems (which I’ll consider to be a traditional “systems” problem). Into the 1990’s, issues of scale forced the server to be divided up with one machine hosting the database and one or more hosting the server-side of the application, furthering the distributed computing problem. Then the database was too big for a single machine and was split across many machines. Then the applications started using not just their own database but also other systems as backends and truly became a distributed system because they had to confront distributed transactions (or confront the lack of them).
This cartoon glosses over the fact that different developers confronted these changes at different times and, for example, many of them had to handle multiple backends and no transactions even in the early client-server days. But it highlights an important trend with respect to IT vs “systems” development: For a couple of decades, IT developers could often build their applications without seriously confronting “systems” problems because the presumptive architectural patterns were so effective. In the early days, they had to confront systems problems, then they could mostly ignore them (say 1990 - 2010), but they now have to confront them again.
To be clear, the kinds of systems problems that IT developers confront include: RPC failures, load balancing, server stickiness and statefulness, distributed cache coherence, network partitioning, and distributed transactions (or the lack of them). It also importantly includes the need to predict data access patterns and carefully choose data structures in advance that can execute quickly – something that was a non-issue when the application had a single normalized relational database.
Added to the modern challenges is the need to configure a system for testing. Test-driven development (TDD) has been one of the best and most widely adopted Agile practices, so IT developers must figure out how to execute not just unit tests on their business logic but also integration tests on what is now a distributed system. Patterns include re-using existing “dev” or “test” servers carefully such that different developers and different tests do not interfere (this is hard), and bringing up fresh copies of the servers for each developer or test. The former pattern is perhaps the most common but perhaps just as common is that it fails to properly isolate developers or tests, leading to false positive failures and developers wasting time debugging ghost issues. The latter pattern has historically been impossible (eg not enough resources on the developer’s local machine to run all the backends and databases) or too expensive (virtual machines require heavy resources and take a long time to initiate, usually too long for tests when the developer is waiting but OK for batch testing).
Before discussing containers directly, it’s important to look at the hardware that IT applications have run on.
Software teams choose their hardware becomes homogenous datacenters
In the old days, you bought hardware that was suited to the task of running your software. Thought went into choosing the right mix of processing speed, memory, storage, and database tech. Datacenters were heterogenous. Worse, since your hardware was not commodity, you had to order it before you had finished (or even started) writing the software. Sometimes a higher priority project would steal your hardware order before you even received it, delaying your launch.
But then there was an inversion of primacy and the ability to scale datacenters efficiently became the most important consideration. The hardware was homogenized. In a reversal, software developers had to figure out how to design software that ran on the available hardware.
The pendulum has started swinging the other direction. Datacenters now offer many types of machines (high ram, fast network, high cpu) to better match the workloads.
Current pattern languages (for code) and hardware capabilities (eg memory, cpu, io) are sufficient to handle the hardware primacy for IT and web applications. Some large scale and compute intensive tasks must still invent their own patterns to deal with hardware primacy.
Modern containers are the “Mama Bear” of containers
1990s container tech (application engines, eg servlets, .Net services) was too tilted towards the IT problems. How so? Limited access to tune performance. Few (or single) languages. Certainly not process-level tuning. More of a “trust the container to load balance” etc to achieve scale, performance, etc. sometimes with horrible results compared to hand-tuned solutions.
2000’s container tech (virtual machines, eg VMWare) was too tilted towards systems problems. Full control but … you have to do it all yourself. Still very manual (until the late 2000’s as scripting started to arrive). Convenience of running a VM, relocating it, sharing images, but per-machine efficiency not great.
2010’s container tech (lightweight containers, eg Docker, Kubernetes) seems like the right fit between IT and Systems problems.
As described above, current IT systems are in some ways confronting systems problems but aren’t ready (and arguably shouldn’t be distracted) with all the knobs and details of pure systems problems and their container tech. Scripting and out-of-the-box usability is much better, as is per-machine efficiency. Developers can spend less % of their attention on the deployment aspects compared to earlier container tech while getting better results.
WICSA 2016: Building Theories is Building Value
Abstract:
I find a program much more valuable when I can read it and understand the abstractions that collectively explain a theory of the problem and solution. In contrast, I find programs with minimal abstractions and lots of conditional logic hard to understand because there is little or no theory to be inferred. The object-oriented programming community has long sought correspondence between models of the domain and the classes in a program, but other communities strive for this too.
But the theory goes beyond just domain models, as solution models (design patterns, architecture patterns) and “critical thinking models” (eg design by contract and the functional thinking embodied in Z and VDM) are equally important.
For several years I’ve been trying to relate the many models that I build on software projects and I believe it’s a combination of models of the domain, the solution, and these critical thinking models. This leads to the question of what value a programming team is contributing, especially when “repeated delivery of stakeholder value” is the primary metric used today.
I argue that:
(1) programs that embody the theories of their developers are more valuable,
(2) how well developers can evolve a program is related to their ability to build and evolve theories, and
(3) a key distinguishing characteristic of a company’s most senior developers are their theory-building traits.
Delivered at WICSA/CompArch 2016
Losing the battle with complexity
Complexity is the #1 problem in software. Fred Brooks was right with No Silver Bullet, but there are levels of understanding his point. A trivial understanding has us realizing that no new tech will slay the werewolf, much as we hope it will. I’ve been through several ups and downs of anticipation of improvements, and indeed there have been improvements.
Deep down we hope that we are Neo and that if we achieve enlightenment that we can stop bullets and no foe can defeat us. I’m not downtrodden but today I realize that Neo can choke on a potato chip like anyone else. In fact, he may do that while he’s stopping bullets (or some other heroic task). Having mad samurai skills just increases your understanding of what’s happening and the chances you’ll not lose stupidly – but it’s not possible to eliminate the chances.
I hope that with more knowledge and experience that I can build simple systems more often, maybe even all the time. But I must acknowledge that no amount of knowledge or expertise will slay essential complexity. And seeming progress is an illusion based on accidental complexity that I now know how to slay.
“Essential” and “accidental” are great names for powerful ideas. In many ways I want to think that being insightful or clever reduces the essential complexity of my code, but by definition it cannot. What’s worse is that as requirements evolve some of my clever ways to simplify the problem may become invalid. For example, today I realized that our convenient use of an environment flag to set configuration is effective but quite fragile and likely not to survive in the face of changing requirements.
So, I’m not Neo and neither is anyone else. Every day we wake up, tie our shoes, and do battle with complexity. It can be worse than thankless because many of our co-workers aren’t as enlightened as us, they don’t see the complexity that they have created. (Indeed, being somewhat enlightened I realize how much simplicity my code lacks!) They resent the pressure and the criticism, despite how lightly I try to deliver it.
Injection, Modularity, and Testing: An Architecturally Interesting Intersection
Dependency injection, code modularity, and testing often seem like staid, even boring, topics but there are surprises when you put all three together. Seemingly independent decisions about each influence the others. Making rational development decisions each day can still lead you into a thicket of complexity and loss of maintainability. So, when should you give it your “architectural attention”? This talk discusses the problem and suggests some limited solutions.
Building Models Quickly and Carefully
This video was done as a PechaKucha talk at SATURN 2015, meaning that the slides were set to auto-advance every 20 seconds and it’s just 20 slides. I have been giving a whiteboard talk to co-workers to explain this technique for years, so it seemed like a fun idea to try to do it in this rigid format. As you can hear in the video, Michael Keeling was skeptical but I think it works pretty well.
Software Architecture Book Recommendations
Also see my older writeup on recommended architecture books.
Princeton ACM 2015: Model minded development
I gave a talk to the Princeton chapter of the ACM on Model Minded Development.
Many slides have no text, so you’ll want to go to the Full slide deck with speaker notes.
Abstract
- Domain-Driven Design says that we should be mindful of domain models and embed them in our code.
- Design Patterns say that we should know a catalog of patterns so we can solve recurring problems that arise in OO code.
- Software Architecture says that unless we are mindful of large-scale patterns and models then our systems will not achieve the qualities we seek.
- Test-Driven Design imposes a structure on our code so that it can be more easily tested.
- Programming styles (functional, object-oriented, procedural, etc.) impose structure on our code.
The common denominator here is that software developers are expected to keep in mind many abstract yet complex models that constrain the code they write. In some ways these constraints are a burden and in other ways they are light that illuminates a path forward.
I discussed an idea called Model Minded Development that generalizes across DDD, Design Patterns, architecture, TDD, and coding styles. The defining characteristic of senior software developers is their facility with Model Minded Development and it enables them to operate at an advanced level.
Agile, Discipline, and Microservices
Position paper for the SATURN 2015 Workshop on Microservices.
Abstract
Agile processes and thinking have helped software development overall but they require discipline from the software teams and that discipline is often lacking. Developers that fall short of the required discipline are branded as Coding Cowboys but that term may describe the majority of us. Microservices architecture, an increasingly popular architectural style, is a kind of distributed system that requires even more discipline, prompting the question, “Does our field collectively have enough discipline for Microservices to become commonplace?”
2001-2015: Agile raises the bar and removes guilt
Here is a broad and biased account of software development now versus a decade and a half ago when XP and Agile started out. Back then, teams used to feel guilty about not “doing software right”, meaning that they had notions of some idealized process they were supposed to follow, usually involving analysis and documentation in addition to the code, and they were somewhat embarrassed when outsiders or newcomers saw what they were actually doing. And what they were actually doing was some variant of a waterfall process with design document templates guiding them to write down some requirements, some analysis of the problem, and a sketch of their design often including a class diagram. The popular analysis was object-oriented, perhaps with flavor from a pundit. Software delivery happened as often as every couple weeks or as infrequently as a couple times per year. Integration was done just before the whole thing was handed off to a separate testing team, who probably suffered similar guilt about “doing testing right”. Infrastructure to support other processes was thin, though some built their own.
There were teams back then doing iterative development with robust automated test harnesses, but they were the minority. I recall the huge improvement in camaraderie and quality our team experienced in the late 90’s when we adopted a 2-week iterative process but at the time short iterations were viewed as radical and risky. William Gibson has said that the future is already here but it’s not evenly distributed. So the right question is not whether Agile is the ideal way to do things but whether in aggregate do more team do better with it.
From the companies that have seen through my consulting work, the state of the practice today is much different and on balance better, largely because of the Agile community, its ideas, and the tools that come with it. Teams are more likely than not to be building incrementally, have automated tests and builds, and sometimes have automated deployments. Most of what I consider bad bureaucracy in software process has been swept away cleanly. Teams focus on developing features and no longer feel guilt about some mythical development process ideal that they are not living up to.
Given the widespread improvements, It would be foolish to wish for the old days, but perhaps I can be forgiven for hoping that we can bring back attention to some things that were perhaps over-hastily or accidentally discarded during the revolution. I find that domain/requirements analysis and software design abstractions, including architecture models, get less attention than they used to. You rarely need to remind a developer to write code – that’s a basic instinct for us – so the guilt about “I really should have analyzed this problem better” and “I really should read more books on software design” were an effective counter-balance to those instincts.
Despite warnings that you should not simply dump your old process and grab a miscellaneous bag of agile practices, I have seen more a-la-carte Agile processes in practice than XP or other curated practices. One of the Agile principles is that Agile processes should be sustainable, with feature delivery or quality not slipping over time. The mutt processes I see are more often focused on short-term wins at the expense of sustainable development.
What I rarely observe on teams is any guilt about doing things the wrong way. Today, being Agile means not having to say you are sorry. This ranges from teams that are effectively coding cowboys using “we are Agile!” as a shield to teams that are well-intentioned but whose collection of Agile practices are not a substitute for the old processes they dropped. Not only are teams guiltless that they have dropped waterfall processes, design documents, etc., they do not seem to feel guilt about dropping the discipline of applying a coherent set practices of that Agile process developers exhort.
Managers and teams no longer hear their guilt talking when someone says that they should choose a suitable architecture. Teams focused on features pay little attention to architecture abstractions – perhaps no more or less than they used to, but now it is harder to get their attention.
Enter Microservices
It is with this jaded, skeptical, and even curmudgeonly attitude about team discipline and attention to abstractions that I view the recent excitement about using the Microservices architectural style to convert monolithic systems into distributed systems. Let’s start with some good qualities of monoliths compared to distributed systems.
The big chunks of software in a monolith are usually written in the same programming language and an IDE can easily figure out what connects to what, so with a keypress a developer can find all the callers of a method. This is generally not true of distributed systems. Similarly, an IDE with all the source code loaded will tell you instantly if A calls B with the wrong number or wrong type of parameters. Distributed systems often lack this info and instead rely on tests or even production logs to find the bugs. For a developer, it is much easier to read the source code and reconstruct flows through a monolith than it is through an otherwise isomorphic distributed system. With monoliths, developers have to reason about evolution of persistent schemas, such as records in a database. Some techniques for handling it include migration scripts and methods that tolerate the old and new versions of the data. The burden is higher for distributed systems because each node could be running a different version of the software, sending messages in old or new formats.
These challenges from distributed systems are not just another arithmetic addition to a developer’s load but a multiplicative factor in the complexity of the system. They make everything a developer does harder by some percent. Given that we each have cognitive limits, this means that we will need to be more disciplined when building a Microservices system than an equivalent monolith. Where will that extra discipline come from?
Optimism
Despite my concerns about Agile processes needing more discipline than many teams seem able to muster, there are solid reasons for optimism that the practice of software development is moving in a good direction and Microservices will contribute positively.
Big monoliths are undeniably worse than Microservices for all the reasons that Microservices are touted, including ability to experiment with different technologies, ability to rewrite chunks as needed, enforced modularity boundaries since each service runs in its own process, and better sub-team velocity because of smaller more reliable test suites. Some monoliths are big balls of mud and the bigger the monolith the more of a problem that is.
Enterprises already build and run on systems-of-systems, even if they do not think of it that way. Microservices and SOA force that recognition, which in turn makes it more likely that the problems that arise will get attention. The future of software is not un-networked desktops but a sea of networked devices, so we should choose architectural patterns that make building those systems easier.
Frameworks are the unsung champions of modern software development, though no champion is more often maligned. Frameworks are what allow a 10-line program to respond to an HTTP request or render “Hello, world” on an anti-aliased display. Some in the Microservices community are reasonably skeptical of frameworks because they can lessen the independence of each microservice but I expect that frameworks will play a major role in reducing or eliminating pain points in microservice adoption.
When you reflect on software development as it was done 15 years ago, it is difficult to overlook the support tools that we take for granted now, including automated and continuous test and build environments. Such tooling will continue to improve in ways that we cannot yet forecast and it will lessen the burden of building and debugging distributed systems.
One last bit of optimism is about the Agile community’s drive to improve. The attention of the community has shifted over the years, focusing on the topics that seem most pressing. The Agile manifesto signatories had their roots in object-oriented development and design patterns. As the community grew, it turned its focus to software development processes, testing, and refactoring. Its current focus is on automated deployment and DevOps – and perhaps Microservices. It is my hope that with the best Agile teams operating with good processes, testing, deployment, etc. that the next item on the agenda will be improving sustainable development. Solid software architecture, design, and abstractions are essential to maintaining developer velocity and I welcome the Agile community’s focus in this area that we SATURN folks hold dear.
IEEE Software - Architectural Hoisting
This column was published in IEEE Software, The Pragmatic Architect column, July-Aug 2014, Vol 31, number 4.
Abstract: Architectural hoisting is a design technique where the responsibility for an intentional design constraint (that is, a guiderail) is moved away from developer vigilance into code, with the goal of achieving a global property on the system.
Teaching Architecture Metamodel-First
Despite software architecture being recognized as a discipline for about two decades, it is not commonly taught in universities or to practitioners. One potential obstacle is the highly abstract nature of the subject: Less experienced students quickly forget the abstract lessons and experienced students feel that software architecture is disconnected from their daily work.
In this talk, I discuss a novel approach to teaching software architecture based on metamodels. Students start by sketching a simple system. I then discuss typical flaws (e.g., showing a runtime “box” talking to a compile-time “box”) and build their understanding of the metamodel behind the diagram, including views for compile time, runtime, and allocation. I provide a standard palette (i.e., metamodel) for each view, and students learn why the constrained metamodel aids their reasoning.
The overall message is that a diagram is really a model, and models elide details, so it is essential to know what question a model should answer so you can reveal (and elide) the right details. Then the students repeat the same exercise of sketching a simple system, with improved results. I have applied this approach with about 50 software engineers at my current company with good results: students stay engaged and report that the lessons are directly relevant to their daily work.
Architecture Patterns vs. Architectural Styles
People often ask about the difference between a pattern and a style. Michael Keeling and I recorded a discussion about architectural styles (like pipe and filter, client server) compared to architectural patterns. Not to spoil the surprise, but we ended up mostly agreeing that it’s good to think of the style as providing the vocabulary of elements (clients, servers, …) and the patterns as providing conventional arrangements (ie patterns) of those elements.
What Agilists Should Know About Software Architecture
Michael Keeling and I had a video Hangout on Air today, chatting about the intersection of software architecture and agile.
Some of the topics discussed include: 3-tier architecture, YAGNI, quality attributes, big design up front, little design up front, design patterns, architecture patterns, walking skeletons, thin vertical slices, spiral model, system metaphor, extreme programming (XP), application frameworks, MVC, publish-subscribe, refactoring, architectural refactoring, refactoring vs rework, refactoring-to-patterns, lean thinking, last responsible moment, keeping options open as long as possible, Michael Waterman (NZ), cost-value-curve, “I’d love to have that problem”, architecture mismatch, conscious design decisions.
Architecture vs. Design
In this video, I try to tease out the difference between architecture and design. It took a million takes to get the production right and dissuaded me from making more short videos :-)
Video & Lecture on Intro to Software Architecture
This lecture, recorded at the University of Colorado Boulder in September 2012, is an introduction to the major concepts of software architecture. The audience consisted of 70 seniors majoring in computer science. It is based on material taken from the book Just Enough Software Architecture.
Most developers have no formal eduction in software architecture, yet a system’s architecture has a big influence on its success. Software architecture has been intensively researched for more than twenty years now and studying it will help you do a better job of designing systems, give you names for the concepts and patterns, and help you choose between competing designs.
Conceptual Models of Software Architecture
Earlier this year, I gave talks at the GOTO Copenhagen and GOTO Amsterdam conferences. A few folks at the conferences called it “the dandelion talk” because there is an example in the middle of my conceptual model of how to get rid of dandelions in your yard. Of course the real idea is that conceptual models of architecture can help you become a better software engineer – the difference between the 20-year-old version of yourself and the older, master-builder version of yourself. This is perhaps my favorite of all the talks I’ve given, so I hope that you enjoy it too.
Architectural Hoisting - video of Atlanta talk
I presented my work on Architecture Hoisting last week in Atlanta. The big idea is that software often relies on global design constraints (guiderails) to achieve its qualities (e.g., reliability, security).
But you’ve really only got two options for ensuring those guiderails: (1) developer vigilance and (2) architecture hoisting. In the small (say inside of a data structure), using vigilance to keep an invariant is possible. In the large, architectural guiderails (self-imposed constraints that simplify the software) are much harder to ensure, as the recent remote exploit in Google Chrome (a use-after-free bug) illustrates.
The other option, architecture hoisting, has not been widely recognized so this talk gives it a clear definition, shows examples of it in use, and discusses its applicability and trade-offs.
Abstract
Software architecture focuses on quality attribute requirements, such as scalability or performance, that are overall properties of a system. This talk describes Architecture Hoisting, a a design technique where the architecture ensures an intensional design constraint (i.e., a guiderail) to achieve a global property. Discussed examples include the NASA JPL Mission Data System, Enterprise Java Beans, and the Apache Portable Runtime.
Speaking at Atlanta IASA, Weds 14th, 2012
I will be in Atlanta giving a talk on Architecture Hoisting next week at the local IASA chapter. The topic of architecture hoisting is in my book but I’ve had several requests to dig into it further and this talk is the result. Here’s the talk abstract:
Solve Problems by Hoisting Them into your Architecture
Architecture hoisting is an established technique that has only recently been identified and named. It is suitable when you have a challenging design problem that requires consistency in the implementation. Architecture hoisting assigns responsibility for a problem to the architecture itself with the intent of guaranteeing a goal or property of the system. Once a goal or property has been hoisted into the architecture, developers should not need to write any additional code to achieve it.
Book on sale: Now just $19.50 with free shipping (limited time)
A little while ago, several cases of my books were being shipped to Amazon by the US postal service but those cases went missing. The USPS claims to have never found them despite being well labeled and indeed having an Amazon packing list inside the box.
Fast forward to now and those books are showing up on Amazon and eBay. But all is not well! It seems they were sold by the USPS to scavengers who have put them up for sale. I’d thought that this would be no problem, but it appears that Amazon does not like it when others sell books below their wholesale price, so they’re penalizing my book and selling it at list price – but nobody pays retail prices for books.
So, rather than let the scoundrels profit, I’m letting readers profit. I’m selling the book myself as an Amazon Seller for $19.50 until the scavenged inventory makes its way through the system. So grab yourself an extra one and tell your friends since I’m not sure how long this will last.
–George
More book citations: Muddy architecture
Franco Martinig has another blog post about my book, this time on Muddy architecture.
New review of my book
Franco Martinig of Methods and Tools has reviewed my book. Review of Just Enough Software Architecture.
Talk on expressing architecture in code: AgileRoots 2010
This talk that I delivered at AgileRoots 2010 was recently posted (or at least I recently learned it was posted). It’s one of my favorite talks to deliver and the topic is important for all developers since writing code that expresses your design intent is hard but so very helpful for everyone who must read it later (including you!).
Abstract
Because of Eric Evans’ Domain Driven Design, agile developers are familiar with embedding their domain models in their code, but architecture and design remain hard to see from the code. How can we improve that? This session presents a new agile technical practice, an architecturally-evident coding style, that lets you drop hints to code readers so that they can correctly infer the design and architecture. It builds upon ideas like Kent Beck’s Intention Revealing Method Name pattern and provides a set of lightweight coding patterns and idioms that let you express your design intent in the code.
CompArch/WICSA 2011 - Panel discussion and Haiku tutorial
Last week was an exciting one as the CompArch/WICSA conference came to my hometown of Boulder. I was one of the conference chairs, the one in charge of the unglamorous task of organization, but the good news is that everyone was delighted by that! We had a great mix of folks from two different backgrounds and there was lots of mingling and ideas shared between them. The reception dinner at the Red Lion Inn on the Boulder Creek (swollen with snow melt runoff) was really great. (And yes, that restaurant choice was Judy Stafford’s idea!)
I presented a tutorial on the idea of an “Architecture Haiku”, which is a super-short architectural description suitable for use in Agile projects. Architecture Haiku tutorial slides PDF. Here’s the description:
Abstract
You can’t say anything in one page — or can you? An Architecture Haiku is a one-page, quick-to-build, über-terse design description. No project wants “shelfware” documentation, but many must communicate their design to others. Brevity is hard — what would you say in one page? With no page limit, experts suggest documenting everything but the kitchen sink. The one-page limit forces you to convey the essence of your design using compact language and concepts.
Normally, when someone starts talking about architecture, they can go on forever — it seems like almost anything can be considered architectural. Constraining the description to one page (an architecture haiku) forces people to make choices. Consequently, they have to be direct and use the most compact language possible. Another interesting thing is that it clearly isn’t BDUF (Big Design Up Front).
Most folks who are asked to write their architecture on one page have a hard time. The point of the tutorial is to demonstrate that it is hard — hence asking the participants to do it themselves — but also go give a whirlwind tour of the essential ingredients of an architecture description. E.g., modules vs. components, viewtypes (module, runtime, allocation), quality attributes, tradeoffs, drivers, architecture patterns (map-reduce, client-server, n-tier, …). These are elements that aren’t directly expressible in programming languages (i.e., bigger than classes), so most developers won’t have a crisp idea about them.
This tutorial’s direct objectives are to teach the participants how to tersely describe their architecture. This is desirable because few developers want to spend much time on architecture descriptions and one-page descriptions cannot take too long. The indirect objectives are to teach the canonical basics of software architecture including its terminology and standard abstractions. This normally dry subject is made interesting and relevant through the artificial one-page limit. Just as design patterns enabled compact, efficient communication about object- and class-level designs, software architecture concepts and terminology enables efficient communication about larger design ideas.
Panel on Agile and Architecture
I also moderated a panel discussion on Agile and Architecture featuring Philippe Kruchten, Dean Leffingwell, Ryan Martens, and Michael Stal. In a straw poll of the audience, only a couple of people said they were skeptical that Agile and Architecture could work together – the vast majority thought there would be friction but that they could coexist on projects.
Much good news: Second printing, Amazon top-10
Book second printing
As of right now, the book has seven 5-star reviews on Amazon.com. The demand for the book has been so strong that I’ve had to order a second printing! That should arrive within a month. I’m also going to be distributing the book to international bookstores including Amazon in the UK and Germany. There have been quite a few e-book sales there, so now they can get the hardback too.
Amazon Top-10
I just noticed that the book has nudged its way into the top-10 list for its category on Amazon (system analysis and design books). That makes me feel really good and I hope this helps it get better visibility.
Another great Amazon review of my book
There’s a new review of my book up on Amazon today. Here are some choice words:
Just Enough Software Architecture: A Risk-Driven Approach is the book I will now be recommending to software developers who either have no architecture-level experience, or who need to get back to the basics of what software architecture is really all about.
…for a cohesive treatment of software architecture, the goal of which is to start students and professionals on the right track, there are no other texts than can stand up to this one…
Thank you to all of you who have provided such kind words about the book. Yes, people really do read those reviews to decide what book to get!
Interview in InfoQ -- and in Japan
I was interviewed in InfoQ by Srini Penchikala. My hope was to lay out the major themes of the book and build bridges to the Agile programming community. I’ve gotten a few supportive messages so I think people liked the interview overall.
Earlier, at the OOPSLA/SPLASH 2010 conference, I was interviewed by Kenji Hiranabe, whose company makes the best-selling UML tool in Japan. Here is part 1 and part 2 of the video interview.
Amazon pricing craziness
My book is for sale on Amazon but its price has fluctuated wildly. This has happened enough times now that I’m willing to venture a guess. It seems that when the book is out of stock, the price drops to just under $44. When it is in stock, it sells for list price, $69.75.
Article published in CrossTalk magazine
First, apologies for the slow blogging recently. Things have been very busy and I’ve got a pile of topics I want to cover very soon. OOPSLA/SPLASH was great and there is plenty to talk about from that, plus I’ve picked up a few new tricks to explain architecture from a consulting project.
But today the news is my article on the Risk-Driven Model just published in CrossTalk magazine. The issue starts with a Q&A session with Grady Booch – quite nice to have an informal piece to find out what someone is really thinking – then my article, then an article on architecture and agility from the SEI (Nanette Brown, Rod Nord, and Ipek Ozkaya).
The article is derived from Chapter 2 of my book. I’ve attached the article (5.6MB) in case it disappears from their website.
Being helpful on the web by answering questions
Recently, I’ve been posting answers about architecture questions I’ve seen pop up in a few places. I’ll collect them here so that you can find them.
Hardback book for sale on Amazon now, $39.75! Huzzah!
It sure took a long time, but the hardback book of Just Enough Software Architecture: A Risk-Driven Approach is now available on Amazon for $39.75. Makes an excellent present, or buy 4 at a time to hold your newspaper down flat. Great as a flyswatter. Kids and mothers love them!
And if you prefer, the PDF and e-book version is also available for $25. It takes up less space in your carry on luggage. As they say, “you can’t grep a dead tree”.
But the cheapest price on the hardback is to get it from Softpro books in Denver. Just $39.50 and you’re supporting a local bookstore.
Hardbacks arrive at the end of the month
I just got word from the printer that the hardbacks are due to arrive around August 31. That means they’ll be for sale at Amazon maybe a week later.
The printing job is kind of a leap of faith. I did get a proof of the insides and a proof of the cover, separately, but no complete book. And the process for creating the proof is different than for creating the real book, which is printed on an offset press. And what if the cover alignment isn’t right – will the spine text be off-center? Let’s just hope it looks good.
Speaking at Boulder Java User Group, 7 Sept 2010: Architecture Haiku
I’ll be delivering the Architecture Haiku presentation, possibly in an expanded form, at the Boulder Java User Group on 7 Sept 2010. The slides for the talk have been posted (thanks to DOSUG and Matthew McCullogh). They are also available as PDF.
E-book available now
As promised last week, the e-book is now for sale. For $25, you get three DRM-free versions of the book: PDF, ePub, and Mobi. The ePub format works on pretty much every e-book reader – except the Kindle, which reads the Mobi format. Apple iPad owners will be happy to hear that the PDF page size is almost identical to the hardback page size, so the PDF looks great on an iPad. (Kindle owners say the same thing).
Also as promised, the price is pretty darn low for an e-book on software architecture. It beats the pants off of the price of similar books, which sell for twice the price or more.
So go buy the e-book now!! And tell your friends!
E-book available soon! (Preview chapters available now)
While the “real book” is at the printer, I’ve been working on getting the electronic version of my book ready. If you have an e-book reader, please download the book preview as an ePub or as a PDF. Please let me know if you discover any problems and I’ll fix them before the e-book goes on sale.
Speaking at Northern Colorado Architect's Group, Weds 21 July 2010
Just a reminder that I’ll be speaking about Architecture Haikus at the Northern Colorado Architect’s Group in Fort Collins, Weds 21 July 2010. I don’t yet have details on the venue or time, but I’ll edit this post when I get them.
An Architecture Haiku is a one-page, quick-to-build, uber-terse design description. No project wants “shelfware” documentation, but many must communicate their designs. 20 years of architecture research suggests that tradeoffs, quality attribute priorities, architecture styles, and constraints are short yet valuable.
Edit 9 Aug 2010: Slides from Architecture Haiku talk
Speaking at Denver Open Source User Group, Tuesday, 3 August 2010
I will be speaking about Architecture Haiku, a one-page architecture description, at the Denver Open Source User Gro up on Tuesday, 3 August 2010.
An Architecture Haiku is a one-page, quick-to-build, uber-terse design description. No project wants “shelfware” documentation, but many must communicate their designs. 20 years of architecture research suggests that tradeoffs, quality attribute priorities, architecture styles, and constraints are short yet valuable.
Edit 9 Aug 2010: Slides from Architecture Haiku talk
Book copyedits
My apologies if small updates like this are boring, but the book copyedits came in today and they’re all trivial. This is good. They’re things like missing periods, a few misspelled words, missing/extra hyphens (ie usage differences), and so on. I should be able to incorporate these into the manuscript today and have it back to the publisher.
I don’t expect that this will hasten publication because they have to schedule time with the printer. But I’d be happy to be wrong!
And I’ve just noticed the book price is down to $40.46 on Amazon. I wonder what their low price guarantee means for pre-orders. In particular, will Nels (the first order) get the new, lower price?
Edit 11 July 2010: Price is now $39.75, set by me. Note the new Amazon page for Just Enough Software Architecture.
Book price and Boulder architecture group
Let me be clear about something you probably already know — writing computer books is a terrible idea for a way to get rich. For each book that the publisher sells, I get a small percentage of the profit. Yes, profit, not the gross, so I’ll be surprised by how the accounting works. A friend of mine who is a prominent author at the SEI told me that he’s happy to get a nice dinner out of his royalty check each year!
Book news!
I just got back from AgileRoots 2010 in Salt Lake City (more on that soon) and have gotten two bits of good news about my book. First, Lisa Haney has finished the cover illustration! She is also doing the rest of the cover design and that’s not done yet, but I’m really excited just to see the illustration. I went ahead and put some placeholder text on it so that you can get an idea of what it will look like. Here’s the larger version.
The other bit of interesting book news is that there is now an Amazon page for Just Enough Software Architecture. I still have to put up a page for me as an author, but this is a good first step. BTW, the latest news is that the book will be out in October, but I see that Amazon thinks it will be out Sept 21.
Edit 11 July 2010: Revised Amazon link
Speaking at AgileRoots 2010
Somehow I neglected to blog about speaking at AgileRoots 2010 in Salt Lake City this coming week. The talk is on an architecturally evident coding style, an idea invented by David Garlan. It follows from Kent Beck’s Smalltalk Best Practice Patterns, which describe how you should express design intent in your code, even when it makes the code slightly more verbose, because that code will be read by other developers.
WICSA / CompArch 2011 in Boulder
Well, look what I’ve done. Judy Stafford is one of the co-General Chairs for the WICSA/CompArch 2011 conference and has asked me to be the Local Arrangements Chair. That means that if you have preferences, you should let me know and will see what I can do. Want an event at Chautauqua? Hiking? Let me know.
Advance praise for my book Just Enough Software Architecture, Parts V-MCMLXVIII
I was posting the kind words that reviewers had for my book, but then I got overloaded as they came in very quickly and I was working to get the book final draft out to the publisher. So my apologies to those that didn’t get their own blog post. I have posted the complete list of review quotes, and will be updating it as more come in.
Offline, I have gotten a few additional comments that make me really happy:
Page 79 is awesome. Pure gold.
With luck, other pages are at least bronze. And another:
I really like the book., it reminds me of many things we learned in the MSE program. And it also gave me an idea how I could apply that in a “real world” project. This is definitely a book which I will recommend to my friends!
I’ve been working on the book WAY too long to have anything like an objective opinion on it, so it feels really good to hear that it’s helpful for others.
Agile software architecture
Philippe Kruchten’s keynote talk from the SATURN conference has been posted at the SEI. His slides are worth taking a look at: moderate in the agile architecture debate but full of real ideas, not just best hopes of compromise.
Thanks to my friend Alan Birchenough explaining the RUP to me, I was able to see how this presentation hinted at Kruchten’s ideas about risk and architecture (p 17 of the slides). In the RUP, a software project has four phases – not because you impose them, but because they almost always end up that way. The first phase is inception, where you are just trying to understand what the project is about. The second phase is elaboration, where you are deciding how to build it. The third phase is construction, where each iteration focuses on delivering new features. The fourth phase is transition, where the software is introduced into the end-users.
An Architectural Perspective on Software Design
What is architecture design?
For technophiles like me, it was amazing. A friend was playing music at his party via iTunes – nothing unusual there – but he was controlling it from his sofa using his phone. Lots of folks play music at parties from their laptop, but he had an application that showed him the songs in his library and let him queue up songs to play. Since I had a smartphone too, and technical pride was on the line, I decided to find or build a similar application.
On the internet, I found an app for my phone. It was visually attractive and had many features even though it was an early release. However, when I ran it, I found some problems. Initially, I the app would not talk to my media center and it was hard for me to diagnose why. And navigating through the pages of the app was slow – even backing up a page caused a noticeable lag.
After looking at the code, I realized that these problems could be fixed by rewriting the communication library it was using. To solve the problems, the library needed to cache results and report more details about connection errors. In a minor way, the application had an unsuitable architecture.
Whitepaper on Software Architecture Education
Many organizations struggle to choose the most appropriate courses to advance their people. They struggle because, while it is relatively easy to describe their desired end state, the path to get to it is unclear.
Software architecture education is particularly tricky. If you take a class on Java or Struts, you do so because you do not yet know that language or framework. But everyone has some ability to design programs already, so it is not clear what benefit will come from a general class on architecture and design. If you are considering an architecture class, you suspect that your team would benefit from software architecture and design education, but are having a hard time putting your finger on exactly what is missing. As a consequence, it’s hard to be sure if any particular course is what the team needs.
This whitepaper describes a structured approach to thinking about your architecture and design needs. This approach, in brief, consists of the following activities:
- Grouping people into personas
- Identifying problems and connect them to personas
- Connecting education offerings to problems
- Evaluating or developing courses
At the end of this, you should have a better idea of your organization’s needs and therefore be in a good position to decide what kind of education options will be most helpful.
This whitepaper makes three contributions. First, the overall structured approach that aids decision making. Second, the use of personas instead of the obvious choice, job titles/roles. And third, the sets of prototypical personas, problems, and education offerings.
Full PDF: Whitepaper on Software Architecture Education
The Kitchen Drawer Problem
I’ve just had a rather enjoyable “refactoring session” with my book. At first, I had some good content sections lumped together into a catchall chapter — Working with Models. But, as the book fills out, it becomes easier to see where those sections should fit, so I was able to refactor the outline to introduce a new chapter on Encapsulation and Partitioning that will hold several of those sections.
Book chapter: A Conceptual Model of Software Architecture
When a coach and a rookie watch the same game, they both see the same things happening on the field, but despite the rookie’s eyes being younger and sharper, the coach is better at understanding and evaluating the action. As a software developer, you would like to understand and evaluate software as effectively as the coach understands the game. This chapter will help you build up a mental representation of how software architecture works so that when you see software you will understand it better and will design it better.
If you want to become as effective as a coach, you could simply work on software and wait until you are old. Eventually, all software developers learn something about architecture even if they sneak up on it indirectly. It just takes practice, practice, practice at building systems. Consider another path, one where you see farther by standing on the shoulders of others. Perhaps we are still waiting for the Isaac Newton of software engineering, but there is plenty to learn from those who have built software before us. Not only have they given us tangible things like compilers and databases, they have given us a set of abstractions for thinking about programs. Some of these abstractions have been built into our programming languages — functions, classes, modules, etc. Others likely will be, such as components, ports, and connectors.
Book chapter: The Risk-Driven Model of Software Architecture
As they build successful software, software developers are choosing from alternate designs, discarding those that are doomed to fail, and preferring options with low risk of failure. When the risks are low it is easy to plow ahead without much thought, but, invariably, challenging design problems emerge and developers must grapple with high-risk designs, ones they are not sure will work.
To address failure risks, the earliest software developers invented design techniques that helped them build successful software, such as domain modeling, security analyses, and encapsulation. Today, developers can choose from a huge number of design techniques. From this abundance, a hard question arises: Which design and architecture techniques should developers use?
If there were no deadlines then the answer would be easy: use all the techniques. But that is impractical because a hallmark of engineering is the efficient use of resources, including time. One of the risks developers face is that they waste too much time designing. So a related question arises: How much design and architecture should developers do?
Book chapter: Introduction to Software Architecture
Decade after decade, software systems have seen orders-of-magnitude increases in their size and complexity. This is remarkable — and more than a little scary for those of us who build software. In contrast, imagine how hard basketball would be if it scaled up the same way, with 5 people on the floor one decade, then 50, then 500. Because of this growth, today’s software systems are arguably the largest and most complex things ever built.
Software developers are always battling the ever-stronger foes of complexity and scale, but even though their opponent grows in strength, developers have staved off defeat and even reveled in victory. How have they done this?
One answer is that the increases in software size and complexity have been matched by advances in software engineering. Assembly language programming gave way to higher-level languages and structured programming. Procedures have, in many domains, given way to objects. And software reuse, which used to mean just subroutines, is now also done with extensive libraries and frameworks.
Milestone: Manuscript delivered to publisher
Today I sent the electronic book sources to the publisher. What follows is several rounds of copyediting to make sure that you, gentle readers, don’t have to suffer through typos, etc. Sadly, it will take 5-6 months in the production process, so it will arrive on bookshelves (whatever those are) and on Amazon in November or December 2010.
Thank you to everyone who has helped me along the way with supportive comments, reviews, and all kinds of other help!
Advance praise for my book Just Enough Software Architecture, Part IV
I’m honored to have received the following endorsement from Michael Keeling, a blogger on software engineering. I met Michael at the last OOPSLA conference while attending the workshop on Agile and Software Architecture. He is a graduate of CMU’s Master of Software Engineering program.
Just Enough Software Architecture says what everyone is thinking but can’t say because they are tied to various methodologies: Big Design Up Front is bad, you don’t have to document everything, and everyone from the lowest programmer to the project manager needs to understand not just what the architecture of the system is but also why that architecture was chosen. The risk-driven approach to design presented in this book helps software developers understand how much design is needed so they can quickly get back to writing code and providing value to customers in the form of working software.
If you’re only going to read one book on software architecture, start with this one.
Just Enough Software Architecture covers the essential concepts of software architecture everyone – programmers, developers, testers, architects, and managers – needs to know; and it provides pragmatic advice that can be put into practice within hours of reading.
—Michael Keeling, Software Engineer at Net Health Systems Inc.
Advance praise for my book Just Enough Software Architecture, Part III
I’m honored to have received the following endorsement from a client who works at at NASA’s Johnson Space Center.
The Risk-Driven Model approach described in George Fairbanks’ Just Enough Software Architecture has been applied to the eXtensible Information Modeler (XIM) project here at the NASA Johnson Space Center (JSC) with much success.
George’s approach illustrated to the customer why the system was designed the way it was from its service layer, data-access layer, dependency injection support, and even its data architecture. The high-high quality attribute pairs that fell out of the architecture process justified the tasks, schedule, and cost of the project. The modeling constructs have helped communicate the architecture among all team members and even to prospective new customers. The architecture has allowed the XIM project to be a success here at JSC and it can help your project be a success as well.
It is a must for all members of the project from project management to individual developers. In fact, it is a must for every developer’s tool belt. (The Code Model section and the anti-patterns alone are worth the cost of the book!). Using “Just Enough Software Architecture” and involving the whole project team in the approach will go a long way towards project success.
—Christopher Dean, Chief Architect, XIM, Engineering Science Contract Group - NASA Johnson Space Center
Advance praise for my book Just Enough Software Architecture, Part II
I’m honored to have received the following endorsement from a friend who works at Microsoft. I previously worked with Nicholas at Carnegie Mellon in David Garlan’s software architecture research group.
Just Enough Software Architecture will coach you in the strategic and tactical application of the tools and strategies of software architecture to your software projects. The first contribution of this book is a insightful guide that will help you identify the driving characteristics of your project and develop goals, value propositions, and implementation strategies for the role of architectural modeling in your project. Dr. Fairbanks goes on to provide clear instruction in the tactics and the tools of architectural modeling that will enable you to achieve your goals for the overall project. Whether you are a developer or an architect, this book is a solid foundation and reference for your architectural endeavors.
— Nicholas Sherman, Program Manager, Microsoft
Advance praise for my book Just Enough Software Architecture, Part I
I’m honored to have received the following endorsement from a top software architecture expert from the Software Engineering Institute. Paulo has been reading drafts of my book, Just Enough Software Architecture, over the past few months and provided essential insights that have improved the book greatly.
This book reflects the author’s rare mix of deep knowledge of software architecture concepts and extensive industry experience as a developer. If you’re an architect, you will want the developers in your organization to read this book. If you’re a developer, do read it. The book is about architecture in real (not ideal) software projects. It describes a context that you recognize and then it shows you how to improve your design practice in that context.
— Paulo Merson, Visiting Scientist at the Software Engineering Institute, and practicing software architect
.Net is a BIG framework
My friend from CMU (now at Microsoft), Jeff Stylos, mentioned in a presentation that the .Net framework was so big that its documentation was bigger than, well, I’ll let him tell you (excerpted from an email with permission):
There are more than 13,000 classes and structures and hundreds of thousands of methods in the .NET Framework. The documentation is three DVDs worth (~10 gigs) of compressed, textual documentation, compared to say the Pascal reference (~200 printed pages) or the complete works of William Shakespeare (2 megs compressed, 5 megs uncompressed).
So yes, to paraphrase Eddie Murphy, that stuff is big.
Presumptive architectures
People used to say that nobody ever got fired for buying IBM. IBM mainframe systems dominated the market and the assumption was that choosing an IBM system was reasonable. Today, many domains have a software architecture that dominates the same way that IBM mainframes once did. These are presumptive architectures.
A presumptive architecture is a software architecture (or a family of architectures) that is dominant in a particular domain. Rather than justifying their choice to use it, developers in that domain may have to justify a choice that differs from the presumptive architecture. Incurious developers may not even seriously consider other architectures or have the misapprehension that all software should conform to the presumptive architecture.
Philippe Kruchten interview: Telecom-architecture connection & level of architecture abstraction
InfoQ has posted an interview with Philippe Kruchten from SDC. In it, he talks about agile, architecture, and process among other topics. Here are some bits I found interesting with some commentary.
SATURN conference registration: Early bird ends tomorrow
I’m speaking at the SATURN conference in Minneapolis next month. If you are thinking of going, now is the time to register so that you can get the $650 registration (instead of $950).
Interesting software architecture links 15 April 2010
Once I am done with this book, I have good intentions of creating a Slashdot-like site devoted to software architecture. I’m frustrated with how hard it is to find interesting material and I hope others would be willing to help me feed the site. Until then, here are some interesting links I’ve come across:
- An overview of cloud computing. Also, a Collection of links on cloud data storage.
- Towards physical laws for software architecture. Statistical analysis of open source software to investigate modularity rules.
- IEEE on Agility and Architecture. Several articles discussing the intersection of two popular topics.
- Another interesting architecture description from HighScalability.com, this one on the Poppen.de software architecture
- 10 Papers Every Software Architect Should Read Twice. Interesting list, though I should put out my own. What would be on your list?
Comments and suggested links are welcome.
Edit 30 May 2010: The new software architecture site is live.
The joy of pro bono consulting
An old friend called me up last week to have lunch and pick my brain on software architecture. We’ve worked together building software in the past but he’s been tempted by the dark side and is now the CEO of a startup company. He needed his engineers to build an architecture model of their running system ASAP. Fortunately, I had some materials ready for him to glance over.
Who would like to review a book chapter?
I’ve been quiet on the blogging front but that is because I’m in the final push on the book, which should be going to the publisher in about a month. If any of you would like to review a chapter, please let me know and I’ll send it off to you. At this point I’m not looking for wordsmithing/typo feedback, but things like missing references to related work or arguments that need to be strengthened. There are already a couple chapters posted for you to get a taste of what the final book will be like.
WICSA 2011 will be in Boulder!?
Why am I the last one to learn these things? :-) This from the IFIP page:
WICSA 2011 will be held June 2011 in Boulder, CO co-located with CompArch.
Even if they’re not here because of me, I’m happy.
Speaking at Denver (ITARC) Conference, 6 May 2010, Architecture Haiku
I will be speaking on 6 May 2010 at the Denver IASA IT Architecture (ITARC) Conference. And yes, I have the coveted 5:30-6:30pm slot :-) The talk will be about Architecture Haiku: One-Page Architecture Descriptions.
You can’t say anything in one page — or can you? An Architecture Haiku is a one-page, quick-to-build, uber-terse design description. No agile project wants “shelfware” documentation, but many must communicate their design to others. Brevity is hard — what would you say in one page?
Video: Intro to Software Architecture and the Risk-Centric Model
In case you were wondering what my book on software architecture will be like, you can now see the video that describes its basics. This talk was given at a joint session of the Denver IASA and the Agile Denver groups on 16 November 2009. The topic is the Risk-Centric Model of software architecture, which helps you answer the question, “How much architecture/design should I do?”
The first 17 minutes of the talk are a quick summary of software architecture; the remainder describes the Risk-Centric Model.
Architecture refactoring
Here is a short excerpt from my forthcoming book on software architecture:
Over time, a team understands increasingly well how a system should be designed. This is true regardless of which kind of process a team follows (e.g., waterfall or iterative processes). In the beginning, they know and understand less. After some work (design, prototyping, iterations, etc.) they have better grounded opinions on suitable designs.
Once they recognize that their code does not represent the best design (e.g., by detecting code smells), they have two choices. One is to ignore the divergence, which yields technical debt. If allowed to accumulate, the system will become a Big Ball of Mud. The other is to refactor the code, which keeps it maintainable. This second option is well described by Brian Foote and William Opdyke in their patterns on software lifecycle.
Refactoring, by definition, means re-design and its scale can vary. Sometimes the refactorings involve just a handful of objects or some localized code. But other times it involves more sweeping architectural changes, called architecture refactoring. Since little published guidance exists, architectural refactoring is generally performed ad hoc.
Dilbert on quality attributes, and what you should know about quality attribute scenarios
Yes, embedding a Dilbert cartoon is just too easy – but it provides a memorable lesson about why quality attributes, by themselves, are not as good as quality attribute scenarios. Before I bore you, the cartoon:
Perhaps it is important for your system to have security. We could all guess what that means, but we’d all likely guess slightly different things. I might guess it means that bad guys could not retrieve info. You might guess it means bad guys could not alter info. And someone else might point out that there is no currently known way to allow good guys to retrieve and alter info without a dedicated bad guy having some chance of access.
Talk about Design Fragments at CU Boulder
I will be speaking about Design Fragments and software frameworks at the University of Colorado (Boulder) computer science colloquium on 4 March 2010 at 3:30pm.
Frameworks are a valuable way to share designs and implementations on a large scale. Client programmers, however, have difficulty using frameworks. They find it difficult to understand non-local client-framework interactions, design solutions when they do not own the architectural skeleton, gain confidence that they have engaged with the framework correctly, represent their successful engagement with the framework in a way that can be shared with others, ensure their design intent is expressed in their source code, and connect with external files.
A design fragment is a specification of how a client program can use framework resources to accomplish a goal. From the framework, it identifies the minimal set of classes, interfaces, and methods that should be employed. For the client program, it specifies the client-framework interactions that must be implemented. The structure of the client program is specified as roles, where the roles can be filled by an actual client s classes, fields, and methods. A design fragment exists separately from client programs, and can be bound to the client program via annotations in their source code. These annotations express design intent; specifically, that it is the intention of the client programs to interact with the framework as specified by the design fragment.
The thesis of this dissertation is: We can provide pragmatic help for programmers to use frameworks by providing a form of specification, called a design fragment, to describe how a client program can correctly employ a framework and by providing tools to assure conformance between the client program and the design fragments.
We built tools into an IDE to demonstrate how design fragments could alleviate the difficulties experienced by client programmers. We performed two case studies on commercial Java frameworks, using demo client programs from the framework authors, and client programs we found on the internet. The first case study, on the Applet framework, yielded a complete catalog of twelve design fragments based on our analysis of fifty-six Applets. The second case study, on the larger Eclipse framework, yielded a partial catalog of fourteen design fragments based on our analysis of more than fifty client programs.
This work provides three primary contributions to software engineering. First, it provides a new technique to help programmers use frameworks. Second, it provides a systematic way to increase code quality. Design fragments provide a means to communicate known-good designs to programmers, and, unlike simple copying of examples, a means of influencing the uses of that design so that revisions can be propagated. Third, it provides an empirically-based understanding of how clients use frameworks, which aids researchers in choosing research directions and aids framework authors in delivery of new frameworks.
Updated book chapters available
I’ve posted a couple updated book chapters here. The biggest change is a reorganization of the book. The first part now deals with risk-centric architecture and does not delve into specific ways of modeling architecture, reserving that for the second part of the book.
The two revised chapters are:
- Software Architecture & Modeling
- Risk-Driven Model
If you are new to software architecture, these provide a good introduction to what it is and how you can do “just enough” architecture (rather than wasting lots of time or painting your project into a corner).
Speaking at SATURN conference, 21 May 2010, in Minneapolis
I am giving a tutorial at the SATURN software architecture conference in Minneapolis in May. The tutorial is on (you guessed it) risk-centric architecture. Here is the abstract:
Architectural design is expensive, so how can you do “just enough” architecture? Today, you must rely upon your intuition. Smaller, simpler projects often thrive with agile processes and evolutionary design. Bigger, complex projects benefit from architectural planning. Between the two is a huge hole —- but little guidance on how to choose “just enough” architecture. This tutorial provides guidance via the Risk-Centric Model, inspired by Attribute Driven Design, Global Analysis, and the Spiral model.
Risks are central in the Risk-Centric Model, so developers:
- prioritize the risks they face,
- choose appropriate architecture techniques to mitigate those risks, and
- re-evaluate remaining risks.
It encourages “just enough” architecture by guiding developers to a prioritized subset of architecture activities. The mappings between risks and corresponding architecture techniques are explicit. Since it is not a full software development process, it can be used inside of a process such as XP or RUP.
Speaking at Boulder Java User Group, 9 February 2010
I will be presenting a lecture on Design Fragments at the Boulder Java User Group meeting on 9 February 2010 at 6pm (before the pizza break). The lecture will be based on my thesis work. You can find a preview slide presentation on the work here. More details, including the full thesis, are available here.
You can watch a video of this talk, given at CU Boulder.
Rebecca Wirfs-Brock on Agile Design and Architecture
InfoQ has a nice interview with Rebecca Wirfs-Brock on agile and architecture. From the interview, it is clear she is not a 100% emergent-design proponent, yet she has helpful advice on how design/architecture can fit into agile processes.
Here are some architecture-related quotes of interest:
“My words of advice about people who want to think about emergent design is that it is not a replacement for design thinking at the most responsible time, so there is still places at it that you might need to take things and put them aside and think about it and try alternatives.”
“Well some people might say that if you are, for example, doing test first development that design happens between the keystrokes as you are planning the next test to write. That’s where it happens, right there. But I take a little bit different view in that I think that there are certain rhythms to design, when I am doing something new I may want to do some design thinking, hit the pause button on writing the test and thinking about structuring stuff that could have me explore some alternatives before I turn to the code. That code is the easy part, so I think there is a role for design and architecture and there are different rhythms for when you do it on Agile projects. ”
“… a very simple technique of forming a team is sitting down and after they have been presented [a few hundred story cards], instead of just going into the planning game, to sit the developers of the team write where they think are the easy bits the design challenges, write for ten minutes and then share their stories amongst each other…”
“On projects that have significant amounts of architecture decisions in an Agile world it’s not always that architecture emerges one story card at a time. I think that the most responsible moment might be taking aside and figuring out some of the hard bits just like the user experience folks these days, user experience designers, are recognizing that there is a cycle of discovery and confirmation and then there is this more regular cycle of doing and filling out of the details once you figured it out. Some of those things need to be done in a side band from the regular beat of all the rhythms of an iterative sprint if you will.”
Nice Dilbert cartoon about training
Speaking at IASA Denver, Monday, November 16th
I will be speaking about Risk-Centric Architecture at the joint IASA Denver / Agile Denver meeting on Monday Nov 16th, at 5pm. Location: PPA Event Center (2105 Decatur St Denver, CO 80211-5125).
http://www.iasahome.org/web/denver/home
Here are the slides on risk centric software architecture.
What is a framework?
Kevin Bierhoff, Ciera Jaspan, Jonathan Aldrich, and I submitted a paper to ICSE that tries to describe what frameworks are today, since the academic literature on them is becoming a bit dated. The paper draft is here. The abstract is:
Software frameworks have changed significantly since they were described by researchers more than a decade ago. They have entered mainstream use in most domains of software development, and their structure and interaction mechanisms have evolved. This paper provides a revised definition of frameworks and surveys commonly used frameworks to extract two categorizations: a categorization of the different kinds of frameworks found in the wild and a summary of common interaction mechanisms employed by frameworks. Four popular frameworks are described in detail to illustrate these findings. The paper also discusses unique framework challenges and avenues for future research on frameworks.
Definition of software architecture
I believe there is a difference between architecture and detailed design, but it’s hard to make a clean distinction. The problem is that some small details are architectural while others are not. Unfortunately, this has led some authors to say that architects get to decide what is architecture and what is not, which is unsatisfactory. I am in the process of developing my own definition which explains more. Here’s my best shot:
Risk-centric model of software architecture
It’s been a while since I posted a blog entry because I’ve been head-down working on the book. Here’s a position paper for an OOPSLA workshop that I will be attending. The abstract is:
Developers are faced with a smorgasbord of architecture activities but few models telling them which activities to use. Alternatives include the documentation package model, which advocates a complete architectural description from many perspectives, and the evolutionary design model, which advocates no up-front architectural work. This paper introduces the risk-centric model, inspired by Attribute Driven Design (ADD) and the Spiral model. Risks are central, so developers: (1) prioritize the risks they face, (2) choose appropriate architecture techniques to mitigate those risks, and (3) re-evaluate remaining risks. It encourages “just enough” architecture by guiding developers to a prioritized subset of architecture activities. Like ADD, and unlike the Spiral model, the risk-centric model is not a full software development process and can instead be used inside a process such as XP or RUP.
New boogeyman: Documentation packages
In general I think the work done by the SEI is valuable and an asset to our field. My main concern is that the nature of projects they tend to be involved with is different than the ones I see. My guess is that they help out on the development of more tanks, electrical grids, and avionics systems than I do. I tend to see business systems (IT, banking) and web applications. The good news is that despite these differences I think we are describing different parts of the elephant rather than describing different animals.
When I started my book I had a boogeyman in mind: the chief architect.
Quality Attribute Workshops: Guidance on how long they take
I searched around and could not find information on how long a Quality Attribute Workshop (QAW) should take. My guess was between 1 and 3 days, depending on the project size, state of the design, and number of stakeholders.
The risk-centric approach I propose isn’t exactly what the Software Engineering Institute proposes. They tend to be more bureaucratic and methodical, which makes sense if you’re developing projects like the electric grid, a fighter jet, or even just software that controls the brakes in your car. The QAW, however, is one of their lighter-weight techniques, especially compared to things like the ATAM. My work assumes you know what the risks are, so the QAW is a nice compliment in that it describes how to solicit and prioritize quality attributes from a diverse group of stakeholders.
I asked Tony Lattanze, one of the authors of the QAW, for advice on its duration. Here’s his reply:
Selling architecture to agile developers
I’m having a great discussion with Larry Maccherone over on his blog. I’ve included my reply here, which will make more sense if you’ve read his original posting. Excerpts from his posting are in the block quotes below.
Hi Larry,
Thanks for chatting with me about this. It’s helping me to understand and articulate my ideas, and it’s stretching my thinking.
It struck me when you said this:
Agilest will argue that feedback from actual running code is the best way to battle complexity and scale. Tighten the feedback loop with actual code. You can make more rapid progress by building something, even the wrong thing, and re-building it, than you can by wasting time modeling it and thinking about it because the assumptions you make when thinking about it always miss something important. You’ll call that something an insignificant detail. They’ll say, it’s the details that end up getting you.
Engineering activities vs. management activities
This is a letter to my friend Larry Maccherone, who is an expert on software processes and Agile in particular.
Larry,
Here’s something I’d like your opinion on, and it might become another blog post for you. I believe we should try to understand engineering activities separately from management activities. It’s tough to disentangle them, but regardless of management process you’ll need to decide what the system should do, assign responsibilities to parts, and think about (analyze) if indeed the thing you designed/coded will do what you were asked functionally, and if it will have acceptable qualities (performance, modifiability, etc.)
Architectural hoisting (original blog post)
Following architecture-centric design means that you are using the architecture to ensure a property or to achieve a quality attribute. Code can diverge from the design, often unintentionally, and disrupt your plans. We can use architecture hoisting to ensure that the code has those same properties or qualities as the design.
Architectural hoisting is the direct ownership, management, or guarantee by the architecture of a feature, property, or quality attribute. It lets developers depend on the architecture for the hoisted feature, property, or quality. This usually entails writing the application code separate from the architecture code, so the architecture has a tangible existence in the source code.
Object-oriented decomposition -- why and why not
My friend Nels Beckman is writing on how to do functional programming in Java. He asked me why I thought people build systems that follow an OO decomposition. Since my boss has been relentless about getting the book done, I have not posted a blog recently, so here’s my note to Nels.
When you decompose systems, at some point you have a chunk (module, component, …) that you want to structure internally. How do you do that? One of the strategies is to mirror domain concepts. This actually works fine in most IT cases (and probably most others too, once you get used to it). There is an argument that the domain concepts (nouns) change very slowly, while the their behaviors seem to always be evolving (verbs). So if you align your structure with the nouns you get less churn induced by the domain changing. This OO strategy implicitly promotes modifiability.
All hail the mighty connector
Components and connectors are the backbone of the runtime view in software architecture, but the connector appears to play second fiddle to the component. The 1968 NATO conference that coined the term “software architecture” specifically asked for components, but did not mention connectors. Components are shown as two-dimensional boxes, while connectors must make do as one-dimensional lines.
Despite the perception of connectors as simple data movers, real work can be done in connectors. Connectors can convert, transform, or translate datatypes between components. They can adapt protocols and mediate between a collection of components. They can broadcast events, possibly cleaning up duplicate events or prioritizing important ones. Significantly, they can do the work that enables quality attributes, such as encryption, compression, synchronization / replication, and threadsafe communication. It is hard to imagine systems achieving qualities like reliability, durability, latency, and auditability if their connectors are not contributing.
Introductions are hard
You may recall writing your first essays in high school and struggling with introductions. There was probably some advice about going from the general and honing in on your specific topic, and ending up with a topic sentence. Over time this got less painful (both for you the writer and whoever had to suffer through your essays), but the introduction to a book is a whole new world of hurt. You’re trying to hook in and interest folks who come with any number of prejudices about your topic, experts and novices, academics and practitioners. Of course here when I say “you” I really mean “me” because that’s what
I’m struggling with for my book intro. What I’ve posted below is the N-th rewrite of the first few paragraphs of the intro. I’m not happy that it starts out with Agile programming, and may paint them too much as the enemy of design, but here it is for your enjoyment. Comments are, as always, gratefully appreciated.
Architecture abstractions: Prescriptive and descriptive
If you decided to create some abstractions to explain the architecture of existing software, it is unlikely that you would end up with the standard set of architecture abstractions (components, connectors, ports, roles, etc.) The standard set of architecture abstractions seeks to point future software development in the right direction, and so it is prescriptive in that it encourages encapsulation and clear channels of communication.
When we document existing systems, we are simply describing them as they are. Consequently, there are some challenges when we try to simply describe software that does not conform to our ideals. Much of what we find is beautifully designed, but not everything. We will have trouble with systems that have no internal subdivisions and are a bowl of object soup. We will have trouble with “glue code” that expediently interconnects other systems and is often written in terse scripting languages. Above all, we will have trouble with frameworks because while our abstractions encourage encapsulation with components and narrow interfaces defined in ports, frameworks often reveal their implementations through inheritance hierarchies and have wide and poorly defined interfaces.
Note to Kevin and Ciera, if you are reading this: we still need to write that paper on the framework definition language, which would be the first step towards reconciling architecture abstractions and frameworks.
What is (still) hard about software architecture
A while back I posted a list of things that are hard about software architecture, and this posting is an elaboration on that list that will make its way into the software architecture book. Comments on this list are welcome. In fact, if you have a pet peeve about software architecture then it can probably be refactored into an entry in this list.
Information model : Module view :: Snapshots : Runtime view
The Unified Modeling Language has a diagram type for showing classes: a class diagram. A class will have instances, but UML does not have a straightforward diagram type for them. People often use the UML collaboration diagram to show a collection on instances, which is also called a “snapshot” because it shows just an instant in time of a running program.
Separately, software architects have identified three primary viewtypes: Module, Component & Connector, and Allocation. (See Documenting Software Architectures).
Frameworks are Mad Libs
I realized yesterday that a great way to explain how frameworks work, and how they are different from code libraries, is to compare them to a Mad Lib. This metaphor isn’t for computer science types, but for “lay people” like my mom. Until now, I hadn’t come up with an example of something in the real world that acts like a framework, where the skeleton is separate and you fill in parts.
Humans as architecture analysis machines
Once you articulate a vision of how software architecture could be done well, which includes the creation of models that enable analysis, the next question is, “What analyses can I do?” At this point we can either go down the path of somewhat obscure academic efforts such as rate monotonic analysis or queuing theory and pretend that they are appropriate for mainstream use. Alternately, we could admit that there really aren’t many architecture analyses yet. Let me suggest a third path.
Human brains are remarkable at processing information, but quite dependent on the form of that information. Imagine trying to navigate from NY to LA if you were given an alphabetical listing of the street segments in the country. This would thwart a human but impede a computer only slightly. That’s because people have lots of visual processing capabilities that we use to find a route on a map. Maps are drawn to work with those capabilities, for example by drawing major roads thicker than minor roads.
Analysis of software architecture can do something similar. If a human is searching for single points of failure in an architecture, he’s going to have a hard time if he only looks at the source code. However, the fact that all requests flow through a single load balancer would jump out when looking at an allocation view.
The follow-on question is: Do standard architecture views (like allocation, runtime, and module) provide the right representations for humans to leverage their built-in analysis? I do not (yet) have a way to give a refutable answer to this question, but my experience is that Yes, the standard architecture views are good, but No, they are not ideal. The standard architecture views are general-purpose, which gives them leverage on almost all domains. The flip side to this is that they provide no domain-specific leverage. For example, a calendar has the nice property that all the Mondays are in the same column – a domain-specific encoding that you wouldn’t get from the standard architecture views.
Is data modeling (entity-relationship) part of software architecture?
David Garlan recently asked me what I thought about data modeling and its relationship to software architecture. I’m including the answer here because it was effective at articulating some of my philosophical assumptions about software architecture that are not shared by all researchers and practitioners.
- The currently defined field of software architecture has a bias to address quality attributes to the exclusion of functionality. I think this may have originated because researchers wanted to differentiate their work from earlier work on high level design, which primarily addressed functionality.
- It is impossible to meaningfully discuss many architecturally-relevant topics without referencing domain-specific types, e.g., cars, accounts, sensor readings. You need to reference them when defining ports/roles and behaviors. Modeling these types has historically been covered under high-level design, not software architecture.
- In order to effectively zoom-out of implementation complexities (and premature commitments), software architecture should seek out surrogate representations of low-level commitments. Properties in Acme are a good example of this because they do not commit to a data representation.
- In MAp (and CAM, Catalysis, …) we commit to types of information, but not their representations. This allows us to refer to essential types (cars, accounts, sensor readings) while maintaining a zoomed-out view of a large system.
Since data(base) modeling commits to a data representation, it hurts your ability to zoom out. It also introduces discussions of N-th normal forms and efficient use of varchar vs. string(20). What MAp calls “information modeling” commits you to the existence and relationship between types, but not their concrete data representations, which allows it to uncover design defects related to the problem domain.
Data modeling is a large and specific concern at companies, because the data and its schema may span applications and outlive all of them. It may have a place in architecture descriptions simply because its significance to companies, but I am unconvinced that it is an architectural style or at the same level of abstraction as the rest of the software architecture ideas.
Components, connectors, and associations
This was a question from the CMU software architecture class that I think highlights a common misunderstanding.
Question
Is there any real difference between a component and a connector? I tend to take connectors as:
- Interface and protocol between 2 components
- A component whose major purpose is conveying information
For example, in a typical drawing of the architecture of web services, you wouldn’t see the web server. (You would see a web container, but that is something different).
The web server is reduced to a connector between the client and the web-tier components, because we do not care too much at this level from this angle.
Frameworks and linguistics
My friend George Figgs is a linguist and has the following to say about the mad libs metaphor for frameworks:
the madlib metaphor/concept also conjures up Chomsky’s arguments, where you can have a grammatically, i.e. syntactically, correct sentence that doesn’t make any sense semantically, as in the famous “colorless green ideas sleep furiously”, and from Lewis Carroll’s jabberrwocky - “twas brillig and the slithy toves did gyre and gimble in the wabe”.
these sentences are grammatical, as all of the necessary syntactic slots are filled with grammatically appropriate lexical items, but are ‘buggy’ in these sense that they’re filled with semantically infelicitous lexical items.
i don’t know much about frameworks, but this seems similar to unification based grammars, such as construction grammar, where, in order for sentences to ‘work’, they have to unify both lexically (‘type of food’), and semantically (“commonly eaten by humans”), to avoid the utterance being semantic gibberish.
Dimensions of software frameworks
This past weekend I was chatting with Ciera Jaspan about her work on frameworks. We chatted about Bill Scherlis’s observation that framework interfaces are wide, while traditional library interfaces are narrow. As I mentioned briefly in a prior blog posting There is another difference you see in frameworks — some interfaces are deep, others are shallow.
Framework mini-taxonomy and API's
Framework mini-taxonomy
Here’s my mini-taxonomy of object-oriented frameworks that I have not seen elsewhere. I define frameworks as code that uses the inversion of control pattern, where it “calls you back” (the so-called Hollywood Principle).
-
Domain frameworks. These were the original frameworks, from Smalltalk. They gave you a bunch of objects that worked together, effectively showing you how a domain works. You have specific places where you can extend the framework.
-
Quality attribute frameworks. These are newer. They grab a quality attribute and hoist it such that framework users don’t have to worry about that quality attribute. They are generally domain-neutral. Example: While Struts is just a domain framework (handles web requests), EJB is a quality attribute framework that hoists concurrency, distribution, and coupling pretty well, and security quite poorly.
Qualities of API’s
Bill Scherlis likes to point out that framework interfaces are wide and deep, while traditional API’s are narrow and shallow. I’ve been experimenting with refactoring a framework interface so that it is narrow and deep (and yes, I need to blog about that too…), which makes me wonder if other combinations in the {wide,narrow} x {deep,shallow} space are interesting. Perhaps it is interesting to have a wide interface where you only get back simple objects, with no chaining like you get in deep interfaces, for example in Eclipse: “grab the Workspace, then the window, then the menu manager, then the menu, then ask the menu to add an item”.
Short review: Domain Driven Design
Book: Domain Driven Design, Tackling Complexity at the Heart of Software
By: Eric Evans
Publisher: Addison-Wesley, ISBN 0321125215
This is the only book I’ve seen that explains what good object-oriented developers do when they incrementally discover a domain by interviewing subject matter experts and building models. The book uses fictitious dialogues between a developer and a SME, which accurately capture the SME’s initial lack of confidence in the process and the transition into trust and effective collaboration. The book also documents a number of useful patterns.
However, it would have been a better book, a book with a wider audience, if it had refrained from preaching about agile software development. Or even if the book were organized around a series of essays, and just one of the essays was about agile. But instead I found myself arguing with the author when the commentary about integrating modeling with code came up. This commentary didn’t have the requisite “agile hedge” regarding “don’t try this at scale”.
SOA definition, part 2
I got a great email from my friend Alan B. today about my SOA definition blog posting. He too has been concerned about its definition and offers the following to work towards a definition.
- “Functionality is exposed via discoverable and/or published interfaces only.” “Picture a cloud with interface jacks sticking out of it, and that’s about it.”
- “Component boundaries (if any) are completely hidden.” In old CBD you could tell where one component ended and another one began. Indeed to assemble an implementation you had to choose components explicitly and compose them.”
SOA definition, part 1
I’d like to find a definition of Service Oriented Architecture (SOA) as an architectural style. The definitions I’ve seen so far are not clear enough in that they do not exclude enough.
The Wikipedia entry is clear enough about lack of consensus:
There is no widely agreed upon definition of service-oriented architecture other than its literal translation that it is an architecture that relies on service-orientation as its fundamental design principle.
One problem with using this to define an architectural style is that it would include client-server, n-tier systems, and other styles that SOA presumably wants to be distinct from.
When Paulo Marques visited our research group, he provided a reasonable definition. His definition was that the SOA style prohibited certain kinds of connections to components, such as directly accessing its database, or dropping a chunk of data, or screen scraping. I like the trajectory that thinking takes, because it both excludes certain kinds of systems and, like any good style definition, does not refer to implementation decisions (like SOAP messages).
I’ll continue to look for a definition that is helpful to me.
Refactoring the Eclipse Framework API into Features
As you doubtless know, gentle reader, I did my thesis research on software frameworks. Specifically, I found that client code interacted with frameworks quite consistently, and I developed a technique, called design fragments, to specify patterns of client-framework interaction. Client code is still scattered around many classes, but the design fragments make the connections between these classes apparent.
An open hypothesis was that it was possible to refactor the frameworks to avoid the scattering of code in client classes. If such refactoring were possible, client code dealing with a particular feature, say adding a menu item, would be located in a single place. Client code would be easier to read and might not need design fragments. So if a client wanted to create a user interface with a tree view and a menu, the code might look something like this:
What is hard about software architecture
Here’s my initial list of things that are hard about software architecture. Many are open research questions.
- Non-static component configurations. That is, dynamic architectures.
- Bridging objects to components. Each has a different vocabulary and communication idioms.
- Cost-benefit analysis. How much benefit do you get from doing some architecture modeling, and when should you stop?
- Modeling of connectors and ports. When to use an N-way connector?
- When to attach two connectors to the same port?
- Refinement. Code-architecture unity / correspondence / assurance.
- Frameworks.
- Multiple languages (especially scripting languages) in one system.
- Evaluating alternative architectures.
- Choosing which qualities to promote to the architectural level.
- Using architecture models to discover problems in advance, not just document designs after the fact.
- Shoehorning your abstractions into {component, connector, port, role}
- Behavior. More specifically, how to abstractly model architectural behavior without continuing on into detailed behavior descriptions.
- Designs or patterns that span components.
