Software Architecture 3 Ways
Abstract: Architecture is a specific abstraction, but it is hard to describe it to others. Most of our explanations refer back to shared experiences – but early career developers don’t yet have those experiences. In this talk, I will describe software architecture three ways, none of which refer to shared experiences.
- By showing a system that was redesigned several times with the same requirements.
- By charting the critical insights that added up to architecture, starting in the 1970s.
- By walking through the design of an example system and knocking down some misunderstandings, such as “the code is the truth” and “there’s one right way to build it”.
If you are new to architecture, I hope you will understand architecture better. If you already know architecture, I hope you will have new ways of explaining your passion to others.
This invited talk was delivered at the 2021 Codecamp Conference - The One With Architecture, 24 February 2021.
Contract-based design
This invited talk was delivered at the University of York in February 2021. Here’s a link to the YouTube playlist of all the lectures.
Contracts: an overview
Contracts will make you a stronger software developer. They change your thinking, actions, and values. As you internalize the contract metaphor, you’ll look at software differently.
You will apply logic to source code, not just animate code in your head. You will write contracts in your code, so both you and your team benefits. You will reward clean code and designs, not short-term hacking.
Contractual thinking applies to any kind of programming language, So you can use it with C, Java, Python, Haskell, or anything else.
It’s a transformation that takes time and effort, but the investment is worth it.
O'Reilly Software Architecture NYC 2020: Intellectual Control
In the early days of software engineering, Edsger Dijkstra warned us not to let the size and complexity of our programs cause us to lose “intellectual control” due to the limited nature of our minds. Dijkstra never defined precisely what intellectual control was. Our software today is staggeringly larger than the programs of the 1960’s, so does that mean we have it under our intellectual control or did we find ways to make progress without Dijkstra’s high standards?
If you found this interesting, you’ll want to try these next:
- Intellectual Control
- Testing Numbs Us to Our Loss of Intellectual Control
- Code is King; Let’s Think in Code
- Continuous Design of IT Systems
This talk was a keynote at the O’Reilly Software Architecture conference NYC, 26 February 2020.
Here are the slides:
GSAS 2019: Code is King; Let's Think in Code
The goal of expressing design intent in code has been around since at least the 1970’s. What’s changed is that now code is king: Developers stare at code all day long, our processes track features, and our success metric is deployed code. Other, formerly first-class activities – like modeling, design docs, and software architecture – now happen largely to support code.
As a result, coding is a different cognitive task. Instead of code being the final expression of developers’ thoughts, it’s now a partner in their thinking. In a real sense, the thoughts they think are made possible only because they are staring at code.
This talk builds a cognitive model of software development in three parts.
- Theory building (from Peter Naur)
- Extended & distributed cognition
- Situational awareness
The implication is that we must go beyond the old advice to “express design intent in code” and actually change our development practices. To succeed like other types of engineering, we must support theory building and expression in teams of software developers.
Invited talk at the GSAS 2019 conference Barcelona, Spain.
SATURN 2019: Continuous Design of IT Systems
I’ve been applying a mixture of agile, Domain Driven Design, functional programming, and architecture ideas to Information Technology (IT) systems. Most architectural case studies report on one round of design but my projects were being designed and redesigned continuously. This talk covers some insights I’ve had and some ways of working that may help you on your projects.
-
Agile and architecture differ in their use of control systems (open loop, closed loop) to guide projects. We can mix them but only when the team agrees to the mixture, eg agilists don’t shout YAGNI and architects don’t stop the presses. We should use open loop control for some things and closed loop for others.
-
The team won’t have a single architect but neither will it have silos of expertise. Instead, the team creates and evolves a shared understanding of the architecture that is communicated verbally and represented in the codebase (including types, framework components, comments, and READMEs).
-
The team maintains intellectual control of the system. They keep the state space small and sufficiently well-understood to reason about it and derive conclusions. The design is continuously changing so they evolve the code to match that understanding and actively communicate relevant changes to the rest of the team.
-
Humans are not ideal computing machines and work better with some external representations (ie source code) than others. To facilitate human reasoning, teams should mercilessly refactor their code so that it matches their internal theories of the requirements and the design. Without aggressive refactoring, complexity quickly accumulates, preventing axiomatic reasoning and intellectual control. Theory building is refactoring.
-
We should expect the code tied to the domain to converge as the team masters the requirements. The architectural infrastructure, however, may never converge. As the team’s theories become clearer, the source code (in the small) will take on a functional, definitional flavor regardless of programming language.
Invited talk at the SATURN 2019 conference.
SATURN 2018: Refactoring to Functional Architecture Patterns
Last year at SATURN I spoke about ideas from the functional programming (FP) community that are relevant to software architecture. This talk is an experience report based on applying those ideas and represents one way to marry functional programming with traditional OO design advice.
We followed a pattern we call the Rich Service layer. It is implemented as pure functions plus a modified version of what Fowler calls an Anemic Domain Model. Historically, the Transaction Script pattern has been contrasted with the Domain Model pattern, where objects have methods expressing the business logic.
Our domain model expresses strictly defined types with as little optional data as possible, not just jumbled data bags. Our domain types are as strictly defined as possible. We check integrity at the system boundary. Compared to object-oriented programming, we don’t push much behavior onto the types, which are immutable. We grew an (internal) Domain Specific Language (DSL) organically so the code reads naturally. But the DSL mainly expresses predicates on state rather than mutations. There is little business logic, which is instead in functions in the rich service layer.
We Are Developers 2017: Model-Minded Development (part 2)
Abstract:
We want to write good code but we don’t know exactly how to do that. This talk suggests one way, which is to focus on expressing your theories (i.e, models) in the code. That has three parts: a model of the domain, a model of the design, and an argument about why this design has the desired effect on the domain.
While this talk can stand alone, its best to see part 1 first.
Delivered at We Are Developers 2017 in Vienna.
SATURN 2017: Functional Programming Invades Architecture
Abstract:
A few decades ago when the first architecture books were being written, today’s large-scale web systems were an oddity, but now they are mainstream. Our architecture pattern languages have changed as have our development, deployment, and operating procedures. During this transition, one source of ideas has been the functional programming (FP) community.
FP itself mostly happens within a module, and many software architects treat it as an implementation choice, unrelated to architecture. But some ideas from the FP community are helpful in architectural design: statelessness, immutability, and pure functions. They underlie DevOps practices and are at the core of many distributed systems patterns in our architectures.
Although functional programming ideas have been around for a long time, conditions are ripe for applying them now. This talk surveys how modern web systems are built with FP patterns, shows how they work, and considers why they are increasingly popular. We will dig into one client-side example and demonstrate how FP ideas can simplify tasks.
Delivered at SATURN 2017
GOTO Berlin 2016: Model-Minded Development
Abstract:
Senior software developers can walk up to a whiteboard and give an impromptu talk on how their software works, explaining both the details and the broad strokes. Most importantly, they connect the specifics of this system to general architecture patterns, domain models, design patterns, development patterns, or programming styles. How do we become “senior software developers”, other than waiting until we are older? The answer is models.
This talk covers Model-Minded Development. It briefly shows how we underestimate the cognitive task of programming and how we consequently use models to amplify our own cognitive abilities and to coordinate our ideas with others. Then we’ll dig into source code and compare equivalent programs that do better and worse jobs of expressing models in the code.
Delivered at GOTO Berlin 2016
GOTO London 2016: Model-Minded Development
Abstract:
Senior software developers can walk up to a whiteboard and give an impromptu talk on how their software works, explaining both the details and the broad strokes. Most importantly, they connect the specifics of this system to general architecture patterns, domain models, design patterns, development patterns, or programming styles. How do we become “senior software developers”, other than waiting until we are older? The answer is models.
This talk covers Model-Minded Development. It briefly shows how we underestimate the cognitive task of programming and how we consequently use models to amplify our own cognitive abilities and to coordinate our ideas with others. Then we’ll dig into source code and compare equivalent programs that do better and worse jobs of expressing models in the code.
Note: I had flown overnight and was jetlagged when delivering this talk. The content is OK but you can tell I’m nursing a cold and had 4 cups of coffee. The GOTO Berlin talk is the same content with better delivery.
Delivered at GOTO London 2016
SATURN 2016: Centralized vs. Decentralized Approaches to SOA: Hamilton vs. Jefferson
Talk from SATURN 2016 with Michael Keeling.
Modern service-oriented architecture (SOA) systems force teams to reconcile a multitude of organizational and technology decisions. With each decision, the team reaffirms allegiance to its chosen message passing, platform governance, and quality assurance philosophy. Which side are you on: Centralized SOA or Decentralized SOA?
During this session, we will explore essential topics in modern SOA including governance, message passing strategies, orchestration, “smart” and “dumb” pipes, quality assurance strategies, deployment, and many other topics. For each topic, we’ll cover the most important information you need to know and debate the pros and cons of a centralized and decentralized approach.
And because it’s Michael and George hosting this session, we can’t just leave it at that. In the spirit of the American Federalists (strong central government) vs. Antifederalists (decentralized government) of the 1790s, George, playing the role of a modern Jefferson, will advocate for decentralized SOA while Michael, as a modern Hamilton, will attempt to convince you, the audience, that centralized SOA is the best path forward. This is a session you will not want to miss!
SATURN 2016: Model-Minded Development
This is a shorter (30 minute) and less theoretical version of the WICSA 2016 keynote talk from earlier this year. I’d recommend watching this first until you decide you are curious for more.
- Domain-Driven Design (DDD) says that we should be mindful of domain models and embed them in our code.
- Design Patterns say that we should know a catalog of patterns so we can solve recurring problems that arise in object-oriented (OO) code.
- Software Architecture says that unless we are mindful of large-scale patterns and models, then our systems will not achieve the qualities we seek.
- Test-Driven Design (TDD) says that we should structure our code so that it can be more easily tested.
- Programming styles (functional, OO, procedural, etc.) say what the core abstractions of our programs should be.
The common denominator here is that software developers are expected to keep in mind many abstract yet complex models that constrain the code they write. In some ways, these constraints are a burden, and in other ways, they are the light that illuminates a path forward.
I will discuss an idea called Model-Minded Development that generalizes across DDD, design patterns, architecture, TDD, and coding styles. The defining characteristic of senior software developers is their facility with Model-Minded Development, and it enables them to operate at an advanced level.
WICSA 2016: Building Theories is Building Value
Abstract:
I find a program much more valuable when I can read it and understand the abstractions that collectively explain a theory of the problem and solution. In contrast, I find programs with minimal abstractions and lots of conditional logic hard to understand because there is little or no theory to be inferred. The object-oriented programming community has long sought correspondence between models of the domain and the classes in a program, but other communities strive for this too.
But the theory goes beyond just domain models, as solution models (design patterns, architecture patterns) and “critical thinking models” (eg design by contract and the functional thinking embodied in Z and VDM) are equally important.
For several years I’ve been trying to relate the many models that I build on software projects and I believe it’s a combination of models of the domain, the solution, and these critical thinking models. This leads to the question of what value a programming team is contributing, especially when “repeated delivery of stakeholder value” is the primary metric used today.
I argue that:
(1) programs that embody the theories of their developers are more valuable,
(2) how well developers can evolve a program is related to their ability to build and evolve theories, and
(3) a key distinguishing characteristic of a company’s most senior developers are their theory-building traits.
Delivered at WICSA/CompArch 2016
Injection, Modularity, and Testing: An Architecturally Interesting Intersection
Dependency injection, code modularity, and testing often seem like staid, even boring, topics but there are surprises when you put all three together. Seemingly independent decisions about each influence the others. Making rational development decisions each day can still lead you into a thicket of complexity and loss of maintainability. So, when should you give it your “architectural attention”? This talk discusses the problem and suggests some limited solutions.
Building Models Quickly and Carefully
This video was done as a PechaKucha talk at SATURN 2015, meaning that the slides were set to auto-advance every 20 seconds and it’s just 20 slides. I have been giving a whiteboard talk to co-workers to explain this technique for years, so it seemed like a fun idea to try to do it in this rigid format. As you can hear in the video, Michael Keeling was skeptical but I think it works pretty well.
Software Architecture Book Recommendations
Also see my older writeup on recommended architecture books.
Teaching Architecture Metamodel-First
Despite software architecture being recognized as a discipline for about two decades, it is not commonly taught in universities or to practitioners. One potential obstacle is the highly abstract nature of the subject: Less experienced students quickly forget the abstract lessons and experienced students feel that software architecture is disconnected from their daily work.
In this talk, I discuss a novel approach to teaching software architecture based on metamodels. Students start by sketching a simple system. I then discuss typical flaws (e.g., showing a runtime “box” talking to a compile-time “box”) and build their understanding of the metamodel behind the diagram, including views for compile time, runtime, and allocation. I provide a standard palette (i.e., metamodel) for each view, and students learn why the constrained metamodel aids their reasoning.
The overall message is that a diagram is really a model, and models elide details, so it is essential to know what question a model should answer so you can reveal (and elide) the right details. Then the students repeat the same exercise of sketching a simple system, with improved results. I have applied this approach with about 50 software engineers at my current company with good results: students stay engaged and report that the lessons are directly relevant to their daily work.
Architecture Patterns vs. Architectural Styles
People often ask about the difference between a pattern and a style. Michael Keeling and I recorded a discussion about architectural styles (like pipe and filter, client server) compared to architectural patterns. Not to spoil the surprise, but we ended up mostly agreeing that it’s good to think of the style as providing the vocabulary of elements (clients, servers, …) and the patterns as providing conventional arrangements (ie patterns) of those elements.
Architecture vs. Design
In this video, I try to tease out the difference between architecture and design. It took a million takes to get the production right and dissuaded me from making more short videos :-)
Video & Lecture on Intro to Software Architecture
This lecture, recorded at the University of Colorado Boulder in September 2012, is an introduction to the major concepts of software architecture. The audience consisted of 70 seniors majoring in computer science. It is based on material taken from the book Just Enough Software Architecture.
Most developers have no formal eduction in software architecture, yet a system’s architecture has a big influence on its success. Software architecture has been intensively researched for more than twenty years now and studying it will help you do a better job of designing systems, give you names for the concepts and patterns, and help you choose between competing designs.
Conceptual Models of Software Architecture
Earlier this year, I gave talks at the GOTO Copenhagen and GOTO Amsterdam conferences. A few folks at the conferences called it “the dandelion talk” because there is an example in the middle of my conceptual model of how to get rid of dandelions in your yard. Of course the real idea is that conceptual models of architecture can help you become a better software engineer – the difference between the 20-year-old version of yourself and the older, master-builder version of yourself. This is perhaps my favorite of all the talks I’ve given, so I hope that you enjoy it too.
Architectural Hoisting - video of Atlanta talk
I presented my work on Architecture Hoisting last week in Atlanta. The big idea is that software often relies on global design constraints (guiderails) to achieve its qualities (e.g., reliability, security).
But you’ve really only got two options for ensuring those guiderails: (1) developer vigilance and (2) architecture hoisting. In the small (say inside of a data structure), using vigilance to keep an invariant is possible. In the large, architectural guiderails (self-imposed constraints that simplify the software) are much harder to ensure, as the recent remote exploit in Google Chrome (a use-after-free bug) illustrates.
The other option, architecture hoisting, has not been widely recognized so this talk gives it a clear definition, shows examples of it in use, and discusses its applicability and trade-offs.
Abstract
Software architecture focuses on quality attribute requirements, such as scalability or performance, that are overall properties of a system. This talk describes Architecture Hoisting, a a design technique where the architecture ensures an intensional design constraint (i.e., a guiderail) to achieve a global property. Discussed examples include the NASA JPL Mission Data System, Enterprise Java Beans, and the Apache Portable Runtime.
Talk on expressing architecture in code: AgileRoots 2010
This talk that I delivered at AgileRoots 2010 was recently posted (or at least I recently learned it was posted). It’s one of my favorite talks to deliver and the topic is important for all developers since writing code that expresses your design intent is hard but so very helpful for everyone who must read it later (including you!).
Abstract
Because of Eric Evans’ Domain Driven Design, agile developers are familiar with embedding their domain models in their code, but architecture and design remain hard to see from the code. How can we improve that? This session presents a new agile technical practice, an architecturally-evident coding style, that lets you drop hints to code readers so that they can correctly infer the design and architecture. It builds upon ideas like Kent Beck’s Intention Revealing Method Name pattern and provides a set of lightweight coding patterns and idioms that let you express your design intent in the code.
Interview in InfoQ -- and in Japan
I was interviewed in InfoQ by Srini Penchikala. My hope was to lay out the major themes of the book and build bridges to the Agile programming community. I’ve gotten a few supportive messages so I think people liked the interview overall.
Earlier, at the OOPSLA/SPLASH 2010 conference, I was interviewed by Kenji Hiranabe, whose company makes the best-selling UML tool in Japan. Here is part 1 and part 2 of the video interview.
Video: Intro to Software Architecture and the Risk-Centric Model
In case you were wondering what my book on software architecture will be like, you can now see the video that describes its basics. This talk was given at a joint session of the Denver IASA and the Agile Denver groups on 16 November 2009. The topic is the Risk-Centric Model of software architecture, which helps you answer the question, “How much architecture/design should I do?”
The first 17 minutes of the talk are a quick summary of software architecture; the remainder describes the Risk-Centric Model.
Talk about Design Fragments at CU Boulder
I will be speaking about Design Fragments and software frameworks at the University of Colorado (Boulder) computer science colloquium on 4 March 2010 at 3:30pm.
Frameworks are a valuable way to share designs and implementations on a large scale. Client programmers, however, have difficulty using frameworks. They find it difficult to understand non-local client-framework interactions, design solutions when they do not own the architectural skeleton, gain confidence that they have engaged with the framework correctly, represent their successful engagement with the framework in a way that can be shared with others, ensure their design intent is expressed in their source code, and connect with external files.
A design fragment is a specification of how a client program can use framework resources to accomplish a goal. From the framework, it identifies the minimal set of classes, interfaces, and methods that should be employed. For the client program, it specifies the client-framework interactions that must be implemented. The structure of the client program is specified as roles, where the roles can be filled by an actual client s classes, fields, and methods. A design fragment exists separately from client programs, and can be bound to the client program via annotations in their source code. These annotations express design intent; specifically, that it is the intention of the client programs to interact with the framework as specified by the design fragment.
The thesis of this dissertation is: We can provide pragmatic help for programmers to use frameworks by providing a form of specification, called a design fragment, to describe how a client program can correctly employ a framework and by providing tools to assure conformance between the client program and the design fragments.
We built tools into an IDE to demonstrate how design fragments could alleviate the difficulties experienced by client programmers. We performed two case studies on commercial Java frameworks, using demo client programs from the framework authors, and client programs we found on the internet. The first case study, on the Applet framework, yielded a complete catalog of twelve design fragments based on our analysis of fifty-six Applets. The second case study, on the larger Eclipse framework, yielded a partial catalog of fourteen design fragments based on our analysis of more than fifty client programs.
This work provides three primary contributions to software engineering. First, it provides a new technique to help programmers use frameworks. Second, it provides a systematic way to increase code quality. Design fragments provide a means to communicate known-good designs to programmers, and, unlike simple copying of examples, a means of influencing the uses of that design so that revisions can be propagated. Third, it provides an empirically-based understanding of how clients use frameworks, which aids researchers in choosing research directions and aids framework authors in delivery of new frameworks.
