solving for better small-data systems

Simple Software, a Tutorial

I’m a compulsive note-taker (using OneNote, which I think tops all other products in the category). I like to condense information, in plain english, and codify insights so I can refer to them later when I need a refresher.

Here’s my notes from Rich Hickey’s 2011 talk “Simple Made Easy” (augmented with bits from his 2012 RailsConf talk). This is extraordinary! It’s near the top of my list of software engineering resources (across all formats - talks, books, etc.). If you haven’t seen the talk, go watch it and I promise you will be so much wiser for it.

It was a small bit of effort to find the deck, so I’m hosting it for your convenience. My notes are a synthesis of the talk, the deck, and some of my own material where I felt it aided in understanding.

The Simple

  • A simple piece of software means that there is no interleaving. It doesn’t combine things. It has focus. Something is simple when it addresses: one role, one task, one concept, one dimension
  • Simple doesn’t mean there is only one of them. It's not about an interface with one operation or a class with one instance. It’s not about cardinality.
  • By contrast, a complex system is braided or folded together. There are interleaving roles, tasks, concepts, or dimensions.
  • Identifying a system as simple or complex is thus objective. If there are twists, it's complex. If there is no interleaving, it’s simple.

The Easy

  • Simple is often (disadvantageously) confused with easy. Easy means near:
  • near at hand - ability to start using it locally. Hickey says, “We are infatuated with this aspect of easy. Can I get this instantly and start installing it in 5 seconds. It could be this giant hairball…”
  • my take: It’s almost a requirement these days that a new framework or library boasts a 4-lines-of-code quick-start. The implicit message is “this is so easy, it must be good.” A year later you wonder if you should have explored it some more before taking the dependency and now you’re tangled in the technology.
  • near to our understand or skillset. It’s familiar. It’s like something we already know, plus one new thing. Hickey says, “If you want everything to be familiar you will never learn anything new because it cannot be significantly different from what you already know and not drift away from the familiarity.”
  • near to our capabilities, within our mental capacity, not overly taxing on our brain. Hickey says, “this starts trampling on our egos…due to combination of hubris and insecurity we never talk about something being outside of our capabilities”
  • Easy is relative to a specific person. Easy for whom? Simple is objective.
  • We are too focused on the experience of programming rather than the artifacts - the programs that gets produced and run. We should assess quality based on our artifacts, not on the experience of typing programs into an editor!

Reasoning

  • Humans have limitations:
  • If we can’t understand our software, we can’t make it reliable. As we make things more flexible and dynamic, we tradeoff ability to understand their nature and ensure they work properly.
  • my take: We’ve all been there - you’re traversing a hierarchy of abstract classes and interfaces and the control was certainly inverted and the dependencies were injected and various design patterns were used and you were just trying to figure out how this one thing works and you’re asking yourself “why is this so complicated?”
  • If two things are intertwined, you have the burden of taking both of them into your mind when you are working on one.
  • my take: Again, we’ve all been there - system A depends on system B being in a certain state, which in turn depends on certain features in system C which behave differently based on the configuration file in system A and luck would have it you’ve long forgotten how system C works.
  • Humans are limited in the number of things they can hold in their head.
  • We need to reason about our programs to change them - to add new capabilities and to fix bugs. This is not about proving programs nor category theory, just informal reasoning.
  • “Apparently I heard in a talk today that Agile and Extreme programming have shown that refactoring and tests allow us to make change with zero impact. I never knew that. I still do not know that. That’s not a knowable thing. That’s fooey! If you’re going to change software you need to analyze what it does and make decisions about what it ought to do…if you can’t reason about your program you can’t make these decisions”
  • Beware guardrail programming. The idea that you can make changes because you have tests. “Who drives their car around banging against the guardrail saying woah I’m glad I’ve got these guardrails because I’d never make it to the show on time…do the guardrails help you get to where you want to go?”

Agility

  • Developing easy systems (i.e. using Agile) is faster than developing simple systems at first. Simple systems require deliberation and thought. My take: easy systems tend start at a frantic pace. But if you ignore complexity you will slow down over the long-haul. In contrast, simple systems accelerate progress over time:
  • complex constructs are often easy - they are familiar, available, etc.
  • what matters is the complexity they yield. Easy constructs often yield incidental complexity.
  • my take: You didn’t mean to take on the complexity, but it came for the ride and now what was easy has become a burden.
  • Architectural agility is the agility you get from building a system that is fundamentally simple. This kind of agility dominates all other kinds of agility. Hickey says, “Simplicity enables change…it’s the primary source of real agility. Agility means to do something. It doesn’t mean to do it over, it doesn’t mean to redo it, it doesn’t mean to undo it….It means to directly do. That’s agility. If you’re dragging an elephant around you’re never going to be agile”

Examples

  • Let's make this more concrete (I tried linking to resources that explain why a construct is better or worse, with a fallback to definitions):
complex simple
state, objects values
methods functions, namespaces
vars managed references
inheritance, switch statements, pattern matching polymorphism a la carte
syntax data
imperative loops, fold set functions
actors queues
ORM declarative data manipulation
conditionals rules
inconsistency (eventually consistent) consistency
  • What makes each construct in the table above complex? The deck contains a table with two columns: the constructs and the concepts that each construct intertwines. He discusses each one briefly in the talk. I’m omitting the table. It wasn’t as helpful for me as researching each construct and reasoning about its complexity.
  • How do you leverage the simple constructs in the table above? Links from the first table also apply here:
construct get it via…
values final, persistent collections, immutables
functions stateless methods
namespaces language support
data maps, arrays, sets, JSON, etc.
polymorphism a la carte protocols, type classes
managed refs clojure/haskell refs
set functions libraries
queues libraries
declarative data manipulation SQL/LINQ/Datalog
rules libraries, prolog
consistency transactions, values

Designing Simple Systems

  • Composing simple components is the key to robust software. Composing is about placing things together, not interleaving them. Modules and layers don’t imply simplicity (but are enabled by it).
  • Architecture and design has become demonized. People think it means making grand plans and thus equate it with heavy, failure-prone models such as waterfall. Good design is actually about taking things apart.
  • Abstract means to draw something away, usually from the physical nature of something.
  • It’s not about hiding stuff
  • It’s the job of designing and as such it's hard to explain how to do it properly.
  • One way to create abstractions is to ask: who, what, when, where, why, how? Rich explores each of these but I’m skipping this because it’s partially a repeat of the above and also presented with too broad a brushstroke.
  • Another way is to say “I don’t know, I don’t want to know.”
  • my take: If you’re working on System A and the topic of System B comes up, you say “I don’t know, I don’t want to know.” As soon as you want to know, you risk interleaving the two systems. System A plays a role independent of System B.
  • Is your system simple? You can apply a litmus test: Ask the question, “Can I move my subsystems around without changing much?” Can you move it out of process? To a different language? To a different thread?
  • Information is simple, don’t ruin it. Represent data as data, don’t hide it behind classes and methods. His talk on the “Value of Values” delves deep into this and is chock-full of value =)
  • Simplicity doesn’t fall out of tooling. You have to do the work! You need to gain sensibilities around what is simple and what is complex. Don’t lean on tools such as testing, refactoring, or type systems. They don’t care if your program is simple or not.

Thoughts?

I welcome your reaction and/or a comment (anonymous or not), below!

Subscribe to get new posts via e-mail.
[email protected]
Subscribe