Patterns for Sustainable Software Development

# Patterns for Sustainable Software Development

## A (permanent) Work in Progress

### Stefan Magnuson / @styrmis

---

# Overview

- Operational & Social Patterns
- Patterns for Code
  - In Ruby
  - In Rails
- Personal Patterns

---

# Disclaimer

- I'm ~~probably~~ definitely preaching to the choir here
- Nothing in here is a silver bullet
- Everything has its caveats
- Some patterns have been tested with a very small team size only, YMMV

---

# Your Thoughts

## At any point in this talk please feel free to raise your hand and contribute your perspective

---

# My Background (on sustainability)

- Longest tenure on a single (active) code base: 11 years
  - A Django system which serves multiple front and backends
- 6 and 8 years maintaining two iOS apps
- Currently at ~4 years working on a Rails codebase
  - From EOL Ruby 1.9.3 and Rails 3.2 to Ruby 2.5.3 and Rails 5.2.2.1
- Still have a great deal to learn...
- Permanent tension between getting things done *now* vs. maintaining future
  pace
- Patterns herein are just a small selection that I have seen deliver results

---

# Divining what will definitely work is hard

## Thinking about what will absolutely not work is easier

---

# Unsustainable Software Development

## What might it look like?

- Everyone committing to master directly
- Test suite is not run regularly
  - Suite is unbearably slow
  - Tests have accumulated more technical debt than the main codebase
  - Everything is tested in multiple places, and not once at the right level
- Pressure from the business to only add features, and to do so rapidly
- Every request is top priority
- Production console sessions used regularly to debug and fix issues
- Individuals/teams work in silos with a spirit of protectionism/antagonism towards one
  another

---

## What might it look like?

- Code is thrown over the wall to operations
- Rate of hiring is driven significantly by staff turnover due to burnout
- Running EOL Rails on EOL Ruby with a huge Gemfile
- Code runs on snowflake servers
- True integration happens every few months, with guaranteed downtime
- Could go on...

---

# Imagining the Inverse

## What would it look like if we did the opposite of these undesirable scenarios?

---

# Operational & Social Patterns

---

# Use PRs

## If you aren't using PRs at all, I strongly recommend that you use them

### Basic Benefits

- Defined units of work
- Dedicated comment threads including per-line threads on Github
- Easy integration with CI, other code review tools, deployment pipelines
- Convenient unit of work for (occasional) reverts

---

### Some Tips

- Push up from the earliest commits to get early feedback
- Put effort into the PR description to include:
  - What does the PR aim to achieve and why?
  - Background information that would be required to understand the change
- Keep them small and focussed
- If you find an issue to resolve, do so in an upstream PR

---

# Package work/changes in small PRs

- "Obvious", but this has really paid off for us
- Initial tension in that it could:
  - Expose how slow your test suite is
  - Put pressure on the CI queue
  - Put pressure on the deployment process
  - Leave people waiting for reviews on their work
  - Increase the number of times people get branches tangled
- We experienced all of these to some extent
- Improvements made to any of these accumulate nicely

---

## Advantages we accrued from keeping PRs small

- Positive pressure on the test suite runtime
- Advances in our CI setup
- A significantly smoother and safer deployment process
- Easier-to review PRs
  - Avoiding the, "looks fine I guess?" trap of complex PRs
- Everyone is now confident in their git branch handling ;)

---

## Maintain staging as a clone of production

- As far as possible, make the setups identical
  - Differences in setups can cause production issues to be missed
- Ours is identical in all respects except instance size (for cost)
- Rebuild automatically on a daily basis

---

## Introduce smoke tests

- Unit/feature/integration tests can't catch all issues
- In our case, "first do no harm" is our policy
  - No place for "move fast and break things" in the enterprise space
- Run them on staging as part of deployments
- Report smoke tests failure diffs to e.g. Slack
- Have smoke test failures pause the pipeline
  - But make it easy to restart

---

# Infrastructure as Code

- Still early days for us, but results have been positive
- For an easy on-ramp CloudFormation is a good start if on AWS
- Automate as much as possible
  - (AWS) Use CloudWatch to run Lambda functions via a cron-like schedule
  - (AWS) Use a Lex bot with Slack to automate common commands
    - Save developers from signing into AWS
    - Do enable MFA for Slack

---

# Write first and foremost for your team

## And for your future self

- It can be frustrating, but that code you're happy with may be hard for others
  to follow
- Pairing is probably the easiest way to guard against this

---

# Introduce Pair Programming

- Code is reviewed as it is written
- Knowledge is transferred at point of creation
- Intention is made clear
- Mistakes that might otherwise be missed *may* be caught
- But be mindful of energy levels, some find pairing draining/intense

---

# Encourage Natural Work Styles

- Everyone is different
  - What drains me could be easy to you
  - And vice versa
- Can be hard to infer, so best to ask/discuss
- My opinion: work is better when you get to spend more time in flow states
- I find Jung's theory of cognitive functions useful as a guide

---

# Example

- Energising work for me involves:

- Complex problems with no obvious solution

- Slight time pressure

- Freedom to focus

- Energy drains for me include:

- Rote/repetitive detail work

- Writing process/compliance documentation

- Easygoing/recuperative work includes:

- Refactoring code/specs

- Development spikes on ideas

- Pairing, brainstorming (most of the time)

---

# Patterns for Code & System Design

---

# Build the system to <u>use</u> Rails

## Don't build the (whole) system <u>in</u> Rails

- Limit model code to logic related to management of the data
- Keep the domain logic on 'your' side
- Avoid model callbacks (e.g. `before_save`, `after_commit`)
  - Unless it is to update the value of an attribute in the database
- Discuss and agree where domain logic should go

---

# Use immutable types

- Struct, now with keyword arguments
- [Functional Core, Imperative Shell](https://www.destroyallsoftware.com/screencasts/catalog/functional-core-imperative-shell)
  - One of Gary Bernhardt's free screencasts
  - Highly recommend it and the Destroy All Software series in general
- But don't write your own value types
  - Several easy traps to fall into here
  - Avdi has covered this in depth in a Tapas episode
- Makes memoization safer
  - Often a cause of bugs in stateful code

---

# Write Confident Code

- Avoid "Defensive programming all the way down"
- Identify the boundaries and enforce there only
- Code inside the boundaries can focus on the work to be done
- Simple example: use optional arguments with care
- Avdi has [written a book on the topic](https://pragprog.com/book/agcr/confident-ruby)

---

# Design to avoid `nil`

## [Nothing is Something](http://confreaks.tv/videos/bathruby2015-nothing-is-something)

> Our code is full of hidden assumptions, things that seem like nothing, secrets that we did not name and thus cannot see.
>
> These secrets represent **missing concepts** and this talk shows you how to expose these concepts with code that is easy to understand, change and extend.
>
> Being explicit about ideas will make your code simpler, your apps clearer and your life better. Even very small ideas matter.
>
> Everything, even nothing, is something.

— *Sandi Metz, Bath Ruby Conference 2015*

---

# Nothing is Something

- `NoMethodError: undefined method 'capitalize' for nil:NilClass`
  - We've all experienced this
  - Worst (IMO) is `@ivars` in views/partials
  - Codebases can be littered with checks like `if current_user.present?`
  - This happens in system code too
- I'd recommend taking the time to watch the talk because:
  - Coding without conditionals (which Smalltalk omits) is intriguing
  - Surfacing missing/hidden concepts in your system is valuable
  - Eradicating `nil` from your system entirely could be very valuable

---

# Avoiding `nil` using the Special Case pattern

## [Two ways to eradicate Ruby nil values (Avdi Grimm, Upcase)](https://www.rubytapas.com/2017/01/31/two-screencasts-two-ways-eradicate-ruby-nil-values/)

```ruby
def current_user
  if session[:user_id]
    User.find(session[:user_id])
  end
end
```

---

# Conditional placeholder content

```ruby
def greeting
  "Hello, " +
    current_user ? current_user.name : "guest" +
    ", how are you today?"
end
```

---

# Conditional rendering of the UI

```ruby
if current_user
  render_logout_button
else
  render_login_button
end
```

---

# Conditional rendering based on user role

```ruby
if current_user && current_user.has_role?(:admin)
  render_admin_panel
end
```

---

# Conditional filtering of data

```ruby
if current_user
  @listings = current_user.visible_listings
else
  @listings = Listing.publicly_visible
end
```

---

# Conditional modification of the user

```ruby
if current_user
  current_user.last_seen_online = Time.current
end
```

---

# Adding an item to a cart

```ruby
cart = if current_user
         current_user.cart
       else
         SessionCart.new(session)
       end

cart.add_item(some_item, 1)
```

---

# The Problem

- We are representing the case of "no user" with `nil`
- As such we must check repeatedly for the presence of a user object

---

# One Possible Solution

## The Special Case Pattern

- A similar pattern exists by default in Django

```ruby
class GuestUser
  def initialize(session)
    @session = session
  end
end
```

---

```ruby
def current_user
  if session[:user_id]
    User.find(session[:user_id])
  end
end
```

```ruby
def current_user
  if session[:user_id]
    User.find(session[:user_id])
  else
    GuestUser.new(session)
  end
end
```

---

```ruby
class GuestUser
  def initialize(session)
    @session = session
  end

* def name
*   "Guest"
* end
end
```

---

```ruby
def greeting
  "Hello, " +
    current_user ? current_user.name : "guest" +
    ", how are you today?"
end
```

```ruby
def greeting
  "Hello, #{current_user.name}, how are you today?"
end
```

---

```ruby
class User
* def authenticated?
*   # ...
* end

# ...
end

class GuestUser
  def initialize(session)
    @session = session
  end

def name
    "Guest"
  end

* def authenticated?
*   false
* end
end
```

---

```ruby
if current_user
  render_logout_button
else
  render_login_button
end
```

```ruby
*if current_user.authenticated?
  render_logout_button
else
  render_login_button
end
```

- A check for presence is converted into an intention-revealing statement

---

```ruby
if current_user && current_user.has_role?(:admin)
  render_admin_panel
end
```

- We can again add a negative implementation to `GuestUser`

---

```ruby
class GuestUser
  def initialize(session)
    @session = session
  end

def name
    "Guest"
  end

def authenticated?
    false
  end

* def has_role?(role)
*   false
* end
end
```

```ruby
if current_user.has_role?(:admin)
  render_admin_panel
end
```

---

```ruby
if current_user
  @listings = current_user.visible_listings
else
  @listings = Listing.publicly_visible
end
```

---

```ruby
class GuestUser
  def initialize(session)
    @session = session
  end

def name
    "Guest"
  end

def authenticated?
    false
  end

def has_role?(role)
    false
  end

* def visibile_listings
*   @listings = Listing.publicly_visible
* end
end
```

```ruby
@listings = current_user.visible_listings
```

---

```ruby
if current_user
  current_user.last_seen_online = Time.current
end
```

---

```ruby
class GuestUser
  def initialize(session)
    @session = session
  end

def name
    "Guest"
  end

def authenticated?
    false
  end

def has_role?(role)
    false
  end

def visibile_listings
    @listings = Listing.publicly_visible
  end

* def last_seen_online=(time)
*   # NO-OP
* end
end
```

```ruby
current_user.last_seen_online = Time.current
```

---

```ruby
cart = if current_user
         current_user.cart
       else
         SessionCart.new(session)
       end

cart.add_item(some_item, 1)
```

---

```ruby
class GuestUser
  def initialize(session)
    @session = session
  end

def name
    "Guest"
  end

def authenticated?
    false
  end

def has_role?(role)
    false
  end

def visibile_listings
    @listings = Listing.publicly_visible
  end

def last_seen_online=(time)
    # NO-OP
  end

* def cart
*   SessionCart.new(session)
* end
end
```

```ruby
current_user.cart.add_item(some_item, 1)
```

---

# Over-Coupling in a (Rails) Codebase

- Rails autoloading implicitly encourages over-coupling
- Models are referred to throughout the system, wherever needed
- Result is a small change over here breaking things over there unexpectedly

---

# Taming Over-Coupling with Concept APIs

- A basic but effective technique for clarifying concepts and their interfaces
- Adds a new tree to the code, `app/concepts`
- Rule #1: top-level files define the sole API for the concept
- Rule #2: any type/function defined in a given tree may not be accessed from
  other concept trees
- Result: changing concrete implementations becomes safe and straightforward

---

# Example

```bash
app/concepts
├── applicants
├── survey
│   ├── sending.rb
└── survey.rb
```

---

```ruby
# app/concepts/applicants.rb

module Applicants
  module_function

def survey_to_send(applicant)
    Applicants::Survey.to_send(applicant)
  end

end
```

---

```ruby
app/concepts/applicants/survey.rb

module Applicants
  module Survey
    module_function

def to_send(applicant)
      Sending.new(applicant).to_send
    end

end
end
```

---

```ruby
# app/concepts/applicants/survey/sending.rb

module Applicants
  module Survey
    class Sending
      attr_reader :applicant

def initialize(applicant)
        @applicant = ApplicantDecorator.new(applicant)
      end

def to_send
        # Concrete implementation
        # ...
      end

end
  end
end
```

---

# Be quicker to remove and slower to add gems

---

# Personal Patterns

---

# Take time to review the full standard library

---

# Study during work hours

- *Positive and rapid ROI* in my experience
- Slow down to speed up
- A good workplace should encourage and facilitate

---

# Really get to know your editor

- It doesn't matter which one you choose, but know it well
- The following should all be effortless:
  - Opening relevant files (in splits)
  - Running tests (under cursor, last run, etc.)
  - Search and replace
  - Autocompletion (still working on this one in vim for Ruby!)
- Start from scratch
- Do not add config that you don't fully understand

---

# And your shell / terminal / multiplexer

- I recommend zsh + tmux (+ vim/neovim)
- Again start from scratch (no Oh-my-zsh)
- Again only add config that you fully understand
- Get to know the most useful UNIX tools
  - e.g. `grep`, `find`, `xargs`, `sed`, `fzf`

---

# Not to mention Git

- Configure aliases for common commands
- Keep practising until rebasing holds no fear of lost/mangled work
- Make small commits and squash them later

---

# Practice touch typing on an ongoing basis

- This is the upper bound on your productivity
- Optimise for accuracy as well as speed
- http://www.speedcoder.net/lessons/ruby is basic but free
- Also Typist on the Mac App Store is free and works well

---

# And finally, words of wisdom from NASA:

---

# Fail Early

---

# Then Stop Failing

---

# Thank you!

## Discussion / Questions