Fighting

Speed & Complexity with

Technology & Automation

"How to handle even more complexity with a small brain."

Hi!

Johann
@ Mayflower

CTO

"Chief Tailwind Officer"

Hacker at Heart

Developer?

Java, JavaScript, PHP, C,
C#, Python, Ruby?

#mainstreamdevs #ftw

Rust, Kotlin,
TypeScript, Dart?

#ilikenewstuff #hipster

WebAssembly, Elixir, ELM?

#impressfriends #nerd

Is learning new languages more important in 2019?

What the Business
expects from us 2019

1958: 60 Years

Average Age of S&P Fortune 500

2017: 19 Years

Average Age of S&P Fortune 500

March 2009

Founding

November 2011

330.000.000 Valuation

December 2014

41.000.000.000 $

May 2019

82.400.000.000 $ IPO

10 years from 0 to 82 Billion

September 2017

founded

September 2018

2.000.000.000 $ Valuation

November 2018

100.000.000 $ Revenue Run Rate

Available in 120 cities

Zero to Unicorn in a few months

Could i get the software faster, please?

Universal C-Level Person, since 2000

Speed?

 

Yep, that's something we can deliver.

Prototype in two weeks?
I'll do it in one.

It's a prototype, right?

We are going to throw it all away, right?

#greenfield #ftw

One Week later

One Week later

  • You know the overall structure

  • The APIs are simple

  • Development is fun

    • Low cognitive load

    • Fast Changes

Hey, Your software is great!

Let's add some features ...

Brilliant, the customers love it.

They just need one more thing.

"Hmm, i thought there was already a method for this ... but nevermind..."

6 months later ...

  • You know the overall structure

  • The APIs are simple

  • Development is fun

    • Low cognitive load

    • Fast Changes

Uncle Bob

“Indeed, the ratio of time spent
reading versus writing is
well over 10 to 1.
We are constantly reading old code as part of the effort to write new code. ...[Therefore,] making it easy to read makes it easier to write.”

10 : 1 !

Fast comprehension >
Fast keyboard

How we understand software

(with a small brain)

Top-Down Comprehension:
 

  • Hypotheses based on  existing knowledge

  • Search for places to prove or falsify them

Bottom-Up Comprehension:
 

  • Search for things you recognize

  • Combine known units to understand
    larger sections

Default for most Developers:
 

Start with top-down analysis
 

Apply Bottom-up  when You
recognize something
while validating

How many items can you remember at the same time?

Your Working Memory:

max. 7 (+2) items

Developers chunk and slice code to understand it.

Known Patterns &
Ignored Lines

Spaghetti Monolith

How about a nice monolith?

Predictable Implementations
a.k.a.

Design Patterns

ca 1995

Well, this is better, but ...

As long your brain stays small
large software stays slow.

Project Size Lines of Code/Day/Dev Lines of Code / Day
10.000 10 - 125 10-125
100.000 5 - 100 5-100
1.000.000 3,5 - 50 3,5-50
10.000.000 1.5 - 25 1,5-25

Source: Steve McConnell, Software Sizing

Die, Monolith, Die!

Cutting Software into Pieces

Frontend

Backend

SOAP
+ ESB

ca 2002

Frontend

Backend

REST+
JSON

ca 2005

... because it's already complicated enough without SOA.

Frontend

Backend

REST+
JSON

JavaScript-Frontend

REST+
JSON

ca 2010

Frontend

Backend

REST+
JSON

JavaScript-Frontend

REST+
JSON

Every
Requirement

ca 2010

I still need to understand every layer for debugging :/

 

 



Catalog


 

 

 

Domain Driven Design

 

 



Inventory


 

 

 

 

 



Orders


 

 

 

 

 



Payment


 

 

 

ca 2005

Ok, that's better.

(Ok, i still had to learn Design Patterns  & DDD)

It's not about the tactical design patterns, though.

DevOps

DEV

OPS

ca 2009

Automate all the
Standard Tasks

DEV

OPS

Faster Feedback =
Less things to remember

DEV

OPS

Faster Understanding
with more Observability

DEV

OPS

 

 



Catalog
Service


Elastic
Search

 

 

MicroServices

 

 



Inventory
Service


PostgreSQL
 

 

 

 

 



Orders
Service


MySQL
 

 

 

 

 



Payment
Service


-
 

 

 

ca 2013

MicroServices

Your software is broken down into multiple services that

  • can be deployed independently -> fast feedback
  • provide one business capability -> smaller domain
  • have no dependency on central governance
  • have no dependency on external data management
  • rely on automated infrastructure / integration / deployment

Microservices: software that fits in your head

Dan North

MicroServices

If  a monolith would be easier to understand than  ...

  • network latency

  • distributed transactions

  • fault tolerance

  • operational complexity

  • technology diversity

... you should do a monolith first

Couldn't just somebody else care about the complicated stuff?

Serverless

Automate All The Things!

ca 2015

  • should be "service-full"
    • Function as a Service / Lambda /λ
    • Serverless databases / Aurora etc
    • SQS, SNS, ELB, Route53,
  • function as a deploy unit
  • Pay-as-you-go
  • Scaling is a solved problem
  • Event-driven
  • You won't get fired for choosing AWS

Observability included

Text

5 lines of λ

 

5 lines of
configuration

But ...

But, but ...

  • Take the afternoon off,
    Amazon/Auth0/Slack/Github is down

     
  • Vendor Lock-in and - even worse: Vendor Control
    "External Services are the New Silo"
     
  • Innovation:
    OpenSource -> cloud provider company secrets

     
  • Pricing by value, not by costs:
    Expensive API Gateways, anyone?

Kubernetes is the new OS

ca 2019

... on premise

... in the cloud

... on your laptop

... on the edge (tbd)

Service Meshes

Let's bundle & automate all the production stuff
You shouldn't think about.

  • ISTIO, Linkerd (2),Consul
  • Load Balancing, FailOver, Retries
  • Traffic control, Routing, Service Discovery
  • CircuitBreakers, Health Checks
  • access control, rate limits
  • rich metrics, logs, traces
  • secure, authenticated & authorized communication
  • Canary deployments, Fault Injection

Ok, Microservices: check.

What about serverless?

Knative

Vendor-agnostic serverless plattform for kubernetes

Component
Serving Auto-Scaling with scale to zero
Build Cloud-native from source to production
Events Event-infrastructure for container services and functions

Google Cloud Run, Gitlab Serverless, Triggermesh

Bloody edge.
It still hurts.

1. Craftership & Design Patterns

2. Proper Decomposition / DDD

3. Automate all the things

4. Even better: let others automate it

5. Tooling for Transparency

 

Howto: speed & complexity
with a small brain 2019

Ok, and what should i do about it?

Rule #1:
Don't focus on
hype technology

Gartner HypeCycle

Eary adopters come for new technology -
and leave for new technology

Cool, new
shiny stuff!

I'm way
ahead

Uh-Oh, get out of here!

Gone

That's why we've
got the Chasm

Rule #2
Mastery in One Language

Needed for:

  • Top-Down Comprehension

  • Design Patterns

  • Craftership

Mastery in One Language

5 years,  full-time

  • "10.000 hours for mastery"

  • backed by scientific research
    for program comprehension


Coding-Dojos &  Pair Programming:

  • smart algorithms
  • common idioms
  • design patterns
  • solution strategies

 

Rule #3
One more language...

Understand your work/
co-workers:

  • JavaScript/TypeScript for
    Browser-Dev

     
  • Kotlin/Swift/Dart for
    mobile Development

     
  • Python/R for data analytics
     
  • Go for fast services ...

Languages to know 2019

Stackoverflow

Wanted Github Pushes Google Trends Recommended
Python 25.6 14.2 ++ First or Second
JavaScript 17.8 26.8 o First
Go 15.0 4.2 ++ First or Second
TypeScript 14.6 3.8 ++ First or Second
Kotlin 11.1 0.6 ++ Second
Rust 9.5 0.6 + Second
C++ 9.1 7.4 o Second
Webassembly 8.9 0.0 o Not recommended
Java 8.3 11.1 - First
PHP 3.5 5.7 - First
Scala 4.3 1.5 - Not recommended

Rule #4: a helpful environment 1

IDE Support

Use a helpful environment 2

Automation

Use a helpful environment 3

Observability

The Secret:How to become a
10-times programmer

  1. Organize coding dojos at in your team
  2. Do a lot of pair programming
  3. Practice clean code
  4. Train until 5 of you are 3 times more effective
  5. Done, that's even better than one 10 times programmer!

What comes next

Genesis & Prophet (2016)

Automatic error detection & Patch Generation
16 of 39 Patches are correct

DeepCode.ai

Trained on existing fixes from Github

 

Analyses your code and recommends known patches

askbayou.com

 

  • Military Project
  • Create new Programs based on sketches
import java.io.*;
import java.util.*;
public class TestIO {
    void read(File file) {
        {
            /// call:readLine type:FileReader type:BufferedReader
        }
    }
}
import java.io.*;
import java.util.*;
import java.io.FileReader;
import java.io.File;
import java.io.IOException;
import java.io.FileNotFoundException;
import java.io.BufferedReader;

public class TestIO {
  void read(File file) {
    {
      FileReader fr1;
      BufferedReader br1;
      String s1;
      try {
        fr1 = new FileReader(file);
        br1 = new BufferedReader(fr1);
        while ((s1 = br1.readLine()) != null) {}
        br1.close();
      } catch (FileNotFoundException _e) {
      } catch (IOException _e) {
      }
      return;
    }
  }
}

So, i still have to work in future?

The amount of time understanding sourcecode has not decreased notably in 30 years.

We didn't win, but we didn't loose, either.

Fighting Speed & Technology with Technology & Automation

By Johann-Peter Hartmann

Fighting Speed & Technology with Technology & Automation

The Time to Market of our products decreases every year, we get earlier customer feedback, we implement their wishes faster while integrating more and scaling better. To achieve this we reinvent ourselves and our methods every year: incremental spaghetti code became a pattern oriented monolith, where we added layers and replaced them with DDD contexts. We added APIs, made it SOA, moved to microservices and serverless, switching from RPC to REST to reactive. The one-trick-pony "Java in university and job" lives in the animal sanctuary right now, while we try to learn the new AWS service and the JavaScript-Framework of the week.

  • 538