Fighting
Speed & Complexity with
Technology & Automation
"How to handle even more complexity with a small brain."
Hi!
Johann
@ Mayflower
CTO
"Chief Tailwind Officer"
Hacker at Heart
Developer?
Java, JavaScript, PHP, C,
C#, Python, Ruby?
#mainstreamdevs #ftw
Rust, Kotlin,
TypeScript, Dart?
#ilikenewstuff #hipster
WebAssembly, Elixir, ELM?
#impressfriends #nerd
Is learning new languages more important in 2019?
What the Business
expects from us 2019
1958: 60 Years
Average Age of S&P Fortune 500
2017: 19 Years
Average Age of S&P Fortune 500
March 2009
Founding
November 2011
330.000.000 Valuation
December 2014
41.000.000.000 $
May 2019
82.400.000.000 $ IPO
10 years from 0 to 82 Billion
September 2017
founded
September 2018
2.000.000.000 $ Valuation
November 2018
100.000.000 $ Revenue Run Rate
Available in 120 cities
Zero to Unicorn in a few months
Could i get the software faster, please?
Universal C-Level Person, since 2000
Speed?
Yep, that's something we can deliver.
Prototype in two weeks?
I'll do it in one.
It's a prototype, right?
We are going to throw it all away, right?
#greenfield #ftw
One Week later
-
You know the overall structure
-
The APIs are simple
-
Development is fun
-
Low cognitive load
-
Fast Changes
-
Hey, Your software is great!
Let's add some features ...
Brilliant, the customers love it.
They just need one more thing.
"Hmm, i thought there was already a method for this ... but nevermind..."
6 months later ...
-
You know the overall structure -
The APIs are simple -
Development is fun-
Low cognitive load -
Fast Changes
-
Uncle Bob
“Indeed, the ratio of time spent
reading versus writing is
well over 10 to 1.
We are constantly reading old code as part of the effort to write new code. ...[Therefore,] making it easy to read makes it easier to write.”
10 : 1 !
Fast comprehension >
Fast keyboard
How we understand software
(with a small brain)
Top-Down Comprehension:
-
Hypotheses based on existing knowledge
-
Search for places to prove or falsify them
Bottom-Up Comprehension:
-
Search for things you recognize
-
Combine known units to understand
larger sections
Default for most Developers:
Start with top-down analysis
Apply Bottom-up when You
recognize something
while validating
How many items can you remember at the same time?
Your Working Memory:
max. 7 (+2) items
Developers chunk and slice code to understand it.
Known Patterns &
Ignored Lines
Spaghetti Monolith
How about a nice monolith?
Predictable Implementations
a.k.a.
Design Patterns
ca 1995
Well, this is better, but ...
As long your brain stays small
large software stays slow.
Project Size | Lines of Code/Day/Dev | Lines of Code / Day |
---|---|---|
10.000 | 10 - 125 | 10-125 |
100.000 | 5 - 100 | 5-100 |
1.000.000 | 3,5 - 50 | 3,5-50 |
10.000.000 | 1.5 - 25 | 1,5-25 |
Source: Steve McConnell, Software Sizing
Die, Monolith, Die!
Cutting Software into Pieces
Frontend
Backend
SOAP
+ ESB
ca 2002
Frontend
Backend
REST+
JSON
ca 2005
... because it's already complicated enough without SOA.
Frontend
Backend
REST+
JSON
JavaScript-Frontend
REST+
JSON
ca 2010
Frontend
Backend
REST+
JSON
JavaScript-Frontend
REST+
JSON
Every
Requirement
ca 2010
I still need to understand every layer for debugging :/
Catalog
Domain Driven Design
Inventory
Orders
Payment
ca 2005
Ok, that's better.
(Ok, i still had to learn Design Patterns & DDD)
It's not about the tactical design patterns, though.
DevOps
DEV
OPS
ca 2009
Automate all the
Standard Tasks
DEV
OPS
Faster Feedback =
Less things to remember
DEV
OPS
Faster Understanding
with more Observability
DEV
OPS
Catalog
Service
Elastic
Search
MicroServices
Inventory
Service
PostgreSQL
Orders
Service
MySQL
Payment
Service
-
ca 2013
MicroServices
Your software is broken down into multiple services that
- can be deployed independently -> fast feedback
- provide one business capability -> smaller domain
- have no dependency on central governance
- have no dependency on external data management
- rely on automated infrastructure / integration / deployment
Microservices: software that fits in your head
Dan North
MicroServices
If a monolith would be easier to understand than ...
-
network latency
-
distributed transactions
-
fault tolerance
-
operational complexity
-
technology diversity
... you should do a monolith first
Couldn't just somebody else care about the complicated stuff?
Serverless
Automate All The Things!
ca 2015
- should be "service-full"
- Function as a Service / Lambda /λ
- Serverless databases / Aurora etc
- SQS, SNS, ELB, Route53,
- function as a deploy unit
- Pay-as-you-go
- Scaling is a solved problem
- Event-driven
- You won't get fired for choosing AWS
Observability included
Text
5 lines of λ
5 lines of
configuration
But ...
But, but ...
-
Take the afternoon off,
Amazon/Auth0/Slack/Github is down
-
Vendor Lock-in and - even worse: Vendor Control
"External Services are the New Silo"
-
Innovation:
OpenSource -> cloud provider company secrets
- Pricing by value, not by costs:
Expensive API Gateways, anyone?
Kubernetes is the new OS
ca 2019
... on premise
... in the cloud
... on your laptop
... on the edge (tbd)
Service Meshes
Let's bundle & automate all the production stuff
You shouldn't think about.
- ISTIO, Linkerd (2),Consul
- Load Balancing, FailOver, Retries
- Traffic control, Routing, Service Discovery
- CircuitBreakers, Health Checks
- access control, rate limits
- rich metrics, logs, traces
- secure, authenticated & authorized communication
- Canary deployments, Fault Injection
Ok, Microservices: check.
What about serverless?
Knative
Vendor-agnostic serverless plattform for kubernetes
Component | |
---|---|
Serving | Auto-Scaling with scale to zero |
Build | Cloud-native from source to production |
Events | Event-infrastructure for container services and functions |
Google Cloud Run, Gitlab Serverless, Triggermesh
Bloody edge.
It still hurts.
1. Craftership & Design Patterns
2. Proper Decomposition / DDD
3. Automate all the things
4. Even better: let others automate it
5. Tooling for Transparency
Howto: speed & complexity
with a small brain 2019
Ok, and what should i do about it?
Rule #1:
Don't focus on
hype technology
Gartner HypeCycle
Eary adopters come for new technology -
and leave for new technology
Cool, new
shiny stuff!
I'm way
ahead
Uh-Oh, get out of here!
Gone
That's why we've
got the Chasm
Rule #2
Mastery in One Language
Needed for:
-
Top-Down Comprehension
-
Design Patterns
-
Craftership
Mastery in One Language
5 years, full-time
-
"10.000 hours for mastery"
-
backed by scientific research
for program comprehension
Coding-Dojos & Pair Programming:
- smart algorithms
- common idioms
- design patterns
- solution strategies
Rule #3
One more language...
Understand your work/
co-workers:
-
JavaScript/TypeScript for
Browser-Dev
-
Kotlin/Swift/Dart for
mobile Development
-
Python/R for data analytics
- Go for fast services ...
Languages to know 2019
Stackoverflow
Wanted | Github Pushes | Google Trends | Recommended | |
---|---|---|---|---|
Python | 25.6 | 14.2 | ++ | First or Second |
JavaScript | 17.8 | 26.8 | o | First |
Go | 15.0 | 4.2 | ++ | First or Second |
TypeScript | 14.6 | 3.8 | ++ | First or Second |
Kotlin | 11.1 | 0.6 | ++ | Second |
Rust | 9.5 | 0.6 | + | Second |
C++ | 9.1 | 7.4 | o | Second |
Webassembly | 8.9 | 0.0 | o | Not recommended |
Java | 8.3 | 11.1 | - | First |
PHP | 3.5 | 5.7 | - | First |
Scala | 4.3 | 1.5 | - | Not recommended |
Rule #4: a helpful environment 1
IDE Support
Use a helpful environment 2
Automation
Use a helpful environment 3
Observability
The Secret:How to become a
10-times programmer
- Organize coding dojos at in your team
- Do a lot of pair programming
- Practice clean code
- Train until 5 of you are 3 times more effective
- Done, that's even better than one 10 times programmer!
What comes next
Genesis & Prophet (2016)
Automatic error detection & Patch Generation
16 of 39 Patches are correct
DeepCode.ai
Trained on existing fixes from Github
Analyses your code and recommends known patches
askbayou.com
- Military Project
- Create new Programs based on sketches
import java.io.*;
import java.util.*;
public class TestIO {
void read(File file) {
{
/// call:readLine type:FileReader type:BufferedReader
}
}
}
import java.io.*;
import java.util.*;
import java.io.FileReader;
import java.io.File;
import java.io.IOException;
import java.io.FileNotFoundException;
import java.io.BufferedReader;
public class TestIO {
void read(File file) {
{
FileReader fr1;
BufferedReader br1;
String s1;
try {
fr1 = new FileReader(file);
br1 = new BufferedReader(fr1);
while ((s1 = br1.readLine()) != null) {}
br1.close();
} catch (FileNotFoundException _e) {
} catch (IOException _e) {
}
return;
}
}
}
So, i still have to work in future?
The amount of time understanding sourcecode has not decreased notably in 30 years.
We didn't win, but we didn't loose, either.
Fighting Speed & Technology with Technology & Automation
By Johann-Peter Hartmann
Fighting Speed & Technology with Technology & Automation
The Time to Market of our products decreases every year, we get earlier customer feedback, we implement their wishes faster while integrating more and scaling better. To achieve this we reinvent ourselves and our methods every year: incremental spaghetti code became a pattern oriented monolith, where we added layers and replaced them with DDD contexts. We added APIs, made it SOA, moved to microservices and serverless, switching from RPC to REST to reactive. The one-trick-pony "Java in university and job" lives in the animal sanctuary right now, while we try to learn the new AWS service and the JavaScript-Framework of the week.
- 538