Development Craftsmanship

Agenda

  • Object Oriented Programming
  • Clean Code
  • Migrations
  • Feature Toggles
  • Testing
  • Logging & Monitoring
  • CI/CD
  • MVP

Hey, I'm Boaz

Full Stack Developer

4.5 years in Shavit

Object Oriented Programming

Abstractions & Encapsulation

method signatures

public class Random {
  public byte[] generate(int length) {
    ...
  }
}

Polymorphism & Inheritance

overload an action

public class SecureRandom extends Random {
  public byte[] generate(int length) {
    ...
  }
}

Principles

Good OOP

Single Responsibility

Open/Closed Principle

Liskov Substitution Principle

Interface Segregation Principle

Dependency Inversion Principle

public class StringProcessor {
    private final StringReader stringReader;
    private final StringWriter stringWriter;
     
    public StringProcessor(StringReader stringReader, 
                StringWriter stringWriter) {
        this.stringReader = stringReader;
        this.stringWriter = stringWriter;
    }
 
    public void printString() {
        stringWriter.write(stringReader.getValue());
    }
}

Composition Over Inheritance

Advanced Topics

In Software

Clean Code

No abbreviations

public int cntEvn(Collection<int> cln) {
    int cnt = 0;
    for(int i : cln) {
        if(i % 2 == 0) {
            cnt++;
        }
    }
    return cnt;
}
public int countEvenItems(Collection<int> collection) {
    int evenItems = 0;

    for(int i : collection) {
        if(item % 2 == 0) {
            evenItems++;
        }
    }

    return evenItems;
}

Be Consistent

(No synonyms)

public void countThings(Collection<int> things) {
    ...
}
public void numerateThings(Collection<int> things) {
    ...
}

You Ain't Gonna Need It (YAGNI)

Occam's razor /

Keep It Simple Stupid

Don't Repeat Yourself but Write Everything Twice

Database Migrations

(Go) Migrate

Why database migrations?

mongodb-migrations

Maven (Java):
​src/main/resources/db/migration/V1/V1.0__Create_Table_users.sql
CREATE TABLE users (
  id bigint(20) NOT NULL AUTO_INCREMENT,
  username varchar(100) NOT NULL,
  first_name varchar(50) NOT NULL,
  last_name varchar(50) DEFAULT NULL,
  PRIMARY KEY (id),
  UNIQUE KEY UK_username (username)
) DEFAULT CHARSET=utf8;
​src/main/resources/db/migration/V1/V1.1__Create_Table_admins.sql
​src/main/resources/db/migration/V2/V2.0__Create_Table_things.sql

Never rename.

Add column, n versions later remove old column.

Feature Toogles & Experiments

Feature Toggles (aka Feature Flags)

  function reticulateSplines(){
    // current implementation lives here
  }
  function reticulateSplines(){
    var useNewAlgorithm = false;
    // useNewAlgorithm = true; // UNCOMMENT IF YOU ARE WORKING 
    // ON THE NEW SR ALGORITHM
  
    if( useNewAlgorithm ){
      return enhancedSplineReticulation();
    }else{
      return oldFashionedSplineReticulation();
    }
  }
  
  function oldFashionedSplineReticulation(){
    // current implementation lives here
  }
  
  function enhancedSplineReticulation(){
    // TODO: implement better SR algorithm
  }

How do I introduce change?

Experiments

A/B Testing

Canary Release

Stateless

Push state to the boundaries (Databases)

What is a stateless app?

Testing

End To End Tests

(System Tests)

  • Slowest
  • Focus on making sure no major breaks
  • No edge cases
  • Close to business language
public class GoogleSearch {
    static WebDriver driver = new FirefoxDriver();
    static Wait<WebDriver> wait = new WebDriverWait(driver, MAX_30_SECONDS);

    public static void main(String[] args) {
        boolean result;

        driver.get("http://www.google.com/");
        try {
            result = firstPageContainsQAANet();
        } catch(Exception e) {
            result = false;
        } finally {
            driver.close();
        }
 
        System.out.println("Test " + (result? "passed." : "failed."));
    }
 
    private static boolean firstPageContainsQAANet() {
        //type search query
        driver.findElement(By.name("q")).sendKeys("qa automation\n");
 
        // click search
        driver.findElement(By.name("btnG")).click();
 
        // Wait for search to complete
        wait.until(webDriver -> {
                System.out.println("Searching ...");
                return webDriver.findElement(By.id("resultStats")) != null;
        });
 
        // Look for QAAutomation.net in the results
        return driver.findElement(By.tagName("body")).getText().contains("qaautomation.net");
    }
}

Integration Tests

 (Black-Box Testing)

  • Slower
  • Focus on validation correctness of single service at a time
  • Validate correctness, not code quality
  • Boost confident while preserving test isolation and speed
@Test
public class SomeKeyGoogleService {

    public SearchServiceConnection searchService;

    @Before
    public void init() {
        // Initialize service
        searchService = ...;
    }

    @After
    public void clean() {
        // Do some cleaning
    }

    @Describe("Given Youtube service has results and there are existing indexed pages " +
        "When a phrase searched Then return Youtube with indexed results")
    public void usuallyThisWillBeAMeaningfullName() {
        // given
        youtubeMock.when("/searchResults?search=my%20search%20phrase")
            .thenReturn("{video: \"http://youtube.com/v?asd123@!ASd2134=\"}");
        db.insert(new IndexedPage("Welcome to my search phrase"));
        // when
        var returnContent = searchService.call("/search?s=my%20search%20phrase").result().asJson();
        // then
        expect(returnContent).toConsistOf(youtubeVideo("http://youtube.com/v?asd123@!ASd2134="),
             pageInfo("Welcome to my search phrase"));
    }
}

Unit

  • Fastest
  • Focus on validation correctness of single [public] function at a time
  • Validate mostly code quality, but also correctness as a side effect*
  • Encourage clean code
  • Easiest to write
public class AlertAdderTest
{
    public SlackAlertAdder slackAlertAdder = mock(SlackAlertAdder.class);
    public FacebookAlertAdder facebookAlertAdder = mock(FacebookAlertAdder.class);

    public AlertAdder alertAdder = new AlertAdder(slackAlertAdder, facebookAlertAdder);

    @Test
    public void whatAnAlertIsAddedThenSlackAlertIsAdded()
    {
        Alert anAlert = new Info("Holy shit, watch out");

        alertAdder.add(anAlert);

        verify(slackAlertAdder).add(anAlert);
    }
}

Also

Consumer Driven Contract

    (PACT)

Component Testing

Mutation Testing

    (PIT)

Test Driven Development

TDD

Understood?

NO

TDD Cycle

  1. Failing test
  2. Minimal implementation
  3. Refactor

Understood?

Yes

TDD Cycle

  1. Write the smallest possible failing test.
  2. Write the most naive implementation possible.
  3. Refactor for a better quality code.

Understood?

Yes

Central Logging

Produce    ->    Collect    ->    Index       ->    Use

Service-wise

Fluentd/ Logstash

Elasticsearch

Kibana

Also: Correlation Ids, Schema

Kibana Example

Tracing

Using correlation Ids we can create a flow of the data:

Monitoring

Counter counter = Counter
  .builder("instance")
  .description("indicates instance count of the object")
  .tags("dev", "performance")
  .register(registry);
 
counter.increment(2.0);
  
assertTrue(counter.count() == 2);
  
counter.increment(-1);
  
assertTrue(counter.count() == 2);
SimpleMeterRegistry registry = new SimpleMeterRegistry();
Timer timer = registry.timer("app.event");
timer.record(() -> {
    try {
        TimeUnit.MILLISECONDS.sleep(1500);
    } catch (InterruptedException ignored) { }
});
 
timer.record(3000, MILLISECONDS);
 
assertTrue(2 == timer.count());
assertTrue(4510 > timer.totalTime(MILLISECONDS) 
  && 4500 <= timer.totalTime(MILLISECONDS));

Micrometer (Java)

What to monitor?

Business Metrics

A Key Performance Indicator (KPI)

Mean time to failure (MTTF)

Mean Time to Repair (MTTR)

Continues Integration

Continues Deployment

Continues Delivery

Automation Server (Jenkins)

pipeline {
    agent any
    tools {
        maven 'Maven 3.3.9'
        jdk 'jdk8'
    }
    stages {
        stage ('Initialize') {
            steps {
                sh '''
                    echo "PATH = ${PATH}"
                    echo "M2_HOME = ${M2_HOME}"
                '''
            }
        }

        stage ('Build') {
            steps {
                sh 'mvn -Dmaven.test.failure.ignore=true install' 
            }
            post {
                success {
                    junit 'target/surefire-reports/**/*.xml' 
                }
            }
        }
    }
}

Automation Server (CircleCI)

version: 2
jobs:
   build:
     docker:
       - image: circleci/<language>:<version TAG>
     steps:
       - checkout
       - run: echo "hello world"
   deploy:
     docker:
       - image: java:8-jre-alpine
     steps:
       - run: java -jar my-app-*.jar
FROM alpine/git
WORKDIR /app
RUN git clone https://github.com/spring-projects/spring-petclinic.git

FROM maven:3.5-jdk-8-alpine
WORKDIR /app
COPY --from=0 /app/spring-petclinic /app
RUN mvn install

FROM openjdk:8-jre-alpine
WORKDIR /app
COPY --from=1 /app/target/spring-petclinic-1.5.1.jar /app
CMD ["java -jar spring-petclinic-1.5.1.jar"]

Docker

Kubernetes

The Children's Illustrated Guide to Kubernetes

Microservices

Single application as a suite of small services.

 

Service is built around business capabilities and independently deploy-able.

N-Tier Architecture

Microservices

Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.

-- Melvyn Conway, 1967

Domain Driven Design

How do I "built around business capabilities"

Bounded Context

Ubiquitous Language

Minimum Viable Product

Build a simple solution, put it in front of customers, enhance incrementally based on customer feedback.

Resources

  • Clean Code, Robert C. Martin
  • Growing Object Oriented Software, guided by tests- Steve Freeman & Nat Pryce
  • The Twelve Factor App- https://12factor.net/ 
  • Domain Driven Design- Eric Evans
  • Building Microservices- Sam Newman
  • The Pheonix Project- Gene Kim, Kevin Behr, George Spafford
  • Continues Delivery- Jez Humble & David Farley
  • Medium- https://medium.com/
  • Twitter
  • StackShare.io- https://stackshare.io/
  • Newsletters- SWLW, Alligator.io, DevOpsLinks

Development Craftsmanship

By Boaz Berman

Made with Slides.com

Development Craftsmanship

  • 205

More from Boaz Berman