Modern Java Concurrency



Adam Retter

Evolved Binary Ltd


@adamretter / adam.retter@googlemail.com

Adam Retter

  • First and foremost... I am very under prepared!

  • Consultant

    • Scala

    • Java

    • XQuery, XSLT

  • Open Source Hacker

    • Predominantly NoSQL Database Internals

    • e.g. eXist, RocksDB, Shadoop (Scala Hadoop M/R framework)

  • W3C Invited Expert for XQuery WG

  • Author of  "eXist" book for O'Reilly

  • ...Recently moved to Bristol!

What I want to talk about...

  • Kind of a whirlwind tour...

    • May introduce many concepts and topics

    • Little detail... lots for you to research later

    • Scare you with JVM and Java SE concurrency

    • Calm you with other concurrency options



  1. Concurrency Fundamentals

  2. Concurrency on the JVM

  3. Modern Java SE Concurrency

  4. Better Concurrency Options

My Java Concurrency Journey

  1. Do some basic multi-threaded stuff

    • Appears to work

    • Think concurrency is simple

  2. Hear other people complaining concurrency is hard

    • Don't really believe them

    • TMy experience was positive... I must know more!?!

    • Blissful ignorance ☺☺

  3. Fix some deadlock/livelock bugs

    • Feel like a concurrency ninja ☺☺☺

My Java Concurrency Journey

  1. Learn and read more about Java Concurrency in depth

    • The less we feel we know

    • The more insecure we feel ☹☹

    • Concurrency is hard! ☹☹☹

  2. Search for higher level abstractions

    • Feel much happier

    • Be more productive

    • Do not be ignorant of the hard stuff!

Concurrency Principles

  • What is concurrency?

    • Several computations are executing simultaneously

    • ...and possibly interacting with each other!

Concurrency Principles

  • Why do we need concurrency?

    • CPU Clock Speeds are not increasing

      • Often decreasing to save power!

      • Many small cores

    • Need to make use of many compute cores

      • Project Sumatra - GPU from Java

      • Sparc T-5, / Tilera / Azul Vega 3 / Compute clouds etc

    • Not linear speed, rather throughput

    • Not just speed but also isolation and containment

Concurrency Principles

  • Why do we need concurrency?

    • CPU Clock Speeds are not increasing

      • Often decreasing to save power!

      • Many small cores

    • Need to make use of many compute cores

      • Tilera (64 cores)

      • Oracle Sparc T-5 (128 cores)

      • Azul Vega 3 (864 cores)

      • Project Sumatra - GPU from Java

      • Compute clouds etc

    • Not linear speed, rather throughput

    • Not just speed... also isolation and containment

Concurrency Primitives

  • Process vs. Thread

    • Processes

      • Managed by the OS

      • Own resources (file handles, memory, etc)

      • Have their own address space

      • e.g. Java


    • Threads

      • Within a process

      • Lightweight compared to process

Thread Safety




"A procedure is thread safe when it is logically correct when executed simultaneously by several threads."


- Multithreaded Programming Guide, Oracle.
https://docs.oracle.com/cd/E26502_01/html/E35303/compat-14994.html

Is Concurrency Simple?

  • Executing independent Tasks is simple

  • Parallel decomposition of existing linear workloads is hard

  • Mutable shared state is hard

  • Task/Data access coordination is hard

    • Consistency vs Eventual Consistency

    • Most likely requires synchronisation

Synchronisation

  • Critical Section

    • Code that accesses a shared resource that must not be concurrently accessed by more than one thread

    • Requires synchronising thread access!

  • Mutual Exclusion

    • Only 1 process concurrently within the same critical section

    • Semaphore, Binary or Counting

    • Mutex, has an owner

    • Lock / Spinlock / Reentrant lock / Readers-writers lock

Synchronisation Issues

  • Caused by interleaved access from multiple threads

  • Issues may rarely manifest themselves

  • When they do manifest, typically under worst conditions

    • i.e. heavy production load

Race Condition

  • Caused by a lack of correct synchronisation

  • 2+ threads dependant on a mutable resource

  • The result is sometimes not as expected


Consider the function:

private int value = 0;

public int getNext() {
    return value++;
}
Q: Is this thread-safe?

Race Condition

A: No! Because....    value++

  • Not an atomic operation!

  • Sugar for 3 operations: get, add and set

Deadlock

  • 2+ threads holding and trying to acquire opposing locks

  • Cyclic locking dependency between threads

    • Directed graph: Nodes are threads, Edges are access relationship


Consider the functions:

public void leftRight() {
    synchronized (left) {
        synchronized (right) {
            doSomething();
        }
    }
}
    
public void rightLeft() {
    synchronized (right) {
        synchronized (left) {
            doSomethingElse();
        }
    }
}
Q: Is this thread-safe?

Deadlock

  • Q: No! Deadlock prone!

  • Tools/Solutions

    • Open Calls

      • Calling an (object) method with no locks held

      • Can help by encapsulating locking

    • Lock ordering

    • Deadlock detection, abort one thread (and retry?)


Livelock

  • Often comes from overeager error-recovery code

    • Mistakes Unrecoverable error as Recoverable and retries

  • Two threads change their state in response to each other

    • e.g. Two polite people walking down a hallway, try to get out of each others way...and again... and again...

    • Can be mitigated by introducing some entropy into the action or delay

Concurrency on the JVM

  • java.lang.Runnable

    • Something that can be executed

    • Possibly in a Thread

  • Threads

    • java.lang.Thread

      • (now) mapped to OS/Kernel threads

      • Daemon or Non-Daemon (user) threads

    • java.lang.ThreadGroup

Synchronisation in Java SE

  • Explicit

    • java.util.concurrent.locks

      • ReentrantLock

      • ReentrantReadWriteLock

        • Supports downgrading

        • Manual lock upgrading is deadlock prone!

      • StampedLock (Java 8), Not Reentrant!

Synchronisation in Java SE

  • Implicit

    • synchronized keyword


      • Used on methods, acquires the objects intrinsic lock

        public synchronized setValue(final int value)


      • On referenced object, acquires intrinsic lock on specified object

        synchronized(thing){ ... }


      • Is Reentrant!

Do you really understand Java Concurrency?

  • What does the volatile keyword do?

  • What issues may occur with escaping references in constructors?

  • What is the JMM (Java Memory Model)?

    • Why is it important for concurrency?

    • Why did it change for Java 1.5?

Java Concurrency Bible

Modern Java SE Concurrency (1.5)

  • JSR-133 Java Memory Model and Thread Specification Revision

    • Corrected and formalized the operation of synchronized and volatile

    • Detailed how immutable objects work with multithreading

      • Thread-safe provided that references aren't allowed to "escape" while the constructor is being executed

    • Enabled non-blocking coordination among threads through the use of volatile.

Modern Java SE Concurrency (1.5)

  • Callable and Future

    • Only really small improvements over java.lang.Runnable

    • Win! Can obtain result from asynchronous computation without shared mutable state

    • Easier to coordinate Threads and their lifetimes

  • java.util.concurrent.Callable

    • Similar to java.lang.Runnable, but...

      • allows you to return a result

      • allows you to throw a checked Exception

Modern Java SE Concurrency (1.5)

  • java.util.concurrent.Future

    • The result of an asynchronous computation

      • e.g. At some point in the future there MAY be a result

    • ExecutionException wraps possible checked exception of Callable

    • Also provides synchronous (blocking) mechanism for getting result

  • java.util.concurrent.ExecutorService

    • The glue between Callable and Future

    • Submit a Callable to an ExecutorService and you are given a Future

Java 1.5 SE Concurrency

  • java.util.concurrent.atomic

    • Lock-free

    • Thread-safe

    • Uses the CPUs CAS (Compare and Swap/Set) instructions

    • Provides atomic conditional update of a value/reference

import java.util.concurrent.atomic.AtomicInteger;

private AtomicInteger value = new AtomicInteger();

public int getNext() {
    return value.getAndIncrement();
}
        

Java 7 SE Concurrency

  • Fork/Join framework

    • The new old stuff!

      • i.e. really just an ExecutorService plus some glitter

    • TODO What sort of parallelism approach does this provide?

    • TODO Examples...

Java 8 SE Concurrency

  • java.util.streams

    • Collection.parallelStream

    • A different approach to parallelism, i.e. SIMD approach

    • Inspired by Scala's Parallel Collections???

    • TODO discuss issues around ordering and side-effects...

    • TODO Examples...

  • java.util.concurrent

    • CompletableFuture

    • CompletionStage

    • TODO Examples...

So, is Java Concurrency Easy?

  • Mutable shared state is the real issue!

  • (Correct) synchronised access is really hard!

  • Threads and Locks are your primitives!

    • Sadly, Callable and Future offer little more

  • Atomics are very helpful... but limited by types

  • TODO check --> Fork/Join does not help with shared state!

  • java.util.streams can help with SIMD like parallel processing of data

Dreaming of an Easier Life

  • Higher-level abstractions

  • Containment of Shared State

  • Correct protected access to Mutable Shared State

  • Concurrency where deadlocks are impossible

    • Actually, why should I even have to care about locks?

  • I want to: focus on what rather than how!

Along Came "Reactive"...

  • It's a Philosophy towards how you compose both your application and larger systems


"We want systems that are Responsive, Resilient, Elastic and Message Driven. We call these Reactive Systems."

- www.reactivemanifesto.org

  • Ah! We are going to need more asynchronous concurrency :-/

Reactive

  • Responsive

  • Resilient

  • Elastic

  • Message Driven



p.s. The "Principles of Reactive Programming" online course with Coursera starts again on April 13th 2015

- https://www.coursera.org/course/reactive

Modern Java Concurrency

So, if we have to do concurrency, what better options exist?

  • ReactiveX

    • Based on Observables

    • Like Iterators but for Concurrency

  • Akka

    • Based on Actors

  • TODO others?