Asynchronous Data Streams with RxJava

ReactiveX

An API / library  for composing asynchronous and event driven programs by using observable sequences

A new idea/paradigm in programming

Ported in more than 10 languages

RxJava ported by Netflix

• Data Stream: A sequence of values
• Micro-services consume & process a ton of Data Streams from apis, dbs, files and internal data structures with different response times
• Non-blocking processing is key for minimizing latency

 Programming with

Asynchronous Data Streams

is essentially

Reactive programming

based on the principle of non-blocking push rather than pull

An easy and scalable way to propagate all data changes to different parts/layers of our app that may want to react on them in parallel

Other paradigms & apis

Event Bus

(Pub-sub pattern):

No clear connection between event producer and event consumer

Difficult to track your code logic

Difficult to test

Single data pipeline and a ton of XXXEvent Classes

Other paradigms & apis

CompletableFuture

(mixes blocking and non-blocking approach into one API ):

More oriented to asynchronously running of tasks with side effects or returning single result

Complex and non intuitive for asynchronous processing of streams

Limited api and composition capabilities

RxJava

(a combination of the Observer pattern, the Iterator pattern, and functional programming)

Observable(source of data stream, Observer(subscribes/listens/reacts)

RxJava

Takes reactive programming to the next level

Functional & reactive programing - asynchronous data streams on steroids:

Easily create data streams of anything, i.e. variables, user inputs, properties, caches, data structures. Data streams are cheap and ubiquitous.
Apply functional composition to the data stream, i.e., have an amazing toolbox of functions to combine, map, filter, retry, throttle, any of those streams and an easy way to create new operators.

RxJava

Easy threading management (abstract away low-level threading issue)

Easy async error handling

More declarative, less side-effects and less mutable state

Higher manipulation, reusability, modularity of code and objects

create data streams of anything

Observable.just(1, 2, 3).toBlocking().subscribe(System.out::println);
Observable.range(100, 120).toBlocking().subscribe(System.out::println);
Observable.fromCallable(() -> 10).toBlocking().subscribe(System.out::println);
Observable.from(ImmutableList.of(1, 2, 3)).toBlocking().subscribe(System.out::println);
Observable.from(new Integer[]{ 1, 2, 3 }).toBlocking().subscribe(System.out::println);
Observable.form(future)...

Observable.create(subscriber -> {
  try {
    if (subscriber.isUnsubscribed()) {
      return;
    }
    subscriber.onNext(1000);
    subscriber.onCompleted();
  } catch (Exception e) {
    if (!subscriber.isUnsubscribed()) {
      subscriber.onError(e);
    }
  }
}).toBlocking().subscribe(System.out::println);

Concurrently get docs from external api

Observable<Doc> getObservableDoc(String docName) { 
    return Observable.create(subscriber -> {
        subscriber.onNext(esClient.getDoc(docName));
        subscriber.onCompleted();
    }
}


Observable
    .from(ImmutableList.of(“doc1”, “doc2”, “doc3”))
    .flatMap(docName -> getObservableDoc(docName))
    .subscribeOn(Schedulers.io())
    .observeOn(Schedulers.computation())
    .subscribe(document -> System.out.println(document::getTitle));

Sum even, GroupBy

Observable
    .just(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
    .filter(item -> item % 2 == 0)
    .scan((sum, item) -> sum + item)
    .last()
    .subscribe(System.out::println);


Observable
    .just("cat", "dog", "ant", "bee")
    .groupBy(animal -> {
        if (animal.equals("cat") || animal.equals("dog") {
           return "mammal";
        } else {
            return "insect";
        }
    })
    .subscribe(groupObservable -> {
        groupObservable.toList()
        .subscribe(itemInGroup -> System.out.println("Group " + 
            groupObservable.getKey() + 
            " items: " + itemsInGroup));
    });

Event Bus with RxJava

public class RxBus {

    private final Subject<Object, Object> rxBus = 
        PublishSubject.create().toSerialized();

    
    public void send(Object event) {
        rxBus.onNext(event);
    }

    public Observable<Object> toObservable() {
        return rxBus;
    }
}

rxBus.toObservable().subscribe(event -> processEvent(event))

Observable<T> to CompletableFuture<T>

public static <T> CompletableFuture<List<T>> fromObservable(Observable<T> observable) {
    final CompletableFuture<List<T>> future = new CompletableFuture<>();
    observable
            .doOnError(future::completeExceptionally)
            .toList()
            .forEach(future::complete);
    return future;
}


public static <T> Observable<T> toObservable(CompletableFuture<T> future) {
    return Observable.create(subscriber ->
        future.whenComplete((result, error) -> {
            if (error != null) {
                subscriber.onError(error);
            } else {
                subscriber.onNext(result);
                subscriber.onCompleted();
            }
        })
    );
}

rx.Completable

Completable task1 = Completable
    .fromAction(() -> System.out.println("Task 1 completed"))
    .subscribeOn(Schedulers.io());

Completable task2 = Completable
    .fromAction(() -> System.out.println("Task 2 completed"))
    .subscribeOn(Schedulers.io());

Completable.merge(task1, task2).subscribeOn(Schedulers.io()).await();

tranform, filter, math, boolean

distinct, elementAt, filter, find, first, last, pausable, skip, skipUntil, take, takeUntil

average, count, max, min, reduce, sum

every, some, includes, sequenceEqual

delay, findIndex, map, scan, debounce

Composing Observables

Observable.merge(o1, o2, ...)
1---2---3---4------|
------5--------6---|
1---2-5-3---4--6---|

Observable.compineLatest(List<Observable<T>>, (List<T>) -> {})
--1------2---------------3--4----5
------A-----B------C-D------------
--1A-----2A-2B----2C-2D--3D-4D---5D

Observable.concat(o1, o2)
---1------1--------------1---
--2---2--
---1------1--------------1---2---2--

Observable.just(1, 2, 3).withLatestFrom(ob2, (x, y) -> x + y);
Observable.zip(List<Observable<T>>, (List<T>) -> {})
Observable.amp(o1, o2); // emit all items from the first to emit

Concat Observables

Merge Observables

Handling Errors

Observable<String> numbers = Observable
    .just("1", "2", "three", "4", "5")
    .map(Integer::parseInt)
    .onErrorReturn(e -> -1)
    .subscribe(System.out::println);

Observable<Integer> defaultOnError =
    Observable.just(5, 4, 3, 2, 1);

Observable<String> numbers = Observable
    .just("1", "2", "three", "4", "5")
    .map(Integer::parseInt)
    .onExceptionResumeNext(defaultOnError);


Observable.just(1, 2, 3, 4, 5, 6)
    .timeout(100, TimeUnit.MILLISECONDS)
    .retryWhen(failedAttempts -> {
        return failedAttempts
            .zipWith(Observable.just(1, 5, 10), (failedAttempt, wait) -> wait)
            .flatMap(wait -> Observable.timer(wait, TimeUnit.SECONDS));
    })
    .subscribe(System.out::println);

Buffering & Throttling

Observable
    .sample(Observable
        .interval(100L, TimeUnit.MILLISECONDS)
        .take(10)
        .concatWith(Observable
          .interval(200L, TimeUnit.MILLISECONDS))
    )
    .debounce(150L, TimeUnit.MILLISECONDS)
    .onBackpressureBuffer(10000)
    .onBackpressureDrop()
    .window(3L, 200L, TimeUnit.MILLISECONDS);

Orion Observable Deploys

Observable<DeployableState> observableDeploy = Observable.create(subscriber -> {
    subscriber.onNext(deployer.deploy())
    subscriber.onComplete()
}
.concatWith(
    Observable.interval(5, TimeUnit.SECOMDS, Schedulers.io())
        .map(tick -> {
            if ((tick + 1L) * 5) > maxWaitTimeSeconds) {return TIMED_OUT}
            return watcherFuction.apply(deployRequest, deployable)
        }
        .lift(new OperatorTakeWhileInclusive(watchedState -> !watchedState.isTerminalState)));

)
.onErrorReturn(throwable -> FAILED_TO_ACTIVATE)
.subcribeOn(Schedulers.io())
.observeOn(Schedulers.computation());


Observable<Deployable> getObservableOfCompinedDeployStates(
    List<Observable<DeployableState>> childObservableDeploys) {
    
    return Observable.combineLatest(childObservableDeploys, (latestStateList) -> {});

};

Cold vs Hot Observables

Observable<Integer> obj = Observable.just(1, 2, 3).publish()

// Observable<Integer> obj = Observable.just(1, 2, 3).replay()
// Observable<Integer> obj = Observable.just(1, 2, 3).publish().refCount()

Subscription sub1 = obj.subscribe(System.out::println);
Subscription sub2 = obj.subscribe(System.out::println);
obj.connect();

Resources

http://reactivex.io/tutorials.html

https://github.com/ReactiveX/RxJava/wiki

https://github.com/ReactiveX/RxNetty

https://github.com/davidmoten/rxjava-jdbc