RX

Intro to Reactive Extensions

http://slides.com/yshayy/rx/live

Agenda

  • What's RX?
    • Observables & Observers
  • Why RX?
    • Reactive manifesto
  • Soluto use cases

ReactiveX

Originated originally by MS  for handling events and
 data-flows based on the Observer pattern and LINQ.

Open source, cross platform and multi-lingual.

Become more popular lately with the increasing 
popularity of reactive and functional programming.

C# Hello rx


 Observable.Interval(TimeSpan.FromSeconds(1))
           .Select( x => x + ".Hello RX");
           .Subscribe( message => Console.WriteLine(message));

Output:
1. Hello RX
2. Hello RX
3. Hello RX
...

CODE - HELLO RX

CODE - HELLO RX

CODE - HELLO RX

ZIP method/operator


Taken from http://rxmarbles.com/






Those are linq methods, right?

RX building blocks


IObservable<T> - Represents a push based data
stream of values of type T. aka stream or publisher.

interface IObservable<T>{
     IDisposable subscribe(IObserver<T> subscriber)
}


RX building blocks

IObserver<T> -> represent a subscriber for 
incoming messages.

interface IObserver<T>
{    void OnNext(T item);
    void OnError(Exception ex);
    void OnComplete();
}


RX BUILDING BLOCKS

Very similar to Enumrable/Iterable, only push based

Event Enumerable - pull Observable - push
retrieve data MoveNext()
Current		 OnNext( (T)=>void)
discover error throws Exception OnError( (Exception)=>void)
complete !MoveNext() OnCompleted(()=> void)

Due to those similarities, we can use most
LINQ operators for manipulating observables

Produce value every second


Enumerable (Pull) Observable (Push)
Consume 
foreach (var x in GetEnumerable())
{
   DoSomething(x)
}

GetObservable()
.Subscribe( x=> DoSomething(x))
Produce 
IEnumerable<Guid> GetEnumerable() {
   while (true)
   {
      Thread.sleep(1000);
      yield return Guid.NewGuid();
   }
}

IObservable<Guid> GetObservable() {
return Observable
.Interval(Timespan.FromSeconds(1))
.Select(x=> Guid.NewGuid());
}
Blocking vs. non blocking

Code - Auto suggest (NO RX)


Code - Auto suggest (NO RX)

Code - Auto suggest (NO RX)

Code - Auto suggest (NO RX)

Code - Auto suggest (NO RX)

DO you follow?

"My second remark is that our intellectual powers are rather geared to master static relations and that our powers to visualize processes evolving in time are relatively poorly developed. For that reason we should do (as wise programmers aware of our limitations) our utmost to shorten the conceptual gap between the static program and the dynamic process, to make the correspondence between the program (spread out in text space) and the process (spread out in time) as trivial as possible."
By "Edsger+W.+Dijkstra" on "Go to considered harmful" paper

Or more simply put...

DO YOU FOLLOW?



Async is hard
Time is hard
State is hard
We are stupid
Plan accordingly

CODE - AUTO SUGGEST (RX)

SELECT (AKA. map)


CODE - AUTO SUGGEST (RX)

WHERE (AKA. Filter)


CODE - AUTO SUGGEST (RX)

Throttle



CODE - AUTO SUGGEST (RX)

DIsTINCTUNTILUNCHANGED


CODE - AUTO SUGGEST (RX)

SWITCH (aka switchonnext)


CODE - AUTO SUGGEST (RX)

Code - Draw rectangle

The need to go reactive


  • Application requirements have changed over the years -
    • More users requesting data more rapidly
    • Data is very large and come with different sources
      and different shapes.
    • Users expect their application to have
      real time response.
    • Stricter SLAs, High availability -
      even in non mission critical applications.

THE NEED TO GO REACTIVE

    • Offloading, multi-threading and asynchronous processing
      can help solve those issue, but they can add real
      complexity to the system and make it impossible to maintainable, and very difficult to scale.

    THE NEED TO GO REACTIVE

    In 2013, the first version of the reactive
    manifesto was introduced.  
    It described the following traits of a reactive system:

    • Responsiveness 
    • Resilient
    • Scalable/Elastic
    • Event-Driven/Message-Driven

    Reactive manifesto


    REACTIVE MANIFESTO


    Responsive -> react to users

    System should be able to give response to users in solid and consistent time.

    REACTIVE MANIFESTO


    Resilient -> react to errors

    System should be highly available in the face of failure.

    REACTIVE MANIFESTO


    Scalable/elastic -> react to load

    System stay responsive and available under any load.

    REACTIVE MANIFESTO


    Message/Event driven -> react to events

    System should use loosely coupled and isolated components.

    Event driven system features


    1. Asynchronous message passing ->
    By passing events     we make our system more :
    decoupled - as components don't need to know each other.
    scalable - no dependency on specific computation model .

    2. Non blocking -> Means the system could make better utilization of resources, since inactive components release their resources.

    .NET Event model

    What's wrong with .net events?

    • Concurrency -> .net events don't allow to make event dispatch on different thread.
    • Completion -> since events don't have any signaling mechanism for ending, events are usually  the source of many memory leaks.
    • Composition -> it's very difficult to compose two or more .net events.
    • All those effects error handling as well...

    Event handling can be a pain


    Adobe desktop apps 2008*

    • 1/3 of the code in Adobe’s desktop applications is devoted to event handling 

    • 1/2 of the bugs reported during a product cycle exist in this code

    http://infoscience.epfl.ch/record/176887/files/DeprecatingObservers2012.pdf?version=1

    RX solution


    • Extensions for creating and composing observables.
    • LINQ Extensions for high lever data manipulation
    • Scheduling services and extensions for handling concurrency
    • .Net IObservable<T> and IObserver<T> implementations
    Android example - Image Loading 

            Observable<Bitmap> imageObservable = Observable.create(observer -> {
            Bitmap bm = downloadBitmap();
            return bm;
        });
        imageObservable.subscribe(image -> loadToImageView(image));
    But it will block the UI... 
             imageObservable
                .subscribeOn(Schedulers.newThread())
                .subscribe(image -> loadToImageView(image));
    But it will not invoke loadImage  on the right thread 
             imageObservable
                .subscribeOn(Schedulers.newThread())
                .observeOn(AndroidSchedulers.mainThread())
                .subscribe(image -> loadToImageView(image));
    Example taken from FRP on android: http://slides.com/yaroslavheriatovych/frponandroid/#/7/1

    Soluto

    • Different roles handle different step in message processing 
      • Listeners/Workers
    • Role to role communication is asynchronous
      • Using queue, low(er) level abstraction 
    • All task based code is internally event-driven (async-await)

    IObservable<T> and Task<T>

    Similarities
    • Can represent future event value.
    • Allows to consume value in non-blocking way.
    • Designed to help asynchronous services composition .
    • Have support for different synchronization context.
    • Help solving the "callback hell" problem

    IOBSERVABLE<T> AND TASK<T>



    Single return value Mutiple return values
    Pull/Synchronous T IEnumerable<T>
    Push/Asynchronous Task<T> IObservable<T>

    IOBSERVABLE<T> VS. TASK<T>

    CAN PLAY TOGETHER!
    • await can work on observable
    • Task.ToObservable -> convert Task to observable
    • Observable.ToTask -> convert observable to Task 
    •  public async Task<int> GetNumberFromServiceWithTimeout()
{

      return await someService.getNumber().ToObservable().Timeout(TimeSpan.FromSeconds(5))
      }



    Some examples in soluto

    TableDataContext.GetObservable() -> 
    Allow us to do a table scan asynchronously and continuesoly

    QueueDataContext.Read/ReadUntilEmpty() ->
    Allow us to read  queue asynchronously in a push style

    Worker.Create ->
    Built on top of QueueDataContext Rx Extensions,
    handle throughput (backpressure), monitoring and handling errors
    for queue worker. (still not complete tough...)

    AnaylyticsDispatcher ->
    Use RX IScheduler for creating dedicated dispatching loop for events
    while maintaining Unit Testability and decoupling for computation model.

    CacheConfigurationLoader -> 
    Get Observable in constructor injection which use for pushing invalidate cache messages.

    IVR Controller


     await queue.ReadUntilEmpty()
                    .Select(x => QueueTask.From(x, () => queue.DeleteMessage(x)))
                    .Select(x => x.AsString)
                    .SelectMany(x => mPiiStorageAccessor.GetPii<SomePiiData>(Guid.Parse(x)))
                    .Aggregate((prev, current) => prev + current)
    Code is actually different,  since we need to add some extensions that handle queue behavior. 
    (deleting message after processing, moving message to error queue, etc...)

    Worker example

     return Worker.Create(mQueue, mErrorQueue, MonitoringObserver))
                .Of(EntityTransformers.Container(mContainer))
                .ProcessMessage(async message =>
                {
                    await someHandler.ProcessMeessage(message);
                });
    Handling throughput and error handling

    Cache Configuration Loader

     public CachedConfigurationLoader(ILogger aLogger, IConfigurationLoader aLoader, IObservable<Unit> aInvalidate)
        {
            mLogger = aLogger;
            mLoader = aLoader;
            aInvalidate.Subscribe(
                x => mConfigurationSetByCategory.Clear(),
                ex => mLogger.Error(
                    "Configuration cache invalidator threw an exception - cache will not refresh", 
                    ex);
        }
    And in DI module
     var policy = Observable.Interval(TimeSpan.FromMinutes(5)).Select(_ => Unit.Default);

 return new CachedConfigurationLoader(ctx.Resolve<ILogger>(), loader, policy);

    Analytics Dispatcher


     public AnalyticsDispatcher(IAnalyticsEnqueuer analyticsEnqueuer, AnalyticsModelCreator modelCreator, IScheduler scheduler = null)
        {
            mAnalyticsEnqueuer = analyticsEnqueuer;
            mModelCreator = modelCreator;
            mScheduler = scheduler ?? TaskPoolScheduler.Default;
        }
    And in DI Module:
     builder.Register((ctx) => new EventLoopScheduler().Catch((Exception ex) =>
            {
                ctx.Resolve<ILogger>().Error("Unhandled error in AnalyticsDispatcher",ex,null,"AnaltyicsDispatcher");
                return true;
            }))
            .Keyed<IScheduler>(SCHEDULER_KEY)
            .SingleInstance();

    Where to use?

    For most async purposes, use Tasks by default!

    Use RX Observables for:
    • Time-based operations
    • Async operations that produce multiple values.
    • Ongoing push stream of data.
    • Non .net environments without async-await or/and inferior "Future<T>" implementations.
    • Better composition capabilities than offered by Task<T>
    • Complex concurrency control.
    • Lazy operations.
    • UI, when not using other data-binding libraries.

    IOBSERVABLE<T> VS. TASK<T>

    Tasks
    • Represent one future value.
    • Simpler for most cases.
    • Have better integration with async-await.
    • Represent a running task, completed or canceled task, never idle one.
    • Offer easier exception handling.

    IOBSERVABLE<T> VS. TASK<T>

    Observables
    • Can receive any number of future values.
    • Offer better composition and manipulation
      based on LINQ/FP operators.
    • More suited for time based operations -
      can replace all .net timers
    • Provide more powerful scheduling abstraction
    • Lazy by default     

    RX iS really big





    This was just intro


    RX is really big




    There are 100+ RX operators for observables,
     and there are many other concepts related.
    RX is almost a programming paradigm rather then just library.

    RX is really big




    It's well suited for polygot programming 
    due to it's wide language and platform support.

    RX is really big





    Everytime you need to do things related to time,
    concurrency, async processing, which feels too
    complex, give RX a spin, or at least 10 minutes.






    Questions?

    ADDITIONAL RESOURCES

    RX

    By yshayy

    Made with Slides.com

    RX

    Introduction to Reactive Extensions

    • 3,270

    More from yshayy