An Example

Projects indexed in ES

Project's Progress is a real time analytics in MongoDB

Query:
Sort "OPEN" "substrate" projects by "progress"

10M Projects
100K Tickets

Basic Example

• val sourceOne = Source(List(1,3,5,7))
  val sourceTwo = Source(List(0,2,4,6))
  
  val merged = Source.combine(sourceOne, sourceTwo)(Merge(_))
  
  val mergedResult: Future[Int] = merged.runWith(Sink.fold(0)(_ + _)
  

Async Calls usually results in "Future"

• def send(email: Email): Future[SMTPResponse] = {
    // Create a connection to SMTP Agent
  ​  // Start SMTP Dialog
  ​  // Handle Exceptions
  ​  // Release resources
  }

• def getEmail(id): Future[Email] = {
    // open a connection to DB
  ​  // Run transaction
  ​  // close the connection
  ​  // log
  }

Buffering problem

List(1 to 100).map(getEmail).map(send)

send is slow
getEmail is fast
 

Problems with chaining

• Complicated debugging
• Lack of parallelism
• Limited Failure Handling
• non-flexible Linear chains
• Buffering problem
• Shared state synchronization

Streams are like gears

Q: How to manage concurrency in ROF programs?

• A: Akka Streaming

Akka Streaming
A Framework

• Concurrency framework
• The backbone for ROF programs in Scala
• State full Stages or stateless stages
• Controllable Parallelism
• No Buffering problem using back pressure
• Extendable
  • By Plugins
• DSL
  • Allows making a blueprint of the app
  • Compose-able stages
• Configurable failure strategy
  • Retries, A/B Testing

Failure Strategy

• val decider: Supervision.Decider = {
    case _: NonFatalException => Supervision.Resume
    case _ => Supervision.Stop
  }

Depending on the Use Case the right Framework should be used

Data Sets at least in Two Machines to run Map/Reduce on

Spectrum

basically to Create Actors

Neutron Timer Service

import scala.concurrent.Future

val futureFunction = Future {

  normalFunction()
}

When you are not sure to use future or not! I recommend please do use, it's very safe

or

When I have IO, (using "blocking" keyword)

Title Text

Subtitle

When and Why to use Futures

• Always
  • For IO tasks
  • Tasks taking more than 10 milliseconds
  • Computations

• Parallelism
• Non Blocking