DM Processor

Case of Study

REPLACING

Spring Integration

Spring Integration

• Message Class
  • Overusing CPU resources (Generating UUID)
  • Impossible to extend the class (Old version 2.2.6)
• Spring Chain
  • Every chain declared in a XML file
  • NO Unit Tests of the chain
  • Depends on Spring core
• Scalability
  • Blocking solution
  • Running on a single JVM
  • Non-able to scale out

Introducing Akka Streams

Why?

• Reactive solution
  • Async
  • Non-Blocking
  • Resilient
  • Message Driven
  • Back-pressure
• Compatible with the current DM technology stack
• Part of the standard Reactive Streams definition
• Easy to test every single flow
• Faster than Spring Integration (~50%)

Introducing Akka Streams

http://www.reactivemanifesto.org/

Responsive

Message Driven

Elastic

Resilient

Introducing Akka Streams

Stream Composition: Single Shapes

Introducing Akka Streams

Stream Composition: Composite Shapes

Introducing Akka Streams

Stream Composition: Partial Flow Graphs

Introducing Akka Streams

Event Inbound Flow

Introducing Akka Streams

<int:channel id="eventInboundChannel" />

<int:chain input-channel="eventInboundChannel" output-channel="eventPreProcessingChannel">
    <int:transformer ref="eventHeaderEnricher" method="applyHeaders"/>
    <int:filter ref="payloadValidationFilter" method="accept" discard-channel="validationFailureSuppressedChannel"/>
</int:chain>

<int:channel id="validationFailureSuppressedChannel" />

<int:chain auto-startup="true" input-channel="validationFailureSuppressedChannel" output-channel="eventSuppressedChannel">
    <int:header-enricher>
        <int:header name="suppressedReason" value="ValidationFailure"/>
    </int:header-enricher>
</int:chain>

<int:channel id="eventPreProcessingChannel" />

<int:header-value-router input-channel="eventPreProcessingChannel" header-name="eventRefreshRequest"
                         default-output-channel="eventProcessingChannel">
    <int:mapping value="true" channel="eventRefreshChannel"/>
</int:header-value-router>

Event Inbound Channel defined using Spring

Introducing Akka Streams

trait EventInboundFlow extends ValidationFailureSuppressedFlow with EventSuppressedPublisherFlow {

  this: Gateway with EventPipelineBeans =>

  private val payloadFilterFlow = filterPartialFlowGraph(payloadValidationFilter.accept(_))

  private val eventRefreshFilterFlow = filterPartialFlowGraph(_.isContained(MessageUtil.EVENT_REFRESH_REQUEST_HEADER))

  lazy val eventInboundFlow = FlowGraph.partial() { implicit b =>
    val headerEnricherFlow = b.add(partialFlow(eventHeaderEnricher.applyHeaders(_)))
    val payloadFilter = b.add(payloadFilterFlow)
    val erf = b.add(eventRefreshFilterFlow)
    val validationSuppressed = b.add(validationFailureSuppressedFlow)

    headerEnricherFlow ~> payloadFilter
                          payloadFilter.out(0) ~> erf
                          payloadFilter.out(1) ~> validationSuppressed

    UniformFanOutShape(headerEnricherFlow.inlet, erf.out(0), erf.out(1), validationSuppressed.outlet)
  }.named("eventInboundFlow")

}

Event Inbound Flow: Coding

Introducing Akka Streams

trait EventInboundFlow extends ValidationFailureSuppressedFlow with EventSuppressedPublisherFlow {

  this: Gateway with EventPipelineBeans =>

  private val payloadFilterFlow = filterPartialFlowGraph(payloadValidationFilter.accept(_))

  private val eventRefreshFilterFlow = filterPartialFlowGraph(_.isContained(MessageUtil.EVENT_REFRESH_REQUEST_HEADER))

  lazy val eventInboundFlow = FlowGraph.partial() { implicit b =>
    val headerEnricherFlow = b.add(partialFlow(eventHeaderEnricher.applyHeaders(_)))
    val payloadFilter = b.add(payloadFilterFlow)
    val erf = b.add(eventRefreshFilterFlow)
    val validationSuppressed = b.add(validationFailureSuppressedFlow)

    headerEnricherFlow ~> payloadFilter
                          payloadFilter.out(0) ~> erf
                          payloadFilter.out(1) ~> validationSuppressed

    UniformFanOutShape(headerEnricherFlow.inlet, erf.out(0), erf.out(1), validationSuppressed.outlet)
  }.named("eventInboundFlow")

}

Introducing Akka Streams

"Event Inbound Flow" should {

    val eventRefreshHeaders = Map(MessageUtil.EVENT_REFRESH_REQUEST_HEADER -> Boolean.box(true))

    "Have every message with the header 'eventRefreshRequest' and value 'true' in filterOut and empty stream in notFilterOut" in withMessageSource(eventRefreshHeaders) { source => messages =>

      val (filterOut, notFilterOut, suppressedOut) = flowGraph(source)

      val filterResult = Await.result(filterOut, 1000.millis)

      // Should be an Empty stream
      intercept[NoSuchElementException] {
        Await.result(notFilterOut, 1000.millis)
      }

      intercept[NoSuchElementException] {
        Await.result(suppressedOut, 1000.millis)
      }

      filterResult.getHeaders should (contain key (MessageUtil.EVENT_REFRESH_REQUEST_HEADER) and contain value (true))
    }

}

Event Inbound Flow: Unit Test

BENCHMARKING

CONCLUSION