Dolganov Sergey @ Evil Martians

Railways, States & Sagas: pure Ruby for Wizards

Sagas Wizards Ruby

What's my problem?

eBay for Business (eBaymag)

Distributed system built upon lots of API calls to external systems, e.g.

eBay, Russian Post, Google Translate

eBaymag Logistics

Order

Order

Order

Parcel Pipeline

def create
  account = Account.find(params[:account_id])
  parcel = account.parcels.find(params[:id]) 
  provider = params[:provider] || "DHL"
  status = 
    Logistics::Pipeline.chain(:Validate)
                       .chain(:GetTransportationCost)
                       .chain(:Register)
                       .chain(:PayOnline)
                       .chain(:PrintLabel)
                       .new(parcel, provider: provider)
                       .call

  render json: status.payload
end

in single database

we're able to rollback sequence of changes with a transaction,

but...

we have separate databases, so

is there a way to have distributed transaction?

Transactions without Transactions

Sagas (with Orchestration):

1. Wrap each mutation in a Command

(each could be done and undone, e.g. #up and #down in AR migration)

3. Chain Commands

4. Execute chain

5. If one fails run `undo`

...

Success

Success

Validate

PrintLabel

Validate#call

PrintLabel#call

...

Failure

Failure

Validate

PrintLabel

Validate#undo

PrintLabel#undo

...

Failure

Failure

Validate

Print

...

Success

Success

Validate

Print

Saga Orchestrator

Parcel

Parcel*

Aaand that's it?!

Real World

What if?

  • Orchestrator runtime error
  • Application shuts down in the middle
  • Another pipeline runs in parallel

Solution from eBaymag

Parcel Pipeline

def create
  account = Account.find(params[:account_id])
  parcel = account.parcels.find(params[:id]) 
  provider = params[:provider] || "DHL"
  status = 
    Logistics::Pipeline.chain(:Validate)
                       .chain(:GetTransportationCost)
                       .chain(:Register)
                       .chain(:PayOnline)
                       .chain(:PrintLabel)
                       .new(parcel, provider: provider)
                       .call

  render json: status.payload
end

...

Success

Success

Action 1

Action N

Pipeline

(Action 1 Attempt)

(Action N Attempt)

Success

Failure

(Done)

Subject

call

call

Events

(Action 1 Attempt)

(Action N Attempt)

(Done)

Cleaner

1

2

Success

...

Success

Success

Action 1

Action N

applied?

applied?

Events

...

Success

Success

Action 1

Action N

Pipeline

(Action 1 Attempt)

(Action N Attempt)

Success

Failure

(Done)

Subject

call

Events

Events

(Action 1 Attempt)

(Action N Attempt)

(Done)

Cleaner

1

2

Failure

...

Success

Success

Action N

Action 1

undo

(reversed)

undo

(Action 1 Attempt)

(Action N Attempt)

(Done)

1

2

Failure

...

Success

Success

Action N

Action 1

undo

Failure

(reversed)

 

lock Subject

with error

Cleaner

Events

Solution from eBaymag

  • Pipeline
  • Lock
  • Action
  • Events
  • Clean
  • Status

Parcel Pipeline

def create
  account = Account.find(params[:account_id])
  parcel = account.parcels.find(params[:id]) 
  provider = params[:provider] || "DHL"
  status = 
    Logistics::Pipeline.chain(:Validate)
                       .chain(:GetTransportationCost)
                       .chain(:Register)
                       .chain(:PayOnline)
                       .chain(:PrintLabel)
                       .new(parcel, provider: provider)
                       .call

  render json: status.payload
end

Parcel Action

module Logistics
  class Pipeline
    class GetTransportationCost < DirtyPipeline::Action
      alias :parcel :subject
      policy contents_completeness: ReadyForTariffRequestPolicy

      def call
        validate!(contents_completeness: parcel)
        response = validate_response!(api_client.tariff(parcel: parcel))
        Success(response.mapped)
      end

      def undo
        Success(CLEAR_TARIFF)
      end

      def applied?
        parcel.tariffs[context["provider"]].values.presence&.all?
      end
    end
  end
end

Pure Ruby Pipeline

class DirtyPipeline::Base
  class << self
    # ...
  end

  attr_reader :subject, :context, :status
  def initialize(subject, context: nil, **opts)
    @subject = subject
    @status = Success.new
    @context = (context || Hash.new).merge(opts)
  end

  def call(**context)
    Lock.new(self.class.root, @subject).lock! do
      shared_context = @context.merge(context.stringify_keys)

      self.class.steps.each do |action|
        @status = action.call(@subject, context: shared_context).status
        break if @status.failure?
      end

      done = Events.publish!(Events.generate(self))
      clean = Clean.new(done).tap(&:call)
      @status.merge(clean.status)
    end
  end
end

Pure Ruby Pipeline

class DirtyPipeline::Base
  class << self
    attr_writer :root
    def root
      @root || self
    end

    def inherited(new_klass)
      new_klass.instance_variable_set(:@steps, [])
      new_klass.root = self.root
    end

    def chain(klass)
      c = Class.new(Base)
      c.instance_variable_set(:@steps, self.steps.to_a + [klass])
      c
    end
  end
end

Pure Ruby Lock

class Pipelines::Lock
  def initialize(pipeline)
    @pipeline = pipeline
  end

  def self.fail!(subject, errors); end
  def failed?; end
  def locked?; end
  def with_lock(&block); end

  def lock!
    return manual_failure if failed?
    return locked_failure if locked?
    with_lock(&block)
  end

  def manual_failure
    Failure.new(:needs_manual_recovery)
  end

  def locked_failure
    Failure.new(:pipeline_locked)
  end
end

class DirtyPipelines::Action
  module WrappedCall
    # ...
  end

  class << self
    # ...

    def call(subject = nil, context: nil, **opts)
      new(subject, context: context, **opts).tap(&:call)
    end
  end

  attr_reader :subject, :status, :context
  def initialize(subject, context: nil, **opts)
    @context = (context || Hash.new).merge(opts)
    @subject = subject 
    @subject ||= self.class.find_subject(**context.symbolize_keys)
    @status = Success.new
  end

  def call; end
  def undo; end

  def applied?
    true
  end
end

Pure Ruby Action

Pure Ruby Action

class DirtyPipelines::Action
  # ...
  class << self
    def inherited(child_klass)
      child_klass.prepend(WrappedCall)
    end

    def from_event(event)
      event = Events.unpack(event)
      event_as_kwargs = event.symbolize_keys
      event_as_kwargs.merge!(subject: find_subject(**event_as_kwargs)
      build_action(**event_as_kwargs)
    end

    def build_action(action_name:, context:, subject:, **)
      Kernel.const_get(action_name).new(subject, context: context.to_h)
    end

    def find_subject(subject_name:, subject_id:, **)
      Kernel.const_get(subject_name).find(subject_id)
    end
  end
end

Pure Ruby Action

class DirtyPipelines::Action
  module WrappedCall
    def call
      Events.publish! Events.generate(self)
      super
    end
  end

  class << self
    # ...
  end
end

Pure Ruby Events

class DirtyPipeline::Events
  # unpack Event's payload to a hash
  def unpack(event)
  end

  # mark Event as processed
  def complete!(event)
  end

  # read Events for a pipeline with subject
  def select_for(pipeline, subject)
  end

  # built an Event
  def generate(action_or_pipeline)
  end

  # publish freshly built Event to the Events stream
  def publish!(event)
  end
end

Pure Ruby Clean


  class DirtyPipeline::Clean
    def initialize(done_event)
      @event = done_event
    end

    def call
      status = @event.call_success ? validate : undo
      return status if status.success?

      Lock.fail!(@subject, status.errors)
      status
    end

    private

    # ...

    def pipeline_events
      Events.select_for(@pipeline, @subject)
    end
  end

Pure Ruby Clean


  class  DirtyPipeline::Clean
    # ...

    def undo
      pipeline_events.reverse_each do |event|
        action = Action.from_event(event).tap(&:undo)
        return undo_failure(action) if action.status.failure?
        Events.complete!(event)
      end
      Success.new
    end

    def undo_failure(action)
      Failure.new(message: :undo_failed, subject: action.subject)
             .merge(action.status)
    end
  end

Pure Ruby Clean


  class DirtyPipeline::Clean
    # ...

    def validate
      pipeline_events.each do |event|
        action = Action.from_event(event)
        return validation_failure(action) unless action.applied?
        Events.complete!(event)
      end
      Success.new
    end

    def validation_failure(action)
      Failure.new(message: :broken_state, subject: action.subject)
    end
  end


class Status::Base
  attr_reader :payload
  alias :data :payload
  def initialize(payload = nil)
    @payload = payload || Hash.new
  end

  def merge(other_status)
    raise ArgumentError unless Base === other_status
    klass = Failure if other_status.failure? || failure?
    klass ||= Success
    klass.new(payload.merge(other_status.payload))
  end

  def failure?
    raise NotImplementedError
  end

  def success?
    !failure?
  end
end

Pure Ruby Status

class Failure < Status::Base
  alias :errors :payload
  def failure?
    true
  end
end

class Success < Status::Base
  def errors
    {}
  end

  def failure?
    false
  end
end

We won the consistency battle!

But...

What's about the war?

Thanks

Twitter: @ss_dolganov

GitHub: @sclinede

 

Our Blog:  evl.ms/chronicles

Pipelines Gist: bit.ly/DP_RUBY_BY19

Sagas. Wizards. Ruby.

By Sergey Dolganov

Made with Slides.com

Sagas. Wizards. Ruby.

Have you ever heard about data consistency problems within the multi-service architecture? That problem could be solved using Sagas design pattern. In fact, it’s quite hard to implement. Want to know how? Welcome, on board, young magicians! Recording is available: https://youtu.be/Ku-IqG4X3q4

  • 1,109

More from Sergey Dolganov