Black Magic

to Build Resilient

API Dependencies

Dolganov Sergey @ Evil Martians

Title Text

What's my problem?

eBay for Business (eBaymag)

Distributed system built upon lots of API calls to external systems, e.g.

eBay, UPS, Google Translate

Contract is:

— Pre-conditions (request)

— Post-conditions (response)

— Invariants (state)

Input

Output

function (input) { ... } # =>

State

Pre-conditions

Post-conditions

Invariants

Contract

Evolution

 

— Input/Output Control

State Control

Homogeneous Design

Problem solving

API Client Problem

Step One

Policy

eBay Orders,

eBaymag Parcels

Layout (for DHL)

Lots of validations

HTTP Client

Confusing

content

module DHL
  module API
    def get_tariff(mapped_request)
      sampler = TariffSampler.new
      sampler.register_request(mapped_request)
      response = http_client.get("/GetQuote", mapped_request.to_xml)
      sampler.register_response(response)
      sampler.tag!
      response
    end

    class TariffSampler < BaseSampler
      def tag!
        tag_success!
        tag_dhl_address_ambiguity_warning!
        tag_dhl_composite_call_warning!
        tag_data_invalid_input_error!
        tag_destination_invalid_for_dhl!
        tag_unexpected_behavior! # if still no tag
      end

      # ...
    end
  end
end

 "Contract"

module DHL
  module Responses
    class TariffPolicy < Tram::Policy
      param :response
      param :mapped_response

      validate :no_response_errors, stop_on_failure: true
      validate :delivery_cost_presence
      validate :delivery_currency_presence
      validate :delivery_date_presence

      # implementation of methods
  end
end
        module DHL
          class Calculator < ::BaseService
            attr_reader :errors
            param :parcel

            def call
              policy = DHL::RegionalPolicy[parcel]
              return if policy.invalid?

              policy = DHL::Requests::TariffPolicy[parcel]
              return if policy.invalid?

              response = DHL::API.get_tariff(Requests::TariffMapper[parcel])
              mapped_response = Responses::TariffMapper[response]

              policy = DHL::Responses::TariffPolicy[response, mapped_response]
              return if policy.invalid?

              parcel.merge_update!(dhl: mapped_response)
            ensure
              @errors = policy&.errors 
            end
          end
        end

        if (service = DHL::Calculator.new(parcel)).call
          render json: { tariff: parcel.reload.dhl["tariff"] }
        else
          render json: { errors: { base: service.errors } }
        end

Dry::Initializer

+

Tram::Policy

+

BloodContracts

Order

Order

Step Two

Sagas

 

Transactions without Transactions

Sagas (with Orchestration):

1. Wrap each mutation in a Command

(each could be done and undone, e.g. #up and #down in AR migration)

2. Chain Command

3. Execute chain

4. If one fails run `undo`

...

Failure

Failure

Validate

Print

...

Success

Success

Validate

Print

Parcel

Parcel*

Saga Orchestrator

...

Success

Success

Validate

PrintLabel

Validate#call

PrintLabel#call

...

Failure

Failure

Validate

PrintLabel

Validate#undo

PrintLabel#undo

Aaand that's it?!

DirtyPipeline

(Sagas)

Real World

What if?

  • Orchestrator runtime error
  • Application shuts down in the middle
  • Another pipeline runs in parallel

RussianPost Pipeline

def create
  account = Account.find(params[:account_id])
  parcel = account.parcels.find(params[:id]) 
  status = 
    RussianPost::Pipeline.chain(:Validate)
                         .chain(:GetTransportationCost)
                         .chain(:Register)
                         .chain(:PayOnline)
                         .chain(:PrintLabel)
                         .new(parcel)
                         .call

  render json: status.payload
end

...

Success

Success

Action 1

Action N

Pipeline

(Action 1 Attempt)

(Action N Attempt)

Success

Failure

(Done)

Subject

call

call

Events

(Action 1 Attempt)

(Action N Attempt)

(Done)

Cleaner

1

2

Success

...

Success

Success

Action 1

Action N

applied?

applied?

Events

...

Success

Success

Action 1

Action N

Pipeline

(Action 1 Attempt)

(Action N Attempt)

Success

Failure

(Done)

Subject

call

Events

Events

(Action 1 Attempt)

(Action N Attempt)

(Done)

Cleaner

1

2

Failure

...

Success

Success

Action N

Action 1

undo

(reversed)

undo

(Action 1 Attempt)

(Action N Attempt)

(Done)

1

2

Failure

...

Success

Success

Action N

Action 1

undo

Failure

(reversed)

 

lock Subject

with error

Cleaner

Events

Solution from eBaymag

— Pipeline

— Lock (interface)

— Action

— Events (interface)

— Clean

— Status

Step Three

Homogeneous Design

 

Functional Way to Validation

Compose

+

Errors Processing

Algebraic Data Types

module UPS
  module Rates
    # types product, 
    # Request is both RegionalParcel and Schema
    Request = RegionalParcel.and_then(Schema)
    
    # types sum, 
    # Error is RecoverableInputError or InvalidRequest
    Error = RecoverableInputError.or(InvalidRequest) 

    # combine
    Response = 
      BaseTypes::JSONResponse.and_then(Tariff.or(Error))
  end
end

Refinement Types

— Type (ADT)

— Predicate

{ n :: Integer | n > 5 }

Context:

"Why did validation fail?"

module BaseTypes
  class JSONResponse < BaseType
    alias :response :value

    def match
      context[:body] = response.body
      context[:parsed] = ::JSON.parse(response.body)
      self
    rescue StandardError => error
      ContractFailure.new(error) # is BaseType, too
    end

    def unpack
      match.context[:parsed]
    end
  end
end

Refinement Type

        class UPS::RatesContract < BaseContract # sampling by Sniffer & types
          def match(*input)
             input_match = Request.match(*input) 
             return input_match if input_match.invalid?
           
             result = yield(input_match)
             Response.match(result)
          end
        end
        class UPS::API
          def rates_request(parcel)
            RatesContract.match(parcel) do |mapped_parcel|
              http_client.get('/1.0/Rates', mapped_parcel.unpack)
            end # returns a typed response
          end
        end
        case (match = UPS::API.rate_request(parcel))
        when ContractFailure
          Honeybadger.notify "Unexpected UPS behavior", context: match.context
          render json: { errors: "Unprocessable now, try later" }
        when Tariff
           render json: { tariff: match.unpack }
        when RecoverableInputError
          render json: { errors: match.unpack }
        when InvalidRequest
          Honeybadger.notify "Invalid UPS request", context: match.context
          render json: { errors: "Unprocessable now, try later" }
        else
          raise "Invalid contract processing"
        end
        class UPS::RatesContract < BaseContract # sampling by types
          def match(*input)
             input_match = Request.match(*input) 
             return input_match if input_match.invalid?
           
             result = yield(input_match)
             Response.match(result)
          end
        end
        class UPS::API
          def rates_request(parcel)
            RatesContract.match(parcel) do |mapped_parcel|
              http_client.get('/1.0/Rates', mapped_parcel.unpack)
            end # returns a typed response
          end
        end
        case (match = UPS::API.rate_request(parcel))
        when ContractFailure
          Honeybadger.notify "Unexpected UPS behavior", context: match.context
          render json: { errors: "Unprocessable now, try later" }
        when Tariff
           render json: { tariff: match.unpack }
        when RecoverableInputError
          render json: { errors: match.unpack }
        when InvalidRequest
          Honeybadger.notify "Invalid UPS request", context: match.context
          render json: { errors: "Unprocessable now, try later" }
        else
          raise "Invalid contract processing"
        end
        class UPS::RatesContract < BaseContract # sampling by types
          def match(*input)
             input_match = Request.match(*input) 
             return input_match if input_match.invalid?
           
             result = yield(input_match)
             Response.match(result)
          end
        end
        class UPS::API
          def rates_request(parcel)
            RatesContract.match(parcel) do |mapped_parcel|
              http_client.get('/1.0/Rates', mapped_parcel.unpack)
            end # returns a typed response
          end
        end
        case (match = UPS::API.rate_request(parcel))
        when ContractFailure
          Honeybadger.notify "Unexpected UPS behavior", context: match.context
          render json: { errors: "Unprocessable now, try later" }
        when Tariff
           render json: { tariff: match.unpack }
        when RecoverableInputError
          render json: { errors: match.unpack }
        when InvalidRequest
          Honeybadger.notify "Invalid UPS request", context: match.context
          render json: { errors: "Unprocessable now, try later" }
        else
          raise "Invalid contract processing"
        end
        class UPS::RatesContract < BaseContract # sampling by types
          def match(*input)
             input_match = Request.match(*input) 
             return input_match if input_match.invalid?
           
             result = yield(input_match)
             Response.match(result)
          end
        end
        class UPS::API
          def rates_request(parcel)
            RatesContract.match(parcel) do |mapped_parcel|
              http_client.get('/1.0/Rates', mapped_parcel.unpack)
            end # returns a typed response
          end
        end
        case (match = UPS::API.rate_request(parcel))
        when ContractFailure
          Honeybadger.notify "Unexpected UPS behavior", context: match.context
          render json: { errors: "Unprocessable now, try later" }
        when Tariff
           render json: { tariff: match.unpack }
        when RecoverableInputError
          render json: { errors: match.unpack }
        when InvalidRequest
          Honeybadger.notify "Invalid UPS request", context: match.context
          render json: { errors: "Unprocessable now, try later" }
        else
          raise "Invalid contract processing"
        end

Sniffer

+

BloodContracts::Core

(Refinement Types)

Outcome

New Horisons

 

Evolution

 

+ Input/Output Control

+ State Control

+ Homogeneous Validations 

 

Leftovers

 

DirtyPipeline is not working

       (valid implementation of Saga is not in gem yet)

 

BloodContracts refactoring

      (with Refinement Types)

 

— Category Theory for Programmers

— Maybe Haskell

— Chaos Engineering

 

Extra Reading

Thanks!

Our Blog: evl.ms/chronicles

Blood Contracts: sclinede/blood_contracts-core

Dirty Pipeline: bit.ly/DP_RUBY_BY19

Made with Slides.com