Black Magic

to Build Resilient

API Dependencies

Dolganov Sergey @ Evil Martians

Title Text

What's my problem?

eBay for Business (eBaymag)

Distributed system built upon lots of API calls to external systems, e.g.

eBay, UPS, Google Translate

Contract is:

— Pre-conditions (request)

— Post-conditions (response)

— Invariants (state)

Input

Output

function (input) { ... } # =>

State

Pre-conditions

Post-conditions

Invariants

Contract

Evolution

— Input/Output Control

— State Control

— Homogeneous Design

Problem solving

API Client Problem

Step One

Policy

eBay Orders,

eBaymag Parcels

Layout (for DHL)

Lots of validations

HTTP Client

Confusing

content

module DHL
  module API
    def get_tariff(mapped_request)
      sampler = TariffSampler.new
      sampler.register_request(mapped_request)
      response = http_client.get("/GetQuote", mapped_request.to_xml)
      sampler.register_response(response)
      sampler.tag!
      response
    end

    class TariffSampler < BaseSampler
      def tag!
        tag_success!
        tag_dhl_address_ambiguity_warning!
        tag_dhl_composite_call_warning!
        tag_data_invalid_input_error!
        tag_destination_invalid_for_dhl!
        tag_unexpected_behavior! # if still no tag
      end

      # ...
    end
  end
end

"Contract"

module DHL
  module Responses
    class TariffPolicy < Tram::Policy
      param :response
      param :mapped_response

      validate :no_response_errors, stop_on_failure: true
      validate :delivery_cost_presence
      validate :delivery_currency_presence
      validate :delivery_date_presence

      # implementation of methods
  end
end

        module DHL
          class Calculator < ::BaseService
            attr_reader :errors
            param :parcel

            def call
              policy = DHL::RegionalPolicy[parcel]
              return if policy.invalid?

              policy = DHL::Requests::TariffPolicy[parcel]
              return if policy.invalid?

              response = DHL::API.get_tariff(Requests::TariffMapper[parcel])
              mapped_response = Responses::TariffMapper[response]

              policy = DHL::Responses::TariffPolicy[response, mapped_response]
              return if policy.invalid?

              parcel.merge_update!(dhl: mapped_response)
            ensure
              @errors = policy&.errors 
            end
          end
        end

        if (service = DHL::Calculator.new(parcel)).call
          render json: { tariff: parcel.reload.dhl["tariff"] }
        else
          render json: { errors: { base: service.errors } }
        end

Dry::Initializer

+

Tram::Policy

+

BloodContracts

Order

Step Two

Sagas

Transactions without Transactions

Sagas (with Orchestration):

1. Wrap each mutation in a Command

(each could be done and undone, e.g. #up and #down in AR migration)

2. Chain Command

3. Execute chain

4. If one fails run `undo`

...

Failure

Validate

...

Success

Validate

Parcel

Parcel*

Saga Orchestrator

...

Success

Validate

PrintLabel

Validate#call

PrintLabel#call

...

Failure

Validate

PrintLabel

Validate#undo

PrintLabel#undo

Aaand that's it?!

DirtyPipeline

(Sagas)

Real World

What if?

Orchestrator runtime error
Application shuts down in the middle
Another pipeline runs in parallel

RussianPost Pipeline

def create
  account = Account.find(params[:account_id])
  parcel = account.parcels.find(params[:id]) 
  status = 
    RussianPost::Pipeline.chain(:Validate)
                         .chain(:GetTransportationCost)
                         .chain(:Register)
                         .chain(:PayOnline)
                         .chain(:PrintLabel)
                         .new(parcel)
                         .call

  render json: status.payload
end

...

Success

Action 1

Action N

Pipeline

(Action 1 Attempt)

(Action N Attempt)

Success

Failure

(Done)

Subject

call

Events

(Action 1 Attempt)

(Action N Attempt)

(Done)

Cleaner

Success

...

Success

Action 1

Action N

applied?

Events

...

Success

Action 1

Action N

Pipeline

(Action 1 Attempt)

(Action N Attempt)

Success

Failure

(Done)

Subject

call

Events

(Action 1 Attempt)

(Action N Attempt)

(Done)

Cleaner

Failure

...

Success

Action N

Action 1

undo

(reversed)

undo

(Action 1 Attempt)

(Action N Attempt)

(Done)

Failure

...

Success

Action N

Action 1

undo

Failure

(reversed)

lock Subject

with error

Cleaner

Events

Solution from eBaymag

— Pipeline

— Lock (interface)

— Action

— Events (interface)

— Clean

— Status

Check out in more details

https://slides.com/sergeydolganov/sagas-wizards-ruby

Step Three

Homogeneous Design

Functional Way to Validation

Compose

+

Errors Processing

Algebraic Data Types

module UPS
  module Rates
    # types product, 
    # Request is both RegionalParcel and Schema
    Request = RegionalParcel.and_then(Schema)
    
    # types sum, 
    # Error is RecoverableInputError or InvalidRequest
    Error = RecoverableInputError.or(InvalidRequest) 

    # combine
    Response = 
      BaseTypes::JSONResponse.and_then(Tariff.or(Error))
  end
end

Refinement Types

— Type (ADT)

— Predicate

{ n :: Integer | n > 5 }

Context:

"Why did validation fail?"

module BaseTypes
  class JSONResponse < BaseType
    alias :response :value

    def match
      context[:body] = response.body
      context[:parsed] = ::JSON.parse(response.body)
      self
    rescue StandardError => error
      ContractFailure.new(error) # is BaseType, too
    end

    def unpack
      match.context[:parsed]
    end
  end
end

Refinement Type

        class UPS::RatesContract < BaseContract # sampling by Sniffer & types
          def match(*input)
             input_match = Request.match(*input) 
             return input_match if input_match.invalid?
           
             result = yield(input_match)
             Response.match(result)
          end
        end

        class UPS::API
          def rates_request(parcel)
            RatesContract.match(parcel) do |mapped_parcel|
              http_client.get('/1.0/Rates', mapped_parcel.unpack)
            end # returns a typed response
          end
        end

        case (match = UPS::API.rate_request(parcel))
        when ContractFailure
          Honeybadger.notify "Unexpected UPS behavior", context: match.context
          render json: { errors: "Unprocessable now, try later" }
        when Tariff
           render json: { tariff: match.unpack }
        when RecoverableInputError
          render json: { errors: match.unpack }
        when InvalidRequest
          Honeybadger.notify "Invalid UPS request", context: match.context
          render json: { errors: "Unprocessable now, try later" }
        else
          raise "Invalid contract processing"
        end

        class UPS::RatesContract < BaseContract # sampling by types
          def match(*input)
             input_match = Request.match(*input) 
             return input_match if input_match.invalid?
           
             result = yield(input_match)
             Response.match(result)
          end
        end

        class UPS::API
          def rates_request(parcel)
            RatesContract.match(parcel) do |mapped_parcel|
              http_client.get('/1.0/Rates', mapped_parcel.unpack)
            end # returns a typed response
          end
        end

        case (match = UPS::API.rate_request(parcel))
        when ContractFailure
          Honeybadger.notify "Unexpected UPS behavior", context: match.context
          render json: { errors: "Unprocessable now, try later" }
        when Tariff
           render json: { tariff: match.unpack }
        when RecoverableInputError
          render json: { errors: match.unpack }
        when InvalidRequest
          Honeybadger.notify "Invalid UPS request", context: match.context
          render json: { errors: "Unprocessable now, try later" }
        else
          raise "Invalid contract processing"
        end

        class UPS::RatesContract < BaseContract # sampling by types
          def match(*input)
             input_match = Request.match(*input) 
             return input_match if input_match.invalid?
           
             result = yield(input_match)
             Response.match(result)
          end
        end

        class UPS::API
          def rates_request(parcel)
            RatesContract.match(parcel) do |mapped_parcel|
              http_client.get('/1.0/Rates', mapped_parcel.unpack)
            end # returns a typed response
          end
        end

        case (match = UPS::API.rate_request(parcel))
        when ContractFailure
          Honeybadger.notify "Unexpected UPS behavior", context: match.context
          render json: { errors: "Unprocessable now, try later" }
        when Tariff
           render json: { tariff: match.unpack }
        when RecoverableInputError
          render json: { errors: match.unpack }
        when InvalidRequest
          Honeybadger.notify "Invalid UPS request", context: match.context
          render json: { errors: "Unprocessable now, try later" }
        else
          raise "Invalid contract processing"
        end

        class UPS::RatesContract < BaseContract # sampling by types
          def match(*input)
             input_match = Request.match(*input) 
             return input_match if input_match.invalid?
           
             result = yield(input_match)
             Response.match(result)
          end
        end

        class UPS::API
          def rates_request(parcel)
            RatesContract.match(parcel) do |mapped_parcel|
              http_client.get('/1.0/Rates', mapped_parcel.unpack)
            end # returns a typed response
          end
        end

        case (match = UPS::API.rate_request(parcel))
        when ContractFailure
          Honeybadger.notify "Unexpected UPS behavior", context: match.context
          render json: { errors: "Unprocessable now, try later" }
        when Tariff
           render json: { tariff: match.unpack }
        when RecoverableInputError
          render json: { errors: match.unpack }
        when InvalidRequest
          Honeybadger.notify "Invalid UPS request", context: match.context
          render json: { errors: "Unprocessable now, try later" }
        else
          raise "Invalid contract processing"
        end

Sniffer

+

BloodContracts::Core

(Refinement Types)

Outcome

New Horisons

Evolution

+ Input/Output Control

+ State Control

+ Homogeneous Validations

Leftovers

— DirtyPipeline is not working

(valid implementation of Saga is not in gem yet)

— BloodContracts refactoring

(with Refinement Types)

— Category Theory for Programmers

— Maybe Haskell

— Chaos Engineering

Extra Reading

Thanks!

Our Blog: evl.ms/chronicles

Blood Contracts: sclinede/blood_contracts-core

Dirty Pipeline: bit.ly/DP_RUBY_BY19

@ss_dolganov

@sclinede

@evilmartians

Black Magic to Build Resilient API Dependencies

By Sergey Dolganov

Black Magic to Build Resilient API Dependencies

Imagine that you want to build a system which depends on external service, e.g., logistics, payments or notifications service. Those systems have its life-cycle which you have to be in sync with. I’ll share how to treat issues you could face, using the examples of DHL, UPS, Russian Post integrations.

Sergey Dolganov

Evil Martians developer, open-source enthusiast, traveller and drummer.