ElixirConf Roundup, Circuit Breaker Patterns

Circuit Breaker Pattern in Elixir

What is the circuit breaker pattern?

A design pattern for detecting and encapsulating failures when invoking a remote system.

Prevents failures from recurring continuously.

Provides a mechanism for reintroducing the (potentially) faulty call back into the execution flow.

http://martinfowler.com/bliki/CircuitBreaker.html

Introducing :fuse

• An Erlang implementation of the circuit breaker pattern - https://github.com/jlouis/fuse
• Super simple/clean/intuitive API.
• Alternatives:
  • Breaky - https://github.com/mmzeeman/breaky
  • circuit_breaker - https://github.com/klarna/circuit_breaker

https://github.com/jlouis/fuse

The :fuse API

• install/2 -- fuse_name, Opts
  • ​Opts -- {{type, attempts, window}, (:reset, time}
• ask/2 -- fuse_name, (:sync | :async_dirty)
• melt/1 -- fuse_name 
• reset/1 -- fuse_name
• run/1 -- fuse_name, function, context

Installation

# apps/twitter/lib/twitter.ex start/2

:fuse.install(
               "twitter_api_fuse", 
               {
                    {:standard, 1, 60_000},
                    #standard fuses (versus :fault_injection). Tolerate one failure/minute

                    {:reset, 900_000}
                    #reset the fuse in 15 minutes after it's :blown
                }
              )

Usage

# apps/twitter/lib/twitter_search.ex

def search(query) do
    case :fuse.ask("twitter_api_fuse", :sync) do       # ask if our fuse is blown
      :ok ->                                           # if we are good...
        ExTwitterBreaker.search(query)                 # wrapper around ExTwitter API
      :blown ->                                        # if we are blown... 
        get_tweet_cache(query)                         # get the results from cache.
    end
  end

Usage con't

# apps/twitter/lib/twitter_search.ex

def search(query) do
  case :fuse.ask("twitter_api_fuse", :sync) do       # ask if our fuse is blown
    :ok ->                                           # if we are good...
      ExTwitterBreaker.search(query)                 # wrapper around ExTwitter API       
      |> parse_tweets                                # parse the results
    :blown ->                                        # if we are blown... 
      get_tweet_cache(query)                             # get the results from cache.
  end

  defp parse_tweets({:ok, tweets}) do
    update_tweet_cache(query)
    Enum.map(tweets, fn(tweet) ->
      result["text"]
    end)
  end

  defp parse_tweets({:error, results}) do
   :fuse.melt("twitter_api_fuse")
   log_error(results)
   get_tweet_cache(query)
  end
end

Other things...

• Can be used to introduce service failures.
• No UI to see the state of your fuses (that I know of).
• Uses ETS.
• Observer bias?
• Other use cases?

Thanks!

Thanks to Mitchell Henke (@MitchellHenke) for the inspiration.

Chris Steinmeyer

@chrisfishwood