Glass Octopus

A Kafka consumer framework. Like Rack but for Kafka.

Secret Sauce Partners

http://www.secretsaucepartners.com

What is Rack?

Rack provides a minimal interface between webservers that support Ruby and Ruby frameworks.

require 'rack'
 
app = Proc.new do |env|
    ['200', {'Content-Type' => 'text/html'}, ['Hello budapest.rb!']]
end
 
Rack::Handler::WEBrick.run app

http://www.rubydoc.info/github/rack/rack/master/file/SPEC

• Phusion Passenger
• Puma
• Thin
• Tiny
• Unicorn
• WEBrick

• Camping
• Merb
• Padrino
• Rails
• Rum
• Sinatra

What is Rack?

Rack provides a minimal interface between webservers that support Ruby and Ruby frameworks.

require 'rack/lobster'

use Rack::CommonLogger
use Rack::Lint
use Rack::ShowExceptions

run Rack::Lobster.new

Middlewares

• Rack::CommonLogger
• Rack::ShowExceptions
• Rack::Lint

Application

• Rack::Lobster

What is Rack?

Rack provides a minimal interface between webservers that support Ruby and Ruby frameworks.

require 'rack'
require 'rack/lobster'

app = Rack::Builder.new do
  use Rack::CommonLogger
  use Rack::ShowExceptions

  map "/lobster" do
    use Rack::Lint
    run Rack::Lobster.new
  end
end

Rack::Handler::WEBrick.run app

Middlewares

• Rack::CommonLogger
• Rack::ShowExceptions
• Rack::Lint

Application(s)

• Rack::Lobster

What is Glass Octopus?

Glass Octopus provides a minimal interface drivers that support Kafka and Kafka consumers.

require "glass_octopus"

app = GlassOctopus.build do
  use GlassOctopus::Middleware::CommonLogger

  run Proc.new { |ctx|
    puts "Got message: #{ctx.message.key} => #{ctx.message.value}"
  }
end

GlassOctopus.run(app) do |config|
  config.adapter :ruby_kafka do |kafka|
    kafka.broker_list = %[localhost:9092]
    kafka.topic       = "mytopic"
    kafka.group       = "mygroup"
    kafka.client      = { logger: config.logger }
  end
end

What is Glass Octopus?

Glass Octopus provides a minimal interface between Kafka drivers and Kafka consumers.

require "glass_octopus"

app = GlassOctopus.build do
  use GlassOctopus::Middleware::CommonLogger

  run Proc.new { |ctx|
    puts "Got message: #{ctx.message.key} => #{ctx.message.value}"
  }
end

GlassOctopus.run(app) do |config|
  config.adapter :ruby_kafka do |kafka|
    kafka.broker_list = %[localhost:9092]
    kafka.topic       = "mytopic"
    kafka.group       = "mygroup"
    kafka.client      = { logger: config.logger }
  end
end

Drivers

• poseidon
• ruby-kafka

Middlewares

• CommonLogger
• JsonParser
• Mongoid
• NewRelic
• Sentry

Where do we use it?

Product pipeline

Live on google.com

Work for Google

Work with us!

@rgabo

gabor@secretsaucepartners.com

https://bit.ly/sspinc-careers

Thanks!

Questions?

@rgabo

gabor@secretsaucepartners.com