deep dive into SIDEKIQ INTERNALS

Tamas Michelberger
Secret Sauce Partners, Inc.
@tmichelberger

Sidekiq at SSP

• We started heavily using Sidekiq in the beginning of 2016
• We migrated from Resque
• ~5M jobs/day

sidekiq:
an extremely reliable
thread-pool

How does a job get executed?

1. Fetch JSON from Redis queue
2. Get the worker class
3. Run the job through the middleware stack

4. Execute job

5. Report statistics (processed and failed jobs)

Main components

• Launcher: heartbeat, coordinate the manager and scheduler
• Manager: keep job processors alive, handle quiet and shutdown signals
• Processor: doing the actual work

Let's look at some code

Tips & Tricks

Pushing jobs in bulk

module Sidekiq
  module Worker
    module Bulk
      def perform_async_bulk(args_set)
        bulk_size = get_sidekiq_options['max_bulk_size'] || 1000
        args_set.each_slice(bulk_size).flat_map do |args|
          client_push_bulk('class' => self, 'args' => args)
        end
      end

      def client_push_bulk(item)
        pool = Thread.current[:sidekiq_via_pool] || 
            get_sidekiq_options['pool'] || Sidekiq.redis_pool

        Sidekiq::Client.new(pool).push_bulk(item.stringify_keys)
      end
    end
  end
end

Gist with comments

Tips & Tricks

Clean stuck workers from Redis

process_ids = Sidekiq::ProcessSet.new
                .select { |p| Time.at(p['beat']) < 2.hours.ago }
                .map { |p| p['identity'] }

Sidekiq.redis { |c| c.del(*process_ids) } if process_ids.any?

Thanks!

Questions?

We are hiring!