Ruby Performance Optimization

@acrogenesis

What can I do to make the code faster?

• I don't know

• Caching and Scaling

What makes Ruby code slow

• Algorithmic Complexity

• Nested Loops

• Computations

Garbage Collector Disabled

• Ruby's GC is slow

• The code uses too much memory

Measuring Memory Used

require "benchmark"

num_rows = 100000
num_cols = 10
data = Array.new(num_rows) { Array.new(num_cols) { "x"*1000 } }

puts "%d MB" % (`ps -o rss= -p #{Process.pid}`.to_i/1024)

GC.disable
time = Benchmark.realtime do
  csv = data.map { |row| row.join(",") }.join("\n")
end

puts "%d MB" % (`ps -o rss= -p #{Process.pid}`.to_i/1024)
puts time.round(2)

> 1060 MB

> 2999 MB

Our inital dataset takes ~1GB, so why processing the data took ~2GB extra instead of just ~1GB.

Where's that extra GB

require "benchmark"

num_rows = 100000
num_cols = 10
data = Array.new(num_rows) { Array.new(num_cols) { "x"*1000 } }

time = Benchmark.realtime do
  csv = data.map do |row|
    row.join(",")
  end.join("\n")
end

puts time.round(2)

The CSV rows that we generate inside that block are actually intermediate results stored into memory until we can finnaly join them by the newline character. 

This is exactly where we use that extra 1GB of memory

Optimizing Memory

require "benchmark"

num_rows = 100000
num_cols = 10
data = Array.new(num_rows) { Array.new(num_cols) { "x"*1000 } }

time = Benchmark.realtime do
  csv = ''
  num_rows.times do |i|
    num_cols.times do |j|
      csv << data[i][j]
      csv << "," unless j == num_cols - 1
    end
    csv << "\n" unless i == num_rows - 1
  end
end

puts time.round(2)

Memory optimization is easy

1. Review
2. Think
3. Rewrite

Best Practices to write code that doesn't use much memory

str = "X" * 1024 * 1024 * 10
str = str.downcase

str = "X" * 1024 * 1024 * 10
str.downcase!

Modifying Strings in Place

10MB

0MB

x = "foo"
x += "bar"

# Equivalent to
x = "foo"
y = x + "bar"
x = y

# Use String::<<
x = "foo"
x << "bar"

Modifying Arrays and Hashes in Place

data = Array.new(100) { "x" * 1024 * 1024 }

measure do
  data.map { |str| str.upcase }
end

data = Array.new(100) { "x" * 1024 * 1024 }

measure do
  data.map! { |str| str.upcase! }
end

100MB

0MB

Best Practices to write code that doesn't use much memory

Write Less Ruby

Ruby is especially bad in two areas

1. Large dataset processing
2. Complex computations