Better Scripting

with Crystal

Lorenzo Barasti

Part 1 A clustering problem

Ingredients

a set of points
a distance function
a clustering rule

A clustering problem

Ingredients

a set of points
a distance function
a clustering rule

Result

Similar things are grouped together

A clustering problem

dist(A, B) = 2

A clustering problem

dist(D, B) = 3

A clustering problem

dist(B, E) = 6

A clustering problem

dist(P,Q) = |x_P - x_Q| + |y_P - y_Q|

A clustering problem

dist(P,Q) \leq 1

A clustering problem

dist(P,Q) \leq 2

A clustering problem

dist(P,Q) \leq 3

A clustering problem

Breaking it down

1. Load points coordinates

2. Compute pairwise distances

3. Update clusters if the rule is satisfied

A clustering problem

Breaking it down

1. Load points coordinates

2. Compute pairwise distances

3. Update clusters if the rule is satisfied

A clustering problem

Breaking it down

1. Load points coordinates

2. Compute pairwise distances

3. Update clusters if the rule is satisfied

A clustering problem

Breaking it down

1. Load points coordinates

2. Compute pairwise distances

3. Update clusters if the rule is satisfied

Out of scope

Part 2 "That's a lot of work"

the CPU

The code - "That's a lot of work"

nodes = File.readlines("./data/nodes.txt")
      .map{|line| line.split(" ").map(&:to_i)}

def dist(u, v)
  (u[0] - v[0]).abs + (u[1] - v[1]).abs
end

dist_list = []
nodes.each_with_index do |u, u_idx|
  nodes[u_idx..-1].each_with_index do |v, v_idx|
    dist_list << [u_idx, v_idx, dist(u, v)]
  end
end

The code - "That's a lot of work"

nodes = File.readlines("./data/nodes.txt")
      .map{|line| line.split(" ").map(&:to_i)}

def dist(u, v)
  (u[0] - v[0]).abs + (u[1] - v[1]).abs
end

dist_list = []
nodes.each_with_index do |u, u_idx|
  nodes[u_idx..-1].each_with_index do |v, v_idx|
    dist_list << [u_idx, v_idx, dist(u, v)]
  end
end

The code - "That's a lot of work"

nodes = File.readlines("./data/nodes.txt")
      .map{|line| line.split(" ").map(&:to_i)}

def dist(u, v)
  (u[0] - v[0]).abs + (u[1] - v[1]).abs
end

dist_list = []
nodes.each_with_index do |u, u_idx|
  nodes[u_idx..-1].each_with_index do |v, v_idx|
    dist_list << [u_idx, v_idx, dist(u, v)]
  end
end

The code - "That's a lot of work"

nodes = File.readlines("./data/nodes.txt")
      .map{|line| line.split(" ").map(&:to_i)}

def dist(u, v)
  (u[0] - v[0]).abs + (u[1] - v[1]).abs
end

dist_list = []
nodes.each_with_index do |u, u_idx|
  nodes[u_idx..-1].each_with_index do |v, v_idx|
    dist_list << [u_idx, v_idx, dist(u, v)]
  end
end

The code - "That's a lot of work"

nodes = File.readlines("./data/nodes.txt")
      .map{|line| line.split(" ").map(&:to_i)}

def dist(u, v)
  (u[0] - v[0]).abs + (u[1] - v[1]).abs
end

dist_list = []
nodes.each_with_index do |u, u_idx|
  nodes[u_idx..-1].each_with_index do |v, v_idx|
    dist_list << [u_idx, v_idx, dist(u, v)]
  end
end

The code - "That's a lot of work"

nodes = File.readlines("./data/nodes.txt")
      .map{|line| line.split(" ").map(&:to_i)}

def dist(u, v)
  (u[0] - v[0]).abs + (u[1] - v[1]).abs
end

dist_list = []
nodes.each_with_index do |u, u_idx|
  nodes[u_idx..-1].each_with_index do |v, v_idx|
    dist_list << [u_idx, v_idx, dist(u, v)]
  end
end

The code - "That's a lot of work"

nodes = File.readlines("./data/nodes.txt")
      .map{|line| line.split(" ").map(&:to_i)}

def dist(u, v)
  (u[0] - v[0]).abs + (u[1] - v[1]).abs
end

dist_list = []
nodes.each_with_index do |u, u_idx|
  nodes[u_idx..-1].each_with_index do |v, v_idx|
    dist_list << [u_idx, v_idx, dist(u, v)]
  end
end

The code - "That's a lot of work"

nodes = File.readlines("./data/nodes.txt")
      .map{|line| line.split(" ").map(&:to_i)}

def dist(u, v)
  (u[0] - v[0]).abs + (u[1] - v[1]).abs
end

dist_list = []
nodes.each_with_index do |u, u_idx|
  nodes[u_idx..-1].each_with_index do |v, v_idx|
    dist_list << [u_idx, v_idx, dist(u, v)]
  end
end

Load

Compute pairwise distance

The code - "That's a lot of work"

nodes = File.readlines("./data/nodes.txt")
      .map{|line| line.split(" ").map(&:to_i)}

def dist(u, v)
  (u[0] - v[0]).abs + (u[1] - v[1]).abs
end

dist_list = []
nodes.each_with_index do |u, u_idx|
  nodes[u_idx..-1].each_with_index do |v, v_idx|
    dist_list << [u_idx, v_idx, dist(u, v)]
  end
end

Load

Compute pairwise distance

~N² operations

Part 3 "I wanna go fast"

Ricky Bobby

Crystal - "I wanna go fast"

Crystal Goals

Expressiveness

Correctness

Performance

Crystal - expressiveness

nodes = File.readlines("./data/nodes.txt")
      .map{|line| line.split(" ").map(&:to_i)}

nodes = File.read_lines("./data/nodes.txt")
      .map{|line| line.split(" ").map(&.to_i)}

API differences

flexible shorthand
block syntax

Crystal - expressiveness

flexible shorthand block syntax

["some", "String", "HeRe"].map(&.downcase.reverse)
# => ["emos", "gnirts", "ereh"]

(-2..2).map(&.abs.+(1).*(2))
# => [6, 4, 2, 4, 6]

Crystal - expressiveness

dist_list = []
dist_list << [u_idx, v_idx, dist(u,v)]

dist_list = [] of {Int32, Int32, Int32}
dist_list << {u_idx, v_idx, dist(u,v)}

Empty array initialization

Crystal - expressiveness

dist_list = []
dist_list << [u_idx, v_idx, dist(u,v)]

dist_list = [] of {Int32, Int32, Int32}
dist_list << {u_idx, v_idx, dist(u,v)}

Tuples are encoded as a separate type

Crystal - expressiveness

Metaprogramming

Classes, variables and methods can be defined at runtime

Code can be injected/removed at compile time

Crystal - "I wanna go fast"

Crystal Goals

Expressiveness

Correctness

Performance

Crystal - correctness

def find_by_id(id, users)
  users.find{ |user| user["id"].to_i == id}
end

def find_similar(user, users)
  users.select{ |other| other["origin"] == user["origin"]}
end

user = find_by_id(240, table)

p find_similar(user, table)

Crystal - correctness

def find_by_id(id, users)
  users.find{ |user| user["id"].to_i == id}
end

def find_similar(user, users)
  users.select{ |other| other["origin"] == user["origin"]}
end

user = find_by_id(240, table)

p find_similar(user, table)

Crystal - correctness

def find_by_id(id, users)
  users.find{ |user| user["id"].to_i == id}
end

def find_similar(user, users)
  users.select{ |other| other["origin"] == user["origin"]}
end

user = find_by_id(240, table)

p find_similar(user, table)

Crystal - correctness

def find_by_id(id, users)
  users.find{ |user| user["id"].to_i == id}
end

def find_similar(user, users)
  users.select{ |other| other["origin"] == user["origin"]}
end

user = find_by_id(240, table)

p find_similar(user, table)

Crystal - correctness

def find_by_id(id, users)
  users.find{ |user| user["id"].to_i == id}
end

def find_similar(user, users)
  users.select{ |other| other["origin"] == user["origin"]}
end

user = find_by_id(240, table)

p find_similar(user, table)

Crystal - correctness

def find_by_id(id, users)
  users.find{ |user| user["id"].to_i == id}
end

def find_similar(user, users)
  users.select{ |other| other["origin"] == user["origin"]}
end

user = find_by_id(240, table)

p find_similar(user, table)

p find_similar(user, table)
  ^~~~~~~~~~~~

in users.cr:13: undefined method '[]' for Nil (compile-time type is (Hash(String, String) | Nil))

  users.select{ |other| other["origin"] == user["origin"]}
                                               ^

Compilation Error

All good

Crystal - correctness

def find_by_id(id, users)
  users.find{ |user| user["id"].to_i == id}
end

def find_similar(user, users)
  users.select{ |other| other["origin"] == user["origin"]}
end

user = find_by_id(240, table)

p find_similar(user, table)

p find_similar(user, table)
  ^~~~~~~~~~~~

in users.cr:13: undefined method '[]' for Nil (compile-time type is (Hash(String, String) | Nil))

  users.select{ |other| other["origin"] == user["origin"]}
                                               ^

Compilation Error

All good

Nil reference check

____

Crystal - correctness

def find_by_id(id, users)
  users.find{ |user| user["id"].to_i == id}
end

def find_similar(user, users)
  return nil if user.nil?
  users.select{ |other| other["origin"] == user["origin"]}
end

user = find_by_id(240, table)

p find_similar(user, table)

All good

_____________

Safer code!

Crystal - correctness

def send(commands)
  cmds = commands.is_a?(Array) ?
    commands :
    [commands]

  cmds.each do |cmd|
    # do something
  end
end

Crystal - correctness

def send(commands)
  cmds = commands.respond_to?(:each) ?
    commands :
    [commands]

  cmds.each do |cmd|
    # do something
  end
end

Crystal - correctness

def send(commands)
  cmds = commands.responds_to?(:each) ?
    commands :
    [commands]

  cmds.each do |cmd|
    # do something
  end
end

The English language is safe!

Crystal - correctness

def send(commands : Array)
  commands.each_with_index do |cmd, idx|
    # do something
  end
end

def send(command)
  send([command])
end

Method overloading based on

The type restrictions applied to arguments
The number of arguments
The names of required named arguments
Whether the method accepts a block or not

Crystal - correctness

def send(commands : Enumerable)
  commands.each_with_index do |cmd, idx|
    # do something
  end
end

def send(command)
  send([command])
end

Method overloading

Arguments type matches on modules too
Clearer intention compared to duck typing
Separation of logic

__________

Crystal - "I wanna go fast"

Crystal Goals

Expressiveness

Correctness

Performance

Crystal - performance

Compiled language speed

Low memory footprint

Event loop powered non-blocking I/O

Channel based communication between fibers

Crystal - performance

Run time / Number of nodes

Crystal - "I wanna go fast"

Crystal Goals

Expressiveness

Correctness

Performance

Part 4 It's not about the destination

I was looking for

Performance

It's not about the destination

What I found was

Correctness

{
  name: lorenzo.barasti,
  works_at:             ,
  blog: lbarasti.github.io,
  twitter: @lbarasti
}

Better scripting with Crystal

Thanks for listening!