(Dynamic|Meta)Programming With Crystal

 

Kirk Haines

Principal Developer Relations Engineer w/ New Relic

 

 

 

@wyhaines everywhere

https://www.therelicans.com/wyhaines

Defining Terms
Dynamic 

 

 

 

This term is heavily overloaded!

Defining Terms

Dynamic

 

https://en.wikipedia.org/wiki/Dynamic_programming

 

I do not mean this:

 

...it refers to simplifying a complicated problem by breaking it down into simpler sub-problems in a recursive manner.

Defining Terms

Dynamic

 

https://en.wikipedia.org/wiki/Dynamic_programming_language

We kind of mean this:

 

...a class of high-level programming languages, which at runtime execute many common programming behaviours that static programming languages perform during compilation...object...alteration...reflection...macros...

Defining Terms

Dynamic

 

 

 

Crystal is a compiled, statically typed language, and it is not a dynamic language.

Defining Terms

Dynamic

 

 

 

All Done! Time for the next talk!

Defining Terms

Metaprogramming

 

https://en.wikipedia.org/wiki/Metaprogramming

 

Metaprogramming is a technique where code is treated as data. Also:

 

Metaprogramming can be used to move computations from run-time to compile-time, to generate code using compile time computations, and to enable self-modifying code.  

Defining Terms

Metaprogramming

 

 

 

Metaprogramming tools overlap Dynamic Language tools,

and they enable those tools in Crystal.

Defining Terms

Metaprogramming

 

 

 

Languages like Javascript and Ruby support eval()

Defining Terms

Metaprogramming

 

 

 

Crystal doesn't support eval()*​

 

(interpreted Crystal, unveiled by Asterite earlier, might, on some levels, change this)

Defining Terms

Metaprogramming

 

 

Crystal does have:

Defining Terms

Metaprogramming

 

 

Crystal does have:

  1. Mixins and reopenable classes

Defining Terms

Metaprogramming

 

 

Crystal does have:

  1. Mixins and reopenable classes
  2. Reflection

Defining Terms

Metaprogramming

 

 

Crystal does have:

  1. Mixins and reopenable classes
  2. Reflection
  3. Macros

Compare & Contrast

Ruby <=> Crystal

Compare & Contrast

Ruby is dynamically typed*

 

Compare & Contrast

Ruby is dynamically typed*

Ruby has eval - method_eval(), class_eval(), and eval()

 

Compare & Contrast

Ruby is dynamically typed*

Ruby has eval - method_eval(), class_eval(), and eval()

Ruby has define_method / remove_method

Compare & Contrast

Ruby is dynamically typed*

Ruby has eval - method_eval(), class_eval(), and eval()

Ruby has define_method / remove_method

Ruby has include / extend

Compare & Contrast

Ruby is dynamically typed*

Ruby has eval - method_eval(), class_eval(), and eval()

Ruby has define_method / remove_method

Ruby has include / extend

Ruby has refinements

Compare & Contrast

Ruby is dynamically typed*

Ruby has eval - method_eval(), class_eval(), and eval()

Ruby has define_method / remove_method

Ruby has include / extend

Ruby has refinements

Ruby has send

Compare & Contrast

Ruby is dynamically typed*

Ruby has eval - method_eval(), class_eval(), and eval()

Ruby has define_method / remove_method

Ruby has include / extend

Ruby has refinements

Ruby has send

Ruby has method_missing

Compare & Contrast

Crystal is statically typed

Compare & Contrast

Crystal is statically typed*

Crystal has include / extend

Compare & Contrast

Crystal is statically typed*

Crystal has include / extend

Crystal has method_missing*

Compare & Contrast

Crystal is statically typed*

Crystal has include / extend

Crystal has method_missing*

Crystal has macros!!!

Compare & Contrast

Crystal is statically typed*

Crystal has include / extend

Crystal has method_missing*

Crystal has macros!!!

(and this means that Crystal can have dynamic things like send...)

Now, Back to that Dynamic Thing...


https://chrisseaton.com/phd/specialising-ruby.pdf




Metaprogramming can also be described as a dynamic language feature. 

Now, Back to that Dynamic Thing...

 

 

 

 

Crystal has powerful metaprogramming features!

Now, Back to that Dynamic Thing...

Now, Back to that Dynamic Thing...

What if you want more information?

Is it really only 40µs to handle a request?
What is the performance breakdown of different phases?
Can I get this without rewriting my app or Kemal?

Now, Back to that Dynamic Thing...

Now, Back to that Dynamic Thing...

Interesting....

How does that work, anyway?

Now, Back to that Dynamic Thing...

Classes in Crystal can be reopened

Now, Back to that Dynamic Thing...

#previous_def is the magic

Now, Back to that Dynamic Thing...

 

 

#previous_def calls the previous definition of the method

 

This permits writing code which injects itself into other classes/structs, wrapping and overriding or proxying those methods.

#send and Dynamic Dispatch

 

 

 

 

In Ruby, #send is used to call methods based on information that can not be known at compile time. 

#send and Dynamic Dispatch

 

 

 

 

#send is commonly used in Ruby metaprogramming.

It can call methods based on their names, which might come from command line inputs or configuration file inputs or other sources not known until runtime.

#send and Dynamic Dispatch

Problem:

Software accepts complex configuration from potentially nested, serialize YAML files. You want to provide a CLI option that can override anyconfig value from the command line. You DO NOT want to attempt to explicitly handle all possible config options, though. You want a generic solution that will just call the right methods based on CLI options.

 

command -k foo:bar -k deeper:nested:key:value

#send and

Dynamic Dispatch

 

in Crystal?

Building #send

Requirements:

Building #send

Requirements:

 

  1. Wrap a method call in something that can be stored in a data structure.

Building #send

Requirements:

 

  1. Wrap a method call in something that can be stored in a data structure.
# With Procs
method  = ->(obj : Foo) { obj.a },
method2 = ->(obj : Foo, val : Int32) { obj.b(val) }

method.call(receiver)
method2.call(receiver, 123)

With Procs:

Building #send

Requirements:

 

  1. Wrap a method call in something that can be stored in a data structure.
# With records (Structs)
record Call_a, obj : Foo do
  def call
    obj.a
  end
end

record Call_b, obj : Foo, val : Int32 do
  def call
    obj.b(val)
  end
end

Call_a.new(receiver).call
Call_b.new(receiver, 123).call

With Records:

Building #send

Requirements:

 

  1. Wrap a method call in something that can be stored in a data structure.
  2. Implement lookup tables that can find the wrapped method calls.

Building #send

Requirements:

 

  1. Wrap a method call in something that can be stored in a data structure.
  2. Implement lookup tables that can find the wrapped method calls.
SendLookup = {
  "a" => Call_a
}

SendLookupInt32 = {
  "b" => Call_b
}

Building #send

Requirements:

 

  1. Wrap a method call in something that can be stored in a data structure.
  2. Implement lookup tables that can find the wrapped method calls.
  3. Implement overloaded #send methods that match the type signatures of the methods to be called.

Building #send

Requirements:

 

  1. Wrap a method call in something that can be stored in a data structure.
  2. Implement lookup tables that can find the wrapped method calls.
  3. Implement overloaded #send methods that match the type signatures of the methods to be called.
def send(method)
  SendLookup[method].call(self)
end

def send(method, arg1 : Int32)
  SendLookupInt32[method].call(self, arg1)
end

Building #send

Consider the following class:

class Foo
  def a(val : Int32)
    val + 7
  end

  def b(x : Int32, y : Int32)
    x * y
  end

  def c(val : Int32, val2 : Int32)
    val * val2
  end

  def d(xx : String, yy : Int32) : UInt128
    xx.to_i.to_u128 ** yy
  end
end

Building #send

Send capability could be built like this:

module FooSends
  SendLookupInt32 = {
    "a": ->(obj : Foo, val : Int32) { obj.a(val) },
  }

  SendLookupInt32Int32 = {
    "b": ->(obj : Foo, x : Int32, y : Int32) { obj.b(x, y) },
    "c": ->(obj : Foo, val : Int32, val2 : Int32) { obj.c(val, val2) },
  }

  SendLookupStringInt32 = {
    "d": ->(obj : Foo, xx : String, yy : Int32) { obj.d(x, y) }
  }
  
  def send(method, arg1 : Int32)
    SendLookupInt32[method].call(self, arg1)
  end

  def send(method, arg1 : Int32, arg2 : Int32)
    SendLookupInt32Int32[method].call(self, arg1, arg2)
  end
  
  def send(method, arg1 : String, arg2 : Int32)
    SendLookupStringInt32[method].call(self, arg1, arg2)
  end
end

Building #send

For a big class, or if there are many classes, this would be no fun at all.

Building #send

Requirements:

 

  1. Wrap a method call in something that can be stored in a data structure.
  2. Implement lookup tables that can find the wrapped method calls.
  3. Implement overloaded #send methods that match the type signatures of the methods to be called.
     
  4. Have Crystal build the code itself instead of requiring that it be hand-built.

Building #send

Requirements:

 

...

4. Have Crystal build the code itself instead of requiring that it be hand-built.

Building #send

Macros!

 

Macros are code, essentially written in an interpreted subset of Crystal, which writes

new code that will be inserted into the codebase and compiled with it. 

Building #send

Macros!

 

Macros have access to reflection data -- types, methods, classes, annotations, etc...

Building #send

Macros!

 

Macros can build everything that we need!

Building #send 

require "send"

class Foo
  include Send

  def a(val : Int32)
    val + 7
  end

  def b(x : Int32, y : Int32)
    x * y
  end

  def c(val : Int32, val2 : Int32)
    val * val2
  end

  def d(xx : String, yy : Int32) : UInt128
    xx.to_i.to_u128 ** yy
  end
end

Send.activate

f = Foo.new
pp f.__send__("b", 23, 37)

Building #send

The full implementation is a talk all by itself.

 

Take a look at:  https://github.com/wyhaines/Send.cr

#method_missing

Building methods on-demand

 

 

 

In Ruby, #method_missing is a method that is called at runtime when an attempt is made to invoke a method that doesn't exist in the current scope.

 

Crystal implements this as a compile-time feature using a macro.

#method_missing

Building methods on-demand

#method_missing

Building methods on-demand

#method_missing

One, of many uses -- a wrapper class

#method_missing

One, of many uses -- a wrapper class

#method_missing

One, of many uses -- a wrapper class

#method_missing

One, of many uses -- a wrapper class

These Examples
Just Scratch The Surface

Crystal's metaprogramming capabilities and the dynamic language qualities that can be built from them are powerful.

Thank You!

Questions?

All of the code for this presentation, and the slide deck, will be pushed to GitHub@

 

https://github.com/wyhaines/crystal-conf-1.0-dynamic-crystal

Dynamic and Metaprogramming with Crystal

By wyhaines

Dynamic and Metaprogramming with Crystal

Compiled, statically typed languages aren't typically known for their dynamic nature or their ability to cater to metaprogramming of the type that languages like Ruby are known for. And while it is true that some of the things that Ruby does aren't easily supported within the Crystal ecosystem, Crystal does provide programmers with a powerful, capable set of tools for building the same sorts of dynamic metaprogramming that Ruby is known for. This talk will survey some of those techniques, and look at how they work to achieve surprisingly powerful results.

  • 561

More from wyhaines