Fast, Simple Concurrency with Scala Native

This talk is about:

bare-metal concurrency on scala native
native as a platform
scala as a platform
sustainable libraries and communities

and it's about native-loop, a new concurrency package for Scala Native 0.4

an extensible event loop and IO system
backed by the libuv event loop
works with other C libraries (libcurl, etc)

http://...

Talk Outline

Scala Native Crash Course
implementation deep dive:
- libuv-based ExecutionContext
- simple streaming IO
HTTP client/server API design
the future

About me:

@RichardWhaling
Author of "Modern Systems Programming in Scala Native"
Lead Data Engineer at M1Finance

Scala Native is:

Scala!
a scalac plugin
targets LLVM rather than JVM bytecode
produces compact, optimized native binaries
a lot of ordinary Scala code "just works"

full control over memory allocation
struct and array layout
C FFI
An embedded scala DSL with the capabilities of C

But that's not all!

Caveat

this low-level functionality is powerful but dangerous
you don't need unsafe functionality to use scala native
ideally: idiomatic Scala API on top of low-level code

Scala Native is (for now) single-threaded
No JDK - essential classes re-implemented in Scala
C FFI fills in the gaps in capabilities

But what is "idiomatic Scala"?

Example: Memory Allocation

val raw_data:Ptr[Byte] = malloc(sizeof[Long])

val int_ptr:Ptr[Int] = raw_data.asInstanceOf[Int]

// the ! operator updates a pointer's contents on the left-hand side
!int_ptr = 0

// on the right-hand side, it dereferences the pointer to read its value
printf(c"pointer at %p has value %d\n", int_ptr, !int_ptr)

!int_ptr = 1
printf(c"pointer at %p has value %d\n", int_ptr, !int_ptr)

//this will segfault
free(raw_data)
printf(c"pointer at %p has value %d\n", int_ptr, !int_ptr)

not shown: arrays and pointer arithmetic

Example: C FFI

@extern
object Quicksort {
  type Comparator = CFuncPtr2[Ptr[Byte],Ptr[Byte],Int]
  def qsort(array:Ptr[Byte],num:CSize,size:CSize,cmp:Comparator):Unit = extern
}

object App {
  type MyStruct = CStruct3[CString,CString,Int]
  val myStructComp = new Comparator {
    def apply(left:Ptr[Byte],right:[Byte]):Int = {
      val l = left.asInstanceOf[Ptr[MyStruct]]
      val r = right.asInstanceOf[Ptr[MyStruct]]
      if (l._3 < r._3) {
        -1
      } else if (l._3 == l._3) {
        0
      else {
        1
      }
    }
  } 
}

Example: C FFI

@extern
object Quicksort {
  type Comparator = CFuncPtr2[Ptr[Byte],Ptr[Byte],Int]
  def qsort(array:Ptr[Byte],num:CSize,size:CSize,cmp:Comparator):Unit = extern
}

object App {
  type MyStruct = CStruct3[CString,CString,Int]
  val myStructComp = new Comparator {
    def apply(left:Ptr[Byte],right:[Byte]):Int = {
      val l = left.asInstanceOf[Ptr[MyStruct]]
      val r = right.asInstanceOf[Ptr[MyStruct]]
      l - r
    }
  }

  def main(args:Array[String]):Unit = {
    // ...
    val data:Ptr[MyStruct] = ???
    val data_size = ???
    qsort(data,data_size,sizeof[MyStruct],myStructComp)
    // ...
  }
}

Example: Function Pointers

instances of CFuncPtr can be passed as callbacks to C
In POSIX this is common for polymorphic functions
sort, bsearch, etc

We'll see more of this soon!

LibUV's IO system

libuv abstracts over different operating systems and different kinds of IO

LibUV's event loop

We just need to adapt a queue-based EC to libuv's lifecycle of callbacks

We queue up work
A prepare handle run immediately prior to IO
It runs tasks until the queue is exhausted
When there are no more tasks and no more IO, we are done!
The catch - how do we track IO that isn't a Future - like our Pipe example?

BYO ExecutionContext

Scala Native includes an EC already
The catch - it runs after main() returns

object ExecutionContext {
  def global: ExecutionContextExecutor = QueueExecutionContext

  private object QueueExecutionContext extends ExecutionContextExecutor {
    def execute(runnable: Runnable): Unit = queue += runnable
    def reportFailure(t: Throwable): Unit = t.printStackTrace()
  }

  private val queue: ListBuffer[Runnable] = new ListBuffer

  private[runtime] def loop(): Unit = {  // this runs after main() returns
    while (queue.nonEmpty) {
      val runnable = queue.remove(0)
      try {
        runnable.run()
      } catch {
        case t: Throwable =>
          QueueExecutionContext.reportFailure(t)
      }
    }
  }
}

EventLoop and LoopExtensions

trait EventLoopLike extends ExecutionContextExecutor {
  def addExtension(e:LoopExtension):Unit 
  def run(mode:Int = UV_RUN_DEFAULT):Unit
}

trait LoopExtension {
  def activeRequests():Int
}

Our EventLoop

object EventLoop extends EventLoopLike {
  val loop = uv_default_loop()

  private val taskQueue = ListBuffer[Runnable]()
  def execute(runnable: Runnable): Unit = taskQueue += runnable
  def reportFailure(t: Throwable): Unit = {
    println(s"Future failed with Throwable $t:")
    t.printStackTrace()
  }
  // ...

Our EventLoop

  // ...
  private def dispatchStep(handle:PrepareHandle) = {
    while (taskQueue.nonEmpty) {
      val runnable = taskQueue.remove(0)
      try {
        runnable.run()
      } catch {
        case t: Throwable => reportFailure(t)
      }
    }
    if (taskQueue.isEmpty && !extensionsWorking) {
      println("stopping dispatcher")
      LibUV.uv_prepare_stop(handle)
    }
  }

  private val dispatcher_cb = CFunctionPtr.fromFunction1(dispatchStep)

  private def initDispatcher(loop:LibUV.Loop):PrepareHandle = {
    val handle = stdlib.malloc(uv_handle_size(UV_PREPARE_T))
    check(uv_prepare_init(loop, handle), "uv_prepare_init")
    check(uv_prepare_start(handle, dispatcher_cb), "uv_prepare_start")
    return handle
  }

  private val dispatcher = initDispatcher(loop)
  // ...

LoopExtensions

  private val extensions = ListBuffer[LoopExtension]()

  private def extensionsWorking():Boolean = {
    extensions.exists( _.activeRequests > 0)
  }

  def addExtension(e:LoopExtension):Unit = {
    extensions.append(e)
  }

Timer

object Timer extends LoopExtension {
  EventLoop.addExtension(this)

  var serial = 0L
  var timers = mutable.HashMap[Long,Promise[Unit]]()

  override def activeRequests():Int = 
    timers.size

  def delay(dur:Duration):Future[Unit] = ???

  val timerCB:TimerCB = ???
}

@extern
object TimerImpl {
  type Timer: Ptr[Long] // why long and not byte?
  def uv_timer_init(loop:Loop, handle:TimerHandle):Int = extern
  def uv_timer_start(handle:TimerHandle, cb:TimerCB, 
                     timeout:Long, repeat:Long):Int = extern
  def uv_timer_stop(handle:TimerHandle):Int = extern
}

Timer

object Timer extends LoopExtension {
  EventLoop.addExtension(this)

  var serial = 0L
  var timers = mutable.HashMap[Long,Promise[Unit]]()

  override def activeRequests():Int = 
    timers.size

  def delay(dur:Duration):Future[Unit] = ???

  val timerCB:TimerCB = ???
}

@extern
object TimerImpl {
  type Timer: Ptr[Long] // why long and not byte?
  def uv_timer_init(loop:Loop, handle:TimerHandle):Int = extern
  def uv_timer_start(handle:TimerHandle, cb:TimerCB, 
                     timeout:Long, repeat:Long):Int = extern
  def uv_timer_stop(handle:TimerHandle):Int = extern
}

Timer

  def delay(dur:Duration):Future[Unit] = {
    val promise = Promise[Unit]()
    serial += 1
    val timer_id = serial
    timers(timer_id) = promise
    val millis = dur.toMillis

    val timer_handle = stdlib.malloc(uv_handle_size(UV_TIMER_T))
    uv_timer_init(EventLoop.loop,timer_handle)
    val timer_data = timer_handle.asInstanceOf[Ptr[Long]]
    !timer_data = timer_id
    uv_timer_start(timer_handle, timerCB, millis, 0)

    promise.future
  }

Timer

  val timerCB = new TimerCB { 
    def apply(handle:TimerHandle):Unit = {
      println("callback fired!")
      val timer_data = handle.asInstanceOf[Ptr[Long]]
      val timer_id = !timer_data
      val timer_promise = timers(timer_id)
      timers.remove(timer_id)
      println(s"completing promise ${timer_id}")
      timer_promise.success(())
    }
  }

Timer

  val timerCB = new TimerCB { 
    def apply(handle:TimerHandle):Unit = {
      println("callback fired!")
      val timer_data = handle.asInstanceOf[Ptr[Long]]
      val timer_id = !timer_data
      val timer_promise = timers(timer_id)
      timers.remove(timer_id)
      println(s"completing promise ${timer_id}")
      timer_promise.success(())
    }
  }

Timer

object Main {
  implicit val ec:ExecutionContext = EventLoop

  def main(args:Array[String]):Unit = {
    println("hello!")
    Timer.delay(3 seconds).onComplete { _ =>
      println("goodbye!")
    }

    EventLoop.run()
  }
}

Streaming with libuv

libuv tracks IO via handles of different types
handles are configured with callbacks
handles all contain "free" space for a custom data pointer

@extern
object LibUV {
  def uv_pipe_init(loop:Loop, handle:PipeHandle, ipc:Int):Int = extern
  def uv_pipe_open(handle:PipeHandle, fd:Int):Int = extern
  def uv_read_start(client:PipeHandle, allocCB:AllocCB, readCB:ReadCB): Int = extern

  type Loop = ???
  type PipeHandle = ???
  type AllocCB = ???
  type ReadCB = ???
}

Streaming with libuv

we can represent opaque structures as Ptr[Byte] (like void *)
libuv provides helpers to allocate and initialize loop/handles
function pointer arguments must be static methods

@extern
object LibUV {
  def uv_pipe_init(loop:Loop, handle:PipeHandle, ipc:Int):Int = extern
  def uv_pipe_open(handle:PipeHandle, fd:Int):Int = extern
  def uv_read_start(client:PipeHandle, allocCB:AllocCB, readCB:ReadCB): Int = extern

  type Loop = Ptr[Byte]
  type PipeHandle = Ptr[Byte]
  type AllocCB = CFunctionPtr3[PipeHandle,CSize,Ptr[Buffer],Unit]
  type ReadCB = CFunctionPtr3[PipeHandle,CSSize,Ptr[Buffer],Unit]

  def uv_handle_size(h_type:Int): CSize = extern

  type Buffer = CStruct2[Ptr[Byte],CSize]
}

An API Sketch

trait Pipe[I,O] {
  def map[U](f: O => U):Pipe[O,U] 
  def feed(i:I):Unit
}

case class SourcePipe(fd:Int) extends Pipe[String,String] 

object SourcePipe {
  val loop = uv_default_loop()

  def stream(fd:Int):SourcePipe = {
    val handle = malloc(uv_handle_size(UV_PIPE_T)) // no need to cast
    uv_pipe_init(loop,handle,0)
    uv_pipe_open(handle,fd)
    uv_read_start(handle,???,???) // we need to supply these callbacks
  }
}

An API Sketch

trait Pipe[I,O] {
  def map[U](f: O => U):Pipe[O,U] 
  def feed(i:I):Unit
}

case class SourcePipe(fd:Int) extends Pipe[String,String] 

object SourcePipe {
  val loop = uv_default_loop()

  def stream(fd:Int):SourcePipe = {
    val handle = malloc(uv_handle_size(UV_PIPE_T)) // no need to cast
    uv_pipe_init(loop,handle,0)
    uv_pipe_open(handle,fd)
    uv_read_start(handle,???,???) // we need to supply these callbacks
  }
}

An API Sketch

trait Pipe[I,O] {
  def map[U](f: O => U):Pipe[O,U] 
  def feed(i:I):Unit
}

case class SourcePipe(fd:Int) extends Pipe[String,String]

object SourcePipe {
  val loop = uv_default_loop()
  var handlers:mutable.Map[Int,SourcePipe] // our "dispatcher"

  def stream(fd:Int):SourcePipe = {
    val handle = malloc(uv_handle_size(UV_PIPE_T)) // no need to cast
    uv_pipe_init(loop, handle, 0)
    uv_pipe_open(handle, fd)

    // store the fd in the pipe's custom data slot
    val pipeDataPointer = handle.cast[Ptr[Int]]
    !pipeDataPointer = fd
    
    uv_read_start(handle, ???, ???) // need to fill these in
    val result = SourcePipe(fd)
    handlers(fd) = result
    result
  }
}

libuv Callbacks

we need to supply callbacks to allocate memory and receive data
an alloc callback runs immediately before data is received
we can supply any memory allocation strategy we like
libuv will adjust its buffering to whatever chunk size we provide
we can be naive for now, but this is very powerful

  def on_alloc(client:PipeHandle, size:CSize, buffer:Ptr[Buffer]):Unit = {
    val bufferData = malloc(4096)
    !buffer._1 = bufferData
    !buffer._2 = 4096
  }
  val allocCB = CFunctionPtr.fromFunction3(on_alloc)

libuv Callbacks

our read callback needs to:
dereference the handle to get the file descriptor
convert the buffer struct to a regular String
look up the right handler and pass it the string
free the temp buffer

  def on_read(handle:PipeHandle,size:CSize,buffer:Ptr[Buffer]):Unit = {
    val pipeDataPointer = handle.cast[Ptr[Int]]
    val fd = !pipeDataPointer
    println(s"read $size bytes from fd $fd")
    if (size < 0) {
      println("size < 0, closing")
      handlers.remove(fd)
    } else {
      val tempBuffer = stdlib.malloc(size + 1)
      strncpy(tempBuffer, !buffer._1, size + 1)      
      val stringData = fromCString(tempBuffer)
      handlers(fd).feed(stringData)
      stdlib.free(tempBuffer)
    }
  }
  val readCB = CFunctionPtr.fromFunction3(on_read)

libuv Callbacks

we can customize string construction and save on copying!

  def on_read(handle:PipeHandle,size:CSize,buffer:Ptr[Buffer]):Unit = {
    val pipeDataPointer = handle.cast[Ptr[Int]]
    val fd = !pipeDataPointer
    println(s"read $size bytes from fd $fd")
    if (size < 0) {
      println("size < 0, closing")
      handlers.remove(fd)
    } else {
      val stringData = bytesToString(!buffer._1, size)
      handlers(fd).feed(stringData)
      stdlib.free(tempBuffer)
    }
  }
  val readCB = CFunctionPtr.fromFunction3(on_read)

  def bytesToString(data:Ptr[Byte],len:Long):String = {
    val bytes = new Array[Byte](len.toInt)
    var c = 0
    while (c < len) {
      bytes(c) = !(data + c)
      c += 1
    }

    new String(bytes)
  }

Tying it all together

The rest is straightforward, idiomatic Scala

case class SourcePipe(fd:Int) extends Pipe[String,String] {
  var destinations:List[Pipe[String,_]] = List.empty
  def feed(t:String) = {
    for (dest <- destinations) {
      dest.feed(t)
    }
  }

  def map[T](f: String => T) = {
    val m = MapPipe(f)
    destinations = destinations + m
    m
  }
}

case class MapPipe[I,O](f: I => O) extends Pipe[I,O] {
  var destinations:List[Pipe[O]] = List.empty
  def feed(t:I) = {
    val o = f(i)
    for (dest <- destinations) {
      dest.feed(o)
    }
  }
  // ...
}

Tying it all together

The rest is straightforward, idiomatic Scala

def main(args:Array[String]):Unit = {
  val STDIN = 0
  
  SourcePipe(STDIN).map { line =>
    print(s"read '${line.trim()}' from standard input")
  }
  uv_loop_run(uv_default_loop,0)
}

With a little work this becomes capable (and complex) quickly
flatMap, mapConcat, mapAsync, reduce, fold, etc.
Does coupling IO and concurrency break abstractions?
What are the core capabilities that other frameworks need?

An API Sketch

object PipeIO extends LoopExtension {
  type ItemHandler = ((String,PipeHandle,Long) => Unit)
  type DoneHandler = ((PipeHandle,Long) => Unit)
  type Handlers = (ItemHandler, DoneHandler)

  var streams = mutable.HashMap[Long,Handlers]()
  var serial = 0L

  override def activeRequests:Int = {
    streams.size
  }

  def stream(fd:Int)(itemHandler:ItemHandler,
                     doneHandler:DoneHandler):Long = ???

  def streamUntilDone(fd:Int)(handler:ItemHandler):Future[Long] = ???
}

Introducing native-loop

Concurrency as a library for Scala Native 0.4
Provides ExecutionContext/Futures/etc
Backed by the C event loop library libuv
Extensible with other C libraries
Cross-platform async IO capabilities

Integrate with curl to provide an async HTTP client
Integrates with node's http-parser for an HTTP server

Example: Future

Concurrency as a library for Scala Native 0.4
Provides ExecutionContext/Futures/etc
Backed by the C event loop library libuv
Extensible with other C libraries

def main(args:Array[String]):Unit = {
    println("hello Scala Days!")
    Future.successful(()).onComplete { v => 
        println(s"Future has completed with value $v")
    } 
    println("about to invoke event loop")
    Loop.run()
}

Example: HTTP Client

    val resp = Zone { implicit z => 
      for (arg <- args) {
        val url = toCString(arg)
        val resp = Curl.get(url)

        resp.onComplete { 
          case Success(data) =>
            println(s"got back response for ${arg} - body of length ${data.body.size}")
            println(s"headers:")
            for (h <- data.headers) {
              println(s"request header: $h")
            }
            println(s"body: ${data.body}")
          case Failure(f) =>
            println("request failed",f)
        }
      }
    }

    loop.run()

Example: HTTP Server

  def main(args:Array[String]):Unit = {
    Service()
      .getAsync("/async/") { r => Future(OK(
        Map("asyncMessage" -> s"got (async routed) request $r")
      ))}
      .get("/") { r => OK( 
        Map("message" -> s"got (routed) request $r")
      )}
      .run(9999)
    println("done")
  }

Adding an HTTP Server

We'll use node/http-parser
same C library that nodejs uses for http parsing
simple API, zero-copy implementation:

object Parsing {
  def http_parser_init(p:Ptr[Parser],parser_type:Int):Unit = extern
  def http_parser_settings_init(s:Ptr[ParserSettings]):Unit = extern
  def http_parser_execute(p:Ptr[Parser],s:Ptr[ParserSettings],
                          data:Ptr[Byte],len:Long):Long = extern
  def http_method_str(method:CChar):CString = extern

  type Parser = CStruct8[
    Long,   // private data
    Long,   // private data
    UShort, // major version
    UShort, // minor version
    UShort, // status (request only)
    CChar, // method
    CChar, // Error (last bit upgrade)
    Ptr[Byte] // user data
  ]

  //...

Adding an HTTP Server

We'll use node/http-parser
same C library that nodejs uses for http parsing
simple API, zero-copy implementation:

object Parsing {
  type HttpCB = CFunctionPtr1[Ptr[Parser],Int]
  type HttpDataCB = CFunctionPtr3[Ptr[Parser],CString,Long,Int]

  type ParserSettings = CStruct8[
    HttpCB, // on_message_begin
    HttpDataCB, // on_url
    HttpDataCB, // on_status
    HttpDataCB, // on_header_field
    HttpDataCB, // on_header_value
    HttpCB, // on_headers_complete
    HttpDataCB, // on_body
    HttpCB  // on_message_complete
  ]
  //...

Adding an HTTP Server

Accept inbound TCP connections
When a connection is established, allocate a parser
When data is received, pass it to the parser
Parser callbacks write HttpRequest state (sync)
When parser's complete callback fires, invoke handler

type ConnectionState
type RequestState
type Router
type Response

Adding an HTTP Server

object Server extends Parsing with LoopExtension {
  import LibUVConstants._, LibUV._,HttpParser._
  implicit val ec = EventLoop
  val loop = EventLoop.loop
  var serial = 1L

  override val requests = mutable.Map[Long,RequestState]()
  var activeRequests = 0

  val urlCB:HttpDataCB = CFunctionPtr.fromFunction3(onURL)
  val onKeyCB:HttpDataCB = CFunctionPtr.fromFunction3(onHeaderKey)
  val onValueCB:HttpDataCB = CFunctionPtr.fromFunction3(onHeaderValue)
  val completeCB:HttpCB = CFunctionPtr.fromFunction1(onMessageComplete)

  val parserSettings = malloc(sizeof[ParserSettings]).cast[Ptr[ParserSettings]]
  http_parser_settings_init(parserSettings)
  !parserSettings._2 = urlCB
  !parserSettings._4 = onKeyCB
  !parserSettings._5 = onValueCB
  !parserSettings._8 = completeCB
  // We'll supply the definitions of these callbacks soon!

Adding an HTTP Server

  var router:Function1[Request[String],Route] = null

  def init(port:Int, f:Request[String] => Route):Unit = {
    EventLoop.addExtension(this)
    router = f
    val addr = malloc(64)
    check(uv_ip4_addr(c"0.0.0.0", 9999, addr),"uv_ip4_addr")
    val server = malloc(uv_handle_size(UV_TCP_T)).cast[TCPHandle]    
    check(uv_tcp_init(loop, server), "uv_tcp_init")
    check(uv_tcp_bind(server, addr, 0), "uv_tcp_bind")
    check(uv_listen(server, 4096, connectCB), "uv_listen")
    this.activeRequests = 1
  }

Adding an HTTP Server


  def onConnect(server:TCPHandle, status:Int):Unit = {
    val client = malloc(uv_handle_size(UV_TCP_T)).cast[TCPHandle]
    val id = serial
    serial += 1

    val state = malloc(sizeof[ConnectionState]).cast[Ptr[ConnectionState]]
    !state._1 = serial
    !state._2 = client
    http_parser_init(state._3,HTTP_REQUEST)
    !(state._3)._8 = state.cast[Ptr[Byte]]
    !(client.cast[Ptr[Ptr[Byte]]]) = state.cast[Ptr[Byte]]

    check(uv_tcp_init(loop, client), "uv_tcp_init (client)")
    check(uv_accept(server, client), "uv_accept")
    check(uv_read_start(client, allocCB, readCB), "uv_read_start")
  }

Adding an HTTP Server

  def onAlloc(handle:TCPHandle, size:CSize, buffer:Ptr[Buffer]):Unit = {
    val buf = stdlib.malloc(4096)
    buf(4095) = 0
    !buffer._1 = buf
    !buffer._2 = 4095
  }

  def onRead(handle:TCPHandle, size:CSize, buffer:Ptr[Buffer]):Unit = {
    val state_ptr = handle.cast[Ptr[Ptr[ConnectionState]]]
    val parser = (!state_ptr)._3
    val message_id = !(!state_ptr)._1
    println(s"conn $message_id: read message of size $size")

    if (size < 0) {
      uv_close(handle, null)
      stdlib.free(!buffer._1)
    } else {
      http_parser_execute(parser,parserSettings,!buffer._1,size)
      stdlib.free(!buffer._1)
    }
  }

Adding an HTTP Server

trait Parsing {
  import LibUV._,HttpParser._
  val requests:mutable.Map[Long,RequestState]

  def handleRequest(id:Long,handle:TCPHandle,request:RequestState):Unit

  type ConnectionState = CStruct3[Long,TCPHandle,Parser]

  val HTTP_REQUEST = 0
  val HTTP_RESPONSE = 1
  val HTTP_BOTH = 2

  def bytesToString(data:Ptr[Byte],len:Long):String = {
    val bytes = new Array[Byte](len.toInt)
    var c = 0
    while (c < len) {
      bytes(c) = !(data + c)
      c += 1
    }

    new String(bytes)
  }
  //...

Adding an HTTP Server

trait Parsing {
  import LibUV._,HttpParser._
  val requests:mutable.Map[Long,RequestState]

  def handleRequest(id:Long,handle:TCPHandle,request:RequestState):Unit

  type ConnectionState = CStruct3[Long,TCPHandle,Parser]

  val HTTP_REQUEST = 0
  val HTTP_RESPONSE = 1
  val HTTP_BOTH = 2

  def bytesToString(data:Ptr[Byte],len:Long):String = {
    val bytes = new Array[Byte](len.toInt)
    var c = 0
    while (c < len) {
      bytes(c) = !(data + c)
      c += 1
    }

    new String(bytes)
  }
  //...

Adding an HTTP Server

  def onURL(p:Ptr[Parser],data:CString,len:Long):Int = {
    val state = (!p._8).cast[Ptr[ConnectionState]]
    val message_id = !state._1
    val url = bytesToString(data,len)
    val m = !p._6
    val method = fromCString(http_method_str(m))
    requests(message_id) = RequestState(url,method)
    0
  }

  def onHeaderKey(p:Ptr[Parser],data:CString,len:Long):Int = {
    val state = (!p._8).cast[Ptr[ConnectionState]]
    val message_id = !state._1
    val request = requests(message_id)
    val k = bytesToString(data,len)
    request.lastHeader = k
    requests(message_id) = request
    0
  }

  def onHeaderValue(p:Ptr[Parser],data:CString,len:Long):Int = {
    val state = (!p._8).cast[Ptr[ConnectionState]]
    val message_id = !state._1
    val request = requests(message_id)
    val v = bytesToString(data,len)
    request.headerMap(request.lastHeader) = v
    requests(message_id) = request
    0
  }

Adding an HTTP Server

  def onMessageComplete(p:Ptr[Parser]):Int = {
    val state = (!p._8).cast[Ptr[ConnectionState]]
    val message_id = !state._1
    val tcpHandle = !state._2
    val request = requests(message_id)
    handleRequest(message_id,tcpHandle,request)
    0
  }

Handling Requests

What Goes Here?

Adding an HTTP Client

We'll use libcurl - c library that powers curl
great support for http, https, ftp, scp
big library with lots of features
complex async story

Adding an HTTP Client

simple libcurl bindings:

@extern 
object CurlBindings {
  // we model the curl handle as an opaque pointer of unknown size
  type Curl = Ptr[Byte]

  // allocate and initialize a curl handle 
  def easy_init():Curl = extern

  // set an option 
  def easy_setopt(handle: Curl, option: CInt, parameter: Any): CInt = extern
  // retrieve any of many named fields
  def easy_getinfo(handle: Curl, info: CInt, parameter: Any): CInt = extern

  // run the query
  def easy_perform(easy_handle: Curl): CInt = extern

  // clean up resources
  def easy_cleanup(handle: Curl): Unit = extern

  // flags and constants
  val WRITEDATA = 10001
  val URL = 10002
  val PORT = 10003
  // ...
}

Adding an HTTP Client

simple libcurl example:

        val url:String = "https://www.example.com"

        // initialize a curl handle
        val curl = easy_init()

        // convert the url to a zero-terminated CString
        // (CString is an alias for Ptr[Byte])
        // we need an implicit Zone allocator for this    
        val url_cstring = toCString(url) 
       
        // set the url on the handle
        easy_setopt(curl, URL, url_cstring)
        
        // set callbacks 
        easy_setopt(curl, HEADERCALLBACK, headerCB)
        easy_setopt(curl, HEADERDATA, req_id_ptr.cast[Ptr[Byte]])
        easy_setopt(curl, WRITECALLBACK, writeCB)
        easy_setopt(curl, WRITEDATA, req_id_ptr.cast[Ptr[Byte]])

        // run the query
        val res = easy_perform(curl) 

        // finish
        easy_cleanup(curl)

Adding an HTTP Client

we'll need callbacks and mutable state to actually see our data!

        val url:String = "https://www.example.com"

        // initialize a curl handle
        val curl = easy_init()

        // convert the url to a zero-terminated CString
        // (CString is an alias for Ptr[Byte])
        // we need an implicit Zone allocator for this    
        val url_cstring = toCString(url) 
       
        // set the url on the handle
        easy_setopt(curl, URL, url_cstring)
        
        // set callbacks 
        easy_setopt(curl, HEADERCALLBACK, headerCB)
        easy_setopt(curl, HEADERDATA, req_id_ptr.cast[Ptr[Byte]])
        easy_setopt(curl, WRITECALLBACK, writeCB)
        easy_setopt(curl, WRITEDATA, req_id_ptr.cast[Ptr[Byte]])

        // run the query
        val res = easy_perform(curl) 

        // finish
        easy_cleanup(curl)

Adding an HTTP Client

To integrate with libuv, we use curl's multi API
Allows multiple requests to advance simultaneously
Allows either libcurl or external event loop to drive IO
Adds a handful of new functions and callbacks:

  type MultiCurl = Ptr[Byte]

  def multi_init():MultiCurl = extern
  def multi_add_handle(multi:MultiCurl, easy:Curl):Int = extern
  def multi_setopt(multi:MultiCurl, option:CInt, parameter:Any):CInt = extern
  
  def multi_assign(
    multi:MultiCurl, 
    socket:Ptr[Byte], 
    socket_data:Ptr[Byte]):Int = extern
  
  def multi_socket_action(
    multi:MultiCurl, 
    socket:Ptr[Byte], 
    events:Int, 
    numhandles:Ptr[Int]):Int = extern

  def multi_info_read(multi:MultiCurl, message:Ptr[Int]):Ptr[CurlMessage] = extern
  def multi_cleanup(multi:MultiCurl):Int = extern

Adding an HTTP Client

multi_socket_action allows us to drive curl with libuv events
multi_assign lets us associate custom data with a socket
libuv's poll handles can detect readiness without doing IO
libuv's timer can be set and reset dynamically by curl

  def multi_assign(
    multi:MultiCurl, 
    socket:Ptr[Byte], 
    socket_data:Ptr[Byte]):Int = extern
  
  def multi_socket_action(
    multi:MultiCurl, 
    socket:Ptr[Byte], 
    events:Int, 
    numhandles:Ptr[Int]):Int = extern

Adding an HTTP Client

multi_socket_action allows us to drive curl with libuv events
multi_assign lets us associate custom data with a socket
libuv's poll handles can detect readiness without doing IO
libuv's timer can be set and reset dynamically by curl

  type SocketCallback = Function5[Curl,      // curl easy handle
                                  Ptr[Byte], // socket
                                  CurlAction,// socket state
                                  Ptr[Byte], // custom CurlMulti data pointer 
                                  Ptr[Byte], // custom per-socket data pointer
                                  CInt]      // returns Int

  type TimerCallback = Function3[MultiCurl,  // curl multi handle
                                 Long,       // time to set next timer period
                                 Ptr[Byte],  // custom CurlMulti data pointer
                                 CInt]       // returns Int

  type CurlAction = CInt
  val POLL_NONE:CurlAction = 0
  val POLL_IN:CurlAction = 1
  val POLL_OUT:CurlAction = 2
  val POLL_INOUT:CurlAction = 3
  val POLL_REMOVE:CurlAction = 4

Adding an HTTP Client

curl creates one or more sockets.
curl notifies us that it has created new sockets.
we create pollhandles for libuv and start polling
libuv sees that a socket is ready and invokes curl
curl performs the appropriate transfer
curl checks to see if the request is complete.
If the request is complete, curl completes the request
When there are no more requests, we're done

Adding an HTTP Client

Request Setup

  val req_promises:mutable.Map[Long,Promise[ResponseState]] = mutable.HashMap.empty

  def get(url:CString, headers:Seq[String] = Seq.empty)
         (implicit ec:ExecutionContext):Future[ResponseState] = {
    req_count += 1
    activeRequests += 1
    val req_id = req_count
    val promise = Promise[ResponseState]()
    req_promises(req_id) = promise

    CurlInternals.beginRequest(req_id, url, headers)

    promise.future
  }

  def beginRequest(reqId:Long, url:CString, headers:Seq[String]):Unit = {
    val curlHandle = easy_init()
    val req_id_ptr = malloc(sizeof[Long]).cast[Ptr[Long]]
    !req_id_ptr = reqId
    requests(reqId) = ResponseState()

    easy_setopt(curlHandle, URL, url)
    easy_setopt(curlHandle, WRITECALLBACK, writeCB)
    easy_setopt(curlHandle, WRITEDATA, req_id_ptr.cast[Ptr[Byte]])
    easy_setopt(curlHandle, HEADERCALLBACK, headerCB)
    easy_setopt(curlHandle, HEADERDATA, req_id_ptr.cast[Ptr[Byte]])
    easy_setopt(curlHandle, PRIVATEDATA, req_id_ptr.cast[Ptr[Byte]])

    multi_add_handle(multi, curlHandle)
  }

Adding an HTTP Client

  // libuv provides helper functions to get the size of opaque structures
  val timer_size = uv_handle_size(UV_TIMER_T)
  // allocate a TimerHandle (really just a Ptr[Byte])
  val timer_handle:TimerHandle = malloc(timer_size)

  def set_timeout(curl:MultiCurl, timeout_ms:Long, data:Ptr[Byte]):Int = {
    val time = if (timeout_ms < 1) {
      // set timeout to minimum of 1, 0 causes problems
      1
    } else {
      timeout_ms
    }

    // we have a single global timer - this can set or reset its period
    uv_timer_start(timer_handle, timerCB, time, 0), "uv_timer_start")

    // complete any requests that have finished    
    cleanup_requests()
    0
  }

  // called by libuv when the timer fires
  def on_timeout(handle:TimerHandle):Unit = {
    val running_handles = stackalloc[Int]
    multi_socket_action(multi,-1.cast[Ptr[Byte]],0,running_handles)
    println(s"on_timer fired, ${!running_handles} sockets running")
  }
  val timerCB = CFunctionPtr.fromFunction1(on_timeout)

Adding an HTTP Client

def state_change(curl:Curl, socket:Ptr[Byte], action:Int, data:Ptr[Byte], 
                 socket_data:Ptr[Byte]):Int = {
    val pollHandle = if (socket_data == null) {
      val newHandle = malloc(uv_handle_size(UV_POLL_T)).cast[Ptr[Ptr[Byte]]
      !newHandle = socket
      uv_poll_init_socket(loop, newHandle, socket)
      multi_assign(multi, socket, newHandle.cast[Ptr[Byte]])
      newHandle
    } else {
      socket_data.cast[Ptr[Ptr[Byte]]]
    }

    val events = action match {
      case POLL_NONE => None
      case POLL_IN => Some(UV_READABLE)
      case POLL_OUT => Some(UV_WRITABLE)
      case POLL_INOUT => Some(UV_READABLE | UV_WRITABLE)
      case POLL_REMOVE => None
    }

    // update or stop the poll handle as needed
    events match {
      case Some(ev) =>
        uv_poll_start(pollHandle, ev, readyCB)
      case None =>
        uv_poll_stop(pollHandle)
        cleanup_requests()
    }
    0
  }

Adding an HTTP Client

  def on_socket_ready(pollHandle:PollHandle, status:Int, events:Int):Unit = {
    println(s"ready_for_curl fired with status ${status} and events ${events}")
    val socket = !(pollHandle.cast[Ptr[Ptr[Byte]]])
    val actions = (events & 1) | (events & 2) // Whoa, nelly!
    val running_handles = stackalloc[Int]
    val result = multi_socket_action(multi, socket, actions, running_handles)
    println("multi_socket_action",result)
  }
  val readyCB = CFunctionPtr.fromFunction3(on_socket_ready)

Adding an HTTP Client

  def cleanup_requests():Unit = {
    val messages = stackalloc[Int]
    val privateDataPtr= stackalloc[Ptr[Long]]
    var message:Ptr[CurlMessage] = multi_info_read(multi,messages)
    while (message != null) {
      val handle:Curl = !message._2
      easy_getinfo(handle, GET_PRIVATEDATA, privateDataPtr)
      val privateData = !privateDataPtr
      val reqId = !privateData
      val reqData = requests.remove(reqId).get
      Curl.complete_request(reqId,reqData)
      message = multi_info_read(multi,messages)
    }
  }

  def complete_request(reqId:Long, data:ResponseState):Unit = {
    val reqId = !request._4
    activeRequests -= 1
    println(s"completing reqId ${reqId}")
    val promise = Curl.req_promises.remove(reqId).get
    promise.success(data)
  }

Whew!

We've covered a LOT of ground
How do we go from slides to a sustainable project?
How do we build an ecosystem with splintering SN?

An Announcement

native-loop is moving into the Scala-Native GH org
Will live under scalanative.loop namespace
Artifacts available for SN 0.4M1 now!

Setting a Course

Improve support in SN for user-supplied event loop
STTP has great (blocking) curl/native support
Plan to spin out server API
Get feedback on streaming IO API

Building a Community

All of this needs contributors to be sustainable
There are a LOT of low-hanging fruit out there
Weekend-scale side projects can have a big impact!
Huge shout out to Scala Native's contributors
Get involved!

Backup Copy of Fast, Simple Concurrency with Scala Native

By Richard Whaling

Backup Copy of Fast, Simple Concurrency with Scala Native

Fast, Simple Concurrency with Scala Native

This talk is about:

Talk Outline

About me:

Scala Native is:

But that's not all!

Caveat

Example: Memory Allocation

Example: C FFI

Example: C FFI

Example: Function Pointers

LibUV's IO system

LibUV's event loop

BYO ExecutionContext

EventLoop and LoopExtensions

Our EventLoop

Our EventLoop

LoopExtensions

Timer

Timer

Timer

Timer

Timer

Timer

Streaming with libuv

Streaming with libuv

An API Sketch

An API Sketch

An API Sketch

libuv Callbacks

libuv Callbacks

libuv Callbacks

Tying it all together

Tying it all together

An API Sketch

Introducing native-loop

Example: Future

Example: HTTP Client

Example: HTTP Server

Adding an HTTP Server

Adding an HTTP Server

Adding an HTTP Server

Adding an HTTP Server

Adding an HTTP Server

Adding an HTTP Server

Adding an HTTP Server

Adding an HTTP Server

Adding an HTTP Server

Adding an HTTP Server

Adding an HTTP Server

Handling Requests

Adding an HTTP Client

Adding an HTTP Client

Adding an HTTP Client

Adding an HTTP Client

Adding an HTTP Client

Adding an HTTP Client

Adding an HTTP Client

Adding an HTTP Client

Adding an HTTP Client

Request Setup

Adding an HTTP Client

Adding an HTTP Client

Adding an HTTP Client

Adding an HTTP Client

Whew!

An Announcement

Setting a Course

Building a Community

Backup Copy of Fast, Simple Concurrency with Scala Native

More from Richard Whaling