Your server as a function

http://monkey.org/~marius/funsrv.pdf

This qual­ity of be­ing able to store strain en­ergy and de­flect elas­ti­cally un­der a load with­out break­ing is called ‘re­silience’, and it is a veryvalu­able char­ac­ter­is­tic in a struc­ture. Re­silience may be de­fined as ‘the amount of strain en­ergywhich can be stored in a struc­ture with­out caus­ing per­ma­nent dam­age to it’.

— Gor­don, J.e.“Struc­tures.”

 

Futures
Services
Filters

Results of asyncronous operations are rapresented as composable futures, expressing the dependencies between operations

Systems boundaries are represented by asynchronous functions called services. Symmetric and uniform API represents both clients and servers

Application-agnostic concerns (e.g. timeouts, retries, authentication) are encapsulated by filters which compose to build services from multiple independent modules

FINAGLE

Finagle is an extensible RPC system for the JVM, used to construct high-concurrency servers. Finagle implements uniform client and server APIs for several protocols, and is designed for high performance and concurrency. Most of Finagle’s code is protocol agnostic, simplifying the implementation of new protocols.

1. connection pools, with throttling to avoid TCP connection churn
2. failure detectors, to identify slow or crashed hosts
3. failover strategies, to direct traffic away from unhealthy hosts
4. load-balancers, including “least-connections” and other strategies
5. back-pressure techniques, to defend servers against abusive clients and dogpiling.
6. standard statistics, logs, and exception reports
7. distributed tracing (a la Dapper) across protocols 
8. optionally uses ZooKeeper for cluster management
9. supports common sharding strategies.

server announcing itself to Zookeeper

client

val client = Http.newService("uwurfl.dev.williamhill.plc:10000")
or..
val client = Http.newService("uwurfl1.dev.wh.plc:10000,uwurfl2.dev.wh.plc:10000")

Await.ready(Httpx.serveAndAnnounce(":*", "zk!zk.dev.wh.plc:2181!/zktest!0", foo))

val client = Httpx.newService("zk2!localhost:2181!/zktest")
val req = RequestBuilder().url("http://foo").buildGet

client using namespace for discovery

LOOK MA, NO LOAD BALANCERS & TOTALY DYNAMIC! 😀

Any RPC protocol can be mapped onto Finagle

• Thrift
• Memcached
  
• MySQL
  
• ...

Unified Metrics
makes it easy to
understand what
you are monitorIng

(and they are always the same)

Look Pa, PErcentiles!

ZIPKIN

Trace all the things

register any finagle client/server with zipkin
val zipkinTracer = ZipkinTracer.mk("zipkin.dev.wh.plc", port = 9410, sampleRate = 1f)

Distributed tracing application based on the Google Dapper paper

Trace Identifiers

• Trace ID
• Span ID
• Parent ID

Common RPC annotations

cs - client start

sr - server receive
ss - server send
cr - client receive

http://research.google.com/pubs/pub36356.html

A common Language for TRAcing

http://www.slideshare.net/johanoskarsson/zipkin-strangeloop

https://blog.twitter.com/2013/observability-at-twitter

https://blog.twitter.com/2013/new-tweets-per-second-record-and-how

Twitter Server

TwitterServer defines a template from which servers at Twitter are built. It provides common application components such as an administrative HTTP server, tracing, stats, etc. These features are wired in correctly for use in production at Twitter.

Features

1. Flags (config)
2. Logging (standard)
3. Metrics exposed as JSON(standard)
4. HTTP admin interface
5. Lifecycle management

Admin console (built in)

FINATRA

Fast, testable, Scala HTTP services

Features

1. Flags (config)
2. Logging (standard)
3. Metrics (standard)
4. HTTP admin interface
5. Lifecycle management

Simple Controllers

Strongly defined Request model & Validation

class Example extends Controller {
  get("/") { request: Request =>
    "hi"
  }
}

http://foo.com/users?max=10&start_date=2014-05-30TZ&verbose=true

case class UsersRequest(
  @Max(100) @QueryParam max: Int,
  @PastDate @QueryParam startDate: Option[DateTime],
  @QueryParam verbose: Boolean = false)

Dependency Injection without XML

class MyController @Inject()(dao: GroupsDAO, service: FooService) extends Controller

WARMUP+AUTOMATIC SMOKE TEST

Send custom stats to local receiver

class WurflWarmupHandler @Inject()(httpWarmup: HttpWarmup) extends Handler {
  override def handle() = {
    httpWarmup.send(
      get("/v1/json"),
      responseCallback = expectOkResponse(_)
    )
  }
}

class MyService @Inject()(
  stats: StatsReceiver)

Injection of environmental variables via flags

object MyModule1 extends TwitterModule {
  val key = flag("key", "defaultkey", "The key to use.")

  @Singleton
  @Provides
  def providesThirdPartyFoo: ThirdPartyFoo = {
    new ThirdPartyFoo(key())
  }
}

Mustache for templated responses

FILTER CHAIN

@Mustache("foo")
case class FooView(
  name: String)

get("/foo") { request: Request =>
  FooView("abc")
}

get("/foo") { request: Request =>
  response.notFound(
    FooView("abc"))
}

class Server extends HttpServer {
  override configureHttp(router: HttpRouter) {
    router.
      filter[CommonFilters].
      filter[LoggingMDCFilter].
      filter[FinagleRequestScopeFilter].
      filter[UserFilter].
      add[MyController1].
      add[MyController2]
  }
}

Logging trait, no boilerplate

class MyClass extends Logging {
  def foo() = {
    info("Calculating...")
    "bar"
  }
}

Test framework included

import com.google.inject.Stage
import com.twitter.finatra.http.test.EmbeddedHttpServer
import com.twitter.inject.server.FeatureTest

class MyServiceStartupTests extends FeatureTest {
  val server = new EmbeddedHttpServer(
    stage = Stage.PRODUCTION,
    twitterServer = new SampleApiServer,
    extraArgs = Seq(
      "-com.twitter.server.resolverMap=myservice=nil!"))

  "SampleApiServer" should {
    "startup" in {
      server.assertHealthy()
    }
  }
}

Access Log (same pattern as apache)

service log (application Log)

0:0:0:0:0:0:0:1 - - [16/Oct/2015:00:59:45 +0000] "GET /v1/json HTTP/1.1" 200 202 5 "curl/7.43.0" 3dad52fbec16da6a
0:0:0:0:0:0:0:1 - - [16/Oct/2015:00:59:46 +0000] "GET /v1/json HTTP/1.1" 200 202 1 "curl/7.43.0" 520d2b910eef9eea

2015-10-16 02:59:26,604 INFO                  HttpWarmup                Warmup Request("GET /v1/json", from 0.0.0.0/0.0.0.0:0) complete with Status(200)
2015-10-16 02:59:27,138 INFO                  DefaultTracer$            Tracer: com.twitter.finagle.zipkin.thrift.SamplingTracer
2015-10-16 02:59:27,145 INFO                  WurflServerMain$          http server started on port: 8888
2015-10-16 02:59:27,146 INFO                  WurflServerMain$          Enabling health endpoint on port 9990
2015-10-16 02:59:27,146 INFO                  WurflServerMain$          App started.
2015-10-16 02:59:27,147 INFO                  WurflServerMain$          Startup complete, server ready.
2015-10-16 02:59:45,185 INFO 3dad52fbec16da6a WurflController           Entering /v1/json with curl/7.43.0
2015-10-16 02:59:45,186 DEBUG 3dad52fbec16da6a WurflServiceImpl          Looking up capabilities.
2015-10-16 02:59:45,187 INFO 3dad52fbec16da6a ScientiaWurflEngine       capabilities=[generic_curl_based_crawler, match=conclusive]
2015-10-16 02:59:45,187 INFO 3dad52fbec16da6a WurflController           Capabilities found DeviceCapabilities(Map(device_os -> , is_wireless_device -> false, physical_screen_height -> 400, is_mobile -> false, is_tablet -> false, device_os_version -> , physical_screen_width -> 400))
2015-10-16 02:59:46,024 INFO 520d2b910eef9eea WurflController           Entering /v1/json with curl/7.43.0
2015-10-16 02:59:46,024 DEBUG 520d2b910eef9eea WurflServiceImpl          Looking up capabilities.
2015-10-16 02:59:46,025 INFO 520d2b910eef9eea ScientiaWurflEngine       capabilities=[generic_curl_based_crawler, match=cached]
2015-10-16 02:59:46,025 INFO 520d2b910eef9eea WurflController           Capabilities found DeviceCapabilities(Map(device_os -> , is_wireless_device -> false, physical_screen_height -> 400, is_mobile -> false, is_tablet -> false, device_os_version -> , physical_screen_width -> 400))
2015-10-16 02:59:46,633 INFO aa0cbfd36849d955 WurflController           Entering /v1/json with curl/7.43.0
2015-10-16 02:59:46,633 DEBUG aa0cbfd36849d955 WurflServiceImpl          Looking up capabilities.
2015-10-16 02:59:46,633 INFO aa0cbfd36849d955 ScientiaWurflEngine       capabilities=[generic_curl_based_crawler, match=cached]
2015-10-16 02:59:46,634 INFO aa0cbfd36849d955 WurflController           Capabilities found DeviceCapabilities(Map(device_os -> , is_wireless_device -> false, physical_screen_height -> 400, is_mobile -> false, is_tablet -> false, device_os_version -> , ph

With MDC in the logs I can follow the requests!

UNIFICATION OF LOGS

UWURFL Customisations

Bridged the Twitter Metrics to Dropwizard metrics and then exported to JMX in order to integrate with Wily

https://github.com/rlazoti/finagle-metrics

https://dropwizard.github.io/

UWURFL Customisations

Exported Dropwizard metrics to Graphite and then Graphana (free)

SBT-native-packager

Universal deployment

μWURFL

All of this in order to get a bit of json

curl -H "user-agent: Mozilla/5.0 (iPhone; U; CPU like Mac OS X; en) Mobile/1A543a Safari/419.3" localhost:8888/v1/json

{
   "timestamp" : 1445003791106,
   "capabilities" : {
      "is_mobile" : true,
      "device_os" : "iOS",
      "device_os_version" : 1,
      "is_tablet" : false,
      "is_wireless_device" : true,
      "physical_screen_height" : 74,
      "physical_screen_width" : 50
   }
}