Serverless Benchmarks

Round 2

 

Sydney Lambda Meetup September 2018

About Me Us

Round 1
Round 2
Winner

2018 Round 1 Multi Lingual

Round 1 Feedback

Done

  • Lambda Signatures were inconsistent
  • Java had way too much overhead, log4j etc
  • No cold start
  • No detailed metrics on median, p95, p99 etc

Pending

  • No actual workload, add downstream service call like DynamoDB or S3
  • .Net Core is fast? That's unpossible!
  • Add other JVM flavours Scala/Kotlin since F# maded it in
  • Add Google Cloud and Azure

Not Doing

  • Not "realistic"! Add universal "realistic" scenarios

Round 1 Results

  1. .Net 2.0 Outperforms
  2. Go and Java is about the same
  3. Consistent Performance between Dynamic vs Static
  4. Go Packages are bigger than Java

Round 2

Including Cold Starts

Round 2 Detailed Results

Runtime Min Max Median P95 P99
aws-csharp2 242ms 8.447s 331.6ms 1.132s 5.332s
aws-csharp2.1 243ms 11.078s 328ms 1.247s 5.297s
aws-fsharp2 237ms 10.938s 338ms 1.340s 5.328s
aws-fsharp2.1 238ms 11.120s 329ms 1.762s 5.306s
aws-go 243ms 11.390s 332ms 1.262s 5.301s
aws-java8 241ms 7s 335ms 1.095s 5.284s
aws-node6 238ms 9.132s 327ms 1.078s 5.285s
aws-node8 231ms 8.558s 324ms 1.064s 5.272s
aws-python2 239ms 8.516s 326ms 1.316s 5.307s
aws-python3 242ms 10.852s 337ms 1.430s 5.326s
gcp-node8 267ms 17.127s 305ms 7s 11.342s

The Winner

Is You!

The median results are quite consistent between runtimes

 

But the real benefit of Serverless is in the paradigm shift towards Simplified Operating Model and Focus on Business Value

Questions

Serverless Benchmarks 2018 Round 2

By Yun Zhi Lin

Serverless Benchmarks 2018 Round 2

  • 1,555