The improbable story of:

FP AND ENTREPRENEURSHIP

Who ARE WE?

Xavier TORDOIR

ANDY PETRELLA

Studying =:= Future[Option[Job]]

Grab the basis

Find your ways

Pick some and dig

Mémoire: polishing phase

Job =:= (Work => Option[Experience]) => Option[Money]

Working will probably raise your experience

Based on this experience, you might get some money

Experience might not be enough

It will depend on the kind of experience

It's not that easy

Proposal:

Job =:= ( (Work, Explore, Learn) => Option[ Expertize ]) =>

Option[(Money, () => Future[Option[Job]] )]

Still some work *sight*

But never stop learning, nor exploring

Expertize is like experience, yet rarer

Money, the unfortunate obligation

Create your secret sauce, a lazy call to Job

Working in IT

Everything is distributed

Clues for answers are in data

Answer questions

More and More data

Industrial data

energy grid
production lines
finance
transport
...

More and More data

People data

web 1.0 : few control content
web 2.0 : lots of end-users manual feed
web x.y : lots of captors per user

More and More data

Why do we collect data?

We always miss some at some point
something of value is encoded
collaborative consiousness

Mobile? REALLY?

Providing content = aggregate distributed sources

BUT in small quantity (network is the bottleneck)

Yes data is available from anywhere

IT is a commodity

...even development

Everything done is distributed

Software (SaaS):

user management, social login
product support
online billing/payment
access to data sources

Infrastructure (IaaS): storage, servers...

A tech startup

Managing distributed resources is key!

Gathers distributed datasources
Calls distributed services
collects raw data
does some aggregation
Must be able to scale resources and costs

FP?

Asynchronous calls

FP describe the result you want, not how you get it

Failure management

Threads, cores, servers, datacenters

If FP gets a global adoption it will be because it solves issues related to the distributed environment.

Complexity of distribution

FP in the industry?

Examples of popular languages with OO and FP

javascript
ruby
Java 8
scala

Await[_]

Eval[Expression]

Scala

val getQueryParameter[T] = 
  (request:Request) =>
    (name:String) =>
      request.query.parameters.get(name).as[T]

abstract class Request[T](val body:Body[T]) 
  extends Query with Header {
  def parser:BodyParser[T]

  val content:Option[T] = parser(body)

  def send(to:Service) = ???
}

case class HttpRequest[T](
    method:Method, 
    body:Body)(implicit parser:BodyParser[T]) 
  extends Request(body) {
  // ...  
}

Play! Framework 2

Async HTTP

Event driven/Concurrency using Actor

Non-blocking IO (reactive: Iteratee/ CPS)

Stateless

object Dummy extends Controller {
  // return async'ly the # of users w/ name start'g w/ q"char"
  def countAs = Action.async { request =>
    val char = request.queryString("char")
    val all:Future[Seq[User]] = users.all // or WS.url
    Ok(all.map(_.count(_.name startsWith "a")))
  }

}

Akka

Message passing style

Messages are immutable

Actors are mutable (stateful)

Thread safety by confinement

Location transparency

class UserRepo extends Actor {
  var users = Set.empty

  def receive = {
    case Add(user:User) => 
      users = users + user
    case All => 
      sender ! users
    case Get(id:String) => 
      user = users.find(_.id == id)
      sender ! user
  }
}

object AkkaApp extends App {
  //...

  val repo = system.actorOf(Props[UserRepo])

  // add
  repo ! Add(User(1L, "")

  //get
  val user = repo ? Get(1L)
}

Spark

Tends to resolve Big Data solution flaws

Provides an abstraction on distributed data

Using an extended functional style (not only MR)

Represents a distributed computation as a Graph

In-Memory!

val sc = new SparkContext(/*...*/)
lines = sc.textFile("hdfs://...")
errors = lines.filter(_.startsWith("ERROR"))
errors.filter(_.contains("HDFS"))
      .map(_.split('\t')(3))
      .collect()