Taming The Beast of Asynchronous PHP

Oleksii Petrov
DreamTeam

Who am I? 

Solution Acrhitect

Back End Tech Lead

PHP Developer

Find me on

@alexhelkar

alexhelkar

https://github.com/alexhelkar

Developer's Toolbox

Parallel

Serial

Multi-threaded

Single-threaded

Concurrent

Monopoly

Synchronous

Asynchronous

Serial vs Parallel

Tasks

t

Tasks

Parallel Processing

Single- vs Multi-threaded

Tasks

Process

CPU

Execution Thread

Memory

Tasks

Process

CPU

Execution Thread

Memory

Execution Thread

Multithreaded vs Parallel

Tasks

Process

CPU

Execution Thread

Memory

Execution Thread

Tasks

Process

CPU

Execution Thread

Memory

Execution Thread

Monopoly vs Concurrent

Tasks

Processing

Processing

Processing

Summary

Serial or Parallel

Execution

Single- or Multithreading

Planning

Monopoly or Concurrent

Access

Synchronous or Asynchronous

Communication

Synchronous vs Asynchronous

Tasks

Tasks

Processing

Processing

Asynchronous Morning

  • Make some food
  • Make a cup of tea
  • Brush your teeth

Morning Routine

Brush Teeth

Make tea

Make food

Tasks

Brush Teeth

Make tea

Make food

Estimated Tasks

4 min.

5 min.

3 min.

Queue Tasks

Brush Teeth

Make tea

Make food

Brush Teeth

Make tea

Make food

Make food

Worker

Make tea

Brush Teeth

4 min

5 min

3 min

12 minutes

Schedule Tasks

Serial

Hell No!

Brush Teeth

Make tea

Make food

Make food

Worker

Make tea

Brush Teeth

4 min

5 min

3 min

5 minutes

Schedule Tasks

Parallel

Worker

Worker

People don't do things in parallel

Brush Teeth

Make tea

Make food

~ 5 minutes

Schedule Tasks

Serial Asynchronous

Worker

Food

Make tea

Brush Teeth

Microwave

Kettle

Blocking Operation

Non-Blocking Operation

Know your Tools

Blocking Operation

Non-Blocking Operation

Know your Tools

Why to write

Async Programs?

Single Process Blocking Server

Multiprocess Blocking Server

Multiprocess Blocking Server

...

...

CGI

Issues?

Context Switch

CPU

CPU

Process State
Process number
CPU Scheduling info
Registers
List of open files
....
Priority
Memory limit
Process State
Process number
CPU Scheduling info
Registers
List of open files
....
Priority
Memory limit

Save

Restore

Invalidate

CPU Cache

Timeline

0%

100%

Context Switch Problem

CPU

0%

100%

Timeline

Multiprocess vs Multithreaded

Timeline

Profit

Threads

0%

100%

0%

100%

0%

100%

Now We Can Run More Of These, Right?

DDoS

Multithreaded Blocking Server

FCGI

Could be better?

Process Queue

CPU

Ready Queue

0%

100%

I/O operation

I/O Request Queue

I/O Queue

I/O

100 ms

Process

0%

100%

What is that?

  • Parse params
  • Validate
  • Go to DB
  • Do stuff
  • Create a response

Process

0%

100%

  • Parse params
  • Validate
  • Write some logs
  • Go to DB
  • Do stuff
  • Write some more logs
  • Create response

I/O Wait

Multithreaded Blocking Server

Async Approach

Async Approach

Event Loop

Worker

Asynchronous Server
Let you serve more clients

Sync Approach

https://www.nginx.com/blog/thread-pools-boost-performance-9x/

Async Approach

https://www.nginx.com/blog/thread-pools-boost-performance-9x/

Reactor Pattern

Reactor Pattern

NodeJS Architecture

NodeJS Architecture

Multithreaded Event Loop

Multi-Reactor Pattern

Vert.x Architecture

Nginx Architecture

https://www.nginx.com/blog/thread-pools-boost-performance-9x/

How to write Asynchronous PHP Programs?

Async Programming in PHP

Brave New World

We went with Swoole

Swoole

Written in C

 

PHP Extension

 

Async I/O

 

Reactor Pattern

 

Super fast

 

Coroutines

 

HTTP Server

 

Task Queues

 

MMap files

 

Shared hash table

 

Event loop API

 

Websocket Server

 

Sync/Async Workers

 

Async Redis/MySQL/DNS/Http clients

 

Simple Benchmark, Node.js

var http = require('http');
http.createServer(function (req, res) {
    res.writeHead(200, {
        'Server': "node.js"}
    );
    res.end("<h1>Hello World</h2>");
}).listen(8080, '127.0.0.1');
console.log('Server running at http://127.0.0.1:8080/');

https://github.com/swoole/swoole-src/blob/master/benchmark/http.js

Benchmark Results, Node.js

Simple Benchmark, Golang

func main() {
    runtime.GOMAXPROCS(runtime.NumCPU() - 1)

    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        w.Header().Add("Last-Modified", "Thu, 18 Jun 2015 10:24:27 GMT")
        w.Header().Add("Accept-Ranges", "bytes")
        w.Header().Add("E-Tag", "55829c5b-17")
        w.Header().Add("Server", "golang-http-server")
        fmt.Fprint(w, "<h1>\nHello world!\n</h1>\n")
    })

    log.Printf("Go http Server listen on :8080")
    log.Fatal(http.ListenAndServe(":8080", nil))
}

https://github.com/swoole/swoole-src/blob/master/benchmark/http.go

Benchmark Results, Golang

Simple Benchmark, PHP/Swoole

<?php
$http = new swoole_http_server("127.0.0.1", 9501, SWOOLE_BASE);

$http->set([
    'worker_num' => 4,
]);

$http->on('request', function ($request, swoole_http_response $response) {
    $response->header('Last-Modified', 'Thu, 18 Jun 2015 10:24:27 GMT');
    $response->header('E-Tag', '55829c5b-17');
    $response->header('Accept-Ranges', 'bytes');
    $response->end("<h1>\nHello Swoole.\n</h1>");
});

$http->start();

https://github.com/swoole/swoole-src/blob/master/benchmark/http.php

Benchmark Results, PHP/Swoole

Questions?

Why node.js is so slow?

Single threaded web-server

Why golang is so slow?

Single threaded event-loop

Why swoole is so fast?

Multi-process event-loop

Traditional PHP Lifecycle

Swoole PHP Lifecycle

All in memory

Swoole Design

Async Primitives

Timers

Callbacks/Promises

Coroutines

Async/Await

Callbacks

$db = new swoole_mysql();
$server = ['host' => '192.168.56.102'];

$db->connect($server, function ($db, $r) {
    if ($r === false) {
        die($db->connect_error);
    }

    echo "Connected to DB";
});

Callbacks Nested

$db = new swoole_mysql();
$server = array('host' => '192.168.56.102');

$db->connect($server, function ($db, $r) {
    if ($r === false) { die( $db->connect_error); }

    $sql = 'show tables';

    $db->query($sql, function(swoole_mysql $db, $r) {
        if ($r === false){
            var_dump($db->error, $db->errno);
        } elseif ($r === true ){
            var_dump($db->affected_rows, $db->insert_id);
        }
        var_dump($r);
        $db->close();
    });
});

Coroutines, No I/O Waiting

echo "main start\n";
go(function () {
    echo "coro ".co::getcid()." start\n";
});
echo "end\n";

/*
main start
coro 1 start
end
*/

Coroutines, With I/O Waiting

echo "main start\n";
go(function () {
    echo "coro ".co::getcid()." start\n";
    co::sleep(.1); //switch at this point
    echo "coro ".co::getcid()." end\n";
});
echo "end\n";
/*
main start
coro 1 start
end
coro 1 end
*/

Coroutine Clients

$mysql = new Swoole\Coroutine\MySQL();
$mysql->connect([
    'host' => '127.0.0.1',
    'user' => 'user',
]);
$mysql->setDefer();
$mysql->query('select sleep(1)');

// ...

$result = $mysql->recv();

Blocking Operations in PHP

All I/O operations in PHP are blocking

  • Mysql, mysqli, pdo, and other DB operation functions
  • sleep, usleep
  • curl
  • Stream, socket extension function
  • Memcache, redis extension function
  • File_get_contents/fread and other file read functions

Coroutine Hooks

Swoole\Runtime::enableCoroutine();

go(function () {
    $redis = new redis;
    $retval = $redis->connect("127.0.0.1", 6379);
    var_dump($retval, $redis->getLastError());
    var_dump($redis->get("key"));
    var_dump($redis->set("key", "value2"));
    var_dump($redis->get("key"));
    $redis->close();
});

Available Hooks

  • Redis Extension
  • Mysqli Extension
  • Mysqlnd/PDO
  • SOAP Extension
  • file_get_contents, fopen
  • stream_socket_client (predis)
  • stream_socket_server
  • fsockopen

Not Available Hooks

  • mysql extension
  • curl extension
  • mongo extension
  • pdo_pgsql
  • pdo_ori
  • pdo_odbc
  • pdo_firebird

Useful Links

  • https://github.com/swooletw/awesome-swoole
  • https://github.com/swoole/ide-helper
  • https://wiki.swoole.com/

Don't Afraid the Docs

Should we all go async?

Not yet...

Questions?

Could use Swoole in Prod?

In a blocking mode - Hell, yeah!

One of our APIs

Swoole ❤️ Symfony

https://github.com/k911/swoole-bundle

Swoole ❤️ Symfony

https://github.com/k911/swoole-bundle

swoole:
    http_server:
        port: 9501
        host: 0.0.0.0
        running_mode: process
        settings:
            reactor_count: 2
            worker_count: 4

Common Issues

With Long Running Apps

Resources Leaks

Memory Leaks

Our Issues

With Long Running Apps

Resources Leaks

Memory Leaks

Memory Leaks

Preventing Memory Leaks

/**
 * {@inheritdoc}
 *
 * @throws \Exception
 */
public function handle(SwooleRequest $request, SwooleResponse $response): void
{
    $httpFoundationRequest = $this->requestFactory->make($request);
    $httpFoundationResponse = $this->kernel->handle($httpFoundationRequest);
    $this->responseProcessor->process($httpFoundationResponse, $response);

    if ($this->kernel instanceof TerminableInterface) {
        $this->kernel->terminate($httpFoundationRequest, $httpFoundationResponse);
    }
}

Contracts Component

Reset Interface

<?php
/**
 * Provides a way to reset an object to its initial state.
 *
 * When calling the "reset()" method on an object, it should be put back to its
 * initial state. This usually means clearing any internal buffers and forwarding
 * the call to internal dependencies. All properties of the object should be put
 * back to the same state it had when it was first ready to use.
 *
 * This method could be called, for example, to recycle objects that are used as
 * services, so that they can be used to handle several requests in the same
 * process loop (note that we advise making your services stateless instead of
 * implementing this interface when possible.)
 */
interface ResetInterface
{
    public function reset();
}

Service Resetter

abstract class Kernel implements KernelInterface, RebootableInterface, TerminableInterface 
{    
    /**
     * {@inheritdoc}
     */
    public function boot()
    {
        if (true === $this->booted) {
            if (!$this->requestStackSize && $this->resetServices) {
                if ($this->container->has('services_resetter')) {
                    $this->container->get('services_resetter')->reset();
                }
                $this->resetServices = false;
                if ($this->debug) {
                    $this->startTime = microtime(true);
                }
            }

            return;
        }
     
        // ...
        $this->booted = true;
    }

More Radical Approach...


/**
 * https://roadrunner.dev/docs/integrations-symfony
 */
while ($req = $psr7->acceptRequest()) {
    try {
        $request = $httpFoundationFactory->createRequest($req);
        $response = $kernel->handle($request);
        $psr7->respond($diactorosFactory->createResponse($response));
        $kernel->terminate($request, $response);
        $kernel->reboot(null);
    } catch (\Throwable $e) {
        $psr7->getWorker()->error((string)$e);
    }
}

Kernel Reboot

public function reboot($warmupDir) {
    $this->shutdown();
    $this->warmupDir = $warmupDir;
    $this->boot();
}

public function shutdown() {
    if (false === $this->booted) {
        return;
    }

    $this->booted = false;
    foreach ($this->getBundles() as $bundle) {
        $bundle->shutdown();
        $bundle->setContainer(null);
    }

    $this->container = null;
    $this->requestStackSize = 0;
    $this->resetServices = false;
}

Doctrine EM Clear

namespace K911\Swoole\Bridge\Doctrine\ORM;

final class EntityManagerHandler implements RequestHandlerInterface
{
    /**
     * {@inheritdoc}
     */
    public function handle(Request $request, Response $response): void
    {
        if (!$this->connection->ping()) {
            $this->connection->close();
            $this->connection->connect();
        }

        $this->decorated->handle($request, $response);

        $this->entityManager->clear();
    }
}

Resources Leaks

Network Connections 

App

Load Balancer

RabbitMQ

6 min

  • Added Connect Retry to Producers
  • Added Heartbeat settings for RabbitMQ Bundle

Network Connections 

RabbitMQ

Consumer

  • Separate Connections for Consumers/Producers
  • Longer Heartbeats for Consumers

Service Response Time

Linear scale

Service Response Time

Log scale

What could be a root cause?

  • Application?
  • Api Gateway?
  • Kubernetes?
  • Services/Balancers?
  • Host network?
  • Bad weather?

What did we do?

  • Debug Application (local)
  • Logs enhancements
  • Load tests
  • Volume tests
  • Stabillity tests

What did we find out?

Nothing...

App Tuning

<?php

$serv = new swoole_server("127.0.0.1", 9501);
$serv->set(array(
    'worker_num' => 4,    
    'max_request' => 1000,  
    'dispatch_mode'=>3,
));

Release Planning

Release Date

Release Issues

Liveness and Readiness Probes

spec:
  containers:
  - name: liveness
    livenessProbe:
      httpGet:
        path: /healthz
        port: 8080
      initialDelaySeconds: 10
      periodSeconds: 5
    readinessProbe:
      httpGet:
        path: /healthz
        port: 8080
      initialDelaySeconds: 10
      periodSeconds: 5
    

LiipMonitorBundle

<?php
namespace Liip\Monitor\Check;
class RabbitMQCheck extends Check {
    /**
     * {@inheritdoc}
     * @see \Liip\MonitorBundle\Check\CheckInterface::check()
     */
    public function check() {
        try {
            $conn = new AMQPConnection(
                $this->host,
                $this->port,
                $this->user,
                $this->password,
                $this->vhost
            );
            $ch = $conn->channel();
            $result = $this->buildResult('OK', CheckResult::OK);
        } catch (\Exception $e) {
            $result = $this->buildResult($e->getMessage(), CheckResult::CRITICAL);
        }

        return $result;
    }
}

Zend Diagnostrics

<?php
namespace ZendDiagnostics\Check;
class RabbitMQ extends AbstractCheck {
    /**
     * @see \ZendDiagnostics\Check\CheckInterface::check()
     * @return Failure|Success
     */
    public function check()
    {
        if (! class_exists('PhpAmqpLib\Connection\AMQPConnection')) {
            return new Failure('PhpAmqpLib is not installed');
        }

        $conn = new AMQPConnection(
            $this->host,
            $this->port,
            $this->user,
            $this->password,
            $this->vhost
        );

        $conn->channel();

        return new Success();
    }
}

Thanks

Taming The Beast of Asynchronous PHP

By Oleksii Petrov

Taming The Beast of Asynchronous PHP

  • 2,206