(Ab)using process control for powerful CLI applications

AustinPHP October 2018

Ian Littman / @iansltx

follow along at https://ian.im/pcntl18aus

Not a normal PHP CLI TAlk

Steve Grunwell has a presentation about that

What We'll Cover

• Signal Handling (you may have seen some of this before)
• Concurrent execution
  • forking
  • threading (briefly, via pthreads and amphp)
  • child processes, with e.g. Symfony Process
• (bonus) Async execution in a web request context

Grab code + a container with required extensions at

github.com/iansltx/pcntl-demo

POSIX-y things ahead

...but some pieces work on Windows as well

Signals

Kill -9 runaway-process

github.com/iansltx/pcntl-demo

pcntl

available from the CLI

github.com/iansltx/pcntl-demo

pcntl_signal(signal, callback)

• signal == integer signal number (use built-in constants!)
  • SIGINT (ctrl-c)
  • SIGHUP (restart)
  • SIGTERM (normal kill behavior)
  • SIGCONT, SIGUSR1, etc.
  • SIGKILL (kill -9)
• Callback works with any callable
  • 1st arg: integer signal #
  • 2nd arg (PHP 7.1+): SIGINFO_T struct, as an array

getmypid() or posix_getpid()
posix_kill(pid, signal)

Signal handlers are one level deep

Listening for signals

• pcntl_async_signals(true) - Fast, automatic, >= 7.1.0
• pcntl_signal_dispatch() - Fast, manual, >= 5.3.0
• declare(ticks = 1) - Slow, >= 4.3.0

Masking signals with pcntl_sigprocmask()

• SIG_BLOCK queues up one or more signal types rather than dispatching them as they come in
• SIG_UNBLOCK stops queueing and dispatches any queued signals of the specified type immediately
• SIG_SETMASK swaps in a new list of signals to block
• Assumes you're dispatching signals somehow

Waiting on signals

• Can wait with or without a timeout
  • pcntl_sigtimedwait() with timeout
  • pcntl_sigwaitinfo() without
• Don't need to use ticks/dispatch/etc.
• Signal callbacks MUST be set but WILL NOT be called
• Can wait on a set of signals (e.g. SIGCONT + SIGHUP)
• Consumes any queued signals from pcntl_sigprocmask()
• Unavailable on macOS, FreeBSD

SIGALRM

• pcntl_alarm(int)
  • Fires a SIGARLM at this process in int seconds
  • Replaces any set, pending alarm
• If the script is sleep()ing, ends that statement early (other signals behave the same way)

Putting it all together

zombie.php

CONCURR TASKSENT

Concurrent Tasks

pcntl_fork()

• Copies RAM snapshot to a child process, continues executing at the same point in both parent and child
• switch ($ret = pcntl_fork())
  • case -1: // fork failed
  • case 0: // in child process
  • default: // in parent process; $ret is the child PID
• Wait for child processes to exit via pcntl_waitpid()
• Each child process will need its own DB connection

$pidsByTable = [];
foreach (array_keys(static::TABLE_TO_QUERY_MAPPING) as $table) {
    $pid = pcntl_fork();
    if ($pid === -1) { // fork failed
        $this->logger->warning("Couldn't fork process for " . $table); continue;
    } elseif ($pid) { // in parent
        $pidsByTable[$table] = $pid; continue;
    } // everything below this line in this loop is in the child process
    
    \DB::purge(); // we need to create a new connection in the forked process
    $queryClass = static::TABLE_TO_QUERY_MAPPING[$table];
    $this->logger->debug('Starting to warm cache for ' . $table . ' in pid ' . $pid);
    (new $queryClass(\DB::connection(), $this->logger))->warmCache();
    $this->logger->debug('Warmed cache for ' . $table . ' in pid ' . $pid);
    die(0); // immediately exit from child processes so we don't have additional forks
}
foreach ($pidsByTable as $table => $pid) { // if we're here, we're in the parent
    pcntl_waitpid($pid, $status);
    $this->logger->debug("Cache warm process for $table ($pid) done w\status $status");
}

Real world example

pthreads

• PHP Extension
• Current version is PHP 7.2+
• Requires ZTS (Zend Thread Safe) build of PHP
• Threads, Threadables (data stores), Workers, Stackables
• You have to manage your own reference counting
  (keep vars in scope)
• As with forking, one DB connection per thread
• Dockerfile available in the companion code repo
• Examples available in its repo...
• ...and in ours

amphp/parallel

• Uses generators/coroutines
• Inter-process communication built in
• Will use threads for workers if available via pthreads
• Uses shared memory via the shmop extension
• Synchronizes access via mutexes with the sysvmsg ext
• Examples available...
• ...including one in our repo

sleepandwrite.php

#!/usr/bin/env php
<?php

if ($argc < 4) {
    die("Usage: ./sleepAndWrite.php sleep_in_seconds filename output\n");
}

sleep($argv[1]);
error_log('Done sleeping. Writing contents to file.');
file_put_contents($argv[2], implode(' ', array_slice($argv, 3)));

composer require symfony/process

Background Processing in-request

fastcgi_finish_request()

• Anything after call doesn't block response
• php-fpm only (function doesn't exist on php -S)
• Ties up an FPM worker process
• Don't expect anything to print after calling
• If you're logging to STDERR and picking it up on your web server, you'll lose that too

Example: raphple

exact code changes here

nohup

nohup cmd > /dev/null 2>&1 &

Doesn't quite work right in php -S

you may want a queue instead

Thanks! QUestions?

• ian.im/pcntl18aus - these slides
• github.com/iansltx/pcntl-demo - most code
• github.com/iansltx/raphple - raffle code
• twitter.com/iansltx - me
• github.com/iansltx - my code