(Ab)using process control for powerful CLI applications

AustinPHP October 2018

Ian Littman / @iansltx

follow along at https://ian.im/pcntl18aus

Not a normal PHP CLI TAlk

What We'll Cover

  • Signal Handling (you may have seen some of this before)
  • Concurrent execution
    • forking
    • threading (briefly, via pthreads and amphp)
    • child processes, with e.g. Symfony Process
  • (bonus) Async execution in a web request context

 

Grab code + a container with required extensions at

github.com/iansltx/pcntl-demo

POSIX-y things ahead

...but some pieces work on Windows as well

Signals

Kill -9 runaway-process

 

github.com/iansltx/pcntl-demo

available from the CLI

 

github.com/iansltx/pcntl-demo

pcntl_signal(signal, callback)

  • signal == integer signal number (use built-in constants!)
    • SIGINT (ctrl-c)
    • SIGHUP (restart)
    • SIGTERM (normal kill behavior)
    • SIGCONT, SIGUSR1, etc.
    • SIGKILL (kill -9)
  • Callback works with any callable

getmypid() or posix_getpid()
posix_kill(pid, signal)

Signal handlers are one level deep

Listening for signals

Masking signals with pcntl_sigprocmask()

  • SIG_BLOCK queues up one or more signal types rather than dispatching them as they come in
  • SIG_UNBLOCK stops queueing and dispatches any queued signals of the specified type immediately
  • SIG_SETMASK swaps in a new list of signals to block
  • Assumes you're dispatching signals somehow

Waiting on signals

  • Can wait with or without a timeout
    • pcntl_sigtimedwait() with timeout
    • pcntl_sigwaitinfo() without
  • Don't need to use ticks/dispatch/etc.
  • Signal callbacks MUST be set but WILL NOT be called
  • Can wait on a set of signals (e.g. SIGCONT + SIGHUP)
  • Consumes any queued signals from pcntl_sigprocmask()
  • Unavailable on macOS, FreeBSD
  • pcntl_alarm(int)
    • Fires a SIGARLM at this process in int seconds
    • Replaces any set, pending alarm
  • If the script is sleep()ing, ends that statement early (other signals behave the same way)

Putting it all together

CONCURR TASKSENT

Concurrent Tasks

  • Copies RAM snapshot to a child process, continues executing at the same point in both parent and child
  • switch ($ret = pcntl_fork())
    • case -1: // fork failed
    • case 0: // in child process
    • default: // in parent process; $ret is the child PID
  • Wait for child processes to exit via pcntl_waitpid()
  • Each child process will need its own DB connection
$pidsByTable = [];
foreach (array_keys(static::TABLE_TO_QUERY_MAPPING) as $table) {
    $pid = pcntl_fork();
    if ($pid === -1) { // fork failed
        $this->logger->warning("Couldn't fork process for " . $table); continue;
    } elseif ($pid) { // in parent
        $pidsByTable[$table] = $pid; continue;
    } // everything below this line in this loop is in the child process
    
    \DB::purge(); // we need to create a new connection in the forked process
    $queryClass = static::TABLE_TO_QUERY_MAPPING[$table];
    $this->logger->debug('Starting to warm cache for ' . $table . ' in pid ' . $pid);
    (new $queryClass(\DB::connection(), $this->logger))->warmCache();
    $this->logger->debug('Warmed cache for ' . $table . ' in pid ' . $pid);
    die(0); // immediately exit from child processes so we don't have additional forks
}
foreach ($pidsByTable as $table => $pid) { // if we're here, we're in the parent
    pcntl_waitpid($pid, $status);
    $this->logger->debug("Cache warm process for $table ($pid) done w\status $status");
}

Real world example

  • PHP Extension
  • Current version is PHP 7.2+
  • Requires ZTS (Zend Thread Safe) build of PHP
  • Threads, Threadables (data stores), Workers, Stackables
  • You have to manage your own reference counting
    (keep vars in scope)
  • As with forking, one DB connection per thread
  • Dockerfile available in the companion code repo
  • Examples available in its repo...
  • ...and in ours
  • Uses generators/coroutines
  • Inter-process communication built in
  • Will use threads for workers if available via pthreads
  • Uses shared memory via the shmop extension
  • Synchronizes access via mutexes with the sysvmsg ext
  • Examples available...
  • ...including one in our repo

sleepandwrite.php

#!/usr/bin/env php
<?php

if ($argc < 4) {
    die("Usage: ./sleepAndWrite.php sleep_in_seconds filename output\n");
}

sleep($argv[1]);
error_log('Done sleeping. Writing contents to file.');
file_put_contents($argv[2], implode(' ', array_slice($argv, 3)));

composer require symfony/process

Background Processing in-request

nohup cmd > /dev/null 2>&1 &

Doesn't quite work right in php -S

you may want a queue instead

Thanks! QUestions?

(Ab)using process control for powerful CLI applications - AustinPHP October 2018

By Ian Littman

(Ab)using process control for powerful CLI applications - AustinPHP October 2018

  • 206
Loading comments...

More from Ian Littman