(Ab)using process control for powerful CLI applications
Not a normal PHP CLI TAlk
What We'll Cover
- Signal Handling
- Concurrent execution
- forking
- threading (briefly, via pthreads and amphp)
- Child processes, with e.g. Symfony Process
- (bonus) Async execution in a web request context
Grab code + a container with required extensions at
POSIX-y things ahead
...but some pieces work on Windows as well
Signals
pcntl_signal(signal, callback)
- signal == integer signal number (use built-in constants!)
- SIGINT (ctrl-c)
- SIGHUP (restart)
- SIGTERM (normal kill behavior)
- SIGCONT, SIGUSR1, etc.
SIGKILL (kill -9)
- Callback works with any callable
- 1st arg: integer signal #
- 2nd arg (PHP 7.1+): SIGINFO_T struct, as an array
getmypid() or posix_getpid()
posix_kill(pid, signal)
Signal handlers are one level deep
Listening for signals
- pcntl_async_signals(true) - Fast, automatic, >= 7.1.0
- pcntl_signal_dispatch() - Fast, manual, >= 5.3.0
- declare(ticks = 1) - Slow, >= 4.3.0
Masking signals with pcntl_sigprocmask()
- SIG_BLOCK queues up one or more signal types rather than dispatching them as they come in
- SIG_UNBLOCK stops queueing and dispatches any queued signals of the specified type immediately
- SIG_SETMASK swaps in a new list of signals to block
- Assumes you're dispatching signals somehow
Waiting on signals
- Can wait with or without a timeout
- pcntl_sigtimedwait() with timeout
- pcntl_sigwaitinfo() without
- Don't need to use ticks/dispatch/etc.
- Signal callbacks MUST be set but WILL NOT be called
- Can wait on a set of signals (e.g. SIGCONT + SIGHUP)
- Consumes any queued signals from pcntl_sigprocmask()
- Unavailable on macOS, FreeBSD
- pcntl_alarm(int)
- Fires a SIGARLM at this process in int seconds
- Replaces any set, pending alarm
- If the script is sleep()ing, ends that statement early (other signals behave the same way)
Putting it all together
CONCURR TASKSENT
Concurrent Tasks
- Copies RAM snapshot to a child process, continues executing at the same point in both parent and child
- switch ($ret = pcntl_fork())
- case -1: // fork failed
- case 0: // in child process
- default: // in parent process; $ret is the child PID
- Wait for child processes to exit via pcntl_waitpid()
- Each child process will need its own DB connection
$pidsByTable = [];
foreach (array_keys(static::TABLE_TO_QUERY_MAPPING) as $table) {
$pid = pcntl_fork();
if ($pid === -1) { // fork failed
$this->logger->warning("Couldn't fork process for " . $table); continue;
} elseif ($pid) { // in parent
$pidsByTable[$table] = $pid; continue;
} // everything below this line in this loop is in the child process
\DB::purge(); // we need to create a new connection in the forked process
$queryClass = static::TABLE_TO_QUERY_MAPPING[$table];
$this->logger->debug('Starting to warm cache for ' . $table . ' in pid ' . $pid);
(new $queryClass(\DB::connection(), $this->logger))->warmCache();
$this->logger->debug('Warmed cache for ' . $table . ' in pid ' . $pid);
die(0); // immediately exit from child processes so we don't have additional forks
}
foreach ($pidsByTable as $table => $pid) { // if we're here, we're in the parent
pcntl_waitpid($pid, $status);
$this->logger->debug("Cache warm process for $table ($pid) done w\status $status");
}
Real world example
- PHP Extension
- Current version is PHP 7.2+
- Requires ZTS (Zend Thread Safe) build of PHP
- Threads, Threadables (data stores), Workers, Stackables
- You have to manage your own reference counting
- As with forking, one DB connection per thread
- Dockerfile available in the companion code repo
- Examples available in its repo...
- ...and in ours
- Uses generators/coroutines
- Inter-process communication built in
- Will use threads for workers if available via pthreads
- Uses shared memory via the shmop extension
- Synchronizes access via mutexes with the sysvmsg ext
- Examples available...
- ...including one in our repo
sleepandwrite.php
#!/usr/bin/env php
<?php
if ($argc < 4) {
die("Usage: ./sleepAndWrite.php sleep_in_seconds filename output\n");
}
sleep($argv[1]);
error_log('Done sleeping. Writing contents to file.');
file_put_contents($argv[2], implode(' ', array_slice($argv, 3)));
composer require symfony/process
Background Processing in-request
- Anything after call doesn't block response
- php-fpm only (function doesn't exist on php -S)
- Ties up an FPM worker process
- Don't expect anything to print after calling
- If you're logging to STDERR and picking it up on your web server, you'll lose that too
nohup cmd > /dev/null 2>&1 &
Doesn't quite work right in php -S
you may want a queue instead
Thanks! QUestions?
- ian.im/pcntl18 - these slides
- joind.in/talk/ac687 - rate this talk!
- github.com/iansltx/pcntl-demo - most code
- github.com/iansltx/raphple - raffle code
- twitter.com/iansltx - me
- github.com/iansltx - my code
(Ab)using process control for powerful CLI applications - CascadiaPHP 2018
By Ian Littman
(Ab)using process control for powerful CLI applications - CascadiaPHP 2018
- 2,401