Use Cases for Drupal Queues
BCO-DMO
Help researchers w. their data
- Prepare data for reuse
- Generate metadata standards
- Improve discovery w. semantics
- Submit data to archives
Metadata
Collated...
- RDF
- ISO 19115-2
- Dublin Core
- Datasets
- Instruments
- Measurements
- People
- Cruises
- Funding
- Projects
Stored.
- Caches
- Indexes



When metadata changes...
what to do with cached docs?
When metadata changes...
what to do with cached docs?
- Figure out what needs updating
- Re-generate
- Update stores
"...I'll just update right away!"
When metadata changes...
"...I'll just update right away!"
meaning...
When a Data Manager DID ANYTHING, they had to wait.
Finding all referenced entities


http://vignette1.wikia.nocookie.net/r2d/images/5/59/Yo-hurry-up.jpg/revision/latest?cb=20141011053551
How do I...
remember the relevant info,
so I can defer the work,
and ensure the work is done?
Drupal Queue API
Drupal Queue API
1. Remember the relevant info
/**
* Implements hook_entity_update().
*/
function mymodule_entity_update($entity, $entity_type) {
...
// Remember relevant info.
$item = array(
'entity_type' => $entity_type,
'entity' => $entity,
'op' => 'update',
);
...
}
Drupal Queue API
2. Defer the work
...
// Create a queue to store items.
// 'TRUE' means get a reliable queue type
// - guarantees every item executes at least once
$queue = DrupalQueue::get('myWorkQueue', TRUE);
// Remember relevant info.
$item = array(
'entity_type' => $entity_type,
'entity' => $entity,
'op' => 'update',
);
// Put an item into the queue.
$queue->createItem($item);
...
Drupal Queue API
3. Ensure the work gets done
Somehow the queue needs to be processed
Our solution was to:
- Setup a drush command to process the queue
- Setup a cron job to run the drush command
Drupal Queue API
3. Ensure the work gets done: DRUSH
// Get the queue.
// 'TRUE' means get a reliable queue type.
$queue = DrupalQueue::get('myWorkQueue', TRUE);
// Iterate over the items.
while($item = $queue->claimItem()) {
// Get the relevant information.
$entity_type = $item['entity_type'];
$entity = $item['entity'];
$operation = $item['op'];
$success = FALSE;
// Do some work with the information you stored.
// Specify if the task was successful $success = ???.
// If successful, delete the item as it is no longer needed.
if ($success) {
$queue->deleteItem($item);
}
}
Drupal Queue API
3. Ensure the work gets done: SUCCESS
// Iterate over the items.
while($item = $queue->claimItem()) {
...
// If successful, delete the item as it is no longer needed.
if ($success) {
$queue->deleteItem($item);
}
}
When an item is not deleted,
it stays in the queue for next time.
RESOURCES
http://rbayliss.net/drupal-queue-api
http://www.slideshare.net/philipnorton42/drupal-7-queues
http://atendesigngroup.com/blog/batch-api-cron-queues
https://api.drupal.org/api/examples/queue_example!queue_example.module/
https://api.drupal.org/api/drupal/modules!system!system.queue.inc/group/queue/7
or hit me up:
- Twitter: @ashep_15
- Email: adam@whoi.edu
Use Cases for Drupal Queues
By Adam Shepherd
Use Cases for Drupal Queues
- 489